Running the Pipeline

1 Before running the pipeline

Before running the pipeline, make sure that

  • the names of the variables of the dataset are appropriately defined according to GPMelt nomenclature (see here),
  • the experimental design parameters of the experiment have been translated into a hierarchy (see here),
  • the choice of the null distribution has been made and NumberSamples_perID.csv file has been defined and saved (see here),
  • a subset of IDs to test the model specifications and resource requirement have been defined and saved as subset_ID.csv (if running the pipeline for the first time on this dataset, see here)
  • appropriate values for the parameters of the parameters.txt have been defined (see here),
  • all the required .config files have been updated (see here),

Note:

  • The workflow report named GPMelt_workflow_report.html will be saved in the reports folder, together with the co2footprint reports.
  • Please take a look at GPMelt_workflow_report.html and the co2footprint reports obtained on a subset of IDs before running GPMelt on your full dataset! This will help improve the resource requirement definition, improve your running time and reduce your carbon imprint! Thank you!

2 Run the Nextflow pipeline on an HPC cluster:

Now, go in the Nextflow folder and run the following command from the terminal.

nextflow run GPMelt_workflow.nf -c nextflow.config  -c nextflow_cluster.config  -c user_params.config  -with-report reports/GPMelt_workflow_report.html

3 Run the Nextflow pipeline locally:

Now, go in the Nextflow folder and run the following command from the terminal.

nextflow run GPMelt_workflow.nf -c nextflow.config  -c nextflow_local.config -c user_params.config  -with-report reports/GPMelt_workflow_report.html

4 Resume

If only one to few steps of the pipeline did fail, it is possible to rerun only these steps using the keyword -resume. For example, locally, we would have:

nextflow run GPMelt_workflow.nf -c nextflow.config  -c nextflow_local.config -c user_params.config  -with-report reports/GPMelt_workflow_report.html -resume

See details on the Nextflow tutorial.

5 Troubleshooting

If you encounter any issues, ensure that:

  • Docker is running correctly on your machine.
  • You have pulled the correct Docker image (ceccads/gpmelt-image-with-dask:0.0.1).
  • Your Nextflow configuration file (nextflow_local.config or nextflow_cluster.config) is set up to use Docker.

For further assistance, refer to the Nextflow documentation or seek help from the pipeline repository’s issue tracker.

6 Session info

sessionInfo()