Exporting Pipelines to your Local Machine
Steps for running code as you find it on Code Ocean locally.
Last updated
Was this helpful?
Steps for running code as you find it on Code Ocean locally.
Last updated
Was this helpful?
Code Ocean allows editors and viewers to download an entire Pipeline. Click the Pipeline menu and select Export:
This will download the /pipeline folder, /metadata folder, and instructions for running this code outside of Code Ocean. When you unzip this, you will see something like the following (this screenshot comes from a Mac):
The /metadata folder has a file called metadata.yml with information about the Capsule
The /pipeline folder contains the Nextflow script in the main.nf file and Nextflow configurations in the nextflow.config file. If a user has defined Secrets in the Environment Editor of any Capsule, these variables will be white-listed in the nextflow.config file and the REPRODUCING.md file will provide instructions for creating the appropriate Secrets locally. In addition, nextflow.config will include the Pipeline's App Panel parameters.
Alternatively, all Git tracked files can be exported as a Pipeline repository via Pipeline -> Clone via Git.
To access this Pipeline, the username and password will be the email used to log into your Code Ocean deployment and the personal access token generated on the box with the link to clone
(This is the recommended practice if you are going to continue to make commits and changes to the Pipeline, so that changes can continue to be tracked). The data will have to be transported separately.
To reproduce your Pipeline outside of Code Ocean, the steps are to complete the following:
Ensure Nextflow is installed and can be used on your local machine
Ensure docker is installed and can be used on your local machine
In the exported/cloned capsule directory on your local machine, run the command
The registry for docker login can be found on the FROM statement on the first line of your Capsule Dockerfile (for example, the registry from the below example is registry.codeocean.com)
The registry can also be found at the end of a buildLog of a successful capsule environment build
After you execute the docker login command you will be prompted for a password - enter the API key previously generated
In your terminal, navigate to the pipeline folder within the folder where you've extracted the Pipeline and execute the following command, adjusting parameters as needed:
Edit the nextflow.config file found inside the same folder as needed:
Enter your parameters (by process name) into the params section. For named parameters, the format is "--name1=value --name2=value". For ordered parameters, the format is "param1_value param2_value".
Capsule secrets can be set in the containerOptions parameter in each of the withName sections within the process section. You must replace the environment variable values with the actual secret values. Alternatively, if you have any of the required environment variables already set in your environment, you can add an envWhitelist parameter in the docker section and specify the relevant environment variable names there, and remove them from the relevant containerOptions parameters.
Generate an API key in your Code Ocean User Account (More information can be found )
Required Data Assets must be downloaded separately, and can be saved into the /data folder. If the Data Assets are External, they can be downloaded during run time. See the for how to connect to AWS.
See for further details.
