Adding workflow tests with Planemo
Ensuring a Tutorial has a Workflow
-
Find a tutorial that you’re interested in, that doesn’t currently have tests.
This tutorial has a workflow (
.ga
) and a test, notice the-test.yml
that has the same name as the workflow.ga
file.machinelearning/workflows/machine_learning.ga machinelearning/workflows/machine_learning-test.yml
You want to find tutorials without the
-test.yml
file. The workflow file might also be missing. - Check if it has a workflow (if it does, skip to step 5.)
- Follow the tutorial
- Extract a workflow from the history
- Run that workflow in a new history to test
Extract Tests (Online Version)
If you are on UseGalaxy.org or another server running 24.2 or later, you can use PWDK, a version of planemo running online to generate the workflow tests.
However if you are on an older version of Galaxy, or a private Galaxy server, then you’ll need to do the following:
Extract Tests (Manual Version)
-
Obtain the workflow invocation ID, and your API key (User → Preferences → Manage API Key)
-
Install the latest version of
planemo
# In a virtualenv pip install planemo
-
Run the command to initialise a workflow test from the
workflows/
subdirectory - if it doesn’t exist, you might need to create it first.planemo workflow_test_init --from_invocation <INVOCATION ID> --galaxy_url <GALAXY SERVER URL> --galaxy_user_key <GALAXY API KEY>
This will produce a folder of files, for example from a testing workflow:
$ tree . ├── test-data │ ├── input dataset(s).shapefile.shp │ └── shapefile.shp ├── testing-openlayer.ga └── testing-openlayer-tests.yml
Adding Your Tests to the GTN
-
You will need to check the
-tests.yml
file, it has some automatically generated comparisons. Namely it tests that output data matches the test-data exactly, however, you might want to replace that with assertions that check for e.g. correct file size, or specific text content you expect to see. -
If the files in test-data are already uploaded to Zenodo, to save disk space, you should delete them from the
test-data
dir and use their URL in the-tests.yml
file, as in this example:- doc: Test the M. Tuberculosis Variant Analysis workflow job: 'Read 1': location: https://zenodo.org/record/3960260/files/004-2_1.fastq.gz class: File filetype: fastqsanger.gz
-
Add tests on the outputs! Check the planemo reference if you need more detail.
- doc: Test the M. Tuberculosis Variant Analysis workflow job: # Simple explicit Inputs 'Read 1': location: https://zenodo.org/record/3960260/files/004-2_1.fastq.gz class: File filetype: fastqsanger.gz outputs: jbrowse_html: asserts: has_text: text: "JBrowseDefaultMainPage" snippy_fasta: asserts: has_line: line: '>Wildtype Staphylococcus aureus strain WT.' snippy_tabular: asserts: has_n_columns: n: 2
-
Contribute all of those files to the GTN in a PR, adding them to the
workflows/
folder of your tutorial.