-
Notifications
You must be signed in to change notification settings - Fork 517
MultiQC: add sourmash module (compare + gather) #8112
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
nekrut
wants to merge
1
commit into
galaxyproject:main
Choose a base branch
from
nekrut:multiqc-sourmash-module
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,80 @@ | ||
| <macros> | ||
| <token name="@SOURMASH_COMMAND@"><![CDATA[ | ||
| #for $j, $repeat2 in enumerate( $repeat.software_cond.output ) | ||
| #if str($repeat2.type.type_selector) == "compare" | ||
| ## The MultiQC sourmash module (search pattern "*.labels.txt") finds the | ||
| ## labels file, then loads the matrix from the same path minus ".labels.txt". | ||
| ## So the matrix and its labels MUST share a basename in the same directory: | ||
| ## <base> <- numpy matrix from `sourmash compare -o <base>` | ||
| ## <base>.labels.txt <- the accompanying labels | ||
| #set $base = os.path.join($software_dir, 'compare_' + str($i) + '_' + str($j)) | ||
| #set $at_least_one_input = "T" | ||
| ln -s '$repeat2.type.matrix' '$base' && | ||
| ln -s '$repeat2.type.labels' '${base}.labels.txt' && | ||
| #elif str($repeat2.type.type_selector) == "gather" | ||
| #if str($repeat2.type.input) != "None" | ||
| #set $at_least_one_input = "T" | ||
| #for $k, $file in enumerate($repeat2.type.input) | ||
| #set $identifier = re.sub('[^\s\w\-]', '_', str($file.element_identifier)) | ||
| ln -s '$file' '$software_dir/gather_${i}_${j}_${k}_${identifier}.csv' && | ||
| #end for | ||
| #end if | ||
| #end if | ||
| #end for | ||
| ]]></token> | ||
| <xml name="sourmash_form"> | ||
| <repeat name="output" title="sourmash output" min="1"> | ||
| <conditional name="type"> | ||
| <param name="type_selector" type="select" label="Type of sourmash output"> | ||
| <option value="compare">compare (similarity matrix + labels)</option> | ||
| <option value="gather">gather (CSV)</option> | ||
| </param> | ||
| <when value="compare"> | ||
| <param name="matrix" type="data" format="data" label="compare similarity matrix" | ||
| help="The numpy matrix written by 'sourmash compare -o NAME' (the binary file, not the --csv output)."/> | ||
| <param name="labels" type="data" format="txt" label="compare labels" | ||
| help="The matching 'NAME.labels.txt' written alongside the matrix by 'sourmash compare' / 'sourmash plot --labels'."/> | ||
| </when> | ||
| <when value="gather"> | ||
| <param name="input" type="data" format="csv" multiple="true" label="Output of sourmash gather" | ||
| help="CSV produced by 'sourmash gather -o'. It must contain the columns 'f_match', 'f_unique_weighted', 'name' and 'query_name'." optional="true"/> | ||
| </when> | ||
| </conditional> | ||
| </repeat> | ||
| </xml> | ||
| <!-- add here your test files and tests, the more stringent the better --> | ||
| <xml name="sourmash_test"> | ||
| <test expect_num_outputs="3"> | ||
| <repeat name="results"> | ||
| <conditional name="software_cond"> | ||
| <param name="software" value="sourmash"/> | ||
| <repeat name="output"> | ||
| <conditional name="type"> | ||
| <param name="type_selector" value="compare"/> | ||
| <param name="matrix" value="sourmash_compare_matrix" ftype="data"/> | ||
| <param name="labels" value="sourmash_compare.labels.txt" ftype="txt"/> | ||
| </conditional> | ||
| </repeat> | ||
| <repeat name="output"> | ||
| <conditional name="type"> | ||
| <param name="type_selector" value="gather"/> | ||
| <param name="input" value="sourmash_gather.csv" ftype="csv"/> | ||
| </conditional> | ||
| </repeat> | ||
| </conditional> | ||
| </repeat> | ||
| <param name="title" value="Title of the report"/> | ||
| <param name="comment" value="Commment for the report"/> | ||
| <param name="flat" value="true"/> | ||
| <param name="export" value="true"/> | ||
| <output name="html_report"> | ||
| <assert_contents> | ||
| <has_text text="Title of the report"/> | ||
| <has_text text="Commment for the report"/> | ||
| <has_text text="sourmash-compare-heatmap"/> | ||
| <has_text text="Sample similarity"/> | ||
| </assert_contents> | ||
| </output> | ||
| </test> | ||
| </xml> | ||
| </macros> | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| strainA | ||
| strainB | ||
| strainC |
Binary file not shown.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| intersect_bp,f_orig_query,f_match,f_unique_to_query,f_unique_weighted,average_abund,median_abund,std_abund,name,filename,md5,f_match_orig,unique_intersect_bp,gather_result_rank,remaining_bp,query_filename,query_name,query_md5,query_bp | ||
| 442000,0.0890,0.0908,0.0890,0.0949,1.5,1.0,0.5,"CP001509.3 Escherichia coli str. K-12 substr. MG1655",db.sbt.zip,abc123,0.0908,442000,0,4525000,query.sig,sampleA,def456,4967000 | ||
| 210000,0.0423,0.0451,0.0423,0.0388,1.2,1.0,0.4,"NC_000913.3 Salmonella enterica",db.sbt.zip,ghi789,0.0451,210000,1,4315000,query.sig,sampleA,def456,4967000 |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we have repeat here with min=1?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is one enough but wyou can provide 2 different one? What happens if you provide two gathers?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This
<repeat name="output" min="1">mirrors the established pattern used by the other MultiQC plugins in this tool (bismark, deeptools, fastqc, gatk, picard, rseqc, samtools, star, vcftools all use the identical<repeat name="output" ... min="1">).min="1"just requires that a sourmash result block contributes at least one output — otherwise nothing would be staged for that block.The repeat is there precisely so you can provide more than one output and/or mix types: e.g. one
compareplus onegather, or severalgathers. Each repeat entry is uniquely indexed in the staged filename, so there are no collisions:compare_<i>_<j>+compare_<i>_<j>.labels.txtgather_<i>_<j>_<k>_<identifier>.csvI verified two
gatherentries end-to-end: they stage asgather_0_0_0_*.csvandgather_0_1_0_*.csv, and MultiQC reportsFound 2 gather resultsand renders both. So providing two gathers (or two compares, or any mix) works as expected.Note also that within a single
gatherentry theinputparam already hasmultiple="true", so a user can either add multiple files to one entry or add multiple repeat entries — both are handled.