Skip to content

MultiQC: add sourmash module (compare + gather)#8112

Open
nekrut wants to merge 1 commit into
galaxyproject:mainfrom
nekrut:multiqc-sourmash-module
Open

MultiQC: add sourmash module (compare + gather)#8112
nekrut wants to merge 1 commit into
galaxyproject:mainfrom
nekrut:multiqc-sourmash-module

Conversation

@nekrut

@nekrut nekrut commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

What

Adds a sourmash module to the IUC MultiQC wrapper, exposing MultiQC's native
sourmash module (present in MultiQC core since 1.x but not previously selectable
in this wrapper). Implemented as a new sourmash_plugin.xml following the
template_plugin.xml pattern, plus the usual five wiring points in multiqc.xml.

Two submodules are supported:

  • compare — MultiQC's sourmash/compare module searches for *.labels.txt
    and then loads the matrix from the same path minus .labels.txt
    (numpy.load). The plugin therefore symlinks the matrix produced by
    sourmash compare -o NAME and its accompanying NAME.labels.txt into the
    software dir under a shared basename, so both are discovered and the
    clustered sample-similarity heatmap is rendered.
  • gather — symlinks sourmash gather CSV(s) for the sourmash/gather
    submodule (content-matched header).

Why

sourmash compare produces an N×N genome/sample similarity matrix; surfacing its
clustered heatmap inside a MultiQC report is a natural QC summary for
multi-genome / multi-sample panels. Until now Galaxy users had to embed the PNG
via custom/image content; this wires up the real interactive MultiQC module.

Changes

  • New tools/multiqc/sourmash_plugin.xml (@SOURMASH_COMMAND@ token + sourmash_form + sourmash_test).
  • tools/multiqc/multiqc.xml: <import>, command #elif, software <option>, <when>, and <expand macro="sourmash_test"/>.
  • tools/multiqc/macros.xml: @VERSION_SUFFIX@ 1 → 2.
  • New test-data: sourmash_compare_matrix + sourmash_compare.labels.txt (real sourmash compare output over the existing-style tiny k=21 signatures) and sourmash_gather.csv.

Testing

  • planemo lint tools/multiqc/multiqc.xml → passes (no errors/warnings).
  • Verified locally with MultiQC 1.35: the staging layout makes the compare
    module find both matrix + labels and the gather module parse the CSV; the
    report contains sourmash-compare-heatmap and Sample similarity
    (the test's <assert_contents>).

🤖 Generated with Claude Code

Expose the native MultiQC sourmash module via a new sourmash_plugin.xml.
- compare: stages the numpy similarity matrix and its NAME.labels.txt with a
  shared basename so MultiQC's compare module (search pattern *.labels.txt)
  finds both and renders the clustered similarity heatmap.
- gather: stages gather CSV(s) for the gather submodule.
Bumps VERSION_SUFFIX 1 -> 2; adds a test with real sourmash compare output.
#end for
]]></token>
<xml name="sourmash_form">
<repeat name="output" title="sourmash output" min="1">

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we have repeat here with min=1?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is one enough but wyou can provide 2 different one? What happens if you provide two gathers?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This <repeat name="output" min="1"> mirrors the established pattern used by the other MultiQC plugins in this tool (bismark, deeptools, fastqc, gatk, picard, rseqc, samtools, star, vcftools all use the identical <repeat name="output" ... min="1">). min="1" just requires that a sourmash result block contributes at least one output — otherwise nothing would be staged for that block.

The repeat is there precisely so you can provide more than one output and/or mix types: e.g. one compare plus one gather, or several gathers. Each repeat entry is uniquely indexed in the staged filename, so there are no collisions:

  • compare: compare_<i>_<j> + compare_<i>_<j>.labels.txt
  • gather: gather_<i>_<j>_<k>_<identifier>.csv

I verified two gather entries end-to-end: they stage as gather_0_0_0_*.csv and gather_0_1_0_*.csv, and MultiQC reports Found 2 gather results and renders both. So providing two gathers (or two compares, or any mix) works as expected.

Note also that within a single gather entry the input param already has multiple="true", so a user can either add multiple files to one entry or add multiple repeat entries — both are handled.

@bgruening bgruening left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm! I'm just wondering about the repeat, and what happens if a user just provides one, or multiple times the same. Should ir have a max=2?

@nekrut

nekrut commented Jun 17, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @bgruening! On the max question: I'd prefer to leave the repeat uncapped (no max), to stay consistent with the other MultiQC plugins here — none of bismark/deeptools/fastqc/gatk/picard/rseqc/samtools/star/vcftools put a max on their output repeat. A cap of 2 would be a bit arbitrary: a realistic use is, say, one compare plus several gather CSVs from different queries in the same MultiQC report, so an upper bound would get in the way.

On the edge cases you raised, I verified both locally:

  • Just one output: works (that's the common case; min="1" guarantees at least one).
  • The same output provided multiple times: also fine — each repeat entry is indexed in the staged filename (gather_<i>_<j>_<k>_<id>.csv, compare_<i>_<j>), so even providing the identical file twice stages as gather_0_0_0_* and gather_0_1_0_* with no collision. MultiQC reports Found 2 gather results and renders the report without error.

So there's no crash/collision risk that a max would be protecting against. Happy to add max if you feel strongly, but my preference is to keep it consistent with the sibling plugins.

@SaimMomin12

Copy link
Copy Markdown
Contributor

Sourmash is WIP here: #7491

I will push it to get is merged in coming days

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Triage/Discuss

Development

Successfully merging this pull request may close these issues.

3 participants