Add deacon and deacon DM by SantaMcCloud · Pull Request #7473 · galaxyproject/tools-iuc

SantaMcCloud · 2025-11-17T09:56:32Z

FOR CONTRIBUTOR:

I have read the CONTRIBUTING.md document and this tool is appropriate for the tools-iuc repo.
License permits unrestricted use (educational + commercial)
This PR adds a new tool or tool collection
This PR updates an existing tool or tool collection
This PR does something else (explain below)

SantaMcCloud · 2025-11-17T09:57:15Z

Test will error since there is still missing the format for deacon which has an open PR: galaxyproject/galaxy#21289

SantaMcCloud · 2025-11-18T23:04:38Z

I will add the change later here since i have some testing to be done which i could done yet but some of reviews input was used to change the current state of the wrappers!

SantaMcCloud · 2025-11-19T20:10:12Z

@bernt-matthias i did change all and test all wrapper manually. When the format was added they all should be positiv!

bede · 2025-11-21T19:51:11Z

Deacon developer here. I'm unfamiliar with Galaxy development and how to test a PR like this in a running environment, but I'm happy to answer questions, review screenshots etc.

bernt-matthias · 2025-11-21T20:05:12Z

Deacon developer here. I'm unfamiliar with Galaxy development and how to test a PR like this in a running environment, but I'm happy to answer questions, review screenshots etc.

I'm sure @SantaMcCloud can provide some screenshots of the tool if you like.

bernt-matthias · 2025-11-21T20:06:10Z

Btw. wonderful the get feedback from the developer of the underlying software. This is super helpful and much appreciated.

SantaMcCloud · 2025-11-21T21:34:53Z

Deacon developer here. I'm unfamiliar with Galaxy development and how to test a PR like this in a running environment, but I'm happy to answer questions, review screenshots etc.

The way a PR is tested is that the xml file has a test section where a test is written. This means the paramter which can be used are set therer together with how many outputs are generated and then this output can be check with different ways like size compare or if certain text is in the file for example. The Problem now is that in GitHub a Galaxy instance will be simulated and the each test will be run via commandline. Now the problem is that my test doesnt work is that Galaxy has datatype which are allowed to be used and i did use a datatype which is not yet implemented in Galaxy so Galaxy blocking the files which are set to this datatype to be used which will end up in a errored Test. I did test all test localy where you can add datatype without an problme and run the same simulation as GitHub does there all test where fine but when there is something where i have some troubles i will come back to this offer thank you! Also if you want too know more feel free to ask i can try to answer them or can provide any data/screenshots from implemente you tool to galaxy! :)

Btw. wonderful the get feedback from the developer of the underlying software. This is super helpful and much appreciated.

Yes this is really helpful thank you for this!

SantaMcCloud · 2026-05-28T20:50:03Z

Will fix the new test tomorrow but in the meantime the checksum check can be reviewed

pvanheus · 2026-06-25T08:53:21Z

@SantaMcCloud can we replace the hard-coded URLs in the data manager with a TSV of URLs and expected checksums? This will make it more maintainable. I can work on that while I'm here at the GCC 2026 CoFest.

SantaMcCloud · 2026-06-25T09:01:17Z

@SantaMcCloud can we replace the hard-coded URLs in the data manager with a TSV of URLs and expected checksums? This will make it more maintainable. I can work on that while I'm here at the GCC 2026 CoFest.

We can replace this yes. And if you can or want to do it feel free to. Happy to learn from you then for futher tools and DMs! :)

pvanheus · 2026-06-25T17:00:35Z

The download from zenodo could be much simplified by using zenodo_get.

SantaMcCloud · 2026-06-25T17:50:44Z

The download from zenodo could be much simplified by using zenodo_get.

i will change it then

SantaMcCloud · 2026-06-26T02:21:33Z

hm the linting error now comes from the r linting which i dont know how to fix since it seems like this is an error from the Pipeline setup?

bernt-matthias · 2026-06-26T06:58:33Z

+                <param name="note" type="text" label="Add a note" help="Here some notes can be set for example from where the data comes, who created the data and so on"/>
+            </when>
+            <when value="fasta">
+                <param name="file_name" type="text" label="Enter filename" help="Please enter the name of the file which will be downloaded without the extension" />


We should maybe consider the fasta data table here as input as well.

So just to check if im right this should work with the current fasta data table or a new data table?

The all_fasta data table

bernt-matthias · 2026-06-26T06:59:17Z

+                    <option value="mouse">panmouse-1 (k=31, w=15, e=0.5)</option> 
+                </param>
+            </when>
+            <when value="url">


Do we need the url option?

can we replace the hard-coded URLs in the data manager with a TSV of URLs and expected checksums?

Is there so much pre-built data out there anyway? If so, we should also store the origin URL in the loc file?

If the number of available pre-built files is small, I would prefer to have them hardcoded in the DM.

Im not sure i just add if there might be a lot of usage later on.

Currently only the 2 hardcoded pre-builds are out so this was also the reason i did hardcode them. I can add the URL/Zenodo record ID in the loc file/data table if wanted

bernt-matthias · 2026-06-26T07:03:05Z

+                <param name="version" type="text" label="Which version was used to build the file to copy" help="State the tool version used to build the index file"/>
+                <param name="format_version" type="text" label="Set the index format version" help="The current index format version for Deacon v.@TOOL_VERSION@ is 3"/>


The DM should set this information and not the DM user.

This referred to the fasta input case. Your change seems to be regarding url input (which should be removed?)

bernt-matthias · 2026-06-26T07:04:07Z

+            </when>
+            <when value="fasta">
+                <param name="file_name" type="text" label="Enter filename" help="Please enter the name of the file which will be downloaded without the extension" />
+                <param name="link" type="text" label="Input the URL to download a fast file for using it to build an index file"/>


This parameter seems not necessary to me.

i will remove/change this based on the fasta table comment.

bernt-matthias · 2026-06-26T07:05:27Z

+    </command>
+    <inputs>
+        <conditional name="index_type">
+            <param name="is_select" type="select" label="Select if index file will be used from DM or own index file">


"from DM" is pretty technical. Just something like "use a pre-built index"?

Please do the same for the option.

SantaMcCloud · 2026-06-26T07:28:25Z

@bernt-matthias i Will do the chnages later which i did not comment :)

pvanheus · 2026-06-26T10:45:32Z

+            #if $input.is_select == "prebuild"
+                #if $download == "human"
+                    zenodo_get 17288185 -m -g "*.idx" -o '$out_file.extra_files_path' &&
+                    md5sum -c md5sums.txt || (echo "Checksum verification failed" >&2; exit 1) 


You don't need the md5sum - the -m to zenodo_get does the MD5 check

pvanheus · 2026-06-26T12:06:38Z

+                ln -s '$fastx_input.single_file' "${identifier}.${fastx_input.single_file.ext}" &&
+            #else
+                #set $identifier = re.sub("[^\s\w\-]", "_", str($fastx_input.pair.forward.element_identifier))
+                ln -s '$fastx_input.pair.forward' "${identifier}.${fastx_input.pair.forward.ext}" &&


All of this use of identifier is not needed. One can simply ln -s '$fastx_input.single_file' "single.${fastx_input.single_file.ext}" && and similar for the paired reads.

Maybe plus the complication that ext is "fastqsanger", but this might not be a problem ... do we have a test covering these datatypes?

Most simple would be to use $fastx_input.single_file directly if it works and skip the linking.

Agreed - deacon uses paraseq to read sequence data which in turn uses niffler which sniffs the files to detect datatype, so exposing the extension of the datatype is not needed. A simple use of the relevant variable (like $fastx_input.single_file would work.

I did change all tools beside of union and intersect since there can be x amout of file used and for debugging it might be better to have numbered files. If wanted i can remove this aswell!

bernt-matthias

Thanks for the changes. A few more comments

bernt-matthias · 2026-06-27T04:00:23Z

+                <param name="note" type="text" label="Add a note" help="Here some notes can be set for example from where the data comes, who created the data and so on"/>
+            </when>
+            <when value="fasta">
+                <param name="file_name" type="text" label="Enter filename" help="Please enter the name of the file which will be downloaded without the extension" />


The all_fasta data table

bernt-matthias · 2026-06-27T04:01:26Z

+                <param name="version" type="text" label="Which version was used to build the file to copy" help="State the tool version used to build the index file"/>
+                <param name="format_version" type="text" label="Set the index format version" help="The current index format version for Deacon v.@TOOL_VERSION@ is 3"/>


This referred to the fasta input case. Your change seems to be regarding url input (which should be removed?)

bernt-matthias · 2026-06-27T04:04:25Z

+                ln -s '$fastx_input.single_file' "${identifier}.${fastx_input.single_file.ext}" &&
+            #else
+                #set $identifier = re.sub("[^\s\w\-]", "_", str($fastx_input.pair.forward.element_identifier))
+                ln -s '$fastx_input.pair.forward' "${identifier}.${fastx_input.pair.forward.ext}" &&


Maybe plus the complication that ext is "fastqsanger", but this might not be a problem ... do we have a test covering these datatypes?

Most simple would be to use $fastx_input.single_file directly if it works and skip the linking.

SantaMcCloud · 2026-06-29T20:36:27Z

@bernt-matthias

The all_fasta data table

I did start change the DM to use this data table but im not finish since i have a question where you might could help me since i need a test DB for this DM so i can use it in a test but for deacon i have the test files. So my question is can i put both into one config file [tool_data_table_conf.xml.test] or do they need each of them there own file and if yes can i name then different?

This referred to the fasta input case. Your change seems to be regarding url input (which should be removed?)

yes i did forgot to change this sorry it should be removed now!

pvanheus · 2026-07-08T18:29:05Z

@bernt-matthias

The all_fasta data table

I did start change the DM to use this data table but im not finish since i have a question where you might could help me since i need a test DB for this DM so i can use it in a test but for deacon i have the test files. So my question is can i put both into one config file [tool_data_table_conf.xml.test] or do they need each of them there own file and if yes can i name then different?

It can all go into the sample tool_data_table_conf.xml.test file. Look here for an example.

SantaMcCloud · 2026-07-08T22:36:50Z

@bernt-matthias

The all_fasta data table

I did start change the DM to use this data table but im not finish since i have a question where you might could help me since i need a test DB for this DM so i can use it in a test but for deacon i have the test files. So my question is can i put both into one config file [tool_data_table_conf.xml.test] or do they need each of them there own file and if yes can i name then different?

It can all go into the sample tool_data_table_conf.xml.test file. Look here for an example.

Thank you this helped me a lot. The DM should be done now and i think all tools can be reviewed again!

pvanheus · 2026-07-09T04:56:13Z

+        #end if
+        "format_version":"3",
+        #if $input.is_select == "prebuilt"
+            "note":"Prebuilt index files from the devs of deacon"


Replace devs with developers

SantaMcCloud added 3 commits November 17, 2025 10:47

Add deacon

130daa1

add .shed files

0e1bba1

typo

438cb98

SantaMcCloud mentioned this pull request Nov 17, 2025

Add deacon idx format galaxyproject/galaxy#21289

Closed

4 tasks

SantaMcCloud changed the title ~~Add deacon~~ Add deacon and deacon DM Nov 17, 2025

SantaMcCloud mentioned this pull request Nov 17, 2025

New tool request: Host decontamination tools #7438

Open

5 tasks

SantaMcCloud added 4 commits November 17, 2025 11:31

fix some tests

aae6216

change one test file

6e9014f

little changes

4e11579

little changes

8474b57

bernt-matthias reviewed Nov 18, 2025

View reviewed changes

SantaMcCloud added 9 commits November 18, 2025 23:10

typos

a7fc57f

add deacon build into dm

19ddd38

add dash to value for dm

f0ca31f

typo

18b9283

shorter desc in deacon_filter

8eff8c0

change unneeded double to single qoutes

57dfe9b

change fastq to sanger/illumina

142e8e3

change to paired collection input

42ee2f2

change test data format

6288d4e

split index into multiple wrappers

3c9a09d

bede reviewed Nov 21, 2025

View reviewed changes

Comment thread data_managers/data_manager_deacon/data_manager/deacon_datamanager.xml Outdated

bede mentioned this pull request Nov 21, 2025

Deposit panmouse-1 to Zenodo, all indexes going forwards bede/deacon#74

Closed

add packages

1963c70

fix test

082d276

SantaMcCloud added 2 commits June 26, 2026 01:49

use zenodo_get

8ea49d5

fix linting

082bde1

bernt-matthias reviewed Jun 26, 2026

View reviewed changes

pvanheus requested changes Jun 26, 2026

View reviewed changes

SantaMcCloud added 5 commits June 27, 2026 01:56

remove format version param

2907580

fix some wording

dc38c66

remove md5sum command

68c1977

add some new requiremnt

c2dc2dd

change file naming in filter.xml

6c66baf

bernt-matthias reviewed Jun 27, 2026

View reviewed changes

SantaMcCloud added 4 commits June 29, 2026 17:41

remove url option

6c42b02

set up selection from other DM

2cbc608

change some command in deacon commands

71cd4f7

fix test

cb4b14b

finalize DM

56c409f

pvanheus reviewed Jul 9, 2026

View reviewed changes

Update deacon_datamanager.xml

27d09cf

		<param name="version" type="text" label="Which version was used to build the file to copy" help="State the tool version used to build the index file"/>
		<param name="format_version" type="text" label="Set the index format version" help="The current index format version for Deacon v.@TOOL_VERSION@ is 3"/>

Uh oh!

Conversation

SantaMcCloud commented Nov 17, 2025

Uh oh!

SantaMcCloud commented Nov 17, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SantaMcCloud commented Nov 18, 2025

Uh oh!

SantaMcCloud commented Nov 19, 2025

Uh oh!

bede commented Nov 21, 2025

Uh oh!

Uh oh!

bernt-matthias commented Nov 21, 2025

Uh oh!

bernt-matthias commented Nov 21, 2025

Uh oh!

SantaMcCloud commented Nov 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SantaMcCloud commented May 28, 2026

Uh oh!

pvanheus commented Jun 25, 2026

Uh oh!

SantaMcCloud commented Jun 25, 2026

Uh oh!

pvanheus commented Jun 25, 2026

Uh oh!

SantaMcCloud commented Jun 25, 2026

Uh oh!

SantaMcCloud commented Jun 26, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SantaMcCloud commented Jun 26, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SantaMcCloud commented Nov 21, 2025 •

edited

Loading