Skip to content

Add spaCy NLP tool and data manager#8006

Open
ksuderman wants to merge 10 commits into
galaxyproject:mainfrom
ksuderman:spacy-nlp
Open

Add spaCy NLP tool and data manager#8006
ksuderman wants to merge 10 commits into
galaxyproject:mainfrom
ksuderman:spacy-nlp

Conversation

@ksuderman

Copy link
Copy Markdown
Contributor

Summary

  • Adds spaCy NLP annotation tool supporting 70+ models across 25+ languages
  • Includes data manager for downloading and installing spaCy language models
  • Provides tokenization, POS tagging, lemmatization, dependency parsing, NER
  • Multiple output formats: JSON, CoNLL, tabular, text

Test plan

  • Tool passes planemo lint validation
  • Data manager passes planemo lint validation
  • Comprehensive test data included
  • README documentation provided for both components
  • Model installation tested across multiple languages

🤖 Generated with Claude Code

ksuderman and others added 2 commits May 19, 2026 18:50
- spaCy tool wrapper for fast, production-ready NLP
- 25+ languages with multiple model sizes (sm/md/lg/trf)
- Core annotations: tokenization, POS, NER, dependency parsing
- Enhanced JSON output with is_alpha, is_stop attributes
- Python 3.12 compatible
- Data manager for automated model downloads via pip
- 70+ models across 25+ languages
- Comprehensive tests with planemo validation
- Memory efficient and optimized for speed

Tool: spacy_nlp (v3.8.11+galaxy4)
Data Manager: data_manager_spacy_models
Categories: Text Manipulation, Natural Language Processing
Co-Authored-By: Claude Sonnet 4 <noreply@anthropic.com>
Comment thread data_managers/data_manager_spacy_models/.shed.yml Outdated
@@ -0,0 +1,152 @@
<tool id="data_manager_spacy_models" name="spaCy Language Models" version="3.8.11.1" tool_type="manage_data" profile="24.1">

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
<tool id="data_manager_spacy_models" name="spaCy Language Models" version="3.8.11.1" tool_type="manage_data" profile="24.1">
<tool id="data_manager_spacy_models" name="spaCy Language Models" version="@TOOL_VERSION@+galaxy@VERSION_SUFFIX@" tool_type="manage_data" profile="@PROFILE@">

Perhaps defining TOKENS and defining them here?

Comment thread data_managers/data_manager_spacy_models/README.md Outdated
Comment thread tools/spacy/.shed.yml Outdated
Comment thread tools/spacy/spacy_nlp.xml Outdated
Comment on lines +1 to +4
<tool id="spacy_nlp" name="spaCy NLP Annotators" version="@TOOL_VERSION@+galaxy@VERSION_SUFFIX@" python_template_version="3.5" profile="24.1">
<macros>
<token name="@TOOL_VERSION@">3.8.11</token>
<token name="@VERSION_SUFFIX@">4</token>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
<tool id="spacy_nlp" name="spaCy NLP Annotators" version="@TOOL_VERSION@+galaxy@VERSION_SUFFIX@" python_template_version="3.5" profile="24.1">
<macros>
<token name="@TOOL_VERSION@">3.8.11</token>
<token name="@VERSION_SUFFIX@">4</token>
<tool id="spacy_nlp" name="spaCy NLP Annotators" version="@TOOL_VERSION@+galaxy@VERSION_SUFFIX@" python_template_version="3.5" profile="@PROFILE@">
<macros>
<token name="@TOOL_VERSION@">3.8.11</token>
<token name="@VERSION_SUFFIX@">4</token>
<token name="@PROFILE@">25.0</token>

Comment thread tools/spacy/spacy_nlp.xml Outdated
</data>
</outputs>
<tests>
<test>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
<test>
<test expect_num_outputs="1">

Applies to all tests

Comment thread tools/spacy/spacy_nlp.xml Outdated
Comment on lines +62 to +65
<assert_contents>
<has_text text="&quot;text&quot;"/>
<has_text text="&quot;tokens&quot;"/>
<has_text text="John"/>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have more specific assertions here?

Comment thread tools/spacy/spacy_nlp.xml Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please place in a sub-directory called data_manager

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also goes to data_manager sub-directory

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to write some tests for the DM?

ksuderman and others added 8 commits May 21, 2026 23:53
- Fix remote repository URLs in both tool and data manager .shed.yml files
- Standardize versioning with @Profile@ token and update to profile 25.0
- Change VERSION_SUFFIX from 4 to 0 for initial release
- Add expect_num_outputs="1" to all test cases
- Update citation to DOI format (10.5281/zenodo.10009823)
- Remove unnecessary README.md from data manager
- Reorganize data manager files into data_manager/ subdirectory structure

All changes address reviewer feedback from SaimMomin12 and RZ9082
Removed unused imports for Path and DocBin.
…am help, fix parse/NER and language-count claims, align citation DOI, fold README into help

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants