Add Stanza NLP tool and data manager

data_managers/data_manager_stanza_models/.shed.yml

@@ -0,0 +1,15 @@
+name: data_manager_stanza_models
+owner: iuc
+description: Data manager for downloading and installing Stanza language models
+long_description: |
+  This data manager allows Galaxy administrators to download and install Stanza
+  language models for use with the Stanza NLP annotation tool. It supports 80+
+  languages with models for tokenization, POS tagging, lemmatization, dependency
+  parsing, NER, sentiment analysis, and constituency parsing.
+homepage_url: https://stanfordnlp.github.io/stanza/
+remote_repository_url: https://github.com/galaxyproject/tools-iuc/tree/main/data_managers/data_manager_stanza_models
+type: unrestricted
+categories:
+  - Data Managers
+  - Text Manipulation
+  - Natural Language Processing

data_managers/data_manager_stanza_models/README.md

@@ -0,0 +1,150 @@
+# Galaxy Data Manager for Stanza Language Models
+
+This Galaxy data manager downloads and installs Stanza language models for use with the Stanza NLP annotation tool, supporting 80+ languages with neural models trained on Universal Dependencies.
+
+## Features
+
+- **80+ languages**: Comprehensive language support for multilingual NLP
+- **Direct HuggingFace download**: Downloads models directly from HuggingFace without requiring stanza installation
+- **Multiple language installation**: Select and install multiple languages simultaneously
+- **Progress reporting**: Shows download progress for each language model
+- **Duplicate prevention**: Checks existing installations to avoid redundant downloads
+- **Data table integration**: Automatically registers models in Galaxy's data table system
+
+## How It Works
+
+This data manager:
+1. **Connects to HuggingFace**: Downloads default_fast model packages directly from Stanford's HuggingFace repository
+2. **No dependencies**: Uses only Python's `urllib.request` - no stanza installation required
+3. **Extracts models**: Unzips model packages to Galaxy's managed storage
+4. **Registers models**: Updates the `stanza_models.loc` data table for tool access
+5. **Version control**: Downloads models compatible with Stanza 1.11.1
+
+## Supported Languages
+
+The data manager supports **80+ languages** including:
+
+### European Languages
+- **Western**: English, German, French, Spanish, Italian, Portuguese, Dutch
+- **Nordic**: Swedish, Danish, Norwegian (Bokmål/Nynorsk), Finnish
+- **Slavic**: Russian, Ukrainian, Polish, Czech, Slovak, Croatian, Serbian, Bulgarian
+- **Other**: Greek, Hungarian, Romanian, Estonian, Latvian, Lithuanian
+
+### Asian Languages
+- **East Asian**: Chinese (Simplified/Traditional), Japanese, Korean
+- **South Asian**: Hindi, Tamil, Telugu, Marathi, Urdu
+- **Southeast Asian**: Vietnamese, Thai, Indonesian
+- **Middle Eastern**: Arabic, Persian, Hebrew, Turkish
+
+### Other Languages
+- **African**: Afrikaans
+- **Minority**: Basque, Galician, Catalan, Armenian, Georgian
+
+See [Stanza's complete model list](https://stanfordnlp.github.io/stanza/available_models.html) for detailed language coverage.
+
+## Model Details
+
+### Model Type
+- **default_fast**: Memory-efficient models without character-level processing
+- **Neural networks**: Pretrained on Universal Dependencies v2.12 treebanks
+- **Multi-task**: Single package includes tokenization, POS, lemma, parsing, and NER models (where available)
+
+### Model Sizes
+- **Typical size**: 50-200MB per language
+- **Variation**: Depends on language complexity and available training data
+- **Storage**: Models persist in Galaxy's `tool-data/stanza_models/` directory
+
+### Model Components
+Each language package may include:
+- **Tokenization**: Sentence and token segmentation
+- **POS tagging**: Universal POS tags and morphological features
+- **Lemmatization**: Base form reduction
+- **Dependency parsing**: Universal Dependencies syntax
+- **NER**: Named entity recognition (available for subset of languages)
+
+## Installation Process
+
+### Admin Setup
+1. **Install this data manager**: `data_manager_stanza_models`
+2. **Install the Stanza tool**: `stanza_nlp`
+3. **Navigate to Admin → Local Data**
+4. **Select "Stanza Language Models"**
+
+### Model Installation
+1. **Choose languages**: Select checkboxes for desired languages
+2. **Run installation**: Data manager will download and extract models
+3. **Monitor progress**: Download status shown for each language
+4. **Verify installation**: Models appear in the Stanza tool's language dropdown
+
+### Post-Installation
+- Models are immediately available to the Stanza NLP tool
+- No restart required
+- Models persist across Galaxy restarts
+- Multiple installations of the same language are prevented
+
+## Data Table Format
+
+Models are registered in `stanza_models.loc` with this format:
+```
+<lang_code>    <display_name>    <lang_code>    <models_path>
+```
+
+Example:
+```
+en    English    en    /galaxy/tool-data/stanza_models/en
+de    German     de    /galaxy/tool-data/stanza_models/de
+```
+
+## Technical Details
+
+### Download Source
+- **Repository**: https://huggingface.co/stanfordnlp/
+- **Model naming**: `stanza-{lang}` (e.g., `stanza-en`, `stanza-de`)
+- **Version**: Models tagged with `v{resources_version}` from Stanford's resources.json
+
+### Storage Structure
+```
+tool-data/
+└── stanza_models/
+    ├── en/
+    │   └── [English model files]
+    ├── de/
+    │   └── [German model files]
+    └── stanza_models.loc
+```
+
+### Dependencies
+- **Python 3.12**: Standard library only
+- **No stanza package**: Downloads directly from HuggingFace
+- **urllib.request**: For HTTP downloads
+- **zipfile**: For model extraction
+
+## Troubleshooting
+
+### Common Issues
+- **Network connectivity**: Ensure access to huggingface.co
+- **Disk space**: Large language sets require substantial storage
+- **Permissions**: Galaxy must have write access to tool-data directory
+
+### Model Verification
+- Check `stanza_models.loc` for registered models
+- Verify model files exist in expected directories
+- Test with Stanza NLP tool after installation
+
+## Citation
+
+This data manager installs models created by the Stanford NLP Group. Please cite:
+
+```
+Qi, Peng, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. 
+"Stanza: A Python Natural Language Processing Toolkit for Many Human Languages." 
+In Proceedings of the 58th Annual Meeting of the Association for Computational 
+Linguistics: System Demonstrations, 2020.
+```
+
+## Version History
+
+- **1.11.1.3**: Enhanced duplicate prevention and error handling
+- **1.11.1.2**: Improved download progress reporting
+- **1.11.1.1**: Direct HuggingFace download implementation
+- **1.11.1.0**: Initial release

data_managers/data_manager_stanza_models/data_manager/data_manager_stanza_models.py

@@ -0,0 +1,133 @@
+#!/usr/bin/env python
+"""
+Data Manager for Stanza Language Models
+
+Downloads Stanza language models with ``stanza.download()`` into the data
+manager output's extra files directory and registers them in Galaxy's
+``stanza_models`` data table. Galaxy then moves the models into the managed
+data directory (see data_manager_conf.xml).
+
+Each language is installed as a self-contained stanza_resources directory
+(resources.json plus a per-language model subdirectory) using the
+memory-efficient ``default_fast`` package, matching the Stanza NLP tool which
+loads it via ``stanza.Pipeline(dir=models_path, package="default_fast")``.
+"""
+
+import argparse
+import json
+from pathlib import Path
+
+import stanza
+
+
+# Language display names
+STANZA_LANGUAGES = {
+    "en": "English",
+    "zh-hans": "Chinese (Simplified)",
+    "zh-hant": "Chinese (Traditional)",
+    "ar": "Arabic",
+    "fr": "French",
+    "de": "German",
+    "es": "Spanish",
+    "it": "Italian",
+    "pt": "Portuguese",
+    "nl": "Dutch",
+    "ru": "Russian",
+    "uk": "Ukrainian",
+    "pl": "Polish",
+    "ja": "Japanese",
+    "ko": "Korean",
+    "hi": "Hindi",
+    "tr": "Turkish",
+    "el": "Greek",
+    "hu": "Hungarian",
+    "sv": "Swedish",
+    "da": "Danish",
+    "nb": "Norwegian Bokmål",
+    "nn": "Norwegian Nynorsk",
+    "fi": "Finnish",
+    "ro": "Romanian",
+    "ca": "Catalan",
+    "cs": "Czech",
+    "sk": "Slovak",
+    "sl": "Slovenian",
+    "hr": "Croatian",
+    "sr": "Serbian",
+    "bg": "Bulgarian",
+    "lv": "Latvian",
+    "lt": "Lithuanian",
+    "et": "Estonian",
+    "he": "Hebrew",
+    "fa": "Persian",
+    "vi": "Vietnamese",
+    "th": "Thai",
+    "id": "Indonesian",
+    "af": "Afrikaans",
+    "eu": "Basque",
+    "gl": "Galician",
+    "hy": "Armenian",
+    "ka": "Georgian",
+    "ta": "Tamil",
+    "te": "Telugu",
+    "mr": "Marathi",
+    "ur": "Urdu",
+}
+
+
+def main():
+    parser = argparse.ArgumentParser(description="Download and register Stanza language models")
+    parser.add_argument("data_manager_json",
+                        help="Galaxy data manager JSON file (prepopulated with output paths)")
+    parser.add_argument("--model", action="append", required=True,
+                        help="Language code(s) to download (can be specified multiple times)")
+    args = parser.parse_args()
+
+    # Galaxy prepopulates the data manager JSON with the output dataset's
+    # extra files path, which is a writable directory. Models are downloaded
+    # there and Galaxy moves them into the managed data directory afterwards.
+    with open(args.data_manager_json) as fh:
+        params = json.load(fh)
+    target_dir = Path(params["output_data"][0]["extra_files_path"])
+    target_dir.mkdir(parents=True, exist_ok=True)
+
+    data_table_entries = []
+
+    for lang in args.model:
+        display_name = STANZA_LANGUAGES.get(lang, lang)
+
+        # Install each language as its own stanza_resources directory so the
+        # tool can load it directly with stanza.Pipeline(dir=models_path).
+        lang_dir = target_dir / lang
+        print(f"Downloading {display_name} ({lang}) models with the default_fast package...")
+        stanza.download(
+            lang=lang,
+            model_dir=str(lang_dir),
+            package="default_fast",
+            verbose=False,
+        )
+
+        data_table_entries.append({
+            "value": lang,
+            "name": display_name,
+            "lang": lang,
+            # Relative to extra_files_path; data_manager_conf.xml moves it into
+            # ${GALAXY_DATA_MANAGER_DATA_PATH}/stanza_models/${value}.
+            "models_path": lang,
+        })
+
+        print(f"Successfully downloaded {display_name} ({lang})")
+
+    data_manager_output = {
+        "data_tables": {
+            "stanza_models": data_table_entries
+        }
+    }
+
+    with open(args.data_manager_json, "w") as fh:
+        json.dump(data_manager_output, fh, sort_keys=True)
+
+    print(f"Summary: Successfully registered {len(data_table_entries)} model(s)")
+
+
+if __name__ == "__main__":
+    main()