Skip to content

add rastervision check command to validate config URIs - #2328

Open
Pranava-Kumar wants to merge 1 commit into
azavea:masterfrom
Pranava-Kumar:feat/check-command
Open

add rastervision check command to validate config URIs#2328
Pranava-Kumar wants to merge 1 commit into
azavea:masterfrom
Pranava-Kumar:feat/check-command

Conversation

@Pranava-Kumar

Copy link
Copy Markdown

Summary

Adds rastervision check CLI command that validates all URIs in a PipelineConfig without running the pipeline. Users can verify their config files are accessible before kicking off a long-running job, avoiding late-stage failures from bad S3 paths or missing files.

Changes

  • collect_uris() utility (file_system/utils.py): Recursively walks a pydantic BaseModel tree to find all fields ending in uri/uris. Handles nested models, lists, dicts, and None values.
  • check CLI subcommand (cli.py): Loads a config module via get_configs(), collects all URIs, checks each with file_exists(), and reports results with color-coded output and a summary line.
  • 19 tests (tests/pipeline/test_cli_check.py): 12 unit tests for collect_uris() and 7 integration tests for the CLI command using CliRunner.

Usage

rastervision check path/to/config.py -a root_uri /tmp/output

Output:

============================================================
  PipelineConfig [0]
============================================================
  [✓] dataset.train_scenes[0].raster_source.uris[0]: /data/img.tif
  [✓] dataset.train_scenes[0].aoi_uris[0]: /data/aoi.geojson
  [ ] dataset.validation_scenes[0].raster_source.uris[0]: /data/missing.tif

  Summary: 3 URI(s) checked, 2 found, 1 missing

Related Issue

Closes #1154

Adds a new CLI subcommand that recursively walks a PipelineConfig tree,
collects all URI fields via collect_uris(), and verifies they are
accessible using file_exists() before running the pipeline.

Implements issue azavea#1154.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

REQUEST: rastervision check command

1 participant