Skip to content

TRR BACKEND: Operational visibility for pipeline runs #32

Description

@therealityreport

Goal: Make pipeline runs easier to debug without digging through raw DB rows.

TODO (pick subset)

  • Add CLI command: python -m trr_backend.cli pipeline status --verbose <run-id> (stage timing + error_details)
  • Add --json flag for machine-readable output (future automation)
  • Stage timings summary output:
    • stage name, status, duration_ms, processed/skipped/failed, error_message (if any)
  • Add CLI command: pipeline logs <run-id> (if logs are stored somewhere)
  • Include manifest_key in status output + optional presigned URL generator (service role only, if AWS tooling exists)
  • Add pipeline.run_events table (optional) for structured events:
    • stage started/finished
    • counts
    • warnings
  • Add S3 manifest enrichment:
    • include tool versions / git SHA / runtime metadata
  • Add optional Sentry breadcrumbs (if Sentry is configured)

Acceptance

  • Operator can run one command and know which stage failed and why, without opening Supabase UI
  • An operator can identify "what broke and where" within 60 seconds using CLI + DB

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions