Skip to content

Releases: astronomer/astronomer-cosmos

v1.14.2

Choose a tag to compare

@tatiana tatiana released this 21 May 12:20

Behaviour Changes

These changes adjust observable behaviour of the ExecutionMode.WATCHER execution mode.
None of them break the public Cosmos API, but users relying on undocumented internals
(graph wiring assertions, XCom backup Variable names, retry-on-recovery semantics, or
retry log format) should review before upgrading.

  • ExecutionMode.WATCHER + depends_on_past=True: when the producer task has
    depends_on_past=True (typically set via default_args), the producer-done gateway
    task inside DbtTaskGroup is now wired downstream of every consumer task, in addition
    to the producer. This is required so that wait_for_downstream gating behaves
    correctly across DAG runs and the task group acts as a single unit that must fully
    succeed before the next run starts. Users with depends_on_past=False (the default)
    see no topology change. See #2615.
  • ExecutionMode.WATCHER downstream retry on upstream recovery: dbt models that were
    skipped after an upstream-failure event are now retried in the same DAG run when the
    upstream task succeeds on retry. Previously these models remained skipped for the run.
    See #2684.
  • ExecutionMode.WATCHER consumer-retry log format: the consumer's fallback dbt
    invocation no longer inherits the producer's internal --log-format json flag, so
    retry task logs now default to dbt's normal text format. Users who relied on JSON output
    in retry logs can opt in via operator_args={"dbt_cmd_flags": ["--log-format", "json"]}.
    See #2713.
  • ExecutionMode.WATCHER XCom-backup Variable key scheme: the per-model XCom backup
    Variable key now includes the full task-group path and sanitises disallowed characters
    (+ / :) from run_id. External monitoring or cleanup scripts that match the
    old key pattern will need updating. See #2629 and #2683.

Bug Fixes

  • Sanitize disallowed characters from XCom backup variable key by @MichaelRBlack in #2629
  • Prevent watcher producers from colliding on one XCom-backup key by @tatiana in #2683
  • Retry watcher downstream models on upstream-failure recovery by @tatiana in #2684
  • Fix ExecutionMode.WATCHER interaction with depends_on_past by @johnhoran in #2615
  • Strip --log-format from producer flags on watcher consumer retry by @tatiana in #2713
  • Fix duplicate deferrable kwarg in DbtRunAirflowAsyncBigqueryOperator by @pankajastro in #2616
  • Fix dbt docs iframe src missing deployment path prefix by @pankajastro in #2640
  • Defer TaskInstance import in cluster policy to fix Sentry init crash by @pankajastro in #2662
  • Restore type hints broken by lazy imports in cosmos/__init__.py by @pankajastro in #2647
  • Fix ExecutionMode.WATCHER non-dbt stdout being suppressed from logs by @pankajastro in #2654
  • Fix test sensor retry behaviour in ExecutionMode.WATCHER by @pankajkoti in #2658
  • Fix watcher fallback selector for versioned dbt models by @pankajkoti in #2659
  • Break out of iframe from Airflow 2 dbt Docs 404 link by @pankajastro in #2685

Docs

Others

v1.14.2a4

v1.14.2a4 Pre-release
Pre-release

Choose a tag to compare

Pre-release alpha of Cosmos 1.14.2.

Refresh of a3 with eleven additional milestone PRs (nine from @pankajastro, two from @pankajkoti). All applied cleanly on top of the a3 cherry-picks. See the release PR for cherry-pick provenance and manual conflict resolution: #2708.

Bug Fixes

  • Sanitize disallowed characters from XCom backup variable key by @MichaelRBlack in #2629
  • Prevent watcher producers from colliding on one XCom-backup key by @tatiana in #2683
  • Retry watcher downstream models on upstream-failure recovery by @tatiana in #2684
  • Fix ExecutionMode.WATCHER interaction with depends_on_past by @johnhoran in #2615
  • Fix duplicate deferrable kwarg in DbtRunAirflowAsyncBigqueryOperator by @pankajastro in #2616
  • Fix dbt docs iframe src missing deployment path prefix by @pankajastro in #2640
  • Defer TaskInstance import in cluster policy to fix Sentry init crash by @pankajastro in #2662
  • Restore type hints broken by lazy imports in cosmos/__init__.py by @pankajastro in #2647
  • Fix ExecutionMode.WATCHER non-dbt stdout being suppressed from logs by @pankajastro in #2654
  • Fix test sensor retry behaviour in ExecutionMode.WATCHER by @pankajkoti in #2658
  • Fix watcher fallback selector for versioned dbt models by @pankajkoti in #2659

Docs

Others

Milestone: Cosmos 1.14.2

Full changelog: astronomer-cosmos-v1.14.2a3...astronomer-cosmos-v1.14.2a4

v1.14.2a3

v1.14.2a3 Pre-release
Pre-release

Choose a tag to compare

Pre-release alpha of Cosmos 1.14.2.

First alpha cut from release-1.14 with the full milestone applied (previous alphas a1 / a2 were point fixes off feature branches). See the release PR for cherry-pick provenance and manual conflict resolution: #2708.

Bug Fixes

  • Sanitize disallowed characters from XCom backup variable key by @MichaelRBlack in #2629
  • Prevent watcher producers from colliding on one XCom-backup key by @tatiana in #2683
  • Retry watcher downstream models on upstream-failure recovery by @tatiana in #2684
  • Fix ExecutionMode.WATCHER interaction with depends_on_past by @johnhoran in #2615
  • Fix duplicate deferrable kwarg in DbtRunAirflowAsyncBigqueryOperator by @pankajastro in #2616
  • Fix dbt docs iframe src missing deployment path prefix by @pankajastro in #2640
  • Defer TaskInstance import in cluster policy to fix Sentry init crash by @pankajastro in #2662
  • Restore type hints broken by lazy imports in cosmos/__init__.py by @pankajastro in #2647
  • Fix ExecutionMode.WATCHER non-dbt stdout being suppressed from logs by @pankajastro in #2654

Docs

Others

Milestone: Cosmos 1.14.2

Full changelog: astronomer-cosmos-v1.14.1...astronomer-cosmos-v1.14.2a3

v1.14.2a2

v1.14.2a2 Pre-release
Pre-release

Choose a tag to compare

Test: Forward non-dbt stdout to task log in WATCHER mode in #2654

v1.14.2a1

v1.14.2a1 Pre-release
Pre-release

Choose a tag to compare

Test: Fix dbt docs iframe src missing deployment path prefix PR: #2640

v1.14.1

Choose a tag to compare

Bug Fixes

  • Fix ExecutionMode.WATCHER producer retry behaviour by @tatiana in #2559
  • Prevent watcher producer skip propagating to downstream tasks via gateway task by @johnhoran and @tatiana in #2597
  • Keep watcher sensor polling when producer is still running by @pankajkoti in #2592
  • Fix circular import error in Cosmos plugin discovery under Astro Runtime by @tatiana in #2538
  • Fix CosmosRichLogger crash on None log message by @tatiana in #2540
  • Enable inlets and outlets using dbt Fusion on Airflow 3 by @ichirotakami in #2561
  • Fix incorrectly skipped source downstream tasks in ExecutionMode.WATCHER by @pankajastro in #2563
  • Fix duplicate logs in dbt build when source freshness is enabled by @pankajastro in #2564
  • Warn and normalize when source_rendering_behavior=None is passed by @pankajastro in #2570
  • Gracefully handle Variable.set() failures on Astro Remote Execution by @hkc-8010 in #2573
  • Skip malformed YAML selectors instead of failing entirely by @YourRoyalLinus in #2577

Docs

  • Update watcher test behavior docs for Cosmos 1.14.0 by @tatiana in #2549
  • Add redirect for moved partial-parsing docs page by @tatiana in #2550
  • Document ExecutionMode.WATCHER and depends_on_past limitation by @tatiana in #2602
  • Restore memory-optimised imports docs for Cosmos < 1.14.0 by @pankajkoti in #2604

Others

v1.14.1a4

v1.14.1a4 Pre-release
Pre-release

Choose a tag to compare

Cut from release PR: #2601

v1.14.1a3

v1.14.1a3 Pre-release
Pre-release

Choose a tag to compare

Cut from release PR: #2601

v1.14.1a2

v1.14.1a2 Pre-release
Pre-release

Choose a tag to compare

  • Fix watcher producer retries behaviour by @tatiana in #2559
    • Fixes ExecutionMode.WATCHER_KUBERNETES as well

#2559

v1.14.1a1

v1.14.1a1 Pre-release
Pre-release

Choose a tag to compare

#2559