Releases: astronomer/astronomer-cosmos
Release list
v1.14.2
Behaviour Changes
These changes adjust observable behaviour of the ExecutionMode.WATCHER execution mode.
None of them break the public Cosmos API, but users relying on undocumented internals
(graph wiring assertions, XCom backup Variable names, retry-on-recovery semantics, or
retry log format) should review before upgrading.
ExecutionMode.WATCHER+depends_on_past=True: when the producer task has
depends_on_past=True(typically set viadefault_args), the producer-done gateway
task insideDbtTaskGroupis now wired downstream of every consumer task, in addition
to the producer. This is required so thatwait_for_downstreamgating behaves
correctly across DAG runs and the task group acts as a single unit that must fully
succeed before the next run starts. Users withdepends_on_past=False(the default)
see no topology change. See #2615.ExecutionMode.WATCHERdownstream retry on upstream recovery: dbt models that were
skipped after an upstream-failure event are now retried in the same DAG run when the
upstream task succeeds on retry. Previously these models remained skipped for the run.
See #2684.ExecutionMode.WATCHERconsumer-retry log format: the consumer's fallbackdbt
invocation no longer inherits the producer's internal--log-format jsonflag, so
retry task logs now default to dbt's normal text format. Users who relied on JSON output
in retry logs can opt in viaoperator_args={"dbt_cmd_flags": ["--log-format", "json"]}.
See #2713.ExecutionMode.WATCHERXCom-backup Variable key scheme: the per-model XCom backup
Variable key now includes the full task-group path and sanitises disallowed characters
(+/:) fromrun_id. External monitoring or cleanup scripts that match the
old key pattern will need updating. See #2629 and #2683.
Bug Fixes
- Sanitize disallowed characters from XCom backup variable key by @MichaelRBlack in #2629
- Prevent watcher producers from colliding on one XCom-backup key by @tatiana in #2683
- Retry watcher downstream models on upstream-failure recovery by @tatiana in #2684
- Fix
ExecutionMode.WATCHERinteraction withdepends_on_pastby @johnhoran in #2615 - Strip
--log-formatfrom producer flags on watcher consumer retry by @tatiana in #2713 - Fix duplicate
deferrablekwarg inDbtRunAirflowAsyncBigqueryOperatorby @pankajastro in #2616 - Fix dbt docs iframe
srcmissing deployment path prefix by @pankajastro in #2640 - Defer
TaskInstanceimport in cluster policy to fix Sentry init crash by @pankajastro in #2662 - Restore type hints broken by lazy imports in
cosmos/__init__.pyby @pankajastro in #2647 - Fix
ExecutionMode.WATCHERnon-dbt stdout being suppressed from logs by @pankajastro in #2654 - Fix test sensor retry behaviour in
ExecutionMode.WATCHERby @pankajkoti in #2658 - Fix watcher fallback selector for versioned dbt models by @pankajkoti in #2659
- Break out of iframe from Airflow 2 dbt Docs 404 link by @pankajastro in #2685
Docs
- Document source freshness aware execution for
ExecutionMode.WATCHERby @pankajastro in #2617 - Add reference docs for
DbtRunLocalOperator,DbtTestLocalOperator,DbtSnapshotLocalOperatorandDbtBuildLocalOperatorby @pankajastro in #2643 - Add watcher retry behaviour history documentation by @tatiana in #2600
- Add Apache Airflow® trademark on first prominent mention by @pankajkoti in #2624
- Sentence-case section headings by @pankajkoti in #2630
- Use
-for bullet points by @pankajkoti in #2631 - Drop decorative separator lines by @pankajkoti in #2632
- Normalize heading underlines in
docs/guides/anddocs/index.rstby @pankajkoti in #2664 - Fix broken cross-directory doc links by @pankajastro in #2694
- Fix broken external links in hand-written docs by @pankajastro in #2696
- Document support for Airflow 3.2 in the compatibility policy by @pankajastro in #2652
- Refresh the dbt/Airflow conflicts table to match the compatibility policy by @pankajastro in #2653
- Document incremental model limitation for
ExecutionMode.AIRFLOW_ASYNCby @pankajastro in #2642
Others
- Import
ParamValidationErrorfromairflow.sdkto silence deprecation warning by @pankajastro in #2645 - Import
DAGfromairflow.sdkto silence deprecation warning by @pankajastro in #2644 - Enforce docs style guide via pre-commit hook by @pankajkoti and @tatiana in #2633
- Add Airflow 3.2 to the test matrix in
CLAUDE.mdby @pankajastro in #2646 - Document the lazy-logging standard in
CLAUDE.mdby @pankajastro in #2679 - Extract watcher XCom-key helpers and inline single-use bindings by @pankajastro in #2673
- Remove leftover
scripts/airflow3directory by @pankajastro in #2661 - Fix
altered_jaffle_shopseed-dep CTE references by @pankajastro in #2690 - Skip Airflow 3.0 integration test stuck on
example_watcher_with_freshnessby @pankajastro in #2692 - Fix typo "constrantis" → "constraints" in tests env comment by @pankajastro in #2669
v1.14.2a4
Pre-release alpha of Cosmos 1.14.2.
Refresh of
a3with eleven additional milestone PRs (nine from @pankajastro, two from @pankajkoti). All applied cleanly on top of the a3 cherry-picks. See the release PR for cherry-pick provenance and manual conflict resolution: #2708.
Bug Fixes
- Sanitize disallowed characters from XCom backup variable key by @MichaelRBlack in #2629
- Prevent watcher producers from colliding on one XCom-backup key by @tatiana in #2683
- Retry watcher downstream models on upstream-failure recovery by @tatiana in #2684
- Fix
ExecutionMode.WATCHERinteraction withdepends_on_pastby @johnhoran in #2615 - Fix duplicate
deferrablekwarg inDbtRunAirflowAsyncBigqueryOperatorby @pankajastro in #2616 - Fix dbt docs iframe
srcmissing deployment path prefix by @pankajastro in #2640 - Defer
TaskInstanceimport in cluster policy to fix Sentry init crash by @pankajastro in #2662 - Restore type hints broken by lazy imports in
cosmos/__init__.pyby @pankajastro in #2647 - Fix
ExecutionMode.WATCHERnon-dbt stdout being suppressed from logs by @pankajastro in #2654 - Fix test sensor retry behaviour in
ExecutionMode.WATCHERby @pankajkoti in #2658 - Fix watcher fallback selector for versioned dbt models by @pankajkoti in #2659
Docs
- Document source freshness aware execution for
ExecutionMode.WATCHERby @pankajastro in #2617 - Add reference docs for
DbtRunLocalOperator,DbtTestLocalOperator,DbtSnapshotLocalOperatorandDbtBuildLocalOperatorby @pankajastro in #2643 - Add watcher retry behaviour history documentation by @tatiana in #2600
- Add Apache Airflow® trademark on first prominent mention by @pankajkoti in #2624
- Sentence-case section headings by @pankajkoti in #2630
- Use
-for bullet points by @pankajkoti in #2631 - Drop decorative separator lines by @pankajkoti in #2632
- Normalize heading underlines in
docs/guides/anddocs/index.rstby @pankajkoti in #2664 - Fix broken cross-directory doc links by @pankajastro in #2694
- Fix broken external links in hand-written docs by @pankajastro in #2696
- Document support for Airflow 3.2 in the compatibility policy by @pankajastro in #2652
- Refresh the dbt/Airflow conflicts table to match the compatibility policy by @pankajastro in #2653
Others
- Import
ParamValidationErrorfromairflow.sdkto silence deprecation warning by @pankajastro in #2645 - Import
DAGfromairflow.sdkto silence deprecation warning by @pankajastro in #2644 - Enforce docs style guide via pre-commit hook by @pankajkoti and @tatiana in #2633
- Add Airflow 3.2 to the test matrix in
CLAUDE.mdby @pankajastro in #2646 - Document the lazy-logging standard in
CLAUDE.mdby @pankajastro in #2679 - Extract watcher XCom-key helpers and inline single-use bindings by @pankajastro in #2673
- Remove leftover
scripts/airflow3directory by @pankajastro in #2661 - Fix
altered_jaffle_shopseed-dep CTE references by @pankajastro in #2690 - Skip Airflow 3.0 integration test stuck on
example_watcher_with_freshnessby @pankajastro in #2692 - Fix typo "constrantis" → "constraints" in tests env comment by @pankajastro in #2669
Milestone: Cosmos 1.14.2
Full changelog: astronomer-cosmos-v1.14.2a3...astronomer-cosmos-v1.14.2a4
v1.14.2a3
Pre-release alpha of Cosmos 1.14.2.
First alpha cut from
release-1.14with the full milestone applied (previous alphasa1/a2were point fixes off feature branches). See the release PR for cherry-pick provenance and manual conflict resolution: #2708.
Bug Fixes
- Sanitize disallowed characters from XCom backup variable key by @MichaelRBlack in #2629
- Prevent watcher producers from colliding on one XCom-backup key by @tatiana in #2683
- Retry watcher downstream models on upstream-failure recovery by @tatiana in #2684
- Fix
ExecutionMode.WATCHERinteraction withdepends_on_pastby @johnhoran in #2615 - Fix duplicate
deferrablekwarg inDbtRunAirflowAsyncBigqueryOperatorby @pankajastro in #2616 - Fix dbt docs iframe
srcmissing deployment path prefix by @pankajastro in #2640 - Defer
TaskInstanceimport in cluster policy to fix Sentry init crash by @pankajastro in #2662 - Restore type hints broken by lazy imports in
cosmos/__init__.pyby @pankajastro in #2647 - Fix
ExecutionMode.WATCHERnon-dbt stdout being suppressed from logs by @pankajastro in #2654
Docs
- Document source freshness aware execution for
ExecutionMode.WATCHERby @pankajastro in #2617 - Add reference docs for
DbtRunLocalOperator,DbtTestLocalOperator,DbtSnapshotLocalOperatorandDbtBuildLocalOperatorby @pankajastro in #2643 - Add watcher retry behaviour history documentation by @tatiana in #2600
- Add Apache Airflow® trademark on first prominent mention by @pankajkoti in #2624
- Sentence-case section headings by @pankajkoti in #2630
- Use
-for bullet points by @pankajkoti in #2631 - Drop decorative separator lines by @pankajkoti in #2632
- Normalize heading underlines in
docs/guides/anddocs/index.rstby @pankajkoti in #2664 - Fix broken cross-directory doc links by @pankajastro in #2694
- Fix broken external links in hand-written docs by @pankajastro in #2696
Others
- Import
ParamValidationErrorfromairflow.sdkto silence deprecation warning by @pankajastro in #2645 - Import
DAGfromairflow.sdkto silence deprecation warning by @pankajastro in #2644 - Enforce docs style guide via pre-commit hook by @pankajkoti and @tatiana in #2633
Milestone: Cosmos 1.14.2
Full changelog: astronomer-cosmos-v1.14.1...astronomer-cosmos-v1.14.2a3
v1.14.2a2
v1.14.2a1
v1.14.1
Bug Fixes
- Fix
ExecutionMode.WATCHERproducer retry behaviour by @tatiana in #2559 - Prevent watcher producer skip propagating to downstream tasks via gateway task by @johnhoran and @tatiana in #2597
- Keep watcher sensor polling when producer is still running by @pankajkoti in #2592
- Fix circular import error in Cosmos plugin discovery under Astro Runtime by @tatiana in #2538
- Fix
CosmosRichLoggercrash onNonelog message by @tatiana in #2540 - Enable inlets and outlets using dbt Fusion on Airflow 3 by @ichirotakami in #2561
- Fix incorrectly skipped source downstream tasks in
ExecutionMode.WATCHERby @pankajastro in #2563 - Fix duplicate logs in
dbt buildwhen source freshness is enabled by @pankajastro in #2564 - Warn and normalize when
source_rendering_behavior=Noneis passed by @pankajastro in #2570 - Gracefully handle
Variable.set()failures on Astro Remote Execution by @hkc-8010 in #2573 - Skip malformed YAML selectors instead of failing entirely by @YourRoyalLinus in #2577
Docs
- Update watcher test behavior docs for Cosmos 1.14.0 by @tatiana in #2549
- Add redirect for moved partial-parsing docs page by @tatiana in #2550
- Document
ExecutionMode.WATCHERanddepends_on_pastlimitation by @tatiana in #2602 - Restore memory-optimised imports docs for Cosmos < 1.14.0 by @pankajkoti in #2604
Others
- Speed up Airflow 3.1+ integration tests by caching InProcessExecutionAPI by @pankajkoti in #2547
- Improve stability of cache hash unit tests by @tatiana in #2539
- Fix mypy 1.20.0 type check failures by @pankajkoti in #2546
- Fix CI failures caused by docs build memory exhaustion by @pankajkoti in #2580
- Fix dbt Fusion broken integration tests by @tatiana in #2581
- Fix flaky
cosmos_manifest_selectors_exampleDAG in CI by @pankajkoti in #2593 - Reduce pre-commit autoupdate frequency PRs by @tatiana in #2544
- Bump
reviewdog/action-actionlintfrom 1.71.0 to 1.72.0 by @dependabot in #2542 - Skip watcher gateway test on Airflow 3.0 by @tatiana in #2607