v1.11.3

Bug Fixes

• (back-ported) Fix resolution of packages-install-path when it uses env_var by @tatiana in #2194

v1.10.3

Bug Fixes

• (back-ported) Fix resolution of packages-install-path when it uses env_var by @tatiana in #2194

v1.12.0a4

Everything in v1.12.0a3 and additionally:

Breaking changes

• Drop Airflow 2.4 support by @pankajastro in #2161
• Drop Airflow 2.5 support by @pankajastro in #2165

Features

• Support real-time consumer updates when using ExecutionMode.WATCHER and InvocationMode.SUBPROCESS by @pankajastro in #2152

Enhancements

• Refactor ExecutionMode.WATCHER InvocationMode.SUBPROCESS log parser by @tatiana in #2183

Bug fixes

• Add Databricks Oauth mock profile by @fjmacagno in #2164
• Register listeners in Airflow 3 plugin implementation by @pankajastro in #2187

Docs

• Add docs for setup_operator_args param by @pankajastro in #2136
• Remove experimental flag for ExecutionMode.AIRFLOW_ASYNC by @pankajastro in #2153
• Clarify ExecutionMode.AIRFLOW_ASYNC dataset limits by @pankajkoti in #2167

Others

• Drop Airflow 2.4 support by @pankajastro in #2161
• Drop Airflow 2.5 support by @pankajastro in #2165
• Improve example DAG jaffle_shop_kubernetes.py by @tatiana in #2140
• Enable tests for Python 3.13 by @pankajastro in #2154
• Add Python 3.12 to CI integration tests matrix by @pankajastro in #2168
• Run pre-commit on ExecutionMode.WATCHER modules by @pankajkoti in #2150
• Refactor: Use shared airflow version constant by @pankajkoti in #2157
• Pin pydantic<2.0 for Airflow 2.6 compatibility by @pankajastro in #2172
• Remove duplicate dbt-duckdb dependency by @pankajastro in #2170
• Add targeted type: ignore for untyped decorators to fix mypy errors by @pankajastro in #2174
• Replace Legacy typing Aliases with Built-in Types for Python 3.10+ by @pankajastro in #2175
• Refactor to reuse load_method_from_module from _utils/importer.py by @pankajastro in #2176
• Remove try except block for cache import and unused python_version variable by @pankajastro in #2186
• Unpin Airflow to satisfy GitHub Security tab requirements by @pankajastro in #2171
• Update Python version for pyupgrade in pre-commit config by @pankajastro in #2190
• Add cooldown config in dependabot config by @pankajastro in #2189
• Bump zizmorcore/zizmor-action from 0.2.0 to 0.3.0 by @dependabot in #2156
• Bump actions/checkout from 5.0.1 to 6.0.0 by @dependabot in #2155
• Bump actions/checkout from 6.0.0 to 6.0.1 by @dependabot in #2178
• pre-commit autoupdate by @pre-commit-ci[bot] in #2162, #2173, #2191

Release prepared via PR: #2149 (branch: https://github.com/astronomer/astronomer-cosmos/tree/release-1.12)

v1.11.2

Bug fixes

• Force DbtProducerWatcherOperator retries to zero by @pankajkoti in #2114
• Fail DbtConsumerWatcherSensor tasks immediately when the DbtProducerWatcherOperator fails using Airflow context by @pankajkoti in #2126
• Fix forwarding DbtProducerWatcherOperator dbt build flags by by @michal-mrazek in #2127

Documentation

• Expand ExecutionMode.KUBERNETES guidance by @tatiana in #2139
• Document dataset-event limitation when using ExecutionMode.AIRFLOW_ASYNC by @varaprasadregani in #2143

v1.12.0a3

Breaking changes

• Introduced in the PR #2080. The following functions are expected to be used internally only to Cosmos, so we hope these won't impact end-users, but we are documenting the changes just in case:
  • generate_task_or_group receives render_config instead of its individual configurations, such as test_behavior, source_rendering_behavior and enable_owner_inheritance
  • create_task_metadata receives render_config instead of its individual configurations, such as test_behavior, source_rendering_behavior and enable_owner_inheritance
  • create_task_metadata now expects the node_converters argument
• Drop Python 3.9 support by @pankajastro in #2118

Features

• Support applying node_converter at a task level instead of task group level by @anyapriya in #1759
• Allow overriding DbtProducerWatcherOperator parameters via ExecutionConfig.setup_operator_args by @pankajastro in #2133
• Use deferrable sensors by default in ExecutionMode.WATCHER by @pankajastro in #2084

Enhancements

• Unify Airflow version handling into constants.py by @tatiana in #2089
• Refactor airflow/graph.py to simplify the code base by @tatiana in #2080
• Force watcher producer retries to zero by @pankajkoti in #2114
• Fail ExecutionMode.WATCHER consumer sensors immediately when the producer fails using Airflow context by @pankajkoti in #2126
• ExecutonMode.Watcher: fetch producer status asynchronously from the Airflow runtime so deferrable sensors fail immediately when the producer task fails by @pankajkoti in #2144

Bug fixes

• Clarify watcher deferrable failure messaging by @pankajkoti in #2124
• Remove empty test tasks when all tests are detached by @anyapriya in #2010
• Fix forwarding DbtProducerWatcherOperator dbt build flags by @michal-mrazek in #2127

Docs

• Document dataset-event limitation when using ExecutionMode.AIRFLOW_ASYNC by @varaprasadregani in #2143
• Expand ExecutionMode.KUBERNETES guidance by @pankajastro in #2139
• Add docs for deferrable DbtConsumerWatcherSensor by @pankajastro in #2115
• Fix reStructuredText formatting by @dnskr in #2132

Others

• Retry flaky Telemetry success test to stabilise CI by @pankajkoti in #2138
• Drop unused producer state xcom handling in ExecutionMode.WATCHER by @pankajkoti in #2145
• Drop Python 3.9 support by @pankajastro in #2118
• Remove unused Python3.9 uses from Github action CI by @pankajastro in #2117
• Bump actions/checkout from 5.0.0 to 5.0.1 by @dependabot in #2135
• Bump actions/checkout to 6.0.0 in GitHub workflows by @dependabot in #2147
• pre-commit autoupdate by @pre-commit-ci[bot] in #2134

Release prepared via PR: #2149 (branch: https://github.com/astronomer/astronomer-cosmos/tree/release-1.12)

v1.11.1

Bug fixes

• Fix ExecutionMode.WATCHER deadlock in Airflow 3.0 & 3.1 by @tatiana in #2087
• Fix ExecutionMode.AIRFLOW_ASYNC TaskGroup XCom issue by @tatiana in #2088
• Guard watcher callback exceptions to avoid hanging producer tasks by @pankajkoti in #2101
• Fix SQL templated field rendering for dynamically mapped tasks in Airflow 2 by @tatiana in #2119
• Fix ExecutionMode.WATCHER to use install_dbt_deps from ProjectConfig by @michal-mrazek in #2112

Enhancements

• Remove usage of contextmanager in plugins for accessing connections in Airflow >= 3.1.2 by @pankajkoti in #2073

Docs

• Improve ExecutionMode.AIRFLOW_ASYNC docs by @tatiana in #2103
• Add note about experimenting threads count for the Watcher Execution mode by @pankajkoti in #2083
• Fix minor documentation formatting issue by @dnskr in #2098
• Correct example YAML key from operator_args to operator_kwargs by @jx2lee in #2091

Others

• Fix broken CI due to fastapi incompatibility with cadwyn for Airflow 3 by @pankajkoti in #2076
• pre-commit autoupdate in #2078, #2104

v1.11.1a1

Bug fixes

• Fix ExecutionMode.WATCHER deadlock in Airflow 3.0 & 3.1 by @tatiana in #2087
• Fix ExecutionMode.AIRFLOW_ASYNC TaskGroup XCom issue by @tatiana in #2088

Enhancements

• Remove usage of contextmanager in plugins for accessing connections in Airflow >= 3.1.2 by @pankajkoti in #2073

Others

• Fix broken CI due to fastapi incompatibility with cadwyn for Airflow 3 by @pankajkoti in #2076
• pre-commit autoupdate in #2078

v1.12.0a1

Breaking changes

• Introduced in the PR #2080. The following functions are expected to be used internally only to Cosmos, so we hope these won't impact end-users, but we are documenting the changes just in case:
  • generate_task_or_group receives render_config instead of its individual configurations, such as test_behavior, source_rendering_behavior and enable_owner_inheritance
  • create_task_metadata receives render_config instead of its individual configurations, such as test_behavior, source_rendering_behavior and enable_owner_inheritance
  • create_task_metadata now expects the node_converters argument

Features

• Support applying node_converter at a task level instead of task group level by @anyapriya in #1759

Enhancements

• Remove usage of contextmanager in plugins for accessing connections in Airflow >= 3.1.2 by @pankajkoti in #2073
• Refactor airflow/graph.py to simplify code-base by @tatiana in #2080

Others

• Fix broken CI due to fastapi incompatibility with cadwyn for Airflow 3 by @pankajkoti in #2076

v1.12.0a2

Breaking changes

• Introduced in the PR #2080. The following functions are expected to be used internally only to Cosmos, so we hope these won't impact end-users, but we are documenting the changes just in case:
  • generate_task_or_group receives render_config instead of its individual configurations, such as test_behavior, source_rendering_behavior and enable_owner_inheritance
  • create_task_metadata receives render_config instead of its individual configurations, such as test_behavior, source_rendering_behavior and enable_owner_inheritance
  • create_task_metadata now expects the node_converters argument

Features

• Support applying node_converter at a task level instead of task group level by @anyapriya in #1759

Enhancements

• Remove usage of contextmanager in plugins for accessing connections in Airflow >= 3.1.2 by @pankajkoti in #2073
• Refactor airflow/graph.py to simplify code-base by @tatiana in #2080

Others

• Fix broken CI due to fastapi incompatibility with cadwyn for Airflow 3 by @pankajkoti in #2076
• Pre-commit updates: #2078

v1.11.0

Features

• Introduce ExecutionMode.WATCHER to reduce DAG run time by 1/5 in several PRs. Learn more about it here. This feature was implemented via multiple PRs, including:

  • Expose new execution mode by @tatiana @pankajastro @pankajkoti in #1999
  • Add DbtProducerWatcherOperator for the proposed ExecutionMode.WATCHER by @pankajkoti in #1982
  • Add DbtConsumerWatcherSensor for the proposed ExecutionMode.WATCHER by @pankajastro in #1998
  • Push producer's task completion status to XCOM by @pankajkoti in #2000
  • Add default priority_weight for DbtProducerWatcherOperator by @pankajkoti in #1995
  • Add sample dbt events for the dbt watcher execution mode by @pankajkoti in #1952
  • Add compiled_sql as a template fields on ExecutionMode.WATCHER when using run_results.json by @pankajastro in #2070
  • Set push_run_results_to_xcom kwargs correctly for invocation mode subprocess and Watcher mode by @pankajastro in #2067
  • Store compiled SQL as template field for dbt callback events in ExecutionMode.WATCHER by @pankajkoti in #2068
  • Add initial documentation for ExecutionMode.WATCHER by @tatiana in #2046
  • Support running State.UPSTREAM_FAILED tasks when WATCHER consumer upstream tasks fail by @tatiana in #2062
  • Fail sensor tasks immediately if the ExecutionMode.WATCHER producer task fails by @pankajastro in #2040
  • Add ``WATCHER``` to GitHub issue template by @tatiana in #2056
  • Add support for TestBehavior.AFTER_ALL with ExecutionMode.WATCHER by @pankajastro in #2049
  • Add support for TestBehavior.NONE with ExecutionMode.WATCHER by @pankajastro in #2047
  • Fix ExecutionMode.WATCHER behaviour with DbtTaskGroup by @tatiana in #2044
  • Fix Cosmos behaviour when using watcher with InvocationMode.DBT_RUNNER by @tatiana in #2048

• Add Airflow 3 plugin for dbt docs with multiple dbt projects support by @pankajkoti in #2009, check the documentation.

• Initial support to dbt Fusion by @tatiana in #1803. More details here.

• Support to prune sources without downstream references in dbt projects by @corsettigyg in #1988

• Allow to set task display name as a user-defined function by @corsettigyg in #1761

• Add dbt project's hash to dag docs to support dag versioning in Airflow 3 by @pankajkoti in #1907

• feat: Add Jinja templating support for dbt_cmd_flags by @skillicinski in #1899

• Add Scarf metric to collect the execution mode uses by @pankajastro in #1981

• Support Airflow 3.1 by @tatiana in #1980

• Add MySQL profile mapping by @Lee2532 in #1977

• Add sqlserver profile mapping by @pankajastro in #1737

Enhancement

• Improved the runtime speed of ExecutionMode.AIRFLOW_ASYNC by 36% , via the following tickets:
  • Use XCom to store sql when using ExecutionMode.AIRFLOW_ASYNC by @pankajastro in #1934
  • Refactor AIRFLOW_ASYNC teardown so it doesn't install the virtualenv by @pankajastro in #1938
  • Reuse the virtual env for AIRFLOW_ASYNC setup task by @pankajastro in #1939
• Improve dataset/asset experience in Cosmos by @tatiana in #2030
• Add downstreams to DbtNode by @wornjs in #2028

Bug fixes

• Fix tags extraction by @ms32035 in #1915
• Fix task flow operator args by @anyapriya in #2024

Documentation

• Add documentation for Airflow 3 Plugin supporting dbt docs for multiple dbt projects by @pankajkoti in #2063
• Add Cosmos Deferrable Operator Guide by @pankajastro in #1922
• Add dbt Fusion documentation by @tatiana in #1824 #1830
• Update dbt-fusion.rst to explicitly highlight it is in alpha by @tatiana in #1838
• Fix a bunch of docs build errors and warnings by @pankajkoti in #1886
• Add docs note for param virtualenv_dir for async execution mode by @pankajastro in #1969
• Use pepy.tech downloads badge in README by @pankajkoti in #1920
• Correct the default value of cache_dir by @seokyun.ha in #2027

Others

• Promote @corsettigyg to committer by @tatiana in #1985
• Add @pankajkoti and @pankajastro to contributors.rst by @tatiana in #1983
• Update setup script for airflow3 script by @dwreeves in #2023
• Prevent pytest from trying to test classes that aren't actually tests by @anyapriya in #2032
• Fix dag.test() for Airflow 3.1+ by syncing DAG to database bby @kaxil in #2037
• Disable Scarf in CI by @pankajastro in #2016
• Fix failing dbt Fusion tests when run in parallel in CI by @pankajkoti in #1896
• Fix MyPy issues related to ObjectStoragePath in main branch by @tatiana in #2012
• Cleanup example dbt event JSON dictionaries kept for XCOM referencby @pankajkoti in #1997
• Bump min hatch version that includes fixes for click>=8.3.0 by @pankajkoti in #1996
• Use official postgres image from Docker hub for kubernetes setup by @pankajkoti in #1986
• Use click<8.3.0 for hatch as click 8.3 breaks hatch by @pankajkoti in #1987
• Pin Airflow version in type check CI job by @pankajastro in #2003
• Improve comments after feedback on #1948 by @tatiana in #1963
• Fix running tests with dbt Fusion 2.0.0 preview versions by @tatiana in #1948
• Test hardening of dbt node having tags as unset or missing by @pankajkoti in #1918
• Fix Sphinx issue in the main branch by @tatiana in #2064
• pre-commit autoupdate in #2065, #2043, #2033, #2019, #1990, #2019, #2008, #1941, #1935, #1924
• GitHub dependabot update in #2051, #2050, #2038, #2022, #1947, #1955, #1946, #1944, #1945, #1928, #1921, #1917

Co-authored-by: Pankaj Koti pankaj.koti@astronomer.io
Co-authored-by: Pankaj Singh pankaj.singh@astronomer.io