denis256 notes and projects

Apache Airflow 3.3.0 was published on July 6, 2026, with asset partitioning as the change most likely to affect larger data platforms. The release also adds a task and asset state store, custom retry policy support, and experimental Java and Go task SDK work. It is not marked as a prerelease.

The full release notes and downloads are on the GitHub release page.

Asset partitions get more explicit

Airflow 3.3.0 expands asset partitioning beyond the base that landed in 3.2.0. A single upstream asset event can now map into downstream Dag runs with RollupMapper, FanOutMapper, and FixedKeyMapper with SegmentWindow. The useful part is not only the mapping. It is the control around it.

The new mappers cover many to one, one to many, and categorical rollup cases. They compose with time windows such as day, week, month, quarter, and year. The wait_policy setting decides when downstream partitioned runs fire, using WaitForAll or MinimumCount(n).

There is also a scheduler guardrail. Total fan out per upstream event is bounded by [scheduler] partition_mapper_max_downstream_keys, with mapper level override support. That cap matters because fan out mistakes are expensive in a scheduler. Airflow 3.3.0 also adds PartitionedAtRuntime, so a Dag can assign partition keys when the run starts instead of mapping them from an upstream event.

The REST API catches up with the CLI for partition clearing. The release notes mention a clearPartitions endpoint and partition_key or partition_date selectors on clearDagRuns. Producer Dag runs can also propagate partition_date to consumers of partitioned assets.

State and retries get real APIs

The new task and asset state store is the second large user facing change. Tasks can write state through task_state_store, and assets can do the same through asset_state_store. That state can survive retries and later runs, which is a practical answer to workflows that need remembered progress without building a custom table for every Dag.

The default storage is the metadata database. Deployments that need something else can set [workers] state_store_backend for a custom worker side backend. The store also supports per key retention, periodic garbage collection, and clear_on_success. Core API and Execution API management is included, so this is not only a Task SDK helper hidden inside worker code.

Retry behavior is also no longer only a fixed retries count. Airflow 3.3.0 adds pluggable retry policies, so a task can decide whether and when to retry based on custom logic. The obvious use case is exception aware retry behavior, where not every failure deserves the same delay or the same number of attempts.

Language tasks move outside Python

Airflow also adds a Coordinator layer for task implementations in Java and Go while the Dag stays in Python. A Dag can declare a stub with @task.stub(queue=...), then route execution to JavaCoordinator for JVM work or ExecutableCoordinator for self contained native binaries such as Go.

The release notes say the runtime proxies Variables, Connections, and XComs through the Execution API. That gives non Python task code a path into the same Airflow data plane, while scheduling and Dag authoring remain Python based.

This area needs a careful read before production use. The Coordinator layer and the Java and Go SDKs are experimental in 3.3.0 and may change in later versions. The cautious reading is simple: good release for early tests, not a reason to move critical task logic blindly.

Upgrade notes for operators

Airflow 3.3.0 adds rerun_with_latest_version for cleared, rerun, and backfilled Dag runs. Airflow 2.x reran with latest code. Airflow 3.x bundle versioning keeps the original version by default. This setting gives operators a direct way to choose the bundle version used for each path, with precedence from request parameter or CLI flag, then Dag setting, then [core] rerun_with_latest_version, then the built in defaults.

Provider example Dags are now separate Dag bundles. The bundle name format is apache-airflow-providers-<distribution>-example-dags, or <distribution>-example-dags for third party providers. If REST API clients filter for dags-folder, they need an update. Dag identifiers do not change.

Remote logging is the sharper migration point. Remote task log handler resolution moved to airflow_shared.logging.factory, and airflow.logging_config.load_logging_config is deprecated. Providers should expose a no argument from_config() classmethod that reads airflow.providers.common.compat.sdk.conf. The legacy fallback through airflow_local_settings.py is transitional and planned for removal in Airflow 4.0.

Metrics also need a dashboard pass. OpenTelemetry timer metrics now use Histograms instead of Gauges. dag_processing.last_run.seconds_ago now carries file_path, bundle_name, and file_name tags, while the old per file metric stays on by default unless [metrics] legacy_names_on disables it. There is also a new Deadlines page under Browse for roles that already have Dag Runs read and menu access.

Where to get it

• Release page: Apache Airflow 3.3.0
• Repo: apache/airflow
• Tag: 3.3.0