Airflow TaskFlow API

Python syntax constructs a workflow

The Airflow TaskFlow API uses decorators such as @dag and @task to construct workflows from Python functions. In Airflow 3.1.0 these decorators can be imported from airflow.sdk. Calling a decorated task while constructing a DAG creates a task reference. Its function body runs later as a task instance; the call does not immediately return the calculated business value.

Read a nested call correctly

In check_quality(calculate()), calculate() creates a reference to the calculation task. Passing that reference into check_quality() adds the dependency and connects the future result to its input. During execution, calculate produces a value and TaskFlow uses XCom to make it available to check_quality. The two functions need not run on the same machine or in the same Python process.

Choose what to return

Return a small count or a durable reference rather than a large DataFrame. For example, a task can write a report to shared storage and return its immutable version URI. The consuming task still needs permission to read it, and retention must keep that version available. XCom transports a value; it does not grant storage access or preserve a worker-local file.

Keep execution rules explicit

Decorators do not remove retry, timeout, or data-quality responsibilities. If a task commits a result and then fails, another attempt can repeat the external effect. Test transformation logic separately, inspect the constructed task graph, and then verify task execution in the intended deployment. Inspecting the graph checks dependencies; verifying task execution also requires running the tasks through Airflow with the intended XCom backend.

Worked example: Apache Airflow Fundamentals: DAGs, Tasks, and Scheduling.

Further reading

• Apache Airflow 3.1.0 — Pythonic Dags with the TaskFlow API
• Apache Airflow 3.1.0 — XComs