Data Interval

A data interval is the period of data that one scheduled run is responsible for, as distinct from the moment the run executes. Airflow assigns every DAG run such an interval and defines its logical date as “the start of the data interval, not when the Dag is actually executed”; a run is normally scheduled after its interval has ended, which the documentation explains is “to ensure the run is able to collect all the data within the time period.” Azure Data Factory’s tumbling window trigger expresses the same thing as fixed-size, non-overlapping, contiguous windows, passing WindowStart and WindowEnd into the pipeline. Descriptions follow documentation checked in September 2026.

Why the distinction matters

A job that filters on “yesterday” computed from the clock produces different output every time it runs. A job that filters on the interval it was handed produces the same output whenever it runs — this morning, or in three months when someone reprocesses. That single property is what makes the following possible:

• Reruns that mean something. Re-running the window 2026-03-02 00:00–01:00 reprocesses that hour, not the hour in which you clicked the button.
• Filling gaps. Because the scheduler knows which intervals have run, it can create the ones that have not. Airflow calls this catchup, and notes that it is off by default in current versions; ADF’s tumbling window trigger generates the missing windows from its start time, oldest first, within the concurrency limit.
• Partition-aligned writes. When a run owns exactly one interval, it can overwrite exactly one destination partition, which turns a rerun into a replacement rather than an addition.

Where it goes wrong

• Reading “now” inside the task. One current_date in the query undoes the whole arrangement, because the run no longer depends only on its interval.
• Boundary and time zone mismatches. The interval is half-open — start inclusive, end exclusive — and the data’s timestamps may be in a different zone than the schedule. A row at exactly midnight belongs to one interval, and it should be the same one on every engine that reads it.
• Confusing the interval with data arrival. Records for an interval can arrive after that interval’s run has completed. ADF’s delay property postpones the start of processing without changing the window’s start time, which helps with small lags; genuinely late data needs a lookback or a reprocessing policy instead.
• Changing the size later. ADF’s documentation notes that a tumbling window trigger’s frequency and interval cannot be edited after publication, which is a reasonable constraint given that reruns and dependencies are evaluated against those boundaries.

How intervals interact with triggers, dependencies, and backfills is worked through in Running Pipelines in the Right Order. For the run objects themselves in one implementation, see Airflow DAG and DAG run.

References: Apache Airflow documentation, Dag Runs; Microsoft Learn, Create tumbling window triggers.