Data Backfill

A data backfill populates or recomputes a selected historical range. It can initialize a new table, apply a corrected calculation, or repair missed input. Specify the output dates, historical input version, transformation and reference-data versions, and publication plan. A date range alone does not identify a reproducible computation.

Rebuilding March 1–7 for a trailing seven-day feature requires input starting six days before March 1. The calculation for March 1 needs the six preceding dates as well. Rebuilding only the output dates as input silently changes the feature. Inventory downstream aggregates and caches so consumers do not combine old derived values with new facts. Where the rebuild changes figures people have already reported, the change is a restatement and needs to be announced as one.

Reuse versioned business logic, but limit concurrency and coordinate overlapping daily loads. Publishing an old backfill last can overwrite fresher data even when every transaction is atomic — the ordering problem that keyed, version-aware writes and idempotent application exist to prevent. Per-day commits may expose mixed versions while the range is rebuilt; use a versioned publication boundary when consumers need a consistent range — see atomic publication. Verify the result against the source before declaring the gap closed: that is what reconciliation is for. An orchestrator schedules runs but does not supply missing source history.

Further reading

• Apache Airflow documentation (stable), Backfill