Data Lineage
Data lineage records relationships between data sources, transformations, and outputs. It helps explain where an output came from and which downstream resources may depend on a changed input. Depending on the system, records can describe tables, columns, or particular processing runs.
Trace a changed payment state
An order source feeds a cleaned table, which feeds a daily report and an assistant's search index. If a new payment state changes the cleaned table, lineage identifies paths that may carry the change to both consumers. The team can then contact their owners and inspect the relevant rules.
For an incorrect result, work backward along the recorded path: which input version, transformation, and run produced it? This narrows investigation but does not automatically identify the cause.
A recorded path has a coverage boundary
Manual spreadsheets, exports, dynamic queries, or unsupported services may be missing. An absent edge does not prove that no dependency exists. Confirm important consumers before making a breaking change.
Lineage shows derivation and dependency, not factual truth. A perfectly recorded pipeline can faithfully propagate an incorrect source value. Combine lineage with versioned definitions, quality evidence, and responsible contacts.
Reference: Google Cloud: Data lineage.
Discover more from Insightful Data Lab
Subscribe to get the latest posts sent to your email.
