Changing a Data Model Without Breaking the People Using It

A team improves a definition. Cancelled orders were being counted in revenue, which was wrong, so they are excluded from now on. The schema does not change. No column is added or removed, no type changes, every automated check passes, and the deployment is uneventful. Three weeks later a regional manager asks why their year-on-year growth collapsed in March, and it takes two days to find out that nothing happened in March except that the number started meaning something slightly different.

This is the shape of the problem. The changes that tooling catches are rarely the ones that hurt, and the ones that hurt most leave no trace in any schema. This article is about the full range — what actually breaks a consumer, what compatibility rules do and do not promise, why a changed definition passes every schema check and what kind of test does catch it, and what a workable process for publishing a change looks like. Product-specific statements are dated; the examples are invented.

Five kinds of change, and how visible each is

“Schema change” is too coarse a category to reason with. Sorting changes by what they affect makes the risk legible, and the last row is where most of the damage lives.

Change	Example	How a consumer finds out	Caught by tooling?
Column added	A new optional attribute	Usually not at all	Yes, and usually allowed
Column removed or renamed	order_total becomes order_amount	The query fails	Yes
Type changed	int to decimal; a string becomes nullable	The query fails, or silently rounds or nulls	Usually
Key or relationship changed	A dimension gains a second row per member; one-to-one becomes one-to-many	Totals inflate; joins fan out	Rarely
Meaning changed	Cancelled orders excluded; tax now included	Someone notices a trend break, eventually	Not by schema checks; by a test that fixes inputs and expected output

The first three are the ones everybody plans for, and they are genuinely the easy cases: they fail loudly, at a known moment, with an error message naming the column. The bottom two fail quietly and at an unknown moment, which is why they cost more even though they are less common.

The relationship row deserves more attention than it gets. Nothing in the column list changes when a dimension that used to hold one row per customer starts holding several — but every query joining on the natural key now fans out, and every sum over those rows inflates. The change is real, the schema is identical, and the failure appears as numbers that are wrong by a plausible-looking factor.

What compatibility rules actually promise

The formal machinery for the first three rows comes from message serialization, and it is worth understanding precisely because it is frequently over-trusted. Confluent’s Schema Registry documentation, checked in September 2026, defines compatibility in terms of who can read what, and the operational content is in the upgrade order.

Setting	Promise	Upgrade first
Backward	Consumers on the new schema can read data written with the old one — add fields that carry a default, remove fields	Consumers
Forward	Consumers on the old schema can read data written with the new one — add fields, remove fields that carry a default	Producers
Full	Both directions — add or remove fields that carry a default	Either; they can be upgraded independently
None	No checking	Everyone at once

“Optional” is doing specific work in that table, and it is format-dependent. In Avro a field is made nullable with a union type that includes "null", but nullable is not the same as safe to add. Confluent’s documentation is explicit that a new field needs a default value so that data encoded with the old schema can be read with the new one: had the default been omitted, “the new schema would not be backward compatible with the old one since it’s not clear what value should be assigned to the new field, which is missing in the old data.” Protobuf has an optional keyword and different rules again. Before relying on a row of that table, check what your serialization format means by the word.

Two refinements matter in practice. The transitive variants check a new schema against all previous versions rather than only the immediately preceding one, which is the setting you want for data anyone might re-read from an archive — non-transitive compatibility permits a chain of individually safe steps that together cannot read version one. And turning checking off is not neutral: the documentation is explicit that you then need to upgrade all producers and consumers at the same time, which is an operational commitment rather than an absence of one.

One naming caution, since it causes real confusion: different products use the words backward and forward in opposite senses. Rather than memorizing a label, ask the question the label encodes — who has to be upgraded first for this to be safe? That phrasing survives moving between tools, and it is the thing you actually need to know on the day of the change. The mechanics underneath, where a reader resolves data written under a different schema, are the subject of writer and reader schemas, and the registry that enforces the policy per subject is where the setting lives.

Now the limit, which is the reason this section exists. Compatibility guarantees readability, not meaning. A message that deserializes correctly can still carry a value computed under a new definition. Every rule in that table would pass the change in the opening paragraph, because no field was touched. Schema compatibility is a floor, and mistaking it for a ceiling is how teams end up confident about changes that are about to cause a two-day investigation.

Contracts, and where they stop

Inside the warehouse, the equivalent mechanism is a data contract, and transformation tools have made this concrete. In dbt’s implementation, checked in September 2026, an enforced contract must include every column’s name and data type, and may add constraints such as not_null depending on the platform. The enforcement is a preflight check at build time: dbt verifies that the model’s transformation will produce a dataset matching the contract, “or it will fail to build.”

That is a meaningful improvement over discovering the problem downstream, and it is worth adopting for anything others depend on — the guidance is to define contracts for the models that are public and relied on downstream, which is also a useful forcing function for deciding which of your tables those are. The listed breaking changes are the expected ones: removing a column, changing a data type, removing or modifying a constraint, or removing the model by deleting, renaming, or disabling it.

Read that list again with the five-row table in mind, and the gap is the same one. A contract enforces shape. Nothing in it prevents the values from being computed differently tomorrow, because the contract has no vocabulary for what a column means. A column typed decimal(18,2) named revenue satisfies its contract whether or not cancellations are included.

Which is not an argument against contracts; it is an argument for pairing them with the two things that do carry meaning. The first is a metric definition stated in words, versioned alongside the model, and reviewed when it changes. The second is a test that pins the definition to an example.

A definition change is detectable, just not by a schema check. dbt’s unit tests, for instance, “validate your SQL modeling logic on a small set of static inputs” before the model is materialized, with the expected output written out in the test. Give such a test three orders, one of them cancelled, and assert the revenue the model should produce: excluding cancellations from now on makes that test fail, at the moment the change is made, in the change author’s own pull request. That is a much earlier signal than a regional manager noticing a trend break. What the test cannot decide is whether the new number is the right one — it fails, someone reads the diff, and a person still judges whether the definition should move and who needs to be told. Automation turns a silent change into a visible one; the judgment stays human.

When the past changes

Correcting historical data is a separate kind of change, and it is easy to misclassify because nothing about the model moves at all. The structure is identical, the contract holds, and yesterday’s query returns a different answer today.

From a consumer’s position, this is indistinguishable from a breaking change, and it is worth treating with the same seriousness rather than as routine maintenance. Three questions decide how much ceremony it needs: how far back the correction reaches, whether any of the affected periods were reported externally, and whether a consumer has already acted on the old figures. A correction confined to last week’s unpublished data needs an entry in a log. A correction that moves a quarter someone filed needs to be announced before it lands, not explained afterwards.

The practical minimum is to make restatements detectable without anyone asking. A published dataset that carries a version or a last-restated timestamp lets a consumer notice that their number moved for a reason, which converts a mystery into a lookup. Without it, the only detection mechanism is a reconciliation against a figure someone wrote down, which is to say, luck.

Knowing who breaks

Every process for changing a model assumes you can find the consumers. That assumption is usually weaker than it feels.

Lineage answers part of it. Automatically captured lineage traces tables through transformations to dashboards and models, and where column-level lineage exists it can narrow the blast radius from “everything touching this table” to “these seven queries reading this column.” That is a genuine improvement and it is where impact analysis should start.

What it does not see depends on what has been connected to it, and the gaps are worth listing as categories of consumer: a spreadsheet someone refreshes through an ODBC connection, an application querying the warehouse directly with SQL built at runtime, an export that leaves the platform and is joined to something else elsewhere, a notebook on an analyst’s laptop, and a report whose logic lives in the BI tool rather than in a model.

Some of those are recoverable with the right integration, and it is worth knowing which. Platforms record what queries actually touched: Snowflake’s access history captures, per query, the source columns read and the projected columns returned, which covers the application and the notebook — though its own documentation warns that not every query in query history is recorded there, since “the structure of the SQL statement determines whether Snowflake records an entry.” Catalogs integrate with BI tools: Microsoft Purview inventories Power BI workspaces, datasets, reports, and dashboards and captures lineage among them and to external data assets, within documented limits on which source types are supported and on cases such as dynamic query parameters, where lineage is not captured.

So the honest statement is narrower than “lineage cannot see these.” A consumer is missing from lineage when it sits outside the tool’s integrations, permissions, or collection scope — which is a gap you can close by connecting something, at some cost. One category is different in kind: once data has been exported out of the platform, what happens to it afterwards leaves no trace any tool inside the platform can follow. Lineage sees what it is connected to; it does not see the organization.

Two things narrow that gap, and neither is a tool purchase. Query logs show who actually ran what against the table in the last ninety days, including the consumers no catalog knows about — this is the single most useful artifact for impact analysis and it is usually already being collected. And catalog coverage, in the sense of tables having a recorded owner and a stated set of consumers, is what turns “we think three teams use this” into a list you can email.

Where you genuinely cannot enumerate consumers, that is itself the finding, and it argues for the conservative path: keep the old shape available alongside the new one and let usage decline before removing anything.

Publishing a change

The software industry solved a version of this problem and the solution transfers, with one addition. Semantic Versioning‘s three rules are familiar — MAJOR “when you make incompatible API changes,” MINOR “when you add functionality in a backward compatible manner,” PATCH “when you make backward compatible bug fixes” — but the requirement that carries the weight is the one before them: software using it “MUST declare a public API,” and that declaration “SHOULD be precise and comprehensive.” Without a stated interface, nothing can be called incompatible, because nothing was promised.

That translates directly. The declared interface for a dataset is its columns, their types, its grain, and the definitions of its measures. Anything else — physical layout, intermediate tables, the SQL that produces it — is implementation you remain free to change, and should say so, because consumers otherwise assume that whatever they can see is guaranteed.

dbt’s framing of why this matters is worth borrowing whole, because it states the trade in one line: producers need the ability to modify logic and structure, there is a real cost to maintaining legacy endpoints forever, “but losing the trust of downstream users is far costlier.” The mechanism it offers is a deprecation date, and the reason a date helps is precise — it puts “a known boundary on the cost of that migration.” An old version that might be removed someday gets migrated by nobody; one that will be removed on a stated date gets scheduled.

A workable sequence for a breaking change, then:

• Classify it. Which of the five rows is it? Shape changes are caught by schema checks; relationship and meaning changes need tests that fix inputs and expected outputs, plus a human to trace the consumers and judge whether the new definition is the intended one.
• Find the consumers from lineage and query logs, and accept that the list is incomplete.
• Publish the new version alongside the old. Both available, both correct under their own definitions.
• Announce with a date. What changed, why, what it does to the numbers, and when the old one goes away.
• Watch usage fall in the query logs rather than waiting to be told migration is complete.
• Remove it on the date you said, because a deprecation date that slips twice stops being a date.

On frequency, the guidance is against versioning every small change: prefer “a predictable cadence (once or twice a year, communicated well in advance)” for bumping the current version and dropping columns nobody uses. The reasoning is that each version carries a real cost on both sides — migration work for consumers, and materializing several versions for producers — so batching breaking changes into an announced window is cheaper than a steady drip of individually small migrations.

One asymmetry is worth stating plainly for anyone deciding how much of this to adopt. Making a change reversible is cheap at the moment of the change and expensive afterwards. Keeping the old column for two quarters costs storage; removing it and discovering a regulatory report depended on it costs an incident. When the consumer list is uncertain — which is most of the time — the conservative option is the correct one, and it is only conservative in the short run.

Questions that reveal the wrong choice

Question	If the answer is unclear
Which of the five kinds of change is this?	A meaning change is handled with the process meant for a column rename
Is the declared interface written down — columns, types, grain, definitions?	Nothing can be called breaking, because nothing was promised
Does your compatibility setting check against all previous versions or only the last?	A chain of safe steps leaves archived data unreadable
Who must be upgraded first for this change to be safe?	Producers and consumers move in an order that breaks one of them
Does anything verify that a measure still means what it meant?	Definitions drift, and every automated check passes
Have you checked query logs, not just lineage, for consumers?	Spreadsheets, apps, and exports break without warning
When a period is restated, can a consumer tell?	A moved number becomes a mystery instead of a lookup
Does the deprecation have a date, and has that date ever slipped?	Nobody migrates, because nothing is actually going away

References

Product documentation was checked on September 16, 2026; compatibility policies and tool features change, so confirm them against current documentation. The versioning and interface principles are not time-sensitive.

• Confluent Platform Documentation, Schema Evolution and Compatibility
• dbt Documentation, Model Contracts
• dbt Documentation, Model Versions
• dbt Documentation, Unit Tests
• Snowflake Documentation, Access History
• Microsoft Learn, Metadata and lineage from Power BI into Microsoft Purview
• Semantic Versioning 2.0.0