Data Platform Layers: What Each One Promises and Who Owns It

Open almost any data platform diagram and you will find boxes in a row: ingest, store, transform, serve. Below them, another row: bronze, silver, gold. The diagram is usually accurate as far as it goes. What it does not show is what any of it guarantees or who is answerable for it, which is what a reader needs. An analyst who asks “can I use this table?” is not asking which box it came out of. They are asking whether the numbers are right, whether they will still be right tomorrow, and whom to call when they are not.

This article is about turning those boxes into answers. It separates two things the diagram merges — the functions a platform performs and the states data passes through — explains what each data layer has to guarantee before the name means anything, works through where the popular layer naming misleads, and then takes on the question that decides how a platform scales organizationally: which data belongs to a central team and which belongs to the domain that produces it. The examples are invented for illustration.

Two different axes, drawn as one row of boxes

The confusion starts because the same diagram is asked to show two unrelated things.

Functional layers are the capabilities the platform provides: getting data in, storing it, transforming it, serving it, and managing it. They are about what the platform does, and they map to systems and to the teams that operate them.

Data layers are the states a given dataset passes through on its way from a source system to a consumer: raw as it arrived, cleaned and conformed, shaped for a particular use. They are about how far along a dataset is, and they map to tables and to the people accountable for their contents.

The two are orthogonal. Every data layer uses every functional capability: the raw layer is stored, transformed into the next layer, served to someone, and governed. Drawing them as one left-to-right flow implies that transformation happens once, between two boxes, which is exactly the misreading that produces platforms where nobody can say where a business rule lives.

Functional layer	What it provides	What breaks when it is weak
Ingestion	Getting data from sources reliably, with a record of what arrived and when. Covers batch extracts, change data capture, events, and files	Silent gaps and duplicates that surface as wrong numbers weeks later
Storage	Durable, queryable retention with a table abstraction, versioning, and retention policy	Cost surprises, unreadable history, and erasure requests that cannot be satisfied
Transformation	A place to express, test, schedule, and deploy the logic that turns one layer into the next, whichever of the ETL and ELT patterns it follows	The same business rule implemented three times with three answers
Serving	Delivering data in the shapes consumers need: SQL, extracts, APIs, feature values, files. Includes the arrangements that let one engine read another’s storage, such as external tables at the lake and warehouse boundary	Every consumer builds its own pipeline from the layer beneath
Management	Catalog, lineage, access control, quality checks, cost attribution, and the record of who owns what	Nobody can answer “where did this number come from” or “who may see this”

The management row is the one most often treated as a later phase. It is also the one that determines whether the other four can be operated by anyone other than their builders, which is why a platform without it tends to become the permanent responsibility of whoever set it up.

What a data layer has to promise

The data layers are worth defining by their guarantees rather than their contents, because the guarantee is what a consumer actually depends on.

Layer	What it promises	What it deliberately does not promise	Typical reader
Raw	This is what the source sent, unaltered, with a record of when it arrived and from where. It can be replayed. Where it holds data only until the next step consumes it, it is a staging area rather than an archive, and the two have different retention needs	That it is correct, complete, deduplicated, or comparable across sources	Pipeline engineers, and anyone reconstructing history after a bug
Cleansed and conformed	Types are enforced, duplicates resolved, keys made consistent, and the same entity means the same thing across sources	That it answers any particular business question, or is shaped conveniently for one	Data engineers and analysts building on top
Serving	These are the agreed definitions of business measures, shaped for a use, with a stated freshness and an owner	That it contains everything, or that unusual questions can be answered from it	Analysts, dashboards, applications, models

Stated that way, the layers are not degrees of quality on one scale. They differ in which guarantee they offer, and the raw layer’s refusal to promise correctness is a feature: it is the only copy that can settle an argument about what the source actually said.

A layer boundary becomes real when four things are true of it. Without these, the names are folder conventions.

• A stated contract covering two different things. Structure — field names, types, and whether a change is compatible with existing readers — can be checked automatically: a schema registry stores versioned schemas and validates evolution against a configured compatibility setting. Meaning cannot. What a field denotes, the grain of a row, and what a null signifies have to be written down and enforced by data tests and change review, because a registry sees no difference between an amount in dollars and the same field in cents.
• A stated freshness. Not “hourly” as an aspiration but a target with a measurement, since data freshness is what consumers silently assume when nobody states it.
• A named owner. Someone who decides what correct means for this dataset and is reachable when it is wrong, which is the distinction drawn in data owner, steward, and custodian.
• A change process. How a breaking change is announced, how long consumers have, and how the old shape is kept available in the meantime.

The practical test is to ask, for one table in each layer, who would be told if its meaning changed next week. Where the answer is “whoever notices,” that boundary is decoration.

Medallion, and what the name leaves out

The most widely used naming for these layers is the medallion pattern. Databricks describes it as a data design pattern for organizing data in a lakehouse, with the goal of incrementally improving the structure and quality of data as it moves through the layers.

• Bronze holds raw data from external sources, with table structures corresponding to the source systems as-is, plus metadata columns such as load date and time and the process that wrote it. Its stated focus is quick change data capture and keeping a historical archive of the source.
• Silver is matched, merged, conformed, and cleansed, producing what the description calls an enterprise view of key business entities and transactions. It emphasizes “just-enough” transformation and a model closer to third normal form, favoring speed and agility.
• Gold is consumption-ready and project-specific, de-normalized and read-optimized with fewer joins, carrying the final presentation-layer transformations and quality rules.

This is a sound pattern, and the same idea appears independently elsewhere. dbt’s recommended project structure uses staging, intermediate, and marts, and states the underlying principle in a way the metal names do not: the arc moves data from source-conformed to business-conformed, and stacking transformations in modular layers means each transformation is applied in only one place. That sentence is the actual point of layering. The medals are a label for it.

Two features of the naming cause trouble in practice. The first is that bronze, silver, and gold sound like grades of quality, which invites the reading that gold is the good data and bronze is the bad data. The layers differ in purpose, not merit: a raw record is exactly as it should be, and a gold table that answers the wrong question is not better for being gold. The second is that a three-medal ladder implies every dataset should climb it, which is not what the pattern requires and not what most platforms need.

The Databricks description does not discuss limits or failure modes — it is a glossary entry defining the pattern, not a critique of it. The following are the ways the pattern goes wrong in practice, and they are this article’s assessment rather than the source’s.

Failure	What it looks like	What to do instead
Layers as a mandatory ladder	A reference table with twelve rows exists three times because “everything goes through all three”	Require layers where they earn their keep. A small, stable, single-source table can be one table
Silver as a dumping ground	Hundreds of tables, no consistent grain, nobody sure which are current	Treat conformed entities as a curated set with owners, not as everything that has been cleaned once
Business rules in the wrong layer	Revenue is defined slightly differently in four gold tables	Define each rule once, at the layer where it stops being source-specific, and reuse it
Gold per dashboard	Each request produces a new gold table, and the layer becomes a report archive	Distinguish shared business entities from one project’s presentation table, and let the second be cheap and disposable
Bronze as an unmanaged archive	It grows without retention, nobody reads it, and it cannot actually be replayed because the pipeline changed	Give it a retention policy and test the replay path, or accept that it is not a replay mechanism

The cost side is worth stating plainly, because layering is usually presented as free. What it costs depends on how the layer is built, and the two cases are different enough that collapsing them leads to the wrong conclusion.

• Materialized as its own tables, a layer is another copy to store, another job to run and monitor, and another hop of latency between the source and the consumer. dbt’s guidance on materializations puts the trade directly: a table is fast to query but has to be rebuilt and does not pick up new source data on its own.
• Expressed logically — as a view, or as dbt’s ephemeral materialization, which is not built in the database at all but interpolated into dependent models as a common table expression — the layer costs no storage and no refresh job of its own, and is computed at query time. dbt’s guidance describes a view as containing the latest records from its source data, which is the right way to read it: the layer adds no staleness, but it inherits whatever its inputs have. A view over a table refreshed nightly is exactly as current as that table. Ephemeral models need one more step in that reasoning, because they have no result of their own: their SQL is compiled into whichever model selects from them, so freshness follows how that model is built. Interpolated into a view, the logic runs at query time; interpolated into a table, the result changes only when that table is rebuilt. The costs move to read time, and the same guidance notes that views become slow when they carry significant transformation or are stacked on other views; ephemeral models also cannot be queried directly, which makes them harder to debug.

The distinction matters because the two questions are separable. Is this a real boundary — does it promise something the previous layer does not, with its own contract, owner, and response when it breaks? And should it be materialized — is it queried often enough, or expensive enough to compute, to be worth storing? A boundary worth having but not worth storing becomes a view. A layer that fails the first question is overhead in either form, and the honest move is to remove it rather than to make it cheaper. Having several consumers makes a boundary easier to justify, but it is not the test: a single consumer that depends on a stated guarantee, or a step that exists to isolate one team from another’s changes, is a real boundary with one reader.

Where transformation actually belongs

The recurring argument on data teams is not whether to have layers but which rule goes in which one. A workable test is to ask what the rule depends on.

• Depends on the source system’s quirks — a column that encodes status as single letters, a timezone the source records in local time, a legacy identifier format. This belongs at the first transformation, because the knowledge is about the source and nothing downstream should need it.
• Depends on identifying an entity across systems — what counts as one customer when three systems each have a record, or which order records refer to the same order. This belongs in the conformed layer, because every domain that joins to it needs the same answer. Note that this is identity, not measurement: when an order counts as recognized revenue is an accounting definition, and it belongs wherever that definition is owned, which is usually finance rather than the shared layer.
• Depends on one use — a dashboard’s chosen comparison period, a model’s feature window, a report’s rounding convention. This belongs in serving, and should be easy to change without disturbing anyone else.

The second category needs one qualification, because it is easy to over-apply. Identity has to be shared: if two domains disagree about which records are the same customer, nothing joins. Measures computed about that entity often should not be shared, because different domains legitimately measure different things. What has to be common is the key and the meaning of the entity; what each domain does with it can differ as long as it says so.

The rule that is usually placed wrong is the second. Entity definitions pushed into serving get re-implemented per consumer, and the failure that follows is a specific one: four teams set out to implement the same definition, write four slightly different versions of it, and the same question returns four answers. That is not the same as four domains reporting different revenue figures because they deliberately measure different things, which is legitimate as long as each states what it measures. The first is drift in one definition; the second is several definitions, and only the first is a defect. Pushed too early, into the raw layer, they destroy the ability to recover the source’s actual statement, which is the one thing the raw layer exists to preserve. Neither mistake is visible on a diagram; both are visible in a reconciliation against the source.

Shared data and domain data

Layers organize data. They do not say who is responsible for it, and that is the question that decides whether a platform keeps working as an organization grows.

The default arrangement has one central data team own every transformation. It works at small scale and fails predictably at larger scale, for a reason that is structural rather than a matter of effort: the central team is the only group permitted to change the data, and it is the group that knows least about any particular domain’s meaning. Every domain’s change queues behind every other domain’s.

Zhamak Dehghani’s data mesh writing is the best-known articulation of the alternative, built on four principles: domain-oriented decentralized ownership, data as a product, self-serve data infrastructure as a platform, and federated computational governance. Two of them bear directly on layer boundaries.

Data as a product means a dataset published for others carries obligations, not just bytes. Her list of required characteristics includes discoverability, security, explorability, understandability, and trustworthiness, with a data product owner accountable for them. That is the same set of conditions as the layer contract above, applied to a team’s output, and it is the reason data product is a useful term rather than a rebranding of “table.”

Federated computational governance answers the question that decentralization otherwise leaves open. Some decisions have to be global or nothing interoperates — her example is how to identify a podcast listener consistently — while modeling within a domain stays local. The word computational is doing work: global policies are meant to be embedded in the platform and enforced automatically rather than published as guidance that teams are asked to follow.

Adopting the full model is a large organizational change and not the only option. The useful part for any platform, mesh or not, is the question of which decisions are global. Three tests separate them.

• Does more than one domain have to agree for the data to join? Customer and product identifiers, currency and time handling, and the identity of shared entities are global, because divergence there makes the data unusable together. This is narrower than it sounds: agreeing on who a customer is does not require agreeing on what makes one active.
• Does an outside obligation attach to it? Classification of personal data, retention periods, and access rules follow from regulation and contracts, and cannot be set per team.
• Would divergence be invisible until it caused harm? The problem is not that two measures differ but that they carry the same name and nobody knew. Marketing counting anyone who opened the app and finance counting anyone who paid are both legitimate definitions of an active customer, and forcing them to agree would make one of them useless. Two numbers differing is not by itself a defect. It becomes one when both are published as “active customers” with no qualification, so that a slide combines them or a reader assumes they are comparable. The remedy is naming, scope, and a stated definition — not a single global number.

Everything failing all three tests should stay with the domain. Pulling those decisions to the center produces the bottleneck decentralization was meant to remove, and it does so while adding the cost of a group deciding things it does not have the context to decide well.

Who owns which layer

Combining the two axes gives a division that survives contact with an incident. The point of writing it down is that during an incident nobody negotiates; they follow whatever was already agreed.

Layer	Platform team owns	Domain or producing team owns	Consumer owns
Raw	The ingestion mechanism and retention; that what arrived was stored; and that gaps or delays against the agreed ingestion window are detected and re-ingested	The source contract: what is emitted, its schema, notice before it changes, and enough information to tell whether a record was emitted at all	Nothing; consumers should not read it for production use
Cleansed and conformed	The transformation runtime, scheduling, testing framework, and lineage	The rules themselves, what correct means for its entities, and the response when a check fails	Raising a defect against a stated contract
Serving	Access mechanisms, performance, cost attribution	Shared business definitions and their freshness targets	Use-specific shaping, and its own dashboards and features
Management	Catalog, policy enforcement, quality tooling, and the audit trail	Populating it: ownership, classification, and descriptions for its own data	Discovering through it rather than by asking colleagues

Two rows deserve emphasis. In the raw row, the platform’s guarantee is about delivery, not content. If the source emitted a wrong value, the platform stored a wrong value correctly, and the defect belongs to the producer. Making this explicit prevents the common failure in which the platform team becomes the debugger of everyone else’s source systems.

Delivery, though, is more than storing what showed up. A record the source emitted and the platform never received is not the producer’s defect, and “we stored what arrived” says nothing about it. Someone has to notice, and the platform is the only party positioned to: it owns the connector, the offsets, and the schedule. That responsibility is worth stating as a specific obligation — for an agreed window, detect missing or late data and re-ingest it — which in practice means tracking completeness against something the source can confirm, such as a sequence, a log position, or a count for the period. This is distinct from guaranteeing the data is complete at the source, which the platform cannot know and should not claim.

In the management row, the split between providing a catalog and populating it is where most catalog programs fail. A platform team can deliver a complete tool and still end up with a catalog nobody trusts, because ownership and descriptions were left to be filled in by people with no reason to do it. If domains are expected to populate it, the expectation has to be stated, made easy, and checked — catalog coverage exists as a metric because the alternative is assuming.

A short checklist for a layer boundary

Question	If the answer is unclear
What does this layer promise that the one before it does not?	The layer is a copy, not a boundary, and should be removed rather than optimized
Who is told when a table here changes meaning, and how far in advance?	Consumers find out from a broken dashboard
Who decides what correct means for this data?	Defects route to whoever runs the pipeline rather than whoever knows the answer
Which business rules live here, and where else are they implemented?	The same measure is computed differently in several places
What is the stated freshness, and is it measured?	Consumers assume a number nobody promised
Which definitions here must be identical across domains?	Teams diverge quietly and reconcile in a meeting
Is this layer materialized, and if so why is a view not enough?	Storage and refresh cost is being paid for a boundary that could have been logical

Questions to explore further

• Pick one number your executives look at. How many layers does it pass through, and at which one is its definition fixed?
• Which of your layers has a stated freshness target that is actually measured, and which has one that is only assumed?
• If a producing team changed a field’s meaning tomorrow, who would find out first, and how?
• Which of your shared definitions are enforced by the platform, and which are enforced by a document nobody reads?

References

All sources were checked on September 15, 2026. The layer patterns cited are design conventions rather than product features and are described by their authors.

• Databricks Glossary, Medallion Architecture
• dbt Developer Hub, How We Structure Our dbt Projects
• Zhamak Dehghani, Data Mesh Principles and Logical Architecture (martinfowler.com, 2020)
• dbt Developer Hub, Materializations
• Confluent Documentation, Schema Registry