Data Fabric
A data fabric integrates access and management across distributed data systems. IBM describes it as an architecture using automation to improve access and reduce silos. It does not require consolidating every source first, but it can include ingestion, transformation, and orchestrated data movement as well as virtualization.
Active metadata connects information about assets, relationships, and policies to discovery and automated actions. Its coverage and timeliness depend on supported connectors, collection intervals, and source metadata. An automatically collected catalog is not necessarily a complete or real-time picture of the estate.
What it typically bundles
“Data fabric” names a category rather than a single product, and vendors include different things under it. Four capabilities account for most of what is offered, and they are worth separating because they have different value and different failure modes.
- Automated cataloging. Scanning sources to register what exists, with classification inferred where possible — the foundation the rest sits on, and a close relative of a data catalog.
- Lineage collection. Recording how datasets derive from each other, gathered from pipelines rather than documented by hand — see data lineage for what that can and cannot answer.
- Virtualization or federated query. Querying across systems without copying, which removes a copy and adds a dependency on the source’s capacity at query time.
- Policy enforcement. Applying access and classification rules through supported enforcement points. Its value depends on the risks and control gaps it addresses; verify the access paths covered rather than assuming every consumer is protected.
Which is why the useful question about a proposal is not whether to adopt a data fabric but which of those four it delivers, how, and against which of your systems. A product strong at cataloging and weak at policy enforcement is a different purchase from the reverse, and both are sold under the same word.
It answers a different question from data mesh
The two terms are often presented as rival strategies, which makes an easy decision hard. IBM’s own contrast points at the difference: “a data mesh is a decentralized data architecture that organizes data by a specific business domain … to provide more ownership to the producers.”
| Data fabric | Data mesh | |
|---|---|---|
| Question it answers | How do people find and access data spread across systems? | Who is responsible for a dataset’s meaning and quality? |
| Means | Integration and automation driven by metadata | Domain teams, product owners, contracts, federated standards |
| Acquisition and operation | Components can be bought or built; integration and ongoing operation still need owners | Requires organizational decisions and a supporting technical platform; purchasing software alone does not establish the model |
| Leaves open | Who decides what a field means, and who answers when it is wrong | How discovery, lineage, and enforcement are implemented |
Fabric capabilities can support part of a mesh platform. Dehghani’s self-service platform also covers infrastructure provisioning, product development and deployment, and functions spanning the mesh. Cataloging, lineage, queries, and access controls alone do not establish that full lifecycle support. Check provisioning, versioning, testing, monitoring, retirement, domain autonomy, and automated policy enforcement before claiming that the platform requirement is met.
What automation does not settle
Two limits are worth stating plainly, because they are where fabric projects disappoint.
Metadata collected automatically describes structure and derivation. It does not decide whether “active customer” excludes trials, and no amount of inference settles that — someone has to choose, and the choice has to be recorded where consumers read it. A platform that has cataloged ten thousand tables and defined none of them has made the ambiguity easier to find.
And discovery does not create accountability. When a consumer finds a dataset through the fabric and the numbers look wrong, they need someone to ask. That is an ownership question, which is why governance and the owner, steward, and custodian distinction remain necessary whatever the technology does.
How fabric and mesh fit with data products, contracts, and adoption is worked through in Who Is This Table For?.
References
Discover more from Insightful Data Lab
Subscribe to get the latest posts sent to your email.