Data Platforms and Developer Platforms: Roles and Boundaries
Picture a company with two teams that both call themselves the platform team. One runs the Kubernetes clusters, the CI/CD system, and the secrets store for application developers. The other runs the lakehouse, the orchestration service, and the data catalog. A product team wants to ship a feature that scores customers with a model trained on warehouse data. It opens one ticket for a service environment and another for access to training data. The two platform teams then disagree about who provides the scheduled training job, who grants the service’s identity read access to customer tables, and who is paged when the nightly scoring run fails. The company is invented for illustration, but the argument is a common one.
Notice what the three open questions have in common. None of them is answered by knowing which cluster the job runs on or which catalog holds the table. Each asks who is accountable for something that both teams touch, and the word platform is covering several different scopes without anyone having written down where one ends and the next begins. This article separates those scopes, explains what turns a set of shared tools into a platform, and shows how to draw the line between what a platform team runs and what its users own.
Four things that get called a platform
The four terms below overlap in practice. The useful question is not which label is correct but what each one hides from its users, and who those users are.
| Term | Who uses it directly | What it hides | Typical capabilities |
|---|---|---|---|
| Infrastructure | Platform builders, operators, and a few advanced users | Hardware and the mechanics of provisioning resources | Compute, storage, networks, identity, managed databases and queues |
| Data platform | Teams that produce, transform, govern, and consume data | How data is ingested, stored, processed, secured, and served | Ingestion, storage and table formats, processing engines, orchestration, catalog and lineage, access control, serving |
| Developer platform | Teams that build, deliver, and run software | How code becomes a running, observable, secure service | Development environments, build and deployment automation, runtimes, observability, secrets, service templates |
| Internal developer platform | The same users as a developer platform | The specific choices one organization made behind its developer platform | Self-service workflows and golden paths exposed through portals, command-line tools, and APIs |
Infrastructure and data infrastructure
Infrastructure is the set of resources that workloads run on: compute, storage, networks, and identity, whether a company owns them or rents them from a cloud provider. The cloud infrastructure overview covers these building blocks. Data infrastructure is the part specialized for data, such as object storage, distributed query engines, streaming clusters, and the systems that control access to them.
Infrastructure answers what runs and where. It does not decide how a team should combine those resources to get its work done. A team handed a cloud account and a storage bucket has infrastructure. It still has to design ingestion, choose formats, secure the data, and keep everything running, so it does not yet have a platform.
Data platform
A data platform is the shared foundation used to collect, store, process, govern, and serve data. It runs on data infrastructure but presents its capabilities in terms of data work: registering a source, scheduling a transformation, publishing a table with an owner and a freshness expectation, or granting a consumer access. Its users include data engineers, analysts, and data scientists, and increasingly application teams that read or write analytical data.
Zhamak Dehghani’s description of a self-serve data platform shows that one data platform can present more than one surface. She describes a provisioning plane that manages the lifecycle of the underlying data infrastructure, which she expects mostly other planes and advanced developers to use directly. Above it sits a developer experience plane: the main interface for people building data products, which offers declarative ways to manage a product’s lifecycle and applies organization-wide standards automatically. A third plane works across all data products, for example to discover them. She notes that these planes are not strict layers.
The idea is useful outside data mesh as well. A platform that exposes only its provisioning level forces every user to understand the infrastructure, and one that exposes only a curated surface leaves advanced users stuck. Most users of a data platform should work through an interface designed for their task, with the lower level available rather than mandatory.
Developer platform
A developer platform does the same job for software delivery. The Cloud Native Computing Foundation’s platforms white paper describes a platform as an integrated set of capabilities, defined and presented according to what its users need, that gives many applications a consistent way to obtain common services. Its list of capability domains includes development environments, build and delivery automation, observability, infrastructure services, identity and secret management, security services, and artifact storage. Application developers are the main users, though the paper also names data scientists among a platform’s internal customers.
The same list shows where the emphasis differs. The white paper’s data services are the databases, caches, and object stores that applications use, described in terms of provisioning and operating them. A data platform is built around a further set of concerns that those services do not model: where data came from, what a column means, whether today’s load is complete, and who may see which rows. The two overlap in the middle, and the overlap is the subject of a later section.
Internal developer platform
An internal developer platform, often shortened to IDP, is the developer platform a particular organization assembles from its own cloud accounts, clusters, pipelines, and policies, and offers to its own teams. People use the term in different ways. Some mean the whole assembled capability, while others mean a product layer that orchestrates the tools beneath it. Two confusions are worth avoiding whichever meaning is in use.
- A developer portal is not the platform. A portal, such as one built with the open source Backstage framework, is one interface to platform capabilities. The CNCF white paper expects platforms to reach users through several interfaces, including web portals, APIs, command-line tools, and IDEs. A portal whose buttons open tickets for manual work adds a front end without adding self-service.
- An IDP is not a different kind of thing from a developer platform. It is the same concept scoped to one organization. The distinction matters mainly when someone compares an internal platform with a commercial product sold under the same name.
What turns shared tooling into a platform
Most organizations already share tools. The difference lies in how much work stays with the users. Four arrangements are easy to mistake for one another:
| Arrangement | What users get | What users still carry |
|---|---|---|
| Tool collection | Approved or licensed tools, often with a wiki page | Integration, configuration, operation, and upgrades |
| Shared infrastructure | Access to common accounts, clusters, or engines | Most configuration, plus requests to a central team for anything outside a fixed template |
| Managed service | A single capability that someone else operates, often a cloud provider | Integration with everything else, access policy, and cost control |
| Platform product | Integrated capabilities obtained through self-service interfaces, with a team accountable for how well they serve users | What they build with those capabilities, and decisions within the limits the platform sets |
In a 2018 article, Evan Bottcher defined a digital platform as a foundation of self-service APIs, tools, services, knowledge, and support, arranged as a “compelling internal product.” Two parts of that definition carry most of the weight.
The first is self-service. The CNCF white paper says users should be able to request and receive capabilities on their own and automatically, and gives the example of requesting a database and receiving its location and credentials through a command-line tool or a web form. Bottcher describes what happens when self-service stops halfway. In his example, a self-service tool let delivery teams create virtual machines from a fixed template, but installing packages, attaching storage, configuring a load balancer, or setting up monitoring still meant filing requests with central teams. Provisioning was automated while the work around it was not. He uses the term backlog coupling for the dependency this creates: a delivery team’s progress waits in another team’s queue.
The second is product ownership. The white paper states that a platform exists to serve its users’ requirements and should be designed and evolved around them, as any software product is. Bottcher argues that a platform becomes compelling when using it is easier than building and maintaining an alternative, and he points to Netflix’s description of its central tooling as a paved road that teams may leave if they take on the cost of their own tools. When a platform is adopted only because it is mandated, that is worth investigating. It may mean the platform is not yet easier than the alternative.
Team Topologies describes the same relationship in terms of teams. A platform team provides services that stream-aligned teams, the teams delivering along a flow of business value, consume in an X-as-a-Service mode with little ongoing interaction. The aim is to lower those teams’ cognitive load. Team Topologies also recommends a thinnest viable platform, meaning just enough capability to help without adding its own complexity. The CNCF white paper gives similar advice: build the thinnest viable platform layer over what managed providers already offer.
These criteria explain why a single managed service is usually not a platform. A managed warehouse removes server operation, but someone still has to decide how teams get access, how data is organized, how costs are attributed, and how changes are reviewed. If every team makes those decisions separately, the organization has a shared service, not a platform.
Where data and developer platforms meet
Data work is software work. Pipelines are code, transformations need tests and review, scheduled jobs need deployment, and models are served by services. Data work also has concerns that general software delivery tools do not model. The two platforms therefore tend to share lower-level capabilities and differ in the experience each offers its users.
| Capability | How it is usually divided | Reason |
|---|---|---|
| Identity, secrets, and access policy mechanisms | Common foundation, separated boundaries | One system of record for people and workloads, and one policy language, keep the two platforms from drifting apart. That is not the same as one credential everywhere: production and non-production, and workloads with different blast radius, stay separate. SPIFFE makes this explicit with trust domains, which its documentation advises separating for different locations or for environments where different security practices apply. |
| Account structure, networking, and cost allocation | Shared | Both platforms sit on the same cloud foundation and draw on the same budget. |
| Build, test, and deployment automation | Shared mechanism, different checks | A transformation project and a web service can use the same CI system, but one runs schema and data tests while the other runs unit and integration tests. |
| Observability | Shared collection, different signals | Services are judged mainly by latency and errors. Data products are also judged by freshness, completeness, and the correctness of values, which need their own service level objectives. |
| Compute runtime | Often shared | Batch jobs and services may run on the same clusters, but their resource profiles differ enough to require workload isolation. |
| Data catalog, lineage, schema management, data quality rules, metric definitions | Data-specific emphasis | They describe the meaning and trustworthiness of data. A developer platform’s catalog may list the same pipeline as a component with an owner; it does not usually hold what a column means or whether today’s load is complete. |
| Service templates, release strategies, API gateways | Software-delivery emphasis | They describe how software is released and exposed to callers. Data teams use them too, for a pipeline repository scaffold or an API that serves a data product, but they are not where a data platform’s distinctive value sits. |
The last two rows say emphasis rather than ownership on purpose, because the boundary is genuinely porous. Backstage, the open source catalog most often used as a developer portal, describes its software catalog as tracking all the software in an ecosystem, and names data pipelines in that list alongside services, websites, and libraries. A developer platform catalog that knows who owns a pipeline and a data catalog that knows what its output column means are both legitimate, and an organization that runs both needs to decide which one is authoritative for ownership before the two answers diverge.
Two separate decisions follow from this, and they get confused with each other often enough to be worth pulling apart. One is how the capabilities are shared, which is a technical dependency. The other is who owns the two products, which is an organizational arrangement. They combine freely: a single platform organization can still build its data experience on top of its developer platform, and two separate organizations can still agree to share an identity system.
| Decision | Common answers | What it costs |
|---|---|---|
| How capabilities are shared | The data platform builds on the developer platform’s compute, delivery, identity, and observability, adding data-specific capabilities on top | No duplicate stacks, but the data platform depends on another roadmap and must negotiate changes such as cluster upgrades that interrupt long-running batch work |
| Each platform has its own stack from the infrastructure up | Each can move at its own pace and accommodate its own workloads. Identity, policy, and cost reporting then have to be kept aligned deliberately, and users of both learn two ways to do the same thing | |
| Some capabilities shared, others duplicated on purpose | The common real answer. It works when the split is deliberate and written down, and decays when it is the residue of past decisions nobody revisited | |
| Who owns the products | One platform organization running two products | Each shared capability is owned once, but the organization must do product management for two very different user populations |
| Separate organizations | Each stays close to its users, at the price of coordinating anything shared through agreements between teams rather than inside one backlog |
None of these is right in general. The choice depends on how many teams use both platforms, whether a mature developer platform already exists, how different the data workloads are from service workloads, and which team has the skills to run the shared capabilities. What holds in every arrangement is narrower: each shared capability should have exactly one owner, each user group should have an interface designed for its work, and where a capability is deliberately duplicated, somebody should be able to say why.
Drawing the responsibility boundary
A platform takes work off its users, but it does not take all of their responsibility. The CNCF white paper states one side of the line directly: users should not have to operate the services a platform offers, so a team that needs a database should not manage the database server. It also expects platforms to be optional and composable, which means a team may run a capability the platform does not provide. In that case the team owns what it runs.
A workable rule follows from this. The platform team owns the capability and the promises made about it. The user team owns what it builds with the capability and the decisions only it has the knowledge to make. The situations below are illustrations, not a complete policy.
| Situation | Platform team owns | User team owns |
|---|---|---|
| A managed database obtained from the platform | Provisioning, patching, backups, availability of the service, secure defaults | Schema, queries, retention choices within policy, and how the application uses the database |
| A nightly pipeline fails | Availability of the orchestrator, clear failure signals, and retry mechanisms | The pipeline’s logic, handling of bad source data, and the decision to rerun, skip, or hold downstream use |
| A consumer requests access to a sensitive table | The mechanism to request, approve, log, and revoke access | The approval decision, which belongs to the data owner rather than to whoever runs the mechanism |
| Data quality checks | A way to define, run, and alert on checks | Which rules define correct data for the data product, and the response when checks fail |
| A golden path service template | Maintaining the template and publishing updates | Changes made after copying it, and adopting updates |
| Cloud spend | Cost visibility, attribution, and guardrails | Sizing workloads and spending decisions within the team’s budget |
Three questions resolve most gray areas. Who can change it? Who is alerted when it breaks? Who decides what correct looks like? When the answers point to different teams and nobody has agreed on how they coordinate, the boundary is not yet defined.
Applied to the argument at the start of this article, the rule produces a workable answer. The feature needs a scheduled training job on warehouse data and a scored output the product team uses, and three teams are involved. One reasonable division, for a company that runs a developer platform and a data platform separately, looks like this.
| Question raised in the ticket | Who owns it | Why |
|---|---|---|
| Who provides the scheduled training job | The data platform team provides the scheduling and execution capability; the product team owns the job definition | Running a scheduler is operating a service, which users should not have to do. What the job computes is a product decision. |
| Who grants the service read access to customer tables | The data platform team provides the request, approval, logging, and revocation mechanism; the owner of the customer data approves | The team that runs the mechanism is rarely the team that knows whether this use of the data is acceptable. |
| Who owns the model and the scoring logic | The product team | Only they know what a correct score is, and they are the ones who will change it. |
| Who is paged when the nightly run fails | The product team first, with a documented handoff | Most failures are in the job, not the platform. Routing every failure to the platform team makes it the debugger of everyone’s business logic. |
| Who handles a failure whose cause is not yet known | The product team opens, the platform team joins when the evidence points to the capability, and the agreement says what that evidence looks like | This is the case the other four rows do not cover, and the one that produces the argument if it is not settled in advance. |
The last row is the one worth agreeing before an incident rather than during one. A useful form is a short list of signals that move ownership: if the scheduler did not start the run, or the identity was refused, or the compute never became available, it is the platform’s; if the job ran and produced a wrong or missing result, it is the product team’s. Neither team needs to win the argument in general, only to know which of them picks up the phone tonight.
Boundaries tend to fail in recognizable ways. If the platform team starts writing business rules because it is the only team able to change a pipeline, the platform has taken on product decisions it is poorly placed to judge. If users must file tickets for routine changes, backlog coupling has returned. If teams quietly build their own tooling because the platform is slower, the organization pays for two stacks while believing it has one. Each of these shows up in ticket queues, duplicated infrastructure, and incident reviews well before it appears on a roadmap.
Questions to explore further
- Which shared capabilities in your organization do users experience mainly as tickets, and what would self-service for them require?
- If your data platform and developer platform were combined tomorrow, which capability would suddenly have two owners?
- Which access or release approvals are made by the team that runs the mechanism rather than by the owner of the data or service?
References
All sources were checked on September 15, 2026. The definitions cited are conceptual and do not depend on product versions.
- CNCF TAG App Delivery, Platforms White Paper
- Evan Bottcher, What I Talk About When I Talk About Platforms (martinfowler.com, 2018)
- Zhamak Dehghani, Data Mesh Principles and Logical Architecture (martinfowler.com, 2020)
- Team Topologies, Key Concepts
- Backstage Documentation, Software Catalog
- SPIFFE Documentation, SPIFFE Concepts
Discover more from Insightful Data Lab
Subscribe to get the latest posts sent to your email.