Data Platform Leadership Decisions for AI

A fictional regional insurer hired a new vice president of data platform in January. By the end of the first quarter, four requests had reached her. Finance wanted a reconciled monthly close dataset. The claims organization wanted an assistant that would answer adjusters’ questions from claim notes and policy documents. The claims and policy administration teams each wanted to own their data instead of waiting in the central data team’s queue, which held 90 open requests. And the chief financial officer pointed out that the platform bill had grown 15 percent over the year and asked for a 10 percent cut. The insurer and every figure in this case are invented for illustration, and amounts are in thousands of dollars unless stated otherwise.

Each request has a reasonable sponsor, and each draws on the same engineers, the same source systems, and the same budget. Deciding one changes the options for the others, so she treats them as a single allocation problem: what to build, who should own it, what the AI work needs that analytics did not, and what each choice costs.

Funding data products by what depends on them

A data product is data deliberately maintained for a defined group of consumers, with a stated meaning, a freshness commitment, an owner, and support. Maintenance is the recurring cost, so the first question is which shared datasets justify it. Her team lists four candidates. The consumer counts come from query logs and request history, and the costs are team estimates. “Duplicate pipelines retired” counts separately maintained pipelines that produce the same entity for different teams and could be switched off once the product exists.

Candidate	Build cost	Run cost per year	Analytics consumers	AI use cases	Duplicate pipelines retired	Run cost avoided per year
Policy	300	60	9	1	6	90
Party (policyholders, claimants, agents)	500	80	11	1	8	120
Claims events	400	90	7	2	4	70
Claim documents	450	120	1	2	0	0

To compare them over three years, add the build cost to three years of run cost and subtract three years of avoided run cost. Policy comes to 300 + 3 × 60 − 3 × 90 = 210. Party comes to 500 + 240 − 360 = 380, Claims events to 400 + 270 − 210 = 460, and Claim documents to 450 + 360 − 0 = 810. Dividing by the number of consumers, analytics and AI together, gives what each dependent team adds to the cost.

Candidate	Three-year net cost	Consumers	Net cost per consumer
Policy	210	10	21.0
Party	380	12	31.7
Claims events	460	9	51.1
Claim documents	810	3	270.0

The avoided cost counts only when the duplicate pipelines are actually switched off. A product that coexists with its predecessors saves nothing, which is why data product adoption is read from retired work as well as from registered consumers. The per-consumer figure is also crude, since it weighs a month-end close process and a one-off exploration equally. It supports one comparison. Policy, Party, and Claims events consolidate existing duplicated work, so part of their cost is paid back. Claim documents is a new capability with nothing to retire, and at 270 per consumer it costs about 13 times as much as Policy for each dependent team. That does not make it a bad investment. Its worth depends on what the assistant is worth, which platform economics cannot answer.

Her decision follows that split. The year-one build budget is 1,200, and Policy, Party, and Claims events cost exactly 1,200 to build (300 + 500 + 400). They serve 31 consumers (10 + 12 + 9) at a combined three-year net cost of 1,050, about 33.9 per consumer. Claim documents is not funded from the platform budget. Its 810 becomes a line in the assistant’s own total cost of ownership, because a case that counts only model usage and licenses understates what the assistant takes to run. The sponsor can pilot on a bounded snapshot of one claim type without the product. If the pilot meets its exit criteria and the other two consumers confirm they need the same documents, the product can move to the shared budget. Funding the foundation a use case needs while testing the use case on a bounded sample is the pattern described in the data and AI strategy article.

Central or domain ownership, decided product by product

Zhamak Dehghani’s data mesh principles, published on Martin Fowler’s site, are the standard argument for domain ownership. They describe the centralized alternative as producing hyper-specialized teams and pipelines coupled to one another, and they move responsibility for quality upstream, as close to the source as possible. The article Who Is This Table For? covers the obligations a data product carries and the conditions under which domains succeed. The leadership question here is narrower: for each product, who should own it?

Ownership follows what each party can do better. A domain team notices first when a source change alters meaning, for example when the claims system adds a status code, and can repair the product before consumers see wrong numbers. A central team notices when a consumer complains. Against that, a domain team cannot define an entity that spans domains. Party records combine policyholders, claimants, and agents from three source systems, and if claims owned them, the other domains would depend on claims’ priorities for a definition they all use. The third factor is whether the domain can do the work, which depends on skills, headcount, and whether the product appears in the domain leader’s goals. She assigns Policy to policy administration and Claims events to claims. Party stays with the central team, because it has 12 consumers across domains and needs one definition. Claim documents would go to claims if it is funded, since claims knows which documents replace which.

Moving ownership relocates work and adds overhead. The central request group has six engineers who each give about 20 working days a month to requests, and an average request takes 20 engineer-days, so the group closes six requests a month. Five new requests arrive each month, which means the backlog of 90 shrinks by one a month and would take 90 months to clear. If claims and policy administration take over the 2 of those 5 monthly requests that concern their own source data, the central group faces 3 arrivals against 6 completions and clears 3 a month, or 30 months for the backlog. That is still slow. Closing requests that the new products make unnecessary shortens it further.

The domains pay for the transfer. Two requests a month at 20 engineer-days is 40 engineer-days. Domain teams are less practiced on the platform, so assume 25 percent overhead, which makes 50 engineer-days, or 2.5 engineers. The central group needs 3 × 20 = 60 engineer-days, or 3 engineers. Together that is 5.5 engineers, compared with the 5 engineers of work (5 × 20 = 100 engineer-days) the central group handled before the transfer. The enterprise pays about 10 percent more engineering time for a shorter queue and repairs closer to the source, and it gets that only if the 2.5 engineers exist. Claims has one data engineer today. The transfer therefore comes with headcount, a place for the product in the domain leader’s goals, and on-call coverage. Without them the tables are owned in name only.

Who may decide what also has to be written down. Decision rights state which questions the domain owner settles alone, such as what a field means or when to deprecate it, which need the central team, such as identifiers shared across domains or access to restricted classes, and where disagreements go. A small set of rules binds every product: the contract format, the identifier standard, and how freshness is published. In federated computational governance, domain and platform owners set those rules together and enforce them in the platform rather than in meetings.

AI data readiness depends on the use and the step

Readiness means the conditions for a defined action are met. It is not a statement about the organization’s general maturity. The assistant’s first step is a pilot with a few adjusters on one claim type. Rollout to every adjuster is a different step with different conditions.

Analytics data passes through a person. An analyst reads a figure beside its definition and can ask about an odd number. In a retrieval-augmented assistant, a retriever selects passages, a model composes an answer, and the adjuster reads text that sounds authoritative. A wrong or outdated passage is consumed without anyone examining its source. Retrieval must also respect each adjuster’s permissions at query time, because the index spans material that not every adjuster may see. The article Last Year’s Policy, Cited Correctly lays out these failure patterns, including an outdated policy version that stays searchable beside the current one.

Research on practitioners shows how common data problems are in AI work. Sambasivan and colleagues interviewed 53 practitioners working on high-stakes AI and found that 92 percent had experienced what they call data cascades: compounding downstream failures that begin with data issues and often stay invisible until late. They attribute the cascades to conventional practices that value model work above data work. DORA’s research on AI-assisted software development reaches a similar conclusion from the organizational side. The benefits of AI depend on a healthy data ecosystem, meaning internal data that is high quality, easy to access, and unified.

The team tests the corpus against the use. From the claim-note and policy-document stores they draw a random sample of 400 documents and check two properties the assistant depends on: whether a newer version supersedes the document while it is still searchable, and whether it carries a classification label. Of the 400, 62 are superseded and still searchable (15.5 percent), and 28 have no label (7.0 percent). With a sample of 400, the 95 percent interval around 15.5 percent runs from about 12 to 19 percent. If the corpus holds 120,000 documents, a hypothetical size, that means roughly 14,000 to 23,000 superseded documents.

Deleting those documents once would fix the sample but not the cause. Nothing at the source records which version is in force, so superseded documents reappear with every policy revision. The repair belongs with the document’s owner, who records effective dates and the version that replaces it, and the index build filters on that record. Lineage lets the team trace a cited passage back to its source document and forward from a changed document to every index built on it. Classification labels feed the access rules discussed below, and the 7 percent without a label need a default. Here the default is exclusion from the index until the document is labeled.

For the pilot, readiness becomes a condition the team can set before it starts. For example, no superseded passage appears in the pilot index, and a set of test queries written in advance retrieves nothing outside the asking adjuster’s permissions. Those thresholds are hypothetical. Rollout to all adjusters would need broader evidence, such as coverage of every claim type and a measured failure rate under real use. The pilot exit criteria record the thresholds, who decides, and what happens if the pilot misses them.

For uses classified as high-risk under the EU AI Act, readiness is also a legal question. Article 10 requires data governance practices for training, validation, and testing data, covering design choices, collection, preparation, examination for bias, and identification of gaps. It sets the standard that the data be relevant, sufficiently representative, and, to the best extent possible, free of errors and complete. Whether an insurer’s use falls in that category depends on its purpose, and counsel decides. The data platform leader’s task is to hold the evidence, including ownership, lineage, and quality measures, before anyone asks for it.

One set of products, two kinds of demand

Across the three funded products, 27 consumer relationships are analytics (9 + 11 + 7) and 4 are AI use cases (1 + 1 + 2). AI is 4 of 31, or 12.9 percent. It draws most of the executive attention, while analytics consumers carry most of the products’ current value. A product shaped only for the loudest consumer fails the rest.

The demands differ. Analytics runs on schedules, tolerates next-morning data, and ends with a person interpreting the result. The assistant is interactive, applies per-user permissions on every retrieval, and ties freshness to individual answers. AI use also reaches material that analytics rarely touched, such as documents and free-text notes. These differences become requirements in each product’s data contract: freshness for the fields the assistant uses, effective-date and supersession fields, and classification labels. Most of those requirements help analytics consumers too, since a finance analyst reconciling claim reserves should not read a superseded rule either.

Two controls keep one set of products from turning into a contest. A single intake applies the same questions to every request, whether or not it is labeled AI: which decision changes, what freshness and access it needs, who funds the incremental cost, and what it costs to serve. And the product owner, not the loudest requester, orders the roadmap within the decision rights above. Serving paths can differ while the product is shared. The assistant’s retrieval index and its query load run on separate capacity with their own cost report, so a spike in assistant use does not slow the month-end close and its cost is visible to the team that causes it.

Trading quality, access, and cost

Each choice above has a price on three axes, and each price can be computed.

Freshness

The assistant’s sponsor asks for hourly updates of claim status, while analytics consumers need daily ones. Assume, for the arithmetic, that each refresh costs a flat \$18 in compute, which approximates a job dominated by fixed start-up and scan costs. An incremental refresh could cost less, and a real figure needs measuring. Months have 30 days. The freshness requirement sets the cadence, and the cadence sets the bill.

Refresh cadence	Refreshes per month	Cost per month	Cost per year
Daily	30	\$540	\$6,480
Hourly	720	\$12,960	\$155,520
Every 15 minutes	2,880	\$51,840	\$622,080

Hourly costs \$12,960 − \$540 = \$12,420 more per month than daily. If the assistant helps with 4,000 claims a month, that is \$3.11 per claim (\$12,420 / 4,000 = \$3.105). With adjuster time valued at a hypothetical \$60 an hour, or \$1 a minute, the faster cadence pays for itself only if it saves more than about 3.1 minutes of adjuster time per claim. The pilot’s measurements can show whether it does, and the sponsor, who sees the charge in a showback report, is the person to weigh it. Alternatives exist. Only the status fields could refresh hourly, or the assistant could call the source system for current status at answer time and avoid storing a copy that goes stale.

Quality

Strictness costs delay. A quality threshold that blocks publication holds back every consumer when it trips, while a warning lets suspect data through with a flag. The choice follows from what the consumer does with the data. A month-end close dataset feeds numbers that finance reports outside the company, and a count that fails to reconcile to the ledger should block. A dashboard read by a person who sees a warning banner can tolerate a warning. For the assistant, superseded documents are consumed silently, which argues for a block when the index is built rather than a warning nobody reads. Strict gates belong where a wrong value does irreversible damage or goes unexamined.

Access

Access trades speed against exposure. Suppose the central team receives 24 access requests a month and each waits 9 days on average for an owner’s review. That is 24 × 9 = 216 requester-days of waiting. If classification and attribute-based rules decide the 18 low-risk requests automatically (75 percent) and only the other 6 go to owners, waiting falls to 6 × 9 = 54 requester-days, a reduction of 162. The precondition is coverage. The audit found 7 percent of documents unlabeled, and unlabeled data needs a default of deny. The risk is scale, because a wrong rule grants wrongly to everyone it matches, so rules are reviewed and tested like code. Role-based and attribute-based access control explains what each model makes easy and what it makes fragile.

Cost

Last is the CFO’s request. Platform spend rose from \$6.0 million to \$6.9 million, up 15 percent. Over the same year the number of teams actively consuming platform data rose from 30 to 46. Cost per active consuming team fell from \$200,000 to \$150,000, a drop of 25 percent. Unit economics reads cost against what it buys, and a rising total can accompany a falling unit cost. A 10 percent cut to \$6.21 million across 46 teams would be \$135,000 per team, another 10 percent below today’s \$150,000.

She also tells the CFO what the funded products do not do. Retiring 18 duplicate pipelines (6 + 8 + 4) avoids 280 a year, but the three products cost 230 a year to run (60 + 80 + 90). The net run saving is 50 a year, about 0.7 percent of \$6.9 million, in exchange for 1,200 of build cost. The plan lowers cost per consumer by consolidating, and it does not deliver a 10 percent cut. A cut of \$690,000 would have to come from serving less or paying less for what is served, and she brings the CFO options with their consequences instead of a promise.

FinOps Foundation guidance describes allocation through account structures, tags, and explicit rules for shared costs, and it presents showback and chargeback as differing in formality rather than in maturity. Showback fits the insurer first. Chargeback could follow once domain teams control the drivers of their own cost, and a cost allocation tag on each product’s resources makes either possible. The article on cloud cost operations and FinOps covers the operating practice.

The decision, and what would change it

She takes a one-page decision to the executive team. The platform budget funds Policy, Party, and Claims events, 1,200 in all. Policy and Claims events move to policy administration and claims with about 2.5 engineers of capacity between them, on-call coverage, and the product in their leaders’ goals; Party stays central. Claim documents waits for the assistant’s case, which carries its 810 and runs a pilot on one claim type against written exit criteria. Hourly refresh of claim status is approved only if the sponsor accepts \$12,420 a month in showback. Low-risk access requests move to rules once classification covers the corpus. The CFO receives the unit-cost trend and the 50-a-year net run saving, along with options for reaching a 10 percent reduction.

The decision names its own review at six months. Consumer counts are checked against query logs, because the ranking above rests on them: if verified consumers of Claims events fall well below nine, its 51 per consumer climbs and its place in the funded set is open again. The 18 duplicate pipelines must actually be off, or the avoided cost does not exist. And the domains must have the engineers on call, or ownership reverts to the central team. Each check turns an assumption in the decision into a measurement.

References

• Zhamak Dehghani, Data Mesh Principles and Logical Architecture, martinfowler.com.
• Sambasivan et al., “Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI, CHI 2021.
• DORA, Capabilities: Healthy data ecosystems.
• EU Artificial Intelligence Act, Article 10: Data and data governance.
• FinOps Foundation, Framework: Unit Economics.
• FinOps Foundation, Framework: Allocation.