Data and AI Platform Architecture Explained

One platform, several kinds of work

A shop wants a daily sales report, a product recommendation when a customer opens the app, and an assistant that answers questions about its return policy. All three use data, but they do not do the same work. The report aggregates orders. The recommendation runs a previously prepared model. The assistant retrieves relevant documents and asks a language model to compose an answer. Putting them on the same cloud account does not, by itself, make them a coherent platform.

Platform architecture describes how responsibilities, interfaces, and operating rules fit together. An interface is the agreed way one component requests work or exchanges data with another. We will use four responsibilities—storage, compute, serving, and control—to trace the shop’s workloads. These are a way to reason about a design, not four products every team must buy. No Python, SQL, or model mathematics is required here.

Four responsibilities, not four separate servers

Responsibility	Question it answers	In the shop
Storage	What persists, and which version is authoritative?	Order history, prepared sales tables, policy documents, model files
Compute	What work transforms an input into an output?	Aggregate orders, prepare documents, train a model, calculate a prediction
Serving	How does a consumer obtain a usable result?	A dashboard query, a recommendation API, an assistant response
Control	Who may do the work, when, and under which configuration?	Schedules, identities, permissions, deployments, resource limits, recovery rules

A database may store tables and compute queries. An API service may both calculate a prediction and return it. A managed service may hide many machines behind one interface. An endpoint identifies where a request is sent within an interface. Name the responsibility first; then decide whether to combine it with another responsibility or separate it for performance, permissions, or recovery.

Cloud changes resource delivery, not the need for ownership

Cloud computing provides network access to configurable computing resources that can be provisioned and released on demand. Those resources still run on physical machines. A cloud can be public or private; access need not travel over the public internet. Merely putting a file online does not establish all the characteristics of a cloud service. NIST’s cloud definition describes this resource model.

For inference, the choice is where a prepared model runs. A cloud service can pool capacity and centralize updates; an on-device model can avoid sending raw inputs over a network. Neither placement is always faster, cheaper, or easier to maintain. Compare end-to-end delay, connectivity, device capacity, permitted data movement, and how models are updated. A hybrid design can prepare inputs locally and send selected features, provided the receiving model expects that representation.

Managed ML platforms can supply training jobs, artifact records, prediction services, and monitoring facilities. The team still chooses compatible inputs and models, configures access and capacity, evaluates outputs, and responds to failures. An external model API supplies a callable model capability; a custom-model platform also provides facilities for operating models the team selects or trains. Actual service scope must be checked rather than inferred from a brand or an “end-to-end” label. Integration with a cloud ecosystem can reduce setup work while tying artifacts, identities, and workflows to its interfaces. Consider export formats, portability, and migration effort when choosing the boundary.

The daily report: prepare data before the reader arrives

Order system → permitted collection → retained source snapshot
                                         ↓
                               validate and aggregate
                                         ↓
                               publish sales table v17
                                         ↓
                               dashboard query → reader

Control: schedule work · assign identity · set limits · approve publication
Observe: record run status, source coverage, output version, and failures

A snapshot is a copy representing a source at a defined point or interval. The collection step must record what that copy covers; a filename containing yesterday’s date is not evidence that all of yesterday’s orders arrived. Compute reads the selected input, applies the agreed sales definition, and produces a candidate table. Checks compare the candidate with the expected input and business rules before consumers receive it.

Publication makes an accepted version available through the serving interface. For example, consumers can keep reading v17 while a job builds and checks v18, then switch to v18 when it is accepted. The storage system must actually support the chosen publication mechanism; drawing an arrow does not make the change atomic. Atomic means readers do not see a half-published version. If publication fails, the report can show the last accepted version with its age, or become unavailable, according to the agreed policy.

The recommendation: separate preparation from a live request

Training fits a model to examples; inference uses a prepared model to produce an output for a new input. A model artifact is the saved representation needed to run that model, together with the configuration or dependencies required to interpret it. A model registry tracks versions and associated records; it is not the service that computes every prediction. A team can also use an externally supplied model instead of training one.

Preparation: permitted history → feature preparation → training → evaluation
                                                                   ↓
                                                     accepted model version
                                                                   ↓ deployment
Live request: app → identity and input checks → feature lookup → inference → response

A feature is an input value used by the model, such as a customer’s recent purchase count. A value prepared overnight can be cheap to retrieve, but it may not include a purchase made one minute ago. The architecture must state which features may be stale and how online inputs match the definitions used in training. Training can take hours without making every app request wait hours, because it runs on a separate preparation path.

Google’s MLOps architecture distinguishes data preparation, training, evaluation, deployment, and prediction services. Here the useful point is the separation of responsibilities and version handoffs, rather than a requirement to adopt a particular implementation. Google Cloud’s MLOps architecture develops those stages further.

The assistant: documents have a preparation path too

For a document-grounded assistant, preparation collects permitted policy documents, divides them into searchable pieces, and builds an index that helps locate relevant material. A search index is a derived lookup structure, not necessarily the authoritative document store. One possible retrieval method uses embeddings: numeric representations that help compare the similarity of a question and a passage. That method is optional, and similarity is not proof that a passage answers the question.

At request time, the service identifies the user, retrieves material that user may access, passes the question and selected material to the model, and returns a response with references where appropriate. This is the basic flow of retrieval-augmented generation, or RAG. Retrieved text supplies context; it does not normally retrain the model on each question. Retrieval, access control, and answer quality each need checks. A fluent answer can still misinterpret a valid document.

If a return policy changes, updating the original document is only the first step. The team must refresh the derived index and relevant caches, and ensure obsolete or newly restricted passages cannot continue leaking through the serving path. A cache is a stored result reused to avoid repeating work. Its key, expiry, and invalidation rules determine when reuse is acceptable, especially when users have different permissions.

Control configures the work; the running path enforces it

The data plane is the path that moves or processes application data and serves requests. The control plane manages desired configuration and coordinates resources: what version should run, how much capacity it may use, and which identity may access which resource. These terms describe responsibilities, not a guarantee of separate machines. A permission policy may be managed centrally while a query service enforces it on each request.

Kubernetes provides a concrete example of a control plane managing worker nodes that run workloads. A data and AI platform may use that mechanism, another scheduler, or a managed service. Do not assume that control-plane failure always stops every existing request—or that it leaves them unaffected. Existing work may continue, while new scheduling, credential checks, or configuration changes fail; the outcome depends on runtime dependencies. Kubernetes component documentation illustrates the separation.

A fast response and fresh data are different promises

Suppose a recommendation request spends 20 ms on identity and input checks, 30 ms retrieving features, 40 ms waiting for compute, 60 ms on inference, and 10 ms preparing the reply. If these steps are sequential, the measured path totals 160 ms. These are invented values for this example, excluding network time outside the measured boundary. The feature values could still be twelve hours old. Response latency measures elapsed request time; freshness measures how current the underlying data is.

A faster model would not remove the waiting time or refresh the features. Measure the path before changing it. An average also hides slow requests; a percentile such as the 95th percentile describes the point at or below which roughly 95% of observed request durations fall. Set the measurement boundary and workload before comparing latency targets. Adding component percentiles does not generally produce the end-to-end percentile.

Shared infrastructure also shares failure and contention

A nightly training job can compete with live inference for memory or processors even when the two have different names in the diagram. A shared storage service can connect their failures too. Isolation means limiting how one workload affects another, using resource quotas, separate queues or compute pools, and appropriate storage boundaries. Each separation adds operating cost, so choose it according to the service requirement rather than separating every box automatically.

Failure	Decision to make before it happens
The new sales table fails validation	Keep the previous accepted version with a stale label, or block the report; define the owner and maximum tolerated age.
The feature service times out	Use a previously approved fallback, such as a generic ranking, or return an error; do not silently substitute arbitrary feature values.
Document access cannot be verified	Do not return potentially restricted passages. Decide whether a public-only path can safely continue.
A new model is worse than the accepted version	Keep or restore a known version together with its compatible feature definitions and runtime.

Read the architecture as a set of contracts

For every arrow, identify the input, output, owner, and failure response. Record the version or time range of the data, the permitted caller, and whether the caller waits for the result. For every stored copy, record why it exists and how it is updated or retired. These details explain more than a page of product logos. A small platform may start with a database, scheduled preparation, and one application service, provided those responsibilities are explicit.

Check what you can explain

Try answering before opening the solutions. If you can name a component but cannot explain its input, output, and failure behavior, mark that relationship for review. These questions check the concepts in this article; they do not certify implementation skill.

1. A dashboard answers in 50 ms but shows yesterday’s policy version. Which promise failed, and where would you investigate?

Solution

If the agreed policy requires the current version, freshness failed even though the measured response was fast. Trace source publication, collection, transformation or indexing, publication to serving, and cache invalidation. Establish which version each step used before blaming compute speed.

2. The team puts a model registry directly between every app request and the prediction result. What question should you ask?

Solution

Ask whether the arrow means retrieving deployment metadata or actually computing a prediction. A registry tracks artifacts and versions; inference needs a runtime that executes the selected model. The runtime may load a model at startup and serve many requests without fetching it again each time. Show that handoff explicitly.

3. A restricted policy is deleted from the source, but the assistant still quotes it. Name the copies and controls you would inspect.

Solution

Inspect retained source copies, indexed passages, retrieval caches, and response caches. Check whether deletion reached each derived store and whether current permissions are enforced when content is retrieved or reused. Also inspect what was sent to the model for the request. Updating the source alone does not demonstrate that the serving path stopped using old content.