Quality Attributes and Trade-offs

Imagine a requirements document for a new customer data platform that says the platform must be fast, reliable, and scalable. Everyone signs it. Six months later the teams are arguing. The web team meant that profile lookups return in under 200 milliseconds. The analytics team meant that yesterday’s purchases appear by 7 a.m. The site reliability engineers meant the service answers requests. Finance meant that the monthly revenue figure matches the ledger. Each group reads the same three words and finds its own expectation unmet. The document is invented, but the pattern is familiar to anyone who has written requirements.

Words like fast and reliable name quality attributes: properties that describe how well a system does its job rather than what job it does. They are the vocabulary for judging designs, setting targets, and settling disputes, but only if each word is pinned to something measurable. This article defines the attributes that matter most for data and platform systems, shows how each is measured and where it is commonly confused with a neighbor, and explains why improving one attribute can weaken another.

From adjectives to measurable attributes

A quality attribute becomes useful when four things are stated: what is measured, where and for whom it is measured, over what window, and what value counts as acceptable. “Fast” is an adjective. “95 percent of profile lookups complete within 200 milliseconds, measured at the load balancer over each five-minute window” is an attribute with a target. The difference between a function the system performs and the quality with which it performs it is the subject of functional and quality requirements.

Standard vocabularies exist. ISO/IEC 25010:2023 defines a product quality model built from nine characteristics, each divided into subcharacteristics, and intends it for specifying, measuring, and evaluating products. Operational practice has its own vocabulary: Google’s site reliability engineering book describes a service level indicator as a carefully defined quantitative measure of some aspect of service. This article uses a smaller working set chosen for data and platform systems rather than reproducing either list.

Correctness, consistency, and durability

These three are about the data itself, and they are the attributes most often left implicit.

Correctness means outputs are what they should be, given the inputs and the business rules. A revenue table that double-counts refunds is incorrect even if every query against it succeeds quickly.
Consistency means that separate copies or views of the same information agree. The term covers replicas of one database, a source system and the warehouse copied from it, and two dashboards that compute the same metric. Consistency and correctness are independent: two systems can agree perfectly on a wrong number. Distributed systems also offer weaker forms of consistency, in which copies may disagree for a while before converging, and those models deserve a treatment of their own.
Durability is how likely data is to be retained over a long period, which the SRE book singles out as especially important for storage systems. It is separate from availability. A storage service can be unreachable for an hour without losing anything, and a service can answer every request while silently having lost last week’s writes.

For analytical data, a fourth attribute sits alongside these: data freshness, or how old the newest data a consumer can see is. Freshness is not latency. A dashboard can load in 300 milliseconds and still show numbers from two days ago.

Correctness and consistency are measured indirectly, usually by comparison. Reconciliation compares totals or records between a source and its copy. Validation rules check values against expected ranges and relationships. Durability is estimated from how storage is replicated and verified, and tested by actually restoring data.

Availability

AWS’s Well-Architected Framework defines availability as the percentage of time a workload performs its agreed function successfully when required, calculated over a period such as a month or a year. The SRE book gives two ways to calculate it, and the choice changes what the number means.

Time-based availability is uptime divided by total time. A target of 99.99 percent over a year allows about 52.56 minutes of downtime, because a year has 525,600 minutes and 0.01 percent of that is 52.56.
Request-based availability is the share of well-formed requests that succeed. A system that receives 2.5 million requests in a day with a daily target of 99.99 percent can return up to 250 errors that day. The SRE book prefers this form for globally distributed services, which are rarely completely down but often partly failing.

Two consequences matter for design. First, availability multiplies through hard dependencies. AWS gives the example of a system designed for 99.99 percent that depends on two other independent systems at 99.99 percent each: in theory the whole reaches only about 99.97 percent, since 0.9999 × 0.9999 × 0.9999 ≈ 0.9997. Every synchronous dependency a design adds lowers the ceiling unless the design can tolerate that dependency failing. Second, each additional nine costs more than the last. AWS notes that higher availability requires more exhaustive failure testing and automated recovery for every kind of failure, including deployments and capacity changes. The SRE book adds that past some point users cannot tell the difference: someone on a phone that works 99 percent of the time will not notice whether a service is at 99.99 or 99.999 percent.

Reliability is often used as a synonym for availability, but many teams use it more broadly to include correct behavior over time. When someone sets a reliability target, ask which they mean.

Latency, throughput, and saturation

Latency is how long a request takes to get a response. Throughput is how much work a system completes per unit of time, such as requests per second or rows processed per hour. The two are related but distinct, and the relationship is not linear.

The SRE book makes two points about latency that are easy to lose in dashboards. Latency should be read as a distribution, not an average. In its example, a typical request is served in about 50 milliseconds while 5 percent of requests are 20 times slower, and an average-based chart shows no change even as that slow tail grows over the day. That is why targets are usually written in percentiles, such as the 95th or 99th. The book also recommends measuring the latency of failed requests separately, because a fast error can make an average look healthy.

Throughput and latency interact through load. The same chapter observes that more requests per second often lead to higher latency, and that many services hit a performance cliff past some load threshold. Saturation, one of the book’s four golden signals for monitoring, describes how full a service is on its most constrained resource. Many systems degrade well before 100 percent utilization, which is why capacity is planned against a utilization target rather than against the physical limit. Concurrency and request rate are the load measures most often used for this.

Data systems add a variant. A batch job has a duration rather than a per-request latency, and what consumers usually care about is end-to-end freshness, which includes waiting for the job to start, running it, and publishing the result. A query engine can have low latency for interactive queries while the pipeline behind it delivers stale data.

Scalability and elasticity

The next definitions reflect common engineering usage rather than a single standard. Scalability describes how the cost and effort of meeting the other targets change as load grows. A system that holds its latency target at ten times today’s traffic by adding proportionally more machines scales well. One that needs a redesign at twice the traffic does not. Scalability is a statement about growth, so a system can be fast today and still scale poorly.

It is worth naming which dimension is growing, because systems scale differently along each: request rate, data volume, number of tenants or teams, number of data products, or geographic regions. A warehouse may handle ten times the data easily yet struggle with ten times as many concurrent analysts. A platform may handle ten times the traffic yet require its central team to grow in proportion to the number of teams it supports, which is an organizational scaling limit.

Elasticity is narrower. It describes how quickly and automatically capacity follows demand up and down. A system can scale well only through manual capacity planning, in which case it is scalable but not elastic. Elasticity mostly affects cost, since unused capacity is paid for, and the ability to absorb sudden spikes.

Changeability, recoverability, and operability

Changeability, often called maintainability or modifiability, describes how the cost and risk of making a change grow over time. It can be observed through the lead time from an agreed change to its release, the number of components or teams a typical change touches, and how often changes cause incidents. It rarely appears in requirements, yet it decides whether the other attributes can be kept as needs shift.

Recoverability describes how quickly and completely a system returns to service after a failure. It is usually expressed through two objectives that AWS’s disaster recovery guidance defines and that the organization, not the technology, sets. The recovery time objective is the maximum acceptable delay between an interruption and restored service. The recovery point objective is the maximum acceptable time since the last recovery point, which determines how much data may be lost. The two are chosen independently and are covered in more detail under recovery time objective and recovery point objective. Recoverability is also different from availability: a system with rare but long outages and one with frequent short outages can have the same availability percentage and very different recovery characteristics.

Operability describes how easily people can run, observe, and diagnose a system, including deploying it, understanding its state, and finding the cause of a problem. Poor operability shows up as long incidents and heavy reliance on a few experts. Security also belongs in any evaluation, but it cuts across all of these attributes rather than sitting beside them.

Why improving one attribute can weaken another

Most design choices move several attributes at once, but none of these trade-offs is automatic. Each one holds under a condition, and the condition is the part worth arguing about, because a choice that is decisive in one setting is free in another.

Design choice	Improves	Weakens, and under what condition	When the cost matters
Serve reads from a cache or replica	Latency, availability of reads	Consistency and freshness, if the copy is updated asynchronously or may return a value known to be stale. A cache invalidated synchronously on write, or a replica read only after its write is acknowledged, does not pay this	When users act on the latest value, such as stock levels or account balances
Process records in larger batches	Throughput, cost per record	Latency for each record, and freshness, because a record waits for its batch to fill or for the schedule	When consumers need results within minutes rather than hours
Replicate writes synchronously to another region	Durability, and reduced potential data loss, which supports a tighter recovery point objective	Write latency, cost, and availability of writes if the other region is unreachable and the write requires it	When losing seconds of data is unacceptable and write volume is high
Add a required synchronous call to another service	Correctness or richness of the response	Availability and latency, because the caller now cannot succeed unless the dependency does. A call that can be skipped or served from a fallback does not bind the caller this way	When the called service has a lower availability or a slower tail than the caller’s target
Engineer the system to meet a higher availability target	Availability, once the redundancy, testing, and automated recovery are actually built	Cost and speed of change, through stricter testing, more automation, and more parts to operate	When users can actually perceive the difference
Build a general shared platform instead of a tuned per-team solution	Changeability across teams, consistency of operations	Peak performance for a single workload, where the shared runtime, resource quotas, or a common configuration prevent an optimization that workload would otherwise use	When a few workloads have extreme performance needs
Add validation and approval steps to data publishing	Correctness	Freshness and speed of change, to the extent the steps are serial and human	When errors reach high-stakes decisions such as financial reporting

The fourth row is worth separating into two claims that are often merged. Adding a required dependency lowers the ceiling on availability, and that is arithmetic: if the caller needs all of them and their failures are independent, the achievable availability is the product, which is the AWS example above. Adding a component raises operational burden, which is a different and softer claim: more parts can fail, be misconfigured, or need changing, so changeability and operability tend to suffer.

They come apart in the case that matters most. A redundant component added so that the system survives the failure of another is also a new thing to operate, but it does not lower the availability ceiling; AWS’s reliability guidance computes redundant arrangements with a different formula, under which availability goes up rather than down. The question to ask about any added part is therefore not “how many parts are there” but “must this one work for the request to succeed.”

Cost, complexity, and team capability belong in the judgment

Quality attributes are never free, and three constraints decide which levels are worth buying. The first can be estimated with arithmetic. The other two usually cannot, which is why they get dropped from the conversation.

Cost, and how to put a number on it

The SRE book states the shape of the problem plainly: cost does not rise linearly as reliability increases, and an incremental improvement may cost a hundred times what the previous increment cost. It separates that cost into two parts. One is the redundant compute and storage that a higher target requires, along with the equipment that makes offline maintenance or durability guarantees possible. The other is opportunity cost, which it defines as what an organization gives up when it puts engineers on work that reduces risk instead of on features users can see.

Against that cost, the book puts a way to value the improvement. Its worked example takes a service earning \$1 million and asks what moving from 99.9 percent to 99.99 percent availability is worth. The availability gain is 0.09 percent, so if revenue tracks availability the improvement is worth \$1,000,000 × 0.0009, which is \$900. In this simplified calculation, spending more than \$900 to get that nine costs more than the revenue it returns. The arithmetic is simple, and it is the part teams skip.

Three cautions before applying it. First, both sides have to be measured over the same period. The \$1 million is revenue over some window, and \$900 is the additional revenue over that same window, so the cost it should be compared against is the cost over that window too: build effort amortized plus the recurring infrastructure and operational cost, not a one-time project estimate set against a single year’s benefit.

Second, the revenue-tracks-availability assumption holds for some services and not others. An outage during a product launch or a month-end close does not cost the same as one at 3 a.m. on a Sunday, and for internal platforms the loss is measured in other teams’ blocked work rather than in revenue. Third, the calculation gives the value of the improvement, not its cost, so it only decides anything once someone has estimated what the redundancy and the engineering would actually take. The useful result of doing all three is that a target becomes defensible in a budget conversation rather than a matter of taste.

This is also why the SRE book treats an availability target as both a minimum and a maximum, and aims for a service that is reliable enough rather than as reliable as possible. Running consistently above a target is not by itself a problem: it can simply mean a quiet period, or a design that turned out to be simple and robust. What deserves a look is whether anything is still being spent to hold that level — redundancy, review steps, engineering attention — and what those resources would be doing otherwise. The point of the target is to stop buying reliability nobody asked for, not to treat good fortune as waste.

Complexity and team capability

Complexity is the second cost, and it is paid later. Designs that reach high targets on paper usually do so by adding parts: replicas, failover paths, queues, coordination. Each of those is something that can fail on its own, something that has to be understood during an incident at four in the morning, and something that must be kept working through every future change. A failover mechanism that has never been exercised is a common example, because it converts a single failure into two: the original one and the failover that did not work.

Team capability is the third, and it is the one most often treated as though it were fixed by the design. A target is only met by the system as operated, not as drawn. An active-active multi-region database can satisfy a recovery objective in a design document while exceeding what a team of four can safely run, upgrade, and debug. When that is the case, the honest options are to lower the target, to buy the capability as a managed service and accept its limits, to fund the people, or to choose a simpler design that the team can actually operate. What does not work is to keep the target, keep the design, and hope.

These two are hard to quantify, so a practical substitute is to ask what the design adds to the list of things the team must operate, and who has run something like it before. Where the answer to the second question is nobody, recoverability is worth checking first, because a recovery procedure is the one that gets practiced least and is needed under the worst conditions. Checking it means the same thing here as anywhere else: run the recovery, measure how long it took and how much data was lost, and compare those two numbers against the stated objectives rather than assuming the design delivers them.

A usable evaluation therefore states targets for the attributes that matter to this system and its users, names the attributes that were deliberately given lower priority, and records what the chosen levels cost in money, in parts to operate, and in skills the team needs. Operational targets of this kind are usually expressed as service level indicators and service level objectives.

Questions to explore further

For one important data product, which attribute do its consumers care about most: correctness, freshness, or availability? Is that written down anywhere?
How many required synchronous dependencies does your most critical service have? If their failures are independent and there is no fallback path, what availability does their product allow, and which of them share a cause of failure such as one region or one identity provider?
Which of your latency targets are reported as averages, and what would the 99th percentile show?
For availability, which regions, customer segments, or times of day are failing inside an overall success rate that looks acceptable?
When did you last run a recovery for real, and how did the measured recovery time and data loss compare with your stated objectives?
For your highest availability target, what would one more nine cost, and what is that nine worth to the people who depend on the service?
Which part of your current design has nobody on the team operated before, and what happens to recovery if it fails first?

References

All sources were checked on September 15, 2026. Calculations in the text were recomputed from the figures in the sources.

Discover more from Insightful Data Lab

Subscribe to get the latest posts sent to your email.

Quality Attributes: A Shared Language for Judging Systems