API Contracts, Compatibility, and Retirement

In this hypothetical example, a team stops populating a field nobody was supposed to depend on. The schema still declares it, the responses still contain it, and it is now always null. Three days later two consumers are broken: one was displaying it, and one was using it to decide whether to retry. Neither had asked permission, and the change went through review as a cleanup.

Publishing an API creates support obligations that extend beyond its schema. Changing it requires a clear contract and evidence about the consumers affected. This article works through compatibility, error behavior, retry safety, and the decisions needed to retire a version.

The contract is larger than the schema

A schema describes the shape that tooling can check. An interface contract also needs to state the behavior consumers may rely on. The following areas deserve explicit decisions.

• Shape and types. The fields, their types, what is optional. The checkable part.
• Meaning. What status = active includes, whether amount is gross or net, which timezone a date is in, whether a list is complete or a page. A schema validator cannot decide these, and they are the source of the worst incidents — but once decided they can often be pinned with tests, which is a different thing from being expressible in a schema.
• Error behavior. Which failures are permanent and which are worth retrying, and how the client tells the difference. Consumers program against this whether or not you documented it.
• Retry safety. Whether repeating a call is harmless. Addressed below, because the standards are specific and widely ignored.
• Ordering and timing. Whether responses reflect writes immediately, whether events arrive in order, how stale a read may be. A consumer that tested against a fast, empty environment has assumed something here.
• Limits. Page sizes, maximum payloads, rate limits, timeouts. Tightening any of these is a change to what callers may do.

Write down field meanings before consumers have to infer them. An undocumented interpretation can become a real dependency, even though it does not automatically become a supported promise. Investigate those dependencies, clarify the contract, and plan the transition when changing behavior would affect consumers.

For data delivered rather than called, the equivalent agreement is a data contract, and the reasoning is the same with different mechanics.

Check the contract and the consumers

A breaking change violates compatibility with the supported interface. That interface includes behavior and meaning as well as structure. Also check actual consumer impact: code may fail or return wrong results because it relied on undocumented or unsupported behavior. Those failures matter operationally, but they do not automatically establish a contract violation or a requirement to preserve that behavior forever.

Change	Usually safe	Breaks someone
Adding an optional field to a response	Yes — provided consumers ignore unknown fields, which is a property of their code rather than your intent	A consumer with strict validation, or one that persists the whole payload into a fixed schema
Adding a required request field	No	Callers that omit the newly required field
Removing a field, or always returning null in it	No	Anyone reading it. Deprecated in the docs is not the same as unused
Adding a new enum value	Sometimes	Consumers that switch exhaustively on the old set — common, and usually undiscovered until it happens
Tightening validation	No	Callers whose previously accepted input is now rejected, including ones relying on a bug
Changing a field’s meaning while keeping its type	No	Consumers that depend on the old meaning; validation may catch the change, or results may change silently

A meaning change can be hard to detect because the results still look plausible. Redefining active users to exclude trials changes counts for consumers that included those accounts. Without a check for that rule, the difference may first surface in a meeting.

But structural changes can be just as quiet, so do not read the table as structure fails loudly, meaning fails silently. Remove an optional field and a consumer that substitutes a default keeps running with a different answer. Add an enum value and a consumer whose switch has a catch-all branch classifies it as something else without raising anything. In both cases parsing succeeded and the result changed. Visibility depends on what the consumer does with the thing you changed — reads it, branches on it, validates it, persists it — which is why the judgment in the next section asks that question rather than sorting changes into structural and semantic.

Provider-side assumptions can hide those dependencies. A bug fix can affect consumers that adapted to the bug, while documented deprecation does not establish disuse. A field may be used without generating a ticket, and reasoning about “the client” as one system can miss eleven distinct consumers, including a partner you cannot call. The direction of the compatibility check — old data against new readers, or new data against old readers — is also a choice with consequences, which is the same reasoning as schema compatibility applied to an interface.

Versions are a cost, so postpone them honestly

A new version is useful when a change cannot preserve the supported contract and consumers cannot all migrate together. Each supported version adds behavior to maintain, test, and diagnose. Judge the version count against the promised support period, deployment model, and migration progress; the number alone does not show whether retirement is working.

Before adding one, four techniques cover most cases.

• Add rather than change. A new field alongside the old one, with both populated during the transition. Ugly and cheap.
• Make the new behavior opt-in, through a parameter or a header, so existing callers are unaffected and new ones get the better interface.
• Expand, migrate, contract. Accept both shapes, move consumers, then remove the old one — the same three-step move used for database schema changes, and it works here for the same reason.
• Version the resource, not the API. Where only one endpoint has to change incompatibly, a new endpoint is a smaller commitment than a new version of everything.

Where a version is genuinely needed, two decisions matter more than the syntax of it. Decide what a version number promises: if it follows semantic versioning, then, for a stable public API, an incompatible change requires a major bump. Consumers must assess the change before upgrading; relying on unsupported behavior does not by itself determine that bump. And decide the support commitment at the same time as the version — how long the previous one lives — because deciding that later means deciding it under pressure from whoever has not migrated.

Errors are an interface, not a message

Consumers write code against failures: retry a request, alert an operator, or explain a rejection to a user. RFC 9457 supplies a common error-body format for details that an HTTP status code alone cannot express.

The problem details format answers that with a small JSON object, and three of its rules are worth adopting whatever format you use.

• Use a stable identifier for each problem type. RFC 9457 requires consumers to use the type URI, resolved when necessary, as the primary problem identifier. Branch on that identifier and defined structured fields instead of matching an error message.
• Keep human text out of machine decisions. The advisory title should stay stable for a problem type except for localization. The per-occurrence detail helps a person understand and correct the problem; RFC 9457 recommends against parsing it for information. Put values needed by client logic in structured extensions.
• Allow unknown extensions. Problem types can define additional members, and RFC 9457 requires clients to ignore extensions they do not recognize. Adding such information preserves compatibility with clients that follow that rule.

The consumer contract must cover unknown error types and retry guidance. Introducing a new error type is generally compatible only if clients treat unrecognized types as some failure of this class rather than crashing — worth stating in your documentation as an expectation of consumers. And the retry guidance has to be explicit somewhere: which failures are transient, and how long to wait. HTTP has a field for the second half of that, and how to use it is covered under Retry-After and backoff.

Retry safety is specified, and usually assumed instead

A connection can fail after the server acts but before the client receives the response. RFC 9110 defines idempotency by the intended effect of repeating an identical request: it is the same as for one request. PUT, DELETE, and safe methods have this property under HTTP semantics.

Three details in that section are more useful than the definition.

• The guarantee concerns the intended effect. Repeating a PUT can apply the same representation more than once while leaving the same intended state. Separate logs or revision records are allowed. This does not promise one execution, nor does it prevent an old retry from overwriting an intervening write; use concurrency controls when that matters.
• Idempotency permits recovery from a lost response. A client can retry an idempotent request after a communication failure. The response may differ even when the intended effect is unchanged.
• Automatic retries have limits. RFC 9110 advises clients against automatically retrying a non-idempotent method unless they know the operation is effectively idempotent or can detect that the original was never applied. It prohibits proxies from automatically retrying non-idempotent requests and advises clients against automatically repeating a failed automatic retry.

For operations without natural retry safety, an idempotency key can identify retries as the same business action. The provider must honor that identity across failures. State the key scope, retention period, request-mismatch handling, and how clients recover a result or an unresolved status. A key outside its supported lifetime no longer provides the promised duplicate protection.

Providers who skip this are not avoiding the problem, only moving it: clients will retry anyway when the alternative is losing a customer’s order, and duplicates will appear in your data. Making the retry safe is cheaper than reconciling the result, which is the practical content of idempotency as a design property.

Retirement needs a measurement, not an announcement

When retiring a version, establish who may still depend on it. Deprecation discourages new dependencies and encourages migration; it does not necessarily set a removal date. For a planned shutdown, combine usage evidence with the support policy, notices, and migration commitments.

Gather evidence early enough to contact affected consumers and resolve exceptions before shutdown.

1. Instrument by consumer and version. Record client identities, then map them to applications and owners. A shared account may represent several teams. Include scheduled and rarely run jobs: zero observed calls during a short window does not prove there are no dependencies.
2. Announce the lifecycle decision and migration path. Explain what users should move to, or how to stop depending on a capability with no replacement. When removal is planned, state its date and the support window; if no date is decided, say so explicitly.
3. Signal lifecycle changes in responses. RFC 9745 defines Deprecation for the deprecation time. RFC 8594 defines Sunset for the time a resource is expected to become unresponsive. These describe different events; if both are sent, Sunset must not precede deprecation. Link to the migration and support policy.
4. Contact the remaining callers individually once the long tail is small. This is the step that actually finishes migrations, and it is why step one exists.
5. Retire in a way you can reverse for a short window — a brownout before the shutdown, and the ability to re-enable briefly if something unexpected breaks.

RFC 8594 treats Sunset as a hint, not a guarantee of availability before or after the timestamp. Keep it consistent with the support policy. A past date signals that shutdown may occur now; if the provider formally extends support, update both the policy and the advertised date rather than silently moving the deadline.

Events change three of the answers

Everything above applies to a published event stream, with three differences that catch teams moving from request-response.

• Consumer evidence comes from the delivery platform too. Depending on the system, use subscription registrations, consumer-group state, access or consumption logs, and owner records. Kafka also supports manual partition assignment without group coordination, so group membership alone is not a complete inventory. State what the evidence can and cannot reveal.
• Retained events outlive producer releases. A replay can encounter any payload versions still present in the topic or archive. Time and size limits, compaction, and archival policy determine what remains. Preserve the schemas and readers needed for that history, with an event envelope or equivalent record metadata identifying the applicable schema.
• Delivery and ordering are contract terms. Whether duplicates are possible, whether order holds, and how late an event may arrive are promises consumers build on — and they are easy to weaken accidentally when the producer is re-partitioned or re-implemented.

Other protocols pose the same questions through different mechanisms, such as field nullability or graph-query arguments. Check the promised contract and the behavior of affected consumers separately, then use both to decide whether and how to release the change.

What to put in place before the first consumer

Most of this is cheap in advance and expensive retrofitted. Establish consumer evidence and field meaning alongside error handling, retry safety, and the support policy.

1. Per-consumer usage evidence. Combine metrics with registrations, lineage, and owner confirmation. Record observation gaps and assess them alongside the support policy, notification process, exceptions, and rollback plan.
2. A written statement of meaning for every field whose interpretation could be argued about — which is most of them.
3. A stable error identifier scheme and a documented statement of which errors are retryable.
4. A retry-safety position per operation, with an idempotency key mechanism for the ones that create or charge something.
5. A support policy: how long a version lives after the next one ships, and how deprecation is communicated.

Agreeing on what active means is a business decision. Tests can defend the agreed rules for the cases they cover: dbt unit tests, for example, exercise model logic on small static inputs. Include trial accounts, refunds, cancellations, and relevant time boundaries in fixtures and assertions. A fixture with no trial accounts will not detect a change that excludes them. Consumer contract tests and reconciliation of affected totals provide additional evidence; a passing fixture does not prove every interpretation is preserved. Keeping the published surface small also reduces the contracts to maintain. An endpoint believed to be unused still needs a support-policy and dependency check before removal, but it is not inherently unchangeable. At a bounded context boundary, the interface also translates between models, giving another reason to keep its responsibilities focused.

One consequence of all this is worth stating for readers designing their first public interface: reads that return lists need a paging contract from the start — see keyset pagination — and any read served from a replica needs its staleness stated, since eventual consistency that consumers discover experimentally can become an undocumented dependency that needs investigation.

References

• RFC 9110, HTTP Semantics, Idempotent Methods
• RFC 9457, Problem Details for HTTP APIs
• RFC 8594, The Sunset HTTP Header Field
• dbt Documentation, Unit tests
• RFC 9745, The Deprecation HTTP Response Header Field
• Semantic Versioning 2.0.0
• Apache Kafka 4.1, KafkaConsumer: Manual Partition Assignment
• Apache Kafka 4.1, Design: Log Compaction