Version Skew
Version skew is the version difference between components upgraded separately. A supported skew policy defines permitted combinations; Kubernetes deployment tools and providers can impose tighter restrictions than the project policy.
The API server, control-plane controllers, node components and kubectl can move separately. Check their installed and target versions, including the mixed-version interval during an upgrade. The table summarizes the current documented component rules; verify the policy for older versions.
The documented distances
Versions are “expressed as x.y.zโฆ following Semantic Versioning terminology,” and each component’s allowance is stated against the API server.
| Component | Allowed relative to kube-apiserver |
|---|---|
| kube-apiserver peers in HA | Newest and oldest “must be within one minor version“ |
| kubelet | “Must not be newer“; “may be up to three minor versions older“ |
| kube-proxy | Same as kubelet, plus within three minors of “the kubelet instance it runs alongside” |
| controller-manager, scheduler, cloud-controller-manager | “Must not be newer”; “may be up to one minor version older (to allow live upgrades)” |
| kubectl | “Within one minor version (older or newer)” |
Two patterns are visible in the table. No cluster component may be newer than the API server it talks to, which makes the API server the pacer for everything else โ with kubectl the deliberate exception, allowed one minor version in either direction because the person running it is often working against several clusters at once. And the tolerances are deliberately uneven: nodes get three minor versions of slack because there are many of them and upgrading each one costs something, while the control plane’s own components get one, because they are expected to move together.
The distances dictate an order
The policy draws the conclusion itself: the supported skew “has implications on the order in which components must be upgraded.” Working through the constraints gives one sequence.
- kube-apiserver โ first, because no other component may be newer than it (kubectl aside).
- kube-controller-manager, kube-scheduler, cloud-controller-manager โ “in any order, or even simultaneously,” once the API servers they talk to have moved.
- kubelet โ optional. It “can be left at” any of the three preceding minors.
- kube-proxy โ optional, on the same terms.
And one hard rule removes the shortcut people reach for when they are far behind: project policies “require kube-apiserver to not skip minor versions when upgrading, even in single-instance clusters.” Three minors behind is three upgrades, run in sequence, each with its own validation.
Three moments where skew actually bites
Nobody has an incident because of a table. The trouble comes from three specific situations.
Deferral can block the next minor upgrade. A kubelet persistently three minors behind the API server must catch up before that server advances another minor. Leaving operational margin below the maximum can make the next transition easier, but the chosen target depends on the fleet and upgrade procedure.
This constraint does not itself block patches within the current control-plane minor. For a security fix, check which supported branches provide it and any advisory-specific prerequisites. If the required fix needs a newer minor, node catch-up may become part of the critical path.
The supported procedure requires draining Pods before a minor kubelet upgrade; in-place minor kubelet upgrades are unsupported. Availability also depends on spare capacity, replicas, placement and disruption budgets, together with graceful shutdown. Do not generalize the minor-upgrade requirement to every kind of node maintenance.
The middle of an upgrade is the most constrained state. A note repeated through the policy: if skew exists between API server instances in an HA cluster, “this narrows the allowed kubelet versions,” and the same narrowing applies to kube-proxy, to the control plane components behind a load balancer, and to kubectl. So the window while your control plane is half-upgraded is when your nodes’ and your operators’ tooling have the least room โ which is an argument for making that window short rather than for pausing inside it.
One prerequisite is easy to forget because it is not a Kubernetes component at all. The requirement is that registered admission webhooks “are able to handle the data the new kube-apiserver instance will send them,” including new resource versions and new fields, with matchPolicy: Equivalent available as a cushion. Whatever you installed to enforce policy is now something the upgrade waits on.
Prepare by moving to the latest patch of the current minor, then the latest patch of the target minor, following the deployment toolโs supported procedure. The typical monthly patch cadence is a planning input, not evidence that testing will be short.
Why the allowance is finite
Supported component combinations and API removal schedules are separate contracts. The deprecation documentโs rationale mentions a two-release skew, while the current component policy permits up to three older minors for supported kubelet versions. Use the component-specific policy for upgrade combinations and the deprecation policy for API availability.
The patch policy describes roughly twelve months of standard support and two months of maintenance. Maintenance releases may address assigned CVEs, dependency issues and critical core-component issues; coverage is narrower, not a promise that every bug fix stops or every eligible fix ships. Check branch dates and advisories when planning the support lifecycle.
Three neighboring ideas, kept separate
| Term | Answers |
|---|---|
| Semantic versioning | How a version number is constructed and what a change in each position promises |
| Version skew | How far apart two running components may be right now, and therefore in what order they move |
| Deprecation policy | How long something keeps working after it has been announced for removal |
| Support lifecycle | How long a release receives fixes at all, and of what kind |
Skew is the one that constrains your calendar rather than your code. A deprecation tells you to change something eventually; a skew limit tells you what you can and cannot do this week.
Finally, the general form beyond Kubernetes. Any system whose parts are deployed independently has an effective version skew policy; the open question is whether it is written down. A client library and a service, a database and its schema migrations, a message producer and its consumers, an agent fleet and its control plane: each pair has some range of combinations that actually work, and when nobody has stated it, an unrecorded combination may first be discovered during an incident. Kubernetes is unusual mainly in publishing the numbers. For how the published numbers interact with policy, permissions and isolation on a shared cluster, see There Is No Deny Rule.
References
Discover more from Insightful Data Lab
Subscribe to get the latest posts sent to your email.
