Skip to main content

Failed or stuck upgrades

An upgrade that stops partway through is almost always blocked on one node, not broken across the cluster. Syself Autopilot rolls an upgrade by replacing nodes one step at a time, so a single node that cannot drain or cannot join holds the whole rollout. Find the blocked node, clear it, and the rollout continues.

How the rolling replace works 

An upgrade replaces nodes with fresh sealed nodes; nothing is patched in place. The control plane rolls one node at a time, so etcd never loses more than one member and keeps quorum. Worker pools roll in parallel with each other, but within a pool the platform respects your PodDisruptionBudgets. Knowing this tells you where to look: a stalled control-plane upgrade is one control-plane node; a stalled worker upgrade is one pool, usually one drain.

The common blocks 

  • A drain blocked by a PodDisruptionBudget. If a workload's PDB will not allow the eviction, the drain waits, up to nodeDrainTimeoutSeconds: 180 on the pool, then proceeds and the pods go with the node. A PDB that can never be satisfied (a single replica with maxUnavailable: 0) makes every drain wait the full 180 seconds. Fix the workload's replicas and PDB; see .
  • Capacity blocking a new node. The replacement node cannot be created because Hetzner has no capacity in the region, so the rollout waits for a machine that never comes. This is ; move the pool to bare metal or wait for capacity.
  • A new node failing to join. The replacement provisions but does not become Ready. Read its Machine events and, if you can reach it, .

The one-minor-at-a-time rule 

You upgrade one Kubernetes minor at a time, on the latest patch first. A skipped minor is rejected the moment you apply it rather than stalling part way through: the ClusterClass pins one Kubernetes version, and the release that classRef.name points at decides which one the cluster runs. To move two minors, do it as two steps: upgrade to the intermediate minor, let it finish and settle, then upgrade again. See .

There is no rollback 

An upgrade cannot be undone. Moving a cluster back to an older Cluster Stack release is not supported, and Kubernetes versions cannot be decreased. The recovery from a failed upgrade is to fix the block and let the rollout finish forward, or in the worst case rebuild from your Git declaration. See .

When to escalate 

If the control plane itself is unhealthy during the upgrade, an etcd member is down or the API is flapping, stop and treat it as , and open a support case rather than forcing more changes onto a control plane that is already short of quorum.