You change one field on the `Cluster` object, and Syself Autopilot moves the whole cluster to a new tested version. It replaces nodes one at a time, and the control plane stays available the whole way through.

The API server, kubelet, etcd, the operating system, and the network layer each have their own version. They have to move in a strict order, or the cluster breaks. Syself knows that order and tests it on real clusters, along with the migrations that run during the move.

Nodes are never patched, so the new version arrives only as a new node. Every move uses the same cordon, drain, replace steps the platform runs to heal a broken node. An upgrade runs those same steps across every node instead of one broken node. To see why a running node cannot be patched at all, read [the node lifecycle](/docs/hetzner/apalla/concepts/operations/node-lifecycle).

## Updates and upgrades use the same steps

Both an update and an upgrade run through the same node-replacement steps. The only difference is which version you end up on.

An update stays inside one Kubernetes minor. You get patches, CVE fixes, and OS and cluster component revisions. The Kubernetes API and your cluster's behavior do not change. An upgrade moves to a new minor, where new features arrive and behavior can change.

|                        | Update (new release, same track)                                                                                                      | Upgrade (new track)                                       |
| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------- |
| What changes           | Kubernetes patch version, OS and cluster component revisions, CVE fixes                                                               | A new Kubernetes minor version and the stack built for it |
| How it starts          | A new release ships on your track and is made available automatically; you move the cluster onto it by pointing it at the new release | You point the cluster at a release on the next track      |
| Kubernetes API surface | Unchanged                                                                                                                             | New features arrive; deprecated APIs can be removed       |
| What you prepare       | PodDisruptionBudgets and graceful shutdown                                                                                            | The same, plus scanning your manifests for removed APIs   |

On your track you still get every security fix. You only miss new features.

### Tracks and releases

A track is named `<major>-<minor>`, for example `1-36`, and targets one Kubernetes minor. It stays supported until upstream ends support for that minor, plus three more months. After that it stops getting security patches. The live status of every track and its supported releases is in the [version hub](/versions).

While a track is live, new releases ship about every four weeks, sooner when a security fix forces it. Upstream ships a patch for a minor roughly once a month, and those patches go into your track's releases. A new track lands a few times a year, one per upstream Kubernetes minor.

Your cluster's behavior changes only when you move to a new minor. Syself batches its feature work into each new minor, so the features built since the last minor all arrive together with the new track. Moving to a new minor is occasional; updating within your track is routine.

### Why a new track lands later than upstream

A new track does not follow the upstream Kubernetes release right away. It usually lands about three months later, around the third upstream patch of the new minor. The delay is deliberate, because after upstream cuts a minor the projects Syself integrates all need time to support it: Cilium (the network layer), the cloud controller manager, the CSI drivers, and the rest. Once that ecosystem is ready, Syself pins the set together, builds it, and tests the whole stack as one unit. That test includes the exact upgrade from the previous track.

Networking is the slow part of that testing. It takes more than setting a few Cilium values. It covers how Kubernetes does networking, the kernel modules in use, and what the OS provides. Those layers all have to work together.

## What triggers a rollout

The version fields are the usual trigger: change the Kubernetes version or the cluster stack version and the control plane rolls, then the workers. A handful of other variables also patch the control-plane template and roll it, among them the OIDC and etcd-encryption settings, `dnsServers`, and machine-type or placement changes; control-plane certificates near expiry roll it too. The [cluster variables reference](/docs/hetzner/apalla/reference/cluster-variables) marks which variables trigger a roll.

One case is easy to get wrong. Changing the OIDC or etcd encryption _variable_ patches the control-plane template and rolls the control plane. Editing the _Secret those variables point at_, say rotating the etcd encryption key inside it, rolls nothing. Each node reads Secret contents once, at boot. So the edit has no effect until the control-plane machines are next replaced for some other reason. To make a Secret edit take effect now, roll the control plane yourself.

## The control plane rolls one node at a time

Control-plane nodes roll strictly one at a time.

The reason is etcd. Each control-plane node runs an etcd member, and etcd needs a majority of its members to accept a write. In a group of three you can lose one member and keep the majority, but not two.

So only one etcd member joins or leaves at a time. A three-node control plane keeps its majority, and the API server stays up. The rollout waits for each new node to report a healthy API server before it moves to the next one.

Which way the roll goes depends on the backend. A Hetzner Cloud server can run beside its replacement. A bare-metal host has to be freed before the same physical machine is reprovisioned, so it goes the other way.

| Roll          | Backend    | During each step                                                        |
| ------------- | ---------- | ----------------------------------------------------------------------- |
| Control plane | Cloud      | A new node is created first, then the old one is deleted; no gap        |
| Control plane | Bare metal | The host is freed first, then reprovisioned; one fewer node for a while |
| Workers       | Cloud      | A new node joins first, then the old one leaves; capacity stays stable  |
| Workers       | Bare metal | The same host is freed and reprovisioned; one fewer worker for a while  |

For bare-metal pools, keep one spare slot so the dip during each step does not reach your workloads.

The API server, controller-manager, scheduler, and etcd are never touched in place. A fresh node replaces the old one, and the components come up from the new image. That node runs an immutable operating system. Immutable means the system cannot change after it is built: there is nothing to patch on a running node, nothing drifts over time, and you can check that a node runs exactly what Syself shipped.

The full rollout, from your one field change to the final lockdown:

```mermaid
flowchart TB
    A["You point the Cluster at a new release"] --> B["Control plane rolls, strictly one node at a time"]
    B --> C["New node's API server healthy? Rollout waits for it"]
    C --> D["Next control-plane node, until all are replaced"]
    D --> E["Worker pools roll, each pool independently"]
    E --> F["Final steps: last cluster components and the API server lockdown apply once every node runs the new image"]
```

## How workloads stay up during a rollout

Every replacement drains the node first, and the drain respects PodDisruptionBudgets. A PodDisruptionBudget can delay a drain; it cannot stop the rollout. Set one on anything that matters, and make sure your pods shut down cleanly when asked to stop. The full cordon, drain, replace sequence, and the workload rules around it, live in [self-healing and node replacement](/docs/hetzner/apalla/concepts/operations/self-healing-and-node-replacement).

Worker pools use `RollingUpdate` by default. Switch a pool to `OnDelete` and nodes are replaced only when you delete one yourself. That is what you want for stateful workloads, where you pick the moment of disruption.

During an upgrade, an `OnDelete` pool holds back the final steps. The last cluster components and the strictest firewall rule (the lockdown that lets only the load balancer reach the API server) apply only after _every_ node runs the new image, control plane and all workers.

While an `OnDelete` pool still runs old nodes you have not drained, the cluster sits in a mixed state. The held-back cluster components stay pending, and a more permissive migration firewall stays in place. The rollout also takes longer.

Drain those nodes to finish the upgrade.

## Upgrades move one minor at a time, control plane first

Kubernetes only guarantees that the API server and kubelet work together within one minor of each other, and each track covers exactly one minor. So you cannot jump straight from `1-34` to `1-36`.

Moving to a new minor goes through your current track first. Upgrade to the latest release on your current track and let that rollout finish, then point the cluster at the latest release of the next track. Going straight from an older `1-35` release to `1-36` is not a tested path. Every release is tested against the upgrade from the release before it, not from any release you pick. How each release is built and frozen is the [cluster stack](/docs/hetzner/apalla/concepts/foundations/cluster-stacks).

The control plane always upgrades before the workers. Workers can run one minor behind for a while, which is normal version skew, but the supported end state is the whole cluster on the new version.

You control the timing. A pinned version stays pinned until you change the field. New releases wait until you point the cluster at them; they never roll a node on their own. An `OnDelete` pool waits for you to delete each node.

## Check your manifests for removed APIs

The release tests cover Syself's own components, mainly the Cilium CNI. They do not read your manifests, so the platform cannot tell that one of your Deployments still uses an API the new minor removes, which can fail the upgrade or break the application when the endpoint disappears. Scanning your manifests and live cluster for removed APIs is a step you run before moving to a new minor; [plan maintenance and pre-checks](/docs/hetzner/apalla/clusters/upgrades/plan-maintenance-and-prechecks) covers the tools and the full pre-flight.

## Stay current

Take each release on your track as it ships, and start the minor move as soon as the next track is out. Each step stays small and is tested against the release before it.

A cluster left until its track nears end of support has more minor moves to make, and it makes them against a deadline. There is less slack if an `OnDelete` pool or a blocking PodDisruptionBudget is in the way.

Next: each new node boots the immutable [Syself Linux](/docs/hetzner/apalla/concepts/internals/syself-linux) image instead of being patched in place, and [prepare workloads for upgrades](/docs/hetzner/apalla/clusters/availability/prepare-workloads-for-upgrades) keeps an application serving while its nodes roll.