Taints and tolerations
Set taints on worker pools or the control plane in the Cluster topology. Taints set this way are propagated to the nodes and survive node replacements, upgrades, and scaling. A taint repels pods that do not tolerate it.
Set taints in the Cluster topology
Add taints to a worker pool or the control plane in the Cluster object:
spec:
topology:
workers:
machineDeployments:
- class: workeramd64hcloud
name: md-jobs
replicas: 2
failureDomain: fsn1
taints:
- key: dedicated
value: jobs
effect: NoSchedule
propagation: Always
controlPlane:
replicas: 3
taints:
- key: example.com/maintenance
effect: PreferNoSchedule
propagation: Always
Taints are applied to existing Machines and Nodes without a rollout. Syself Autopilot propagates the taints from the Cluster to the Machines and the Nodes.
Adding an Always taint with the NoExecute effect to a pool that is already running evicts every pod on those nodes that does not tolerate it. Make sure the workloads have matching tolerations before you apply the taint.
Taint fields
| Field | Required | Values |
|---|---|---|
key | Yes | A valid label key, max 253 characters |
value | No | Max 63 characters |
effect | Yes | NoSchedule, PreferNoSchedule, NoExecute |
propagation | Yes | Always, OnInitialization |
key plus effect must be unique within a taint list. A list holds at most 64 taints.
Reserved keys
The webhook rejects taints with these keys:
- Any key starting with
node.kubernetes.io/, exceptnode.kubernetes.io/out-of-service - Any key starting with
node.cloudprovider.kubernetes.io/ node.cluster.x-k8s.io/uninitializednode.cluster.x-k8s.io/outdated-revisionnode-role.kubernetes.io/master
Use your own domain or a plain key like dedicated.
Always vs OnInitialization
| Always | OnInitialization | |
|---|---|---|
| Applied to | All nodes of the pool, existing and new | Only new nodes, once, when they join |
| Removed from a node by hand | Added back within seconds | Stays removed |
| Removed from the Cluster | Removed from the nodes | Nodes keep it |
| Typical use | Dedicated pools (GPU, database, edge) | Block pods until a setup job removes the taint |
Changing a taint from Always to OnInitialization leaves it on the nodes but stops managing it. Changing it back makes Syself Autopilot manage it again.
Built-in taints
The platform sets these taints by default:
- Control-plane
NoSchedule: keeps workloads off control-plane nodes. Pods land only on workers unless they tolerate this taint. node.cloudprovider.kubernetes.io/uninitialized: sits on a brand-new node for a few seconds while the platform wires it up, then clears itself.
When you taint a pool that runs a platform DaemonSet (for example, the nvidia device plugin on GPU nodes), use a taint key the DaemonSet already tolerates. The nvidia device plugin tolerates nvidia.com/gpu. A custom key like dedicated=gpu blocks the DaemonSet from scheduling, which makes the GPU unusable.
Tolerations
A pod that must run on tainted nodes needs a matching toleration.
A DaemonSet that must cover every node, including tainted ones:
tolerations:
- operator: Exists
A pod that targets one specific taint:
tolerations:
- key: dedicated
value: jobs
effect: NoSchedule
operator: Equal
Taints and the Cluster Autoscaler
When a tainted pool has running nodes, the Cluster Autoscaler copies a real node to simulate scheduling, so taints are included automatically. When a pool is scaled to zero, the autoscaler builds the simulated node from the MachineDeployment. Cluster Autoscaler 1.36.0 and later reads the taints field from the MachineDeployment and no extra configuration is needed. If you run an older version, add the capacity.cluster-autoscaler.kubernetes.io/taints annotation to the pool in the Cluster topology.
Taints set with kubectl
A taint added by hand with kubectl taint lives only on that node. It is lost when the node is replaced, which happens on upgrades and maintenance. Set taints in the Cluster manifest instead.