Skip to main content

Taints and tolerations

Set taints on worker pools or the control plane in the Cluster topology. Taints set this way are propagated to the nodes and survive node replacements, upgrades, and scaling. A taint repels pods that do not tolerate it.

Set taints in the Cluster topology 

Add taints to a worker pool or the control plane in the Cluster object:

yaml
				spec:
  topology:
    workers:
      machineDeployments:
        - class: workeramd64hcloud
          name: md-jobs
          replicas: 2
          failureDomain: fsn1
          taints:
            - key: dedicated
              value: jobs
              effect: NoSchedule
              propagation: Always
    controlPlane:
      replicas: 3
      taints:
        - key: example.com/maintenance
          effect: PreferNoSchedule
          propagation: Always
			

Taints are applied to existing Machines and Nodes without a rollout. Syself Autopilot propagates the taints from the Cluster to the Machines and the Nodes.

Adding an Always taint with the NoExecute effect to a pool that is already running evicts every pod on those nodes that does not tolerate it. Make sure the workloads have matching tolerations before you apply the taint.

Taint fields 

Field Required Values
key Yes A valid label key, max 253 characters
value No Max 63 characters
effect Yes NoSchedule, PreferNoSchedule, NoExecute
propagation Yes Always, OnInitialization

key plus effect must be unique within a taint list. A list holds at most 64 taints.

Reserved keys 

The webhook rejects taints with these keys:

  • Any key starting with node.kubernetes.io/, except node.kubernetes.io/out-of-service
  • Any key starting with node.cloudprovider.kubernetes.io/
  • node.cluster.x-k8s.io/uninitialized
  • node.cluster.x-k8s.io/outdated-revision
  • node-role.kubernetes.io/master

Use your own domain or a plain key like dedicated.

Always vs OnInitialization 

Always OnInitialization
Applied to All nodes of the pool, existing and new Only new nodes, once, when they join
Removed from a node by hand Added back within seconds Stays removed
Removed from the Cluster Removed from the nodes Nodes keep it
Typical use Dedicated pools (GPU, database, edge) Block pods until a setup job removes the taint

Changing a taint from Always to OnInitialization leaves it on the nodes but stops managing it. Changing it back makes Syself Autopilot manage it again.

Built-in taints 

The platform sets these taints by default:

  • Control-plane NoSchedule: keeps workloads off control-plane nodes. Pods land only on workers unless they tolerate this taint.
  • node.cloudprovider.kubernetes.io/uninitialized: sits on a brand-new node for a few seconds while the platform wires it up, then clears itself.

When you taint a pool that runs a platform DaemonSet (for example, the nvidia device plugin on GPU nodes), use a taint key the DaemonSet already tolerates. The nvidia device plugin tolerates nvidia.com/gpu. A custom key like dedicated=gpu blocks the DaemonSet from scheduling, which makes the GPU unusable.

Tolerations 

A pod that must run on tainted nodes needs a matching toleration.

A DaemonSet that must cover every node, including tainted ones:

yaml
				tolerations:
  - operator: Exists
			

A pod that targets one specific taint:

yaml
				tolerations:
  - key: dedicated
    value: jobs
    effect: NoSchedule
    operator: Equal
			

Taints and the Cluster Autoscaler 

When a tainted pool has running nodes, the copies a real node to simulate scheduling, so taints are included automatically. When a pool is scaled to zero, the autoscaler builds the simulated node from the MachineDeployment. Cluster Autoscaler 1.36.0 and later reads the taints field from the MachineDeployment and no extra configuration is needed. If you run an older version, add the capacity.cluster-autoscaler.kubernetes.io/taints annotation to the pool in the Cluster topology.

Taints set with kubectl 

A taint added by hand with kubectl taint lives only on that node. It is lost when the node is replaced, which happens on upgrades and maintenance. Set taints in the Cluster manifest instead.