Spread pods with topology constraints

Topology spread constraints keep a workload's replicas apart, across nodes and across zones, so a single node replacement takes at most one of them. On a platform where node replacement is routine, it is important to ensure a drain does not remove all replicas once.

maxSkew, and the default you are overriding

maxSkew is how uneven the spread may get: the busiest topology domain may hold at most maxSkew more matching pods than the emptiest. A pod that carries no spread rule of its own still gets one, because the cluster applies a default. But that default is deliberately loose:

In the first row we see maxSkew: 3 across nodes, which means three replicas can additionally be put onto one node, and losing that node takes all three. For anything that has to survive a single node replacement, write your own tighter rule:

		spec:
  topologySpreadConstraints:
    - maxSkew: 1
      topologyKey: kubernetes.io/hostname
      whenUnsatisfiable: DoNotSchedule
      labelSelector:
        matchLabels:
          app: web
	

maxSkew: 1 limits the difference between any two nodes to a single pod, so replicas are placed on separate nodes. Note that a constraint you define replaces the default constraint for that pod rather than applying in addition to it.

DoNotSchedule or ScheduleAnyway

whenUnsatisfiable is where you decide what matters more: the placement, or the pod running at all.

• DoNotSchedule holds a pod Pending until it can spread. This is strict high availability: an imbalanced placement is not allowed to happen.
• ScheduleAnyway places the pod even when it cannot spread evenly. Use it when you prefer a running but unevenly placed pod rather than no pod.

Zones on bare metal

The CCM gives every node a zone label, bare metal included. The values differ in detail: cloud nodes get a datacenter like fsn1-dc14, Robot nodes get a location like fsn1. So a pool of Robot servers in one location is a single zone.

A single zone does not block scheduling. The skew is the gap between the fullest and the emptiest zone, and with only one zone that gap is always 0. A DoNotSchedule zone constraint is therefore satisfied straight away, and every replica lands.

What does block scheduling is a topologyKey that no node carries. Under DoNotSchedule those nodes are skipped entirely and every replica stays Pending. On cloud, minDomains goes the other way, requiring a minimum number of zones to actually get used before the constraint counts as satisfied.

Verify it survived a drain

We will verify whether the spread holds when a node is removed. Start with three replicas across worker-1, worker-2, and worker-3, then drain the middle one:

		$ kubectl drain worker-2 --ignore-daemonsets --delete-emptydir-data
	

Watch where the evicted replica lands. Under maxSkew: 1 it must reschedule onto a node that does not already hold a matching pod, never doubling up on worker-1 or worker-3. If instead it sits Pending, the spread is working exactly as told: there is no node left that satisfies the rule, and your choices are to add a node or relax that one constraint to ScheduleAnyway.

Spread decides where pods land; it does not cap how many are evicted at once. Pair it with a PodDisruptionBudget to keep enough replicas running through the drain itself.