Keep ingress and load balancers available
Traffic from outside the cluster reaches your app through two hops that an upgrade disrupts: the ingress controller pods, and the load balancer that routes to the nodes those pods run on. Both need protecting, on top of the four rules for the app itself .
Your ingress controller is a workload#
An ingress controller (Traefik, ingress-nginx, or similar) is a Deployment like any other, and it sits in the path of every request. If all of its replicas land on the node being drained, everything behind it goes down, no matter how well the backend apps are spread.
Apply the same four rules to the ingress controller:
- Run at least two replicas.
- Spread them one-per-node with a hard topology spread constraint.
- Give it a PodDisruptionBudget with
minAvailable: 1. - Keep its readiness probe, so the load balancer only sends traffic to a controller that is ready.
Most ingress controller charts expose all four as values. Set them rather than relying on the chart defaults, which are often a single replica with no budget.
The load balancer adds one more path#
A type: LoadBalancer Service on Hetzner is backed by a cloud load balancer that targets the cluster's nodes. When a node is drained and replaced, the load balancer has to drop the old node and register the new one as a target. That registration is not instant, so there is a short window on each node replacement where the load balancer may still send to a node that is going away, or not yet send to its replacement.
This window is the gap left over after the app and the ingress controller are configured correctly. It is a few seconds per node replacement, and it comes from the load balancer catching up with the node change, not from the upgrade taking your pods down. Two settings shrink it.
Choose the right externalTrafficPolicy#
The Service's externalTrafficPolicy decides which nodes the load balancer can use:
| Policy | Behavior during a roll | Client source IP |
|---|---|---|
Cluster (default) | Any node forwards to a ready pod anywhere in the cluster, so a surviving node keeps serving while another is replaced. The most forgiving choice for availability. | Lost (pods see the node IP) unless you use PROXY protocol. |
Local | Only nodes that run a ready pod receive traffic. The Hetzner load balancer health-checks each node and stops sending to a node with no ready pod; the cloud-controller-manager configures that check. Preserves the client IP, but you must spread the pods so surviving nodes have one, and the health check governs how fast a draining node is dropped. | Preserved. |
If you do not need the real client IP, Cluster gives the smoothest upgrades: every surviving node can forward to your spread-out pods. If you need the client IP, use Local and make sure the pods are spread one-per-node so no surviving node is left without one. See Preserve the client source IP for the full trade-off.
Tune the load balancer health check#
The load balancer stops sending to a node when that node fails its health check. A shorter health-check interval drops a draining node sooner and shortens the window where traffic lands on a node that is leaving. Set the health-check interval and thresholds on the Service; the annotations are listed in Configure a load balancer .
Put it together#
For an ingress controller behind a Hetzner load balancer:
- Run two or more controller replicas, spread one-per-node, with a
minAvailable: 1PodDisruptionBudget. - Spread the backend app pods the same way, so a surviving node always has a ready pod to forward to.
- Use
externalTrafficPolicy: Clusterunless you need the client IP, in which case useLocalwith pods spread across nodes. - Tighten the health-check interval so the load balancer drops a draining node quickly.
With all four in place, external traffic keeps flowing through the upgrade, down to the few-second re-registration window on each node swap.
Related#
- Configure a load balancer : the full annotation list, including health checks and algorithm.
- Expose an application : Service, ingress, and
externalTrafficPolicyfrom the start. - Keep stateless workloads available : the four rules the ingress controller also needs.
- Keep databases available : the stateful side of surviving a node roll.
Keep stateless workloads available
Configure replicas, a hard topology spread, a PodDisruptionBudget, readiness probes, and graceful shutdown so a stateless workload stays served through a Syself Autopilot upgrade.
Keep databases available
How a database behaves when its node is replaced during a Syself Autopilot upgrade, why a single instance has a short gap, and how a replicated database with failover stays available.