Syself Autopilot provisions fresh nodes for every upgrade, patch, and self-healing event. During a node replacement, the Hetzner Load Balancer relies on active health checks to detect when a node is being drained and remove it from the backend pool. Configuring proper health check intervals and timeouts ensures leaving nodes drop out of the target pool before their workloads stop answering requests.

## What happens on a node swap

When Syself Autopilot replaces a node, target deregistration is not instantaneous. If health checks are too slow, the Hetzner Load Balancer may continue routing traffic to a terminating node after its pods have already begun shutting down.

To eliminate dropped connections, the load balancer must detect the draining state and drop the node from rotation before local pods terminate, while remaining cluster nodes absorb ongoing traffic.

## The health-check annotations

Set these on the `type: LoadBalancer` Service (or on the ingress controller's Service). Pick the check protocol first, then the timing.

<Tabs id="health-check-protocol" items={['HTTP check', 'TCP check']}>
<TabItem value="HTTP check">

An `http` (or `https`) check performs a `GET` request against the specified path and expects a `2xx` or `3xx` HTTP status code, reflecting the application's actual readiness. If the application begins failing or enters shutdown, the node is dropped from the load balancer pool even while its `NodePort` socket remains open. Use this whenever your workload provides a `/healthz` or readiness endpoint.

```yaml
metadata:
  annotations:
    load-balancer.hetzner.cloud/health-check-protocol: http
    load-balancer.hetzner.cloud/health-check-http-path: /healthz
    load-balancer.hetzner.cloud/health-check-interval: "3s"
    load-balancer.hetzner.cloud/health-check-timeout: "2s"
    load-balancer.hetzner.cloud/health-check-retries: "2"
```

</TabItem>
<TabItem value="TCP check">

A `tcp` check only opens the port. It is the default and requires no path, but it cannot distinguish a hung application from a healthy one as long as the port still accepts connections. Use it for non-HTTP backends.

```yaml
metadata:
  annotations:
    load-balancer.hetzner.cloud/health-check-protocol: tcp
    load-balancer.hetzner.cloud/health-check-interval: "3s"
    load-balancer.hetzner.cloud/health-check-timeout: "2s"
    load-balancer.hetzner.cloud/health-check-retries: "2"
```

</TabItem>
</Tabs>

- **Write `interval` and `timeout` with a time unit.** Use `"15s"` for 15 seconds or `"1m30s"` for a minute and a half. A plain number such as `15` is rejected, and the Service fails to reconcile.
- A shorter `interval` and fewer `retries` drop a failing node from the pool sooner, which shrinks the swap window. Do not set them so aggressively that a single slow response ejects a healthy node.

For the complete list of supported Service annotations see [Configure a Hetzner load balancer](/docs/hetzner/apalla/network/load-balancing/configure-a-load-balancer).

## Workload redundancy during node drains

Tuning load balancer health checks ensures fast target removal, but workloads must also be designed to absorb traffic smoothly during node evictions:

- **Multiple replicas:** Run at least two replicas of any externally exposed service.
- **Topology spread:** Spread pods across nodes using [topologySpreadConstraints](https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/) so a single node replacement never evicts all replicas simultaneously.
- **PodDisruptionBudgets:** Configure a [PodDisruptionBudget](/docs/hetzner/apalla/workloads/production/pod-disruption-budgets) to prevent the drain controller from evicting the last remaining replica before a replacement is ready.

To learn how traffic routes through ingress controllers and load balancers during cluster upgrades, see [Keep ingress and load balancers available](/docs/hetzner/apalla/clusters/availability/keep-ingress-and-load-balancers-available). For details on the underlying node lifecycle, see [Self-healing and node replacement](/docs/hetzner/apalla/concepts/operations/self-healing-and-node-replacement).

> [!NOTE]
> When `externalTrafficPolicy: Local` is enabled, the Hetzner Load Balancer checks whether a healthy pod exists on the node itself rather than just checking node reachability. This prevents traffic from landing on nodes that have no local endpoints. See [Preserve the client source IP](/docs/hetzner/apalla/network/load-balancing/preserve-client-source-ip).