Bare metal control planes and the load balancer

Difference between bare metal and cloud servers#

CAPH attaches every control plane machine to the control plane load balancer, but it cannot do it the same way for both kinds of server.

An HCloud server is a resource of the HCloud API, the same API the load balancer belongs to, so CAPH attaches it by its server ID. This is a target of type server, and the load balancer resolves the address of that server on its own. There is nothing to choose.

A bare metal server is a Hetzner Robot resource. It has a server ID too, but not one the HCloud load balancer can reference, so CAPH attaches it by address instead. This is a target of type ip, and such a target holds exactly one address. Reaching one server over both protocols therefore takes two targets, and CAPH has to decide which of them to create.

spec.controlPlaneLoadBalancer.targetAddressFamily selects which of them CAPH creates.

The field only affects bare metal control plane servers. It does not change worker machines, which are never attached to the control plane load balancer, and it does not change a Service of type LoadBalancer in the workload cluster, which gets its own load balancer from the cloud controller manager rather than from CAPH.

Choosing the address family#

		apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: HetznerCluster
spec:
  controlPlaneLoadBalancer:
    targetAddressFamily: ipv4
	

The field is optional. Leaving it out is the same as setting dualstack.

An address the server does not have is never attached. If a server has no IPv6 address, dualstack attaches its IPv4 address and nothing else.

When to set ipv4#

Whether you want IPv6 depends on how you set up your nodes and their images, so it is your decision to make. Hetzner routes an IPv6 subnet to every bare metal server, so an IPv6 address exists in the Robot API, and CAPH records it as spec.status.ipv6 on the HetznerBareMetalHost. But an address existing does not mean the server uses it: the installed OS still has to configure it, and an image that only sets up IPv4 is common. CAPH cannot see from the Robot API what the OS configured, so it cannot decide for you.

When the server does not answer on the address, the target is still created and simply never passes its health check. The load balancer then reports an unhealthy target for as long as the machine exists, which buries a genuinely unhealthy control plane in the noise, and one target slot is spent on a target that cannot serve traffic.

A quick way to check what a server actually configured, from a shell on the machine:

		ip -6 addr show scope global
ip -6 route show default
	

If both are empty, the server has no usable IPv6 address and an IPv6 target for it cannot become healthy.

Changing the value later#

You can change the field at any time. On the next reconcile of each bare metal control plane machine, CAPH attaches the addresses of the newly selected family that are missing, and detaches the targets of the addresses the family no longer selects. No manual cleanup in the HCloud API is needed.

Detaching is not held back by the kube-apiserver health gate that governs attaching, because a target that is no longer selected cannot serve traffic and there is nothing to wait for.

Each change is visible in status.controlPlaneLoadBalancer.targets on the HetznerCluster, which lists the targets as they exist on the load balancer:

		kubectl get hetznercluster <name> -o jsonpath='{.status.controlPlaneLoadBalancer.targets}'
	

CAPH also emits events on the HetznerCluster when it attaches or detaches an address.