Skip to main content

Dedicate node pools per client

Inspect 1.36

Namespaces separate clients logically, but their pods still share nodes. A namespace is a logical partition inside one cluster.

When tenants must not share hardware, give each client its own pool of nodes. This puts isolation at the machine level, not just the namespace level. It is the pattern agencies and MSPs (managed service providers) use for clients that need a hard boundary.

Why node-level isolation matters#

Sensitive tenants, compliance rules, and noisy neighbors all need the same fix: separate machines. A noisy neighbor is one tenant's load slowing another's on a shared node.

The strongest boundary short of a separate cluster is a separate set of machines. A dedicated pool means a client's pods run only on that client's nodes, and no other tenant's pods land there.

Two levers: dedicated machines and sandboxed workloads#

Dedicating machines is one lever. The other is the runtime a workload runs under. Besides giving a tenant its own pool, you can run a specific workload with the secure runtime class, which runs each pod inside its own lightweight virtual machine (Kata Containers). A container escape then stops at the VM boundary instead of reaching the host or the other pods on the node.

yaml
		spec:
  runtimeClassName: secure
  containers:
    - name: app
      image: ...
	

The secure class works out of the box on bare-metal nodes with hardware virtualization. The RuntimeClass includes its own node selector, so you do not need to add one. Each pod uses a little more startup time and memory than a normal container, so use it where you need the stronger boundary, not everywhere. Together, the two levers let a sensitive tenant run sandboxed workloads on its own dedicated hardware.

One pool per client#

Give each client a pool with a client label (see ):

yaml
		workers:
  machineDeployments:
    - class: workeramd64hcloud
      name: md-client-acme
      replicas: 3
      metadata:
        labels:
          node.cluster.x-k8s.io/client: acme
    - class: workeramd64hcloud
      name: md-client-globex
      replicas: 2
      metadata:
        labels:
          node.cluster.x-k8s.io/client: globex
	
Tip

Only labels in the node-role.kubernetes.io, node-restriction.kubernetes.io, and node.cluster.x-k8s.io domains reach the node. A label in your own domain, for example acme.com/client: acme, is silently dropped. Use node.cluster.x-k8s.io/client as shown above.

Steer each client's pods, and keep others off#

Pin a client's workloads to its nodes with a nodeSelector or . A nodeSelector is a rule that keeps a pod on nodes carrying a given label:

yaml
		spec:
  nodeSelector:
    node.cluster.x-k8s.io/client: acme
	

A label attracts but does not repel. A pod without a nodeSelector can still land on a client's node. Syself Autopilot does not support node taints (see ), so the boundary is the pool label plus discipline on your side: give every workload a nodeSelector, so nothing lands on a client's pool by accident.

For a boundary that does not depend on every pod being labelled correctly, put sensitive tenants on their own bare-metal pool, where the hardware itself is the separation.

Dedicated bare-metal pools for sensitive tenants#

For a tenant that must be on physically separate hardware, not just a separate pod, give it a bare-metal pool. Bare-metal (Robot) servers are physical machines you own. A pool that only claims that tenant's hosts means no other tenant's pod can ever land on that hardware.

First label the tenant's HetznerBareMetalHost objects (see ):

		$ kubectl label hbmh baremetal-1 client=acme
	

Then point the pool at those hosts with workerHostSelectorBareMetal:

yaml
		- class: workeramd64baremetal
  name: md-client-acme-bm
  replicas: 2
  variables:
    overrides:
      - name: workerHostSelectorBareMetal
        value:
          matchLabels:
            client: acme
	

This is a second, separate label mechanism from the client pool label:

Label Lives on Decides
client: acme (host selector) HetznerBareMetalHost Which physical machine joins the pool
node.cluster.x-k8s.io/client: acme (pool label) Node, once the machine joins Where pods land

You need both: the host selector picks which hardware becomes Acme's, the pool label steers Acme's pods onto it.

Warning

Without a host selector, any free HetznerBareMetalHost can be claimed by any bare-metal pool in the same namespace, including ones a different client's pool is scaling into. Set workerHostSelectorBareMetal on every bare-metal pool, not only the sensitive one.

Note

A per-client selector is the isolation you want, but it also leaves self-healing no other host to fall back to: if that tenant's server fails, the replacement re-claims the same machine, and a hardware fault survives the reinstall. Alert on the hardware so a broken server surfaces instead of quietly churning that client's pool. See .

You can run several bare-metal pools in one cluster, each selecting a different set of hosts, so each sensitive tenant gets its own machines. Check kubectl get machines -o wide: a bare-metal machine shows an hrobot:// provider ID, confirming it landed on Robot hardware and not a cloud VM.

Failure isolation per pool#

For cloud pools, a Hetzner placement group keeps a pool's VMs on separate physical hosts. Without one, Hetzner can put two of a client's VMs on the same physical host, so one hardware failure takes both. spread is the only placement group type Hetzner Cloud supports.

Declare a group under spec.topology.variables, then reference it by name from the client's pool:

yaml
		variables:
  - name: hcloudPlacementGroups
    value:
      - name: acme-spread
        type: spread
workers:
  machineDeployments:
    - class: workeramd64hcloud
      name: md-client-acme
      replicas: 3
      variables:
        overrides:
          - name: workerMachinePlacementGroupNameHcloud
            value: acme-spread
	

Give each client pool its own group. Then one physical-host failure takes at most one node from one client, and a single failure never spans tenants.

Group names are scoped per cluster. Hetzner creates the real group prefixed with the cluster name (<cluster>-acme-spread in the console), so the same naming scheme reused across every cluster in a fleet never collides, and each cluster's groups stay separate even when they share one Hetzner project.

Bare-metal pools do not need this. A Robot server already is one physical host, so the dedicated bare-metal pool above already gives you that isolation.

Cost attribution and hand-off#

Each client's capacity is its own pool of named machines. You can read the spend per client from the machine count and type in that pool, and adjust each pool on its own.

Keeping clusters on the customer's own Hetzner account makes the hand-off clean. The machines, and their bill, already sit with the customer.

Dedicated pools make "these nodes are yours" concrete. This is how you organize pools, not a separate feature to enable.

Where to go next#