Share a GPU across pods
The most misunderstood aspect of GPUs on Kubernetes is what a slice provides. Every card here advertises 4 slices as nvidia.com/gpu, so up to four pods can schedule onto one card. This reads like a quarter each. It is nothing of the sort, and getting it wrong is how you spend an afternoon debugging a "bug" that is really two workloads competing for the same memory.
A slice is an allocation token, not a share#
Requesting nvidia.com/gpu: 1 does not cap your pod to a quarter of the card. Time-slicing does not partition the hardware; it lets several CUDA contexts take turns on the whole device. A pod that runs alone gets everything: full compute, full memory, no throttling. Add a second and they alternate, each still seeing the entire GPU. The number is a ceiling on how many pods may land on a card, not a portion handed to each: a pod cannot request more than one (nvidia.com/gpu: 2 is rejected), and never needs to.
The failure mode: shared memory#
Warning
Time-slicing shares compute time but does not isolate memory. Co-scheduled pods share the card's VRAM with no quota, in one fault domain: one can allocate all of it and leave its neighbours with out-of-memory errors, and a pod that fails this way appears to have a bug of its own. Only co-schedule workloads you trust to stay inside the card's memory budget.
No mode here changes that. MPS, which can cap memory per client, is not enabled, and the device-plugin config is not user-overridable; MIG, which partitions a card in hardware, does not exist on the GPU models Hetzner offers. There is no middle setting available.
Choose deliberately: share or dedicate#
Two valid positions remain. The choice turns on a single question:
Tip
Do you trust the co-tenants' combined VRAM to stay inside the card? Share it: it is the cheapest option, and fine for several small inference pods on a large card. Cannot verify it? Dedicate the card to one workload.
Dedicating is a scheduling problem, not a partitioning one: a lone GPU pod already owns the whole card, so you only have to keep a second one off the node. Do not use a taint: custom node taints are not available, and a hand-applied kubectl taint is lost the moment the node is replaced (see Reserve nodes with taints and tolerations ). Use a pool instead: put the GPU servers in a dedicated worker pool with a distinct label, and give the workload a nodeSelector for it. You decide what targets the pool, so the card stays with that one workload. For a hard, enforced guarantee that nothing can ever share it, contact Syself.
Run GPU workloads
Schedule a pod onto an NVIDIA GPU node with no manual driver setup, because the sealed node OS ships the driver, CDI spec, and device plugin before the node boots.
Run NUMA-aligned workloads
Pin a pod's CPUs and memory to one socket on multi-socket bare metal so latency-sensitive databases and real-time services stop paying the cross-socket tax.