Share a GPU across pods
Users very often misunderstand what a GPU slice provides. If a GPU card advertises 4 slices as nvidia.com/gpu, that means up to four pods can schedule onto one card. This seems like each pod gets a quarter of the GPU. However, that is not the case. Instead it is four workloads competing for the same memory.
A slice is an allocation token, not a share
Requesting nvidia.com/gpu: 1 does not cap your pod to a quarter of the card. Time-slicing does not partition the hardware; it lets several CUDA contexts take turns on the whole device. A pod that runs alone gets everything: full compute, full memory, no throttling. Add a second and they alternate, each still seeing the entire GPU. The number is a limit on how many pods may use the card at the same time.
A pod cannot request more than one slice. nvidia.com/gpu: 2 is rejected since one slice already exposes the whole GPU.
The failure mode: shared memory
Warning
Time-slicing shares compute time but does not isolate memory. Co-scheduled pods share the card’s VRAM with no per-pod quota: one pod can allocate all of it and leave the other pods with out-of-memory errors, even though they were using only their normal share. Only co-schedule workloads you trust to stay inside the card’s memory budget.
No configuration option changes this behavior. NVIDIA MPS, which can enforce a per-client memory cap, is not enabled, and the device-plugin configuration cannot be overridden by users. MIG, which partitions a card into isolated instances in hardware, is not supported on the GPU models Hetzner offers. Memory isolation is therefore unavailable, and time-slicing is the only sharing mode.
Choose deliberately: share or dedicate
Share a card when you can be confident the co-tenants’ combined VRAM stays within its limit. This is the cheapest option and works well for several small inference pods on a large card. When you cannot guarantee that, dedicate the card to a single workload.
Two valid positions remain. The choice turns on a single question:
Dedicating is a scheduling problem, not a partitioning one: a singular GPU pod already owns the whole card, so you only have to keep a second GPU pod off the node. Because each GPU server is a single node, required pod anti-affinity on kubernetes.io/hostname does this directly: label your GPU pods and use podAntiAffinity so no two land on the same node, and each card stays with one pod. It only repels pods that match its selector, so it keeps your own replicas one-per-node.
To reserve the nodes from every other workload too, put the GPU servers in a dedicated worker pool with a distinct label, and give the workload a nodeSelector for it. Do not use a taint: custom node taints are not available, and a hand-applied kubectl taint is lost the moment the node is replaced (see Taints and tolerations ). For a hard, enforced guarantee that nothing can ever share it, contact Syself.
Run GPU workloads
Schedule a pod onto an NVIDIA GPU node with no manual driver setup, because the sealed node OS ships the driver, CDI spec, and device plugin before the node boots.
Run NUMA-aligned workloads
Pin a pod's CPUs and memory to one socket on multi-socket bare metal so latency-sensitive databases and real-time services stop paying the cross-socket tax.