Run GPU workloads

Running a GPU workload on Syself Autopilot takes no driver setup. When a bare-metal server with a supported NVIDIA GPU joins the cluster, the driver, the CDI spec, and the device plugin are already in place, because they ship inside the sealed node OS . You request a GPU the way you request memory, and the pod runs. There is no operator to install, no driver DaemonSet to reconcile, and no image variant to pick.

What happens when a GPU server joins#

The work that usually falls to a driver operator happens earlier here, before the kubelet ever registers the node:

1. During provisioning, the server's PCI devices are matched against the NVIDIA models the platform recognizes.
2. On a match, the driver, CDI spec, and device plugin are already present in the node OS when it boots. A non-match receives nothing extra; the card is left as a plain display adapter.
3. The node boots with the driver, the CDI (Container Device Interface) spec, and the device plugin already resident.
4. The device plugin advertises nvidia.com/gpu and labels the node autopilot.syself.com/gpu: "true", and the scheduler can place work.

Because the driver is built in rather than pulled at runtime, a GPU node that reaches Ready is a GPU node that can run CUDA. There is no window where the node is up but the GPU is still being configured.

Supported cards#

The match in step 1 fires only for cards the platform recognizes. See supported GPU cards for the current list. Any other NVIDIA card is treated as a plain display adapter and receives no GPU support: no GPU driver, no device plugin, no advertised resource. There is no separate GPU image to select and no cluster-class option to set; a recognized card is enough.

Schedule a GPU pod#

Request nvidia.com/gpu in the container's limits. The scheduler places the pod on a GPU node, and the CDI spec injects the device into the container.

		apiVersion: v1
kind: Pod
metadata:
  name: gpu-check
spec:
  restartPolicy: Never
  containers:
    - name: cuda
      image: nvidia/cuda:12.4.1-base-ubuntu22.04
      command: ["nvidia-smi"]
      resources:
        limits:
          nvidia.com/gpu: 1 # whole GPUs only; this request is what pins the pod to a GPU node
	

You do not need a nodeSelector. The nvidia.com/gpu request already steers the pod, because non-GPU nodes never advertise the resource and so never match. Add nodeSelector: { autopilot.syself.com/gpu: "true" } only when you want the intent spelled out in the manifest.

Run it, then kubectl logs gpu-check. Seeing nvidia-smi list the card model and driver version from inside the container is the signal that matters: it means the CDI injection reached the pod, not merely that the scheduler found a node with a free slice.

Read Capacity against Allocated#

Every GPU advertises a fixed slice count, and the common mistake is reading the wrong field:

		$ kubectl describe node <gpu-node> | grep nvidia.com/gpu
  nvidia.com/gpu:  4      # Capacity and Allocatable: the fixed slice count
	

Capacity and Allocatable report the total slices (commonly 4), and they do not shrink as pods claim them. To see what is actually in use, read the Allocated resources section further down the same describe output. Allocatable: 0 means the device plugin is missing or unhealthy, not that the card is busy. What a slice is, and how many pods should share one card, are covered in Share a GPU across pods .

Troubleshooting#

Related: Add GPU nodes for the hardware and a dedicated pool, GPU not scheduling to fix a pod stuck Pending, and Share a GPU across pods for slicing. For adding GPU servers, see Add bare-metal servers .