Skip to main content

Workload runtimes and isolation

Inspect 1.36

Syself Autopilot offers several container runtimes and isolation levels, from a shared kernel up to a separate VM for each pod. Pick the one that matches how much you trust the workload.

Syself Autopilot secures the host OS and the network. Inside the cluster, the platform confines every non-privileged container with an enforcing and the RuntimeDefault seccomp profile by default. The other workload controls (Pod Security Standards, network policy) are yours to set.

Container runtimes#

Your containers run on a fast, minimal runtime by default. The cluster ships RuntimeClasses for cases where you need something different.

standard (the default). A pod with no runtimeClassName uses this runtime. Behind it is crun, a container runtime written in the C programming language. Most Kubernetes platforms run runc, which is written in Go. Syself chose crun because it starts containers faster and needs less memory for every container it runs. Two things improve: pods start faster when your applications scale up, and each node keeps more memory free for your workloads. It works on every node type.

runc (fallback). A small number of container images behave differently on crun. If your workload does not start correctly, set runtimeClassName: runc to switch to the fallback.

secure (Kata Containers). Each pod runs in its own lightweight VM (QEMU/KVM). A container escape stays inside that VM's kernel and cannot reach the host. Set runtimeClassName: secure and the RuntimeClass's node selector sends the pod to a node carrying the virtualization label, which the provisioner sets only on bare-metal nodes where it detected the CPU's hardware virtualization support. Scheduling fails closed: with no capable node in the cluster, the pod stays Pending. It never silently falls back to a shared-kernel runtime.

The Kata runtime is part of the sealed OS, not something an installer writes onto the host. It ships inside the one immutable node image, pinned by content hash, and verified block by block by the kernel while the node runs (dm-verity is the kernel feature that checks every disk read against a hash tree). How that sealing works is . Two limits: secure works on bare metal only, and there is no confidential computing, because Hetzner servers do not offer those CPU features. See .

Note

On a shared-kernel runtime (standard or runc), a container that breaks out reaches the host kernel, which every pod on that node shares. So one escape puts the whole node at risk. The kernel is hardened against this: it loads only modules signed at build time, so a compromised container cannot load a kernel-module rootkit (see ). It is still one kernel shared by every pod on the node. For hostile or untrusted workloads, use the secure runtime class for VM-level isolation, or place them on dedicated node pools and apply the controls below.

Isolation levels#

Isolation levels available, from lightest to strongest:

Level Kernel Container root is Runs on
standard or runc (default) shared with host host root (UID 0) every node
standard or runc + user namespace shared with host an unprivileged host user every node
secure (Kata Containers) its own guest kernel isolated inside a VM bare metal only

User namespaces: root inside, unprivileged on the host#

Between the shared-kernel runtimes and the secure VM runtime, there is a lightweight isolation layer that works on every node: a user namespace.

By default, UID 0 (root) inside a container is the same as root on the node. If a container escape happens while the container runs as root, the attacker lands as root on the host.

A user namespace removes that problem. The pod sets hostUsers: false. A UID (numeric Linux user ID) is what identifies a user to the kernel. The node maps the container UIDs 0-65535 onto a different, higher range of host UIDs, and the kubelet picks a different range for each pod. Inside the container, the process still sees itself as root, so images that need root keep working. On the node, that process is an ordinary user with no host privileges. A container escape lands as a "nobody" user instead of host root.

User namespaces have very low overhead. They work on cloud VMs and bare metal alike. No special hardware is needed.

To use a user namespace, set hostUsers: false on the pod spec. Nothing needs to be installed or enabled at the cluster level. The kernel is built with CONFIG_USER_NS and idmapped-mount support. containerd 2.3 and crun 1.28 both support the idmapped mounts a user namespace needs. In Kubernetes 1.36, user namespace support is on by default with no feature gate to set.

Why it is opt-in, not a default. A user namespace cannot be used with hostNetwork: true, hostPID: true, or hostIPC: true. Platform components use host namespaces, so they cannot run in user namespaces. A cluster-wide default would break them. Turn user namespaces on per workload.

See .

Workload security controls#

Beyond the runtime, other controls shrink what a container can do. Two are on by default; the rest are yours to set. Each has its own page:

  • Seccomp RuntimeDefault is on by default, so a pod that names no profile gets the runtime's syscall filter instead of running unconfined. See .
  • The enforcing AppArmor profile confines every non-privileged container by default, with nothing to set. See .
  • Security contexts (non-root, dropped capabilities, no privilege escalation, read-only root) are yours to set per pod. See .
  • Pod Security Standards are not enforced by default; no cluster-wide level is set, so every namespace runs privileged until you label it. Roll them out audit-first. See .

Resource governance for multi-tenant clusters#

Isolation is not only about security. A single namespace can also starve the cluster by consuming all CPU, memory, or object count. Set these per namespace:

  • ResourceQuota. Caps total CPU, memory, storage, and object counts in the namespace.
  • LimitRange. Sets default and maximum per-container resource requests and limits.

Pair these with a default-deny NetworkPolicy and a Pod Security Standard level. Together they cover network segmentation, pod constraints, and resource caps for any multi-tenant namespace.

Related#

Encrypt pod traffic with WireGuard

Enable WireGuard transparent encryption in Cilium so cross-node pod traffic is encrypted at the kernel level with no changes to your applications or workloads.

Set security contexts

How to set pod and container security contexts to limit what running containers can do, covering non-root users, privilege escalation, capabilities, and seccomp.