How Syself Builds a Secure Kubernetes Node OS

Every Syself node runs Syself Linux, an operating system we spent about a year building for one job: running Kubernetes, and nothing else. It ships as a single sealed image, tested as a whole before any node boots, and once it's built it never changes. A node is replaced from a fresh image, never patched in place. We control every input that goes into that image, so you know exactly what runs on your nodes, down to the last package.

We've maintained cluster-api-provider-hetzner, the open-source Cluster API provider for Hetzner, since 2020, and we contribute upstream to the Kubernetes projects it runs on. In 2023 we won the public tender for the Kubernetes-as-a-Service part of the Sovereign Cloud Stack, a German government initiative, with the best concept. By then we ran everything around the node. The node OS underneath was the last piece we didn't build ourselves, so we built it.

You can see exactly what a node runs. Its full bill of materials is in the Version Hub, every package and the version we shipped.

What runs on a node, and how you check it

You can check two things about any node yourself: what's on it, and whether it still runs what we put there.

The platform provisions each node and confirms it booted the correct sealed image before the node joins the cluster. From then on, a health daemon on the node watches for tampering and reports what it finds to the Kubernetes API. So you can check any node's integrity state with kubectl, without logging in:

		$ kubectl get node <name> -o jsonpath='{range .status.conditions[?(@.type=="NodeTampered")]}{.status}{"\n"}{end}'
	

True means the daemon saw a watched file change since it took its baseline; a clean node reads False. If the OS integrity check itself ever fails on a running node, the platform raises a separate condition, and on a cloud pool it replaces that node automatically. The full walkthrough, with real conditions and output, is in Verify node integrity.

How the seal works

A Linux feature called dm-verity is responsible for the sealing. The image is split into fixed-size blocks. Each block is hashed, and those hashes combine upward into a single root hash fixed when the image is built. On every read, the kernel hashes the block it just read and checks it against the tree. A block that doesn't match fails the read, so no block is trusted past the moment it's touched. A changed layer, or one re-sealed under a new hash, is caught: it either fails the read or no longer matches the root hash the node was provisioned to enforce.

The image is sealed in parts, and the kernel checks each part on its own.

There's no signing key anywhere in this, no signature and no key file on the node. Authenticity comes from the content hash, not from a key someone holds. A swapped or edited layer hashes to a different value, and the mismatch is the alarm. That's what "keyless" means here: you don't have to protect a key, because there isn't one to steal. If you want to confirm the live seal on a node yourself, Verify node integrity walks each layer with real device names and output.

One image, tested then shipped

We don't rebuild the image when we release it. We build it once, test that exact image, and promote the same bytes to release. Nothing is recompiled between the version we tested and the version you run, so the two can't drift apart.

Plenty of projects claim you can rebuild their image from source and land on identical bytes, but in practice that's not always the case: the environment where its built influences the final artifact. In our case, there's only ever one image, and every node either runs it or is caught not running it.

What's in the auditable supply chain?

The seal proves a node runs the image we published. It doesn't tell you what's inside that image, and for that you don't have to take our word either. Every release ships an SBOM in SPDX 3.0.1 format: the full list of what's in the image and the version of each part. Feed it to grype or trivy, the common open-source scanners, and you get the same findings our own scan does, because you're scanning the same list.

Alongside the SBOM we publish machine-readable advisories that say whether a given CVE actually affects the image, in both OpenVEX and CSAF 2.0. Each release carries a stack security report, and a CVE feed we re-audit daily, so the picture stays current as new issues appear.

The details are in Supply chain security and Security evidence per release.

How does this support SOC 2, ISO 27001, and BSI C5?

We build in the technical controls those frameworks require, so a cluster comes with what they ask for. The node audit ruleset is written against BSI C5:2020, the cloud-security catalogue from the German Federal Office for Information Security. The SBOM format conforms to BSI TR-03183-2. And splitting the SBOM from the advisory feed matches what the EU Cyber Resilience Act asks for.

What we hand you is the evidence: a provable seal, a real bill of materials, and advisories that stand up to inspection. You operate the controls and make the case to your auditor.

Where a hardware root of trust would go

The biggest attack surface on most nodes builds up over time. You install an OS, run it for years, and it drifts: packages patched at different moments, config piling up, every change a window where something could slip in unaudited. Syself Linux takes that surface away. The image is written whole before a node boots, it configures nothing at startup, it's immutable, and it's never patched. A node that fails verification or shows tampering is replaced from a fresh image, not repaired.

The seal is software, not hardware. Only a small share of servers have a TPM module, so we don't build the root of trust on one. That leaves one attack open: rewriting the boot line to point the kernel at a different image. But it can't just be run. It means booting the server into Hetzner's rescue system, which takes access to the Hetzner account, and the platform watches the node the whole time, so it wouldn't pass unnoticed. The node runs under a system that provisions and vouches for it, so it never has to vouch for itself.

If your threat model demands a hardware root of trust, that's the line to weigh.

FAQ

How do I check what a Syself Linux node is running?

Read its bill of materials in the Version Hub to see every package and version in the image, and scan it with grype or trivy. To check that a specific node hasn't been tampered with, read its integrity conditions from the Kubernetes API with kubectl, no login to the node needed. Verify node integrity shows both.

What SBOM format do you ship?

SPDX 3.0.1. Feed it to grype or trivy and you get the same findings as our own scan, because it's the same bill of materials. We publish advisories in OpenVEX and CSAF 2.0 alongside it.

There's no signing key. What stops someone swapping the image?

The content hash. A swapped or edited layer hashes to a different value than the one the node was provisioned to enforce, and the kernel checks it on every read, so a tampered layer either fails to boot or fails the read. There's no key to steal, because authenticity comes from the content itself.

Can I rebuild the exact image from source?

No, and we don't rebuild it either. We build the image once, test that exact image, and ship the same bytes as the release, so what you run is what we tested.

We're transparent about what a node runs, and you can see it for yourself. The full bill of materials is on the Version Hub, and any node's integrity state is one kubectl call away. You don't have to trust us to know what a node runs.