A Syself Autopilot node runs [Syself Linux](/docs/hetzner/apalla/concepts/internals/syself-linux), a sealed OS: the root filesystem is read-only, and only `/var` is writable. Every log stream lives on `/var`, which means every log stream disappears when the node is reprovisioned. Knowing what is there, and that it is temporary, is the reason the rest of this sub-topic ships logs off-node. ## The streams | Source | Path | Nodes | What it records | | ------------------------------- | -------------------------------------------- | -------------- | --------------------------------------------------------------------------------------------------------- | | Container logs | `/var/log/pods/` | all | stdout and stderr from every container | | journald (the systemd log) | `/var/log/journal/` | all | systemd service output (kubelet, containerd, the node daemons) | | auditd (the Linux OS audit log) | `/var/log/audit/audit.log` | all | OS-level events: file and identity changes, privilege use, module loads, every command in a login session | | Kubernetes API audit | `/var/log/kube-apiserver/kube-apiserver.log` | control planes | who did what to which Kubernetes object | | KubeGate audit | `/var/log/kubegate/audit.log` | control planes | the allow and deny decisions the API server's front gate made (TLS and source-address checks) | The first three are on every node. The two API-side audit streams exist only on control-plane nodes, because that is where the API server and its front gate run. Each API server logs only the requests it handled, so on a multi-control-plane cluster the full API trail is the union of the per-node files. > [!WARNING] > Everything in `/var` is lost when a node is replaced, and the platform replaces nodes rather than repairing them. The on-node copies rotate and are a bounded hot buffer, not a system of record. Ship what you need off-node before a node goes. See [Collect container and journald logs](/docs/hetzner/apalla/observability/logs/collect-container-and-journald-logs) and [Ship audit logs off-node](/docs/hetzner/apalla/observability/logs/ship-audit-logs). ## Why you cannot install a log agent in the OS The root filesystem is read-only and there is no package manager, so you cannot install a collector into the OS the way you would on a normal server. That is not a limitation to work around; it is why every collector here runs as a Kubernetes DaemonSet, a pod that runs on every node. The DaemonSet mounts the host log paths and streams them out. The node stays sealed, and the shipping is a workload you own and upgrade like any other. For the on-node size caps and how to size retention in your own store, see [Log retention and sizing](/docs/hetzner/apalla/observability/logs/log-retention-and-sizing).