Skip to main content

Log sources on a sealed node

A Syself Autopilot node runs , a sealed OS: the root filesystem is read-only, and only /var is writable. Every log stream lives on /var, which means every log stream disappears when the node is reprovisioned. The next pages cover shipping these logs off the node.

The streams

Source Path Nodes What it records
Container logs /var/log/pods/ all stdout and stderr from every container
journald (the systemd log) /var/log/journal/ all systemd service output (kubelet, containerd, the node daemons)
auditd (the Linux OS audit log) /var/log/audit/audit.log all OS-level events: file and identity changes, privilege use, module loads, every command in a login session
Kubernetes API audit /var/log/kube-apiserver/kube-apiserver.log control planes who did what to which Kubernetes object
KubeGate audit /var/log/kubegate/audit.log control planes the allow and deny decisions the API server's front gate made (TLS and source-address checks)

The first three are on every node. The two API-side audit streams exist only on control-plane nodes, because that is where the API server and its front gate run. Each API server logs only the requests it handled, so on a multi-control-plane cluster the full API trail is the union of the per-node files.

Warning

Everything in /var is lost when a node is replaced, and Syself Autopilot replaces nodes instead of repairing them. Logs are capped in size and rotate on a running node, so older entries get deleted long before a replacement. Ship logs off-node continuously if you need to keep them. See and .

Why you cannot install a log agent in the OS

The root filesystem is read-only and there is no package manager, so you cannot install a collector into the OS the way you would on a normal server. That is why every collector here runs as a Kubernetes DaemonSet, a pod that runs on every node. The DaemonSet mounts the host log paths and streams them out. The node stays sealed, and the shipping is a workload you own and upgrade like any other.

For the on-node size caps and how to size retention in your own store, see .