Every node reports its own health as **conditions**, the status flags Kubernetes keeps on a `Node`. On top of the standard Kubernetes conditions, Syself Autopilot adds its own, so trouble shows up as a clear signal before a node fails outright. You get early warning, automatic recovery where it is safe, and one clean place to hook your own alerting. ## What conditions give you - **Early warning.** A node tells you it is degrading, a disk wearing out, a certificate not renewing, a service struggling, before it drops out of the cluster. - **Automatic recovery.** For the problems a restart or a replacement can fix, Syself Autopilot acts on its own. You do not wire anything up for that. - **One place to alert from.** The conditions that actually need a person are the ones you route to your monitoring, and there are only a handful. ## What the conditions cover Beyond the standard `Ready`, `MemoryPressure`, `DiskPressure`, and `PIDPressure` that every Kubernetes node reports, Syself Autopilot adds health signals for: - the sealed OS and kernel - the kubelet and container runtime - core node services and networking - disks: wear, temperature, and failures - certificates - GPUs - hardware faults and node integrity `Ready` is still the signal that drives replacement. The rest give you and the platform a head start. The full catalog, with every signal and what it means, is in the [Node health conditions reference](/docs/hetzner/apalla/reference/node-health-conditions). ## What happens when one fires Each signal has a fixed outcome, so the behavior is predictable: - **Fixed in place.** For a service that has stumbled, Syself Autopilot restarts it on the node. No machine is touched. - **Reboot, then replace.** For a problem that means the node is unlikely to recover, a cloud node is rebooted and, if that does not help, replaced. - **Reported for a human.** Integrity and hardware signals are report-only. They tell you something needs a look; the platform does not act on its own. Bare metal is more conservative: a dedicated server is not reprovisioned for a fault a reboot would not fix, or for one that could hit many servers at once, so those signals alert a person instead of churning hardware. For exactly what gets replaced and when, see [Machine health checks and remediation](/docs/hetzner/apalla/servers-and-nodes/maintenance/machine-health-checks-and-remediation). ## See a node's conditions List a node's conditions and the short reason on each: ```console $ kubectl get node -o jsonpath='{range .status.conditions[*]}{.type}={.status} {.reason}{"\n"}{end}' ``` Syself Autopilot also writes a fuller health report onto the node, so you can read the detail behind a signal without connecting to it. See [Node labels and annotations](/docs/hetzner/apalla/reference/node-labels-and-annotations) for how to read it. > [!NOTE] > Watch that the report keeps updating. If it goes stale while `Ready` still looks fine, the health reporting itself has stopped, which is worth an alert of its own. ## Alert on the ones that need you Most signals either self-heal or trigger a replacement, and you can leave those to the platform. Route the rest, the certificate, disk-failure, hardware, and integrity signals, to your own monitoring, because they do not fix themselves and a person has to act. Syself Autopilot exposes node health as metrics you can pull into your existing stack; see [Scrape control-plane metrics](/docs/hetzner/apalla/observability/metrics/control-plane-metrics). For the security and integrity signals, forward the records to your SIEM: see [Ship audit logs to a SIEM](/docs/hetzner/apalla/security/ship-audit-logs-to-a-siem). At a minimum, alert on two things: `Ready` going bad (the node is gone) and the health report going stale (the reporting is gone). When an integrity signal fires, treat the node as untrusted, reprovision it, and follow [Respond to a tampered node](/docs/hetzner/apalla/security/respond-to-a-tampered-node). For a failing certificate, see [Certificate expiry and rotation](/docs/hetzner/apalla/servers-and-nodes/access/certificate-expiry-and-rotation).