Node problem detection and conditions
Every node reports its own health as conditions, the status flags Kubernetes keeps on a Node. On top of the standard Kubernetes conditions, Syself Autopilot adds its own, so trouble shows up as a clear signal before a node fails outright. You get early warning, automatic recovery where it is safe, and one clean place to hook your own alerting.
What conditions give you
- Early warning. A node tells you it is degrading, a disk wearing out, a certificate not renewing, a service struggling, before it drops out of the cluster.
- Automatic recovery. For the problems a restart or a replacement can fix, Syself Autopilot acts on its own. You do not wire anything up for that.
- One place to alert from. The conditions that actually need a person are the ones you route to your monitoring.
What the conditions cover
Beyond the standard Ready, MemoryPressure, DiskPressure, and PIDPressure that every Kubernetes node reports, Syself Autopilot adds health signals for:
- the sealed OS and kernel
- the kubelet and container runtime
- core node services and networking
- disks: wear, temperature, and failures
- certificates
- GPUs
- hardware faults and node integrity
Ready drives replacement on every pool, and on cloud pools a few other conditions do too. The rest give you and the platform a head start. The full catalog, with every signal and what it means, is in the Node health conditions reference .
What happens when one fires
Each signal has a fixed outcome, so the behavior is predictable:
- Fixed in place. For a service that has stumbled, Syself Autopilot restarts it on the node. No machine is touched.
- Reboot, then replace. For a problem that means the node is unlikely to recover, a cloud node is rebooted and, if that does not help, replaced.
- Reported for a human. Hardware signals and
SealedOSTampered/NodeTamperedare report-only on every pool. On bare metal, every signal exceptReadyis report-only. They tell you something needs a look and the platform does not act on its own.
Bare metal is more conservative: a dedicated server is not reprovisioned for a fault a reboot would not fix, or for one that could hit many servers at once, so those signals alert a person instead of churning hardware. For exactly what gets replaced and when, see Machine health checks and remediation .
See a node's conditions
Node health conditions shows how to list a node's conditions, read the health report behind each one, and notice when that report stops updating.
Alert on the ones that need you
Set up alerts for certificate, disk, hardware, and integrity issues that require your attention. The Platform alert rules ship rules for the node integrity conditions and show how to add one for Ready.
When an integrity signal fires, treat the node as untrusted, reprovision it, and follow Respond to a tampered node . For a failing certificate, see Certificate expiry and rotation .
Related
Machine health checks and remediation
How Syself Autopilot detects an unhealthy node, what each condition triggers, the exact windows before a reboot or a replacement on cloud and bare-metal pools, how to pause remediation for planned maintenance, and how to turn off health checks for a pool.
Certificate expiry and rotation
Fix a stuck CertRenewalFailing condition or an expired certificate that's breaking connections.