Monitor cluster and machine health

Your cluster's health has a layer above the nodes: the Cluster API objects that describe it. Syself runs the management cluster and the controllers that turn your Cluster object into Machines and Nodes, and that heal them when they fail. Watching the objects is how you catch a cluster that is not reaching the state you declared.

What the management cluster does for you

You declare a Cluster (and its pools) once. In the management cluster Syself runs , the controllers reconcile that declaration continuously: they create Machines, provision servers, join Nodes, and step in when a node stops being healthy. On a cloud node that means a reboot, and if the node does not come back, replacing it with a fresh one from a clean image. On a bare-metal server it means rebooting it, and re-provisioning the same physical host if the reboots do not help. See self-healing for the full loop.

The objects you watch

You have kubectl access to your objects in the management cluster, so you read their status directly:

		$ kubectl get clusters,machinedeployments,machines
	

• Cluster carries top-level conditions: whether the control plane is ready and the infrastructure is provisioned.
• MachineDeployment is a pool. Compare its ready replicas against its desired count; a gap that does not close means the pool cannot reach its target.
• Machine is one server on its way to becoming a Node. Its phase walks Pending, Provisioning, Provisioned, then Running. A Machine stuck in Provisioning is the signal behind most cases where a cluster does not come up (see Cluster creation fails ).

What to alert on

Watch for the states that mean your declared cluster and the real one have drifted apart:

• A Machine stuck in Provisioning past the normal few minutes: capacity, credentials, or a bad manifest.
• A MachineDeployment not reaching its replicas: it cannot get the servers it asked for (common on bare metal when no free host matches, or on cloud during a capacity shortage).
• Repeated remediation on one pool: Machines being rebooted or replaced in a loop points at a persistent fault, a bad image reference, or on bare metal a hardware fault a reboot does not clear.

To alert on these, run a kube-state-metrics custom-resource-state configuration, or a small exporter, that reads the Cluster API objects from the management cluster and turns their conditions into metrics you can put a PrometheusRule on.

That exporter runs as a pod, so it needs credentials it can read on startup. The management cluster's default login is OIDC through a browser, so give the exporter a Service Account kubeconfig from Headless access instead. Machine and Cluster events are worth watching too, since they narrate provisioning and remediation as it happens.

Where the boundary sits

Syself watches the management cluster's own health. You watch whether your declared state (your pools, replicas, and versions) is actually met. When it is not, and it is not a capacity or manifest issue you can fix, open a support case. To run this watch across many clusters from one place, see Multi-cluster observability .