Maintaining a cluster means checking regularly that it is healthy. A healthy cluster keeps your workloads running without problems. Run the checks below manually, or set up alerts for them in your observability stack. ## Prerequisites - [`kubectl`](/docs/hetzner/apalla/platform/setup/prerequisites) access to the workload cluster. All commands on this page run against it. ## Check the control plane Every control plane node should be `Ready`: ```console $ kubectl get nodes --selector='node-role.kubernetes.io/control-plane' ``` The control plane runs the Kubernetes API. When it has a problem, the whole cluster is affected. If a control plane node is `NotReady`, see [Control-plane and etcd recovery](/docs/hetzner/apalla/clusters/troubleshooting/control-plane-and-etcd-recovery). ## Check that all nodes are Ready Every node in the cluster should be `Ready`: ```console $ kubectl get nodes ``` Kubernetes stops placing new pods on a `NotReady` node. To find out why a node is `NotReady`, see [Debug a node](/docs/hetzner/apalla/servers-and-nodes/access/debug-a-node). ## Check for pending certificate requests A new node asks the cluster for certificates when it joins. Syself Autopilot approves these requests on its own. A request that stays `Pending` can slow down a node or stop it from joining. ```console $ kubectl get csr ``` Look for requests that have been `Pending` for more than a few minutes. Approve the legitimate ones. Find out why the others are stuck. ## Check that your workloads can handle a node replacement Nodes are replaced from time to time, for example when Syself Autopilot heals a broken node. Your pods then move to other nodes. Make sure your workloads stay available while that happens. See [Keep workloads available](/docs/hetzner/apalla/workloads/availability/keep-workloads-available). ## Back up your cluster regularly It is good practice to back up your cluster regularly. A backup lets you rebuild your cluster if something goes wrong. Back up two things: - **Your manifests.** Commit your cluster and application manifests to Git. Make sure the repository is up to date. - **Your data.** Back up your PersistentVolumeClaims and anything you applied by hand with [Velero](/docs/hetzner/apalla/storage/backup/back-up-with-velero). ## Related - [Update within a version](/docs/hetzner/apalla/clusters/upgrades/update-within-a-version), patch and stack version bumps - [Upgrade to a new Kubernetes version](/docs/hetzner/apalla/clusters/upgrades/upgrade-to-a-new-kubernetes-version), moving to the next minor version - [Cluster upgrades](/docs/hetzner/apalla/concepts/operations/cluster-upgrades), rollout mechanics, surge rules and drain behavior - [Self-healing and node replacement](/docs/hetzner/apalla/concepts/operations/self-healing-and-node-replacement), how Syself detects and replaces unhealthy nodes - [Bare metal and cloud](/docs/hetzner/apalla/concepts/internals/bare-metal-and-cloud), the capacity and surge differences between backends