The control plane reports the numbers that tell you whether the cluster's own machinery is healthy: how long etcd takes to commit a write, whether the scheduler's queue is draining, how many reconciles the controller-manager is behind. All of it is available, and none of it is on the network. These components bind their metrics ports to 127.0.0.1 on purpose, so the numbers stay on the node.

That is why a Prometheus pod cannot collect them: inside a pod, 127.0.0.1 is the pod. The System Alloy runs on each node's own network namespace and reaches them there, which is the supported path and the one the config in Collection already covers. This page covers what it exposes once it is running.

The endpoints#

The API server is the exception in this group: it serves its metrics over HTTPS with a bearer token on the kubernetes Service, so it is an ordinary authenticated target that needs no host network. The full catalog, including the data-plane and node daemons, is Reference: metrics per component .

What to watch#

Watch etcd first. It is the component that degrades cluster-wide responsiveness when it is slow, and it warns you before it fails. Rising fsync latency means the disk is struggling to keep up with commits. A database size climbing toward its quota ends in a read-only cluster. Frequent leader changes mean the members cannot keep a stable quorum, usually from disk or network latency between control planes.

The controller-manager and scheduler tell you about backlog, not health. Their work-queue depth staying high means reconciles are queuing faster than they finish, which is what a cluster looks like just before objects stop converging. A scheduler with pending pods and an empty queue is a capacity problem instead, and kube-state-metrics is where you see that.

The kubelet and cadvisor are the per-node and per-container view. The kubelet reports pod lifecycle timings, volume mounts, and its own runtime operations; cadvisor supplies per-container CPU, memory, and network. Together they answer "which container on this node is using what", which the metrics API deliberately does not keep (metrics-server and live usage ).

KubeGate reports on the gate in front of the API server, the allow and deny decisions it makes on TLS and source address. A climbing deny rate is either a misconfigured client or something worth investigating, and its audit log has the detail (Log sources on a sealed node ).

Node roles and loopback certificates#

Only control planes have most of this. etcd, the controller-manager, the scheduler, and KubeGate exist nowhere else, so a collector that looks for them on every node reports connection refused on workers forever. The System Alloy gates those jobs on the node's role for exactly this reason, and Troubleshoot the collectors covers the symptom if you meet it.

The HTTPS endpoints use certificates that cannot be verified from loopback. The controller-manager and scheduler serve self-signed certificates, and the kubelet's certificate names the node IP rather than 127.0.0.1. Verification fails on the address the collector has to use, so those scrapes skip it. Nothing is exposed by that: the connection is between two processes on the same machine and never reaches an interface.

Once the samples land, alert on them. Rising etcd fsync latency and sustained queue depth are in Platform alert rules , and the control-plane dashboards are in Dashboards for a Syself Autopilot cluster .