node-exporter gives you the per-node hardware and OS metrics: CPU, memory, filesystem, disk, and network. You install it yourself, as a DaemonSet on the host network, and on a Syself Autopilot cluster its metrics stay on the node, so a `ServiceMonitor` does not reach it. Scrape it over loopback instead.

## Install it

The chart's defaults are already what this platform needs: `hostNetwork: true`, a `NoSchedule` toleration so it lands on control-plane nodes too, and no `ServiceMonitor`. There is no values file:

```console
$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
$ helm install node-exporter prometheus-community/prometheus-node-exporter \
  --namespace monitoring \
  --version <chart-version>
```

Pin `--version` to a known chart release so an update does not change behavior under you. Confirm you get one pod per node, control planes included:

```console
$ kubectl -n monitoring get ds node-exporter-prometheus-node-exporter
NAME                                     DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
node-exporter-prometheus-node-exporter   4         4         4       4            4           kubernetes.io/os=linux   1m
```

A node missing from that count is a node whose hardware you are not watching.

## Why the loopback scraper

node-exporter's DaemonSet listens on the node, and Syself Autopilot cluster nodes carry public IPs, so the host firewall does not let an arbitrary Prometheus pod reach it. That is why its `ServiceMonitor` stays off, which is the chart's default, and the [System Alloy](/docs/hetzner/apalla/observability/collection/deploy-the-system-alloy) scrapes `127.0.0.1:9100` instead. Because node-exporter is your install rather than something the platform ships, that job is the one block in the agent's config to delete if you skip this page. Confirm it is listening on a node before chasing an error:

```console
$ ss -tlnp | grep 9100
```

## What you get

The usual node-exporter series, per node, for both cloud and bare-metal pools:

- **CPU** by mode, **memory** used and available, **load average**.
- **Filesystem** free space and inodes per mount, which is how you catch a `/var` filling with logs before it becomes a problem.
- **Disk I/O** per device, including NVMe, useful under a TopoLVM local-storage workload where disk pressure on one node hurts the pods pinned to it.
- **Network** bytes, errors, and drops per interface.
- On bare metal, **hardware sensor** readings where the board exposes them, such as component temperatures (node-exporter's `hwmon` collector).

Label the series by pool (for example a `pool` relabel from the node's pool label) so you can compare a bare-metal database pool against a cloud burst pool in one view.

## What the health daemon already covers

node-exporter is for graphs and trends. The node health daemon does something different: it **detects** the disk and hardware problems that matter. It sets conditions such as `DisksFailure`, `DiskWearHigh`, `DiskTemperatureHigh`, and `DiskUsageHigh` on the Node, and it records a disk inventory, including wear level, in the `autopilot.syself.com/disks` node annotation, so you get that hardware picture without setting anything up.

A disk condition is for you to alert on, not something the platform fixes on its own. The daemon does not reboot or replace a node over a failing disk, on cloud or bare metal, because neither would fix the hardware. Use node-exporter to watch trends and capacity over time, and alert on the daemon's disk conditions (see [Platform alert rules](/docs/hetzner/apalla/observability/alerting/platform-alert-rules)) to catch a failing disk on any pool.

> [!NOTE]
> For GPU hardware health, the node health daemon sets `GpuActivationFailed` and `GpuFallenOffBus` conditions on the Node object, which you can alert on via `kube-state-metrics`. For detailed GPU compute utilization, memory, power, and temperature metrics, install NVIDIA's DCGM exporter as shown below.

## GPU metrics with the NVIDIA DCGM exporter

On GPU worker nodes, NVIDIA's Data Center GPU Manager (DCGM) exporter reports GPU utilization, VRAM use, power, and temperature. Install its Helm chart pinned to the GPU pool, and give the pod one GPU so the exporter can read the hardware:

```console
$ helm repo add gpu-helm-charts https://nvidia.github.io/dcgm-exporter/helm-charts
$ helm upgrade -i nvidia-dcgm-exporter gpu-helm-charts/dcgm-exporter \
  --namespace monitoring \
  --set serviceMonitor.enabled=false \
  --set-string nodeSelector."autopilot\.syself\.com/gpu"="true" \
  --set "resources.limits.nvidia\.com/gpu=1"
```

The exporter runs as a DaemonSet on the GPU nodes and serves metrics on port `9400` (service port name `metrics`) on the pod network, so it is scraped like an application rather than like a platform component: declare a `ServiceMonitor` for it and the [Application Alloy](/docs/hetzner/apalla/observability/collection/deploy-the-application-alloy) picks it up ([Custom application metrics](/docs/hetzner/apalla/observability/metrics/custom-application-metrics)). Requesting `nvidia.com/gpu=1` triggers containerd's Container Device Interface hook to mount the `/dev/nvidia*` devices and driver libraries into the pod. Because Syself Autopilot enables GPU time-slicing by default, the exporter takes one virtual time-sliced slot and your workloads keep running on the same physical GPU alongside it.

The core series to graph are `DCGM_FI_DEV_GPU_UTIL` (compute utilization), `DCGM_FI_DEV_FB_USED` (VRAM used, in MB), and `DCGM_FI_DEV_GPU_TEMP` (temperature).