On a platform that drains and replaces nodes, one upgrade can fire dozens of alerts at once, none of which need a human. Grouping collapses related alerts into one notification, and silences mute expected noise during maintenance. Get both right, or the on-call rotation learns to ignore the pager.

## Group related alerts

Alertmanager groups alerts that share the labels you list in `group_by`, and sends one notification per group instead of one per alert:

```yaml
route:
  group_by: ["alertname", "cluster", "namespace"]
  group_wait: 30s # wait this long to collect the first batch of a new group
  group_interval: 5m # wait this long before sending an update for a group
  repeat_interval: 4h # re-notify an unresolved group this often
```

`group_wait` gives a burst a moment to gather, so a whole pool going `NotReady` arrives as one notification, not twenty. `group_interval` paces updates to a group, and `repeat_interval` re-pages if it is still firing.

## Silence maintenance

A node replacement or a cluster upgrade rolls nodes on purpose, and the alerts that follow (a node briefly `NotReady`, a pod rescheduling) are expected. Silence them for the window rather than paging on your own maintenance. A silence matches alert labels for a set duration, and `amtool`, the CLI that ships with Alertmanager (see [Test each route](/docs/hetzner/apalla/observability/alerting/alert-routing-and-receivers#test-each-route)), adds one:

```console
$ amtool silence add \
  --alertmanager.url=http://localhost:9093 \
  --duration=2h --comment="cluster upgrade, ticket OPS-1234" \
  alertname=~"KubeNodeNotReady|KubePodNotReady" cluster="prod-eu"
```

Scope the silence tightly (the specific alerts and the specific cluster) so you do not blind yourself to a real problem elsewhere while maintenance runs. Set the duration to a little longer than the planned window, and let it expire on its own.

> [!TIP]
> Before a planned [node replacement](/docs/hetzner/apalla/concepts/operations/self-healing-and-node-replacement) or [cluster upgrade](/docs/hetzner/apalla/clusters/upgrades/upgrade-to-a-new-kubernetes-version), add the silence, do the work, then confirm it expired. `amtool` makes this scriptable, so it can be a step in your maintenance runbook.

## Inhibition

Inhibition suppresses one alert while another is firing, which cuts the cascade where one root cause triggers ten symptoms. Do not page for "pod not ready" and "service has no endpoints" on a node whose "node down" alert is already firing:

```yaml
inhibit_rules:
  - source_matchers: ["alertname = KubeNodeNotReady"]
    target_matchers: ["severity = warning"]
    equal: ["node"]
```

## Audit who silenced what

A silence hides alerts, so it is worth an audit trail. Require a `--comment` with a ticket reference on every silence (as above), and periodically list active silences (`amtool silence query`) so a forgotten silence does not quietly hide a real alert for weeks.