Grafana is where you view your metrics and logs as graphs and dashboards. It is its own chart, and it reads the store you set up in [Set up Prometheus](/docs/hetzner/apalla/observability/metrics/set-up-prometheus) over the cluster network. Declare that connection in the values rather than clicking it in later, so a reinstalled Grafana comes back knowing where its data is. Install it ```yaml title="grafana-values.yaml" # A data source is a backend Grafana reads from to draw its graphs. Declared # here, they survive a reinstall; added in the UI, they do not. datasources: datasources.yaml: apiVersion: 1 datasources: - name: Prometheus type: prometheus access: proxy url: http://prometheus-operated.monitoring.svc.cluster.local:9090 isDefault: true # Load dashboards from labelled ConfigMaps, and let each one pick its folder. sidecar: dashboards: enabled: true label: grafana_dashboard folderAnnotation: grafana_folder provider: foldersFromFilesStructure: true # standard volumes provision only on cloud nodes, so keep Grafana there. # Change this if you point storageClassName at a local class instead. nodeSelector: autopilot.syself.com/machine-type: hcloud persistence: enabled: true # standard = Hetzner Cloud volumes, cloud nodes only. The nodeSelector above # keeps this on a cloud node. To run on a bare-metal node instead, drop it and # name the local class matching that node's disks (local-nvme, local-ssd, or # local-hdd through TopoLVM). storageClassName: standard size: 10Gi ``` ```console $ helm repo add grafana https://grafana.github.io/helm-charts $ helm install grafana grafana/grafana \ --namespace monitoring \ --version \ --values grafana-values.yaml ``` Pin `--version` to a known chart release so an update does not change behavior under you. > [!NOTE] > The `nodeSelector` is what keeps the `standard` claim provisionable. On a cluster with bare-metal nodes, a Grafana scheduled onto one leaves its PVC `Pending` with `no topology key found` and the pod never starts. If that has already happened, adding the selector is not enough on its own: the claim keeps a `volume.kubernetes.io/selected-node` annotation pointing at that node, so delete the PVC and `helm upgrade` to have the chart recreate it. To run Grafana on a bare-metal node instead, name the local class matching that node's disks. Reach it and sign in For a single operator, use a port-forward and read the admin password from the secret the chart created. A port-forward opens a private connection from your machine to the service, so Grafana stays off the network: ```console $ kubectl -n monitoring port-forward svc/grafana 3000:80 $ kubectl -n monitoring get secret grafana \ -o jsonpath='{.data.admin-password}' | base64 -d && echo ``` Then open `http://localhost:3000`. Under Connections, the Prometheus data source should already be there; **Save & test** on it reports `Successfully queried the Prometheus API`. Add the other data sources as you install them Add each backend to the same `datasources` list so metrics, logs, and traces share one query surface: - **Loki** (`http://loki-gateway.monitoring.svc.cluster.local`) for logs, once you [set up Loki](/docs/hetzner/apalla/observability/logs/set-up-loki). - **Tempo** for traces, once you [install it](/docs/hetzner/apalla/observability/traces/install-tempo-or-jaeger). ```yaml title="grafana-values.yaml" datasources: datasources.yaml: apiVersion: 1 datasources: - name: Loki type: loki access: proxy url: http://loki-gateway.monitoring.svc.cluster.local ``` Log a team in with OIDC For anything beyond one operator, connect Grafana to your identity provider so access follows your existing accounts and groups. OIDC is a standard login method that lets people sign in with those accounts instead of a shared password. Configure your IdP settings under the `grafana.ini.auth.generic_oauth` key in your Helm values. To avoid handing out the built-in admin password, use the `role_attribute_path` setting to automatically map an IdP group (like `grafana-admins`) to the Grafana `Admin` role. For the full list of configuration options, see the [Grafana Generic OAuth documentation](https://grafana.com/docs/grafana/latest/setup-grafana/configure-access/configure-authentication/generic-oauth/). > [!WARNING] > Do not expose Grafana on a `type: LoadBalancer` Service with only the default admin login. It shows your whole cluster's telemetry. For a team, put it behind an ingress with the OIDC login above. The `persistence` block above keeps the dashboards you build by hand, its users, and its settings across a pod move. It is still better to keep dashboards in Git, so the pod's state does not matter: [Dashboards as code](/docs/hetzner/apalla/observability/dashboards/dashboards-as-code). Next, load dashboards that match a Syself Autopilot cluster's real components: [Dashboards for a Syself Autopilot cluster](/docs/hetzner/apalla/observability/dashboards/dashboards-for-a-syself-cluster).