Kyverno enforces cluster-wide policy at admission: it changes a resource before Kubernetes saves it, creates resources when a trigger fires, and blocks resources that break your rules. It is an admission controller you install on your workload cluster. Choose each webhook's `failurePolicy` deliberately: the platform permits both `Fail` (a hard gate that also blocks when Kyverno is down) and `Ignore` (best-effort, so a Kyverno outage admits the request). The platform's webhook-guard makes that choice safe, because no tenant webhook can lock out the control plane whatever failure policy it uses.

> [!NOTE]
> Read [Control admission](/docs/hetzner/apalla/security/control-admission) first. It explains the webhook-guard contract and when a native ValidatingAdmissionPolicy (VAP) is a better choice than Kyverno.

## Prerequisites

- `kubectl` access to the workload cluster
- `helm` installed

## The platform constraint

The platform ships an admission-policy cluster component called `webhook-guard`. It installs a ValidatingAdmissionPolicy named `syself-restrict-tenant-webhooks` that runs inside the API server and checks every `ValidatingWebhookConfiguration` or `MutatingWebhookConfiguration` you apply. It does not check `failurePolicy`. Instead it blocks a webhook from targeting the admission-registration API (`admissionregistration.k8s.io`), the authentication or authorization APIs, or cluster-scoped RBAC (role-based access control) objects, and requires any webhook touching roles, role bindings, or service accounts to be namespace-scoped. Kyverno's own webhooks do not target any of those, so this is not normally a problem. If `kubectl apply` fails with a `syself-restrict-tenant-webhooks` error, it is because of one of these restrictions, not because of `failurePolicy`.

The guard does not touch `failurePolicy`, so that choice is yours. Because it already stops any tenant webhook from intercepting authentication, admission registration, or cluster RBAC, and it exempts the platform's own admin identities, an admin can always recover a cluster whose Kyverno webhook is misbehaving, whatever failure policy it uses. The tradeoff is availability, not lockout:

- `failurePolicy: Ignore`: if the Kyverno pod is down or unreachable, the API server skips the webhook and admits the request. The rule is best-effort.
- `failurePolicy: Fail`: the request is blocked when the rule fails and also when Kyverno is down. This is a hard gate, but during a Kyverno outage a broadly-matching `Fail` webhook can block normal work in your cluster, both your own workloads and the platform's reconciliation of it.

Prefer `Ignore` for webhooks that match broadly, so a Kyverno outage never stalls the cluster. Reserve `Fail` for tightly-scoped policies in your own namespaces, where blocking during an outage is acceptable. For a hard gate that must survive a Kyverno outage entirely, write a native VAP (ValidatingAdmissionPolicy) instead: it runs inside the API server, so there is no pod to take down. See [Control admission](/docs/hetzner/apalla/security/control-admission).

## Step 1: install Kyverno

Use the official Kyverno Helm chart. Run multiple replicas, and set the chart's webhook failure-policy value to `Ignore`. The key name differs by chart version, so find it first with `helm show values kyverno/kyverno | grep -i failurepolicy` and add it as a `--set` flag to the command below.

```console
$ helm repo add kyverno https://kyverno.github.io/kyverno/
$ helm repo update
$ helm install kyverno kyverno/kyverno \
  --namespace kyverno \
  --create-namespace \
  --set admissionController.replicas=3 \
  --set backgroundController.replicas=2 \
  --set cleanupController.replicas=2 \
  --set reportsController.replicas=2
```

> [!WARNING]
> After installing, verify that every Kyverno webhook configuration uses `failurePolicy: Ignore` (Step 4). The failure policy key differs between Kyverno chart versions, so before installing, run `helm show values kyverno/kyverno | grep -i failurepolicy` to find the key for your version, and set it to `Ignore`.

## Step 2: choose failurePolicy on each policy

Set `spec.webhookConfiguration.failurePolicy` on every Kyverno policy so the choice is explicit rather than inherited from a global Helm value that can change later:

```yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: my-policy
spec:
  webhookConfiguration:
    failurePolicy: Ignore # Ignore for a broad webhook so a Kyverno outage never blocks requests
  rules:
    - name: my-rule
      # ...
```

`validationFailureAction` and `failurePolicy` are two different fields, and they control two different actors:

- `validationFailureAction: Enforce`: Kyverno is up and finds a violation, so it blocks the request.
- `failurePolicy: Ignore`: Kyverno is down, so the API server admits the request instead.

You can set both:

```yaml
spec:
  validationFailureAction: Enforce # block objects that violate the rule when Kyverno is up
  webhookConfiguration:
    failurePolicy: Ignore # admit anyway if Kyverno is down
```

## Which rule types to use

### Mutation

Mutation rules change a resource before Kubernetes saves it: add resource limits, inject labels, set a default security context. These are a good fit for Kyverno. If the mutation does not fire because Kyverno is down, the resource is saved without the change. That is usually safer than blocking the request. Run Kyverno highly available so mutations are reliable.

Example: add a `cost-center` label to every pod in a namespace if it is missing.

```yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: add-cost-center-label
spec:
  webhookConfiguration:
    failurePolicy: Ignore
  rules:
    - name: add-label
      match:
        any:
          - resources:
              kinds: [Pod]
              namespaces: [team-a]
      mutate:
        patchStrategicMerge:
          metadata:
            labels:
              +(cost-center): "team-a"
```

### Generation

Generation rules create resources automatically when a trigger fires: for example, create a default NetworkPolicy when a namespace is created. If the webhook is down when the trigger fires, the generated resource may be missing. That is a gap, not a security hole, so generation rules are a reasonable fit. After applying a generation policy, check that the generated resources exist.

### Validation

Validation rules check a resource and block it when a rule fails (with `validationFailureAction: Enforce`). Whether the rule still blocks during a Kyverno outage depends on the `failurePolicy` you chose: `Ignore` admits the request, `Fail` keeps blocking. Match that to how strict the rule needs to be.

Use Kyverno validation when the rule is a safety net (not the only enforcement line), the policy logic is too complex for CEL (Common Expression Language, the language native VAPs use), or you pair an enforcing policy with a highly available Kyverno deployment.

For a rule that must always block, even during a Kyverno outage, write a native ValidatingAdmissionPolicy: it runs inside the API server and cannot be taken offline. Do not name it `syself-*`; that prefix is reserved by the platform.

### Image verification

Kyverno's `verifyImages` rule checks container image signatures using cosign (a tool for signing and verifying container images). It must be a webhook because it makes network calls and cryptographic checks that a native VAP cannot do. That network dependency makes `Fail` risky here, since a registry or Kyverno hiccup would block deployments, so it usually runs `failurePolicy: Ignore` and best-effort. Run Kyverno highly available so this check rarely misses. For the full setup, see [Verify image signatures](/docs/hetzner/apalla/security/verify-image-signatures).

## Step 3: run Kyverno highly available

Run Kyverno highly available whichever failure policy you pick. With `Ignore`, a Kyverno outage silently skips those policies; with `Fail`, it blocks the requests they match. Three replicas for the admission controller is the minimum. Add a PodDisruptionBudget (a rule that tells Kubernetes to keep a minimum number of replicas running during node drains) to protect against disruption:

```yaml
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: kyverno-admission-controller
  namespace: kyverno
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app.kubernetes.io/component: admission-controller
      app.kubernetes.io/instance: kyverno
```

Spread the admission-controller pods across nodes with `podAntiAffinity` or a `topologySpreadConstraint` in your Helm values so a single node failure does not take down all replicas.

## Step 4: verify the setup

Check the `failurePolicy` on each Kyverno webhook configuration matches what you intended (`Ignore` for broadly-matching webhooks):

```console
$ kubectl get validatingwebhookconfigurations -o json \
    | jq '.items[] | select(.metadata.name | startswith("kyverno"))
          | .webhooks[] | {name: .name, failurePolicy: .failurePolicy}'
$ kubectl get mutatingwebhookconfigurations -o json \
    | jq '.items[] | select(.metadata.name | startswith("kyverno"))
          | .webhooks[] | {name: .name, failurePolicy: .failurePolicy}'
```

Confirm each entry shows the `failurePolicy` you intended, `Ignore` on the broadly-matching webhooks. If a webhook configuration is missing, `webhook-guard` rejected it, most likely because it targets the admission-registration, authentication/authorization, or cluster-scoped RBAC APIs described above. The `Forbidden` error returned by the rejected `kubectl apply` names the rule that fired. To read those rules, describe the policy rather than the binding. Its `validations[].message` entries state each restriction:

```console
$ kubectl describe validatingadmissionpolicy syself-restrict-tenant-webhooks
```

Check that the admission controller has at least two ready replicas:

```console
$ kubectl -n kyverno get pods
```

## Related

- [Control admission](/docs/hetzner/apalla/security/control-admission): the webhook-guard contract and how to choose between Kyverno and a native VAP.
- [OPA Gatekeeper as an alternative](/docs/hetzner/apalla/security/opa-gatekeeper-alternative): the same webhook-guard contract with Rego instead of YAML policies.
- [Verify image signatures](/docs/hetzner/apalla/security/verify-image-signatures): using Kyverno's `verifyImages` rule for image signature checks.
- [Segment with network policies](/docs/hetzner/apalla/security/segment-with-network-policies): adding network controls alongside Kyverno policies.
- [Security architecture](/docs/hetzner/apalla/security/security-architecture): where Kyverno fits in the platform security model.