Use Kyverno policies

Kyverno enforces cluster-wide policy at admission: it changes a resource before Kubernetes saves it, creates resources when a trigger fires, and blocks resources that break your rules. It is an admission controller you install on your workload cluster. Choose each webhook's failurePolicy deliberately: the platform permits both Fail (a hard gate that also blocks when Kyverno is down) and Ignore (best-effort, so a Kyverno outage admits the request). The platform's webhook-guard makes that choice safe, because no tenant webhook can lock out the control plane whatever failure policy it uses.

Prerequisites#

• kubectl access to the workload cluster
• helm installed

The platform constraint#

The platform ships an admission-policy cluster component called webhook-guard. It installs a ValidatingAdmissionPolicy named syself-restrict-tenant-webhooks that runs inside the API server and checks every ValidatingWebhookConfiguration or MutatingWebhookConfiguration you apply. It does not check failurePolicy. Instead it blocks a webhook from targeting the admission-registration API (admissionregistration.k8s.io), the authentication or authorization APIs, or cluster-scoped RBAC (role-based access control) objects, and requires any webhook touching roles, role bindings, or service accounts to be namespace-scoped. Kyverno's own webhooks do not target any of those, so this is not normally a problem. If kubectl apply fails with a syself-restrict-tenant-webhooks error, it is because of one of these restrictions, not because of failurePolicy.

The guard does not touch failurePolicy, so that choice is yours. Because it already stops any tenant webhook from intercepting authentication, admission registration, or cluster RBAC, and it exempts the platform's own admin identities, an admin can always recover a cluster whose Kyverno webhook is misbehaving, whatever failure policy it uses. The tradeoff is availability, not lockout:

• failurePolicy: Ignore: if the Kyverno pod is down or unreachable, the API server skips the webhook and admits the request. The rule is best-effort.
• failurePolicy: Fail: the request is blocked when the rule fails and also when Kyverno is down. This is a hard gate, but during a Kyverno outage a broadly-matching Fail webhook can block normal work in your cluster, both your own workloads and the platform's reconciliation of it.

Prefer Ignore for webhooks that match broadly, so a Kyverno outage never stalls the cluster. Reserve Fail for tightly-scoped policies in your own namespaces, where blocking during an outage is acceptable. For a hard gate that must survive a Kyverno outage entirely, write a native VAP (ValidatingAdmissionPolicy) instead: it runs inside the API server, so there is no pod to take down. See Control admission .

Step 1: install Kyverno#

Use the official Kyverno Helm chart. Run multiple replicas, and set the chart's webhook failure-policy value to Ignore. The key name differs by chart version, so find it first with helm show values kyverno/kyverno | grep -i failurepolicy and add it as a --set flag to the command below.

		$ helm repo add kyverno https://kyverno.github.io/kyverno/
$ helm repo update
$ helm install kyverno kyverno/kyverno \
  --namespace kyverno \
  --create-namespace \
  --set admissionController.replicas=3 \
  --set backgroundController.replicas=2 \
  --set cleanupController.replicas=2 \
  --set reportsController.replicas=2
	

Step 2: choose failurePolicy on each policy#

Set spec.webhookConfiguration.failurePolicy on every Kyverno policy so the choice is explicit rather than inherited from a global Helm value that can change later:

		apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: my-policy
spec:
  webhookConfiguration:
    failurePolicy: Ignore # Ignore for a broad webhook so a Kyverno outage never blocks requests
  rules:
    - name: my-rule
      # ...
	

validationFailureAction and failurePolicy are two different fields, and they control two different actors:

• validationFailureAction: Enforce: Kyverno is up and finds a violation, so it blocks the request.
• failurePolicy: Ignore: Kyverno is down, so the API server admits the request instead.

You can set both:

		spec:
  validationFailureAction: Enforce # block objects that violate the rule when Kyverno is up
  webhookConfiguration:
    failurePolicy: Ignore # admit anyway if Kyverno is down
	

Which rule types to use#

Mutation#

Mutation rules change a resource before Kubernetes saves it: add resource limits, inject labels, set a default security context. These are a good fit for Kyverno. If the mutation does not fire because Kyverno is down, the resource is saved without the change. That is usually safer than blocking the request. Run Kyverno highly available so mutations are reliable.

Example: add a cost-center label to every pod in a namespace if it is missing.

		apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: add-cost-center-label
spec:
  webhookConfiguration:
    failurePolicy: Ignore
  rules:
    - name: add-label
      match:
        any:
          - resources:
              kinds: [Pod]
              namespaces: [team-a]
      mutate:
        patchStrategicMerge:
          metadata:
            labels:
              +(cost-center): "team-a"
	

Generation#

Generation rules create resources automatically when a trigger fires: for example, create a default NetworkPolicy when a namespace is created. If the webhook is down when the trigger fires, the generated resource may be missing. That is a gap, not a security hole, so generation rules are a reasonable fit. After applying a generation policy, check that the generated resources exist.

Validation#

Validation rules check a resource and block it when a rule fails (with validationFailureAction: Enforce). Whether the rule still blocks during a Kyverno outage depends on the failurePolicy you chose: Ignore admits the request, Fail keeps blocking. Match that to how strict the rule needs to be.

Use Kyverno validation when the rule is a safety net (not the only enforcement line), the policy logic is too complex for CEL (Common Expression Language, the language native VAPs use), or you pair an enforcing policy with a highly available Kyverno deployment.

For a rule that must always block, even during a Kyverno outage, write a native ValidatingAdmissionPolicy: it runs inside the API server and cannot be taken offline. Do not name it syself-*; that prefix is reserved by the platform.

Image verification#

Kyverno's verifyImages rule checks container image signatures using cosign (a tool for signing and verifying container images). It must be a webhook because it makes network calls and cryptographic checks that a native VAP cannot do. That network dependency makes Fail risky here, since a registry or Kyverno hiccup would block deployments, so it usually runs failurePolicy: Ignore and best-effort. Run Kyverno highly available so this check rarely misses. For the full setup, see Verify image signatures .

Step 3: run Kyverno highly available#

Run Kyverno highly available whichever failure policy you pick. With Ignore, a Kyverno outage silently skips those policies; with Fail, it blocks the requests they match. Three replicas for the admission controller is the minimum. Add a PodDisruptionBudget (a rule that tells Kubernetes to keep a minimum number of replicas running during node drains) to protect against disruption:

		apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: kyverno-admission-controller
  namespace: kyverno
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app.kubernetes.io/component: admission-controller
      app.kubernetes.io/instance: kyverno
	

Spread the admission-controller pods across nodes with podAntiAffinity or a topologySpreadConstraint in your Helm values so a single node failure does not take down all replicas.

Step 4: verify the setup#

Check the failurePolicy on each Kyverno webhook configuration matches what you intended (Ignore for broadly-matching webhooks):

		$ kubectl get validatingwebhookconfigurations -o json \
    | jq '.items[] | select(.metadata.name | startswith("kyverno"))
          | .webhooks[] | {name: .name, failurePolicy: .failurePolicy}'
$ kubectl get mutatingwebhookconfigurations -o json \
    | jq '.items[] | select(.metadata.name | startswith("kyverno"))
          | .webhooks[] | {name: .name, failurePolicy: .failurePolicy}'
	

Confirm each entry shows the failurePolicy you intended, Ignore on the broadly-matching webhooks. If a webhook configuration is missing, webhook-guard rejected it, most likely because it targets the admission-registration, authentication/authorization, or cluster-scoped RBAC APIs described above. The Forbidden error returned by the rejected kubectl apply names the rule that fired. To read those rules, describe the policy rather than the binding. Its validations[].message entries state each restriction:

		$ kubectl describe validatingadmissionpolicy syself-restrict-tenant-webhooks
	

Check that the admission controller has at least two ready replicas:

		$ kubectl -n kyverno get pods
	

Related#

• Control admission : the webhook-guard contract and how to choose between Kyverno and a native VAP.
• OPA Gatekeeper as an alternative : the same webhook-guard contract with Rego instead of YAML policies.
• Verify image signatures : using Kyverno's verifyImages rule for image signature checks.
• Segment with network policies : adding network controls alongside Kyverno policies.
• Security architecture : where Kyverno fits in the platform security model.