Skip to main content

Set namespace defaults with LimitRange and ResourceQuota

Inspect 1.36

Run an agency on one cluster and every client lives in its own namespace. Each of those namespaces needs two things: pods that are well-formed even when the developer who deployed them forgot to set resources, and a hard ceiling so no single client can starve the others of cluster resources. Kubernetes has one admission controller for each half of that problem, and on Syself Autopilot both controllers are already enabled. That does not mean your namespaces are shaped: the controllers only act on the LimitRange and ResourceQuota objects you create. Add either one to a namespace and it takes effect on the next pod that lands there, with nothing to install.

What each tool owns

  • LimitRange shapes each pod: Fills in a default request and limit for any container that omits one, and caps how large a single container may ask to be. It corrects the for the whole namespace at once, before a scheduler ever sees the pod.
  • ResourceQuota fences the total: Caps what the namespace can consume in aggregate: requests.cpu, requests.memory, limits.memory, and object counts like pods, persistentvolumeclaims, or services.loadbalancers, the last of which stops one tenant from quietly creating a hundred load balancers.

Both are namespaced objects you apply once per client:

yaml
		apiVersion: v1
kind: LimitRange
metadata:
  name: defaults
  namespace: client-acme
spec:
  limits:
    - type: Container
      defaultRequest: # request filled in when a container sets none
        cpu: "250m"
        memory: "256Mi"
      default: # limit filled in when a container sets none
        memory: "256Mi"
      max: # ceiling a single container may ask for
        cpu: "2"
        memory: "2Gi"
---
apiVersion: v1
kind: ResourceQuota
metadata:
  name: client-acme-quota
  namespace: client-acme
spec:
  hard:
    requests.cpu: "8"
    requests.memory: 16Gi
    limits.memory: 32Gi
    pods: "50"
    services.loadbalancers: "2"
	

The quota caps limits.memory but deliberately leaves limits.cpu off, mirroring the : pods here run without a CPU limit so they can burst into spare cores, and quota-ing limits.cpu would force one back on. CPU is still bounded through requests.cpu, which is what the scheduler actually reserves. For the same reason the LimitRange above defaults a memory limit but no CPU limit.

The catch that ties them together

Set a quota on requests.cpu or requests.memory and Kubernetes changes the rules: every pod in that namespace must now declare those requests, or the API server rejects it outright. "Please set requests" becomes a hard requirement.

That would break every developer who forgets, except the LimitRange in the same namespace is exactly what saves them. It defaults the missing request in before admission, so the quota only ever sees a fully-specified pod. The two work as a pair: the quota makes requests mandatory, the LimitRange makes forgetting them harmless. Size both to what the client pays for, then add and per-namespace RBAC for real isolation.

Read what a client has left

Ask a namespace how close it is to its ceiling with describe quota:

		$ kubectl describe quota client-acme-quota -n client-acme
Name:            client-acme-quota
Namespace:       client-acme
Resource         Used   Hard
--------         ----   ----
pods             12     50
requests.cpu     2500m  8
requests.memory  6Gi    16Gi
	

Used against Hard is the headroom a client has before pods start getting rejected. A deploy that fails with a must specify or exceeded quota error is the guardrail doing its job. Fix the pod's requests, or raise the ceiling by editing the hard values on the ResourceQuota.

Verify the guardrails

Prove both halves on a throwaway namespace. First, that the LimitRange fills in a missing request. Deploy a container with no resources and read back what it got:

		$ kubectl run probe --image=nginx -n client-acme
$ kubectl get pod probe -n client-acme -o jsonpath='{.spec.containers[0].resources.requests}'
{"cpu":"250m","memory":"256Mi"}
	

The request you never set is the LimitRange default. Then breach the ceiling on purpose to watch the quota reject it:

		$ kubectl create deployment flood --image=nginx --replicas=100 -n client-acme
$ kubectl get events -n client-acme | grep -i "exceeded quota"
... Error creating: pods "flood-..." is forbidden: exceeded quota: client-acme-quota, requested: pods=1, used: pods=50, limited: pods=50
	

That rejection is the guardrail working, not a bug. Remove the probe and flood when you are done.