Skip to main content

Runtime threat detection

Inspect 1.36

The sealed OS and admission guardrails prevent a lot, but prevention is never complete. Runtime threat detection watches what actually happens inside running containers and alerts on the behavior that gets past the preventive controls: a shell spawned in a container, an unexpected outbound connection, a privilege change. Falco is the common tool; deploy it, tune its rules, and wire it into a detect-and-respond loop.

Deploy Falco as a DaemonSet

Falco runs a sensor on every node. On the sealed OS there is one thing to get right: use the eBPF driver, not a kernel module. The kernel is built with MODULE_SIG_FORCE, so an unsigned kernel module cannot load, and Falco's kernel-module driver would be rejected. Falco's eBPF probe needs no module and is the supported path here. The Helm chart selects it:

		$ helm repo add falcosecurity https://falcosecurity.github.io/charts
$ helm install falco falcosecurity/falco \
  --namespace falco --create-namespace \
  --set driver.kind=ebpf
	

Which rules matter most

Falco ships a default ruleset; the high-value rules for a Kubernetes cluster are the ones that flag post-compromise behavior:

  • A shell in a container (bash, sh) where the workload should never spawn one, the classic sign of an attacker exploring after a breach.
  • Unexpected outbound connections, a pod dialing a destination it never has before (pair this with for the network view).
  • Privilege changes, a process trying to escalate, write to a sensitive path, or load code.

Tune out the noise for your workloads (a legitimate debug container, an expected admin action) so a real alert is not buried, then keep the rules under version control.

Route alerts to your SIEM

A detection is only useful if someone sees it. Send Falco's alerts to the same SIEM or store you , so runtime alerts sit next to the audit trail. Together, Falco (what a process did), the (who touched the API and what the OS saw), and Hubble (what talked to what) give you the three views an investigation needs. Falco is a runtime layer you add on top of the built-in , and its alerts land in storage you own, the same as every other signal.

The detect-and-respond runbook

When Falco fires on a workload you believe is compromised:

Isolate the pod

Apply a that denies the pod all ingress and egress, so it cannot spread or exfiltrate while you look. Do not delete it yet.

Capture evidence

Preserve the Falco alert, the pod's logs, and the relevant audit and Hubble records off-node, before the pod is gone. A replaced node takes its local evidence with it.

Redeploy clean

Once you have what you need, delete the pod and let its controller recreate it from the clean image. If the node itself is suspect, treat it as a : reprovision, never repair in place.

Close the gap

Fix what let it in (the vulnerable image, the missing network policy, the over-broad RBAC) so the same path does not reopen.

Runtime detection complements the sealed OS and admission; it does not replace them. Keep the preventive controls tight and use Falco to catch what gets through, then respond fast enough that a detection does not become an incident.