Control pod egress (and the FQDN limitation)

Inspect 1.36

Egress control limits what your pods can call out to, which is how you contain data exfiltration and a compromised pod's reach. On Syself Autopilot you write egress with Cilium policy at Layers 3 and 4: address, port, and workload identity. There is one limitation to plan around. FQDN and DNS-based rules are rejected the moment you apply them, so you name destinations by address and identity, not by hostname.

Egress is the outbound half of segmentation. Start from Segment with network policies for the two network layers and the default-deny baseline; this page covers what leaves the cluster and the FQDN gate in full.

Why FQDN and DNS rules are rejected here#

Rules that inspect a request need a userspace proxy on the node, and the node's datapath denies the proxy's return path today. A rule you apply would then silently break the traffic it is meant to permit, and a broken DNS rule can cut off name resolution for other workloads on the same node. A built-in admission policy (syself-restrict-l7-proxy-policies) rejects these rules at apply time with a readable error, so a bad rule never ships to break traffic quietly.

Rule family	Status
Application-layer rules (`rules.http`, `rules.kafka`, `rules.l7proto`, `rules.l7`)	Rejected permanently in a `CiliumClusterwideNetworkPolicy`; rejected for now in a namespaced `CiliumNetworkPolicy`
DNS rules (`toPorts[].rules.dns`, and `toFQDNs`, which needs one)	Rejected for now in both kinds
Layer 3 and Layer 4 rules (identity, IP ranges, entities, ports, protocols)	Work today, entirely in the kernel

The application-layer restriction is permanent in a clusterwide policy by design: it applies to every namespace at once, so one wrong request-level match can change how a workload another team owns behaves. In a namespaced policy the same rules lift once the datapath return path works and is verified. The DNS-rule restriction lifts once every node in the fleet runs the fixed node image, not in the release that first ships it, since a policy is cluster-scoped while the fix lands per node. Networking and Cilium explains the datapath and DNS gates under the hood; Control admission covers the admission policy itself.

Egress rules that work today#

Write egress on identity, address, and port. This policy lets a workload reach only its database and DNS, and nothing else outbound:

egress-policy.yamlyaml

		apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: app-egress
  namespace: team-web
spec:
  endpointSelector:
    matchLabels:
      app: web
  egress:
    # to the database pods, by identity
    - toEndpoints:
        - matchLabels:
            app: db
      toPorts:
        - ports:
            - {port: "5432", protocol: TCP}
    # to a specific external API, by CIDR
    - toCIDR:
        - "203.0.113.10/32"
      toPorts:
        - ports:
            - {port: "443", protocol: TCP}
    # DNS to CoreDNS (an L4 allow to kube-dns, not a DNS-content rule)
    - toEndpoints:
        - matchLabels:
            k8s-app: kube-dns
            io.kubernetes.pod.namespace: kube-system
      toPorts:
        - ports:
            - {port: "53", protocol: UDP}
            - {port: "53", protocol: TCP}

The DNS entry here opens port 53 and nothing more. It carries no rules.dns block, so it does not put the pod's lookups through Cilium's DNS proxy and none of the FQDN restrictions apply to it. Allowing DNS this way works today.

A default-deny egress baseline#

Flip a namespace's pods from "reach anything outbound" to "reach only what you allow" with a policy that selects every pod and lists only the egress you permit. With endpointSelector: {} and an egress list, anything not listed is denied. Always include the port-53 allow to CoreDNS, or the pods cannot resolve names and every outbound connection fails at the lookup.

When two rules overlap, the result is deny. There is no priority field, so you cannot write a broad deny and then allow one thing above it. Write each exception as its own selector.

When you need to name a destination by hostname#

Allow-listing an external service by hostname, or restricting by URL path, is not something egress policy does here today. Two options work around this:

Pin the destination by IP where it is stable, using toCIDR as above, or route the workload through an egress gateway so the far end sees one stable source address. The egress gateway works today.
Use a service mesh for request-level HTTP control between your own services (allow only GET /api, per-path rules). Istio is supported on the platform; see Handle unreliable networks with Istio .

Revisit FQDN policy when the datapath return-path limitation lifts. Until then, address and identity rules are the supported egress controls.

FQDN policy, for when the gate lifts#

toFQDNs allows outbound traffic to a named domain like api.github.com instead of an IP range. Cilium reads the pod's own DNS answers, watches which addresses a name resolves to, and permits exactly those. The pattern below is the one to use once the DNS-rule gate lifts, so you can plan for it. Applying it today is rejected.

The policy needs two rules, and both are required. The first names the allowed destination with toFQDNs. The second allows DNS with a rules.dns block on port 53, and that second rule is what puts the pod's lookups through Cilium's DNS proxy so Cilium ever sees an answer. Without it, Cilium never learns the address behind api.github.com, the toFQDNs rule matches nothing, and every connection to that name is dropped. Deleting the second rule to make the example shorter breaks it completely.

yaml

		apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: egress-github-only
  namespace: team-a
spec:
  endpointSelector:
    matchLabels: {app: ci-runner}
  egress:
    - toFQDNs:
        - matchName: "api.github.com"
      toPorts:
        - ports: [{port: "443", protocol: TCP}]
    - toEndpoints:
        - matchLabels:
            k8s:io.kubernetes.pod.namespace: kube-system
      toPorts:
        - ports: [{port: "53", protocol: UDP}]
          rules: {dns: [{matchPattern: "*"}]}

Only the name lookup passes through the DNS proxy. The connection to the resolved address that follows goes straight from the pod to that address on the eBPF fast path, enforced at Layer 3 and Layer 4 like any other connection.

These follow from how name resolution actually happens inside a pod, and they apply wherever such a policy runs, separate from the platform gate above.

A wildcard only ever covers names the pod actually looked up. A * in a matchPattern stands for characters inside one label and never crosses a dot: *.example.com covers api.example.com but not example.com itself and not files.api.example.com. Use the **. prefix form to cover names nested arbitrarily deep: **.example.com covers both api.example.com and files.api.example.com, though still not example.com itself. A bare matchPattern: "*" matches every name, which for a pod that can resolve anything is close to allowing egress everywhere. Either way the rule only permits addresses that came back in an answer Cilium observed. A destination the pod reaches by hard-coded IP, or one it resolved before the policy existed, is not covered; use toCIDR for those.

Kubernetes Service names do not work in FQDN rules. A cluster-internal name like backend.team-a.svc.cluster.local is not a usable target in toFQDNs. A toFQDNs rule turns into an IP-range rule under the hood, and an IP-range rule cannot select pods, so it matches no destination inside the cluster. FQDN rules are for destinations outside the cluster. For traffic inside the cluster use label selectors (toEndpoints), which also survive a pod restart and a new IP.

Every lookup takes the slow path. While such a policy selects a pod, each name lookup that pod makes is handled by the Cilium agent's DNS proxy in userspace instead of in eBPF, and a lookup is answered only while the agent on that node is running. The data connections are unaffected; only lookups pay this. A workload that resolves the same name repeatedly is the case to watch: give it a DNS cache, or select that destination by label or IP range.

Alpine and other musl-based images stop at the first refused lookup. When the DNS proxy sees a query the rules.dns list does not permit, it answers Refused. musl, the small C library Alpine uses in place of glibc, treats Refused as a hard failure and stops walking the search list in /etc/resolv.conf, where glibc would carry on. Kubernetes gives pods ndots: 5, so a short name like api.github.com is tried against the search domains first: a pod in team-a asks for api.github.com.team-a.svc.cluster.local before api.github.com. If your rules.dns list names only the real domain, the proxy refuses that first query and the Alpine pod gives up there. Keep rules: { dns: [{ matchPattern: "*" }] } as the example does so the proxy refuses nothing, or set ndots: 1 on the pod through dnsConfig, or build on a glibc base image.

The symptom: a pod a policy selects cannot resolve anything, including the domain the policy explicitly allows. Inside the pod, nslookup times out with "no servers could be reached" and wget reports bad address. That differs from a normal policy denial, where resolution still works and the connection to the resolved address is what times out.

Read a test in this state carefully. A check that a denied domain is unreachable also fails with bad address, so it passes for the wrong reason: nothing is being enforced, the pod simply cannot resolve any name. Confirm the allowed name resolves before you conclude a deny is working.

Check the rule pairing first. If the policy has a toFQDNs rule but no rules.dns rule permitting port 53 to the cluster DNS service, the policy itself is the cause: under default-deny the pod's queries never reach the resolver. Compare against the two-rule example above. This is the one you own and can fix in a minute, and it produces exactly the same symptom as the platform-side DNS-proxy problem, down to the error wording. Check this first, before you start looking at the nodes.

Then check the reverse-path filter on the node's pod interfaces. Cilium's DNS proxy redirects the pod's queries to a socket in the node's own network namespace, which puts packets on the pod's virtual interface whose source address the kernel's routing table does not expect there. The reverse-path check drops exactly those. Each pod has an lxc-prefixed interface on its node, and the check has to read 0 on all of them:

		$ kubectl debug node/<node> -it --image=busybox -- \
  sh -c 'grep . /proc/sys/net/ipv4/conf/*/rp_filter'

all, default, the cilium_* interfaces, and every lxc* interface have to read 0. Any lxc* interface reading 1 or 2 means this node still drops proxied DNS traffic. The node's uplink and lo read 2 on purpose. The setting is part of the node image, so the fix is to move to a node image that carries it, not to change it by hand: a hand-set value is lost when the node is replaced, and every pod that starts afterwards creates a new interface that comes up wrong again.

Segment with network policies : the two network layers and the ingress-side baseline
Networking and Cilium : the datapath and DNS gates behind the rejection
Control admission : the admission policy that rejects these rules
Handle unreliable networks with Istio : request-level HTTP control through a service mesh

Segment with network policies

How Cilium enforces pod-to-pod traffic policy by label identity, and how to set up a default-deny baseline per namespace using standard NetworkPolicy objects.

Encrypt pod traffic with WireGuard

Enable WireGuard transparent encryption in Cilium so cross-node pod traffic is encrypted at the kernel level with no changes to your applications or workloads.