Diagnose pod DNS resolution
When a pod cannot resolve a name, work from the pod outward: test resolution, check CoreDNS, then check what might be blocking the query. CoreDNS runs in kube-system and answers DNS for every pod in the cluster.
Test resolution from inside a pod#
$ kubectl exec -it <pod-name> -n <namespace> -- nslookup kubernetes.default.svc.cluster.local
A successful answer means DNS works and the problem is elsewhere. A timeout or SERVFAIL means the query is not reaching CoreDNS or CoreDNS cannot answer it.
Check CoreDNS#
$ kubectl get pods -n kube-system -l k8s-app=kube-dns
$ kubectl get endpoints -n kube-system kube-dns
Expect the pods Running and 1/1, and the Service with endpoints. If the pods are unhealthy, kubectl describe them and read their logs. Syself Autopilot autoscales CoreDNS with the cluster, so its replica count is not yours to change; a persistent CoreDNS problem is one to raise with support rather than fix manually. See Understand in-cluster DNS for how it is managed.
A NetworkPolicy that blocks port 53#
This is the most common cause once network policies are in use. A policy that denies all egress also blocks DNS, because DNS uses port 53. The pod then resolves nothing, and every outbound connection fails at the name-lookup step.
Add an egress rule that allows UDP and TCP on port 53 to the CoreDNS pods (label k8s-app=kube-dns) in kube-system:
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
podSelector:
matchLabels:
k8s-app: kube-dns
ports:
- protocol: UDP
port: 53
- protocol: TCP
port: 53
Note
Segment DNS with an L3/L4 rule such as this, not with an L7 DNS policy. HTTP-aware and toFQDNs rules are rejected at apply time on this platform (see Control egress with FQDN policy ), so a policy that tries to filter DNS by name does not take effect. Allow port 53 to CoreDNS and control destinations by another means.
Warning
A DNS allow-rule that selects the pod by a label the rollout changes stops matching after a new image ships, and the new pods lose DNS with no error in the application. It appears that the old image worked and the new one is broken, but the code did not change; the policy no longer matches. After a Deployment rolls, confirm the new pods still match the egress rule that opens port 53, and watch Hubble for DROPPED flows toward k8s-app=kube-dns.
External names fail but internal names work#
If service.namespace.svc.cluster.local resolves but example.com does not, the problem is upstream, not CoreDNS. CoreDNS forwards names it does not own to the node's resolver, and that resolver is set by the dnsServers cluster variable. Check that variable and confirm the upstream resolvers it names are reachable.
A pod's /etc/resolv.conf has a search list and an ndots value (5 by default in Kubernetes). A name with fewer dots than ndots is first tried with each search domain appended, so api becomes api.<namespace>.svc.cluster.local, then the next search domain, before it is tried as-is. That is why an unqualified external name can be slow or fail: it runs through several cluster suffixes first. Use a fully qualified name with a trailing dot (api.example.com.) for an external host to skip the search list.
When a name still will not resolve after CoreDNS looks healthy and policy allows port 53, use Hubble , Cilium's live flow viewer, to see whether the DNS packet is dropped and why.
If the node itself appears faulty, with the tunnel or syself-agent unhealthy, start at Debug node networking .