Fix DNS Resolution Problems in AWS VPCs | Route 53, Lambda, RDS
Your RDS instance is running, the security group allows the port, the subnet looks right. But the connection still fails. In many cases the actual problem is DNS. Inside an AWS VPC, DNS goes through a VPC-specific resolver, and several common misconfigurations can silently break it in ways that look exactly like a network failure.
A Lambda function that can’t reach RDS doesn’t say “DNS is broken.” It says the host is unknown. An EC2 instance that can’t find a private service just times out. This page shows you how to identify which DNS problem you have, test from the right place, and fix it quickly. Theory is here too, but it comes after the fast checks.
What DNS does in AWS and why it breaks
DNS converts a hostname like my-cluster.abc123.us-east-1.rds.amazonaws.com into an IP address. If DNS fails, your application errors out before a single byte of application data is sent. The database could be perfectly healthy. DNS just couldn’t find the address to connect to.
Think of the VPC DNS resolver as an internal company directory. Your code asks for a name, and the directory returns the IP address to call. Without the directory, nothing reaches anything — not because resources moved, but because the address book is missing. The database can be healthy, listening, and fully reachable. If the directory is gone, your code never gets that far.
Inside every VPC, AWS runs a built-in DNS resolver at a fixed address: the VPC’s base CIDR + 2. For a VPC using 10.0.0.0/16, that’s 10.0.0.2. Every EC2 instance, Lambda function, ECS task, and EKS pod inside that VPC automatically uses this resolver, unless a custom DHCP option set overrides it.
That resolver handles everything: EC2 private hostnames, RDS endpoints, S3 URLs, Route 53 private records, and public internet names. If the resolver is disabled, or a custom DHCP option set sends your instances to an external DNS server that doesn’t understand AWS-internal names, resolution breaks in ways that are hard to diagnose from the outside.
When to use this page
Use this page if you’re seeing any of these symptoms:
- A hostname does not resolve inside a VPC (
NXDOMAINor timeout) - A Lambda function cannot resolve an RDS or Aurora endpoint
- Route 53 private hosted zone records don’t resolve inside a VPC
- ECS service discovery or AWS Cloud Map names are not resolving
- EKS pods can’t resolve Kubernetes service names or external hostnames
- An EC2 instance resolves a name differently than your laptop does
- A custom corporate DNS server or DHCP option set is in play
This is not the right page if the hostname resolves to an IP correctly but the connection still fails. That points to routing, security groups, or NACLs. Check debugging VPC connectivity instead. If RDS returns “connection refused” after DNS resolves, check RDS connection refused and the network troubleshooting guide.
Quick diagnosis checklist
Run through these checks before diving deeper. Each one takes under a minute.
enableDnsSupportis enabled on the VPC: iffalse, nothing resolves at allenableDnsHostnamesis enabled on the VPC: required for private hosted zones and EC2 private hostnames; off by default on custom VPCs- The Route 53 private hosted zone is associated with this VPC: a zone not associated with a VPC is invisible to it
- The specific DNS record exists in the hosted zone: test with
dig @10.0.0.2 <name>from inside the VPC - No custom DHCP option set is overriding DNS: check
aws ec2 describe-dhcp-options; if a non-AWS IP is listed, verify it forwards to the VPC resolver - You are testing from inside the VPC: your laptop resolves names differently; use Session Manager to connect to an EC2 instance without opening SSH
- For EKS, CoreDNS pods are running:
kubectl get pods -n kube-system -l k8s-app=kube-dns - Know your error type:
NXDOMAIN(resolver replied, record missing) vs. timeout (resolver unreachable) vs. “connection refused” (DNS worked, TCP failed)
If you’re unsure where to start, skip to How to test DNS correctly, then return here.
DNS failure vs. network connectivity failure
The error message your application shows often looks identical whether DNS failed or the TCP connection failed. This table helps you triage before making changes.
| Symptom | What it likely means | First thing to test |
|---|---|---|
Name or service not known / NXDOMAIN | DNS record missing or hosted zone not associated with VPC | dig @10.0.0.2 <hostname> from inside the VPC |
| Request hangs, then connection times out | DNS resolver unreachable: enableDnsSupport off, or custom DNS server down | dig @10.0.0.2 <hostname> (does it respond at all?) |
Connection refused after a short delay | DNS resolved, but TCP connection was rejected | telnet <ip> <port> or nc -zv <ip> <port> |
Connection refused immediately | DNS resolved from cache, TCP refused instantly | Check security groups and whether the service is listening |
| Name resolves on laptop, fails inside VPC | VPC uses different DNS resolver or custom DHCP option set | cat /etc/resolv.conf on an EC2 instance inside the VPC |
| Name resolves to wrong IP inside VPC | Split-horizon misconfiguration or stale record | Compare dig @10.0.0.2 <name> vs. dig @8.8.8.8 <name> |
The key rule: if dig returns an IP address, DNS worked. Whatever fails after that is a network or application problem, not a DNS problem.
How AWS VPC DNS works
Every VPC has a built-in DNS resolver at base CIDR + 2. For 10.0.0.0/16 that’s 10.0.0.2. This resolver is the single entry point for all DNS inside the VPC: it handles public internet names, AWS service endpoints, private EC2 hostnames, and Route 53 private hosted zone records.
Two VPC settings control how it behaves:
enableDnsSupport: activates the VPC DNS resolver. When false, no DNS resolution works inside the VPC at all. Everything breaks. Default is true.
enableDnsHostnames: gives EC2 instances a private DNS hostname like ip-10-0-1-45.us-east-1.compute.internal. Also required for private hosted zones to resolve correctly inside the VPC. Default is true for the default VPC, but false for every custom VPC you create. This catches a lot of people off guard.
Check both:
aws ec2 describe-vpc-attribute \
--vpc-id vpc-0abc123 \
--attribute enableDnsSupport
aws ec2 describe-vpc-attribute \
--vpc-id vpc-0abc123 \
--attribute enableDnsHostnamesBoth should return "Value": true. Enable them if not:
aws ec2 modify-vpc-attribute \
--vpc-id vpc-0abc123 \
--enable-dns-support
aws ec2 modify-vpc-attribute \
--vpc-id vpc-0abc123 \
--enable-dns-hostnamesChanges take effect immediately. No restart required.
Root causes and fixes
1. VPC DNS support or hostnames disabled
What breaks: If enableDnsSupport is off, all DNS resolution fails inside the VPC. If only enableDnsHostnames is off, private hosted zones stop working and EC2 instances don’t receive private hostnames, but public name resolution still works.
enableDnsSupport = false is a total blackout If this flag is false, no instance, Lambda function, or container inside the VPC can resolve any hostname. RDS, S3, internal services — everything fails. It is the single highest-impact DNS setting in the VPC and is almost never intentional.
Verify:
aws ec2 describe-vpc-attribute --vpc-id vpc-0abc123 --attribute enableDnsSupport
aws ec2 describe-vpc-attribute --vpc-id vpc-0abc123 --attribute enableDnsHostnamesFix:
aws ec2 modify-vpc-attribute --vpc-id vpc-0abc123 --enable-dns-support
aws ec2 modify-vpc-attribute --vpc-id vpc-0abc123 --enable-dns-hostnames2. Route 53 private hosted zone not associated with the correct VPC
What breaks: DNS records in the private hosted zone return NXDOMAIN from any VPC not associated with the zone. The records exist and are correct; the VPC just can’t see them.
Verify:
# List private hosted zones
aws route53 list-hosted-zones \
--query 'HostedZones[?Config.PrivateZone==`true`].[Id,Name]'
# Check which VPCs a hosted zone is associated with
aws route53 get-hosted-zone \
--id /hostedzone/ZXXXXXXXXXXXXX \
--query 'VPCs'If your VPC ID is not in the output, the zone is not associated with it.
Fix:
aws route53 associate-vpc-with-hosted-zone \
--hosted-zone-id /hostedzone/ZXXXXXXXXXXXXX \
--vpc VPCRegion=us-east-1,VPCId=vpc-0newvpc456See Route 53 private hosted zones for the full setup pattern including cross-account associations.
3. Wrong or incomplete record in the hosted zone
What breaks: The hosted zone is associated correctly, but the name you’re querying has a typo, is missing entirely, or points to the wrong IP.
Verify:
# List records in a hosted zone
aws route53 list-resource-record-sets \
--hosted-zone-id /hostedzone/ZXXXXXXXXXXXXX \
--query 'ResourceRecordSets[?Name==`myservice.internal.`]'
# Test from inside the VPC
dig @10.0.0.2 myservice.internalNote the trailing dot: Route 53 stores names in FQDN notation (myservice.internal.). A query for myservice.internal should still match, but verify the record name matches what you expect.
Fix: Create or correct the record in the Route 53 console or CLI. For RDS and Aurora, use the endpoint shown in the RDS console. Do not create a manual A record for RDS unless you know the IP is static.
4. Custom DHCP option set overriding DNS
What breaks: A custom DHCP option set points EC2 instances at a corporate or on-premises DNS server. If that server doesn’t forward AWS-specific queries to the VPC resolver, RDS endpoints, private hosted zones, and .internal hostnames fail.
Verify:
# Find the DHCP option set for a VPC
aws ec2 describe-vpcs \
--vpc-ids vpc-0abc123 \
--query 'Vpcs[0].DhcpOptionsId'
# Inspect the option set
aws ec2 describe-dhcp-options \
--dhcp-options-ids dopt-0abc123If domain-name-servers is not AmazonProvidedDNS, a custom server is in use. Test it from inside the VPC:
# Query the custom server directly
dig @<custom-dns-ip> my-cluster.abc.us-east-1.rds.amazonaws.com
# Compare with the VPC resolver
dig @10.0.0.2 my-cluster.abc.us-east-1.rds.amazonaws.comIf the VPC resolver returns an answer but the custom server doesn’t, forwarding is missing or broken.
Custom DNS without forwarding breaks all AWS names at once Pointing instances at a corporate DNS server is fine. Not configuring it to forward AWS queries back to the VPC resolver breaks RDS endpoints, private hosted zones, and EC2 hostnames for every instance in the VPC simultaneously. It’s an easy mistake to make and a confusing one to diagnose.
Fix: Configure the custom DNS server to forward these query types to the VPC resolver (base CIDR + 2):
*.internal(EC2 private hostnames)*.amazonaws.com(AWS service endpoints: RDS, S3, SQS, etc.)- Your private hosted zone domain names
A correctly configured split-DNS setup using Unbound, BIND, or Windows DNS Server forwards the above to the VPC resolver and everything else to public DNS (like 8.8.8.8).
5. Testing from the wrong place
What breaks: Nothing is actually broken. You’re just unable to reproduce the failure from your laptop because your laptop uses different DNS servers.
Verify: If DNS works on your laptop but fails in your application, you need to test from inside the VPC. Use Session Manager to connect to an EC2 instance without opening any SSH ports:
aws ssm start-session --target i-0abc123Then run your DNS tests from inside that instance. See How to test DNS correctly below.
This is a testing methodology issue. The actual DNS failure is one of the other causes on this page.
6. Lambda in a VPC cannot resolve RDS or private names
Lambda functions placed in a VPC inherit the VPC’s DNS settings. If enableDnsSupport is off on that VPC, or a custom DHCP option set routes DNS to an unreachable server, the Lambda function cannot resolve any hostnames, including RDS cluster endpoints.
Typical error in Python:
socket.gaierror: [Errno -2] Name or service not known
Host: my-cluster.cluster-xyz.us-east-1.rds.amazonaws.comVerify:
aws ec2 describe-vpc-attribute \
--vpc-id vpc-0lambda123 \
--attribute enableDnsSupportFix: Enable DNS support on the VPC. Also check that the Lambda’s subnets have a route to local traffic; custom route tables that remove the local route can also block access to the VPC resolver.
For the complete pattern, see connecting Lambda to RDS securely and verify that the RDS security group allows inbound traffic from the Lambda’s security group on the database port.
7. ECS service discovery / Cloud Map / private namespace issues
What breaks: ECS service discovery uses Route 53 private hosted zones under the hood via AWS Cloud Map. If the namespace is not associated with your VPC, or if ECS tasks haven’t registered yet (due to failed health checks), DNS queries return NXDOMAIN.
Verify:
# List Cloud Map namespaces
aws servicediscovery list-namespaces
# Check which VPC the namespace is associated with
aws servicediscovery get-namespace \
--id ns-XXXXXXXXXX
# List registered instances for a service
aws servicediscovery list-instances \
--service-id srv-XXXXXXXXXXAlso check that ECS tasks are healthy; unhealthy tasks automatically deregister from service discovery.
Fix: Associate the Cloud Map namespace with the correct VPC. If tasks aren’t registering, investigate ECS task health checks and the healthCheckCustomConfig settings on the service.
8. EKS DNS and CoreDNS problems
EKS uses CoreDNS for in-cluster DNS. Pods resolve Kubernetes service names (e.g., my-service.my-namespace.svc.cluster.local) through CoreDNS. External names go through CoreDNS too, forwarded on to the VPC resolver.
Common failure modes:
- CoreDNS pods are crashed or not scheduled
- A pod’s
dnsPolicyis notClusterFirst(the default) - The CoreDNS ConfigMap has a misconfigured or stale
forwardrule - The VPC has
enableDnsSupportdisabled, so CoreDNS cannot resolve external names
Verify:
# Check CoreDNS pods
kubectl get pods -n kube-system -l k8s-app=kube-dns
# Check CoreDNS logs for errors
kubectl logs -n kube-system -l k8s-app=kube-dns
# Test in-cluster resolution from inside a pod
kubectl run dns-test --image=busybox --rm -it --restart=Never \
-- nslookup kubernetes.default
# Test external resolution from inside a pod
kubectl run dns-test --image=busybox --rm -it --restart=Never \
-- nslookup amazonaws.com
# Inspect the CoreDNS ConfigMap
kubectl get configmap coredns -n kube-system -o yamlFix: If CoreDNS pods are crashing, check the logs and the ConfigMap for syntax errors or stale IP addresses in the forward rule. If external resolution fails from pods but in-cluster resolution works, check that enableDnsSupport is enabled on the VPC. CoreDNS depends on the VPC resolver for all non-cluster queries.
See EKS networking model for how CoreDNS integrates with VPC DNS.
9. Peered VPC, multi-VPC, or hybrid DNS assumptions
VPC peering creates a routing path between two VPCs. It does not automatically share Route 53 private hosted zone resolution.
VPC peering does not share DNS This is one of the most common multi-VPC mistakes. Peering connects routes, not DNS namespaces. A private hosted zone associated with VPC A is completely invisible to VPC B even after peering is fully established. You must explicitly associate the zone with both VPCs.
What breaks: VPC B is peered with VPC A. VPC A has a private hosted zone. Engineers assume the peered VPC can resolve records in that zone. It cannot. The hosted zone is only associated with VPC A.
Verify:
# From an instance in the peered VPC, test resolution
dig @10.0.0.2 internal.mycompany.com
# Confirm whether the hosted zone covers the peered VPC
aws route53 get-hosted-zone \
--id /hostedzone/ZXXXXXXXXXXXXX \
--query 'VPCs'Fix: Explicitly associate the private hosted zone with the peered VPC:
aws route53 associate-vpc-with-hosted-zone \
--hosted-zone-id /hostedzone/ZXXXXXXXXXXXXX \
--vpc VPCRegion=us-east-1,VPCId=vpc-0peeredvpc789For cross-account peering, the association requires authorization from the account that owns the hosted zone. This is a two-step process. For hybrid scenarios (Direct Connect or VPN), use Route 53 Resolver inbound and outbound endpoints to forward queries between on-premises resolvers and the VPC resolver instead of relying on DHCP overrides.
How to test DNS correctly
Always test from inside the VPC. Your laptop’s DNS results will mislead you.
Use Session Manager, not SSH Session Manager lets you connect to an EC2 instance with zero open inbound ports and no SSH key needed. It works even when the instance has no public IP and is faster to set up than a bastion host.
Connect to an instance inside the VPC:
aws ssm start-session --target i-0abc123Step 1 — Check which DNS server the instance is using:
cat /etc/resolv.confThe nameserver line should point to the VPC resolver (base CIDR + 2). A different IP means a custom DHCP option set is active.
Step 2 — Query the VPC resolver directly:
# Use your actual base+2 address
dig @10.0.0.2 my-cluster.cluster-abc.us-east-1.rds.amazonaws.comIf this returns an IP, the VPC resolver knows about the name. If it returns NXDOMAIN, the record is missing or the hosted zone isn’t associated with this VPC.
Step 3 — Test default resolution:
dig my-cluster.cluster-abc.us-east-1.rds.amazonaws.com
nslookup my-cluster.cluster-abc.us-east-1.rds.amazonaws.comIf Step 2 worked but Step 3 doesn’t, the instance is using a custom DNS server that can’t resolve this name. Forwarding is broken.
Step 4 — Compare answers for a private name:
# From inside VPC — should return private IP
dig @10.0.0.2 myservice.internal.mycompany.com
# From outside VPC — will return NXDOMAIN or a different IP
dig @8.8.8.8 myservice.internal.mycompany.comStep 5 — For EKS, test from inside a pod:
# Test in-cluster service name resolution
kubectl run dns-test --image=busybox --rm -it --restart=Never \
-- nslookup my-service.my-namespace.svc.cluster.local
# Test that external names resolve through CoreDNS
kubectl run dns-test --image=busybox --rm -it --restart=Never \
-- nslookup amazonaws.comReading dig output:
status: NOERRORwith an answer section: DNS works, the problem is elsewherestatus: NXDOMAIN: resolver responded but the record doesn’t exist;; connection timed out; no servers could be reached: resolver is unreachable- Answer has a public IP when you expected a private one: split-horizon misconfiguration
Real scenarios
Scenario 1: Lambda in a VPC can’t resolve an RDS endpoint
A Python Lambda function deployed inside a VPC logs this error on every invocation:
socket.gaierror: [Errno -2] Name or service not known
Host: my-cluster.cluster-xyz.us-east-1.rds.amazonaws.comThe Aurora cluster is healthy. Security groups allow port 5432 from the Lambda’s security group. But the connection never gets that far. DNS fails before any TCP connection is attempted.
Investigation:
aws ec2 describe-vpc-attribute \
--vpc-id vpc-0lambda123 \
--attribute enableDnsSupportOutput: {"EnableDnsSupport": {"Value": false}}
DNS was disabled when the VPC was created with a custom DHCP option set for a corporate DNS server. The engineer also set enableDnsSupport to false, assuming the custom server would handle everything.
Fix:
aws ec2 modify-vpc-attribute \
--vpc-id vpc-0lambda123 \
--enable-dns-supportDNS resolution starts working immediately. Lambda connects to RDS on the next invocation. No Lambda redeployment needed.
See Lambda VPC access for the full pattern, and connecting to RDS securely for security group and subnet configuration.
Scenario 2: EKS pods can’t resolve external names (CoreDNS forwarding broken)
An EKS cluster has pods that resolve Kubernetes service names correctly (my-service.default.svc.cluster.local works fine), but calls to any external endpoint fail:
nslookup: can't resolve 's3.amazonaws.com'Investigation:
CoreDNS pods are running and not crashing. Inspecting the ConfigMap:
kubectl get configmap coredns -n kube-system -o yamlThe forward rule reads:
forward . 10.1.0.210.1.0.2 is a stale IP from a previous VPC CIDR range. The cluster was migrated to a new VPC but the CoreDNS ConfigMap was never updated. CoreDNS forwards all external queries to an address that no longer has a resolver.
Fix:
kubectl edit configmap coredns -n kube-systemUpdate the forward rule to the correct VPC resolver:
forward . 10.0.0.2CoreDNS detects ConfigMap changes within seconds. External name resolution restores for all pods without a CoreDNS pod restart.
See EKS networking model for how CoreDNS integrates with VPC DNS.
Common mistakes
- Creating a custom VPC without enabling DNS hostnames:
enableDnsHostnamesis off by default on all custom VPCs. Private hosted zones won’t resolve and EC2 instances won’t get private hostnames. Enable it at VPC creation time or immediately after. - Associating a private hosted zone with the wrong VPC: when creating a new environment (staging, prod-eu, etc.), forgetting to associate existing private hosted zones with the new VPC means the new VPC can’t resolve any internal service names.
- Testing DNS from your laptop instead of from inside the VPC: your laptop uses your ISP or corporate DNS, not the VPC resolver. A name that resolves locally may still fail inside the VPC. Always test from an EC2 instance or pod using the VPC resolver address directly.
- Using a custom DNS server without configuring forwarding: pointing instances at an on-premises resolver via a DHCP option set breaks RDS endpoints, private hosted zones, and EC2 private hostnames unless that resolver explicitly forwards AWS-specific queries to the VPC resolver.
- Confusing DNS failure with security group failure: “connection refused” or “host unreachable” after a TCP handshake attempt is a network problem. “Name not resolved” or “Name or service not known” before any connection attempt is a DNS problem. Check which error type you have before touching security groups.
- Assuming VPC peering automatically shares private DNS: peering provides a routing path, not shared DNS. A private hosted zone in one VPC is invisible to a peered VPC unless you explicitly associate the zone with both VPCs.
Summary
- Check
enableDnsSupportandenableDnsHostnamesfirst. Both must betrueon any VPC using private DNS. Custom VPCs haveenableDnsHostnamesoff by default. - Route 53 private hosted zones must be explicitly associated with each VPC that needs to resolve their records. No association means no resolution.
- Always test DNS from inside the VPC using an EC2 instance or pod. Your laptop uses different DNS and will give different results.
- Custom DHCP option sets that point to external DNS servers must forward AWS-internal queries to the VPC resolver (base CIDR + 2) or they break everything.
- VPC peering does not automatically share private hosted zone resolution. Associate the zone with all relevant VPCs explicitly.
NXDOMAINmeans the resolver replied but the record doesn’t exist. A timeout means the resolver couldn’t be reached at all. These have different fixes.- EKS uses CoreDNS for in-cluster resolution; external name resolution also depends on CoreDNS forwarding correctly to the VPC resolver.
Frequently asked questions
Why does DNS work on my laptop but not inside the VPC?
Your laptop uses your ISP or corporate DNS server. Inside a VPC, instances use the VPC DNS resolver at base CIDR + 2 (e.g., 10.0.0.2). If that resolver is disabled or a custom DHCP option set overrides it with an external server, VPC DNS behaves differently from your laptop. Always test from inside the VPC using an EC2 instance or pod.
What is the difference between NXDOMAIN and a timeout?
NXDOMAIN means the DNS server replied and said the name does not exist. The record is missing or the hosted zone is not associated with your VPC. A timeout means no DNS server replied at all, usually because enableDnsSupport is disabled or a custom DNS server is unreachable. These two failures have different root causes and need different fixes.
Can I safely use a custom DNS server in AWS?
Yes, via a custom DHCP option set. But the custom server must forward queries for AWS-internal names (.internal, .amazonaws.com, and your private hosted zone domains) to the VPC resolver at base CIDR + 2. If it does not, RDS endpoints, private hosted zones, and EC2 hostnames will stop resolving inside the VPC.
Does VPC peering make private hosted zones resolve automatically?
No. Peering provides a routing path between VPCs, but it does not automatically share Route 53 private hosted zone resolution. You must explicitly associate the hosted zone with each VPC that needs to resolve records in it, and ensure enableDnsHostnames is enabled on both VPCs.
Why can Lambda fail on hostname resolution even when RDS is healthy?
Lambda functions placed in a VPC inherit that VPC's DNS settings. If the VPC has enableDnsSupport disabled, or a custom DHCP option set points to a DNS server that cannot reach the VPC resolver, Lambda cannot resolve RDS endpoints even though RDS is running normally. The socket.gaierror: [Errno -2] error in Python is the most common symptom.