How to Fix GCP DNS Resolution Problems (Private Zones, VPC Peering, TTL)

Your GCP VM cannot resolve a hostname. You see NXDOMAIN, SERVFAIL, or your application times out on a DNS lookup. The cause is almost always one of five things: the VM is pointing at the wrong resolver, a private DNS zone is not attached to the VM’s VPC, the record does not exist, DNS peering is missing between VPCs, or a cached TTL is serving stale data. This page walks through each cause with the exact commands to diagnose and fix it.

Fastest first check

SSH into the VM and run dig my-hostname @169.254.169.254. If that returns the correct IP, GCP DNS is working and the problem is in your application. If it fails, keep reading.

Simple explanation

DNS translates names like db.internal.example.com into IP addresses. Inside GCP, DNS resolution involves a few moving parts:

  • The VPC resolver (169.254.169.254): Every GCP VM automatically uses this as its DNS server. It resolves built-in VM hostnames, checks any Cloud DNS private zones attached to the VPC, and forwards everything else to Google public DNS. You never create it. For a deeper explanation, see DNS in GCP.

  • Cloud DNS private zones: Zones you create to host internal DNS records visible only inside specific VPCs. See Private DNS Zones for setup details.

  • Forwarding zones: Zones that do not store records. Instead, they forward queries for a domain to external DNS servers, such as on-premises resolvers reachable over Cloud VPN or Cloud Interconnect.

  • DNS peering zones: Zones that delegate queries to the VPC resolver of another VPC network. This is how you share DNS resolution across VPCs without duplicating zone associations.

Analogy

Think of the VPC resolver as a building’s front desk. It knows every room number (internal VM names) by heart and can look up any outside phone number (public DNS) on request. Cloud DNS private zones are like a private directory the front desk keeps in a locked drawer. Only people in approved departments (authorized VPCs) can ask the desk to look up numbers from that directory. A forwarding zone is a sticky note saying “for anything at Corp HQ, call this outside number.” A peering zone says “for anything in Building B, ask Building B’s front desk.”

Most DNS problems come from one of these layers being misconfigured. The troubleshooting steps below work through them in order of likelihood.

What this problem usually looks like

  • A VM can reach an IP address directly but cannot resolve the hostname for that IP
  • Public names like www.google.com resolve but private names do not
  • A name resolves in one VPC but returns NXDOMAIN from a peered VPC
  • An old IP address is still returned after you updated the DNS record
  • dig returns the correct answer but the application still fails
  • Resolution works from one VM but not from another in the same VPC

Start here: 60-second diagnosis

SymptomLikely causeFirst check
Nothing resolves (public or private)VM not using the GCP resolvercat /etc/resolv.conf (should show 169.254.169.254)
Public names work, private names failPrivate zone not attached to this VPCgcloud dns managed-zones list —filter=“visibility:private”
Works in VPC-A, fails in peered VPC-BMissing DNS peering or zone associationCheck if VPC-B is in the zone’s authorized networks
Old IP returned after record changeTTL cachingdig name @169.254.169.254 (check the TTL value in the answer)
dig works but application failsApplication-level DNS cache or resolverRestart the application process and retest
On-prem names fail from GCP VMForwarding zone missing or target unreachableCheck forwarding zone exists and target IP is reachable via VPN/Interconnect

How DNS resolution works in GCP

Every GCP VM is configured via DHCP to use 169.254.169.254 as its DNS server. This is the metadata server, and it runs a DNS resolver for the VM’s VPC. When a VM queries a name, the resolver follows this order:

  1. Internal Compute Engine names: Names matching the pattern INSTANCE.ZONE.c.PROJECT.internal are resolved automatically. No Cloud DNS zone is needed.

  2. Private Cloud DNS zones: If a private zone is attached to this VPC and the queried name falls under the zone’s DNS suffix, the resolver returns records from that zone. When multiple zones match, the resolver picks the one with the longest matching suffix. For example, a query for db.us.internal.example.com matches us.internal.example.com. before internal.example.com.

  3. DNS peering zones: If a peering zone matches, the resolver delegates the query to the target VPC’s resolver.

  4. Forwarding zones: If a forwarding zone matches, the query is sent to the configured external DNS server.

  5. Public DNS: Everything else is resolved via Google’s public DNS infrastructure.

Zone shadowing trap

A misconfigured zone can intercept queries you expect to go elsewhere. For example, a private zone for example.com. attached to your VPC will shadow all public example.com records. If public domains suddenly stop resolving, check whether a broad private zone is swallowing those queries.

Step 1: Verify the VM is using the correct resolver

GCP VMs should use 169.254.169.254 as their DNS server. If someone has manually edited /etc/resolv.conf, installed a local caching resolver, or changed DHCP settings, queries may bypass the GCP resolver entirely.

# Check the configured DNS server
cat /etc/resolv.conf
# Expected output includes: nameserver 169.254.169.254

# Verify the metadata server is reachable
curl -s -H "Metadata-Flavor: Google" http://169.254.169.254/

# Test DNS resolution against the GCP resolver explicitly
dig my-service.internal.example.com @169.254.169.254

# Compare with public DNS to rule out a broader DNS outage
dig www.google.com @169.254.169.254

How to interpret the results:

  • If /etc/resolv.conf shows a different nameserver (not 169.254.169.254), the VM is using a non-default resolver. This is the problem. Fix the resolver config or remove whatever overrode it.

  • If the metadata server is unreachable, the VM’s network configuration is broken at a lower level. Check the VPC connectivity troubleshooting guide.

  • If www.google.com resolves but your private name does not, the resolver is working but the private zone configuration has an issue. Continue to Step 2.

  • If nothing resolves at all, the VM’s connection to the metadata server may be blocked. Verify routing and firewall rules allow traffic to 169.254.169.254.

Step 2: Verify the private zone is visible to this VPC

A private DNS zone only resolves for VMs in VPCs that are listed as authorized networks on the zone. If the zone exists but your VM’s VPC is not in the list, queries for names in that zone return NXDOMAIN or fall through to public DNS.

# List all private zones and their associated VPCs
gcloud dns managed-zones list \
  --filter="visibility:private" \
  --format="table(name,dnsName,privateVisibilityConfig.networks)"

# Describe a specific zone to see its authorized networks
gcloud dns managed-zones describe my-private-zone \
  --format="yaml(privateVisibilityConfig)"

# Add a VPC to an existing private zone
gcloud dns managed-zones update my-private-zone \
  --networks=my-vpc-1,my-vpc-2
Destructive flag

The —networks flag replaces the entire list of authorized networks. Always include all VPCs that need access, not just the one you are adding. If you omit an existing VPC, it loses access to the zone immediately.

Step 3: Verify the record exists and the queried name matches the zone

# List all records in a zone
gcloud dns record-sets list --zone=my-private-zone

# Check for a specific record
gcloud dns record-sets list \
  --zone=my-private-zone \
  --filter="name=my-service.internal.example.com."

Things to check:

  • Trailing dot: Cloud DNS stores record names with a trailing dot (my-service.internal.example.com.). The gcloud CLI adds it automatically in most contexts, but Terraform and direct API calls require it explicitly. A missing trailing dot can cause the record to be created under the wrong FQDN.

  • Suffix matching: The queried name must fall under the zone’s DNS name. If your zone is internal.example.com. but you query my-service.corp.example.com, the zone does not match and the resolver skips it. Double-check that the record’s domain suffix exactly matches the zone.

  • Zone shadowing: If two private zones are attached to the same VPC and both could match a query, the zone with the longest matching suffix wins. An overly broad zone (like example.com.) can shadow more specific zones if misattached.

Quick record sanity check

Run gcloud dns record-sets list —zone=my-private-zone and visually confirm that every record name ends with the zone’s DNS name. If a record looks like my-service.internal.example.com.internal.example.com., someone accidentally included the zone suffix when creating the record.

# Add a new A record to a zone
gcloud dns record-sets create my-service.internal.example.com. \
  --type=A \
  --ttl=300 \
  --rrdatas="10.0.0.5" \
  --zone=my-private-zone

# Update an existing record
gcloud dns record-sets update my-service.internal.example.com. \
  --type=A \
  --ttl=300 \
  --rrdatas="10.0.0.10" \
  --zone=my-private-zone

Step 4: Fix cross-VPC DNS correctly

This is the most commonly misunderstood part of GCP DNS. VPC network peering exchanges routes so VMs can communicate by IP. It does not share DNS visibility. A VM in VPC-B cannot resolve names from a private zone attached to VPC-A just because the two VPCs are peered.

Analogy

VPC peering is like connecting two office buildings with a hallway. People can walk between them. But each building still has its own phone directory. Just because you can physically walk to Building B does not mean Building A’s front desk knows Building B’s extension numbers. You need to either copy the directory (add both VPCs to the zone) or set up call forwarding (DNS peering).

You have two options to fix this:

Option A: Add both VPCs to the private zone

The simplest approach. Associate the private zone with every VPC that needs to resolve its records:

# Associate the zone with both VPCs
gcloud dns managed-zones update my-private-zone \
  --networks=vpc-a,vpc-b

This works well when you own both VPCs and the zone. It does not work across organizations or when you want the target VPC to control its own DNS resolution.

Option B: Create a DNS peering zone

DNS peering delegates resolution for a specific domain to the VPC resolver of another network. The target VPC resolves the query using its own private zones. This is useful when the target VPC owns the zone and you do not want to (or cannot) modify its zone associations.

# Create a DNS peering zone in vpc-a that delegates to vpc-b's resolver
gcloud dns managed-zones create dns-peer-to-vpc-b \
  --dns-name=internal.example.com. \
  --visibility=private \
  --networks=vpc-a \
  --target-network=vpc-b \
  --target-project=my-project \
  --description="DNS peering: delegate internal.example.com to vpc-b"

The —target-network flag is what makes this a peering zone rather than a forwarding zone. The query is handed off to VPC-B’s resolver, which then checks VPC-B’s private zones.

Key distinction

VPC peering = route exchange for IP connectivity.
DNS peering = query delegation for name resolution.
They are independent features. Having one does not give you the other. If you need both IP connectivity and name resolution across VPCs, you must configure both. For centralized networking with shared DNS, consider Shared VPC, which avoids this problem entirely by keeping all resources in one VPC.

Step 5: Hybrid and on-premises forwarding checks

In hybrid architectures, GCP VMs may need to resolve on-premises hostnames. A forwarding zone directs queries for a specific domain to an on-premises DNS server:

# Create a forwarding zone for on-premises DNS
gcloud dns managed-zones create onprem-forwarding \
  --dns-name=corp.internal. \
  --visibility=private \
  --networks=my-vpc \
  --forwarding-targets=192.168.1.53 \
  --description="Forward corp.internal to on-premises DNS"

If on-premises names are not resolving, check:

  1. Network reachability: The target DNS server must be reachable from the VPC over Cloud VPN or Cloud Interconnect. Test with dig @192.168.1.53 hostname.corp.internal from a VM. If it times out, the issue is network connectivity, not DNS config.

  2. Forwarding zone association: The forwarding zone must be attached to the VPC where the querying VM lives.

  3. On-premises firewall: The on-premises DNS server must accept queries from GCP IP ranges. Cloud DNS sends forwarded queries from the 35.199.192.0/19 range, which must be allowed through any on-premises firewalls.

Common pitfall

If your forwarding zone target is an IP on your corporate network, it must be reachable from GCP’s forwarding infrastructure, not just from your VM. Cloud DNS sends forwarded queries from the 35.199.192.0/19 range. Your on-prem firewall needs a rule allowing DNS traffic (UDP/TCP 53) from this range, even if VM-to-server connectivity works fine over VPN.

Step 6: TTL and caching

You changed a DNS record but the old IP is still being returned. This is not a bug. DNS resolvers cache records for the duration of the TTL (time to live). Until the TTL expires, clients continue to see the old value.

Analogy

TTL is like a “best before” date on a cached answer. If you ask the front desk for a phone number, they write it on a sticky note and keep it for 5 minutes (the TTL). During those 5 minutes, they hand you the sticky note instead of checking the directory again. If someone updated the directory in the meantime, you still get the old number until the sticky note expires.

# Check the current TTL on a record
dig my-service.internal.example.com @169.254.169.254
# The number in the ANSWER SECTION is the remaining TTL in seconds

How to handle planned DNS changes:

  1. Before the change: Lower the TTL to a short value (e.g., 60 seconds) and wait for the old TTL to expire. If the old TTL was 300 seconds, wait at least 5 minutes after lowering it.

  2. Make the change: Update the record to the new IP.

  3. After the change: Once everything is confirmed working, raise the TTL back to your normal value.

# Lower TTL before a planned change
gcloud dns record-sets update my-service.internal.example.com. \
  --type=A \
  --ttl=60 \
  --rrdatas="10.0.0.5" \
  --zone=my-private-zone

# Wait for old TTL to expire, then update the IP
gcloud dns record-sets update my-service.internal.example.com. \
  --type=A \
  --ttl=60 \
  --rrdatas="10.0.0.10" \
  --zone=my-private-zone

# After confirming, restore normal TTL
gcloud dns record-sets update my-service.internal.example.com. \
  --type=A \
  --ttl=300 \
  --rrdatas="10.0.0.10" \
  --zone=my-private-zone
Application-level caching

Cloud DNS default TTL is 300 seconds (5 minutes), but applications can add their own caching layer on top. Java caches DNS results indefinitely by default unless you configure networkaddress.cache.ttl in the JVM. If you lowered the TTL and waited but the application still sees the old IP, restart the application process to flush its internal cache.

When to use this guide

Use this page when:

  • A VM or application cannot resolve a specific hostname
  • DNS works from one VPC but not another
  • A DNS record change is not taking effect
  • On-premises names do not resolve from GCP VMs

Go to a different page when:

Common beginner mistakes

  1. Creating a private zone but not associating it with the VPC. A private zone with no authorized networks is invisible to all VMs. Always add the target VPC when creating or updating a zone.

  2. Confusing VPC peering with DNS peering. VPC peering shares routes. DNS peering shares name resolution. They are independent. If you peer two VPCs but do not configure DNS peering (or add the second VPC to the zone), private names will not resolve across the peering connection.

  3. Debugging application code before testing with dig. Always confirm DNS works at the OS level first. Run dig name @169.254.169.254. If dig succeeds, the problem is in the application’s DNS cache or resolver library, not in GCP.

  4. Querying a name that does not match the zone suffix. If your zone is internal.example.com. but you query db.corp.example.com, the zone does not match. Check that the record’s FQDN falls under the zone’s DNS name.

  5. Assuming a TTL change takes effect instantly. Lowering the TTL on a record only affects future caches. Clients that already cached the old record continue to use it until the old TTL expires. Lower the TTL before you plan to change the IP, not at the same time.

  6. Forgetting the trailing dot in API and Terraform calls. Cloud DNS record names end with a dot. The gcloud CLI adds it automatically, but Terraform and the REST API require it explicitly. A missing dot can create the record under the wrong name.

Resolver vs Cloud DNS vs forwarding vs peering

FeatureWhat it doesStores records?When to use
VPC resolverBuilt-in DNS server at 169.254.169.254 in every VPCNo (resolves from other sources)Always active, no setup needed
Private zoneHosts DNS records visible only to authorized VPCsYesCustom internal hostnames (e.g., db.internal.example.com)
Forwarding zoneSends queries for a domain to an external DNS serverNoResolving on-prem names from GCP over VPN/Interconnect
DNS peering zoneDelegates queries to the VPC resolver of another networkNoCross-VPC resolution without modifying zone associations

For a full walkthrough of each zone type, see Private DNS Zones.

Related topics

These pages cover the networking and DNS concepts referenced in this guide.

Frequently asked questions

Why does dig work but my application still fails to resolve?

If dig returns the correct IP but your application cannot resolve the same name, the problem is not GCP DNS. The application is either using its own DNS cache, a different resolver, or a language-specific resolver library that behaves differently from the system resolver. In Java, the JVM caches DNS results aggressively by default. In Go, the pure-Go resolver may read /etc/resolv.conf differently. Check your application runtime DNS settings and restart the process after confirming dig works.

Why does DNS resolve in one VPC but not a peered VPC?

VPC peering shares routes, not DNS. A private DNS zone is only visible to VPCs listed in its authorized networks. A VM in a peered VPC has no access to the other VPC private zones unless you either add the peered VPC as an authorized network on the zone, or create a DNS peering zone that delegates queries to the other VPC resolver.

Why do I still get the old IP after updating the DNS record?

DNS resolvers cache records for the duration of the TTL. If the old record had a TTL of 300 seconds, clients that cached it will keep returning the old IP for up to 5 minutes. The fix is to lower the TTL before making the change, wait for the old TTL to expire, then update the record. After the change, clients pick up the new value within the new (shorter) TTL.

Do I need Cloud DNS for internal VM-to-VM hostname resolution?

No. Every GCP VPC includes a built-in resolver at 169.254.169.254 that automatically resolves Compute Engine VM hostnames in the format INSTANCE_NAME.ZONE.c.PROJECT_ID.internal. You only need Cloud DNS when you want custom internal names like db.internal.example.com, or when you need forwarding zones or DNS peering.

What is the difference between a private zone, a forwarding zone, and a peering zone?

A private zone stores DNS records in Cloud DNS and serves them to authorized VPCs. A forwarding zone does not store records. It sends queries for a domain to external DNS servers you specify, such as on-premises resolvers reachable over VPN or Interconnect. A peering zone also stores no records. It delegates queries to the VPC resolver of another VPC network in your project or organization, letting that network resolve the name using its own private zones.

Last verified: 27 March 2026 Cloud services change frequently. Verify details against official documentation before making infrastructure decisions.