AWS VPC Network Troubleshooting Guide: Routes, Security Groups, NACLs, NAT and ALB Errors

Why AWS networking fails: the plain English version

Most AWS network failures fall into one of two buckets. Knowing which bucket you are in changes where you look first.

Bucket one: traffic cannot get there. A packet is never delivered to the destination. There is no route for it, there is no Internet Gateway or NAT Gateway to send it through, or the instance is in the wrong subnet with no way out. The connection times out.

Bucket two: traffic gets there but is blocked. The packet arrives, but a firewall rule drops it at the destination. Security groups act at the instance level and are stateful. Network ACLs act at the subnet level and are stateless. Either one can silently drop traffic.

The reason this matters: if you spend 30 minutes checking security groups when the real problem is a missing route, you will be just as stuck at the end. Work through each layer in order. VPC networks, subnets, route tables, private and public IPs, NAT, DNS, and the application itself all play a role.

How traffic moves through a VPC

Every packet travelling between two endpoints in AWS passes through this sequence:

Source (EC2, Lambda, ECS task)
  → Route table on source subnet: does a route exist for the destination?
  → NACL on source subnet (outbound): is this outbound traffic allowed?
  → Network path (internet, peering, VPC internal)
  → NACL on destination subnet (inbound): is this inbound traffic allowed?
  → Security group on destination: is this traffic allowed?
  → Application on destination: is the process listening on the expected port?

If any step is missing or blocked, the connection fails. The symptom looks identical regardless of where the break is. That is why working through the layers systematically saves time.

Quick symptom-to-cause lookup

When to use this page

This guide is for:

• EC2 instances that cannot reach the internet or other VPC resources
• SSH or Session Manager connectivity failures
• Lambda functions timing out on database or private resource connections
• ALB targets showing as unhealthy or returning 5xx errors
• Traffic that works in one subnet or AZ but fails in another
• Security group and NACL debugging

This is not the right page if:

• The issue is IAM permissions rather than network access. Check IAM policies and resource-based policies first.
• The issue is application authentication (wrong password, expired token, TLS cert mismatch).
• The issue is a service quota limit rather than connectivity.
• You are debugging DNS specifically. The DNS with Route 53 guide covers that in more depth.

Core troubleshooting workflow

Follow this order before making any changes. Skipping steps and making random adjustments is how misconfigurations accumulate.

1. Confirm the exact symptom. Is it a timeout, a refused connection, or an error response? Timeout usually means routing or firewall. Refused means the host was reached but the port is not open. An error response means the app received the request.
2. Identify source and destination. What is the source IP, instance ID, or resource? What is the destination: another EC2 instance, an RDS endpoint, an ALB, or the internet?
3. Check the route path. Does the source subnet’s route table have a valid route for the destination? Public subnet internet traffic routes to an Internet Gateway. Private subnet internet traffic routes to a NAT Gateway.
4. Check NACLs. Does the NACL on the source subnet allow outbound? Does the NACL on the destination subnet allow inbound? Does the source subnet NACL allow inbound ephemeral return traffic (ports 1024-65535)?
5. Check security groups. Does the destination’s security group have an inbound rule matching this traffic? Does the source security group allow the outbound connection?
6. Check the destination. Is the service running? Is it listening on the expected port? Is the instance healthy?
7. Use Flow Logs and Reachability Analyzer. Enable VPC Flow Logs to see ACCEPT and REJECT decisions. Use Reachability Analyzer to trace the exact blocking element.
8. Make one targeted change at a time. Each change should test one specific hypothesis. Reverting is easier when only one thing changed.

Scenario: EC2 instance cannot reach the internet

You run curl https://example.com from an EC2 instance and it times out. The fix depends on whether the instance is in a public subnet or a private subnet.

Public subnet: instance cannot reach internet

A public subnet routes internet traffic directly through an Internet Gateway. For this to work, all four of these must be true at the same time:

1. The instance has a public IP or Elastic IP. Without this, the Internet Gateway cannot route return traffic back to the instance. Check EC2, then Instances, then the Networking tab.
2. The subnet route table has a 0.0.0.0/0 route pointing to an Internet Gateway (igw-xxxxxxxx). Check VPC, then Route Tables, then the Routes tab for the subnet’s associated table.
3. The security group allows outbound traffic on the relevant port (port 443 for HTTPS). Default security groups allow all outbound. Check if yours was restricted.
4. The NACL allows both outbound traffic and inbound ephemeral return ports (1024-65535). If you restricted the default NACL, return traffic will be silently dropped even though the request went out.

# Check the route table for a specific subnet
aws ec2 describe-route-tables \
  --filters "Name=association.subnet-id,Values=subnet-0abc123def456789a" \
  --query 'RouteTables[].Routes'

# Confirm the instance has a public IP
aws ec2 describe-instances \
  --instance-ids i-0abc123def456789a \
  --query 'Reservations[].Instances[].[PublicIpAddress,SubnetId,State.Name]'

Private subnet: instance cannot reach internet

Private subnet instances route internet traffic through a NAT Gateway. Check these in order:

1. The subnet route table has a 0.0.0.0/0 route pointing to a NAT Gateway (nat-xxxxxxxx). Do not route private subnets to an Internet Gateway. That breaks the private subnet model entirely.
2. The NAT Gateway is in Available state and is in a public subnet. A NAT Gateway placed in a private subnet cannot reach the internet. Check VPC, then NAT Gateways.
3. The public subnet where the NAT Gateway lives has its own route to the Internet Gateway. The NAT Gateway needs internet access to forward outbound traffic.
4. The instance security group allows outbound traffic.

# Check NAT Gateway status and subnet placement
aws ec2 describe-nat-gateways \
  --filter "Name=state,Values=available" \
  --query 'NatGateways[].[NatGatewayId,SubnetId,State]'

# Check the route table for the private subnet
aws ec2 describe-route-tables \
  --filters "Name=association.subnet-id,Values=subnet-0private123" \
  --query 'RouteTables[].Routes'

Scenario: cannot connect to EC2 via SSH

You run ssh ec2-user@<public-ip> and the connection fails. Read the error carefully before you start debugging, because the two most common errors point to completely different problems.

Checklist:

1. Is the instance in a public subnet with a public IP? Private subnet instances are not directly reachable from the internet. Use a bastion host or Session Manager instead.
2. Does the security group allow inbound TCP port 22 from your IP? Use EC2, then Security Groups, then Inbound rules. The console “My IP” option auto-populates your current public IP.
3. Does the NACL allow inbound port 22 and outbound ephemeral ports (1024-65535)? NACLs are stateless. You need both directions explicitly. The return traffic uses a high ephemeral port, not port 22.
4. Is the instance running? A stopped or terminated instance will time out.
5. Are you using the correct key pair? The key pair must match what was assigned when the instance was launched.
6. Are you using the correct username? Amazon Linux 2/2023 uses ec2-user. Ubuntu uses ubuntu. RHEL uses ec2-user or root. The wrong username gives a permission denied error that looks like a key problem.

# Check security group inbound rules for SSH
aws ec2 describe-security-groups \
  --group-ids sg-0abc123def456789a \
  --query 'SecurityGroups[].IpPermissions[?ToPort==`22`]'

# Confirm instance state and public IP
aws ec2 describe-instances \
  --instance-ids i-0abc123def456789a \
  --query 'Reservations[].Instances[].[State.Name,PublicIpAddress,KeyName]'

Scenario: Lambda cannot reach RDS

Your Lambda function times out when trying to connect to an RDS database. This is almost always a combination of VPC configuration and security group issues.

Checklist:

1. Is the Lambda function configured to run inside a VPC? By default, Lambda runs outside any VPC and cannot reach private VPC resources. Check Lambda, then Configuration, then VPC. If the VPC field is empty, Lambda cannot reach your RDS instance regardless of security group settings.
2. Is Lambda in the same VPC as RDS? They must share a VPC (or a peered VPC) to communicate over private IPs.
3. Does Lambda have subnets in multiple Availability Zones with available IP space? Lambda creates elastic network interfaces (ENIs) in your subnets. If a subnet runs out of IP addresses, Lambda invocations fail silently.
4. Does the RDS security group allow inbound from the Lambda security group on the database port? Use a security group reference (source = Lambda SG ID), not a CIDR range, so the rule works regardless of which IP Lambda gets. Port 5432 for PostgreSQL, 3306 for MySQL.
5. Does the Lambda security group allow outbound on the database port? The default allows all outbound. Check if it was restricted.
6. Is the RDS instance running? Check RDS, then Databases, then status.

# Check Lambda VPC configuration
aws lambda get-function-configuration \
  --function-name my-function \
  --query 'VpcConfig'

# Check RDS security group inbound rules
aws ec2 describe-security-groups \
  --group-ids sg-rds-database \
  --query 'SecurityGroups[].IpPermissions'

# Check available IPs in the Lambda subnet
aws ec2 describe-subnets \
  --subnet-ids subnet-0lambda123 \
  --query 'Subnets[].[SubnetId,AvailableIpAddressCount]'

Scenario: ALB returning 502 or 503

Your Application Load Balancer is returning errors to clients. These two codes mean different things and have different fixes.

502 Bad Gateway

The ALB reached a target but the target returned an invalid or unexpected response. Common causes:

• The application is not running on the target instance
• The application crashed and returned a non-HTTP response
• The ALB is forwarding to HTTPS on the target but the target has no valid TLS certificate
• The target is responding on the wrong port

Check application logs on the target instance and confirm the app is running and responding to requests directly, bypassing the ALB.

503 Service Unavailable

The ALB has no healthy targets to send traffic to. The target group is empty or all registered targets are failing health checks.

Checklist for 503:

1. Check target health status. Go to EC2, then Target Groups, then select the group, then the Targets tab. Look at the health check reason for each unhealthy target.
2. Is the health check path returning a 200 status? The ALB sends an HTTP request to a configurable path (default /). If your app returns 404 or 500 on that path, the target is marked unhealthy. Update the health check to a path your app reliably returns 200 on.
3. Does the target security group allow inbound traffic from the ALB security group on the application port? Use the ALB’s security group ID as the source, not 0.0.0.0/0.
4. Is the application listening on the port the target group sends to? Confirm directly on the instance.

# Check target health for a target group
aws elbv2 describe-target-health \
  --target-group-arn arn:aws:elasticloadbalancing:us-east-1:123456789012:targetgroup/my-tg/abc123

# Get the ALB's security group ID (use this as source in target SG rule)
aws elbv2 describe-load-balancers \
  --names my-alb \
  --query 'LoadBalancers[].[LoadBalancerName,SecurityGroups]'

# Verify the app port is open on the target instance (SSH in first)
ss -tlnp | grep <port>

Scenario: DNS and name resolution issues

If a hostname resolves to the wrong IP or does not resolve at all, traffic never reaches the right destination, even if your routing, NACLs, and security groups are all correct.

Common DNS problems in AWS:

• Using the wrong hostname. If an RDS instance is in a private subnet, its public DNS hostname may not resolve inside the VPC, or vice versa. Use the endpoint shown in the RDS console.
• Private hosted zone not associated with the VPC. A Route 53 private hosted zone only resolves for VPCs that are explicitly associated with it. If Lambda or EC2 is in a different VPC, the private records will not resolve.
• DNS resolution disabled on the VPC. VPC DNS resolution can be disabled in VPC settings. Check VPC, then Your VPC, then the DNS resolution and DNS hostnames attributes.

# Test DNS resolution from inside an instance (SSH in first)
nslookup mydb.cluster-xxxx.us-east-1.rds.amazonaws.com
dig mydb.cluster-xxxx.us-east-1.rds.amazonaws.com

# Check VPC DNS settings
aws ec2 describe-vpc-attribute \
  --vpc-id vpc-0abc123 \
  --attribute enableDnsSupport

aws ec2 describe-vpc-attribute \
  --vpc-id vpc-0abc123 \
  --attribute enableDnsHostnames

For detailed DNS troubleshooting, see the DNS with Route 53 guide.

Scenario: traffic reaches AWS but still fails

Sometimes the network path is fully working but the connection still fails. This means the problem is at the application layer, not the network layer.

Signs that it is an application problem, not a network problem:

• VPC Flow Logs show ACCEPT for the traffic (the packet was delivered)
• Reachability Analyzer says the path is reachable
• You can telnet or netcat to the port but the app returns an error

Common application-layer failures that look like network problems:

• The application is not listening on the expected port. Run ss -tlnp on the instance to confirm what is actually listening.
• The service crashed or was never started. Check application logs and service status (systemctl status myapp).
• TLS certificate mismatch. The connection reaches the app but the handshake fails because the certificate does not match the hostname.
• Application-level auth failure. Wrong database password, expired token, or a failed IAM authentication for RDS IAM auth.
• Health check path fails but the app works. The ALB marks the target unhealthy because the health check path returns a non-200 status, even though the actual application endpoints work fine.

If Flow Logs show traffic is being ACCEPTED, the network is not the problem. Check CloudWatch Logs for application errors.

AWS tools for network troubleshooting

VPC Flow Logs

Flow Logs capture metadata about traffic flowing through your VPC: source IP, destination IP, port, protocol, and whether the traffic was ACCEPT or REJECT. They do not capture packet content.

When to use: when you need to confirm whether traffic is arriving at a destination, identify which rule is blocking traffic, or audit network activity.

# Enable Flow Logs on a VPC, sending to CloudWatch Logs
aws ec2 create-flow-logs \
  --resource-type VPC \
  --resource-ids vpc-0abc123 \
  --traffic-type ALL \
  --log-destination-type cloud-watch-logs \
  --log-group-name /aws/vpc/flowlogs \
  --deliver-logs-permission-arn arn:aws:iam::123456789012:role/flow-logs-role

Then query rejected traffic in CloudWatch Logs Insights:

fields @timestamp, srcAddr, dstAddr, dstPort, action
| filter dstAddr = "10.0.1.50" and action = "REJECT"
| sort @timestamp desc
| limit 50

VPC Reachability Analyzer

Reachability Analyzer analyzes your VPC configuration — route tables, security groups, NACLs, peering connections — and tells you whether a path between two endpoints is theoretically possible. It does not send real traffic.

When to use: before traffic starts flowing, for new VPC configurations, for complex multi-VPC environments, or when you cannot reproduce a failure with real traffic.

# Create a network path between an EC2 instance and an RDS endpoint
aws ec2 create-network-insights-path \
  --source i-0abc123def456789a \
  --destination db-instance-endpoint \
  --destination-port 5432 \
  --protocol TCP

# Run the analysis
aws ec2 start-network-insights-analysis \
  --network-insights-path-id nip-0abc123def456789a

# Get results
aws ec2 describe-network-insights-analyses \
  --network-insights-analysis-ids nia-0abc123def456789a \
  --query 'NetworkInsightsAnalyses[0].{Reachable:NetworkPathFound,Blockers:Explanations}'

When NetworkPathFound is false, the Explanations field names the exact resource blocking the path.

Security Groups vs NACLs

Beginners often check the wrong one. Here is the key difference:

Security groups are usually the problem when a specific source is blocked or a rule is missing for a particular port. NACLs are usually the problem when traffic works for some connections but not others from the same subnet, or when you recently added NACL rules to tighten a subnet.

See Network ACLs and Security Groups Explained for rule syntax and examples.

Internet Gateway vs NAT Gateway

Using the wrong one — or pointing a route to the wrong one — is one of the most common reasons EC2 instances cannot reach the internet.

Common misconfiguration patterns:

• Routing a private subnet’s 0.0.0.0/0 to an Internet Gateway instead of a NAT Gateway. This makes the subnet effectively public and defeats the purpose of a private subnet.
• Placing the NAT Gateway in a private subnet. NAT Gateways must be in a public subnet with internet access.
• Forgetting to route the public subnet where NAT lives to the Internet Gateway. The NAT Gateway itself needs internet access to forward traffic.

For full setup details, see the NAT Gateway guide.