SSH into a Compute Engine VM: gcloud, IAP, OS Login, and Keys

SSH access in plain English

SSH (Secure Shell) is how you open a terminal session on a remote server. When you SSH into a Compute Engine VM, you are connecting to a Linux machine running in Google’s data centre and getting a command prompt on it.

In Compute Engine, SSH is not just about network connectivity. It also involves keys, firewall rules, IAM permissions, and optionally an identity proxy. There are four core concepts to understand before anything else:

• Keys: SSH uses public-key cryptography. Your local machine holds the private key; the VM holds the matching public key. They must pair correctly for the connection to succeed.
• Metadata: Compute Engine stores SSH public keys in VM metadata, either at the project level (shared across all VMs) or the instance level (one VM only).
• OS Login: An alternative to metadata keys. Access is granted via IAM roles tied to Google identities rather than static key files.
• IAP tunnelling: Identity-Aware Proxy lets you SSH into a VM with no external IP by routing the connection through Google infrastructure, without a VPN or bastion host.

For most teams, the right default is straightforward: use gcloud compute ssh day-to-day, enable OS Login for multi-user environments, and use IAP tunnelling whenever a VM does not need a public IP.

How SSH access works in Compute Engine

When you run gcloud compute ssh, several things happen behind the scenes:

1. gcloud checks whether you have a temporary SSH key pair on your local machine. If not, it generates one.
2. It uploads your public key to the VM’s metadata, either project-wide or instance-specific, with a short expiry window.
3. It opens a standard SSH connection to the VM’s external IP address on port 22, or through an IAP tunnel if you pass —tunnel-through-iap.
4. The VM’s SSH daemon reads the authorised keys from metadata and either allows or refuses the connection.

When OS Login is enabled, the flow changes. GCP checks the connecting user’s IAM permissions at connection time instead of looking up a key in metadata. This makes access control dynamic rather than static. Revoking an IAM role immediately cuts access without touching any key files.

For VMs on a private VPC with no external IP, the connection cannot reach port 22 directly from the internet at all. IAP tunnelling solves this by creating an authenticated channel through Google’s infrastructure. See private vs public IP addresses for more on when a VM actually needs an external IP.

The main ways to SSH into a Compute Engine VM

1. gcloud compute ssh

The standard approach for daily use. gcloud handles key generation and upload automatically. You do not need to set anything up first. It works for any VM with an external IP and an open firewall rule on port 22, or any VM when combined with —tunnel-through-iap.

2. Browser-based SSH in the Cloud Console

The Cloud Console includes a built-in SSH terminal you can open directly from the VM list. Click the SSH button next to any VM and a terminal opens in your browser. This works even if you have no local SSH tools installed. It uses the same key injection mechanism as gcloud: GCP temporarily adds a key to the VM’s metadata, opens the session, then removes the key.

Browser SSH is useful for quick one-off tasks and lab work. It is less practical for daily engineering where you want a persistent terminal with your own shell configuration.

3. IAP tunnelling for private VMs

If a VM has no external IP address, standard SSH cannot reach it from outside the VPC. Identity-Aware Proxy tunnelling creates a secure connection through Google infrastructure to the VM’s internal IP. No public internet exposure, no VPN required. This is the recommended approach for production VMs that should not be directly reachable from the internet.

The connecting user needs the roles/iap.tunnelResourceAccessor IAM role. Every connection is logged in Cloud Audit Logs.

4. Standard SSH with manually managed keys

If you need to connect using a standard SSH client rather than gcloud, you add your public key to the VM’s metadata directly. Keys can be added at the project level (applies to all VMs) or the instance level (one VM only). This approach requires you to manage key rotation and revocation manually.

5. OS Login

OS Login replaces metadata-based key management with IAM-based access control. SSH access is granted by assigning the roles/compute.osLogin role (standard user) or roles/compute.osAdminLogin role (sudo access). Revoking access is instant: remove the IAM binding and the user cannot connect on any VM where they held that role.

From the user’s perspective, gcloud compute ssh works exactly the same way. The difference is entirely behind the scenes: GCP validates IAM permissions on every connection rather than reading a static key from metadata.

Which method should you use?

The right choice depends on your environment and who needs access.

Solo learning or lab environment

Use gcloud compute ssh with the default settings. It requires zero configuration and works immediately after you create your first VM. For quick experiments, the browser SSH button in the console is also fine. Do not spend time on OS Login or IAP until you understand the basics.

Production team environment

Enable OS Login project-wide. Every developer gets their own Linux user account on each VM, access is controlled through IAM roles, and when someone leaves the team you remove their IAM binding. Access is revoked everywhere, instantly, with no key cleanup. Combine with IAP tunnelling so VMs do not need external IPs. This is the lowest-risk default for any team.

Private VM with no external IP

Use gcloud compute ssh —tunnel-through-iap. If the VM does not have an external IP, this is your primary option short of setting up a VPN or bastion host. IAP is simpler than both, built into GCP, and produces automatic audit logs. Grant the roles/iap.tunnelResourceAccessor role to anyone who needs access.

Standard SSH client requirement

If your workflow requires a standard SSH client, for example a CI/CD system that cannot use gcloud, use instance-level metadata keys rather than project-level. Add each user’s public key to the specific VM they need, and block project-level key inheritance with block-project-ssh-keys=true. This limits the blast radius if a key is ever compromised.

When to use SSH access

• Debugging a running application: SSH in, check logs, inspect processes, then exit. No persistent access needed.
• Running maintenance scripts: SSH in to run a one-off command or patch a package. Consider startup scripts for repeated tasks that do not need a person present.
• Setting up a VM for the first time: Initial software installation, configuration, and testing typically happens over SSH before automation is in place.
• Accessing a database or internal service: Connect to a private VM with no external IP using IAP tunnelling to reach an internal port-forwarded resource.
• Incident response: When something is wrong on a production VM, SSH is often the fastest path to diagnosis.

Step-by-step examples

Basic SSH with gcloud

The simplest case. SSH into a VM that has an external IP. gcloud handles keys automatically.

# SSH into a VM interactively
gcloud compute ssh my-vm --zone=us-central1-a

# SSH and run a single command without opening an interactive session
gcloud compute ssh my-vm --zone=us-central1-a -- "sudo systemctl status nginx"

# SSH as a specific user
gcloud compute ssh alice@my-vm --zone=us-central1-a

# Copy a file to the VM using SCP through gcloud
gcloud compute scp ./config.yaml my-vm:/home/alice/ --zone=us-central1-a

IAP tunnelling for a private VM

If the VM has no external IP, add —tunnel-through-iap. You also need the IAP API enabled and the correct IAM role granted before this will work.

# Enable the IAP API if not already enabled
gcloud services enable iap.googleapis.com

# Grant yourself the IAP tunnel access role
gcloud projects add-iam-policy-binding PROJECT_ID \
  --member=user:you@example.com \
  --role=roles/iap.tunnelResourceAccessor

# SSH into the VM through the IAP tunnel
gcloud compute ssh my-private-vm \
  --zone=us-central1-a \
  --tunnel-through-iap

Adding SSH keys manually

Use this when you need a standard SSH client to connect. Prefer instance-level keys over project-level to limit scope.

# Add a key to a specific VM only (recommended over project-level)
gcloud compute instances add-metadata my-vm \
  --zone=us-central1-a \
  --metadata=ssh-keys="alice:$(cat ~/.ssh/id_ed25519.pub)"

# Block project-level keys from applying to this VM
gcloud compute instances add-metadata my-vm \
  --zone=us-central1-a \
  --metadata=block-project-ssh-keys=true

# Add a key to project metadata (applies to all VMs — use with caution)
gcloud compute project-info add-metadata \
  --metadata=ssh-keys="alice:$(cat ~/.ssh/id_ed25519.pub)"

Setting up OS Login

Enable OS Login project-wide, then grant IAM roles to the users who need SSH access.

# Enable OS Login for all VMs in the project
gcloud compute project-info add-metadata \
  --metadata=enable-oslogin=TRUE

# Grant a developer standard SSH access (no sudo)
gcloud projects add-iam-policy-binding PROJECT_ID \
  --member=user:developer@example.com \
  --role=roles/compute.osLogin

# Grant an admin SSH access with sudo
gcloud projects add-iam-policy-binding PROJECT_ID \
  --member=user:admin@example.com \
  --role=roles/compute.osAdminLogin

# Revoke access immediately when someone leaves
gcloud projects remove-iam-policy-binding PROJECT_ID \
  --member=user:former-employee@example.com \
  --role=roles/compute.osLogin

SSH access methods compared

OS Login vs SSH keys

This is one of the most common questions for engineers new to Compute Engine. Here is the clearest way to think about it.

When OS Login is the better choice

• Multiple people need SSH access to the same VMs
• You need to revoke access quickly when team members leave
• You want SSH access decisions to be part of your IAM audit trail
• You are running a compliance-conscious environment

When manual keys still appear

• CI/CD systems that cannot use gcloud and need static credentials to authenticate
• Legacy tooling that expects a specific key-based workflow
• Single-user scenarios where key management overhead is negligible
• Specific VMs opted out of OS Login via enable-oslogin=FALSE on the instance

For a full walkthrough of enabling and managing OS Login, see OS Login explained.

Security best practices

Use IAP tunnelling for production VMs

Remove external IPs from production VMs where possible and route all SSH access through IAP tunnelling. This eliminates the attack surface of port 22 being reachable from the public internet. See private vs public IP addresses for guidance on when a VM actually needs an external IP.

Enable OS Login for teams

For any environment with more than one person needing SSH access, OS Login is the right default. It ties access to Google identities, makes revocation instant, and produces an automatic audit trail in Cloud Audit Logs.

Follow least privilege for IAM roles

Grant roles/compute.osLogin (no sudo) by default. Grant roles/compute.osAdminLogin only to specific individuals with a documented need for root access. Grant roles/iap.tunnelResourceAccessor at the resource level on specific VMs rather than the project level where possible.

Tighten the default firewall rule

The default VPC firewall rule allows port 22 from all source IPs. For any project beyond a short-lived experiment, delete or disable this rule and replace it with either a rule restricted to your office IP range, or no rule at all combined with IAP tunnelling. The network tags system makes it easy to apply tighter firewall rules to specific VMs.

Audit SSH access regularly

Review project-level SSH keys periodically and remove any that belong to former employees or expired access. If using OS Login, review IAM bindings for compute.osLogin and compute.osAdminLogin across the project. Cloud Audit Logs records all IAP tunnel connections and OS Login sessions for incident investigation.

Frequently asked questions

How do I SSH into a Compute Engine VM?

Run gcloud compute ssh VM_NAME —zone=ZONE. gcloud generates SSH keys automatically and opens the connection. If the VM has no external IP, add —tunnel-through-iap.

How do I SSH into a VM with no external IP?

Use IAP tunnelling: gcloud compute ssh VM_NAME —zone=ZONE —tunnel-through-iap. You need the IAP API enabled and the roles/iap.tunnelResourceAccessor role on your account. No VPN or bastion host required.

What is the difference between OS Login and SSH keys?

SSH keys stored in metadata are static credentials you manage manually. OS Login uses IAM roles tied to Google identities. Access is granted and revoked through IAM rather than key files. For teams, OS Login is simpler to manage and more auditable.

Is opening port 22 required?

Not if you use IAP tunnelling. IAP routes the connection through Google infrastructure so port 22 never needs to be open to the public internet. If you do need port 22 open for a standard SSH client, restrict the source range to known IP addresses rather than allowing all traffic.

Where are Compute Engine SSH keys stored?

In VM metadata. Project-level keys are stored in project metadata and propagate to all VMs in the project. Instance-level keys are stored in the individual VM’s metadata. With OS Login enabled, keys move out of metadata and are managed through the user’s OS Login profile instead.

Can multiple users SSH into the same VM safely?

Yes. With OS Login, each user gets their own Linux account on the VM. No shared credentials. With metadata keys, you add each user’s public key individually. The risk with metadata keys at scale is that they persist indefinitely. OS Login solves this since access is removed by revoking an IAM role.