Blue Green Deployment in GCP: Cloud Run, GKE, Rollbacks

Simple explanation

In deployment terms: blue is your current live production version. Green is the new version you have deployed but not yet exposed to users. You validate green, then flip a switch to make it the live version. Blue stays running and ready so you can flip back instantly.

What is a blue green deployment in GCP?

A blue green deployment maintains two complete, production-equivalent environments at the same time. One environment (blue) serves all live user traffic. The other (green) runs the new version of your application, fully deployed, fully configured, connected to the same production dependencies, but receiving no user traffic yet.

When you are ready to release, you move all traffic from blue to green in one step. The switch is instantaneous from the user’s perspective: one request hits blue, the next hits green, with no in-between state where both versions might handle the same session unpredictably.

Because blue stays running after the switch, rollback is just the reverse operation. There is no redeployment, no image pull, no cold start wait. You shift traffic back to the previous version and it is live again within seconds.

How blue green deployments work

The sequence looks like this:

1. Deploy the new version (green) without traffic. Green runs and is fully started but receives zero user requests. On Cloud Run this uses —no-traffic. On GKE, a separate Deployment runs alongside blue with a different version label.
2. Validate green in isolation. Run smoke tests, integration checks, and health checks against the green environment directly. Because it receives no production traffic, you can test thoroughly without any user impact. See the CI/CD pipeline guide for how to automate this step.
3. Switch all traffic to green. One command moves 100% of traffic from blue to green. Users hit green from this point on.
4. Monitor the post-switch window. Watch error rates and latency for the first few minutes. Automated checks against Cloud Monitoring alerts can trigger automatic rollback if metrics degrade.
5. Roll back if needed. If anything looks wrong, shift traffic back to blue instantly. No redeployment required.
6. Clean up. Once green is confirmed stable (typically after 24 hours or more), decommission blue.

The key distinction from canary: in blue green, green is tested in isolation before any users see it. In a canary, the new version gets a small slice of real user traffic as a validation mechanism. Blue green is faster and simpler; canary provides more evidence from real traffic at the cost of more operational complexity. See canary deployments for when to choose that path instead.

Blue green on Cloud Run

Cloud Run’s revision model makes blue green straightforward. Every deploy creates a new revision, and you control traffic allocation between revisions explicitly. The Cloud Run overview explains the revision model in more detail if this is new to you.

# Step 1: Deploy the new revision with no production traffic
# --no-traffic means this revision starts but receives zero requests
# --tag gives it a stable, predictable URL for testing
gcloud run deploy api-service \
  --image=europe-west2-docker.pkg.dev/my-app-prod/api/api:v2.0.0 \
  --region=europe-west2 \
  --no-traffic \
  --tag=green

# Green is now accessible for testing at its stable tagged URL:
# https://green---api-service-xxxx-ew.a.run.app
# This URL is stable and persists until you remove the tag

# Step 2: Run your validation suite against the tagged URL
# Smoke tests, integration checks, dependency verification
# (these should be automated in your pipeline)

# Step 3: Switch all traffic to green instantly
gcloud run services update-traffic api-service \
  --region=europe-west2 \
  --to-revisions=LATEST=100

# Step 4 (if needed): List revisions to find the previous blue revision name
gcloud run revisions list \
  --service=api-service \
  --region=europe-west2

# Roll back to the previous revision by name
gcloud run services update-traffic api-service \
  --region=europe-west2 \
  --to-revisions=api-service-00049-xyz=100

Cloud Run billing during blue green

Cloud Run charges for actual request processing: CPU time, memory, and requests served. A revision that receives no traffic is not billed for request processing.

After switching, the old blue revision goes idle and its request billing drops to zero. It stays available for rollback without generating costs, unless it also has minimum instances configured.

Blue green on GKE

On GKE, there is no built-in revision model, so you manage it yourself with two Deployments and a Service selector switch. The Service routes traffic to pods based on label selectors. Changing the selector is the traffic switch:

# The Service selects pods by the 'version' label
# Change 'blue' to 'green' to switch all traffic
apiVersion: v1
kind: Service
metadata:
  name: api-service
  namespace: production
spec:
  selector:
    app: api
    version: blue   # <-- this is the only thing that changes during a switch
  ports:
    - port: 80
      targetPort: 8080

# Blue Deployment: current live version
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-blue
  namespace: production
spec:
  replicas: 10
  selector:
    matchLabels:
      app: api
      version: blue
  template:
    metadata:
      labels:
        app: api
        version: blue
    spec:
      containers:
        - name: api
          image: europe-west2-docker.pkg.dev/my-app-prod/api/api:v1.9.0
---
# Green Deployment: new version, not yet selected by the Service
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-green
  namespace: production
spec:
  replicas: 10
  selector:
    matchLabels:
      app: api
      version: green
  template:
    metadata:
      labels:
        app: api
        version: green
    spec:
      containers:
        - name: api
          image: europe-west2-docker.pkg.dev/my-app-prod/api/api:v2.0.0

# Switch all traffic to green by patching the Service selector
kubectl patch service api-service -n production \
  -p '{"spec":{"selector":{"app":"api","version":"green"}}}'

# Rollback: patch back to blue
kubectl patch service api-service -n production \
  -p '{"spec":{"selector":{"app":"api","version":"blue"}}}'

GKE capacity and cost tradeoffs

See monitoring GKE for how to watch both Deployments during the switch window. This is one of the key operational differences from Cloud Run blue green: on Cloud Run the cost concern is minimal, but on GKE it requires deliberate scaling management.

When to use blue green deployments

Good fits for blue green

• High-risk releases. Major version changes, database schema migrations, significant refactors. Anything where you want instant rollback available.
• User-facing services where downtime is unacceptable. Blue green gives you a clean switch with no mixed-version window.
• Changes that are hard to canary safely. If the new version is incompatible with the old version during a gradual rollout (for example, a breaking API change), blue green is cleaner than a canary.
• Regulated or compliance-sensitive workloads. Blue green makes it easy to demonstrate that only one known version was serving traffic at any given time.
• Teams with strong pre-deploy validation automation. The more automated your smoke tests and integration checks, the safer the full-traffic switch becomes.

When to choose something else

• Low-risk, high-frequency deploys. Dependency updates, config tweaks, minor bug fixes. A standard rolling deployment is simpler and fast enough.
• When you need production traffic to validate. If you cannot fully trust pre-switch testing and want real-user signal before committing, use a canary deployment instead.
• When validation automation is weak. Blue green exposes 100% of users to the new version in one step. Without meaningful pre-switch validation, you are just doing a faster, less safe rolling deploy.
• Complex stateful migrations where two-version compatibility is impossible. Some changes cannot safely run against the same database or state store at the same time. In those cases, a maintenance window may be unavoidable.
• GKE workloads with tight budget constraints. If running two Deployments simultaneously is not cost-feasible, consider a rolling deployment with a pause-and-validate checkpoint instead.

The dev vs staging vs production guide is useful context here. Blue green works best when your staging environment already mirrors production closely enough to catch most issues before the green environment is even built.

Blue green vs canary vs rolling deployments

These three strategies solve the same problem differently. Knowing when to reach for each one is part of designing a solid release process.

The short version: blue green is the right choice when your pre-switch testing is thorough and you want the cleanest possible rollback. Canary is better when the change is risky enough that you want production signal before going to 100%. Rolling is fine for routine, low-risk deploys where simplicity and low cost matter more than instant rollback.

Validating green before switching

The safety of blue green depends entirely on what you check before switching. A careless switch is worse than a careful canary, because you expose 100% of users to an unvalidated version at once. Automate as much of this as possible so the pipeline cannot proceed to the traffic switch unless all checks pass.

Smoke and functional tests

• Cover your core user journeys against the tagged URL, not just a /healthz endpoint. A health check returning 200 does not mean your application is processing requests correctly.
• Test the critical paths that would cause user-visible failures: login, checkout, data reads and writes, third-party integrations.

Dependency and configuration checks

• Verify the green revision can connect to its database, caches, and external APIs. A missing or malformed secret causes startup failures that may not surface until the first real request. The secrets in CI/CD guide covers how to catch these problems early.
• Check that the new version is reading the correct config values. Environment variable renames and config schema changes are a common source of silent failures.

Logs and error checks

• Inspect Cloud Logging for the green revision before switching. Look for startup errors, configuration warnings, and any stack traces that appear during your smoke test traffic.
• A revision that has errors in its logs during smoke testing is not ready to serve production traffic.

Baseline metrics

• Check the error rate and latency of the green revision under your synthetic smoke test traffic. Even a small test load can surface obvious problems. A 5xx rate on the green revision during your own tests is a clear stop signal.
• Set a hard threshold: if the error rate on green exceeds X% during validation, the pipeline aborts. Do not let warnings get ignored.

Schema and backward compatibility

• Before switching, confirm that the green version’s expected database schema is in place. Run any necessary migrations before flipping traffic, and make sure those migrations are backward compatible with the previous version.

Database and stateful workload considerations

Blue green is cleanest when your application is stateless. When state is involved, you need to plan more carefully.

Database schema migrations

During and after a blue green switch, both the blue (old) and green (new) versions may be talking to the same database at the same time. Blue is live during the rollback window; green is the live version after the switch. Any schema change must be backward compatible with the old version:

Sessions and user state

If your application stores session state in a cookie, a token, or a server-side session store, check that the new version can read sessions created by the old version. A session format change that silently logs users out after the switch is a real user impact even if no errors appear in monitoring.

Caches and queues

Shared caches (Memorystore, Redis) and queues (Pub/Sub, Cloud Tasks) can cause problems if the new version uses a different data format than the old one. During the rollback window, blue might dequeue a message that green serialised, or vice versa. Design your message and cache formats to be forward and backward compatible across adjacent versions.

API contract compatibility

If blue and green both call downstream services, make sure those services can handle requests from both versions at the same time during the switch window. This matters most in microservice architectures where many services deploy together.

Automated rollback and post-deploy verification

Manual rollback only works if someone notices the problem quickly. For anything beyond a low-traffic service, automate it.

Post-switch monitoring window on Cloud Run

After switching traffic to green, your pipeline should pause and query Cloud Monitoring for the new revision’s error rate. If the error rate exceeds a threshold within the first five minutes, the pipeline triggers the rollback command automatically and posts to your incident channel:

# Example post-switch automated rollback (pseudocode within a pipeline script)
# 1. Switch traffic to green
gcloud run services update-traffic api-service \
  --region=europe-west2 \
  --to-revisions=LATEST=100

# 2. Wait for monitoring data to accumulate
sleep 300

# 3. Query Cloud Monitoring for the new revision's error rate
# (implement via gcloud monitoring read or the Cloud Monitoring API)
# If error rate > threshold:

# 4. Rollback
gcloud run services update-traffic api-service \
  --region=europe-west2 \
  --to-revisions=api-service-00049-xyz=100

For a production-quality implementation, pair this with Cloud Monitoring alerts and a Cloud Run error rate metric. The monitoring Cloud Run guide shows which metrics to watch.

Cloud Deploy verification jobs

If you use Cloud Deploy to manage releases, it supports post-deployment verification via a verify job in your delivery pipeline definition. The verify job runs after the deployment and executes a test container of your choice. If the verify job fails, Cloud Deploy can be configured to automatically roll back to the previous release.

The Cloud Deploy overview explains how to structure a delivery pipeline if you are not already using it.