Canary Deployments in Azure

How canary deployments work

The canary pattern requires running two versions of an application simultaneously and controlling what percentage of traffic each version receives. A typical progression:

1. Deploy the new version alongside the current stable version.

2. Send 5–10% of traffic to the new version. Monitor error rates and latency for 15–30 minutes.

3. If metrics are healthy, increase to 25% traffic. Monitor for another period.

4. Increase to 50%, then 75%, then 100%.

5. At 100%, retire the old version.

At any point in steps 2–4, if error rates or latency increase beyond a defined threshold, shift all traffic back to the stable version. The new version never reached 100%, so the blast radius was limited to the traffic percentage in use at the time of the problem.

Canary with App Service deployment slots

App Service deployment slots are pre-production environments within the same App Service Plan. Each slot has its own hostname, application configuration, and running code. The traffic routing feature sends a configurable percentage of incoming requests to a slot.

Traffic splitting in App Service is sticky by default: a user who is routed to the canary slot continues hitting the canary slot for their session. This prevents a user from experiencing two different application versions in the same browsing session — which could cause confusing state issues.

# Deploy the new version to the staging slot first
az webapp deployment slot create \
  --name app-myapp-prod \
  --resource-group rg-prod \
  --slot staging

# Confirm new version is deployed to staging slot
az webapp show \
  --name app-myapp-prod \
  --resource-group rg-prod \
  --slot staging \
  --query "state"

# Start canary: send 10% of production traffic to the staging slot
az webapp traffic-routing set \
  --name app-myapp-prod \
  --resource-group rg-prod \
  --distribution staging=10

# Monitor for 15-30 minutes, then increase to 25%
az webapp traffic-routing set \
  --name app-myapp-prod \
  --resource-group rg-prod \
  --distribution staging=25

# Increase to 50%
az webapp traffic-routing set \
  --name app-myapp-prod \
  --resource-group rg-prod \
  --distribution staging=50

# Check current routing distribution
az webapp traffic-routing show \
  --name app-myapp-prod \
  --resource-group rg-prod

When confidence is high, complete the promotion with a slot swap. The swap warms up the new version (completes its startup sequence and health checks) before switching the production hostname to point at it:

# Promote: swap staging slot to production
az webapp deployment slot swap \
  --name app-myapp-prod \
  --resource-group rg-prod \
  --slot staging \
  --target-slot production

Canary with Container Apps revisions

Azure Container Apps uses revisions as immutable snapshots of a container app’s configuration and container image. Multiple revisions run simultaneously with configurable traffic percentages. This is a natural fit for canary deployments.

# Deploy a new revision with the updated container image
# The --revision-suffix creates a human-readable revision name
az containerapp update \
  --name myapp \
  --resource-group rg-prod \
  --image myregistry.azurecr.io/myapp:v2.0 \
  --revision-suffix v2-0

# List current revisions and their traffic weights
az containerapp revision list \
  --name myapp \
  --resource-group rg-prod \
  --query "[].{name:name, active:properties.active, traffic:properties.trafficWeight}" \
  --output table

# Set traffic split: 90% stable, 10% canary
az containerapp ingress traffic set \
  --name myapp \
  --resource-group rg-prod \
  --revision-weight myapp--v1-0=90 myapp--v2-0=10

# Increase to 50/50 after validating the canary
az containerapp ingress traffic set \
  --name myapp \
  --resource-group rg-prod \
  --revision-weight myapp--v1-0=50 myapp--v2-0=50

# Full promotion: 100% to new revision
az containerapp ingress traffic set \
  --name myapp \
  --resource-group rg-prod \
  --revision-weight myapp--v2-0=100

# Deactivate the old revision after confirming stability
az containerapp revision deactivate \
  --name myapp \
  --resource-group rg-prod \
  --revision myapp--v1-0

Canary in AKS with replica ratios

In AKS, a basic canary uses two Deployments — one stable, one canary — both with pods matching the same Service selector label. Kubernetes distributes requests across all matching pods proportionally by replica count.

Example: 9 stable replicas + 1 canary replica = roughly 10% canary traffic. Scale to 3 canary + 9 stable = 25% canary.

# Scale the canary deployment to increase traffic percentage
# 3 canary + 9 stable = 25% canary
kubectl scale deployment myapp-canary --replicas=3 -n production

# Watch the canary pods for errors
kubectl logs -l app=myapp,track=canary -n production --since=5m

# Check error rate in Application Insights or Prometheus before increasing
# If healthy, move to 50%: 6 canary + 6 stable
kubectl scale deployment myapp-canary --replicas=6 -n production
kubectl scale deployment myapp-stable --replicas=6 -n production

# Full promotion: update stable deployment to new image, delete canary
kubectl set image deployment/myapp-stable \
  myapp=myregistry.azurecr.io/myapp:v2.0 \
  -n production

kubectl delete deployment myapp-canary -n production

For production AKS canaries, consider Flagger. It automates traffic shifting based on Prometheus or Application Insights metrics, rolls back automatically if thresholds are breached, and integrates with Istio or NGINX ingress for more precise traffic control than replica-count-based splitting.

Monitoring the canary: promotion criteria

Define promotion criteria before starting the canary. Common thresholds:

Error rate. The canary’s HTTP 5xx rate must not exceed the stable baseline by more than 0.1 percentage points (e.g., if stable is at 0.05%, the canary limit is 0.15%). Measure per revision in Application Insights.

P99 latency. The 99th-percentile response time for the canary must not exceed the stable baseline by more than 20%. A 20% latency regression in the 99th percentile indicates a real performance issue.

Soak time per increment. Hold each traffic level for at least 15 minutes before increasing. Memory leaks, connection pool exhaustion, and database contention often do not appear immediately — they need sustained traffic to manifest.

Automated rollback

Manual monitoring is error-prone. Automate rollback using Azure Monitor alerts. Create an alert rule on the canary’s error rate metric. Add an action group that triggers an Azure Logic App or Function. The Logic App calls the Azure REST API to reset the traffic routing to 0% for the canary slot or revision.

For App Service, the rollback action is:

# Rollback: set canary traffic back to 0% (all traffic on production slot)
az webapp traffic-routing clear \
  --name app-myapp-prod \
  --resource-group rg-prod

After rollback, the application is fully on the stable version. Investigate the canary metrics, fix the issue, and start a new canary deployment after the fix is validated in staging.