Horizontal Pod Autoscaling in Kubernetes

How the HPA controller works

The HPA controller runs inside Kubernetes and watches the pods belonging to a target Deployment (or StatefulSet or other scalable resource). It periodically reads metrics — CPU utilization, memory usage, or custom metrics — and calculates the desired replica count using this formula:

desiredReplicas = ceil(currentReplicas × (currentMetricValue / targetMetricValue))

If your target is 50% CPU and your pods are currently running at 90% CPU, HPA calculates: ceil(3 × (90 / 50)) = ceil(5.4) = 6. It then scales up to 6 replicas.

When CPU drops below the target, HPA scales back down — but slowly, with a 5-minute cooldown by default to avoid constant scale-up/scale-down oscillation.

Prerequisites: metrics-server

HPA reads metrics from the Kubernetes metrics API, which is served by the metrics-server component. Verify it is running:

# Check if metrics-server is installed
kubectl get deployment metrics-server -n kube-system

# If missing, install it (for AKS with monitoring add-on, it's usually present)
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

# Verify metrics are available (takes 1-2 minutes after install)
kubectl top nodes
kubectl top pods

Creating an HPA — CPU-based scaling

First, ensure the target Deployment has CPU resource requests defined. HPA uses the request as the baseline for percentage calculations — without a request, CPU-based HPA does not function.

# deployment with resource requests
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web-app
  template:
    metadata:
      labels:
        app: web-app
    spec:
      containers:
      - name: web-app
        image: nginx:1.25
        resources:
          requests:
            cpu: "200m"
            memory: "256Mi"
          limits:
            cpu: "500m"
            memory: "512Mi"

# hpa.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-app-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-app
  minReplicas: 2
  maxReplicas: 20
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 60

kubectl apply -f hpa.yaml

# Check HPA status
kubectl get hpa web-app-hpa
# NAME          REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
# web-app-hpa   Deployment/web-app 12%/60%   2         20        2          5m

The TARGETS column shows current/target. When current usage exceeds 60%, HPA adds replicas. Below target, it scales down (respecting the minimum).

Memory-based scaling

You can scale on memory utilization too, or combine CPU and memory:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: cache-app-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: cache-app
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 75

When multiple metrics are specified, HPA calculates the desired replicas for each metric independently and uses the highest value. If CPU says scale to 5 and memory says scale to 8, it scales to 8.

Scaling on custom metrics

Business metrics often matter more than CPU. Scale on queue depth, request count, or any metric exposed by your application or Azure services. This requires the Azure Kubernetes Metrics Adapter or KEDA.

KEDA (Kubernetes Event-Driven Autoscaling) is built into AKS and provides the richest set of scalers. Enable it:

# Enable KEDA on an AKS cluster
az aks update \
  --resource-group my-aks-rg \
  --name my-cluster \
  --enable-keda

KEDA ScaledObject for Azure Service Bus queue depth:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: order-processor-scaler
spec:
  scaleTargetRef:
    name: order-processor
  minReplicaCount: 0
  maxReplicaCount: 30
  triggers:
  - type: azure-servicebus
    metadata:
      queueName: orders
      namespace: my-servicebus-namespace
      messageCount: "5"   # Scale up when >5 messages per replica
    authenticationRef:
      name: servicebus-auth

This scales the order-processor Deployment to 0 when the queue is empty (saving cost completely) and up to 30 replicas at 5 messages per replica when it fills up.

Testing HPA with a load generator

Verify your HPA works by generating artificial load:

# Deploy a load generator
kubectl run load-generator \
  --image=busybox \
  --restart=Never \
  -- /bin/sh -c "while true; do wget -q -O- http://web-app; done"

# In another terminal, watch the HPA respond
kubectl get hpa web-app-hpa --watch

# And watch pods scale up
kubectl get pods -l app=web-app --watch

# Stop the load generator when done
kubectl delete pod load-generator

# Watch pods scale back down (takes ~5 minutes)
kubectl get hpa web-app-hpa --watch

Tuning scale-up and scale-down behavior

The default HPA behavior is conservative about scaling down. You can tune this with the behavior field:

spec:
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 30    # Wait 30s before scaling up
      policies:
      - type: Pods
        value: 4                         # Add at most 4 pods per scale step
        periodSeconds: 60
    scaleDown:
      stabilizationWindowSeconds: 300   # Wait 5 min before scaling down
      policies:
      - type: Pods
        value: 2                         # Remove at most 2 pods per scale step
        periodSeconds: 60
      - type: Percent
        value: 10                        # Or 10% of current replicas
        periodSeconds: 60
      selectPolicy: Min                  # Use the more conservative policy

Aggressive scale-up and conservative scale-down is the right approach for most services. Scaling up quickly handles traffic spikes. Scaling down slowly avoids thrashing if traffic is fluctuating around the threshold.

What HPA cannot do

HPA has real limitations to be aware of:

• It cannot scale below minReplicas. Setting minReplicas to 0 is technically possible in autoscaling/v2 but requires KEDA for the 0-to-N scaling to work well.
• It does not provision nodes. If HPA wants to add pods but all nodes are full, pods stay Pending until the Cluster Autoscaler adds a node. Plan for this latency.
• It works poorly for very short spikes. A 15-second traffic burst is over before HPA even evaluates metrics. For flash traffic, consider pre-scaling or using caching at the edge.
• It conflicts with manual scaling. If you manually scale a Deployment that has an HPA, the HPA will override your change on the next evaluation cycle. Let the HPA control the replica count.