AKS Node Pools Explained

What is a node pool

A node pool is a group of nodes (Azure VMs) in an AKS cluster that share the same VM size, OS, and configuration. All nodes within a pool are identical. You can have multiple pools in one cluster, each with different characteristics.

When Kubernetes schedules a pod, it picks a node from whichever pool has the right capacity and meets any constraints the pod specifies. By default, pods can run on any node. When you need to control which workloads run on which pools, you use labels, taints, and tolerations.

System node pools vs user node pools

AKS distinguishes between two types of node pools:

The system node pool taint prevents your regular application pods from running alongside cluster-critical pods. This ensures that a misbehaving application that consumes all node resources cannot starve CoreDNS or other essential components.

For small clusters where cost matters more than isolation, you can run workloads on system nodes by adding the CriticalAddonsOnly toleration to your pods. But for production, keep them separated.

Adding a node pool to an existing cluster

Add a user node pool with a different VM size for compute-intensive workloads:

# Add a memory-optimized node pool for database workloads
az aks nodepool add \
  --resource-group my-aks-rg \
  --cluster-name my-cluster \
  --name memorypool \
  --node-count 2 \
  --node-vm-size Standard_E4s_v3 \
  --mode User \
  --labels workload=memory-intensive

# Add a spot node pool for batch jobs (much cheaper, interruptible)
az aks nodepool add \
  --resource-group my-aks-rg \
  --cluster-name my-cluster \
  --name spotpool \
  --node-count 3 \
  --node-vm-size Standard_D4s_v3 \
  --priority Spot \
  --eviction-policy Delete \
  --spot-max-price -1 \
  --mode User \
  --labels workload=batch

The —spot-max-price -1 means you accept Azure’s current spot price up to the regular VM price. Set a specific value (e.g., 0.05) to cap what you pay per hour and risk not getting capacity.

Viewing and managing node pools

# List all node pools in a cluster
az aks nodepool list \
  --resource-group my-aks-rg \
  --cluster-name my-cluster \
  --output table

# Get details on a specific pool
az aks nodepool show \
  --resource-group my-aks-rg \
  --cluster-name my-cluster \
  --name memorypool

# Scale a node pool manually
az aks nodepool scale \
  --resource-group my-aks-rg \
  --cluster-name my-cluster \
  --name memorypool \
  --node-count 4

# From kubectl, see nodes with their pool labels
kubectl get nodes --show-labels
kubectl get nodes -l agentpool=memorypool

Enabling autoscaling per node pool

Each node pool can have its own autoscaler configuration with independent min and max node counts:

# Enable autoscaler on the system pool
az aks nodepool update \
  --resource-group my-aks-rg \
  --cluster-name my-cluster \
  --name nodepool1 \
  --enable-cluster-autoscaler \
  --min-count 2 \
  --max-count 5

# Enable autoscaler on the memory pool
az aks nodepool update \
  --resource-group my-aks-rg \
  --cluster-name my-cluster \
  --name memorypool \
  --enable-cluster-autoscaler \
  --min-count 1 \
  --max-count 8

The autoscaler adds nodes when pods cannot be scheduled due to resource pressure, and removes nodes when they remain underutilized for a configurable period (default 10 minutes).

Directing pods to specific node pools

Use node selectors to pin pods to pools with matching labels. When you created the memory pool, you added the label workload=memory-intensive. Reference it in your pod spec:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgres
spec:
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      nodeSelector:
        workload: memory-intensive
      containers:
      - name: postgres
        image: postgres:15
        resources:
          requests:
            cpu: "500m"
            memory: "2Gi"
          limits:
            cpu: "2"
            memory: "4Gi"

For more flexibility, use node affinity instead of nodeSelector. Node affinity supports preferred (soft) rules and required (hard) rules:

spec:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: workload
            operator: In
            values:
            - memory-intensive

Taints and tolerations

Node selectors push pods toward specific nodes. Taints do the opposite — they push pods away from nodes unless those pods explicitly tolerate the taint.

This is useful for dedicated pools. Add a taint to a GPU node pool so only GPU-enabled pods can run there:

# Add a taint when creating a node pool
az aks nodepool add \
  --resource-group my-aks-rg \
  --cluster-name my-cluster \
  --name gpupool \
  --node-count 1 \
  --node-vm-size Standard_NC6s_v3 \
  --mode User \
  --node-taints sku=gpu:NoSchedule

Now only pods that tolerate sku=gpu:NoSchedule can schedule on the GPU pool:

spec:
  tolerations:
  - key: "sku"
    operator: "Equal"
    value: "gpu"
    effect: "NoSchedule"
  nodeSelector:
    sku: gpu

Use both the toleration (allows landing on tainted nodes) and nodeSelector (requires landing on labeled nodes) together. Toleration alone allows pods on GPU nodes but does not prevent them from running elsewhere too.

Upgrading node pool Kubernetes version

Node pools can be on a different Kubernetes version than the control plane, as long as they are within two minor versions. You can upgrade pools independently, which lets you test upgrades on a non-production pool first:

# Check available upgrade versions
az aks get-upgrades \
  --resource-group my-aks-rg \
  --cluster-name my-cluster \
  --output table

# Upgrade a specific node pool
az aks nodepool upgrade \
  --resource-group my-aks-rg \
  --cluster-name my-cluster \
  --name nodepool1 \
  --kubernetes-version 1.29.2

# Watch upgrade progress
az aks nodepool show \
  --resource-group my-aks-rg \
  --cluster-name my-cluster \
  --name nodepool1 \
  --query "provisioningState" -o tsv

During an upgrade, AKS cordons each node (marks it unschedulable), drains its pods to other nodes, replaces the VM with a new one running the target version, then uncordons it. This happens one node at a time (configurable with surge settings) to minimize disruption.

Removing a node pool

Before deleting a pool, ensure its workloads have been migrated or scaled down:

# Cordon all nodes in the pool to prevent new pods scheduling
kubectl cordon $(kubectl get nodes -l agentpool=spotpool -o name)

# The cordon above marks nodes unschedulable but doesn't move existing pods
# Drain to evict existing pods (will reschedule on other nodes)
kubectl drain $(kubectl get nodes -l agentpool=spotpool -o name | tr '\n' ' ') \
  --ignore-daemonsets \
  --delete-emptydir-data

# Now delete the pool
az aks nodepool delete \
  --resource-group my-aks-rg \
  --cluster-name my-cluster \
  --name spotpool \
  --no-wait