Azure CNI vs Kubenet in AKS

What the CNI plugin does

CNI stands for Container Network Interface. When Kubernetes creates a pod, it calls the CNI plugin to set up the pod’s network namespace: create a virtual network interface, assign an IP address, and configure routing so the pod can communicate.

Different CNI plugins use different strategies for assigning IPs and routing traffic. In AKS, your main choices are Kubenet, Azure CNI (standard), and Azure CNI Overlay.

Kubenet: the overlay approach

With Kubenet, pods get IPs from a separate address space that is not part of your Azure VNet. Each node handles routing for its own pods using a VXLAN overlay network.

How it works:

1. Each node gets one VNet IP address
2. Pods on that node get IPs from a smaller subnet allocated to the node (e.g., node gets 10.244.1.0/24, its pods get 10.244.1.1 through 10.244.1.254)
3. Traffic from pod A (node 1) to pod B (node 2) is encapsulated and sent via the node’s VNet IP
4. Traffic from pods to the internet is NATted to the node’s VNet IP

# Create a cluster with Kubenet
az aks create \
  --resource-group my-aks-rg \
  --name kubenet-cluster \
  --network-plugin kubenet \
  --pod-cidr 10.244.0.0/16 \
  --service-cidr 10.0.0.0/16 \
  --dns-service-ip 10.0.0.10 \
  --node-count 3 \
  --generate-ssh-keys

Advantages of Kubenet:

• Minimal VNet IP consumption — only node IPs come from the subnet
• Works with smaller subnets (/26 or even /28)
• Simple configuration

Disadvantages of Kubenet:

• Pods cannot be reached directly from other VNet resources using pod IPs
• Traffic between nodes goes through user-defined routes (Azure manages these up to 400 routes)
• Does not support Azure Network Policy (only Calico)
• No support for Windows node pools

Azure CNI: VNet-native pod IPs

With Azure CNI (standard mode), every pod gets an IP address directly from your Azure VNet subnet. Pods are first-class citizens of the VNet — they can be reached from other Azure resources, on-premises networks, and peered VNets using their pod IPs without any special configuration.

How it works:

1. Before pods are scheduled, the CNI plugin pre-allocates a batch of IPs from the subnet for each node
2. When a pod starts, it gets one of those pre-allocated IPs
3. Pod-to-pod traffic is routed directly by the Azure VNet — no overlay, no NAT

# Create a cluster with Azure CNI
az aks create \
  --resource-group my-aks-rg \
  --name cni-cluster \
  --network-plugin azure \
  --network-policy azure \
  --vnet-subnet-id /subscriptions/{sub-id}/resourceGroups/my-rg/providers/Microsoft.Network/virtualNetworks/my-vnet/subnets/aks-subnet \
  --node-count 3 \
  --generate-ssh-keys

Advantages of Azure CNI:

• Pods directly reachable from the VNet using real IP addresses
• Supports Azure Network Policy
• Works with Windows node pools
• No user-defined route limits (Azure CNI handles routing at the NIC level)
• Better performance — no overlay encapsulation overhead

Disadvantages of Azure CNI:

• Each pod consumes a VNet IP — IP address planning is critical
• Requires a large enough subnet (easy to underestimate)
• More complex IP planning for multiple node pools

IP address planning for Azure CNI

This is where Azure CNI catches people off guard. The formula for calculating required IPs:

Required IPs = (max nodes × max pods per node) + node count + buffer

Azure CNI pre-allocates IPs before pods are scheduled. By default, each node pre-allocates IPs for up to 30 pods. With autoscaling, you need to plan for the maximum node count.

Example:
- Max nodes: 20
- Max pods per node: 30 (default)
- Pre-allocated per node: 30 IPs
- Node IPs: 20

Total: (20 × 30) + 20 = 620 IPs

Use a /22 subnet (1022 usable IPs) to be safe.
A /24 subnet (254 IPs) is NOT enough.

# Check how many IPs are allocated per node
kubectl describe node aks-nodepool1-12345678-vmss000000 \
  | grep -i "azure.com/podcount"

# See current pod count vs max
kubectl describe node aks-nodepool1-12345678-vmss000000 \
  | grep -A2 "Allocatable"

Azure CNI Overlay — the best of both worlds

Azure CNI Overlay is a newer mode that addresses the IP exhaustion problem. Pods get IPs from an overlay network (not the VNet subnet), but the CNI plugin still handles features like network policies and Azure-native load balancing.

# Create a cluster with Azure CNI Overlay
az aks create \
  --resource-group my-aks-rg \
  --name overlay-cluster \
  --network-plugin azure \
  --network-plugin-mode overlay \
  --pod-cidr 192.168.0.0/16 \
  --network-policy azure \
  --node-count 3 \
  --generate-ssh-keys

With Overlay mode:

• Pods get IPs from the pod CIDR (192.168.0.0/16 in the example) — not from the VNet subnet
• Only node IPs come from the VNet subnet (much fewer IPs consumed)
• Azure Network Policy still works
• Windows node pools are supported
• Pods cannot be directly reached from outside the cluster using pod IPs (similar to Kubenet)

Side-by-side comparison

Which to choose

Choose Kubenet when:

• You have limited VNet IP address space
• You do not need direct pod reachability from outside the cluster
• Your cluster will stay under 400 nodes
• You are comfortable with Calico for network policy

Choose Azure CNI when:

• Other Azure services need to connect directly to pod IPs
• You run Windows containers
• You have planned a sufficiently large subnet
• You need Azure Network Policy

Choose Azure CNI Overlay when:

• You want Azure CNI features without the IP address overhead
• You are starting a new cluster and do not need direct pod-IP connectivity from outside
• This is increasingly the recommended default for new AKS clusters