Skip to content

Upgrade Blocked by Pod Disruption Budget

Symptom

The upgrade reports drain or eviction failure, often with a message that AKS cannot evict a pod because doing so would violate the pod's disruption budget.

Possible Causes

  • minAvailable is too strict for the current replica count.
  • maxUnavailable effectively allows zero disruption.
  • Critical workloads run with insufficient replicas for a rolling drain.
  • Drain timeout expires before the workload can terminate cleanly.

Diagnosis Steps

flowchart TD
    A[Drain fails during upgrade] --> B[Inspect PDBs and replica counts]
    B --> C{Budget allows at least one eviction?}
    C -->|No| D[Fix PDB or increase replicas]
    C -->|Yes| E[Review drain timeout and slow termination]

  1. Inspect budgets and deployment replica counts.

    kubectl get pdb --all-namespaces
kubectl get deploy --all-namespaces

  2. Check for quarantined nodes when undrainable-node behavior is in use.

    kubectl get nodes --show-labels

  3. Review recent events around the blocked workloads.

    kubectl get events --all-namespaces --sort-by=.lastTimestamp

Resolution

  • Increase replicas so the PDB can tolerate at least one eviction.
  • Relax the PDB so the workload can be drained safely.
  • Extend drain timeout if the workload shuts down slowly.
  • Re-run the upgrade only after the budget and replica model are compatible with drain.

Prevention

  • Test PDB behavior in staging during real node drains.
  • Avoid singleton critical workloads on pools that will be upgraded in place.
  • Keep a blue/green option for the most disruption-sensitive services.

See Also

Sources