Adetayo Akinsanya unkletayo.dev

Controllers Deep Dive: ReplicaSets, Deployments, and Rolling Update Mechanics

Understanding maxSurge, maxUnavailable, ReplicaSet revision scaling, and zero-downtime rollbacks.

Part 9 in Series — Catch up on the previous article: The Atomic Unit of Scheduling: Why Kubernetes Uses Pods Instead of Containers (Part 8) before diving into this post.

An engineering team deploys a critical update to a payment microservice, changing the image tag from payment-api:v1.0 to payment-api:v2.0.

Immediately after deployment, a null-pointer exception causes all v2.0 containers to crash on startup.

If this deployment were executed using imperative scripts that stopped all v1.0 containers first, the payment service would be 100% offline, rejecting all customer transactions.

Instead, the application is managed by a Kubernetes Deployment:

kubectl set image deployment/payment-api payment=payment-api:v2.0

The Kubernetes DeploymentController initiates a zero-downtime Rolling Update:

  1. It creates a new ReplicaSet for v2.0 and attempts to launch 1 new pod replica.
  2. The readiness probe on the new v2.0 pod fails.
  3. The DeploymentController halts the rollout automatically, leaving all existing v1.0 pods running and serving traffic.

The team executes a single rollback command:

kubectl rollout undo deployment/payment-api

Within 2 seconds, the cluster reverts to 100% v1.0 state without losing a single customer transaction.

How do Deployments and ReplicaSets manage version revisions, replica scaling, and zero-downtime rolling updates?


1. The Workload Controller Hierarchy

Kubernetes manages application workloads using a 3-tier object hierarchy:

+-------------------------------------------------------------------+
|                        DEPLOYMENT OBJECT                          |
|   - Manages version revisions, rollouts, & rollbacks              |
|   - Declares strategy: RollingUpdate (maxSurge, maxUnavailable)  |
+-------------------------------------------------------------------+
                                  |
            Manages & Scales underlying ReplicaSets
                                  |
            +---------------------+---------------------+
            |                                           |
            v                                           v
+-----------------------+                   +-----------------------+
| ReplicaSet (v1.0)     |                   | ReplicaSet (v2.0)     |
| - Desired Replicas: 0 |                   | - Desired Replicas: 4 |
| - Label Selector:     |                   | - Label Selector:     |
|   pod-template-hash=v1|                   |   pod-template-hash=v2|
+-----------------------+                   +-----------------------+
            |                                           |
     Manages Pods                                Manages Pods
            v                                           v
+-----------------------+                   +-----------------------+
| Pods (v1.0) [OFFLINE] |                   | Pods (v2.0) [ACTIVE]  |
+-----------------------+                   +-----------------------+
  1. Deployment: High-level declarative specification for application versions and update strategies (RollingUpdate vs Recreate).
  2. ReplicaSet: Mid-level controller responsible for maintaining a fixed number of identical Pod replicas based on label selectors (pod-template-hash).
  3. Pod: Low-level execution unit holding running container processes.

Key Rule: Users rarely create ReplicaSets directly. You manage Deployments, and the DeploymentController manages ReplicaSets automatically.


2. Anatomy of a Rolling Update

A Rolling Update replaces old Pod instances with new Pod instances incrementally, ensuring enough healthy Pods remain active to serve traffic throughout the deployment.

Two critical configuration parameters govern rolling update behavior:

spec:
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 25%        # Maximum Pods created ABOVE desired replica count
      maxUnavailable: 25%  # Maximum Pods allowed OFFLINE during update

Understanding maxSurge and maxUnavailable

For a Deployment with replicas: 4:

  • maxSurge: 25% (4×0.25=14 \times 0.25 = 1): The cluster can scale up to 5 total Pods during the update (4+1=54 + 1 = 5).
  • maxUnavailable: 25% (4×0.25=14 \times 0.25 = 1): At least 3 Pods must remain active and healthy at all times (41=34 - 1 = 3).

3. Step-by-Step Rolling Update Execution Trace

Let’s trace how a Deployment updates 4 replicas from v1.0 (RS-Old) to v2.0 (RS-New):

INITIAL STATE: RS-Old (v1.0) = 4 Replicas | RS-New (v2.0) = 0 Replicas
-------------------------------------------------------------------------
Step 1: Deployment creates RS-New (v2.0) and scales it to 1 Replica.
        - Total Pods = 5 (4 v1.0 + 1 v2.0). Matches maxSurge limit (5)!
        - Pods Active = 5.

Step 2: RS-New (v2.0) Pod passes Readiness Probe!
        - Deployment scales RS-Old (v1.0) down to 3 Replicas.
        - Total Pods = 4 (3 v1.0 + 1 v2.0).

Step 3: Deployment scales RS-New (v2.0) up to 2 Replicas...
        - Progressively scales RS-Old DOWN while scaling RS-New UP.

FINAL STATE: RS-Old (v1.0) = 0 Replicas | RS-New (v2.0) = 4 Replicas

4. Revisions and Instant Rollbacks

Why are rollbacks so fast in Kubernetes?

When a Deployment is updated, the DeploymentController does not delete the old ReplicaSet. It retains historical ReplicaSets with replicas: 0.

# Inspect historical revisions:
kubectl rollout history deployment/payment-api
REVISION  CHANGE-CAUSE
1         kubectl apply --filename=deploy.yaml (image: v1.0)
2         kubectl set image deployment/payment-api payment=v2.0

When you execute kubectl rollout undo deployment/payment-api:

  1. The DeploymentController sets replicas: 4 on the historical Revision 1 ReplicaSet.
  2. It sets replicas: 0 on the failed Revision 2 ReplicaSet.
  3. Because the v1.0 image layers already exist on the worker nodes, the rollback completes in seconds!

Deployment Strategies Comparison

Deployment StrategyDowntime RiskMemory / Hardware OverheadRollback SpeedBest Use Case
RollingUpdateZero DowntimeModerate (Incurs maxSurge overhead)Fast (Instant ReplicaSet scaling)Standard production web services & APIs
RecreateBrief Downtime (Kills all v1 first)Zero (No extra pods)FastLegacy apps unable to run dual schema versions concurrently
Blue/Green (Custom)Zero DowntimeHigh (100% duplicate hardware pool)Instant (Service selector switch)Mission-critical financial & core banking services
Canary (Custom)Zero DowntimeLowFlexibleTesting new features against 5% of real user traffic

Summary & Next Steps

Deployments manage software revision rollouts with zero application downtime:

  • Deployments manage version revisions and ReplicaSets.
  • ReplicaSets maintain exact Pod replica counts via label selectors.
  • maxSurge & maxUnavailable bound temporary pod allocation limits during updates.
  • Historical ReplicaSets (replicas: 0) enable instant zero-downtime rollbacks (kubectl rollout undo).

In the next article, we examine Stateful Workloads: StatefulSets, Stable Network Identities, and Ordered Scaling.

References & Further Reading

  1. CNCF SIG Network. Kubernetes Gateway API Specification v1.0.0. CNCF Gateway API Docs.
  2. F5 NGINX Documentation. NGINX Ingress Controller Architecture. NGINX Docs.
  3. CNCF Envoy Project. Envoy Proxy Architecture & xDS API Specification. Envoy Docs.

Up Next in Series →

Part 10: Stateful Workloads: StatefulSets, Stable Network Identities, and Ordered Scaling

Continue to Part 10 →