Kubernetes Control Plane Architecture: API Server, etcd, Scheduler, and Controller Manager

At 04:15 AM, a physical network switch fails in a cloud data center, instantly disconnecting Worker Node 04 from the rest of your production cluster.

Node 04 was running 15 active application containers.

Within 40 seconds, without human intervention:

1. The cluster detects that Node 04 has stopped sending heartbeats.
2. The control plane updates the state of Node 04 to NotReady.
3. Replacement containers are scheduled and launched on surviving worker nodes.
4. Internal cluster routing updates automatically to send web traffic to the new container IP addresses.

How does a Kubernetes cluster make intelligent, high-speed orchestration decisions across hundreds of distributed physical servers?

The secret lies in the architectural division between the Control Plane (The Brain) and Worker Nodes (The Execution Engine).

---

1. The Architectural Split: Control Plane vs Data Plane

A Kubernetes cluster separates responsibilities into two distinct planes:

+-------------------------------------------------------------------+
|                       CONTROL PLANE (MASTER)                      |
|                                                                   |
|   +-----------------------------------------------------------+   |
|   |                    kube-apiserver                         |   |
|   |         (HTTP REST API Gateway & Authentication)          |   |
|   +-----------------------------------------------------------+   |
|         |                     |                     |             |
|         v                     v                     v             |
|    +----------+      +------------------+    +----------------+   |
|    |   etcd   |      |  kube-scheduler  |    | kube-controller|   |
|    | (State)  |      |   (Placement)    |    |  -manager      |   |
|    +----------+      +------------------+    +----------------+   |
+-------------------------------------------------------------------+
                                  |
            gRPC / HTTPS Network Stream (Port 6443)
                                  |
+-------------------------------------------------------------------+
|                        WORKER NODES (DATA PLANE)                  |
|                                                                   |
|   Worker Node 1                      Worker Node 2                |
|   +--------------------------+       +------------------------+   |
|   | kubelet | kube-proxy     |       | kubelet | kube-proxy   |   |
|   | containerd (Pods)        |       | containerd (Pods)      |   |
|   +--------------------------+       +------------------------+   |
+-------------------------------------------------------------------+

• Control Plane: Manages cluster state, makes scheduling decisions, responds to cluster events, and enforces declarative specs.
• Worker Nodes: Host the actual running application container workloads (Pods).

---

2. Core Control Plane Components

The Control Plane consists of four primary software components:

A. kube-apiserver (The Front Door & Gateway)

The API Server is the central communication hub of the entire Kubernetes cluster.

• HTTP REST Gateway: Exposes JSON/YAML endpoints for all cluster resources (Pods, Services, Deployments).
• Authentication & Authorization: Validates user tokens, TLS certificates, and RBAC permissions.
• Single Gateway to etcd: No other component in the cluster reads or writes to etcd directly. The API Server is the sole component allowed to query or update etcd.

---

B. etcd (The Cluster Memory)

etcd is a strongly consistent, distributed key-value storage system based on the Raft Consensus Algorithm.

• Stores the entire state of the Kubernetes cluster (desired state, actual state, node status, secrets).
• If data is not saved in etcd, it does not exist in the cluster.
• Supports HTTP/2 Watch Streams, allowing control plane components to receive instant push notifications when key entries change.

---

C. kube-scheduler (The Placement Engine)

The Scheduler assigns newly created, un-scheduled Pods to suitable worker nodes.

• Filtering (Predicates): Eliminates nodes that lack sufficient CPU/RAM, fail taints/tolerations, or lack requested volume mounts.
• Scoring (Priorities): Ranks surviving nodes based on resource availability, affinity rules, and spread topology to select the optimal host.

---

D. kube-controller-manager (The Reconciliation Engine)

The Controller Manager compiles dozens of individual control loops into a single binary executable.

• NodeController: Monitors node health and handles node failure eviction.
• ReplicaSetController: Ensures the exact requested number of Pod replicas remain active.
• EndpointSliceController: Populates IP endpoint arrays connecting Services to Pods.

Every controller runs a continuous Reconciliation Loop: Observe State -> Compare to Desired Spec -> Execute Corrective Action.

---

3. Core Worker Node Components

Every worker host machine runs three node-level management components:

A. kubelet (The Node Execution Agent)

kubelet is the primary daemon running on every worker node.

• Communicates continuously with kube-apiserver via HTTPS.
• Receives assigned PodSpec manifests and instructs the local container runtime (via gRPC Container Runtime Interface / CRI) to start or stop container processes.
• Executes container readiness and liveness probes.
• Reports node status and resource metrics back to the API Server.

---

B. kube-proxy (The Network Routing Agent)

kube-proxy runs on every worker node to manage virtual network routing.

• Monitors Service and EndpointSlice updates from the API Server.
• Updates local kernel iptables or IPVS routing rules so network traffic sent to a virtual Service IP is load-balanced to target Pod IPs.

---

C. Container Runtime (containerd / cri-o)

The low-level software driver responsible for pulling container images and running container processes (via runc).

---

Component Interaction Matrix

Component	Layer	Communication Mechanism	Primary Responsibility
kube-apiserver	Control Plane	HTTPS / REST (Port 6443)	Validates REST API requests and reads/writes etcd
etcd	Control Plane	gRPC / Raft Protocol	Persists immutable cluster configuration & state
kube-scheduler	Control Plane	Watches kube-apiserver	Selects target worker nodes for unassigned Pods
kube-controller-manager	Control Plane	Watches kube-apiserver	Executes control loops to reconcile actual vs desired state
kubelet	Worker Node	gRPC (CRI) / HTTPS	Manages local container lifecycles and node health
kube-proxy	Worker Node	Kernel Netfilter / IPVS	Configures node network rules for virtual Service IPs

---

Summary & Next Steps

Kubernetes achieves cluster-scale resilience through decoupled component responsibilities:

• kube-apiserver acts as the single authenticated REST gateway to the cluster.
• etcd provides a distributed, strongly consistent Raft storage database.
• kube-scheduler matches unallocated Pods to optimal worker nodes based on resource scores.
• kube-controller-manager continuously reconciles state discrepancies.
• kubelet and kube-proxy manage node-level container processes and virtual network routing.

In the next article, we examine etcd Internals: Raft Consensus, MVCC Key-Value Storage, and Watch Streams.

References & Further Reading

1. Cloud Native Computing Foundation. Kubernetes Components & Control Plane Architecture. CNCF Docs.
2. CNCF etcd Project. etcd v3 Architecture & Raft Consensus Implementation. etcd Docs.
3. Hausenblas, M., & Schimanski, S. (2019). Programming Kubernetes (Chapter 2: Kubernetes API Basics). O’Reilly Media.

---