Service Proxies Under the Hood: Virtual IPs, kube-proxy iptables, and IPVS Routing

A frontend web application sends an HTTP GET request to a stable virtual database service endpoint:

http://10.96.0.100:8080

10.96.0.100 is a Kubernetes ClusterIP Service Virtual IP.

Behind this single virtual IP, 4 backend database Pods run across separate worker nodes with individual IP addresses (10.244.1.15, 10.244.2.88, 10.244.3.12, 10.244.3.44).

When one of the backend database Pods crashes:

The control plane updates the active endpoint list within 100 milliseconds.
Subsequent HTTP requests to 10.96.0.100:8080 route exclusively to the 3 surviving healthy Pods.
The frontend web application never encounters a connection error or configuration change.

If you list network interfaces on the host node, 10.96.0.100 does not exist on any physical or virtual network interface card.

How does Kubernetes route traffic sent to a non-existent Virtual IP address across dynamic backend Pod endpoints?

The answer lies in kube-proxy and Linux kernel packet rewriting engines (Netfilter iptables and IPVS).

1. Why Services and Virtual IPs Exist

Because Pod IP addresses are ephemeral (a Pod receives a new IP address every time it restarts or scales), application microservices cannot hardcode target Pod IPs directly.

A Kubernetes Service solves this by providing:

An invariant, stable Virtual IP Address (ClusterIP) and DNS name.
An automated, load-balanced traffic distributor that maps traffic sent to the Virtual IP across matching backend EndpointSlices (active Pod IPs).

                      +-----------------------------------+
                      |      VIRTUAL SERVICE IP           |
                      |      10.96.0.100:8080             |
                      +-----------------------------------+
                                        |
                 +----------------------+----------------------+
                 | (kube-proxy Packet Load-Balancing)         |
                 v                                             v
    [ Backend Pod A: 10.244.1.15 ]               [ Backend Pod B: 10.244.2.88 ]

2. How `kube-proxy` Operates

kube-proxy is a node-level daemon running on every worker node.

It does not sit in the userland data path acting as a reverse proxy for every byte transferred. If kube-proxy proxied every TCP packet in userland, network throughput would collapse under CPU context-switching overhead.

Instead, kube-proxy acts as a control loop agent:

It watches kube-apiserver for Service and EndpointSlice object updates.
It programs kernel packet routing rules directly into the host kernel’s Netfilter (iptables) or IPVS subsystems.
Network packets are load-balanced and rewritten at kernel wire speed.

3. `kube-proxy` Mode 1: `iptables` Mode (Default)

In iptables mode, kube-proxy translates Services into chains of Netfilter rules using the kernel statistic module for random probability load balancing.

How Random Probability Load Balancing Works

Suppose a Service with Virtual IP 10.96.0.100:80 maps to 4 backend Pods (Pod 1, Pod 2, Pod 3, Pod 4).

kube-proxy constructs a sequence of rules in the KUBE-SERVICES chain:

Rule 1: Match Dst 10.96.0.100 --statistic mode random --probability 0.25000000000 -> Forward to Pod 1 (10.244.1.15)
Rule 2: Match Dst 10.96.0.100 --statistic mode random --probability 0.33333333333 -> Forward to Pod 2 (10.244.2.88)
Rule 3: Match Dst 10.96.0.100 --statistic mode random --probability 0.50000000000 -> Forward to Pod 3 (10.244.3.12)
Rule 4: Match Dst 10.96.0.100                                                   -> Forward to Pod 4 (10.244.3.44)

Probability Math Breakdown:

Rule 1: Has a $25\%$ chance ( $1/4$ ) of matching. If selected, packet routes to Pod 1.
Rule 2: If Rule 1 missed ( $75\%$ remaining), Rule 2 has a $33.3\%$ chance ( $1/3$ ) of matching $75\% \times 33.3\% = \mathbf{25\%}$ . Routes to Pod 2.
Rule 3: If Rules 1 and 2 missed ( $50\%$ remaining), Rule 3 has a $50\%$ chance ( $1/2$ ) of matching $50\% \times 50\% = \mathbf{25\%}$ . Routes to Pod 3.
Rule 4: Catch-all for remaining traffic ( $25\%$ ). Routes to Pod 4.

Result: Traffic is distributed evenly ( $25\%$ per Pod) across all 4 backends without maintaining application state!

The `iptables` Scalability Bottleneck ( $\mathcal{O}(N)$ )

While iptables mode works well for small clusters, iptables stores rules in a sequential linked list.

Evaluating rules takes $\mathcal{O}(N)$ linear time.

In large enterprise clusters with 10,000 Services and 50,000 Pod endpoints:

A single packet must traverse up to 50,000 sequential iptables rules in kernel space.
Adding or updating a single Service requires rewriting the entire 50,000-rule iptables string in kernel RAM, causing severe CPU spikes.

4. `kube-proxy` Mode 2: IPVS Mode ( $\mathcal{O}(1)$ )

To solve iptables linear scaling limits, kube-proxy supports IP Virtual Server (IPVS) mode.

IPVS is an in-kernel Layer-4 load balancing subsystem built into the Linux kernel specifically for high-throughput routing.

iptables Mode: Sequential Linked List O(N)        IPVS Mode: In-Kernel Hash Table O(1)

[ Rule 1 ] -> [ Rule 2 ] -> ... -> [ Rule 50,000 ]    Hash Key: 10.96.0.100:80
(Traverses 50,000 rules sequentially per packet!)     ---> Direct Hash Lookup O(1)
                                                      ---> Selects Target Pod instantly!

Advantages of IPVS Mode:

$\mathcal{O}(1)$ Hash Table Lookup: Network packets execute instant hash table lookups regardless of whether the cluster has 10 Services or 100,000 Services.
Advanced Load Balancing Algorithms: Supports algorithms beyond random probability:
- rr: Round-Robin
- lc: Least Connections
- sed: Shortest Expected Delay
- dh: Destination Hashing

5. Modern Alternative: eBPF Routing (Cilium)

Modern CNI implementations (such as Cilium) bypass kube-proxy, iptables, and IPVS entirely using Extended Berkeley Packet Filters (eBPF).

Cilium attaches eBPF programs directly to Linux kernel socket layers (sock_ops), rewriting destination packet addresses inside socket buffers before Netfilter chains execute, achieving higher throughput and lower latency.

`kube-proxy` Modes Comparison

Mode / Engine	Algorithmic Complexity	Load Balancing Algorithms	Scalability Limit	Resource Overhead
`iptables` (Default)	$\mathcal{O}(N)$ Linear Scan	Random Probability (`-m statistic`)	Degrades past ~5,000 Services	Low initial RAM, high CPU update churn
IPVS	$\mathcal{O}(1)$ Hash Lookup	Round-Robin, Least-Conn, Hashing	Scales to 100,000+ Services	Minimal memory & CPU overhead
eBPF (Cilium)	$\mathcal{O}(1)$ eBPF Map	Custom eBPF algorithms	Massive Hyperscale	Lowest latency (Bypasses Netfilter)

Summary & Next Steps

Service proxies enable stable networking across dynamic Pod endpoints:

ClusterIP Virtual IPs provide invariant network addresses that route traffic across active Pod endpoints.
kube-proxy manages Netfilter rules in kernel space without proxying userland data directly.
iptables Mode uses random probability distribution (-m statistic), but suffers from $\mathcal{O}(N)$ linear scan overhead in large clusters.
IPVS Mode uses in-kernel hash tables to provide $\mathcal{O}(1)$ constant-time routing for large-scale production clusters.

In the next article, we examine CoreDNS, Service Discovery, and Ingress Controller Architecture.

References & Further Reading

Cloud Native Computing Foundation. Encrypting Secret Data at Rest in Kubernetes. CNCF Docs.
HashiCorp. Vault Agent Sidecar Injector for Kubernetes. HashiCorp Vault Docs.
Center for Internet Security. CIS Kubernetes Benchmark v1.8.0 Secret Protection Guidelines. CIS Security.