The Circuit Breaker Pattern: Protecting Services from Cascading Failures

Why You Need This in Real Life

At 02:30 PM on a busy afternoon, a downstream credit scoring microservice experiences a database connection pool exhaustion bug. Requests to the credit service stall for 30 seconds before timing out.

Upstream, the CheckoutService continues sending 1,000 HTTP requests per second to the struggling credit service.

Because each HTTP request blocks a worker thread for 30 seconds while waiting for the timeout, CheckoutService exhausts its own Tomcat thread pool of 200 workers within two seconds.

Suddenly, CheckoutService stops responding entirely—even for shopping cart operations that have nothing to do with credit scoring! The single failing downstream dependency triggers a cascading failure that takes down the entire microservice ecosystem.

The Circuit Breaker Pattern acts as an automated electrical fuse for microservices. When a downstream service degrades, the Circuit Breaker trips open, instantly failing fast without blocking worker threads, giving the struggling dependency time to recover.

Part 1: Circuit Breaker State Machine

A Circuit Breaker operates as a 3-state finite state machine:

                      +-----------------------------+
                      |           CLOSED            |
                      |  (Normal Operation: Pass)  |
                      +--------------+--------------+
                                     |
               Failure Rate Exceeds  |  Success Rate Meets
               Threshold (e.g. 50%)  |  Threshold
                                     v
                      +-----------------------------+
                      |            OPEN             |
                      |  (Fail-Fast: Reject Calls)  |
                      +--------------+--------------+
                                     |
                                     | Sleep Window Expires
                                     | (e.g. 10 seconds)
                                     v
                      +-----------------------------+
                      |          HALF-OPEN          |
                      | (Trial Probe: Test Traffic) |
                      +-----------------------------+

The 3 States Detailed

CLOSED (Normal Operation):
- Requests flow through to the downstream service normally.
- The Circuit Breaker records execution outcomes (successes, failures, slow calls) in a sliding time or count window.
- If the failure rate exceeds the configured threshold (e.g., $> 50\%$ failures over 100 requests), the circuit trips OPEN.
OPEN (Fail-Fast Mode):
- All calls to the downstream service are rejected immediately without attempting a network connection (CallNotPermittedException).
- The caller receives instant fallback responses (sub-1ms latency), preserving upstream thread pools.
- A Sleep Window timer starts (e.g., 10 seconds).
HALF-OPEN (Trial Recovery Mode):
- After the Sleep Window expires, the circuit transitions to HALF-OPEN.
- A limited number of trial probe requests (e.g., 10 requests) are permitted through to the downstream service.
- If all trial requests succeed, the circuit transitions back to CLOSED.
- If any trial request fails, the circuit immediately resets to OPEN for another sleep window.

Part 2: Sliding Window Metrics Mechanics

Modern Circuit Breakers (such as Resilience4j) calculate failure rates using two types of sliding windows:

1. Count-Based Sliding Window

Tracks the outcome of the last $N$ calls (e.g., a ring buffer of 100 calls).

2. Time-Based Sliding Window

Tracks calls executed during the last $N$ seconds (e.g., 10 1-second buckets).

Ring Buffer Metrics (Size N = 10):
[ Success | Success | FAIL | FAIL | FAIL | FAIL | FAIL | FAIL | Success | FAIL ]
Failure Count = 7 / 10 = 70% Failure Rate ---> TRIP CIRCUIT OPEN!

Part 3: Runnable Java Circuit Breaker Engine

package com.example.resilience;

import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicReference;

public class MiniCircuitBreaker {

    public enum State { CLOSED, OPEN, HALF_OPEN }

    private final int failureThresholdPercent;
    private final long sleepWindowMs;
    private final int ringBufferSize = 10;

    private final AtomicReference<State> state = new AtomicReference<>(State.CLOSED);
    private final Boolean[] ringBuffer = new Boolean[ringBufferSize];
    private final AtomicInteger bufferIndex = new AtomicInteger(0);
    private long lastStateChangeTimestamp = System.currentTimeMillis();

    public MiniCircuitBreaker(int failureThresholdPercent, long sleepWindowMs) {
        this.failureThresholdPercent = failureThresholdPercent;
        this.sleepWindowMs = sleepWindowMs;
    }

    public synchronized boolean allowRequest() {
        long now = System.currentTimeMillis();

        if (state.get() == State.OPEN) {
            if (now - lastStateChangeTimestamp > sleepWindowMs) {
                state.set(State.HALF_OPEN);
                lastStateChangeTimestamp = now;
                System.out.println("[CIRCUIT BREAKER] Transitioned OPEN -> HALF-OPEN (Testing probe traffic)");
                return true; // Allow trial probe
            }
            return false; // Reject fail-fast!
        }
        return true; // Allow request in CLOSED or HALF-OPEN
    }

    public synchronized void recordResult(boolean success) {
        int idx = bufferIndex.getAndIncrement() % ringBufferSize;
        ringBuffer[idx] = success;

        if (state.get() == State.HALF_OPEN) {
            if (success) {
                state.set(State.CLOSED);
                lastStateChangeTimestamp = System.currentTimeMillis();
                System.out.println("[CIRCUIT BREAKER] Probe Success! Transitioned HALF-OPEN -> CLOSED");
            } else {
                state.set(State.OPEN);
                lastStateChangeTimestamp = System.currentTimeMillis();
                System.out.println("[CIRCUIT BREAKER] Probe Failed! Transitioned HALF-OPEN -> OPEN");
            }
            return;
        }

        if (state.get() == State.CLOSED) {
            int failures = 0;
            int total = 0;
            for (Boolean res : ringBuffer) {
                if (res != null) {
                    total++;
                    if (!res) failures++;
                }
            }
            if (total >= ringBufferSize) {
                int failureRate = (failures * 100) / total;
                if (failureRate >= failureThresholdPercent) {
                    state.set(State.OPEN);
                    lastStateChangeTimestamp = System.currentTimeMillis();
                    System.out.println("[CIRCUIT BREAKER] Failure rate " + failureRate + "% >= " + failureThresholdPercent + "%. TRIP OPEN!");
                }
            }
        }
    }

    public State getState() {
        return state.get();
    }
}

Part 4: Production Gotchas: Bulkheads & Timeouts

A Circuit Breaker alone is insufficient to guarantee complete isolation. Resilience architectures combine three patterns:

Timeouts: Every network call must enforce an explicit execution timeout (e.g., 1000ms). Without timeouts, a slow dependency will never fail, preventing the Circuit Breaker from recording a failure!
Bulkheads: Isolate thread pools or semaphores per downstream dependency (e.g., Payment pool: 20 threads; Recommendation pool: 10 threads). If recommendations freeze, payment processing threads remain completely unaffected.
Circuit Breakers: Trap failure rates and fail fast during downstream outages.

Next Steps

Now that we understand Circuit Breakers, sliding metrics, and thread pool bulkheads, we will explore Load Balancing Architectures in Part 12: dissecting L4 vs L7 routing and Power of Two Choices.

References & Further Reading

Kleppmann, M. (2017). Designing Data-Intensive Applications (Chapter 11: Stream Processing). O’Reilly Media.
VMware / Pivotal. AMQP 0-9-1 Model Specification and RabbitMQ Architecture. RabbitMQ Docs.
Kreps, J., et al. (2011). Kafka: a Distributed Messaging System for Log Processing. NetDB Workshop.