Distributed Caching Patterns: Cache-Aside, Write-Through, Write-Back, and Cache Stampedes

Why You Need This in Real Life

At 08:00 AM on breaking news day, a popular article key (news:article_101) expires in Redis.

During the single second while the key is absent from the cache, 25,000 incoming HTTP worker threads query Redis, find a cache miss, and simultaneously execute the heavy SQL query SELECT * FROM articles JOIN authors JOIN comments WHERE id = 101 against the primary database.

This phenomenon is the Cache Stampede (Thundering Herd Problem).

Within 500 milliseconds, database CPU utilization hits 100%, query execution times explode, and the database server crashes.

Caching is the single most effective technique for scaling read-heavy systems, but misconfigured caching patterns lead to severe failure modes: stale data reads, cache stampedes, cache penetration, and cache breakdown.

To design high-performance caching layers, you must master Cache-Aside, Write-Through, Write-Back, Singleflight Locks, and Probabilistic Early Expiration (XFetch).

Part 1: Caching Patterns Breakdown

1. Cache-Aside (Lazy Loading)
Application ---> 1. Read Cache (Miss!) ---> 2. Read DB ---> 3. Write Cache

2. Write-Through
Application ---> 1. Write Cache ---> Cache Manager ---> 2. Write DB (Synchronous)

3. Write-Back (Write-Behind)
Application ---> 1. Write Cache (Returns Immediately!) ---> Cache Async Queue ---> 2. Write DB (Batch Async)

1. Cache-Aside (Lazy Loading)

The application code manages cache interactions directly:

Read Path: Application queries cache. On hit, return data. On miss, query database, populate cache, and return data.
Write Path: Application updates database, then deletes (invalidates) the key from the cache.
Pros: Resilient against cache node failures (system falls back to DB). Cache only contains data that is actively requested.
Cons: Initial read latency on cache miss; danger of stale reads if invalidation fails.

2. Write-Through Caching

The application treats the cache as the main data store. The cache manager synchronously writes data to the database before returning success.

Pros: Guarantees cache and database consistency; no stale reads.
Cons: High write latency (writes must wait for DB write completion).

3. Write-Back (Write-Behind Caching)

The application writes data to the cache, which returns success immediately. The cache layer asynchronously flushes dirty data pages to the database in batches.

Pros: Ultra-low write latency and high write throughput (absorbs database write spikes).
Cons: Risk of data loss if the cache node crashes before flushing dirty pages to DB.

Part 2: Severe Caching Anomalies & Defenses

1. Cache Penetration

Problem: Malicious clients query keys that do not exist in either the cache or the database (e.g., user_id = -99999). Every request bypasses the cache and hits the database.
Solution 1: Cache null values with a short TTL (SET user:-99999 "NULL" EX 30).
Solution 2: Use a Bloom Filter at the API gateway to test whether a key exists before querying the database.

2. Cache Breakdown (Hotspot Expiration)

Problem: A highly popular key (e.g., celebrity profile) expires, causing a flood of concurrent requests to hit the database simultaneously.
Solution: Use Mutex Locks (Singleflight) or Probabilistic Early Expiration.

Part 3: Mitigating Cache Stampedes: Singleflight Mutex Lock

To prevent 25,000 requests from querying the database simultaneously when a hotspot key expires, use a Singleflight Mutex Lock. Only the first thread that experiences a cache miss acquires the lock to query the database; all subsequent threads wait for the first thread to populate the cache!

Request 1 (Miss!) ---> Acquires Lock ---> Queries DB ---> Writes Cache ---> Releases Lock
Request 2 (Miss!) ---> Waits for Lock -----------------------------------> Reads Cache!
Request 3 (Miss!) ---> Waits for Lock -----------------------------------> Reads Cache!

Java Singleflight Cache Loader Implementation

package com.example.cache;

import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.CompletableFuture;
import java.util.function.Function;

public class SingleflightCache<K, V> {

    private final ConcurrentHashMap<K, V> cache = new ConcurrentHashMap<>();
    private final ConcurrentHashMap<K, CompletableFuture<V>> inFlightRequests = new ConcurrentHashMap<>();

    public V get(K key, Function<K, V> databaseLoader) {
        // 1. Try cache read
        V value = cache.get(key);
        if (value != null) {
            return value;
        }

        // 2. Singleflight deduplication: Only ONE thread executes databaseLoader per key
        CompletableFuture<V> future = inFlightRequests.computeIfAbsent(key, k -> {
            CompletableFuture<V> newFuture = new CompletableFuture<>();
            // Execute heavy database loader on a background thread
            CompletableFuture.runAsync(() -> {
                try {
                    System.out.println("[SINGLEFLIGHT] Querying Database for Key: " + key);
                    V dbValue = databaseLoader.apply(k);
                    if (dbValue != null) {
                        cache.put(k, dbValue);
                    }
                    newFuture.complete(dbValue);
                } catch (Throwable ex) {
                    newFuture.completeExceptionally(ex);
                } finally {
                    inFlightRequests.remove(k); // Cleanup in-flight tracker
                }
            });
            return newFuture;
        });

        try {
            return future.join(); // Wait for single database query to complete
        } catch (Exception e) {
            throw new RuntimeException("Failed to load key: " + key, e);
        }
    }
}

Part 4: XFetch: Probabilistic Early Expiration

Another advanced technique to eliminate cache stampedes is XFetch (Probabilistic Early Expiration).

As a request reads a cached item, the algorithm probabilistically recomputes and refreshes the cache before the item expires based on how close it is to expiration and how long the database read took:

$\text{Recompute If: } -\beta \times \Delta \times \ln(\text{random}()) > \text{TTL} - \text{Current Time}$

Where $\beta > 0$ is a constant, and $\Delta$ is the delta computation time to query the database. As TTL approaches 0, the probability of background recomputation approaches $100\%$ , guaranteeing the key never expires for active users!

Next Steps

Now that we understand distributed caching patterns and cache stampede mitigations, we will explore Event-Driven Architecture, CQRS, and Event Sourcing in Part 14.

References & Further Reading

Garcia-Molina, H., & Salem, K. (1987). Sagas. ACM SIGMOD ‘87, 249–259.
Gray, J. (1978). Notes on Data Base Operating Systems: Two-Phase Commit Protocol. Springer.
Richardson, C. (2018). Microservices Patterns: With examples in Java. Manning Publications.