Caching Layer Problems: Stale Data & Cache Stampede

A cache trades correctness for speed — it serves a stored copy of data instead of recomputing or refetching it, on the bet that the copy is still good enough to use. That bet is usually right. This chapter is about the two genuinely different ways it goes wrong: a cache quietly serving something that's no longer true, and a cache emptying itself all at once and taking the backend down with it for a few seconds.

Cache Invalidation: The Genuinely Hard Problem

Two common strategies, each with a real, honest tradeoff:

Neither approach is simply "better" — TTL-based caching accepts a known, bounded staleness window up front; explicit invalidation aims for zero staleness but is only as reliable as every single write path that's supposed to trigger it, which in a real system with multiple services and admin tooling is genuinely easy to miss.

Reading a Stale-Data Complaint

Before assuming a "wrong data" complaint is a cache bug, confirm it directly — compare what the cache is actually returning against what the real source of truth currently holds for the same key:

A genuine mismatch confirms a stale cache. If the two actually agree, the cache isn't the problem at all — the "wrong data" complaint is something else entirely (a display bug, a genuine data error, or user confusion), and continuing to chase caching as the cause would be a wasted detour.

Cache Stampede: When Many Requests Miss at Once

A stampede (also called a thundering herd) happens when a popular cache key expires, and many concurrent requests all miss the cache at the exact same instant — all of them then hit the backend simultaneously to regenerate the same value, producing a sudden, sharp spike in backend load precisely at the moment of expiry.

Real mitigations worth knowing: jittered TTLs (randomizing expiry slightly per key, so not every instance expires at the exact same moment), a single-flight pattern (only one request regenerates the value while everyone else waits for that result instead of duplicating the work), and stale-while-revalidate (serving the slightly-stale value immediately while quietly regenerating it in the background).

Recognizing a Stampede From the Outside

A stampede caused by a fixed TTL has a genuinely distinctive signature: it recurs at a suspiciously regular, clock-aligned interval — every hour, on the hour, if the TTL is exactly 3600 seconds. That periodicity itself is real diagnostic evidence, not a coincidence worth ignoring.

Working Example: The Hourly Database Spike

A fresh ticket: every hour, right on the hour, the database briefly spikes to near-100% CPU for about 10 seconds, then returns to normal — otherwise, everything is healthy. The clock-aligned periodicity is the immediate tell. Checking a suspected key's own remaining time-to-live confirms it:

A TTL close to a full hour, on a genuinely popular key — the homepage's own featured-products list, requested by nearly every visitor. Every instance of the application shares the same cache key, so every one of them experiences the exact same expiry moment simultaneously, and every request that arrives in that brief window misses the cache and hits the database at once. Adding a small amount of random jitter to the TTL (so different instances' copies expire at slightly different times) spreads that load out instead of concentrating it into one sharp spike every hour.

Hands-On Exercises