5 Caching Strategies: Tradeoffs, Invalidation, and Stampedes
Compare cache-aside, read-through, write-through, write-behind, and write-around. Design for invalidation races, stampedes, and cache outages.
In this article
Your product page serves 100,000 reads per second. With a 99% cache hit rate, roughly 1,000 reads per second reach the database. Then a deployment empties the cache. The application has not gained any users, but its database suddenly faces up to 100 times the previous read load.
Caching changes the shape of a system’s traffic. A useful design explains both the fast path and what happens when that fast path disappears.
Start with the load you are avoiding
For a simplified read path, database read rate is approximately the incoming read rate multiplied by the miss fraction:
Here is the hit ratio. This calculation excludes writes, refreshes, replication, and retries. It is an estimate for reasoning about capacity, not a benchmark.
- 99% hit rate
- 1,000/s
- Database reads at 100,000 incoming reads/s.
- 90% hit rate
- 10,000/s
- Ten times the database load after a nine-point drop.
- Cold cache
- 100,000/s
- Potential load before admission control intervenes.
Do not size the database around the best observed hit rate without a fallback policy. You may decide that a complete cache outage should shed some traffic. That is a valid architectural choice when it is explicit, measured, and reflected in the service’s availability objectives.
Compare the five caching strategies
Read strategy and write strategy are separate decisions. Cache-aside can coexist with invalidation after writes; read-through can coexist with write-through. The following table describes the common meanings used in this article.
| Strategy | Who loads or updates the cache? | Useful when | Main cost |
|---|---|---|---|
| Cache-aside | Application checks cache, loads a miss, then fills it | Repeated reads with application-owned cache policy | Miss latency and invalidation races |
| Read-through | Cache layer loads from the backing store on a miss | Shared loading logic is worth centralizing | Loader availability and configuration become critical |
| Write-through | Writes go through a layer that updates the backing store and cache before acknowledging under its contract | Recently written values are likely to be read | Write latency and partial-failure handling |
| Write-behind | Acknowledged writes reach the backing store asynchronously | Buffering or batching has a justified benefit | Durability, replay, ordering, and recovery requirements |
| Write-around | Writes update the backing store without populating the cache | Many writes are never read | Existing cached values still need invalidation or expiry |
“Write-through” does not magically make two independently managed systems transactional. You still need an acknowledgment rule and recovery behavior when the database succeeds but the cache update fails. “Write-behind” needs a durable handoff if acknowledged writes must survive a cache failure.
Walk the cache-aside read path
For a product page, first check a cache key that represents the actual response context. A hit returns the cached value. A miss loads the authoritative record and stores a copy with an expiration policy.
Key design is part of correctness. If the response varies by tenant, locale, currency, or authorization scope, those dimensions must be represented or the value must not be shared. A product ID alone is insufficient when different customers see different prices.
product-view:v3:tenant-42:product-918:en-IN:INRThe version makes incompatible representations easy to separate during a rollout. It also means a version change can create a wave of cold misses, so deploy it with the same care as a cache flush.
The Azure cache-aside pattern guide documents this loading approach and its consistency limitations. Do not assume cached and authoritative records remain synchronized simply because both are healthy.
The invalidation race that surprises people
A common write path commits the database update and then deletes the cached copy. This is useful, but it does not close every race.
The reader fetched an older value before the write committed, then filled the cache afterward. The database-first invalidation sequence reduces some races, but it is not a proof of strong consistency.
Choose a remedy that matches the product’s promise:
- Tolerate it deliberately: a short TTL may be sufficient for a description or thumbnail. Define the freshness target and measure stale responses.
- Coordinate versions: retain an authoritative version or invalidation watermark and reject older fills. Ordering and atomic comparison are essential; merely attaching a version field is insufficient.
- Bypass the cache for critical decisions: checkout should validate current price and stock against the authoritative system even if browsing uses a cached view.
- Change the read model: a versioned immutable object can use a version-specific cache key, while a separately coordinated pointer identifies the current version.
A TTL bounds how long an entry remains after it is filled. It does not automatically bound how old the underlying data was when that fill occurred. Replica lag, delayed fills, and stale refreshes all matter to an end-to-end freshness claim.
Prevent a cache stampede
Suppose a popular product expires while 2,000 requests are in flight. If every miss performs the same database query, one expired key can create a burst of duplicate work.
Request coalescing lets one loader fetch a key while other callers wait for that same result. Within one process, this can be an in-flight promise map. Across a fleet, local coalescing still permits one load per process; stronger coordination may require a shared lease or another ownership mechanism.
Bound the waiting time and the number of waiters. Clear an in-flight entry on failure, and ensure a crashed refresh owner cannot block the key forever. A shared lease needs an expiry and a rule preventing an old owner from overwriting a newer fill.
Two complementary techniques solve different problems:
| Technique | What it helps with | What it does not guarantee |
|---|---|---|
| TTL jitter | Spreads expiration of many keys over time | Does not stop a stampede on one very hot key |
| Request coalescing | Reduces duplicate concurrent loads for one key | Does not remove load from many distinct cold keys |
| Serve stale while refreshing | Keeps reads fast during a permitted stale window | Is unsafe for data requiring immediate revocation or freshness |
| Prewarming | Loads an expected working set before traffic arrives | Does not predict every key or eliminate miss traffic |
| Origin concurrency cap | Prevents unbounded fallback load | Some requests may be rejected or delayed |
With stale-while-refresh behavior, specify both a soft refresh threshold and a hard serving limit. If the background refresh repeatedly fails, an item should not remain acceptable forever just because the stale path is fast.
Keep cache failure from becoming database failure
The fallback path needs its own capacity budget. Consider a cache lookup timeout of a few milliseconds as an example policy, not a universal recommendation. Once the cache is classified as unavailable, repeatedly waiting on it can waste the entire request deadline.
Protect the database with a bounded number of concurrent fills. If that budget is exhausted, choose a response appropriate to the data: return an allowed stale value, offer a reduced view, or fail promptly. Avoid retrying a failed cache lookup and a failed database read repeatedly within the same request.
Amazon’s caching challenges and strategies describes why cache loss, cold starts, and traffic changes must be treated as operational scenarios. Its guidance supports testing the service with its cache unavailable rather than relying on normal hit-rate behavior.
Negative caching can also reduce repeated requests for missing records. Give genuine “not found” results a deliberate, usually short lifetime, and invalidate them when a record is created. A timeout is not evidence that a record does not exist; do not convert transient failures into long-lived absence.
Expiration is not eviction
Expiration removes an entry because its lifetime is over. Eviction removes an entry to satisfy a resource policy, often because memory is full. An unexpired value can still disappear.
Redis offers policies based on recency, frequency, TTL, and other choices. Its LRU behavior is approximate, and the selected policy determines which keys are eligible. Use the Redis eviction reference to choose and verify the exact configuration.
A practical review should ask:
- How large is the hot working set, including serialization and metadata overhead?
- Can a small number of oversized values displace useful small entries?
- Does eviction create a sustained miss rate the database can absorb?
- Is the workload mostly repeated hot keys or mostly one-off requests?
- Does the cache contain durable information that must not be silently evicted?
Do not treat an evictable performance cache as the only copy of a queue, idempotency result, or acknowledged business record. Different responsibilities can justify separate stores or configurations.
Measure the slow path, not just the average
For a simple sequential lookup, average latency is roughly cache-lookup latency plus miss probability times database-read latency. If lookup takes 2 ms, a database read takes 20 ms, and hit rate is 99%, the estimate is 2.2 ms, excluding fills and other overhead.
That average hides the misses. At high percentiles, origin latency and stampedes can dominate. Track hit rate alongside miss latency, fill concurrency, eviction rate, cache timeouts, stale responses, and database saturation. Break down useful segments such as endpoint or key family without creating unbounded metric cardinality.
Frequently asked questions
Should every system have Redis?
No. A cache helps when it avoids enough repeated expensive work to justify its network, memory, correctness, and operational costs. First measure the bottleneck and consider query/index improvements. A low-reuse workload may gain little from a cache.
Is deleting the cache after a database write enough?
It is a useful baseline for some freshness requirements, but delayed reads can repopulate stale values after deletion. Stronger guarantees require coordination, version enforcement, or a read path that consults authoritative state.
How do I choose a TTL?
Start with how stale the product permits the value to be, then account for source lag, refill timing, update frequency, load, and eviction. Measure the result. There is no single TTL that is right for descriptions, inventory, authorization, and analytics.
Put the cache under pressure
Review your design with three exercises: flush the working set, expire the most popular key, and update a value while a slow reader is filling it. Explain the response each caller sees and the maximum origin work allowed.
Continue with cache stampedes, cache invalidation, and distributed caching. Apply the read/write tradeoff to a URL shortener, or use a rate limiter to bound admitted work before it reaches the cache.