Caching strategies and invalidation
Pick a caching strategy from the read/write ratio, price the miss path in milliseconds, and defend the cache against stampedes, hot keys and stale reads.
Two decisions define a cache: where the copy lives, and who is responsible for putting it there. Placement is settled by the latency ladder — an in-process map read costs about 100 ns, a Redis read across the datacenter about 0.5 ms, a Postgres read off SSD a few milliseconds — so moving a copy up one rung is worth roughly a factor of a thousand. Strategy is settled by the read/write ratio and by how much staleness the data tolerates. Both go wrong the same way: a design that works while the cache is warm and takes the database down when it is not.
Where the copy lives
A request passes through the layers in order and stops at the first one holding the value.
Client-side. Browsers cache HTML, CSS, JS and images according to Cache-Control and Expires; mobile apps cache API responses locally. This is the only layer that removes the network request entirely, and the one you control least — a bad max-age shipped to a million clients cannot be recalled.
CDN. Geographically distributed proxy servers holding content near users. This is the only layer that helps with distance: a round trip from India to US East is about 200 ms, and no server-side cache removes it. CDNs also absorb volumetric traffic that would otherwise reach the origin. See CDNs.
Load balancer. Some Layer 7 balancers cache backend responses. Cheap to switch on, less capable than a dedicated cache, and worth knowing about rather than designing around.
Application, in-process. A map inside the server process, at about 100 ns — five hundred times faster than Redis because there is no network hop. The cost is copies: with 20 app servers you hold 20 versions of the same entry and an invalidation must reach all 20. Keep it for feature flags and configuration, which tolerate seconds of skew.
Application, distributed. Redis or Memcached, shared by every instance, surviving app restarts, and holding one authoritative copy per key. One network round trip, 0.5 ms, against a database read an order of magnitude slower.
Database. Buffer pools and query caches inside the engine. Transparent, and it optimises queries rather than caching your objects, so it is not a substitute for a layer you control.
Read strategies
Cache-aside (lazy loading). The application checks the cache; on a hit it returns, on a miss it reads the database, writes the value into the cache, and returns it. Only data somebody asked for is ever cached, and if the cache dies the application still works — slowly. The costs are the miss penalty on first read and staleness between updates. This is the default, and the strategy to name unless you have a reason not to.
Read-through. The application queries the cache as if it were the database; the cache itself fetches from the store on a miss, stores the result, and returns it. Application code is simpler because there is one data source, not two. It needs provider support, carries the same miss penalty and staleness, and removes the cache-aside escape hatch: with no code path to the database, a cache outage is a full outage.
Write strategies
Write-through writes to cache and database together and acknowledges only after both. Cache and database never disagree and nothing is lost if the cache node dies. The write pays for both hops, so an acknowledged write costs the database commit — round it to 5 ms rather than 0.5 ms.
Write-back (write-behind) acknowledges as soon as the cache holds the value and flushes to the database asynchronously, usually in batches. Writes are ten times faster and the database sees a fraction of them. An acknowledged write is lost if the node fails before the flush, so the honest way to state it is a bounded loss window: "up to one second of writes."
Write-around writes straight to the database and lets the value enter the cache on a later read. It keeps write-only data from evicting entries people actually read, at the price of a guaranteed miss on the first read of anything just written.
Choosing from the ratio
Take a product catalogue at 1 billion reads/day and 10 million writes/day. Reads are 1B ÷ 86,400 s ≈ 12,000 QPS average and, at a 3× peak, 36,000 QPS. Writes are 10M ÷ 86,400 ≈ 116 QPS. Write-around costs at most one guaranteed miss per write, so its ceiling on hit rate is 116 ÷ 12,000 ≈ 1% — negligible. Cache-aside reads plus write-around writes is the correct default here, and a 95% hit rate leaves 36,000 × 0.05 = 1,800 QPS on the origin, inside the roughly 5,000 simple QPS a single commodity Postgres box handles.
Invert the ratio and the answer inverts. A counter service takes 20,000 writes/s against 50 dashboard reads/s. Write-through would put 20,000 QPS on a database rated for 5,000 — four times over. If those increments land on about 200 hot keys and the cache flushes each key once a second, the database sees 200 writes/s instead of 20,000, a factor of 100. Write-back is the only strategy that fits, and the price is up to a second of acknowledged counts lost on a node failure. For view counts that is acceptable; for account balances it is not, which is the whole decision.
Eviction
When memory is full something must go. LRU evicts the entry unused for the longest and is the right default when access is recency-skewed. LFU evicts the least frequently used and protects a stable hot set from a one-off scan that would flush an LRU cache. FIFO evicts in insertion order and ignores access entirely — cheapest, weakest. Pick from the access pattern, not from habit.
Invalidation
A TTL bounds staleness with no coordination: 60 seconds means readers may see 60-second-old data and you have accepted that. Active invalidation on write is exact and harder, and the verb matters — delete the key rather than overwrite it. Two writers can reach the database in one order and the cache in the other, stranding the older value with only the TTL to clear it; a delete sends the next reader to the source of truth, so the worst case is one extra miss. What staleness costs depends on the guarantee you claimed, which is the subject of consistency models. If a read misses to a follower replica rather than the leader, replication lag adds its own staleness on top: a user who just edited their profile can read back the old one, and read-your-writes is broken.
Three ways a cache fails under load
Stampede. A hot key expires at 36,000 QPS. Refilling takes about 6 ms, and every request arriving in that window also misses: 36,000 × 0.006 ≈ 216 simultaneous database queries for one row. Fix it with single-flight — one request per key goes to the origin, the rest wait on its result — and with TTL jitter, so keys written together do not expire together.
Penetration. Requests for keys that do not exist miss every time and reach the database every time, which is what a scraper or an attacker produces. Cache the negative result with a short TTL, 30 seconds, or keep a Bloom filter of existing ids.
Hot key. One celebrity entry at 36,000 QPS lands on the single shard that owns it. Across 20 shards the average is 1,800 QPS, so that shard carries 20× its share. Put the key in the in-process cache with a 1-second TTL: 20 app servers each refresh once a second, and Redis sees 20 requests/s instead of 36,000. Because the miss path dominates the tail, the number to watch here is p99, not the mean — see tail latency.
In an interview
The interviewer is testing whether you treat a cache as an optimisation with a failure mode, or as free speed. Say where you would place caches in the design — CDN for static assets, a distributed cache for hot rows — and name the strategy with the ratio that justifies it: "reads outnumber writes 100 to 1, so cache-aside with write-around; at a 95% hit rate the primary sees about 1,800 QPS at peak, which one box handles." Then say the eviction policy and the TTL or invalidation rule that handles stale data, and state the trade-off you accepted rather than waiting to be asked.
The mistake that loses points is sizing the origin for cached load and never asking what happens when the cache is empty. A restart at peak sends the full 36,000 QPS at a database provisioned for 1,800; queries slow from 5 ms to 200 ms, and by Little's Law in-flight work goes from 12,000 × 0.0008 ≈ 10 requests to 12,000 × 0.2 = 2,400, which no connection pool holds. If your database survives only while the cache is warm, you have built a single point of failure and called it fast. Two smaller ones: proposing write-back for data whose loss you cannot defend, and saying "invalidate on write" without saying whether you delete or overwrite. Sizing from a hit rate is covered in caches and the cost of a miss; the origin capacity question continues in database scaling.
Check yourself
1. A payments service writes 800 transactions/s and reads each one back rarely. Someone proposes write-back for the speed. What do you say?
No. Write-back acknowledges before the database has the row, so a node failure loses up to a flush interval of acknowledged writes — a second of transactions is 800 payments the user believes succeeded. The write volume is also well inside the roughly 5,000 QPS one Postgres box handles, so there is nothing to buy. Write-through, or write-around with no cache on the write path at all, since the reads are rare.
2. Your cache holds rendered profile pages on a 300-second TTL. Product wants edits visible within 5 seconds. Do you drop the TTL to 5 seconds?
No. A 60× shorter TTL multiplies the miss rate roughly 60-fold on unchanged pages, and profiles change far less often than every 5 seconds, so you would pay for freshness on every key to fix the 1% that were edited. Keep the long TTL and delete the key on write. The TTL becomes the backstop for missed invalidations rather than the mechanism.
3. One key takes 30,000 QPS at peak and its Redis shard is saturating. Estimate the load after adding a 1-second in-process cache on 15 app servers.
Each server serves that key from local memory for a second at a time, so it fetches from Redis at most once per second:
15 servers × 1 fetch/s = 15 QPS, down from 30,000 — a factor of 2,000. The cost is up to 1 second of staleness and up to 15 different answers in flight at once, which is fine for a view count and wrong for a permission check.