CDNs
Size an edge cache from its hit rate, pick a cache key and TTL that survive real traffic, and ship changes with versioned URLs instead of purges.
A user in Mumbai fetching an image from a server in Virginia waits roughly 200 ms before the first byte moves, and no amount of backend optimisation touches that number — it is the speed of light through fibre. A CDN removes the distance instead, serving the bytes from a machine near the user.
The mechanism
A CDN is a fleet of caching reverse proxies run in a few hundred locations. Each location is a point of presence — a PoP — a facility, not a machine: tens of edge servers racked in a carrier building, a local balancer spreading objects across them by consistent hashing. A large provider runs 300 to 400 of them. "The PoP's copy" of an object is fair only because of that hashing; without it, a PoP of 40 servers takes 40 separate origin misses for one file.
Two layers steer a request to a near PoP, and only one is DNS. GeoDNS decides in the answer: the authoritative nameserver reads the resolver's location and returns a different address per region, so DNS resolution hands a Mumbai client the Mumbai PoP's IP. It re-steers no faster than the record's TTL, 30 to 60 seconds, and reads the resolver's position, not the user's. Anycast decides in the routing table: every PoP advertises the same IP, DNS returns that one address to everyone, and BGP delivers the packet to whichever PoP is fewest AS hops away — DNS plays no part in it. Both find the topologically nearest edge, not the geographically nearest. The rest is ordinary caching — a hit at 20 ms, a miss that pays 200 ms to the origin and stores the object under its TTL.
What the hit rate buys
Assume 100M asset requests/day, average object 200 KB. 1M requests/day is about 12 QPS, so 100M/day is roughly 1,200 QPS average and ~3,600 QPS at a 3x peak. Edge round trip 20 ms, origin 200 ms, so a miss costs 20 + 200 = 220 ms.
hit rate 95%: 0.95 × 20 ms + 0.05 × 220 ms = 19 + 11 = 30 ms effective
origin sees 0.05 × 1,200 = 60 QPS, 12 MB/s
hit rate 90%: 0.90 × 20 ms + 0.10 × 220 ms = 18 + 22 = 40 ms effective
origin sees 0.10 × 1,200 = 120 QPS, 24 MB/s
Total egress at the edge is 1,200 × 200 KB = 240 MB/s, close to 2 Gbps, and almost none of it touches our infrastructure. Nobody notices the 10 ms between 90% and 95%; the origin difference is 2x, and that is the whole argument. We buy the last points of hit rate for the origin fleet, not for the user.
Where the hit rate goes to die
The cache key is the URL plus whatever headers we vary on, and its cardinality decides everything above. Append a per-visitor query parameter — a utm_source, a session id, a cache-busting timestamp — and every request becomes a unique key. Hit rate falls from 95% to near zero, and the origin sized for 60 QPS receives the full 1,200 QPS average, 3,600 at peak, 240 MB/s behind it. A 20x step change arriving in one deploy takes the origin out rather than merely slowing it. We normalise the key at the edge instead: whitelist the query parameters that change the bytes, and vary only on things with a handful of values, like Accept-Encoding or a device class.
The second failure is the expiry stampede. One popular object with a 60-second TTL expires at every PoP at roughly the same moment: 400 fetches of one file from the origin, every minute. Three mechanisms cut that down — request coalescing collapses concurrent misses inside one PoP into a single origin fetch, a shield or tiered cache cuts the fan-in to one fetch overall, and stale-while-revalidate serves the expired copy while the refill runs behind it. We check which of the three we have before treating a low TTL as free.
Invalidation: versioned URLs, not purges
A purge is a fan-out write to every PoP. It converges in seconds to minutes, most vendors rate-limit it, and it cannot reach the copy in a user's browser, which honours the TTL we sent whatever the edge now believes. The rejected alternative is the obvious one: short TTLs everywhere. Freshness problems do disappear, and so does the hit rate — every object revalidates constantly and the origin traffic the CDN was bought to remove comes back.
Put the version in the name instead. Ship /static/app.9f2c1b.js on a one-year TTL with Cache-Control: immutable, and change the reference in the HTML document, which gets 60 seconds because it is small and the only thing that must move quickly. New content means a new URL, a guaranteed miss, correct everywhere within the HTML's TTL — nothing is ever invalidated. The full delivery path for large media is built on the same idea.
In an interview
A CDN is standard in almost any large-scale web design, so proposing one earns nothing. What is tested is whether you know which traffic leaves your infrastructure, and whether you can name a cache key and a TTL instead of pointing at a box. Say it with numbers: "Static assets go to a CDN, keyed on the path with tracking parameters stripped, content-hashed filenames on a one-year immutable TTL, and the HTML document on 60 seconds so a deploy propagates. At a 95% hit rate the origin sees 5% of 1,200 QPS, so I size the origin fleet for 60 QPS and 12 MB/s, not 1,200 QPS and 240 MB/s."
The mistake that loses points is treating a CDN as a fix for dynamic, personalised responses — a page carrying a username has one cache entry per user, a hit rate of zero, and every request pays an extra hop before reaching the origin it was always headed for. The related slip is assuming a purge is instant and global; say instead that the URL is versioned so nothing needs purging.
Check yourself
1. 100M asset requests/day, 200 KB average object, edge 20 ms, origin 200 ms. The hit rate improves from 90% to 98%. What did that buy?
100M/day ÷ 86,400 ≈ 1,200 QPS average. Effective latency goes from 0.90 × 20 + 0.10 × 220 = 40 ms to 0.98 × 20 + 0.02 × 220 = 24 ms, which no user reports. The origin goes from 120 QPS and 24 MB/s to 24 QPS and 4.8 MB/s — five times less traffic and egress bill. Hit-rate work is justified by the origin fleet it deletes, not by the milliseconds.
2. A product page is personalised — a name in the header — but 95% of its bytes are identical for every visitor. Does it go through the CDN?
Not as one assembled document: a per-user cache key has one entry per user, so the hit rate is ~0 and each request pays edge plus origin instead of origin alone. Split by cacheability — bundle and images at the edge on long immutable TTLs, the personalised fragment from the origin. If the document must vary, vary on something coarse — locale or device class, tens of values not millions.
3. A hot-fix ships to app.js. Its TTL is 24 hours and 400 PoPs hold a copy. What do we do, and what does the answer say about the filename?
A purge fans out to 400 PoPs, converges in minutes, is usually rate-limited, and still cannot reach browsers holding the file for up to 24 hours — so it does not solve the problem. Ship
app.<hash>.jsand change the reference in the HTML, on a 60-second TTL: a new URL is an unavoidable miss, so every client is correct within a minute. Needing a purge is the signal that the file was named wrong.