From 250ms to 0ms: how requests short-circuit through 4 cache layers
TL;DR
Systems usually have several cache layers: the browser, a CDN, Redis, then the database. A request falls from the top and stops at the first layer holding the data. The first (all-miss) trip is the most expensive, but on the way back each layer stores a copy for next time.
A page loads in 250ms the first time and almost instantly the second. Most of that difference comes from multi-layer caching. Web systems usually stack cache layers in order: the browser, the CDN, Redis (or an application cache), and only then the database. A request travels down and stops at the first layer that has the data. The first time (a miss everywhere) costs the most, but on the way back the data is stored at each layer for next time.
Three requests
| Request | Stops at | Time |
|---|---|---|
| 1: cold cache | Database (missed all 3 cache layers) | ~250ms |
| 2 | Browser | ~0ms |
| 3: upper-layer TTLs expired | Redis | ~80ms |
The role of each layer
| Layer | Stores | Controls |
|---|---|---|
| Browser | Static assets, rarely changing APIs | Cache-Control, ETag |
| CDN | Content shared by many users, close to them | Cache rules, purge, TTL |
| Redis / app cache | Query results, sessions, expensive computed data | TTL, deleting keys when data changes |
| Database | The source of truth | Indexes, the DB's own buffer cache |
The earlier, the cheaper
The deeper the layer, the more each hit costs in resources and time. A browser cache uses no bandwidth and no server; a CDN cache shields the origin; Redis shields the database. The goal is to stop requests at the highest possible layer.
Classic problems
- Stale data: the cache keeps an old value after the data changes.
- Cache stampede: when a popular key expires, many requests hit the database at once; mitigate with locks or early refresh.
- Private data in a shared cache: the CDN mistakenly caches a personal response.
- Hard to debug: you don't know which layer the data came from; add headers/logs showing HIT/MISS.
How to apply it
- Stop requests as early as possible: prefer browser/CDN caching for static or rarely changing content (
Cache-Control,ETag). - Set TTLs by how much staleness each data type tolerates: product prices differ from avatars.
- Have an invalidation strategy for when data changes (delete Redis keys, purge the CDN); don't just wait for TTLs.
- Be careful with private data: don't let the CDN cache responses containing personal information (
Cache-Control: private). - Track each layer's hit rate; a low hit rate means the cache costs money without helping.
Sources & references
- HTTP caching — MDN Web Docs