What it's for
- The same question shouldn't cost a database trip every time. A cache keeps hot answers close, so the database only works on what's new.
- Load balancing — the previous chapter — spreads work across machines. Caching is the complementary move: don't do the same work twice.
- Caching helps most when a few answers are asked for constantly — a homepage, a profile, a price — and barely helps when everything is asked once.
Good to know
The pattern shown is application-managed cache-aside: the server itself checks the cache first, on a miss asks the database and stores a copy, on a hit never touches it — the cache is not an invisible proxy sitting in front of the database. Note that a miss costs the lookup plus the full database trip plus storing the copy, so a request that never repeats is slightly cheaper with no cache at all. The hard part is the copy going stale. Shown here is the usual fix: a TTL, a clock that starts when the copy is stored and expires lazily — nothing refetches until the next request asks, and a write that lands before expiry still leaves the copy stale. Expiry is not eviction (dropping entries when the cache is full, not shown) and not explicit invalidation (the writer deleting the copy the moment the data changes, also not shown). Real systems also deal with stampedes (a hot key expiring under load), cache penetration, and write-through or write-back policies; none of that is shown. Placement varies too: beside the app server like here, in a shared store such as Redis or Memcached, in the browser, or at the edge — the CDN chapter picks that up.
How it works, step by step
Why the steps go in this order
- The cost comes first: every request paying the database is the problem a cache exists to fix.
- Repetition is the opportunity — the same question asked constantly is what makes remembering worthwhile.
- Only then does the cache appear, so its job is obvious: answer without asking.
- A miss before a hit, because the cache starts empty and the first ask still pays full price.
- Staleness and expiry come last: remembering is easy, knowing when to forget is the real design.
Before you rely on it
This is a toy: one server, one cache, one hot answer, hand-set timings ("~100 ms" database, "~1 ms" cache are illustrative, not measurements). "9 of 10 from the cache" is this toy's hit rate over its declared ten requests, not a benchmark — real hit rates depend on the workload, the TTL and the cache size, and a bad one can be worse than no cache. The cache trades memory and freshness complexity for less repeated work; it never removes the database — every miss, cold start and expired copy still needs it, and when the cache is down every request goes back to paying full price. Timings vary by orders of magnitude in practice; the cited sources describe production systems with their own measured numbers.