Debug Foundry

Systems concepts, slowly enough to follow.

Small interactive simulations in a guided order. Pick a path, start at step one, and let each chapter earn the next.

LLM inference

Why a model is slow to answer, and the ten tricks that fixed it.

Start here →
  1. FlashAttentionWatch attention drop its biggest memory bill, tile by tile.
  2. The KV cacheWatch one new token meet the whole past, without re-reading it.
  3. QuantizationWatch a model's weights shrink from 16 bits to 4 — and see what it costs.
  4. Continuous batchingWatch the GPU idle between requests, then fill every gap.
  5. Speculative decodingWatch a small model guess ahead and a big model check four guesses at once.
  6. Attention variants: GQA & MLAWatch eight heads give up their private copies of the past.
  7. The tokenizerWatch a sentence merge into tiles, and tiles become the ids a model sees.
  8. Sampling & temperatureWatch raw scores become shares, temperature reshape them, and one drawn number pick the token.
  9. MoE routingWatch one token visit only the experts it needs — a router scores, a top-2 gate opens, the rest stay dark.
  10. Long contextWatch a prompt outgrow the model twice — positions run out, then memory does — and a rotation trick plus a ring of devices open both doors.

System design

One server is never enough. The standard moves, in the order they earn each other.

Start here →
  1. Load balancerWatch requests queue behind one server, then spread across three — and route around a failure.
  2. CachingWatch a miss fill the cache, hits answer in a blink, and a stale copy expire and refetch.
  3. CDNWatch an edge cache miss once to the origin, then serve its region's hits — while a second region stays cold until its own first ask.
  4. Database replicationWatch a write stream from the primary to the replicas, reads spread out — and one quick read arrive before the copy does.
  5. ShardingWatch one database split into three slices by key, writes and reads route to their own shard — and one celebrity key defeat the whole split.
  6. Message queuesWatch a slow request become a job, a burst turn into a backlog instead of an outage, a crashed worker's job come back — and the duplicate get caught.
  7. Consistent hashingWatch mod-N scatter every key when a server dies, then a hash ring move only one slice — and virtual nodes even the slices out.

C++: owning your memory

Where values live, who owns them, and how they move without copying.

Start here →
  1. Smart pointersWatch C++ memory free itself, owner by owner.
  2. RAII beyond memoryWatch a mutex leak on the early return, then watch one small object release it on every way out.
  3. Move semanticsWatch a copy double a heap block, then watch the same block change owners without duplicating a byte.
  4. The stack and the heapWatch a local die at the closing brace while an owned block survives the function that made it.you are here
  5. Copy elision & RVOThe copies that silently disappear.in the works
  6. Object lifetime & UBWhere "it worked on my machine" comes from.in the works
  7. Allocators & arenasThe garden behind the heap.in the works

C++: the machine

What the hardware really does with your code.

First chapter in the works
  1. Branch prediction & cache-friendly codeThe machine under big-O.in the works
  2. SIMDOne instruction, eight lanes — the speedup hiding inside one core.in the works
  3. The memory model & atomicsWhat one thread is allowed to see. The hardest chapter in the plan.in the works

Agents that act

Models that observe, decide, and do.

First chapter in the works
  1. The agent loopObserve, think, act, repeat.in the works
  2. Tool calling & structured outputThe model hands back a plan, not prose.in the works
  3. Context engineeringWhat fits in the window, and what to drop.in the works