LT LearningTechBasics @amtocbot
How CPU Caches Work
Why the same loop can run 10× faster depending on memory order.
Your CPU is far faster than your RAM. Caches — small, fast memories close to the core — bridge the gap by keeping recently and soon-to-be-used data nearby.
ACPU registers · <1ns
BL1 cache · ~1ns
CL2 cache · ~4ns
DL3 cache · ~15ns
EMain RAM · ~100ns
Legend — how to read this diagram
A–EComponentsthe parts involved, labelled in the diagram
1 2 3Walkthroughnumbered steps below run in order
The memory hierarchy
- Registers. A few dozen slots inside the core, accessed in a single cycle.
- L1/L2. Per-core caches, kilobytes to a megabyte, a handful of nanoseconds away.
- L3. Shared across cores, several megabytes, slower but still far faster than RAM.
- RAM. Gigabytes, but ~100× slower than L1 — a cache miss stalls the core.
Writing cache-friendly code
Locality wins. Data fetched in 64-byte cache lines; touching neighbors is nearly free.
Iterate in memory order. Row-major arrays should be traversed row by row, not column by column.
Compact structures. Smaller, contiguous data fits more per line and misses less often.
One-line mental model:
Speed isn't just how many operations you do — it's how far the data had to travel to reach the core.
No comments:
Post a Comment