Memory hierarchy — registers, cache (L1 - L2 - L3), RAM, SSD, HDD — speed - size trade-offs

1.1.9 · Coding › How Computers Work

WHAT hai memory hierarchy?

Levels, fastest → slowest:

Level	Typical size	Approx. access time	Volatile?
Registers	~1–2 KB (kuch dozen × 64-bit)	~0.3 ns (<1 cycle)	haan
L1 cache	~32–64 KB	~1 ns (~4 cycles)	haan
L2 cache	~256 KB–1 MB	~4 ns (~12 cycles)	haan
L3 cache	~8–32 MB (shared)	~10–20 ns (~40 cycles)	haan
RAM (DRAM)	~8–64 GB	~100 ns	haan
SSD	~256 GB–4 TB	~100,000 ns (100 µs)	nahi
HDD	~1–16 TB	~10,000,000 ns (10 ms)	nahi

Figure — Memory hierarchy — registers, cache (L1 - L2 - L3), RAM, SSD, HDD — speed - size trade-offs

WHY trade-off exist karta hai (first principles)

HOW yeh actually kaam karta hai: locality + caching

Locality ki wajah se, CPU data ko cache mein cache lines (typically 64 bytes) ke chunks mein laata hai, single bytes mein nahi. Ek request jo already cache mein hai woh hit hai; warna woh miss hai aur slower level se fetch karni padti hai.

Average Memory Access Time (AMAT) derive karna

Hum chahte hain expected time pata karna ek program mein memory access karne ki. WHY derive karna? Kyunki yeh hume exactly batata hai ki low hit rate kitna hurt karta hai.

Maano:

$h$ = ek level par hit rate (accesses ka fraction jo wahan milta hai), $0 \leq h \leq 1$
$t_{hit}$ = time agar is level par hit ho
$T_{lower}$ = average time agle (slower) level se lene ka agar miss ho

Expected value ki definition se, probability $h$ se hum $t_{hit}$ pay karte hain, aur probability $(1 - h)$ se hum additionally neeche level tak jaane ka paisa dete hain:

$AMAT = h \cdot t_{hit} + (1 - h) (t_{hit} + T_{lower})$

Expand aur simplify karo ( $t_{hit}$ dono cases mein pay hota hai):

$AMAT = t_{hit} + (1 - h) \cdot T_{lower}$

Worked examples

Common mistakes (Steel-manned)

Recall Feynman: ek 12-saal ke bacche ko samjhao (click to reveal)

Socho tum homework kar rahe ho. Tumhare haath abhi jo pencil use ho rahi hai use pakde hain (registers). Tumhari desk par is ghante ki zaruri kitabein hain (cache). Bookshelf kamre ke doosri taraf par aur kitabein hain (RAM). Library shahر mein har kitaab hai lekin wahan jaana padta hai (SSD/HDD). Tum sabse zyada use hone wali cheez desk par rakhte ho taaki library kabhi jaana na pade. Computer bhi yahi karta hai: jo data sabse zyada use karta hai use sabse paas, fastest jagah rakhta hai, taaki slow trip almost kabhi nahi karni padti.

Active recall

Order the hierarchy fastest to slowest

Registers → L1 → L2 → L3 cache → RAM → SSD → HDD

Fast memory hamesha small aur expensive kyun hoti hai?

Fast cells (SRAM) ~6 transistors/bit use karte hain aur physically core ke paas rakhne padte hain (signal-distance limit), isliye cost aur area capacity limit karte hain.

AMAT derive karo

Hit prob h se t_hit pay karo; miss prob (1−h) se t_hit + T_lower pay karo. AMAT = h·t_hit + (1−h)(t_hit + T_lower) = t_hit + (1−h)·T_lower.

Temporal locality kya hai?

Haal hi mein use kiya gaya data jald hi phir use hone ki sambhavana hai (jaise loop variables).

Spatial locality kya hai?

Haal hi mein access kiye gaye address ke paas ka data jald hi access hone ki sambhavana hai (jaise array elements).

Cache line kya hai?

Fixed-size block (~64 bytes) jo CPU memory levels ke beech transfer karta hai, spatial locality exploit karke.

Cache hit vs miss?

Hit = data is fast level mein mil gaya; miss = nahi mila, slower level se fetch karna padega (miss penalty pay karke).

Kaunse levels volatile hain?

Registers, saare cache levels, aur RAM power-off par data lose karte hain; SSD aur HDD non-volatile hain.

Two-level AMAT with t_L1=1ns, h=0.95, RAM=100ns

1 + 0.05·100 = 6 ns.

Zyada RAM directly CPU kyun fast nahi karta?

CPU speed cache hit rates par depend karti hai; extra RAM sirf tab help karta hai jab aap disk par swap kar rahe the.

RAM vs SSD ka key difference?

RAM volatile working memory hai (~100ns); SSD non-volatile persistent storage hai (~100µs), ~1000× slower.

Miss par 1 byte ki jagah 64 bytes kyun fetch karte hain?

Spatial locality — nearby data ki zarurat hogi, isliye slow fetch ko kaafi useful bytes par amortize karo.

Connections

CPU and instruction cycle — registers har cycle ALU ko feed karte hain
Cache coherence and multicore — L3 shared kyun hai
Locality of reference — woh principle jo hierarchy ko kaam karta hai
Virtual memory and paging — RAM ko SSD/HDD par extend karna
Big-O vs constant factors — cache-friendly code theoretically equal algorithms ko bhi beat karta hai
Volatile vs non-volatile storage