Processing-in-memory (PIM): Visual walkthrough — Hinglish deep dive

6.5.5 · D2 · Hardware › Advanced & Emerging Architectures › Processing-in-memory (PIM)

Neeche sab kuch zero se build kiya gaya hai. Agar koi symbol aata hai, usse pehle draw kiya gaya tha.

Step 1 — Jab ek program data touch karta hai toh actually kya hota hai

Inhein alag kyun karein? Kyunki agar hum "program chalana" ko ek blob of time mein lump kar dein, hum nahi dekh sakte kaunsa part hume slow kar raha hai. Poora PIM argument is fact par tika hai ki movement aur compute alag hain, aur unmen se ek dominate karta hai. Toh pehla kadam yeh hai ki inhe alag naam dein.

Do quantities define karta hoon jinhe hum baaki page par track karenge:

$N$ = bytes ki sankhya jo workload move karna chahta hai (byte = 8 bits, woh unit jismein memory measure hoti hai).
$C$ = compute operations ki sankhya (ek "op" = ek arithmetic action, jaise ek add).

PICTURE. Chef-aur-warehouse cartoon, literal banaya gaya. Number door memory mein rehta hai; ek lamba wire usse processor tak le jaata hai; sirf wahan "+" hota hai.

Do arrows dekho: lamba red wala movement hai, chhota green wala compute hai. Step 2 poochta hai kaunsa zyada costly hai.

Step 2 — Movement expensive part hai (energy picture)

Yeh sab kuch kyun decide karta hai? Agar expensive part compute hai, toh aap faster arithmetic unit bana ke cheezein speed up karte ho. Agar expensive part movement hai, toh faster arithmetic unit kuch nahi badalta — aap sirf faster ALU par wait karte. Toh kuch bhi design karne se pehle hume pata karna hoga kaunsa term dominate karta hai.

Fairly compare karne ke liye, dono energies same size ke ek operand per measure honi chahiye — maan lo ek 4-byte (32-bit) word, woh natural unit jo ek "add" consume karta hai. Is tareh $E_{m o v e}$ aur $E_{o p}$ exactly usi chunk of data ko refer karte hain, aur unka ratio meaningful hai.

PICTURE. Ek hi axis par do bars: ek chhoti green bar (compute energy) ek towering red bar (move energy) ke saath, dono ek word ke liye. Height ka gap hi ek word ke andar memory wall hai energy form mein.

Step 3 — Classic machine par total time likhna

Yeh shape kyun? Time = work / rate sabse basic timing law hai: agar aap $N$ bytes $B$ bytes per second par move karte ho, toh $N / B$ seconds lagte hain. Compute ke liye bhi yahi logic. Hum ise isliye use karte hain kyunki yeh bottleneck ko clearly isolate karta hai — har fraction ek independent stage hai.

Do naye rates enter hote hain, dono use se pehle define kiye gaye hain:

$B$ = bandwidth: external bus (lamba wire) per second kitne bytes carry kar sakta hai.
$R$ = compute rate: processor per second kitne operations execute kar sakta hai.

PICTURE. Ek horizontal timeline do colored segments mein split: ek lamba red segment ( $N / B$ ) uske baad ek chhota green wala ( $C / R$ ). Har segment ki length literally uska formula hai.

Step 4 — Arithmetic intensity: kaunsa segment jeet ta hai?

Yeh ratio kyun invent karein? Kyunki $T_{v N}$ mein do terms hain aur hum jaanna chahte hain kaunsa bada hai machine speeds plug in kiye bina. Compute-work ko move-work se divide karna timeline ko ek comparable number tak strip kar deta hai. Yeh exactly Roofline Model ka axis hai.

Data-hungry job ke liye (streaming, low reuse), $I$ tiny hota hai, toh $C / R$ negligible hota hai $N / B$ ke next to:

$T_{v N} \approx \frac{N}{B} (bandwidth-bound)$

PICTURE. Do timelines stacked: upar wali $I$ small hai (mostly red), neeche wali $I$ large hai (mostly green). Dividing sawaal — "kaunsa color bar fill karta hai?" — woh PIM go/no-go test hai.

Step 5 — Jahan PIM attack karta hai: hidden internal bandwidth

Yeh kyun matter karta hai? Bottleneck $B$ external straw tha. Agar hum memory ke andar compute karte hain, hum kabhi us straw se nahi squeeze karte — hum wide internal read use karte hain. Toh hume internal bandwidth ko naam dena hoga.

$k$ = banks ki sankhya jo parallel mein kaam kar sakti hain.
$B_{bank}$ = ek bank ki internal read ki bandwidth.
$B_{in t}$ = combined internal bandwidth.

PICTURE. Ek wide reservoir (bahut saare parallel bank-pipes, har ek $B_{bank}$ ) sab ek narrow external straw $B$ mein feed ho rahe hain. PIM aapko seedha reservoir se peene deta hai.

Step 6 — PIM timeline aur speedup formula

Sirf $B \to B_{in t}$ swap kyun karein? Kyunki ek cheez jo PIM change karta hai woh hai woh rate jis par operands arithmetic unit tak pahunchte hain. Compute work $C$ aur compute rate $R$ unchanged hain. Movement work $N$ unchanged hai. Sirf road wider ho gayi.

PICTURE. Do timelines scale ke saath drawn, ek ke upar ek: von Neumann bar (lamba red) PIM bar (chhota red) ke upar. Red lengths ka ratio hi speedup hai.

Step 7 — Edge cases (jahan formula jhooth bolta hai)

Yeh failure modes exist hi kyun karte hain? Derivation ke woh do cheezein assume ki gayi thi uspe wapas jao. Step 6 mein $N$ ka cancellation sirf isliye kaam kiya kyunki humne compute term $C / R$ drop kiya tha (assumption 1: bandwidth-bound). Aur speedup ek poore program ko count karta hai jaise woh ek bandwidth-bound kernel ho (assumption 2: sab kuch benefit karta hai). Neeche har case exactly woh hai jo hota hai jab un hidden assumptions mein se ek toot ti hai — toh har ek derivation ki ek specific line modify karta hai, picture ko generally nahi.

Case A — Compute-bound ( $I$ bada). Kaunsa assumption toot ta hai: assumption 1. Agar $C / R$ negligible nahi hai, toh $N$ terms dominate nahi karte, toh Step 6 mein cancellation kabhi nahi hoti. Tab $T_{P I M} \to C / R$ , von Neumann jaisa hi, aur speedup $\to 1$ . Road ko widen karna help nahi karta agar aap kabhi road par stuck nahi the. Yeh roofline ceiling hai.

Case B — Amdahl's ceiling. Kaunsa assumption toot ta hai: assumption 2. Sirf program ka movement-bound fraction speed up hota hai; baaki apni purani speed par chalta hai. Agar kaam ka ek fraction $f$ bandwidth-bound hai aur $s = B_{in t} / B$ se speed up hota hai, toh Amdahl's Law raw speedup $s$ ki jagah leta hai $Speedup_{t o t a l} = \frac{1}{( 1 - f ) + f / s},$ jo $s = \infty$ par bhi $1/ (1 - f)$ se better nahi ho sakta. Un-accelerated $(1 - f)$ portion ceiling set karta hai.

Case C — Degenerate / analog input. Kaunsa assumption toot ta hai: "op exact hai" premise jo $C / R$ ke andar hidden hai. Crossbar-style compute ke liye jo memory ki apni physics use karta hai, "op" digital nahi hai — yeh ek analog current hai jo ek wire par sum hota hai. Fresh symbols define karo (yeh electrical hain, Step 4 ke intensity $I$ nahi):

$G$ = memory cell ki conductance (current kitni aasani se flow karta hai, siemens mein).

$V$ = ek column par apply kiya gaya input voltage (volts mein).

$I_{ce l l} = G V$ = us cell se current (Ohm's law); ek shared row wire par currents add ho jaate hain (Kirchhoff), dot product dete hain.

Zero input voltage ( $V = 0$ ) → zero current (ek harmless degenerate case), lekin conductance drift aur analog-to-digital noise summed result ko approximate banate hain. Speedup real hai; exactness nahi. Ise error-tolerant neural nets ke liye use karo, kabhi exact finance ke liye nahi.

PICTURE. Teen mini-panels: (A) ek timeline PIM ke baad bhi green-dominated (koi win nahi); (B) ek Amdahl curve $1/ (1 - f)$ tak flatten ho raha hai; (C) ek ideal straight line $I_{ce l l} = G V$ uske aas-paas ek fuzzy noisy band ke saath.

Ek-picture summary

Ek frame mein sab kuch: lamba von Neumann timeline shrink hota hai kyunki fat external-bus segment ek thin internal-bandwidth segment se replace ho jaata hai, jabki compute segment (unchanged) woh hai jo eventually aapko limit karta hai.

Recall Feynman retelling — plain words mein poora walkthrough

Ek program do kaam karta hai: numbers fetch karna aur unpar math karna. Unhe lambe wire ke paar fetch karna slow hai aur math ki 100–1000× energy burn karta hai (Steps 1–2, dono 32-bit word per measure kiye gaye). Toh humne total time likhi "move time + compute time" = $N / B + C / R$ (Step 3). Jaanne ke liye kaunsa part hurt karta hai, hum ops ko bytes se divide karte hain arithmetic intensity $I = C / N$ paane ke liye; agar yeh small hai, hum wire par wait karne mein stuck hain (Step 4). PIM ka trick: memory andar se already wide hai — kaafi banks, har ek ek saath thousands of bits read karta hai — toh iska internal bandwidth $B_{in t} = k B_{bank}$ skinny external bus $B$ ko dwarf karta hai (Step 5). Wahan compute karo aur move segment exactly $B_{in t} / B$ se shrink ho jaata hai; woh ratio speedup hai (Step 6). Lekin yeh sirf tab help karta hai jab aap bandwidth-bound the, yeh is baat se cap hai ki aapka kitna program bandwidth-bound hai (Amdahl), aur analog versions exactness ko speed ke liye trade karte hain (Step 7). Ek sentence: data move mat karo — math move karo.