Worked examples — Processing-in-memory (PIM)

6.5.5 · D3 · Hardware › Advanced & Emerging Architectures › Processing-in-memory (PIM)

Scenario matrix

Har PIM sizing question ultimately do quantities tak reduce hoti hai jo tumhe workload se compute karni hoti hain:

• Arithmetic intensity $I=C/N$ — compute ops per byte moved. Yahan $C$ arithmetic operations ki count hai aur $N$ bus ke paar move hone wale bytes ki count hai. Low $I$ = bandwidth-bound = PIM iske liye perfect hai. (Dekho Roofline Model.)
• Bandwidth speedup ceiling $S_{\infty }=B_{int}/B$ — PIM kitna zyada internal bandwidth unlock karta hai vs external bus. Yahan $B$ external bus bandwidth hai (bytes/second, memory aur CPU ke beech wire par), $k$ wo memory banks hain jo parallel mein compute kar sakte hain, aur $B_{bank}$ ek bank ke andar ki bandwidth hai. Unka product $B_{int}=k{B}_{bank}$ woh total internal bandwidth hai jo PIM harvest karta hai.

Neeche ki matrix har possible case class enumerate karti hai jo yeh topic tumhare saamne rakh sakta hai. Baad ke saat examples mein se har ek ek row cover karta hai (ya ek row ki extreme), aur cell ke hisaab se label kiya gaya hai.

Example 1 — Case A: streaming, bahut low intensity

1. Compute ops count karo. Har element par ek add → $C=10^6$. Yeh step kyun? Intensity ops-per-byte hai; pehle op count chahiye.
2. Bytes moved count karo. b read, c read, a write = 3 accesses × 4 bytes × $10^6$ = $12\times 10^6$ bytes. Kyun? Bus cost har byte se set hoti hai jo cross karti hai, reads aur writes dono.
3. Intensity compute karo. $I=10^6/(12\times 10^6)=0.0833$ ops/byte. Kyun? Yeh chota number humein red ridge ke kaafi left mein dalta hai → bandwidth-bound.
4. Internal bandwidth. $B_{int}=16\times 8=128$ GB/s. Kyun? Har bank parallel mein compute karta hai; unki internal bandwidths add hoti hain (DRAM Organization (banks, rows)).
5. Speedup ceiling. $S_{\infty }=128/50=2.56\times$. Kyun $16\times$ nahi? Kyunki per-bank bandwidth ($8$) bus ($50$) se neeche hai; jeet aggregate internal aur external ka ratio hai, bank count nahi.

Verify: Units: $\frac{\text{ops}}{\text{byte}}$ ✓ dimensionless speedup ✓. Sanity: $I\approx 0.08\ll 1$ correctly ek bandwidth-bound streaming kernel flag karta hai — exactly jo PIM target karta hai. $2.56\times$ modest hai par real hai. Parent ke Worked Example 2 se match karta hai.

Example 2 — Case B: dense matmul, high intensity (PIM barely helps)

1. Ops. Ek dense matmul $2n^3$ FLOPs karta hai (har inner term par ek multiply + ek add): $C=2\times (1000{)}^3=2\times 10^9$. Kyun? $n^2$ output cells, har ek ek $n$-length dot product = $n$ MACs = $2n$ flops.
2. Bytes (naïve, no reuse counted). Teen $n\times n$ matrices: $N=3n^2\times 4=3\times 10^6\times 4=1.2\times 10^7$ bytes. Kyun? Hum A, B, aur C ek ek baar move karte hain — minimum traffic yahi hai.
3. Intensity. $I=2\times 10^9/1.2\times 10^7\approx 166.7$ ops/byte. Kyun? Yeh red ridge ke kaafi right mein hai → compute-bound.
4. Verdict. Har byte ~167 ops feed karta hai → CPU/GPU apne ALUs busy rakhta hai aur caches memory ko hide kar dete hain. PIM ka data-movement win yahan almost invisible hai.

Verify: $I\approx 167\gg 1$ ✓ Ex 1 ($0.083$) ki opposite extreme. Ratio $167/0.083\approx 2000$ — do workloads roofline par ~3.3 decades apart hain, correctly ridge ke opposite sides par. Dekho Neural Network Accelerators: woh matmul ko precisely tile karte hain reuse badhane ke liye aur compute-bound rehne ke liye.

Example 3 — Case C: break-even ridge

1. Dono time terms likho. Move time $=N/B$, compute time $=C/R$. Balance matlab dono equal hain. Kyun? Roofline Model ridge exactly woh crossover hai jahan do costs match karti hain.
2. Equal set karo aur solve karo. $N/B=C/R\Rightarrow C/N=R/B\Rightarrow I_{ridge}=R/B$. Kyun? $N$ se divide karne par raw counts intensity mein convert ho jaati hain jise hum lookup kar sakte hain. Yaad karo $R$ = peak compute rate (GFLOP/s), $B$ = bus bandwidth (GB/s).
3. Numbers plug karo. $I_{ridge}=100[\text{GFLOP/s}]/50[\text{GB/s}]=2$ ops/byte. Kyun? $2$ se neeche → bus-bound (PIM help karta hai). $2$ se upar → compute-bound (PIM meh).
4. PIM ka effect. PIM effective bandwidth ko $B$ se $B_{int}$ tak raise karta hai, jo ridge ko right shift karta hai $I_{ridge}=R/B_{int}$ tak. Jo workloads barely compute-bound the woh bandwidth-bound zone mein wapas aa sakti hain — lekin sirf woh jo ridge ke kareeb hain.

Verify: Units: $\frac{\text{GFLOP/s}}{\text{GB/s}}=\frac{\text{ops}}{\text{byte}}$ ✓. $I_{ridge}=2$. Ex 1 ($0.083<2$) bandwidth-bound hai ✓; Ex 2 ($167>2$) compute-bound hai ✓ — dono classifications is ek ridge value ke saath consistent hain.

Example 4 — Case D: degenerate, zero compute (pure copy)

1. Compute ops. Ek copy karta hai koi arithmetic nahi: $C=0$. Kyun? Yeh intensity axis ka degenerate corner hai.
2. Bytes. b read, a write = 2 accesses × 4 bytes × $10^6$ = $8\times 10^6$ bytes. Kyun? Movement phir bhi hoti hai — yahi is case ka poora point hai.
3. Intensity. $I=0/(8\times 10^6)=0$ ops/byte — roofline ka extreme left edge, Ex 1 se bhi neeche. Kyun? Nonzero number par zero ek clean $0$ deta hai; ratio well-defined hai (contrast: agar $N$ bhi $0$ hota, toh $0/0$ hota, undefined — woh corner Ex 8 mein drill hota hai).
4. PIM verdict. Maximally bandwidth-bound. Lekin note karo: PIM compute karta hai, aur yahan kuch bhi compute nahi karna. Near-memory copy (in-DRAM row-clone / RowClone) phir bhi help karta hai kyunki bus kabhi cross nahi hoti; compute-style crossbar nahi karta.

Verify: $I=0$ ✓, intensity axis ka true floor ($0<0.083<2<167$, ordering intact). Degenerate-input handle hua: $C=0$ ek defined $I$ deta hai; $N=0$ corner Ex 8 hai.

Example 5 — Case E: limiting behaviour, infinite banks meets Amdahl

1. Bandwidth ceiling bina bound ke badhta hai. $S_{\infty }=k{B}_{bank}/B\to \infty$ as $k\to \infty$. Kyun? Pure-bandwidth model mein kuch bhi isko cap nahi karta — yahi trap hai. Note karo $S_{\infty }$ sirf parallel (bandwidth-bound) part ka ceiling hai; yeh exactly woh "$S_{\text{parallel}}$" hai jo hum aage Amdahl mein feed karte hain.
2. Amdahl's Law laao. Overall speedup $=1/(s+(1-s)/S_{\infty })$, jahan $s=0.05$ serial fraction hai aur $S_{\infty }$ wahi bandwidth ceiling hai step 1 se — woh ek hi part hai jise Amdahl accelerate karne deta hai. Yeh tool kyun? Amdahl exactly yeh sawaal answer karta hai "jab ek part infinitely fast ho jaaye toh program speedup kya approach karti hai?" — yahan sahi sawaal, raw bandwidth nahi.
3. Limit lo. Jaise jaise $S_{\infty }\to \infty$, $(1-s)/S_{\infty }\to 0$, toh speedup $\to 1/s=1/0.05=20\times$. Kyun? Serial $5\%$ speed up nahi ho sakta; baaki sab free ho jaane ke baad woh dominate karta hai.
4. Interpretation. Perfect infinite-bank PIM ke saath bhi, yeh workload $20\times$ par cap ho jaata hai. Bus/serial residue woh wall hai.

Verify: $1/0.05=20$ ✓. Sanity: smaller serial fraction $s=0.01$ ke saath cap $100\times$ tak rise karti — monotonic aur correct. Limiting case handle hua: $k\to \infty$ ek finite $20\times$ deta hai, infinity nahi. Parent ki warning "always Amdahl se sanity-check karo" ✓.

Example 6 — Case F: analog PUM crossbar accuracy

1. Ideal current per cell (Ohm's law). $I_{ij}=G_{ij}{V}_j$: $2\cdot 1=2$, $4\cdot 0.5=2$, $6\cdot 0=0$ (mA). Ohm's law kyun? Har crosspoint literally ek resistor hai; $I=GV$ exact device physics hai.
2. Row sum karo (Kirchhoff's current law). $I_i=2+2+0=4$ mA. Kyun? Ek wire par milne wale currents add hote hain — woh summation hi dot product ${\sum }_j{G}_{ij}{V}_j$ hai.
3. Drift apply karo. Pehla cell $+3\%$ drift karta hai: $G_1=2\cdot 1.03=2.06$ mS → uska current $=2.06\cdot 1=2.06$ mA. Naya sum $=2.06+2+0=4.06$ mA. Kyun? Real conductances wander karte hain; isliye PIM approximate hai, exact nahi.
4. Relative error. $(4.06-4)/4=0.015=1.5\%$. $3\%$ se kam kyun? Drift us ek term ko hit karta hai jo total ka half contribute karta hai, aur baaki terms unaffected hain — errors sum ke across dilute ho jaate hain.

Verify: Ideal $=4$ mA, drifted $=4.06$ mA, error $=1.5\%$ ✓. Parent ke mistake-callout ko confirm karta hai: crossbars approximate hain, error-tolerant nets ke liye suited hain, exact finance ke liye nahi.

Example 7 — Case G: word problem / exam twist (energy, time nahi)

1. Baseline energy. Move: $12\times 10^6\times 200\text{pJ}=2.4\times 10^9$ pJ. Compute: $10^6\times 1=10^6$ pJ. Total $E_{vN}=2.401\times 10^9$ pJ. Dono alag kyun karo? PIM sirf move term ko attack karta hai; alag karne se pata chalta hai yeh kitna dominant hai.
2. Confirm karo ki movement dominate karta hai. $2.4\times 10^9$ vs $10^6$ → movement yahan compute se $2400\times$ zyada hai. Check kyun? Agar compute dominate karta, toh movement save karna pointless hota — yeh intensity test ka energy version hai.
3. PIM energy. Sirf $10\%$ bytes move karte hain: $0.1\times 2.4\times 10^9=2.4\times 10^8$ pJ. Compute unchanged: $10^6$ pJ. Total $E_{PIM}=2.41\times 10^8$ pJ. Compute unchanged kyun? PIM arithmetic ko sasta nahi banata — woh trips remove karta hai.
4. Saving. $1-E_{PIM}/E_{vN}=1-2.41\times 10^8/2.401\times 10^9=0.8996\approx 90.0\%$. Itna bada kyun? Kyunki movement compute se $2400\times$ zyada tha, $90\%$ movement cut karna ~$90\%$ total energy cut karta hai.

Verify: $E_{vN}=2.401\times 10^9$ pJ, $E_{PIM}=2.41\times 10^8$ pJ, saving $\approx 90.0\%$ ✓. Units: pJ throughout ✓. Parent ke core energy thesis ko confirm karta hai: movement ko attack karo, ALU ko nahi.

Example 8 — Case H: fully degenerate corner, bilkul data nahi

1. Ratio likho. $I=C/N=0/0$. Kyun? Yeh intensity axis ka last unvisited corner hai — numerator aur denominator dono vanish ho jaate hain.
2. Ratio classify karo. $0/0$ indeterminate hai — mathematically undefined, na $0$ na $\infty$. Yeh kyun matter karta hai? Ex 4 (clean $0$ kyunki sirf $C=0$ tha) ke unlike, yahan workload ko roofline par kahi bhi place nahi kar sakte — koi workload hi nahi hai.
3. Physical reading. Koi bytes move nahi karte → bus kabhi bottleneck nahi thi; koi ops nahi chalte → accelerate karne ke liye kuch nahi. Dono time terms $N/B$ aur $C/R$ $0$ hain, toh total time $\to 0$ regardless of architecture.
4. PIM verdict. PIM na help karta hai na hurt: speedup bhi $0/0$ hai. Practice mein ek runtime guard karta hai empty inputs ke against kabhi bhi yeh poochne se pehle "kya main ise PIM karoon?" — intensity test undefined hai aur skip karna chahiye.

Verify: $N=0\Rightarrow I=0/0$ undefined ✓ (contrast Ex 4 ka defined $I=0$). Dono cost terms $N/B=0$ aur $C/R=0$ ✓, toh runtime kisi bhi machine par $0$ hai — empty kernel ki correct limiting reading.

Active Recall