The energy story is even worse. Moving one bit across the chip costs far more energy than doing the math on it:
WHAT is PIM?
HOW does it help? (Derivation from first principles)
Take a workload that touches N bytes and does C compute operations. Total time on a classic von Neumann machine:
TvN=move over bus, bandwidth BBN+compute at rate RRC
For data-intensive jobs C is small relative to N, so TvN≈N/B: bandwidth-bound.
PIM changes the effective bandwidth. If memory has k banks each computing locally in parallel, the internal bandwidth Bint can be much larger than the external bus B:
Bint=k⋅Bbank≫B
So:
TPIM≈BintN+RC,Speedup=TPIMTvN≈BBint=k⋅BBbank
Imagine you're a chef (CPU) and your ingredients are in a warehouse far away (memory). Cooking is fast, but running to the warehouse for every carrot is exhausting and slow. Processing-in-memory is like putting a tiny kitchen inside the warehouse, so you can chop the veggies right where they're stored and only carry the finished dish back. You save a ton of running around — and running around was the tiring part all along.
The memory wall — the bandwidth/energy cost of moving data between memory and the processor.
Why is data movement worse than computation, energetically?
Moving a bit across the chip costs ~100–1000× the energy of a single arithmetic op, so movement dominates total energy in data-intensive workloads.
Define arithmetic intensity and its role in PIM.
I = C/N (ops per byte moved); low I means bandwidth-bound and a good PIM candidate; high I means compute-bound and PIM helps little.
Difference between processing-near-memory and processing-using-memory?
PNM puts ordinary logic physically close to memory (e.g., HBM logic die); PUM computes using the analog physics of the memory array itself (e.g., crossbar MVM).
How does a memristor crossbar compute matrix–vector multiply?
Ohm's law gives I=GV per cell; Kirchhoff's current law sums currents per row into a dot product, so I=G·V for the whole matrix in one analog step.
What is the approximate speedup formula for near-memory PIM?
Speedup ≈ B_int / B = k·(B_bank / B), the ratio of internal parallel bandwidth to external bus bandwidth.
Why isn't PIM the same as adding cache?
Cache helps reused (temporal-locality) data on the CPU side; PIM eliminates the trip entirely, helping streaming low-reuse data.
Main downside of analog PUM crossbars?
Results are approximate due to device variation, drift, and ADC/DAC noise — suited to error-tolerant workloads like neural nets.
Dekho, aaj ke CPU bahut fast hain, lekin data DRAM me door pada hota hai. Har baar number laane ke liye CPU ko ek lambe, slow bus ke through jaana padta hai — aur yahi wait karna sab time kha jaata hai. Isko bolte hain memory wall. Aur mazedaar baat: ek bit ko chip par move karne ki energy, us par calculation karne se ~100 se 1000 guna zyada hoti hai. Matlab problem compute ki nahi, data hilaane ki hai.
Processing-in-Memory (PIM) ka simple idea: data ko CPU tak mat lao, thoda sa computation memory ke andar (ya bilkul paas) le aao. Do flavor hote hain — near-memory (jaise HBM stack ke logic die par chhote ALU) aur using-memory (memory array ki apni physics se compute, jaise memristor crossbar). Crossbar me Ohm's law (I=GV) aur Kirchhoff se ek hi analog step me poora matrix-vector multiply ho jaata hai — isliye AI accelerators ise pasand karte hain.
Kaunsa workload PIM ke liye achha hai? Bas arithmetic intensityI=C/N nikalo — per byte kitne ops. Vector add jaisa low-I kaam bandwidth-bound hota hai, PIM ka bada fayda. Matrix-multiply jaisa high-I kaam compute-bound hota hai, cache already achha karta hai, PIM ka fayda kam. Yaad rakho: PIM ALU ko fast nahi karta, wo data movement kam karta hai — yahi 80/20 wali asli baat hai.
