Processing-in-memory (PIM): Question bank — Hardware deep dive

Symbols and terms used on this page (build them before you quiz them)

Before any trap, here is every letter and abbreviation this page uses, in plain words. Do not skip — the traps only make sense once these are earned.

Formula The one equation behind the crossbar:

$I = G V$ Picture a grid of wires: vertical columns carry input voltages, horizontal rows collect currents. At every crossing sits one memristor cell (see Memristor / ReRAM Crossbars).

$V_{j}$ = the voltage applied to column $j$ (this is one input number of the vector $V$ ).
$G_{ij}$ = the conductance (1/resistance) of the cell where row $i$ meets column $j$ — this stores one weight of the matrix $G$ .
$I_{ij}$ = the current that cell pushes onto row $i$ .

Ohm's law (why: a memristor cell is just a programmable resistor, and $I = G V$ is the resistor law): $I_{ij} = G_{ij} V_{j}$ . So each cell multiplies a weight by an input. Kirchhoff's current law (why: currents entering a shared wire must add up): the total current on row $i$ is $I_{i} = \sum_{j} G_{ij} V_{j}$ . That sum is a dot product — one row of a matrix–vector product. Across all rows the whole grid computes $I = G V$ in a single analog settling step. That is why physics does the multiply-accumulate "for free."

Look at the roofline sketch above: the sloped teal line is the bandwidth roof (performance limited by $B$ ); the flat orange line is the compute roof (limited by $R$ ). The bend where they meet is the ridge point. A workload's arithmetic intensity $I = C / N$ places it left (bandwidth-bound → good PIM target) or right (compute-bound → PIM helps little). Keep this picture in mind for every trap below.

True or false — justify

PIM's main benefit is that it makes the arithmetic units (ALUs) run faster.

False. PIM's win is eliminating data movement, not faster arithmetic; for compute-bound work the same slow-or-fast ALU is doing the math either way. See Memory Wall.

If a workload is compute-bound (high arithmetic intensity), PIM will still give a large speedup.

False. When

I = C / N

is high, data movement is already a small fraction of the time, so removing it barely helps — on the roofline you're right of the ridge, under the flat compute ceiling, not the bandwidth one. See Roofline Model.

Processing-using-memory (PUM) crossbars produce results as bit-exact as a digital multiplier.

False. The equation

I = G V

is exact on paper, but conductances

G_{ij}

drift, and ADCs/DACs add quantization and noise, so PUM results are approximate — fine for neural nets, wrong for IEEE-exact finance.

Adding a bigger cache achieves the same thing as PIM.

False. Cache lives on the CPU side and only helps reused (temporal-locality) data; PIM eliminates the trip entirely and so helps streaming, low-reuse data that a cache cannot capture.

The internal bandwidth

B_{in t}

of a PIM memory can exceed the external bus bandwidth

B

True. Inside DRAM,

k

banks each stream at

B_{bank}

in parallel, so

B_{in t} = k \cdot B_{bank}

can dwarf the single external bus

B

— that untapped parallelism is exactly what PIM harvests. See DRAM Organization (banks, rows).

Because

E_{m o v e} \approx 100 - 1000 \times E_{o p}

, energy savings from PIM are usually larger than the raw speedup.

True (often). Long high-capacitance wires make movement dominate energy even more than time, so cutting movement can slash total energy dramatically even when the timing speedup is modest.

PIM removes the Von Neumann Bottleneck entirely.

False in general. PIM reduces the trips across the bus but rarely eliminates them — results, control, and non-local data still travel; it narrows the bottleneck rather than deleting it.

A memristor crossbar computes a matrix–vector product in

O (1)

analog steps regardless of matrix size.

True for the multiply-accumulate itself. Ohm's law + Kirchhoff sum all

G_{ij} V_{j}

products in one settling step, but you still pay

O (n)

to load inputs and read outputs through DACs/ADCs, so it isn't free end-to-end. See Memristor / ReRAM Crossbars.

Near-memory PIM (PNM) requires a fundamentally new memory technology like ReRAM.

False. PNM uses ordinary logic (ALUs, small cores) placed physically close to conventional memory — e.g., on the logic die of an HBM 3D-Stacked Memory stack. No exotic device needed.

Spot the error

"A workload does

1 0^{6}

ops and moves

1 0^{6}

bytes, so its arithmetic intensity is 1000 ops/byte."

Error: intensity is

I = C / N = 1 0^{6} /1 0^{6} = 1

, not 1000. The claimant likely divided by a mis-scaled byte count; always keep

C

and

N

in the same units and orders of magnitude.

"HBM is 3D-stacked, and 3D-stacking is what makes it a PUM (using-memory) design."

Error: 3D-stacking enables PNM (near-memory) — ordinary logic sits on a nearby die. PUM means computing with the memory array's own analog physics; stacking alone is not PUM. See HBM 3D-Stacked Memory.

"Speedup for near-memory PIM is

B / B_{in t}

, the external over internal bandwidth."

Error: the ratio is inverted. Speedup

\approx B_{in t} / B = k \cdot (B_{bank} / B)

— you want the larger internal bandwidth on top so the speedup exceeds 1.

"Since PIM parallelizes across banks, its speedup can grow without bound as you add banks."

Error: Amdahl's Law bites — the residual compute term

C / R

(time spent by the ALU at rate

R

) and un-parallelizable serial fractions cap the speedup; adding banks past the bottleneck gives diminishing returns.

"Analog crossbars are noisy, so they're useless for real computing."

Error: overgeneralization. Approximate results are perfectly acceptable for error-tolerant workloads like Neural Network Accelerators; "noisy" ≠ "useless," it just narrows the application domain.

"PIM is free — we just bolt some logic onto the DRAM chip."

Error: DRAM uses a process optimized for cheap dense storage and bad for logic; adding compute costs area, heat, and memory density, plus hard programming-model and coherence problems. It's a trade-off, not a free lunch.

Why questions

Why does making the ALU faster do "nothing" for a bandwidth-bound job?

Because total time is dominated by the

N / B

movement term; speeding the tiny

C / R

compute term (where

R

is ALU throughput) leaves the bottleneck untouched — you must attack the movement itself.

Why does PIM specifically favor neural networks and graph analytics?

Both are dominated by data movement over large low-reuse data (matrix–vector multiplies, edge traversals) with modest per-byte compute, i.e. low arithmetic intensity

I = C / N

— exactly where cutting movement pays off. See Neural Network Accelerators.

Why is the internal DRAM bandwidth so much larger than the external bus?

A DRAM row activation reads thousands of bits at once, and

k

banks stream in parallel, but the external bus

B

is a narrow shared pin-limited channel — PIM computes on the wide internal data before it's funneled through the narrow bus. See DRAM Organization (banks, rows).

Why does a shared row wire in a crossbar naturally perform a summation?

Kirchhoff's current law: currents entering a common node add. Each cell injects

G_{ij} V_{j}

onto row

i

, so the wire physically sums them into

\sum_{j} G_{ij} V_{j}

— a dot product for free.

Why does the Roofline Model help decide whether to use PIM?

It plots performance against arithmetic intensity

I = C / N

, showing the bandwidth ceiling (sloped, set by

B

) vs compute ceiling (flat, set by

R

); low-intensity workloads sit left of the ridge under the bandwidth roof, which is precisely the roof PIM raises.

Why isn't temporal locality enough to fix the memory wall for streaming data?

Caches only help when data is reused before eviction; streamed data is touched once, so it never benefits — PIM helps by computing at the source instead of relying on reuse.

Edge cases

What happens to PIM's benefit when arithmetic intensity is extremely high (e.g. dense matmul,

I \approx 167

The workload is compute-bound (right of the ridge); almost no time is spent on movement, so PIM's speedup approaches 1× — CPU/GPU caches already win here.

What happens when

k = 1

(a single bank, no internal parallelism)?

Then

B_{in t} = k \cdot B_{bank} = B_{bank}

and the speedup

B_{in t} / B

has no parallelism to exploit; PIM degenerates to roughly the baseline — the win comes from many banks working at once.

What if a workload is purely serial control logic with almost no data (tiny

N

N / B

is negligible, so there's nothing for PIM to save; it's neither bandwidth-bound nor a data problem — a fast general CPU is the right tool.

What happens to a PUM crossbar result when the required numerical precision is very high?

Analog drift in

G_{ij}

, device variation, and ADC quantization set an accuracy floor; beyond a few bits of effective precision the crossbar can't meet the requirement, so digital compute is needed instead.

What is the limiting speedup as you add infinitely many banks (

k \to \infty

) but keep a fixed serial compute fraction?

By Amdahl's Law it converges to

1/ (serial fraction)

, not infinity — the un-parallelizable

C / R

term and coordination overhead form a hard ceiling.

What happens at the boundary intensity where the workload is neither clearly bandwidth- nor compute-bound?

You sit near the ridge point of the Roofline Model; PIM gives a partial win, and the decision hinges on secondary costs (energy, area, programmability) rather than raw speedup alone.