High Bandwidth Memory (HBM - HBM2 - HBM3): Visual walkthrough

This page slowly derives the parent result HBM uses: $BW = \frac{N \times f}{8}$ Let's build it one picture at a time.

Step 1 — What is a single wire actually doing?

WHAT we are doing: looking at the tiniest possible unit — one wire, one moment in time.

WHY: bandwidth is a big number built from many small events. If we don't understand the smallest event (one wire flipping high/low), the big number is just magic. We refuse magic.

PICTURE: look at the single wire below. Follow the red trace: it steps high, low, high, low as time moves right. Each coloured block is one bit. The wire is like a light switch being flicked.

Figure — High Bandwidth Memory (HBM - HBM2 - HBM3)

Step 2 — How fast can one wire flip? (this is $f$ )

WHAT: we count how many bit-blocks from Step 1 fit into one second.

WHY this symbol: we need a name for "wire speed" so we can compare memories. Calling it $f$ (for frequency of transfers) lets us write formulas instead of paragraphs.

PICTURE: the timeline below packs bits into exactly one second. Slow wire on top (few, wide bits), fast wire on bottom (many, narrow bits). Same second, more bits ⇒ bigger $f$ .

Step 3 — One wire is too slow. Add more wires (this is $N$ ).

WHAT: we lay several Step-1 wires next to each other and read them all in one instant.

WHY $N$ and not just a faster wire? Pushing $f$ higher costs a lot of power and makes the signal noisy (we'll see why in Step 6). Adding wires is a cheap, calm way to move more bits — as long as you have room for the wires. HBM's whole trick is: make $N$ enormous.

PICTURE: 8 wires stacked vertically. Read the highlighted column — in one instant you collect 8 bits at once. That's 8× the data of a single wire for the same $f$ .

Step 4 — Multiply: total bits per second

WHAT: we glue Step 2 ( $f$ ) and Step 3 ( $N$ ) together.

WHY multiply and not add $N + f$ ? Adding would be nonsense — the units don't even match ( $N$ is a plain count, $f$ is bits/s). " $N$ wires, each doing $f$ bits/s" is $N$ groups of $f$ , and "groups of" is multiplication. Always let the units guide you: (wires) × (bits/s per wire) = bits/s. ✓

PICTURE: a grid — $N$ rows (wires) tall, and $f$ bits wide (one second). The area of the grid = total bits in that second. Area = height × width = $N \times f$ .

Step 5 — Why divide by 8? (bits → bytes)

WHAT: we convert our bits per second into bytes per second.

WHY: nobody sells "gigabits per second" of memory bandwidth — the spec sheets, the GPU boxes, the benchmarks all say GB/s (gigabytes). To speak the same language we regroup 8 bits into 1 byte, which means dividing the bit-count by 8.

PICTURE: take the bit-stream and draw brackets around every group of 8. Count the brackets — that's your byte count. Fewer brackets than bits, exactly 8× fewer.

Step 6 — Plug in HBM2 and read the headline number

WHAT: use the real HBM2 numbers — $N = 1024$ wires per stack, $f = 2.0 Gbps$ per wire.

WHY these values? A stack is split into 8 channels of 128 bits each; $8 \times 128 = 1024$ wires. And each wire runs at the modest 2 Gbps we met in Step 2. We're not clocking fast — we're going wide.

$BW_{stack} = \frac{1024 N wires \times 2.0 \times 1 0 ^{9} f bits/s}{bits per byte 8} = 2.56 \times 1 0^{11} B/s = 256 GB/s$

PICTURE: a bar chart contrasting the two philosophies. GDDR = tall-and-thin (huge $f$ , few wires). HBM = short-and-fat (small $f$ , huge $N$ ). Their areas (= bandwidth) are comparable, but HBM's short bars mean far less heat and lower energy per bit.

Step 7 — Scale up and the edge cases

WHAT: put several stacks on one processor, and check the weird inputs so nothing surprises you.

WHY check edge cases? A formula you only trust for "nice" numbers is a trap. Let's push it.

Several stacks (the real GPU case). With $S$ stacks working in parallel on their own channels, total $BW = S \times \frac{N f}{8}$ . HBM2 4-stack: $4 \times 256 = 1024 GB/s \approx 1 TB/s$ . Why multiply by $S$ ? Independent channels don't share wires — they're 4 separate grids from Step 4, added.
$N = 1$ (a single wire). BW $= f /8$ . At $f = 2$ Gbps that's 250 MB/s — you're back to one garden tap. The formula degrades gracefully.
$f = 0$ (idle bus). BW $= 0$ . No flips ⇒ no bits ⇒ no bytes. Sensible: a still wire moves nothing.
Same BW target, two designs. To hit 256 GB/s you could use $N = 1024, f = 2$ (HBM) or $N = 128, f = 16$ (GDDR-ish). Same product $N \times f$ , wildly different power because only $f$ (and $V$ ) blows up the $C V^{2} f$ term. This is the HBM insight in one line.

PICTURE: four side-by-side grids (four stacks), each one the Step-4 area, summing to the total-BW bar on the right.

The one-picture summary

The whole journey on one canvas: one wire → speed $f$ → many wires $N$ → multiply → regroup by 8 → plug in HBM2 → 256 GB/s.

Recall Feynman: tell it back in plain words

Picture one wire between the memory and the chip. It's a light switch that flicks on/off, and each flick is one bit. Count the flicks in a second — that's the wire's speed, $f$ . One wire is too slow, so we run 1024 wires side by side; in one instant we grab 1024 bits at once. Total bits per second = wires × speed = $N \times f$ . But the world counts in bytes (bundles of 8 bits), so we divide by 8. For HBM2 that's $1024 \times 2 billion \div 8 = 256$ billion bytes per second — 256 GB/s per stack. Stack four of them next to the chip on a little glass tray and you're near a terabyte a second. And because every wire is short and slow, the whole thing barely gets warm. Wide, short, and stacked.

Connections

High Bandwidth Memory (HBM - HBM2 - HBM3) — the parent this walkthrough derives.
GDDR Memory and DDR vs GDDR vs HBM — the tall-and-thin rival design.
Memory Wall — why any of this bandwidth chasing matters.
Through-Silicon Vias (TSV), 2.5D and 3D Integration, Silicon Interposer — how the 1024 wires physically exist.
Energy per Bit / pJ per bit — the $C V^{2} f$ payoff of "wide + slow".
GPU Architecture — the customer starved for these bytes.