High Bandwidth Memory (HBM - HBM2 - HBM3): Exercises

The single tool we use everywhere is the bandwidth equation. Let us restate it so nothing is assumed.

Unit reminder so no magic happens later: $1 Gbps = 1 0^{9}$ bits/s, $1 GB/s = 1 0^{9}$ bytes/s, $1 TB/s = 1 0^{12}$ bytes/s.

Figure — High Bandwidth Memory (HBM - HBM2 - HBM3)

Level 1 — Recognition

Recall Solution L1.1

The two factors are $N$ (number of data wires / bus width) and $f$ (per-pin data rate).

HBM makes $N$ large ( $\approx 1024$ bits per stack).
HBM keeps $f$ small (a few Gbps per pin), so each wire runs cool and cheap. This is the "wide and slow" strategy from the parent note. See Memory Wall for why we chase bandwidth at all.

Recall Solution L1.2

(a) TSV — a vertical copper via through a die, connecting one DRAM die to the die above/below inside the stack.
(b) Silicon Interposer — a passive silicon carrier holding thousands of fine wires between the stack(s) and the processor (this is 2.5D integration).
(c) base logic die — the bottom die of the stack that runs the interface/PHY.
(d) microbump — solder connection joining the stack to the interposer (stack-to-interposer, not die-to-die).

WHAT: plug into $BW = N f /8$ . WHY: it is the definition of bandwidth. $BW = \frac{1024 \times 2.0 \times 1 0 ^{9}}{8} = \frac{2.048 \times 1 0 ^{12}}{8} = 2.56 \times 1 0^{11} B/s = 256 GB/s$ Sanity check: $1024 \times 2 G = 2.048$ Tbit/s; a byte is 8 bits so divide by 8 → 256 GB/s. ✅

Recall Solution L2.2

Per stack: $\frac{1024 \times 6.4 \times 1 0 ^{9}}{8} = 819.2$ GB/s. Total: WHY multiply by 4? The stacks sit on independent channels and move data simultaneously, so their bandwidths add. $4 \times 819.2 = 3276.8 GB/s = 3.2768 TB/s$

Recall Solution L2.3

Each stack exposes a 1024-bit data bus, and there are 4 stacks: $4 \times 1024 = 4096 data wires$ Why this matters: 4096 fine-pitch traces cannot be routed on ordinary PCB copper — that is the reason a silicon interposer exists. Compare with GDDR Memory, which uses far fewer wires on the board.

Level 3 — Analysis

Recall Solution L3.1

WHAT: invert the formula to solve for $f$ . WHY: we know BW and $N$ , we want $f$ . $f = \frac{8 \times BW}{N} = \frac{8 \times 1.0 \times 1 0 ^{12}}{1024} = 7.8125 \times 1 0^{9} bits/s \approx 7.81 Gbps$ From the cheat-sheet: HBM3 is 6.4 Gbps (too low for one stack → gives 819 GB/s), HBM3E is 9.6 Gbps (enough). So HBM3E is the first single-stack generation to clear 1 TB/s.

Recall Solution L3.2

With $N$ fixed, bandwidth is proportional to $f$ : $BW \propto f$ . $\frac{BW _{HBM3}}{BW _{HBM2}} = \frac{f _{HBM3}}{f _{HBM2}} = \frac{6.4}{2.0} = 3.2$ So the entire gain is from per-pin rate, not width. (HBM3's channel count doubled to 16×64-bit, but that is 1024 bits still — see the L3 trap below.)

Recall Solution L3.3

WHY does $f$ cancel? Energy per bit = power ÷ bits-per-second = $(C V^{2} f) / f = C V^{2}$ . Frequency drops out — energy per bit depends only on how much charge you shove ( $C$ ) and how far ( $V^{2}$ ). $\frac{E _{HBM}}{E _{GDDR}} = \frac{C _{H} V _{H}^{2}}{C _{G} V _{G}^{2}} = (\frac{1}{4}) (\frac{1}{2})^{2} = \frac{1}{4} \cdot \frac{1}{4} = \frac{1}{16}$ Each HBM wire spends ~1/16 the energy per bit. See Energy per Bit / pJ per bit and DDR vs GDDR vs HBM.

Level 4 — Synthesis

Recall Solution L4.1

Stacks needed: $⌈ 3.0/1.2 ⌉ = ⌈ 2.5 ⌉ = 3$ stacks → $3 \times 1.2 = 3.6$ TB/s ✅ (2 stacks would give only 2.4 TB/s, short). Data wires: $3 \times 1024 = 3072$ wires. Impossible on PCB, routed on the interposer — see GPU Architecture for where these stacks physically sit.

Recall Solution L4.2

Bandwidth constraint: $⌈ 2.4/1.2 ⌉ = 2$ stacks.
Capacity constraint: $⌈ 48/36 ⌉ = 2$ stacks (2×36 = 72 GB ≥ 48). Both give 2, so 2 stacks suffice: $2 \times 1.2 = 2.4$ TB/s ✅ and $72$ GB ≥ 48 ✅. Which binds? Both hit the limit at 2 simultaneously here, but note bandwidth only just meets 2.4 while capacity has 24 GB of slack — so bandwidth is the tight (binding) constraint.

Level 5 — Mastery

Recall Solution L5.1

(a) GDDR pins: each pin = 16 Gbps = $2 GB/s$ . Need $\frac{1.0 \times 1 0 ^{12}}{2 \times 1 0 ^{9}} = 500$ data pins. HBM2E stacks: per stack $= \frac{1024 \times 3.6 \times 1 0 ^{9}}{8} = 460.8$ GB/s. Need $⌈ 1000/460.8 ⌉ = 3$ stacks (3×460.8 = 1382 GB/s ≥ 1 TB/s ✅; 2 stacks = 921 GB/s falls short). (b) Wires: HBM2E $= 3 \times 1024 = 3072$ wires vs GDDR6 $= 500$ pins. HBM uses ~6× more wires. (c) Total energy per bit ratio. Both move the same total bits/s (1 TB/s). Energy per bit is a per-bit quantity — it does not care how many wires split the work. Total energy/bit $\approx$ (energy per wire-bit), and each HBM wire-bit costs 1/16 of a GDDR wire-bit. So HBM moves the same 1 TB/s at about 1/16 the energy per bit. Why "more wires" ≠ "more power": with 6× the wires, each wire only carries $1/6$ the traffic and costs $1/16$ per bit. Power = (total bits/s) × (energy per bit); the bits/s is fixed at 1 TB/s, so the winner is purely whoever has lower energy per bit — HBM, decisively. This is the core lesson of Energy per Bit / pJ per bit.

Recall One-screen summary

Every problem = $BW = N f /8$ , inverted as needed.
HBM pushes $N$ (width); generations mostly raise $f$ ; total $N$ stays 1024/stack even in HBM3.
Requirements → round stacks up (ceiling).
Energy per bit $\propto C V^{2}$ ( $f$ cancels) → HBM's short, low-swing wires win regardless of wire count.
Back to the parent note · Hinglish version: 6.5.04 High Bandwidth Memory (HBM - HBM2 - HBM3) (Hinglish).

Exercises — High Bandwidth Memory (HBM - HBM2 - HBM3)