High Bandwidth Memory (HBM - HBM2 - HBM3): Worked examples

Everything here rests only on three ideas the parent already built:

$N$ = number of data wires (the "width" of the bus).
$f$ = the effective per-pin rate in bits/second per wire (what the wire actually delivers).
Divide the bit total by 8 because there are 8 bits in a byte. See the parent topic and Memory Wall for why we chase bandwidth at all.

The scenario matrix

Every HBM numeric question falls into one of these cells. The example number that covers each cell is in the last column.

Cell	What varies / the twist	Covered by
A. Forward, single stack	Given $N$ , $f$ → find BW	Ex 1
B. Scale to many stacks	Multiply by stack count	Ex 2
C. DDR clock trap	Given a clock, not the effective rate	Ex 3
D. Inverse solve	Given BW → find $f$ or $N$	Ex 4
E. Degenerate — a die dies	Fewer working channels ( $N$ shrinks)	Ex 5
F. Zero / limiting case	What happens as $f \to 0$ or one wire only	Ex 6
G. Comparison (HBM vs GDDR)	Same BW, different $N, f$ → energy angle	Ex 7
H. Real-world word problem	"Can memory feed the compute?"	Ex 8
I. Exam twist	Channels doubled but bus unchanged (HBM3)	Ex 9

The figures below anchor the two mental pictures the whole page needs.

Figure — High Bandwidth Memory (HBM - HBM2 - HBM3)

Example 1 — Cell A: forward, single stack

Worked example The base case (HBM2)

Statement: One HBM2 stack has a $1024$ -bit data bus and each pin runs at an effective $2.0$ Gbps. Find the bandwidth of this one stack.

Forecast: Guess the order of magnitude first. A thousand wires, each ~2 billion bits/s → about $2$ trillion bits/s. Bytes are 8× smaller in count, so a few hundred GB/s. Hold that guess.

Step 1 — Multiply width by rate. Total bits/second $= N \times f = 1024 \times 2.0 \times 1 0^{9} = 2.048 \times 1 0^{12}$ bits/s. Why this step? Each of the $N$ wires independently delivers $f$ bits/s; they run in parallel, so the deliveries simply add up — that is what multiplication is here.

Step 2 — Convert bits to bytes. $\frac{2.048 \times 1 0 ^{12}}{8} = 2.56 \times 1 0^{11}$ B/s $= 256$ GB/s. Why this step? Bandwidth is quoted in bytes/s, and $8$ bits $= 1$ byte, so we divide the bit-count by 8.

Verify: Units check — $wires \times \frac{bits}{wire \cdot s} \div \frac{bits}{byte} = \frac{bytes}{s}$ . ✓ And $256$ GB/s matches our "few hundred GB/s" forecast. ✓ Matches the parent's headline number.

Example 2 — Cell B: scale to many stacks

Worked example Four HBM3 stacks in parallel

Statement: An accelerator uses $4$ HBM3 stacks. HBM3 is $1024$ -bit, $6.4$ Gbps per pin. Total bandwidth?

Forecast: One HBM2 stack was 256 GB/s at 2 Gbps. HBM3 is ~3.2× faster per pin and there are 4 stacks, so roughly $256 \times 3.2 \times 4 \approx 3300$ GB/s. Guess ~3 TB/s.

Step 1 — Per-stack BW. $\frac{1024 \times 6.4 \times 1 0 ^{9}}{8} = 819.2 \times 1 0^{9}$ B/s $= 819.2$ GB/s. Why this step? Same $N f /8$ as Ex 1; only $f$ changed.

Step 2 — Multiply by stack count. $4 \times 819.2 = 3276.8$ GB/s $\approx 3.28$ TB/s. Why this step? The stacks sit on independent channels (GPU Architecture talks to all of them at once), so their bandwidths add — there is no shared wire to bottleneck them.

Verify: $3276.8$ GB/s $= 3.2768$ TB/s, right in the ~3 TB/s forecast. ✓ Sanity: doubling to 8 stacks would give ~6.5 TB/s, which matches top-end 2020s accelerators. ✓

Example 3 — Cell C: the DDR clock trap

Worked example Don't forget the "double"

Statement: A datasheet says the HBM2 memory clock is $1.0$ GHz (DDR). What is the effective per-pin rate, and the per-stack bandwidth?

Forecast: People who forget "DDR" get $1024 \times 1.0 G /8 = 128$ GB/s — half the real answer. Predict whether that's the trap.

Step 1 — Turn clock into effective rate. DDR = Double Data Rate: one bit on the rising clock edge, one on the falling. So $f = 2 \times f_{clk} = 2 \times 1.0 GHz = 2.0$ Gbps. Why this step? The formula $N f /8$ needs the effective bits-per-second per wire, not the clock frequency. A DDR wire delivers two bits per clock cycle.

Step 2 — Apply $N f /8$ . $\frac{1024 \times 2.0 \times 1 0 ^{9}}{8} = 256$ GB/s. Why this step? Now $f$ is the honest delivered rate, so the standard formula applies.

Verify: This equals Ex 1 exactly, because "1 GHz DDR" and "2.0 Gbps effective" are two names for the same thing. ✓ The trap answer $128$ GB/s is exactly half — the tell-tale sign you forgot the ×2. ✓

Example 4 — Cell D: inverse solve

Example 5 — Cell E: degenerate case, a channel fails

Worked example What if one channel dies?

Statement: An HBM2 stack normally has $8$ channels $\times 128$ bits $= 1024$ bits. Suppose one channel is disabled (a defective die block, common in binning). Effective width and bandwidth at $2.0$ Gbps?

Forecast: Losing 1 of 8 channels loses $1/8$ of the width, so ~87.5% of 256 GB/s ≈ 224 GB/s. Guess that.

Step 1 — New width. $7$ working channels $\times 128 = 896$ bits. So $N = 896$ . Why this step? A dead channel contributes zero wires; bandwidth is built per channel, so we just drop its 128 bits.

Step 2 — Apply $N f /8$ . $\frac{896 \times 2.0 \times 1 0 ^{9}}{8} = 224 \times 1 0^{9} = 224$ GB/s. Why this step? The rate per surviving wire is unchanged; only $N$ shrank.

Verify: $\frac{224}{256} = \frac{7}{8} = 0.875$ — exactly the fraction of channels left. ✓ Matches forecast. ✓ This is why real products sometimes ship at odd capacities/bandwidths: partial-good dies get sold with a channel fused off.

Example 6 — Cell F: zero and single-wire limits

Worked example Pushing to the extremes

Statement: Two limiting checks on the formula. (a) If a wire's rate drops to $f = 0$ , what is the bandwidth? (b) What does a single wire ( $N = 1$ ) at $2.0$ Gbps contribute?

Forecast: (a) A wire toggling zero times per second moves nothing → 0. (b) One wire is $1/1024$ of a full bus → about a quarter of a GB/s.

Step 1 — Zero-rate case. $BW = N \times 0/8 = 0$ B/s, for any $N$ . Why this step? Width is useless if nothing toggles. This shows both knobs must be non-zero — the parent's two-knob picture in words.

Step 2 — Single-wire case. $\frac{1 \times 2.0 \times 1 0 ^{9}}{8} = 0.25 \times 1 0^{9} = 0.25$ GB/s $= 250$ MB/s. Why this step? Isolating one wire shows the "atom" of HBM bandwidth. Multiply by $1024$ and you rebuild the full 256 GB/s: $0.25 \times 1024 = 256$ . ✓

Verify: $0.25 \times 1024 = 256$ GB/s recovers Ex 1 exactly. ✓ And $0$ is the only sensible answer when nothing moves. ✓ This is the "degenerate input" guardrail — the formula behaves at the edges.

Example 7 — Cell G: HBM vs GDDR, same bandwidth, different energy

Worked example Why width beats speed on power

Statement: Compare two ways to reach $\approx 256$ GB/s: (i) HBM2-style, $N = 1024$ wires at $2.0$ Gbps; (ii) GDDR-style, $N = 128$ wires at $16$ Gbps. Confirm both hit the same bandwidth, then argue which sips less power.

Forecast: Both are engineered for ~256 GB/s, so bandwidth should tie. Power should favour HBM because each wire is slower and shorter.

Step 1 — HBM bandwidth. $1024 \times 2.0 G /8 = 256$ GB/s. Step 2 — GDDR bandwidth. $128 \times 16 G /8 = 256$ GB/s. Why these steps? Same formula, different $(N, f)$ split — proving bandwidth is set by the product $N f$ , not by $f$ alone.

Step 3 — Power argument. Dynamic wire power $\propto C V^{2} f$ . HBM's on-interposer wires are short (small $C$ ), low-swing (small $V$ ), and low- $f$ . GDDR's long PCB traces at $16$ Gbps have larger $C$ , larger $V$ , and $8 \times$ the $f$ per wire. Why this step? It answers the exam's "which is more efficient?" See Energy per Bit / pJ per bit and DDR vs GDDR vs HBM for the measured figures.

Verify: $256 = 256$ — bandwidths match exactly, so the only differentiator left is energy. ✓ The $N f$ products are equal: $1024 \times 2 = 2048$ and $128 \times 16 = 2048$ (Gbps·wires). ✓ Consistent with the parent's "wide + slow wins on power."

Example 8 — Cell H: real-world word problem

Worked example Can the memory keep the GPU fed?

Statement: A GPU does $200$ TFLOP/s of useful work, and its workload needs (arithmetic intensity) about $2$ bytes of memory traffic per FLOP. Its memory is $4$ HBM3 stacks (each $819.2$ GB/s). Is memory the bottleneck?

Forecast: $200$ TFLOP/s × 2 B/FLOP = 400 TB/s demanded. Four stacks give ~3.3 TB/s. That's ~100× short — memory is the wall.

Step 1 — Bandwidth demanded by compute. $200 \times 1 0^{12} \frac{FLOP}{s} \times 2 \frac{B}{FLOP} = 400 \times 1 0^{12}$ B/s $= 400$ TB/s. Why this step? Multiplying FLOP-rate by bytes-per-FLOP converts "how much math" into "how much data the math needs" — units cancel FLOP.

Step 2 — Bandwidth supplied. $4 \times 819.2 = 3276.8$ GB/s $= 3.2768$ TB/s (from Ex 2). Why this step? That's the physical ceiling the HBM can deliver.

Step 3 — Compare. $\frac{400}{3.2768} \approx 122$ . Demand is ~122× supply → memory-bound. Why this step? The larger of (demand, supply) that the system can't meet is the bottleneck; here supply loses badly, which is exactly the Memory Wall and why we cache/reuse data on-chip.

Verify: $3276.8 \times 122 \approx 3.998 \times 1 0^{5}$ GB/s $\approx 400$ TB/s — the ratio checks. ✓ Interpretation: without heavy on-chip data reuse, this GPU would idle waiting on HBM. ✓

Example 9 — Cell I: the exam twist (channels doubled, bus unchanged)

Worked example "HBM3 doubled channels, so is the bus twice as wide?"

Statement: HBM2 = $8$ channels $\times 128$ bits. HBM3 = $16$ channels $\times 64$ bits. Show the total data width, and where the real HBM3 gain comes from.

Forecast: The "twist" tempts you to say $16 \times 128 = 2048$ . Predict whether that's wrong.

Step 1 — HBM2 width. $8 \times 128 = 1024$ bits. Step 2 — HBM3 width. $16 \times 64 = 1024$ bits. Why these steps? Width is channels × bits-per-channel. HBM3 halved the bits-per-channel while doubling channel count, so the product is unchanged at 1024.

Step 3 — Where the gain is. Both are $1024$ -bit. HBM3's higher bandwidth comes from a bigger per-pin $f$ (e.g. $6.4$ Gbps vs $2.0$ ), not a wider bus. More channels = finer-grained parallelism (more independent requests in flight), not more wires. Why this step? It kills the common misread and points to the true lever ( $f$ ), consistent with the parent.

Verify: $8 \times 128 = 1024 = 16 \times 64$ . ✓ And HBM3 BW $= 1024 \times 6.4 G /8 = 819.2$ GB/s vs HBM2's $256$ — the $3.2 \times$ jump is exactly the ratio $6.4/2.0$ , confirming $f$ (not $N$ ) did the work. ✓

Active recall

Recall Which cell is this? (hide answers)

"Given 1024-bit and 2 Gbps, find BW." ::: Cell A → 256 GB/s.
"Clock is 1 GHz DDR, find effective rate." ::: Cell C → 2.0 Gbps (×2).
"Stack delivers 460.8 GB/s, find f." ::: Cell D → 3.6 Gbps.
"One of 8 channels disabled, find BW." ::: Cell E → 224 GB/s.
"16×64 vs 8×128 — same width?" ::: Cell I → yes, both 1024 bits.
"200 TFLOP/s ×2 B/FLOP vs 3.3 TB/s — bottleneck?" ::: Cell H → memory-bound (~122×).

Connections

Parent topic — the theory these examples exercise.
Through-Silicon Vias (TSV) · Silicon Interposer · 2.5D and 3D Integration — why 1024 wires per stack are even routable.
GDDR Memory · DDR vs GDDR vs HBM · Energy per Bit / pJ per bit — the comparison in Ex 7.
Memory Wall · GPU Architecture — the word problem in Ex 8.