Foundations — High Bandwidth Memory (HBM - HBM2 - HBM3)

This page assumes nothing. Before you read the parent note on High Bandwidth Memory, we build every word and symbol it leans on. Read top to bottom; each idea uses only the ones above it.

1. A "bit" and a "byte" — the units everything is counted in

Why the topic needs this: memory capacity and bandwidth are measured in bytes, but the actual hardware pushes bits down wires. Every time you see "divide by 8" later, it is because we are translating bits the wires carry into bytes we quote.

2. A "wire / pin" and what it means to "toggle"

Look at the figure: each horizontal line is one wire. A wire that toggles quickly sends many bits per second; a wire that toggles slowly sends fewer. A single wire can never send two bits at the same instant — that is why we need many wires to move data fast.

Why the topic needs this: HBM's headline is "1024 wires." You cannot understand why width helps until you see that each extra wire is an independent lane carrying its own stream of bits.

3. The symbol $N$ — how many data wires

4. The symbol $f$ — per-pin transfer rate, and why "Gbps"

The figure shows one clock wave (violet). With single data rate, one bit lands per full cycle. With double data rate (DDR), a bit lands on both the up-edge and the down-edge — twice the bits from the same clock. That is why "1 GHz DDR = 2 Gbps."

5. Putting them together — the bandwidth formula

See Memory Wall for why we are desperate to grow this number, and DDR vs GDDR vs HBM for how the two knobs are split differently across memory families.

6. Power per wire — the symbols $P$, $C$, $V$

Why the topic needs this: it explains HBM's superpower. HBM keeps its wires short (small $C$, because they sit on a tiny carrier next to the chip), low-swing (small $V$), and slow (small $f$). Small $C$, small $V^2$, small $f$ → tiny $P$ per wire. Even with 1024 wires, the total stays low. This is measured as energy per bit (pJ/bit).

7. Why the wires can be short — the physical stack

The figure is a side view. TSVs (magenta) run vertically through the DRAM dies. Microbumps drop the stack onto the interposer (orange), which sits beside the GPU/processor. Because both live on the same tiny tray, the data wires are millimetres long instead of the centimetres of a printed circuit board — that is the geometric reason $C$ is small.

Why the topic needs this: the formula in section 6 says short wires save power, but nothing physically forces wires to be short. TSVs + interposer are the machinery that makes them short. Compare with GDDR Memory, where chips sit far away on the board.

8. "N-Hi" and "channel" — counting height and independence

The prerequisite map

Read it as: units feed the bandwidth formula and the power formula; the physical stack makes wires short (small $C$); together they explain why HBM chooses wide + slow.

Equipment checklist

You are ready for the parent note when you can answer each of these out loud: