High Bandwidth Memory (HBM - HBM2 - HBM3): Exercises

6.5.4 · D4 · Hardware › Advanced & Emerging Architectures › High Bandwidth Memory (HBM - HBM2 - HBM3)

Poore page par hum ek hi tool use karte hain: bandwidth equation. Chalte hain isse dobara state karte hain taaki kuch bhi assumed na ho.

Unit reminder taaki baad mein kuch magic na lage: $1 Gbps = 1 0^{9}$ bits/s, $1 GB/s = 1 0^{9}$ bytes/s, $1 TB/s = 1 0^{12}$ bytes/s.

Figure — High Bandwidth Memory (HBM - HBM2 - HBM3)

Level 1 — Recognition

Recall Solution L1.1

Do factors hain $N$ (number of data wires / bus width) aur $f$ (per-pin data rate).

HBM $N$ ko bada rakhta hai ( $\approx 1024$ bits per stack).
HBM $f$ ko chhota rakhta hai (kuch Gbps per pin), taaki har wire cool aur sasta rahe. Yeh parent note ki "wide and slow" strategy hai. Memory Wall dekho — kyun hum bandwidth ke peeche bhaagte hain.

Recall Solution L1.2

(a) TSV — ek vertical copper via jo die ke through jaata hai, ek DRAM die ko uske upar/neeche wale die se stack ke andar connect karta hai.
(b) Silicon Interposer — ek passive silicon carrier jo stack(s) aur processor ke beech hazaaron fine wires hold karta hai (yeh 2.5D integration hai).
(c) base logic die — stack ka sabse neeche wala die jo interface/PHY run karta hai.
(d) microbump — solder connection jo stack ko interposer se jodata hai (stack-to-interposer, die-to-die nahi).

KYA: $BW = N f /8$ mein plug in karo. KYUN: yeh bandwidth ki definition hai. $BW = \frac{1024 \times 2.0 \times 1 0 ^{9}}{8} = \frac{2.048 \times 1 0 ^{12}}{8} = 2.56 \times 1 0^{11} B/s = 256 GB/s$ Sanity check: $1024 \times 2 G = 2.048$ Tbit/s; ek byte 8 bits ka hota hai isliye 8 se divide karo → 256 GB/s. ✅

Recall Solution L2.2

Per stack: $\frac{1024 \times 6.4 \times 1 0 ^{9}}{8} = 819.2$ GB/s. Total: 4 se multiply kyun? Stacks independent channels par baithe hain aur data simultaneously move karte hain, isliye unki bandwidths add hoti hain. $4 \times 819.2 = 3276.8 GB/s = 3.2768 TB/s$

Recall Solution L2.3

Har stack ek 1024-bit data bus expose karta hai, aur 4 stacks hain: $4 \times 1024 = 4096 data wires$ Yeh kyun matter karta hai: 4096 fine-pitch traces ordinary PCB copper par route nahi ho sakti — isliye hi silicon interposer exist karta hai. GDDR Memory se compare karo, jo board par bahut kam wires use karta hai.

Level 3 — Analysis

Recall Solution L3.1

KYA: formula invert karke $f$ solve karo. KYUN: hume BW aur $N$ pata hai, $f$ chahiye. $f = \frac{8 \times BW}{N} = \frac{8 \times 1.0 \times 1 0 ^{12}}{1024} = 7.8125 \times 1 0^{9} bits/s \approx 7.81 Gbps$ Cheat-sheet se: HBM3 hai 6.4 Gbps (ek stack ke liye kam — 819 GB/s deta hai), HBM3E hai 9.6 Gbps (enough). Toh HBM3E pehli single-stack generation hai jo 1 TB/s cross karti hai.

Recall Solution L3.2

$N$ fixed hone par, bandwidth $f$ ke proportional hai: $BW \propto f$ . $\frac{BW _{HBM3}}{BW _{HBM2}} = \frac{f _{HBM3}}{f _{HBM2}} = \frac{6.4}{2.0} = 3.2$ Toh poora gain per-pin rate se hai, width se nahi. (HBM3 ka channel count 16×64-bit tak double hua, lekin woh abhi bhi 1024 bits hi hai — neeche L3 trap dekho.)

Recall Solution L3.3

$f$ kyun cancel hota hai? Energy per bit = power ÷ bits-per-second = $(C V^{2} f) / f = C V^{2}$ . Frequency drop out ho jaata hai — energy per bit sirf is par depend karta hai ki kitna charge push kiya ( $C$ ) aur kitna door ( $V^{2}$ ). $\frac{E _{HBM}}{E _{GDDR}} = \frac{C _{H} V _{H}^{2}}{C _{G} V _{G}^{2}} = (\frac{1}{4}) (\frac{1}{2})^{2} = \frac{1}{4} \cdot \frac{1}{4} = \frac{1}{16}$ Har HBM wire ~1/16 energy per bit spend karta hai. Energy per Bit / pJ per bit aur DDR vs GDDR vs HBM dekho.

Level 4 — Synthesis

Recall Solution L4.1

Stacks needed: $⌈ 3.0/1.2 ⌉ = ⌈ 2.5 ⌉ = 3$ stacks → $3 \times 1.2 = 3.6$ TB/s ✅ (2 stacks sirf 2.4 TB/s denge, kam). Data wires: $3 \times 1024 = 3072$ wires. PCB par impossible hai, interposer par route hoga — GPU Architecture dekho ki yeh stacks physically kahan baithe hain.

Recall Solution L4.2

Bandwidth constraint: $⌈ 2.4/1.2 ⌉ = 2$ stacks.
Capacity constraint: $⌈ 48/36 ⌉ = 2$ stacks (2×36 = 72 GB ≥ 48). Dono 2 dete hain, toh 2 stacks kaafi hain: $2 \times 1.2 = 2.4$ TB/s ✅ aur $72$ GB ≥ 48 ✅. Kaun sa bind karta hai? Dono yahan 2 par ek saath limit hit karte hain, lekin note karo ki bandwidth barely 2.4 meet karta hai jabki capacity mein 24 GB ka slack hai — isliye bandwidth tight (binding) constraint hai.

Level 5 — Mastery

Recall Solution L5.1

(a) GDDR pins: har pin = 16 Gbps = $2 GB/s$ . Chahiye $\frac{1.0 \times 1 0 ^{12}}{2 \times 1 0 ^{9}} = 500$ data pins. HBM2E stacks: per stack $= \frac{1024 \times 3.6 \times 1 0 ^{9}}{8} = 460.8$ GB/s. Chahiye $⌈ 1000/460.8 ⌉ = 3$ stacks (3×460.8 = 1382 GB/s ≥ 1 TB/s ✅; 2 stacks = 921 GB/s kam padta hai). (b) Wires: HBM2E $= 3 \times 1024 = 3072$ wires vs GDDR6 $= 500$ pins. HBM ~6× zyada wires use karta hai. (c) Total energy per bit ratio. Dono same total bits/s (1 TB/s) move karte hain. Energy per bit ek per-bit quantity hai — ise koi farq nahi ki kitne wires kaam baant rahe hain. Total energy/bit $\approx$ (energy per wire-bit), aur har HBM wire-bit ek GDDR wire-bit ka 1/16 cost karta hai. Toh HBM wahi 1 TB/s ko roughly 1/16 energy per bit par move karta hai. "Zyada wires" = "zyada power" kyun nahi: 6× wires hone par, har wire sirf $1/6$ traffic carry karta hai aur per bit $1/16$ cost karta hai. Power = (total bits/s) × (energy per bit); bits/s 1 TB/s par fixed hai, toh winner purely woh hai jiska energy per bit kam hai — HBM, decisively. Yeh Energy per Bit / pJ per bit ka core lesson hai.

Recall One-screen summary

Har problem = $BW = N f /8$ , zaroorat ke hisaab se invert karo.
HBM $N$ (width) push karta hai; generations mostly $f$ badhati hain; total $N$ HBM3 mein bhi 1024/stack hi rehta hai.
Requirements → stacks ko upar round karo (ceiling).
Energy per bit $\propto C V^{2}$ ( $f$ cancel hota hai) → HBM ke short, low-swing wires wire count se independent jeet-wale hain.
Parent note par wapas jao parent note · Hinglish version: 6.5.04 High Bandwidth Memory (HBM - HBM2 - HBM3) (Hinglish).

Exercises — High Bandwidth Memory (HBM - HBM2 - HBM3)