Visual walkthrough — High Bandwidth Memory (HBM - HBM2 - HBM3)

6.5.4 · D2 · Hardware › Advanced & Emerging Architectures › High Bandwidth Memory (HBM - HBM2 - HBM3)

Is page mein parent result ko dheere dheere derive kiya gaya hai jo HBM use karta hai: $\text{BW}=\frac{N\times f}{8}$ Chalo ise ek ek picture ke saath build karte hain.

---

Step 1 — Ek akela wire actually kya kar raha hai?

KYA kar rahe hain: sabse tiny possible unit dekh rahe hain — ek wire, ek moment in time.

KYUN: bandwidth ek badi number hai jo bahut saare chhote events se bani hai. Agar hum sabse chota event nahi samjhe (ek wire ka high/low flip karna), toh badi number sirf magic hai. Aur hum magic se inkar karte hain.

PICTURE: neeche diye ek wire ko dekho. Red trace follow karo: jaise jaise time right mein move karta hai, yeh high, low, high, low step karta hai. Har coloured block ek bit hai. Wire ek light switch ki tarah hai jo flick ho rahi hai.

---

Step 2 — Ek wire kitni tezi se flip kar sakta hai? (yahi hai $f$)

KYA: hum count karte hain ki Step 1 ke kitne bit-blocks ek second mein fit hote hain.

YE SYMBOL KYUN: hume "wire speed" ke liye ek naam chahiye taaki hum memories ko compare kar sakein. Ise $f$ (transfers ki frequency ke liye) kehna hume paragraphs ki jagah formulas likhne deta hai.

PICTURE: neeche ki timeline bits ko exactly ek second mein pack karti hai. Upar slow wire (kam, wide bits), neeche fast wire (zyada, narrow bits). Same second, zyada bits ⇒ bada $f$.

---

Step 3 — Ek wire bahut slow hai. Aur wires add karo (yahi hai $N$).

KYA: hum kaafi Step-1 wires ko ek dusre ke saath rakhte hain aur unhe sab ek instant mein read karte hain.

SIRF FASTER WIRE KI JAGAH $N$ KYUN? $f$ ko zyada push karne mein bahut power lagti hai aur signal noisy ho jaata hai (Step 6 mein dekhenge kyun). Wires add karna bits move karne ka ek sasta, shaant tarika hai — jab tak wires ke liye jagah ho. HBM ka poora trick yahi hai: $N$ ko enormous banao.

PICTURE: 8 wires vertically stack hue hain. Highlighted column ko dekho — ek instant mein tum 8 bits ek saath collect karte ho. Yeh same $f$ ke liye ek akele wire ka 8× data hai.

---

Step 4 — Multiply karo: total bits per second

KYA: hum Step 2 ($f$) aur Step 3 ($N$) ko jodh dete hain.

MULTIPLY KYUN AUR $N+f$ ADD KYUN NAHI? Add karna bakwaas hoga — units bhi match nahi karti ($N$ ek plain count hai, $f$ bits/s hai). "$N$ wires, har ek $f$ bits/s kar raha hai" $N$ groups of $f$ hai, aur "groups of" matlab multiplication hota hai. Hamesha units ko guide karne do: (wires) × (bits/s per wire) = bits/s. ✓

PICTURE: ek grid — $N$ rows (wires) tall, aur $f$ bits wide (ek second). Grid ka area = us second mein total bits. Area = height × width = $N\times f$.

---

Step 5 — 8 se divide kyun karte hain? (bits → bytes)

KYA: hum apne bits per second ko bytes per second mein convert karte hain.

KYUN: koi bhi memory bandwidth ko "gigabits per second" mein nahi bechta — spec sheets, GPU boxes, benchmarks sab GB/s (gigabytes) kehte hain. Same language bolne ke liye hum 8 bits ko 1 byte mein regroup karte hain, jiska matlab hai bit-count ko 8 se divide karna.

PICTURE: bit-stream lo aur har 8 ke group ke around brackets draw karo. Brackets count karo — yahi tumhara byte count hai. Bits se kam brackets, exactly 8× kam.

---

Step 6 — HBM2 ke numbers plug karo aur headline number padho

KYA: real HBM2 numbers use karo — $N=1024$ wires per stack, $f=2.0\text{Gbps}$ per wire.

YE VALUES KYUN? Ek stack 8 channels mein split hota hai jinmein se har ek 128 bits ka hai; $8\times 128=1024$ wires. Aur har wire Step 2 wale skrompt 2 Gbps par run karta hai. Hum fast clock nahi kar rahe — hum wide ja rahe hain.

${\text{BW}}_{\text{stack}}=\frac{\stackrel{N\text{wires}}{\overbrace{1024}}\times \stackrel{f\text{bits/s}}{\overbrace{2.0\times 10^9}}}{\underset{\text{bits per byte}}{\underbrace{8}}}=2.56\times 10^{11}\text{B/s}=256\text{GB/s}$

PICTURE: ek bar chart jo do philosophies ko contrast karta hai. GDDR = tall-and-thin (huge $f$, few wires). HBM = short-and-fat (small $f$, huge $N$). Unke areas (= bandwidth) comparable hain, lekin HBM ke short bars ka matlab hai bahut kam heat aur kam energy per bit.

---

Step 7 — Scale up karo aur edge cases check karo

KYA: ek processor par kaafi stacks rakho, aur weird inputs check karo taaki kuch surprise na kare.

EDGE CASES KYUN CHECK KAREIN? Jo formula tum sirf "nice" numbers ke liye trust karo, woh ek trap hai. Chalo ise push karte hain.

• Kaafi stacks (real GPU case). Jab $S$ stacks apne channels par parallel kaam karte hain, total $\text{BW}=S\times \frac{Nf}{8}$. HBM2 4-stack: $4\times 256=1024\text{GB/s}\approx 1\text{TB/s}$. $S$ se multiply kyun karte hain? Independent channels wires share nahi karte — yeh Step 4 ke 4 alag grids hain, add kiye gaye.
• $N=1$ (ek akela wire). BW $=f/8$. $f=2$ Gbps par yeh 250 MB/s hai — tum ek akele garden tap par wapas aa gaye. Formula gracefully degrade karta hai.
• $f=0$ (idle bus). BW $=0$. Koi flip nahi ⇒ koi bit nahi ⇒ koi byte nahi. Sensible: ek still wire kuch nahi move karta.
• Same BW target, do designs. 256 GB/s hit karne ke liye tum $N=1024,f=2$ (HBM) ya $N=128,f=16$ (GDDR-ish) use kar sakte ho. Same product $N\times f$, wildly different power kyunki sirf $f$ (aur $V$) $C{V}^{2}f$ term ko blow up karta hai. Yahi ek line mein the HBM insight hai.

PICTURE: char side-by-side grids (char stacks), har ek Step-4 area, right par total-BW bar mein sum ho rahe hain.

---

Ek picture mein summary

Puri journey ek canvas par: ek wire → speed $f$ → bahut saare wires $N$ → multiply karo → 8 se regroup karo → HBM2 plug karo → 256 GB/s.

Recall Feynman: plain words mein batao wapas

Ek wire picture karo memory aur chip ke beech. Yeh ek light switch hai jo on/off flick hoti hai, aur har flick ek bit hai. Ek second mein flicks count karo — yahi wire ki speed hai, $f$. Ek wire bahut slow hai, toh hum 1024 wires side by side chalate hain; ek instant mein hum 1024 bits ek saath grab karte hain. Total bits per second = wires × speed = $N\times f$. Lekin duniya bytes mein count karti hai (8 bits ke bundles), toh hum 8 se divide karte hain. HBM2 ke liye yeh hai $1024\times 2\text{ billion}÷8=256$ billion bytes per second — 256 GB/s per stack. Unhe char ko chip ke paas ek chhoti si glass tray par rakho aur tum ek terabyte per second ke paas pahunch jate ho. Aur kyunki har wire short aur slow hai, poori cheez muskil se garam hoti hai. Wide, short, aur stacked.

---

Connections

• High Bandwidth Memory (HBM - HBM2 - HBM3) — parent jise yeh walkthrough derive karta hai.
• GDDR Memory aur DDR vs GDDR vs HBM — tall-and-thin rival design.
• Memory Wall — kyun koi bhi bandwidth chasing matter karti hai.
• Through-Silicon Vias (TSV), 2.5D and 3D Integration, Silicon Interposer — 1024 wires physically kaise exist karte hain.
• Energy per Bit / pJ per bit — "wide + slow" ka $C{V}^{2}f$ payoff.
• GPU Architecture — customer jo in bytes ke liye starved hai.