Worked examples — NVLink and GPU interconnects

6.3.4 · D3 · Hardware › Interconnects, Buses & SoC › NVLink and GPU interconnects

Yeh page ek drill hall hai. Parent note ne theory build ki; yahan hum har tarah ke NVLink arithmetic question ko hammer karte hain, ek worked example har case ke liye. Har solution se pehle main aapko Forecast karne ko kehta hoon — pehle guess karna hi woh tarika hai jisse number yaad rehta hai.

Scenario matrix

Cell	Case class	Tricky kyun hai	Example
A	8b/10b payload from line rate	$8/10$ multiply karna hoga	Ex 1
B	PAM4 + 128b/130b payload	2 bits/symbol AUR naya encoding ratio	Ex 2
C	Raw vs payload (datasheet twist)	headline number ≠ usable number	Ex 3
D	Direction bookkeeping (uni vs bidir)	factor-of-2 trap	Ex 4
E	Aggregate over many links	per-link ko link count se multiply karo	Ex 5
F	Saturated link (buffer ≥ BDP)	efficiency = 100%	Ex 6
G	Degenerate: buffer too small (< BDP)	stall, efficiency < 100%	Ex 7
H	Limiting case: zero latency / infinite buffer	boundary sanity check	Ex 8
I	Real-world all-reduce word problem	ring algorithm, many peers	Ex 9
J	Exam twist: NVLink vs PCIe crossover	NVLink kab worth it NAHI hai?	Ex 10

Har cell neeche fill ki gayi hai. Chalte hain.

---

Ek aisa tool banana jo hum har jagah reuse karein

Lagbhag har example mein same conversion chain use hoti hai, toh ise ek baar define karo, plain words mein, koi bhi symbol aane se pehle.

Neeche di gayi figure ko ek pipe ki tarah padho jo har stage par narrow hoti jaati hai. Baayi taraf wide mouth se shuru karo — raw $8\times 20=160$ GT/s ke symbols. Pipe apni width "×1 bit/symbol" stage se le jaati hai (on/off signaling kuch add nahi karta). Orange "×8/10" stage par pipe narrow hoti hai: width ka ek panchwa hissa encoding overhead ko jaata hai, aur hum 128 Gb/s par aa jaate hain. Aakhir mein green "÷8" stage phir narrow karti hai bits ko bytes mein convert karne ke liye, 16 GB/s spout par khatam hoti hai. Visual point: har stage flow ko sirf shrink kar sakti hai, kabhi grow nahi — isliye aapka final payload hamesha chain mein sabse chhota number hoga, aur koi stage bhool jaane ka matlab hai over-reporting.

---

Cell A — 8b/10b payload

1. Bits per lane per second $=20\text{GT/s}\times 1\frac{\text{bit}}{\text{symbol}}\times \frac{8}{10}=16\text{Gb/s}$. Yeh step kyun? Raw symbols ko real payload bits mein convert karta hai encoding tax pehle pay karke.
2. Saare 8 lanes $=8\times 16=128\text{Gb/s}$. Yeh step kyun? Lanes parallel mein run karti hain, toh unke payloads simply add ho jaate hain.
3. Gigabits → gigabytes: $128÷8=16\text{GB/s}$. Yeh step kyun? Ek byte 8 bits hota hai; datasheets GB/s mein bolte hain.

Verify: Master formula mein plug karo: $8\times 20\times 1\times \frac{8}{10}\times \frac{1}{8}=16$. ✓ Units: $\text{lanes}\cdot \frac{\text{Gsym}}{s}\cdot \frac{\text{bit}}{\text{sym}}\cdot \frac{\text{byte}}{8\text{bit}}=\text{GB/s}$. ✓ Parent table row NVLink 1.0 se match karta hai.

---

Cell B — PAM4 + 128b/130b

Example se pehle, yahan raw parent-note figures reproduce ki gayi hain taaki aapko context-switch na karna pade:

1. Bits/lane/s $=26.56\times 2\times \frac{128}{130}=52.30\text{Gb/s}$. Yeh step kyun? Ek saath do cheezein change hui hain — bits/symbol AUR encoding ratio — toh dono yahan enter hoti hain.
2. 9 lanes: $9\times 52.30=470.7\text{Gb/s}$. Yeh step kyun? Parallel lanes add hoti hain; note karo odd lane count (9, 8 nahi).
3. Gigabits → gigabytes: $470.7÷8=58.8\text{GB/s}$. Yeh step kyun? Exactly Ex 1 ki tarah — ek byte 8 bits hota hai, toh bit-rate ko 8 se divide karo bytes report karne ke liye. Result GB/s mein hai, Gb/s mein nahi.
4. Parent ke 46.5 se reconcile karo: parent ka datasheet-consistent figure $\approx 46.5$ GB/s/dir hai. Pure master-formula number 58.8 GB/s hai; real silicon mein extra framing/protocol overhead hota hai jo marketing payload fold in karti hai. Is drill ke liye hum formula value 58.8 GB/s report karte hain aur flag karte hain ki datasheet-consistent NVLink 4.0 near 46.5 quoted hai.

Verify: $9\times 26.56\times 2\times \frac{128}{130}÷8=58.8$ GB/s (formula). ✓ $÷8$ aur $b=2$ bits/symbol hi Ex 1 se single structural differences hain. 46.5 se gap protocol overhead hai encoding se pare — mistake box dekho.

---

Cell C — raw line rate vs payload (datasheet twist)

1. 25 decode karo: $25\text{GT/s}\times 1\frac{\text{bit}}{\text{sym}}÷8=3.125\text{GB/s}$ per lane raw symbols ka. 8 lanes par: $8\times 3.125=25\text{GB/s}$ raw. Yeh step kyun? Dikhata hai ki "25" symbols-as-bytes hai, encoding haircut se pehle.
2. 8b/10b apply karo: $25\times \frac{8}{10}=20\text{GB/s}$ payload. Yeh step kyun? Transmitted bits ka sirf $8/10$ aapka data hai.

Verify: $8\times 25\times 1\times \frac{8}{10}÷8=20$ GB/s. ✓ 25 pre-encoding line rate hai; 20 woh hai jis par aapka data actually move karta hai. Timing calculations se pehle hamesha convert karo.

---

Cell D — direction bookkeeping

1. Full duplex matlab simultaneous hai: send-lanes aur receive-lanes physically alag hain, toh directions bandwidth share nahi karti. Yeh step kyun? Isliye aap inhein add kar sakte ho; half-duplex bus par aap nahi kar sakte.
2. Add karo: $20+20=40\text{GB/s}$ bidirectional.

Verify: $2\times 20=40$ GB/s. ✓ Parent table "Bidir/link = 40 GB/s" NVLink 2.0 ke liye match karta hai. Trap: 40 GB/s kabhi one-way transfer time karne ke liye mat use karo — ek single flow sirf 20 dekhta hai.

---

Cell E — kai links par aggregate

1. Per-link bidir = 40 GB/s (Ex 4 se).
2. Chhe links = $6\times 40=240\text{GB/s}$ aggregate. Yeh step kyun? Links independent physical ports hain; unki bandwidths add hoti hain.

Verify: $6\times 40=240$ GB/s. ✓ Parent 240 GB/s aggregate kehta hai, aur $240/16=15\times$ PCIe Gen3 x16 se zyada. ✓ Caution: aggregate ≠ jo koi ek peer dekhta hai; ek peer 40 par capped hai (ya 20 one-way).

---

Cell F — saturated link (efficiency 100%)

Pehle do tools define karo jo saturation govern karte hain: BDP aur efficiency law jise hum Cells F, G aur H mein lean on karenge.

1. BDP compute karo $=20\frac{\text{GB}}{\text{s}}\times 500\text{ns}=20\times 10^9\times 500\times 10^{-9}=10,000\text{B}=10\text{KB}$ (decimal KB). Yeh step kyun? BDP zero stalls ke liye minimum buffer hai — yardstick.
2. Efficiency law apply karo: $\min \left(1,\frac{16}{10}\right)=\min (1,1.6)=1=100\%$. Yeh step kyun? Buffer ≥ BDP matlab ratio ≥ 1, toh cap fire hoti hai aur koi stall nahi.
3. Timeline se cross-check karo: buffer $16\text{KB}÷20\text{GB/s}=800\text{ns}$ mein drain hoti hai; pehla ack $500\text{ns}<800\text{ns}$ par return karta hai, exhaustion se pehle credits replenish karta hai. Yeh step kyun? Efficiency-law verdict ko actual clock ke against confirm karta hai.

Verify: ack 500 ns par aata hai, buffer 800 ns par empty hone se pehle → no stall. Efficiency $=\min (1,16/10)=100\%$. ✓

Neeche di gayi figure ko sender ki fill-and-refill timeline ki tarah padho. Blue curve "in flight" data ki amount hai jaise time aage badhta hai (rising kyunki sender 20 GB/s par pour kar raha hai). Gray dashed line 10 KB par BDP mark karti hai; orange dashed line 16 KB par buffer ceiling mark karti hai. Green dotted vertical 500 ns par hai jab pehla acknowledgement return karta hai. Key cheez jo dekhni hai: green line us point ke left par aati hai jahan blue curve 16 KB ceiling hit karti (800 ns par), toh credits buffer ever empty hone se pehle refresh ho jaate hain — blue curve kabhi flat nahi hoti, matlab link kabhi idle nahi hoti. Yeh geometric "ack-before-ceiling" exactly buffer ≥ BDP condition hai.

---

Cell G — degenerate: buffer too small (stall)

1. 6 KB bhejne mein time $=6\times 10^3\text{B}÷20\times 10^9\text{B/s}=300\text{ns}$. Yeh step kyun? Jab credits zero hite hain, itne time tak useful kaam chala.
2. Ack 500 ns par aata hai; sender 300 → 500 ns tak idle. Yeh step kyun? Yeh gap forced stall hai.
3. Stall $=500-300=200\text{ns}$.
4. Efficiency $=\min \left(1,\frac{6}{10}\right)=0.60=60\%$, $\frac{\text{busy}}{\text{cycle}}=\frac{300}{500}$ se match karta hai. Yeh step kyun? BDP se neeche cap fire nahi hoti, toh efficiency bas raw ratio buffer/BDP hai.

Verify: $300/500=0.6$ aur $6/10=0.6$ agree karte hain. ✓ Effective throughput $=0.6\times 20=12$ GB/s. ✓ Rule confirm: BDP se neeche, efficiency $=\text{buffer}/\text{BDP}=60\%$. ✓

---

Cell H — limiting cases (boundary sanity)

1. (a) $\text{BDP}=20\text{GB/s}\times 0\text{ns}=0\text{B}$. Phir $\text{efficiency}=\min (1,B/0)=\min (1,\infty )=1=100\%$ kisi bhi $B>0$ ke liye. Yeh step kyun? Law ko low boundary par test karta hai — koi in-flight data nahi matlab buffer requirement zero ho jaati hai.
2. (b) Jaise $B\to \infty$, $B/\text{BDP}\to \infty$, toh $\text{efficiency}=\min (1,\infty )=1=100\%$. Yeh step kyun? Dikhata hai ki $\min (1,\cdot )$ cap efficiency ko exactly 100% par rakhta hai, kabhi usse zyada nahi — ek sanity guard.

Verify: $\min (1,\infty )=1$ dono limits mein. ✓ Dono degenerate ends 100% dete hain, intuition se match karta hai: sirf finite latency paired with too-small buffer (Cell G) hi stall kar sakta hai.

---

Cell I — real-world all-reduce word problem

Numbers se pehle, algorithm carefully fix karo — common trap yeh assume karna hai ki har hop poora gradient move karta hai.

1. Steps $=2(N-1)=2\times (8-1)=14$ (reduce-scatter + all-gather). Yeh step kyun? Ring do $N-1$-step passes mein reduction complete karta hai, ek mein nahi; dono count karne chahiye.
2. Chunk per step $=M/N=1.5\text{GB}/8=0.1875\text{GB}$. Yeh step kyun? Yahi poora correction hai — ek hop ek chunk carry karta hai, poora gradient nahi.
3. Per-step time $=0.1875\text{GB}÷20\text{GB/s}=9.375\times 10^{-3}\text{s}=9.375\text{ms}$. Yeh step kyun? Saare links parallel mein apna chunk move karte hain, toh ek step ek single chunk-transfer time karti hai.
4. Total NVLink $=14\times 9.375\text{ms}=131.25\text{ms}$. Yeh step kyun? Steps ek ke baad ek run karte hain; multiply karo.
5. PCIe (5 GB/s effective): per step $=0.1875/5=37.5\text{ms}$; total $=14\times 37.5=525\text{ms}$. Yeh step kyun? Same algorithm aur data volume, slower link.

Verify: NVLink $=2(8-1)\times \frac{1.5/8}{20}=14\times 9.375\text{ms}=131.25$ ms. ✓ PCIe $=14\times 37.5=525$ ms. ✓ Speedup $=525/131.25=4.0\times$. Note karo yeh ek naive model (full 1.5 GB per hop) ko correct karta hai jo NVLink time $\frac{N}{2(N-1)/N}\approx$ kaafi × se over-estimate karta; parent ka ~10× headline ek more pessimistic PCIe figure aur "full-gradient" simplification use karta hai, toh uska arithmetic aur hamara differ karte hain — method yahan correct wala hai. Units: GB ÷ (GB/s) = s throughout. ✓

---

Cell J — exam twist: NVLink vs PCIe crossover

1. Har path model karo fixed cost + transfer time ke roop mein: NVLink time $=C+S/20$; PCIe time $=S/5$. Yeh step kyun? NVLink ka advantage speed hai lekin setup tax pay karta hai; PCIe ka fixed cost ~0 hai yahan.
2. Equal set karo: $C+\frac{S}{20}=\frac{S}{5}$. Yeh step kyun? Crossover exactly wahan hai jahan dono times match karte hain.
3. Solve karo: $C=\frac{S}{5}-\frac{S}{20}=\frac{4S-S}{20}=\frac{3S}{20}$, toh $S=\frac{20C}{3}$.
4. $C=50\mu \text{s}=50\times 10^{-6}\text{s}$ plug karo: $S=\frac{20\times 50\times 10^{-6}}{3}\text{GB}=3.33\times 10^{-4}\text{GB}=333\text{KB}$ (decimal KB). Yeh step kyun? 333 KB se neeche setup overhead NVLink ko plain PCIe se slower bana deta hai.

Verify: $S=333\text{KB}=3.33\times 10^{-4}$ GB par: NVLink $=50\mu s+3.33\times 10^{-4}/20\text{s}=50+16.67=66.67\mu s$; PCIe $=3.33\times 10^{-4}/5=66.67\mu s$. Equal ✓.

Neeche di gayi figure ko do cost lines ki tarah padho jo cross karti hain. Blue line (NVLink) $S=0$ par zero se upar shuru hoti hai — woh raised intercept fixed 50 µs setup tax hai — phir dhheere charhti hai (gentle slope $1/20$). Orange line (PCIe) origin se shuru hoti hai lekin steeply charhti hai (slope $1/5$, chaar guna steeper). Dono red dot par milti hain: $S=333$ KB, $66.7$ µs. Dot ke left orange line lower hai — chhote messages ke liye PCIe jeet ta hai. Right mein blue line lower hai — bade messages ke liye NVLink jeet ta hai. Sabak us intersection mein rehta hai: NVLink bade transfers ke liye shine karta hai (gradients, activations); chatty tiny messages ke liye setup tax winner flip kar sakta hai.

---

Drill hall band karo — das cells ke lessons

Recall Self-test: cell ka naam batao, phir answer do

NVLink 1.0 8-lane 20 GT/s 8b/10b unidirectional payload? ::: 16 GB/s (Cell A) NVLink 4.0 raw master-formula payload/dir (9 lanes, 26.56 GT/s, PAM4, 128b/130b)? ::: 58.8 GB/s formula (46.5 datasheet) (Cell B) NVLink 2.0 ke liye datasheet "25 GB/s/dir" kis payload correspond karta hai? ::: 20 GB/s after 8b/10b (Cell C) Ek V100 ka aggregate NVLink 2.0 bandwidth (6 links)? ::: 240 GB/s bidirectional (Cell E) 20 GB/s par 500 ns latency ke liye BDP? ::: 10 KB decimal (Cell F) 6 KB buffer ke saath efficiency jab BDP 10 KB hai? ::: 60% via min(1, 6/10) (Cell G) Ring all-reduce time, 8 GPUs, 1.5 GB, 20 GB/s (chunks, both phases)? ::: 131.25 ms (Cell I) Crossover payload jahan NVLink overhead 50 µs PCIe ke barabar ho (5 vs 20 GB/s)? ::: 333 KB (Cell J)