Exercises — NVLink and GPU interconnects

KYA karte hain: transfers-per-second ko payload bytes-per-second mein convert karte hain. KYUN: wire symbols (transfers) bhejti hai; har symbol ka sirf $8/10$ hi real data hota hai.

Per lane: $25\text{ GT/s}\times \frac{8}{10}=20\text{ Gb/s}$.

Saare 8 lanes: $8\times 20=160\text{ Gb/s}$.

Bits → bytes convert karo: $\frac{160}{8}=\boxed{20\text{ GB/s per direction}}$.

Full duplex = har direction ke liye alag wires hoti hain, isliye dono directions ek saath full speed pe chalti hain — koi time-sharing nahi.

$20\text{ GB/s}\times 2=\boxed{40\text{ GB/s per link (bidirectional)}}$.

$50\text{ GB/s}=25\text{ GB/s/dir}\times 2$, raw 25 GT/s line rate ko 25 GB/s treat karke — 8b/10b overhead hatane se pehle. Honest payload $=40\text{ GB/s}$ bidirectional (line rate ko $0.8$ se multiply karo).

Har port: $40\text{ GB/s}$ bidirectional. $6\times 40=\boxed{240\text{ GB/s aggregate}}$. Yahi number ek single PCIe Gen3 x16 slot ($16\text{ GB/s}$) se compare karne ka hai.

$\frac{240}{16}=\boxed{15\times }$.

KYUN PAM4 formula ko change karta hai: har transfer ab 2 bits carry karta hai (4 voltage levels), 1 nahi. Toh hum 2 se multiply karte hain.

$\underset{\text{lanes}}{\underbrace{9}}\times \underset{\text{GT/s}}{\underbrace{26.56}}\times \underset{\text{bits/symbol}}{\underbrace{2}}\times \underset{\text{encoding}}{\underbrace{\frac{128}{130}}}÷\underset{\text{bits/byte}}{\underbrace{8}}$

$=9\times 26.56\times 2\times 0.9846÷8\approx \boxed{58.8\text{ GB/s per direction}}$.

(NVIDIA isko round/derate karke $\approx 46.5\text{–}50\text{ GB/s}$ headline deta hai depend karta hai kaise count karte hain; method important hai.)

Per step: $\frac{1.5\text{ GB}}{20\text{ GB/s}}=0.075\text{ s}=75\text{ ms}$. Saat steps: $7\times 75=\boxed{525\text{ ms}}$.

KYUN 8 GPUs ke liye 7 steps: ek ring har chunk ko circle ke around pass karta hai; $N-1$ hand-offs ke baad har GPU ne har contribution dekh li hoti hai. Zyada GPUs = zyada steps, lekin per-step data chunk-sized rehta hai, toh yeh gracefully scale hota hai.

(a) BDP = data "in flight" = bandwidth $\times$ latency: $20\text{ GB/s}\times 500\text{ ns}=20\times 10^9\times 500\times 10^{-9}=10^4\text{ B}=\boxed{10\text{ KB}}.$ Figure dekho: credits poore in-flight region (shaded tube) ko cover karni chahiye ya sender pehli ack aane se pehle dry ho jaata hai.

(b) $6\text{ KB}$ buffer ke saath sender credits exhaust karta hai $t_{\text{drain}}=\frac{6\text{ KB}}{20\text{ GB/s}}=\frac{6\times 10^3}{20\times 10^9}=300\text{ ns}$ ke baad. Pehli ack $500\text{ ns}$ pe hi aati hai, toh sender $300$ se $500\text{ ns}$ tak idle rehta hai: $\text{stall}=500-300=\boxed{200\text{ ns}}.$ Efficiency $=\frac{\text{useful time}}{\text{cycle time}}=\frac{300}{500}=\boxed{60\%}$. Design fix: buffer $\ge$ BDP $=10\text{ KB}$ → $t_{\text{drain}}\ge 500\text{ ns}$ → ack draining se pehle aati hai → 100% efficiency.

PCIe naive: $7\times \frac{1.5\text{ GB}}{5\text{ GB/s}}=7\times 0.30=2.1\text{ s}=\boxed{2100\text{ ms}}$. $300\text{ ms}$ compute se compare karo → communication-bound by $7\times$. GPUs iteration ka zyaada time idle rehte hain.

NVLink: $525\text{ ms}$ comm vs $300\text{ ms}$ compute → abhi bhi communication-bound, lekin sirf $1.75\times$. Agar compute aur comm overlap ho (pipelined), toh effective iteration $\approx \max (525,300)=525\text{ ms}$ hoga $825\text{ ms}$ ki jagah.

Synthesis takeaway: NVLink magically tumhe compute-bound nahi banata yahan — yeh comm wall ko $2100$ se $525\text{ ms}$ tak shrink karta hai ($4\times$), ek hopeless run ko ek merely-comm-limited run mein badal deta hai jise overlap hide kar sakta hai.

Comm time $=7\times \frac{1.5}{B}$ seconds, chahiye $\le 0.420\text{ s}$, toh $B\ge \frac{7\times 1.5}{0.420}=\frac{10.5}{0.420}=25\text{ GB/s}.$

• 1.0 ($16$): $7\times 1.5/16=656\text{ ms}$ — too slow.
• 2.0 ($20$): $7\times 1.5/20=525\text{ ms}$ — too slow.
• 4.0 ($58.8$): $7\times 1.5/58.8=178.6\text{ ms}$ ✅.

Required $B\ge 25\text{ GB/s}$ 1.0 aur 2.0 ko exclude karta hai. Answer: NVLink 4.0 sabse slow listed generation hai jo deadline meet karti hai.

Payload per direction (bytes/s): $B=L\times R\times 2\times \frac{128}{130}÷8(\text{L lanes},R\text{ in GT/s}=\text{Gsym/s}).$ Constraint $B\ge 60$ ke liye solve karo: $L\times R\ge \frac{60\times 8}{2\times 0.9846}=\frac{480}{1.9692}=\mathrm{243.8.}$ Toh humein lanes $\times$ rate $\ge 243.8$ chahiye. $L=9$ choose karo: chahiye $R\ge 243.8/9=27.1\text{ GT/s}$. $R=28\text{ GT/s}$ choose karo ($\le 30$ ✅). Check karo: $9\times 28\times 2\times 0.9846÷8=\boxed{61.9\text{ GB/s}}\ge 60$ ✅. Ek valid design: 9 lanes at 28 GT/s. (Kaafi saare pairs kaam karte hain; e.g. 12 lanes at 21 GT/s deta hai $62.0\text{ GB/s}$.)

Buffer $\ge$ BDP chahiye: $\text{BDP}=61.9\times 10^9\text{ B/s}\times 600\times 10^{-9}\text{ s}=61.9\times 600=37140\text{ B}\approx \boxed{37.1\text{ KB}}.$ Insight: faster + longer-latency links ko bigger buffers chahiye. Bandwidth ya latency double karne se credits double ho jaate hain jo tumhe on-chip hold karne padte hain — yeh ek real silicon-area cost hai jo bound karti hai ki interconnects kitni fast grow kar sakte hain.

PCIe ek tree hai jo CPU se root hoti hai: har GPU-to-GPU byte ko root complex tak chadhna padhta hai aur wapas neeche aana padhta hai, toh saare GPU pairs CPU ke finite lanes share karte hain aur latency ke hops add hote hain. PCIe ko wider karna trunk bandwidth badhata hai lekin trunk abhi bhi shared aur CPU-serialized rehta hai — 8 GPUs gradients synchronize karte hue abhi bhi wahan collide karte hain. NVLink iske bajaye har GPU ko ek mesh mein dedicated point-to-point links deta hai, toh pairs parallel mein communicate karte hain bina CPU path ke; aggregate bandwidth links ki number ke saath scale hoti hai, ek shared root ke saath nahi. Differential signaling (noise $V_{+}-V_{-}$ se cancel hota hai) add karo per-lane rates ko shared bus se bahut upar push karne ke liye, aur tumhein woh $15\times$ jump milta hai jo parent note report karta hai.

NVLink 2.0 payload per direction 20 GB/s hai. Woh factor jo 8b/10b overhead hatata hai woh hai 8/10 (0.8). PAM4 payload ko 2 se multiply karta hai kyunki yeh 2 bits per symbol carry karta hai. Ek receiver buffer kam se kam bandwidth–delay product (BDP) hona chahiye 100% efficiency ke liye. $N$ GPUs mein ring all-reduce ==$N-1$== steps leta hai.