Visual walkthrough — NVLink and GPU interconnects

6.3.4 · D2 · Hardware › Interconnects, Buses & SoC › NVLink and GPU interconnects

Pehle hum target number state bhi nahi kar sakte jab tak ek unit clear na ho. Bandwidth bytes per second mein measure hoti hai, aur ek byte hoti hai 8 bits:

Yeh unit samajh ke, yeh hai woh ek target number jiske peeche hum hain:

$$\underset{\text{answer}}{\underbrace{20}}\text{GB/s per direction (NVLink 2.0)}$$

Neeche har step exactly ek idea add karta hai jab tak yeh number nahi nikal aata.

---

Step 1 — Ek wire, ek bit bhejta hua

KYA. Sabse basic cheez jo ek link karta hai: wire ko high voltage pe rakhna matlab "1", low voltage pe rakhna matlab "0", phir switch karna. Har switch ek naya symbol bhejne ka mauka hai.

YAHAN SE KYU shuru karein. Har bada bandwidth number asliyat mein bas "ek second mein kitni baar voltage change kar sakte hain" multiplied by "har change kitna meaning carry karta hai" hota hai. Hume dono halves earn karne hain. Yeh step pehla half earn karta hai.

PICTURE. Neeche, wire ki voltage time ke saath upar-neeche jaati hai. Har shaded slot ek symbol period hai — woh time jitna wire still rehti hai agle change se pehle.

NVLink 1.0 ke liye yeh rate $20\text{GT/s}$ hai; NVLink 2.0/3.0 ke liye $25\text{GT/s}$ hai. Yeh woh fastest speed hai jis par transmitter wire ko cleanly flip kar sakta hai.

---

Step 2 — Do wires, kyunki ek wire jhooth bolta hai

KYA. Ek wire jo ground ke against measure ho uski jagah, NVLink do wires use karta hai aur unke beech ka difference padhta hai.

YEH TOOL KYU aur single wire kyun nahi? 25 billion flips per second par, noise enormous hoti hai — paas ke switching currents ground reference ko itna hilate hain ki receiver trust nahi kar sakta ki "kya yeh high hai ya low?" Trick yeh hai: agar noise dono wires ko equally hit kare, toh ek ko doosre se subtract karne par noise delete ho jaati hai. Hum subtraction specifically isliye use karte hain kyunki yeh woh ek operation hai jo dono inputs mein common kuch bhi kill kar deta hai.

PICTURE. Upar: do wires, ek doosre ka mirror. Beech mein: same noise bump ($n$) dono pe padhta hai. Neeche: unka difference — clean, noise gone.

---

Step 3 — Har symbol tumhara poora ek bit nahi hota

KYA. Yahan catch hai. Step 1 ke raw symbols sab tumhara payload nahi hain. NVLink 2.0 8b/10b encoding use karta hai: tumhare har 8 bits of data ke liye, yeh wire par actually 10 symbols transmit karta hai.

8 bhejne ke liye 10 kyun kharch karein? Do survival needs:

1. Clock recovery — receiver ke paas koi alag clock wire nahi hota; woh timing data mein edges (transitions) se recover karta hai. Saari-1s ki lambi run mein koi edge nahi hoti → receiver drift karta hai aur track kho deta hai. 8b/10b frequent transitions guarantee karta hai.
2. DC balance — average voltage centered rakhta hai taaki electronics slowly saturate na hon.

Iski price ek fixed tax hai: wire ke symbols ka sirf $\frac{8}{10}$ hi tumhara data carry karta hai.

PICTURE. 10 wire-symbols ka ek bar; 8 "tumhara data" (payload) hain, 2 "encoding tax" hain. Efficiency fraction $\frac{8}{10}=0.8$ highlighted hai.

---

Step 4 — 8 lanes ko ek link mein stack karo

KYA. Ek lane 20 Gb/s deta hai. NVLink 2.0 ek link banane ke liye har direction mein 8 lanes bundle karta hai. Lanes side by side, parallel mein run karti hain, data unke beech split hota hai.

8 parallel mein kyun? Tum ek single wire ko electronics se zyada fast flip nahi kar sakte (Step 1 ki ceiling). Zyada fast jaane ke liye tum wider jaate ho: kai identical lanes ek saath run karo aur unki rates add karo. Yeh same idea hai jaise motorway mein speed limit badhane ki jagah lanes add karna.

PICTURE. 8 lane-arrows side by side, har ek 20 Gb/s labelled, ek mote bundle mein sum hote hue 160 Gb/s labelled.

---

Step 5 — Dono directions ek saath (full duplex)

KYA. Ek link mein har direction ke liye lanes ka ek alag set hota hai, toh data dono taraf simultaneously flow karta hai. Per-link figure ke liye unhe add karo.

Har direction ke liye alag lanes kyun? Training workloads ek saath gradients send aur receive karte hain. Dedicated lanes per direction ka matlab hai ki na koi direction doosre ka wait karta hai — koi turn-taking nahi.

PICTURE. Do GPUs ke beech opposite directions mein point karte do 8-lane bundles, har ek 20 GB/s, total 40 GB/s.

---

Step 6 — Ek GPU, six links

KYA. Ek single V100 mein ek link nahi hota — uske paas 6 NVLink ports hote hain. Poore GPU ke aggregate ke liye unhe sum karo.

Six kyun? 8-GPU mesh wire up karne ke liye jahan har GPU seedhe kai neighbours tak pahunch sake. Zyada links = zyada direct neighbours = all-reduce ke dauran kam hops.

PICTURE. Ek central GPU jisme se 6 bidirectional links fan out ho rahe hain, har ek 40 GB/s labelled, total 240 GB/s.

---

Step 7 — Edge cases (jahan naive number jhooth bolta hai)

KYA. 240 GB/s ek aggregate ceiling hai. Real transfers ise kabhi nahi dekhte. Teeen degenerate situations jo jaanni chahiye:

YEH COVER KYU KAREIN. Agar tum sirf "240 GB/s" yaad rakhoge, tum performance wildly over-predict karoge. Neeche har case ek real reason se number cap karta hai.

PICTURE. Teen mini-scenarios: (a) ek link share karte do GPUs 40 GB/s par cap; (b) mesh mein traffic split karta all-reduce; (c) ek chota buffer jo ack return hone se pehle stall karta hai.

• Case A — ek single pair. GPU0→GPU4 ek link par 20 GB/s per direction se zyada nahi ja sakta, 240 nahi. Aggregate tabhi appear hota hai jab saare 6 links ek saath busy hon.
• Case B — indirect pairs. Agar do GPUs directly linked nahi hain, toh traffic multiple hops leta hai, links ko doosre flows ke saath share karta hai. Effective bandwidth single-link figure se neeche girti hai.
• Case C — buffer too small (flow-control stall). Sender utna hi data transmit kar sakta hai jitna uske credits allow karein acknowledgement return hone se pehle. Agar buffer "in flight" data se chota ho, toh woh khaali ho jaata hai aur stall karta hai.

---

Ek-picture summary

Sab kuch ek canvas par: raw symbols → noise subtract karo → encoding tax bharo → 8 lanes wide karo → duplex ke liye double karo → six links. Arrows follow karo aur target number khud assemble ho jaata hai.

$$\underset{\text{Step 1}}{\underbrace{25\text{GT/s}}}\times \underset{\text{Step 3}}{\underbrace{\frac{8}{10}}}\times \underset{\text{Step 4 lanes}}{\underbrace{8}}÷\underset{\text{bits→bytes}}{\underbrace{8}}=20\text{GB/s}\stackrel{\times 2}{\to }\underset{\text{duplex}}{\underbrace{40}}\stackrel{\times 6}{\to }\underset{\text{per GPU}}{\underbrace{240\text{GB/s}}}$$

Recall Feynman retelling — plain words mein wapas batao

Socho ek wire jise tum 25 billion times per second upar-neeche flick kar sakte ho — yeh tumhari raw speed hai, "transfers" mein measure ki gayi. Lekin ek wire jhooth bolta hai, kyunki noise use hilati hai; toh tum do wires use karte ho aur difference padhte ho, aur noise (jo dono ko equally hit karti hai) subtract hokar kuch nahi reh jaati. Ab, har 10 flicks mein se tum unhe sab real data ke liye use nahi kar sakte — 2 receiver ki clock sync rakhne aur voltage balanced rakhne mein jaate hain, toh sirf 8-of-10 flicks tumhare hain. Yeh chhod deta hai 20 billion real bits per second ek lane par. Tum faster flip nahi kar sakte, toh tum 8 lanes side by side run karte ho aur unhe add karte ho: 160 billion bits = 20 gigabytes per second, ek direction (yaad raho: 8 bits ek byte banate hain). Har direction ko apni lanes do aur tumhe milta hai 40 GB/s per link. Ek GPU par 6 links lagao aur woh apne neighbours se 240 GB/s par gossip kar sakta hai — poori PCIe road se lagbhag 15 guna zyada. Bas yaad raho: woh 240 "sab ek saath" ceiling hai; koi bhi single pair sirf 20–40 dekhti hai, aur agar tumhara buffer "in flight" data se chota hai (500 ns round trip ke liye lagbhag 10 KB), toh link stall karta hai "got it" reply ka wait karte hue.