NVLink and GPU interconnects: Question bank — Hinglish deep dive

6.3.4 · D5 · Hardware › Interconnects, Buses & SoC › NVLink and GPU interconnects

Shuru karne se pehle, kuch words baar baar aayenge. Chalo inhe plain-language meaning aur ek picture ke saath pin kar dete hain taaki neeche ke traps samajh aa sakein.

Definition Woh words jo tumhare dimaag mein clear hone chahiye

Lane = ek differential pair of wires — do taar side by side jo signal ko unke beech ke voltage difference ke roop mein carry karte hain. Lane wo sabse chhoti unit hai jo bits move karti hai. Figure 1 mein taar dekho.
Link = lanes ka ek bundle jo saath milkar ek NVLink connection banaate hain. NVLink 1.0–3.0 mein 8 lanes per link direction hain; 4.0 mein 9 hain. Ek GPU mein kaafi links hote hain (NVLink 2.0 par 6).
Signaling rate (GT/s, "gigatransfers per second") = ek lane per second mein kitne raw symbols flip karta hai. Ye wire par symbols count karta hai, encoding overhead ke saath.
Symbol = ek transfer mein wire jo ek cheez transmit karta hai. Ordinary signaling mein ek symbol 1 bit hota hai; PAM4 ke saath ek symbol 2 bits hota hai (char voltage levels). Figure 2 dekho.
Payload (GB/s) = encoding overhead strip karne ke baad jo actual useful data bachta hai. Ye wohi hai jo tumhara program dekhta hai.
Bidirectional = link ki dono directions ek saath chalti hain (full duplex), isliye tum inhe add kar sakte ho aggregate ke liye — lekin sirf jab question aggregate ke baare mein puchhe.
Flit = "flow-control unit", woh chhota fixed-size chunk jisme link data ko split karta hai taaki usme CRC checksum lagaya ja sake aur har chunk ke liye flow control track kiya ja sake (flit-and-CRC picture, figure 3 dekho).

Parent note NVLink and GPU interconnects dekho agar koi term unfamiliar lage.

Agar tum woh pictures dimaag mein rakhte ho, toh neeche ke aadhe traps waise hi gayab ho jaate hain.

True or false — justify karo

NVLink ek GPU server mein PCIe ko poori tarah replace kar deta hai.

False. GPUs abhi bhi PCIe ke upar boot karte hain, commands receive karte hain, aur CPU se data load karte hain; NVLink ek additional mesh hai sirf GPU-to-GPU (aur kuch GPU-to-CPU) traffic ke liye jo warna PCIe root complex ko choke kar deta.

Ek single NVLink 2.0 link ek direction mein 40 GB/s move karta hai.

False. 40 GB/s bidirectional total hai (20 GB/s har taraf). Ek direction akele 20 GB/s payload carry karta hai.

Kyunki NVLink 2.0 25 GT/s par signal karta hai, ek program per direction 25 GB/s par data read kar sakta hai.

False. 25 GT/s raw signaling rate hai. 8b/10b encoding 20% overhead hatane ke baad per lane 20 Gb/s milta hai, aur 8 lanes ÷ 8 bits/byte phir bhi per direction 20 GB/s payload par aa jaata hai.

Lane aur link ek hi cheez hai.

False. Lane ek differential pair hai (bits ki ek stream); link 8 aisi lanes hai (NVLink 4.0 par 9) jo parallel mein ek single NVLink connection ki tarah kaam karti hain.

NVLink lanes ki sankhya double karne se per-lane speed double ho jaati hai.

False. Lanes parallel mein kaam karti hain, isliye zyada lanes aggregate bandwidth badhati hain, na ki kisi single lane ki speed; per-lane rate signaling rate aur encoding se fixed hoti hai.

8b/10b encoding waste overhead hai jo link ko slow kar deta hai.

False. Iska cost 20% hai (har 10 coded bits mein se sirf 8 data hain), lekin ye guarantee karta hai ki frequent voltage transitions hon taaki receiver ka clock recovery kaam kare — iske bina link sync kho deta aur fail ho jaata, tumhe 0 GB/s milta, zyada nahi.

Differential signaling saara electrical noise cancel kar deta hai.

False. Ye common-mode noise cancel karta hai (wahi noise jo dono wires par equally hit karta hai, jo subtract ho jaata hai). Woh noise jo dono wires ke beech alag hota hai woh abhi bhi through jaata hai, isliye layout aur shielding abhi bhi matter karte hain.

NVLink 4.0, 3.0 se fast hai sirf isliye kyunki wires tez switch karti hain.

False. Signaling rate bahut kam badi (25 → 26.56 GT/s). Jump PAM4 (2 bits/symbol instead of 1) aur saste 128b/130b encoding (~1.5% overhead instead of 20%) se aata hai — do alag wins ek chhote rate bump ke upar layered.

Credit-based flow control bit errors detect karne ke liye exist karta hai.

False. Error detection CRC ka kaam hai (har flit par ek checksum). Credits isliye exist karte hain taaki sender receiver ke buffer ko overrun na kare — ye ek capacity problem hai, correctness problem nahi.

Agar ek link ka buffer bandwidth-delay product se bada ho, toh throughput line rate se aage badh jaata hai.

False. BDP ke barabar ya zyada buffer link ko 100% saturated rehne deta hai; use bada banana sirf memory waste karta hai — tum physical line rate se zyada nahi ja sakte.

Cube-mesh topology ka matlab hai ki har GPU seedha har doosre GPU se wired hai.

False. DGX-1 hybrid cube-mesh mein ek GPU door ke peers tak kuch hops mein pahunchta hai; direct wires sirf uske neighbors aur kuch cross-links ke saath hote hain.

Galti dhundo

"PCIe Gen3 x16 deta hai 16 GB/s, NVLink 2.0 deta hai 20 GB/s, toh NVLink sirf 25% faster hai."

Galti hai ek NVLink link ko poore PCIe se compare karna. Ek single GPU ke paas 6 NVLink connections hain (240 GB/s aggregate) apne ~16 GB/s PCIe path ke muqable mein — ~15× ka gap, 25% nahi.

"Bidirectional bandwidth find karne ke liye, signaling rate lo aur do se multiply karo."

Operations ka order galat hai. Pehle bits/symbol aur encoding efficiency se multiply karna hoga, phir ÷8 karke per direction payload bytes milenge, phir double karo. Raw line rate ko double karna 8b/10b loss ko double count karta hai.

"NVLink GPU1 ko GPU2 ki memory mein likhne deta hai, toh CPU ko pehle data copy karna hoga."

CPU copy exactly wohi cheez hai jo NVLink avoid karta hai. RDMA-style load/store ek GPU ko doosre ki memory directly touch karne deta hai, CPU ko poori tarah bypass karke.

"Flit par 16-bit CRC ka matlab hai ki har flit mein sirf 16 bits usable data hain."

16 bits ek checksum hai jo flit ke saath append hoti hai (flit ek fixed-size flow-control chunk hai), payload ko 16 bits tak shrink nahi karti; flit ka data field kaafi bada aur mostly usable hota hai.

"Pre-emphasis har frequency ko equally boost karta hai signal ko stronger banane ke liye."

Nahi — poora point high frequencies ko selectively boost karna hai, kyunki PCB traces high frequencies ko zyada attenuate karti hain. Sab kuch equally boost karna inter-symbol interference fix nahi karta.

"8 GPUs mein all-reduce ko 8 communication steps chahiye kyunki 8 GPUs hain."

Ring all-reduce ko 8 GPUs ke liye

N - 1 = 7

steps chahiye; count ring algorithm se aata hai, seedha GPUs ki sankhya se nahi.

"Agar credits zero ho jaayein toh link fail ho gaya aur reset hona chahiye."

Zero credits ek normal stall hai, failure nahi. Sender sirf ek acknowledgement ka wait karta hai taaki ek credit wapas mile, phir resume ho jaata hai.

Why questions

Kyun hum sirf PCIe ko wider (zyada lanes) nahi bana sakte NVLink invent karne ki jagah?

Yahan tak ki wide PCIe bhi har GPU-to-GPU transfer ko CPU root complex se funnel karta hai, hops, latency, aur ek shared bottleneck add karta hai; NVLink GPUs ko direct point-to-point paths deta hai jo CPU ko skip karte hain.

High-speed serial ko separate clock wire ki jagah clock recovery kyun chahiye?

Multi-GHz speeds par ek separate clock wire real distance par data ke saath alignment se drift kar jaati hai (skew); data mein transitions embed karna aur unse clock recover karna sender aur receiver ko ek saath lock rakhta hai.

Ek receiver buffer jo kam se kam bandwidth-delay product ke barabar ho kyun full speed ke liye zaroori hai?

BDP (bandwidth × latency) woh data hai jo already in flight hai; agar buffer itna nahi rakh sakta, toh sender apne credits exhaust kar deta hai pehle acknowledgement ke wapas aane se pehle aur stall karna padta hai, efficiency 100% se neeche aa jaati hai.

PAM4 sirf do baar faster switch karne ki jagah 4 voltage levels kyun use karta hai?

Tez switch karna frequency badhata hai, aur PCB traces high frequencies ko severely attenuate karti hain. Char levels 2 bits per symbol encode karte hain, data rate ko (roughly) usi switching speed par double karte hain, frequency-loss wall se bachte hain.

NVLink ek single shared bus ki jagah mesh topology kyun use karta hai?

Ek shared bus sabko baari baari lene par majboor karta hai, isliye bandwidth kaafi talkers mein collapse ho jaati hai. Ek mesh multiple simultaneous point-to-point paths deta hai, toh saare GPUs parallel mein data exchange kar sakte hain.

NVLink 4.0 8b/10b se 128b/130b encoding mein kyun switch karta hai?

8b/10b 20% waste karta hai (har 10 coded bits mein 8 data); 128b/130b sirf ~1.5% waste karta hai (130 mein 128). PAM4 ki higher rates par, sasta encoding us overhead ka zyaadatar hissa recover kar leta hai jo 8b/10b throw away karta.

PAM4 aur 128b/130b alag alag NVLink 4.0 ki near-doubling bandwidth mein kaise contribute karte hain?

PAM4 akela roughly rate double karta hai 2 bits/symbol carry karke instead of 1; encoding switch aur ~

\frac{0.985}{0.8} \approx 1.23 \times

add karta hai ~98.5% bits ko data rakh ke instead of 80%. Dono ek chhote 25→26.56 GT/s bump ke upar ek dusre se multiply hote hain.

Edge cases

Do GPUs ke beech effective NVLink bandwidth kya hogi jinke paas koi direct link nahi hai aur koi free intermediate path bhi nahi hai?

Us pair ke liye NVLink par effectively zero — traffic PCIe par fallback ho jaata hai (CPU ke through), toh "fast" number ab apply nahi hota; topology, spec nahi, decide karta hai.

Agar ek GPU ke paas 6 NVLink ports hain lekin system sirf 4 wire karta hai, toh kya hoga?

Sirf wired ports traffic carry karte hain, isliye aggregate bandwidth scale down hoti hai (jaise 4 × 40 GB/s); unused ports kuch contribute nahi karte — datasheet maximum assume karta hai ki saare ports populated hain.

Agar tum ek raw serial lane mein bina encoding ke identical bits ki endless string bhejo toh payload kya hoga?

Link clock sync kho deta hai (recover karne ke liye koi transitions nahi) aur receiver ka data garbage ban jaata hai, isliye usable payload zero ki taraf collapse ho jaata hai — exactly isliye encoding exist karti hai.

Jab latency double ho jaaye lekin buffer same size rahe toh throughput ka kya hoga?

Bandwidth-delay product double ho jaata hai, isliye fixed buffer ab BDP se chhota ho sakta hai; sender acks ka wait karte karte stall hota hai aur efficiency 100% se neeche aa jaati hai even though link speed unchanged hai.

Ring all-reduce mein, ek single GPU (N = 1) ke liye minimum steps kitne hain?

Zero — ek GPU ke saath exchange karne ke liye kuch nahi hai,

N - 1

formula se match karta hai, jo sahi tarah se 0 par degenerate ho jaata hai rather than break karne ke.

Bilkul us moment jab sender ka credit count zero hit karta hai aur ek acknowledgement simultaneously aata hai, kya link stall hoga?

Nahi — wapas aata ack turant ek credit restore karta hai, isliye sender bina kisi observable stall ke continue karta hai; boundary case resolve hota hai no interruption ke favour mein.

Agar encoding overhead 0% hoti, toh kya payload signaling rate ke barabar hoga (bit terms mein)?

Haan us idealized case mein, kyunki payload = signaling rate × bits/symbol × (data bits / coded bits); koi overhead nahi toh last ratio 1 hai. Real links mein hamesha kuch overhead hota hai, isliye payload hamesha strictly less hota hai.

Recall Ek-line self-test is page band karne se pehle

Yaad karo ki ek link bidirectional hota hai (dono directions ek saath). NVLink 2.0 ke liye — "25 GT/s, 8b/10b, 8 lanes" — sirf ek direction mein ek program kaun sa number experience karta hai? ::: 20 GB/s payload per direction: $25 \times \frac{8}{10} = 20$ Gb/s per lane, $\times 8$ lanes $= 160$ Gb/s, $\div 8 = 20$ GB/s. Bidirectional figure (40 GB/s) bas ye doubled hai.