6.3.4 · D5Interconnects, Buses & SoC
Question bank — NVLink and GPU interconnects
Before we start, several words appear again and again. Let's pin each one to a plain-language meaning and a picture so the traps below make sense.
See the parent note NVLink and GPU interconnects if any term feels unfamiliar.
If you keep those pictures in mind, half the traps below disappear.
True or false — justify
NVLink replaces PCIe entirely in a GPU server.
False. GPUs still boot, receive commands, and load data from the CPU over PCIe; NVLink is an additional mesh only for GPU-to-GPU (and some GPU-to-CPU) traffic that would otherwise choke the PCIe root complex.
A single NVLink 2.0 link moves 40 GB/s in one direction.
False. 40 GB/s is the bidirectional total (20 GB/s each way). One direction alone carries 20 GB/s of payload.
Because NVLink 2.0 signals at 25 GT/s, a program can read data at 25 GB/s per direction.
False. 25 GT/s is the raw signaling rate. After 8b/10b encoding removes 20% overhead you get 20 Gb/s per lane, and 8 lanes ÷ 8 bits/byte still land at 20 GB/s of payload per direction.
A lane and a link are the same thing.
False. A lane is one differential pair (one stream of bits); a link is 8 such lanes (9 on NVLink 4.0) bundled to work in parallel as a single NVLink connection.
Doubling the number of NVLink lanes doubles per-lane speed.
False. Lanes work in parallel, so more lanes raise aggregate bandwidth, not the speed of any single lane; per-lane rate is fixed by the signaling rate and encoding.
8b/10b encoding is wasted overhead that slows the link down.
False. It costs 20% (only 8 of every 10 coded bits are data), but it guarantees frequent voltage transitions so the receiver's clock recovery works — without it the link would lose sync and fail entirely, giving you 0 GB/s, not more.
Differential signaling cancels all electrical noise.
False. It cancels common-mode noise (the same noise hitting both wires equally, which subtracts out). Noise that differs between the two wires still gets through, which is why layout and shielding still matter.
NVLink 4.0 is faster than 3.0 only because the wires switch faster.
False. The signaling rate barely moved (25 → 26.56 GT/s). The jump comes from PAM4 (2 bits/symbol instead of 1) and cheaper 128b/130b encoding (~1.5% overhead instead of 20%) — two separate wins layered on top of a tiny rate bump.
Credit-based flow control exists to detect bit errors.
False. Error detection is the CRC's job (one checksum per flit). Credits exist to stop the sender from overrunning the receiver's buffer — a capacity problem, not a correctness problem.
If a link's buffer is larger than the bandwidth-delay product, throughput increases beyond line rate.
False. A buffer at least equal to the BDP lets the link stay 100% saturated; making it bigger just wastes memory — you cannot exceed the physical line rate.
A cube-mesh topology means every GPU is directly wired to every other GPU.
False. In the DGX-1 hybrid cube-mesh a GPU reaches distant peers in up to a few hops; direct wires only exist to its neighbors plus a couple of cross-links.
Spot the error
"PCIe Gen3 x16 gives 16 GB/s, NVLink 2.0 gives 20 GB/s, so NVLink is only 25% faster."
The error is comparing one NVLink link to all of PCIe. A single GPU has 6 NVLink connections (240 GB/s aggregate) versus its ~16 GB/s PCIe path — a ~15× gap, not 25%.
"To find bidirectional bandwidth, take the signaling rate and multiply by two."
Wrong order of operations. You must first multiply by bits/symbol and encoding efficiency, then ÷8 to get payload bytes per direction, then double it. Doubling the raw line rate double-counts the 8b/10b loss.
"NVLink lets GPU1 write to GPU2's memory, so the CPU has to copy the data first."
The CPU copy is exactly what NVLink avoids. RDMA-style load/store lets one GPU touch another's memory directly, bypassing the CPU entirely.
"A 16-bit CRC per flit means only 16 bits of every flit are usable data."
The 16 bits are a checksum appended to the flit (a flit is the fixed-size flow-control chunk), not a shrinking of the payload to 16 bits; the flit's data field is far larger and mostly usable.
"Pre-emphasis boosts every frequency equally to make the signal stronger."
No — the whole point is selective boosting of high frequencies, because PCB traces attenuate high frequencies more. Boosting everything equally would not fix inter-symbol interference.
"An all-reduce across 8 GPUs needs 8 communication steps because there are 8 GPUs."
A ring all-reduce needs steps for 8 GPUs; the count follows from the ring algorithm, not directly from the number of GPUs.
"If credits reach zero the link has failed and must reset."
Zero credits is a normal stall, not a failure. The sender simply waits for an acknowledgement to return a credit, then resumes.
Why questions
Why can't you just make PCIe wider (more lanes) instead of inventing NVLink?
Even wide PCIe funnels every GPU-to-GPU transfer through the CPU root complex, adding hops, latency, and a shared bottleneck; NVLink gives GPUs direct point-to-point paths that skip the CPU.
Why does high-speed serial need clock recovery instead of a separate clock wire?
At multi-GHz speeds a separate clock wire drifts out of alignment with data (skew) over any real distance; embedding transitions in the data and recovering the clock from them keeps sender and receiver locked together.
Why is a receiver buffer of at least the bandwidth-delay product required for full speed?
The BDP (bandwidth × latency) is the data already in flight; if the buffer can't hold that much, the sender exhausts its credits before the first acknowledgement returns and must stall, dropping efficiency below 100%.
Why does PAM4 use 4 voltage levels rather than just switching twice as fast?
Switching faster raises the frequency, and PCB traces attenuate high frequencies severely. Four levels encode 2 bits per symbol, doubling the data rate at (roughly) the same switching speed, dodging the frequency-loss wall.
Why does NVLink use a mesh topology instead of a single shared bus?
A shared bus forces everyone to take turns, so bandwidth collapses under many talkers. A mesh gives multiple simultaneous point-to-point paths, so all GPUs can exchange data in parallel.
Why does NVLink 4.0 switch from 8b/10b to 128b/130b encoding?
8b/10b wastes 20% (8 data bits per 10 coded); 128b/130b wastes only ~1.5% (128 per 130). At PAM4's higher rates, the cheaper encoding recovers most of the overhead that 8b/10b would have thrown away.
How do PAM4 and 128b/130b separately contribute to NVLink 4.0's near-doubling of bandwidth?
PAM4 alone roughly doubles the rate by carrying 2 bits/symbol instead of 1; the encoding switch adds a further ~ by keeping ~98.5% of bits as data instead of 80%. The two multiply together on top of the small 25→26.56 GT/s bump.
Edge cases
What is the effective NVLink bandwidth between two GPUs that have no direct link and no free intermediate path?
Effectively zero over NVLink for that pair — traffic falls back to PCIe (through the CPU), so the "fast" number no longer applies; topology, not spec, decides.
If a GPU has 6 NVLink ports but the system only wires up 4 of them, what happens?
Only the wired ports carry traffic, so aggregate bandwidth scales down (e.g. 4 × 40 GB/s); unused ports contribute nothing — the datasheet maximum assumes all ports are populated.
What is the payload if you send an endless string of identical bits down a raw serial lane with no encoding?
The link loses clock sync (no transitions to recover from) and the receiver's data becomes garbage, so usable payload collapses toward zero — this is precisely why encoding exists.
What happens to throughput when latency doubles but the buffer stays the same size?
The bandwidth-delay product doubles, so the fixed buffer may now be smaller than the BDP; the sender stalls waiting for acks and efficiency drops below 100% even though the link speed is unchanged.
In a ring all-reduce, what is the minimum number of steps for a single GPU (N = 1)?
Zero — with one GPU there is nothing to exchange, matching the formula, which correctly degenerates to 0 rather than breaking.
At the exact moment a sender's credit count hits zero and an acknowledgement arrives simultaneously, does the link stall?
No — the returning ack immediately restores a credit, so the sender continues without an observable stall; the boundary case resolves in favour of no interruption.
If encoding overhead were 0%, would payload equal the signaling rate (in bit terms)?
Yes in that idealized case, because payload = signaling rate × bits/symbol × (data bits / coded bits); with no overhead the last ratio is 1. Real links always carry some overhead, so payload is always strictly less.
Recall One-line self-test before you close this page
Recall that a link is bidirectional (two directions at once). For NVLink 2.0 — "25 GT/s, 8b/10b, 8 lanes" — which number does a program experience in one direction only? ::: 20 GB/s payload per direction: Gb/s per lane, lanes Gb/s, GB/s. The bidirectional figure (40 GB/s) is just this doubled.