Exercises — NVLink and GPU interconnects

WHAT we do: turn transfers-per-second into payload bytes-per-second. WHY: the wire ships symbols (transfers); only $8/10$ of each symbol is real data.

Per lane: $25\text{ GT/s}\times \frac{8}{10}=20\text{ Gb/s}$.

All 8 lanes: $8\times 20=160\text{ Gb/s}$.

Convert bits → bytes: $\frac{160}{8}=\boxed{20\text{ GB/s per direction}}$.

Full duplex = separate wires carry each direction, so both directions run at full speed simultaneously — no time-sharing.

$20\text{ GB/s}\times 2=\boxed{40\text{ GB/s per link (bidirectional)}}$.

$50\text{ GB/s}=25\text{ GB/s/dir}\times 2$, using the raw 25 GT/s line rate treated as 25 GB/s — before stripping 8b/10b overhead. Honest payload $=40\text{ GB/s}$ bidirectional (multiply the line rate by $0.8$).

Each port: $40\text{ GB/s}$ bidirectional. $6\times 40=\boxed{240\text{ GB/s aggregate}}$. This is the number to compare against a single PCIe Gen3 x16 slot ($16\text{ GB/s}$).

$\frac{240}{16}=\boxed{15\times }$.

WHY PAM4 changes the formula: each transfer now carries 2 bits (4 voltage levels), not 1. So we multiply by 2.

$\underset{\text{lanes}}{\underbrace{9}}\times \underset{\text{GT/s}}{\underbrace{26.56}}\times \underset{\text{bits/symbol}}{\underbrace{2}}\times \underset{\text{encoding}}{\underbrace{\frac{128}{130}}}÷\underset{\text{bits/byte}}{\underbrace{8}}$

$=9\times 26.56\times 2\times 0.9846÷8\approx \boxed{58.8\text{ GB/s per direction}}$.

(NVIDIA rounds/derates this to a headline of $\approx 46.5\text{–}50\text{ GB/s}$ depending on how they count; the method is what matters.)

Per step: $\frac{1.5\text{ GB}}{20\text{ GB/s}}=0.075\text{ s}=75\text{ ms}$. Seven steps: $7\times 75=\boxed{525\text{ ms}}$.

WHY 7 steps for 8 GPUs: a ring passes each chunk around the circle; after $N-1$ hand-offs every GPU has seen every contribution. More GPUs = more steps, but per-step data stays chunk-sized, so it scales gracefully.

(a) BDP = data "in flight" = bandwidth $\times$ latency: $20\text{ GB/s}\times 500\text{ ns}=20\times 10^9\times 500\times 10^{-9}=10^4\text{ B}=\boxed{10\text{ KB}}.$ Look at the figure: the credits must cover the whole in-flight region (the shaded tube) or the sender runs dry before the first ack returns.

(b) With a $6\text{ KB}$ buffer the sender exhausts credits after $t_{\text{drain}}=\frac{6\text{ KB}}{20\text{ GB/s}}=\frac{6\times 10^3}{20\times 10^9}=300\text{ ns}.$ The first ack only returns at $500\text{ ns}$, so the sender idles from $300$ to $500\text{ ns}$: $\text{stall}=500-300=\boxed{200\text{ ns}}.$ Efficiency $=\frac{\text{useful time}}{\text{cycle time}}=\frac{300}{500}=\boxed{60\%}$. Fix the design: buffer $\ge$ BDP $=10\text{ KB}$ → $t_{\text{drain}}\ge 500\text{ ns}$ → ack arrives before draining → 100% efficiency.

PCIe naive: $7\times \frac{1.5\text{ GB}}{5\text{ GB/s}}=7\times 0.30=2.1\text{ s}=\boxed{2100\text{ ms}}$. Compare to $300\text{ ms}$ compute → communication-bound by $7\times$. The GPUs sit idle most of the iteration.

NVLink: $525\text{ ms}$ comm vs $300\text{ ms}$ compute → still communication-bound, but only $1.75\times$. If compute and comm overlap (pipelined), effective iteration $\approx \max (525,300)=525\text{ ms}$ instead of $825\text{ ms}$.

Synthesis takeaway: NVLink doesn't magically make you compute-bound here — it shrinks the comm wall from $2100$ to $525\text{ ms}$ ($4\times$), turning a hopeless run into a merely-comm-limited one that overlap can hide.

Comm time $=7\times \frac{1.5}{B}$ seconds, needs $\le 0.420\text{ s}$, so $B\ge \frac{7\times 1.5}{0.420}=\frac{10.5}{0.420}=25\text{ GB/s}.$

• 1.0 ($16$): $7\times 1.5/16=656\text{ ms}$ �— too slow.
• 2.0 ($20$): $7\times 1.5/20=525\text{ ms}$ �— too slow.
• 4.0 ($58.8$): $7\times 1.5/58.8=178.6\text{ ms}$ ✅.

Required $B\ge 25\text{ GB/s}$ excludes 1.0 and 2.0. Answer: NVLink 4.0 is the slowest listed generation that meets the deadline.

Payload per direction (bytes/s): $B=L\times R\times 2\times \frac{128}{130}÷8(\text{L lanes},R\text{ in GT/s}=\text{Gsym/s}).$ Solve for the constraint $B\ge 60$: $L\times R\ge \frac{60\times 8}{2\times 0.9846}=\frac{480}{1.9692}=\mathrm{243.8.}$ So we need lanes $\times$ rate $\ge 243.8$. Pick $L=9$: need $R\ge 243.8/9=27.1\text{ GT/s}$. Choose $R=28\text{ GT/s}$ ($\le 30$ ✅). Check: $9\times 28\times 2\times 0.9846÷8=\boxed{61.9\text{ GB/s}}\ge 60$ ✅. A valid design: 9 lanes at 28 GT/s. (Many pairs work; e.g. 12 lanes at 21 GT/s gives $62.0\text{ GB/s}$.)

Need buffer $\ge$ BDP: $\text{BDP}=61.9\times 10^9\text{ B/s}\times 600\times 10^{-9}\text{ s}=61.9\times 600=37140\text{ B}\approx \boxed{37.1\text{ KB}}.$ Insight: faster + longer-latency links need bigger buffers. Doubling bandwidth or latency doubles the credits you must hold on-chip — a real silicon-area cost that bounds how fast interconnects can grow.

PCIe is a tree rooted at the CPU: every GPU-to-GPU byte must climb up to the root complex and back down, so all GPU pairs share the CPU's finite lanes and add hops of latency. Widening PCIe raises the trunk bandwidth but the trunk is still shared and CPU-serialized — 8 GPUs synchronizing gradients still collide there. NVLink instead gives each GPU dedicated point-to-point links in a mesh, so pairs communicate in parallel with no CPU in the path; aggregate bandwidth scales with the number of links, not with one shared root. Add differential signaling (noise cancels via $V_{+}-V_{-}$) to push per-lane rates far higher than a shared bus safely allows, and you get the $15\times$ jump the parent note reports.

NVLink 2.0 payload per direction is 20 GB/s. The factor that removes 8b/10b overhead is 8/10 (0.8). PAM4 multiplies payload by 2 because it carries 2 bits per symbol. A receiver buffer must be at least the bandwidth–delay product (BDP) for 100% efficiency. Ring all-reduce across $N$ GPUs takes ==$N-1$== steps.