Visual walkthrough — NVLink and GPU interconnects

Before we can even state the target number we need one unit. Bandwidth is measured in bytes per second, and a byte is 8 bits:

With that unit in hand, here is the one target number we are chasing:

$$\underset{\text{the answer}}{\underbrace{20}}\text{GB/s per direction (NVLink 2.0)}$$

Every step below adds exactly one idea until this number falls out.

---

Step 1 — One wire, sending one bit

WHAT. The most basic thing a link does: hold a wire at a high voltage to mean "1", a low voltage to mean "0", then switch. Each switch is a chance to send a new symbol.

WHY start here. Every giant bandwidth number is really just "how many times per second can we change the voltage" multiplied by "how much meaning each change carries." We must earn both halves. This step earns the first half.

PICTURE. Below, the wire's voltage rises and falls over time. Each shaded slot is one symbol period — the time the wire holds still before the next change.

For NVLink 1.0 this rate is $20\text{GT/s}$; for NVLink 2.0/3.0 it is $25\text{GT/s}$. That's the fastest the transmitter can flip the wire cleanly.

---

Step 2 — Two wires, because one wire lies

WHAT. Instead of one wire measured against ground, NVLink uses two wires and reads the difference between them.

WHY this tool and not a single wire? At 25 billion flips per second, noise is enormous — nearby switching currents shake the ground reference so the receiver can't trust "is this high or low?". The trick: if noise hits both wires equally, then subtracting one from the other deletes the noise. We use subtraction specifically because it is the one operation that kills anything common to both inputs.

PICTURE. Top: two wires, one the mirror of the other. Middle: the same noise bump ($n$) lands on both. Bottom: their difference — clean, noise gone.

---

Step 3 — Each symbol isn't a full bit of your data

WHAT. Here is the catch. The raw symbols from Step 1 are not all your payload. NVLink 2.0 uses 8b/10b encoding: for every 8 bits of your data, it actually transmits 10 symbols on the wire.

WHY spend 10 to send 8? Two survival needs:

1. Clock recovery — the receiver has no separate clock wire; it recovers timing from the edges (transitions) in the data. A long run of all-1s has no edges → the receiver drifts and loses track. 8b/10b guarantees frequent transitions.
2. DC balance — keeps the average voltage centred so the electronics don't slowly saturate.

The price is a fixed tax: only $\frac{8}{10}$ of the wire's symbols carry your data.

PICTURE. A bar of 10 wire-symbols; 8 are "your data" (payload), 2 are "encoding tax". The efficiency fraction $\frac{8}{10}=0.8$ is highlighted.

---

Step 4 — Stack 8 lanes into one link

WHAT. One lane gives 20 Gb/s. NVLink 2.0 bundles 8 lanes in each direction to form one link. The lanes run side by side, in parallel, splitting the data between them.

WHY 8 in parallel? You cannot make a single wire flip faster than the electronics allow (Step 1's ceiling). To go faster you go wider: run many identical lanes at once and add their rates. This is the same idea as adding lanes to a motorway rather than raising the speed limit.

PICTURE. 8 lane-arrows side by side, each labelled 20 Gb/s, summing to a fat bundle labelled 160 Gb/s.

---

Step 5 — Both directions at once (full duplex)

WHAT. A link has a separate set of 8 lanes for each direction, so data flows both ways simultaneously. Add them for the total per-link figure.

WHY separate lanes each way? Training workloads send and receive gradients at the same time. Dedicated lanes per direction mean neither direction waits for the other — no turn-taking.

PICTURE. Two 8-lane bundles pointing opposite ways between two GPUs, each 20 GB/s, total 40 GB/s.

---

Step 6 — One GPU, six links

WHAT. A single V100 doesn't have one link — it has 6 NVLink ports. Sum them for the whole GPU's aggregate.

WHY six? To wire up an 8-GPU mesh where each GPU reaches several neighbours directly. More links = more direct neighbours = fewer hops during an all-reduce.

PICTURE. A central GPU with 6 bidirectional links fanning out, each labelled 40 GB/s, totalling 240 GB/s.

---

Step 7 — The edge cases (where the naive number lies)

WHAT. The 240 GB/s is an aggregate ceiling. Real transfers rarely see it. Three degenerate situations to know:

WHY cover these. If you only remember "240 GB/s," you'll wildly over-predict performance. Each case below caps the number for a real reason.

PICTURE. Three mini-scenarios: (a) two GPUs sharing one link cap at 40 GB/s; (b) all-reduce splitting traffic across the mesh; (c) a small buffer that stalls before the ack returns.

• Case A — a single pair. GPU0→GPU4 over one link tops out at 20 GB/s per direction, not 240. The aggregate only appears when all 6 links are busy at once.
• Case B — indirect pairs. If two GPUs aren't directly linked, traffic takes multiple hops, sharing links with other flows. Effective bandwidth drops below the single-link figure.
• Case C — buffer too small (flow-control stall). The sender may only transmit as much data as its credits allow before an acknowledgement returns. If the buffer is smaller than the data "in flight," it empties and stalls.

---

The one-picture summary

Everything on one canvas: raw symbols → subtract noise → pay the encoding tax → widen to 8 lanes → double for duplex → six links. Follow the arrows and the target number assembles itself.

$$\underset{\text{Step 1}}{\underbrace{25\text{GT/s}}}\times \underset{\text{Step 3}}{\underbrace{\frac{8}{10}}}\times \underset{\text{Step 4 lanes}}{\underbrace{8}}÷\underset{\text{bits→bytes}}{\underbrace{8}}=20\text{GB/s}\stackrel{\times 2}{\to }\underset{\text{duplex}}{\underbrace{40}}\stackrel{\times 6}{\to }\underset{\text{per GPU}}{\underbrace{240\text{GB/s}}}$$

Recall Feynman retelling — tell it back in plain words

Imagine a wire you can flick up and down 25 billion times a second — that's your raw speed, measured in "transfers." But one wire lies, because noise shakes it; so you use two wires and read the difference, and the noise (which hits both equally) subtracts away to nothing. Now, of every 10 flicks you can't use all of them for real data — 2 are spent keeping the receiver's clock in sync and the voltage balanced, so only 8-of-10 flicks are yours. That leaves 20 billion real bits per second on one lane. You can't flick faster, so you run 8 lanes side by side and add them: 160 billion bits = 20 gigabytes per second, one direction (remember: 8 bits make a byte). Give each direction its own lanes and you get 40 GB/s per link. Put 6 links on a GPU and it can gossip with its neighbours at 240 GB/s — about 15 times more than the old PCIe road. Just remember: that 240 is the "everything at once" ceiling; any single pair only sees 20–40, and if your buffer is smaller than the data in flight (about 10 KB for a 500 ns round trip), the link stalls waiting for the "got it" reply.