Infinity Fabric - mesh interconnects: Visual walkthrough

This page builds ONE result from nothing: why a 2D mesh of cores beats a ring of cores as core counts grow. We will not assume you know what a "hop" is, what "average distance" means, or where the number $\frac{2 N}{3}$ comes from. Every symbol is earned. By the end you will be able to draw the whole story on a napkin.

Parent: Infinity Fabric & mesh interconnects (see the Hinglish version too).

Step 1 — What is a "hop"? (the atom of the whole story)

WHAT. A chip is a set of nodes. A node is one stop on the network: think "a core plus its little router". A router is the tiny traffic-cop circuit that receives a data packet and pushes it toward the next node. Every time a packet passes through one router and moves to the next node, we count one hop.

WHY define this first. Latency — the delay before data arrives — is dominated by how many routers the packet must pass through. So if we can count hops, we can compare designs before any chip is built. Hops are the ruler.

PICTURE. Below, a packet leaves node A and reaches node D by passing B and C. Three arrows = three hops.

Figure — Infinity Fabric - mesh interconnects

Fewer hops on average?

lower latency, because

T

is directly proportional to

H

Step 2 — The ring, and why we take the shorter way around

WHAT. Arrange $N$ nodes in a circle. Here == $N$ == is simply "how many nodes (cores) there are". Each node has a link to its left and right neighbour — that is the whole ring.

WHY the shorter way. A real ring is bidirectional (two counter-rotating lanes). From node 0, node 5 is either 5 hops clockwise or $N - 5$ hops the other way. A router always picks the smaller. This is the key rule that makes the average come out to roughly a quarter, not a half, of the ring.

PICTURE. From the red node, the green destination is reached the short way (solid) not the long way (dashed).

The exact discrete count, built term by term. Fix one node as source (every node is identical by symmetry, so one source stands for all). Now list its shortest distance to every node including itself. Take $N$ even for concreteness:

distance $0$ : itself — 1 node
distance $1$ : one neighbour clockwise, one counter-clockwise — 2 nodes (that is the " $1, 1$ ")
distance $2$ : 2 nodes (the " $2, 2$ ")
… this pattern of two nodes at each distance continues up to distance $\frac{N}{2} - 1$ …
distance $\frac{N}{2}$ : the node exactly opposite — reachable equally either way, so it is 1 node, counted once, not twice. This lone far-side node is the reason the two branches don't split it evenly.

So the sum of all shortest distances from the source is

S = self 0 + 2 d = 1 \sum \frac{N}{2} - 1 d + far node \frac{N}{2} = 2 \cdot \frac{( \frac{N}{2} - 1 ) \frac{N}{2}}{2} + \frac{N}{2} = \frac{N ^{2}}{4} .

$2 \sum_{d = 1}^{N /2 - 1} d$ ::: two nodes at each distance $1, 2, \dots, \frac{N}{2} - 1$ , using $\sum_{d = 1}^{m} d = \frac{m ( m + 1 )}{2}$ with $m = \frac{N}{2} - 1$
$+ \frac{N}{2}$ ::: the single far-side node, added once
$= \frac{N ^{2}}{4}$ ::: the neat total after the algebra collapses

Now average. We divide by all $N$ destinations (self included, per our page rule):

E [H_{ring}] = \frac{S}{N} = \frac{N ^{2} /4}{N} = \frac{N}{4} .

dividing by $N$ ::: because there are $N$ possible destinations for the fixed source, and we count "go nowhere" ( $0$ ) as one of them
$= \frac{N}{4}$ ::: the exact even- $N$ result — clean, with no correction needed once the self-pair is included

For $N = 16$ : $\frac{16}{4} = 4$ exactly. (Earlier hand-wavy "excluding self" counts gave $4.27$ ; including the self-pair — our fair rule — gives the clean $4$ .)

Step 3 — The mesh, and why we measure distance as $∣Δ x ∣ + ∣Δ y ∣$

WHAT. Now lay the same $N$ nodes on a square grid, $k$ rows by $k$ columns, so $N = k^{2}$ and $k = N$ . Here == $k$ == is the side length of the grid. Each node links to its North, South, East, West neighbour. (Real chips often use rectangular grids $k_{x} \times k_{y}$ — we cover that in Step 7.)

WHY this distance. On a grid you cannot move diagonally — packets go along links only. So to get from column $x_{1}$ to $x_{2}$ you make $∣ x_{2} - x_{1} ∣$ horizontal hops, and from row $y_{1}$ to $y_{2}$ you make $∣ y_{2} - y_{1} ∣$ vertical hops. The bars $∣ \cdot ∣$ mean "absolute value" — the size of the gap, ignoring whether you go left or right (distance is never negative). Adding them gives the total hops. This grid-distance is called Manhattan distance (like walking city blocks).

PICTURE. From $(0, 0)$ to $(3, 2)$ : three East hops, then two North hops.

Step 4 — The average gap on one axis: the exact discrete sum

WHAT. We want the typical Manhattan distance over all random source–destination pairs (self-pairs included, per our page rule). By symmetry the X-average and Y-average are equal, so we only need one axis: $E [∣ x_{2} - x_{1} ∣]$ — the average column-gap when $x_{1}$ and $x_{2}$ are each equally likely to be any position $0, 1, \dots, k - 1$ .

WHY split into axes. Because $H = ∣Δ x ∣ + ∣Δ y ∣$ is a sum, and the average of a sum is the sum of the averages, we can split a hard 2D average into two easy 1D averages. That is the whole trick — it is why the grid is tractable.

PICTURE. Every ordered pair of columns and the gap between them; we literally add up all the gaps and divide.

Why exactly $2 (k - d)$ pairs have gap $d$ . An ordered pair is $(x_{1}, x_{2})$ — start column then end column, and $(2, 5)$ counts as different from $(5, 2)$ . Fix a gap size $d \geq 1$ . How many ordered pairs sit exactly $d$ apart?

Count the pairs going right, $x_{2} = x_{1} + d$ . The start $x_{1}$ can be $0, 1, \dots, k - 1 - d$ (any further right and $x_{2}$ falls off the grid). That is $k - d$ starting positions → $k - d$ pairs.
Count the pairs going left, $x_{2} = x_{1} - d$ . By the mirror argument, another $k - d$ pairs.
Total: $(k - d) + (k - d) = 2 (k - d)$ ordered pairs with gap $d$ .

The picture above shows this as two staircases (rightward and leftward), each of length $k - d$ , shrinking as $d$ grows (big gaps are rare — only a few positions leave room for them). For $d = 0$ (self-pairs) there are exactly $k$ of them ( $x_{1} = x_{2}$ ), contributing $0$ to the sum but still counted in the denominator. Now sum $d \times (count)$ and divide by all $k^{2}$ ordered pairs:

E [∣ x_{2} - x_{1} ∣] = \frac{1}{k ^{2}} d = 1 \sum k - 1 d \cdot 2 (k - d) = \frac{k ^{2} - 1}{3 k}

$\sum_{d = 1}^{k - 1}$ ::: add over every possible gap size $d \geq 1$ (the $d = 0$ term is $0$ , so it drops out of the numerator but stays in the $k^{2}$ denominator)
$2 (k - d)$ ::: the count just derived — two directions, fewer starting spots as $d$ grows
$\frac{k ^{2} - 1}{3 k}$ ::: the exact answer after the algebra — the number that is truly correct

For $k = 4$ : exact $= \frac{15}{12} = 1.25$ , versus the tidy $\frac{4}{3} = 1.33$ — a 6% overestimate, shrinking as $k$ grows.

Step 5 — Add the two axes: mesh average hops

WHAT. Combine the X-average and Y-average.

WHY addition works. The average of a sum equals the sum of the averages — always, even if X and Y were related (they're not here). So we simply add the two per-axis averages.

PICTURE. The two per-axis averages stack into one L-shaped path length.

E [H_{mesh}] = X hops (exact) \frac{k ^{2} - 1}{3 k} + Y hops (exact) \frac{k ^{2} - 1}{3 k} = \frac{2 ( k ^{2} - 1 )}{3 k} \approx \frac{2 k}{3} = \frac{2 N}{3}

$\frac{k ^{2} - 1}{3 k}$ (twice) ::: the exact per-axis average from Step 4
$\frac{2 ( k ^{2} - 1 )}{3 k}$ ::: their exact sum
$\frac{2 k}{3}$ ::: the clean approximation, using $k = N$ so we can compare directly against the ring's $N$

Step 6 — Plug in $N = 16$ and see the win

WHAT. Put real numbers in. Use $N = 16$ (a $4 \times 4$ mesh vs a 16-node ring) and $T_{hop} = 2$ cycles.

WHY $N = 16$ . It is the smallest size where the gap is obvious and it matches the parent note, so you can cross-check.

PICTURE. Two bars: ring latency towering over mesh latency.

E [H_{ring}] = \frac{16}{4} = 4 \Rightarrow T_{ring} = 4 \cdot 2 = 8 cycles

E [H_{mesh}] \approx \frac{2 16}{3} = \frac{2 \cdot 4}{3} \approx 2.67 \Rightarrow T_{mesh} \approx 2.67 \cdot 2 = 5.3 cycles

(The exact discrete mesh value, $\frac{2 ( 16 - 1 )}{3 \cdot 4} = \frac{30}{12} = 2.5$ , gives the mesh an even bigger edge — the approximation is conservative.)

Step 7 — Edge, non-square & degenerate cases (never let the reader fall off the map)

WHAT. Check the corners of the argument where the neat formulas might lie.

WHY. A formula that only works "on average, for big square grids" must be tested at its extremes, or you will trust it where it breaks.

PICTURE. Four tiny scenarios.

Single node ( $N = 1$ ) — both networks: there is only one node, so the only pair is (self, self) with distance $0$ . Ring: $\frac{N}{4} = \frac{1}{4}$ from the tidy form, but the exact self-included sum is $S = 0$ , average $\frac{0}{1} = 0$ ✓ — the formula's tidy version distorts here, the exact one is correct. Mesh: exact $\frac{2 ( k ^{2} - 1 )}{3 k}$ at $k = 1$ gives $0$ ✓; the tidy $\frac{2 1}{3} \approx 0.67$ over-states it. Both networks correctly give $H = 0$ for one node — nobody to send to.
Same node in a big network (source = dest): $H = 0$ . No hops, latency is just the local access. This pair is the one we deliberately include in every average (Step 0 rule).
$N = 4$ ( $2 \times 2$ mesh vs 4-ring), exact self-included forms: ring $= \frac{S}{N}$ with $S = 0 + 1 + 2 + 1 = 4$ → $\frac{4}{4} = 1.0$ ; mesh $= \frac{2 ( 4 - 1 )}{3 \cdot 2} = 1.0$ . A tie at $N = 4$ — the mesh has not yet pulled ahead this small, which is exactly why rings ruled at low core counts.
Non-square (rectangular) mesh $k_{x} \times k_{y}$ : the axes just use their own lengths — X-average $= \frac{k _{x}^{2} - 1}{3 k _{x}}$ , Y-average $= \frac{k _{y}^{2} - 1}{3 k _{y}}$ , added. A long thin grid ( $k_{x} \neq = k_{y}$ ) has a larger average distance than a square of the same node count, because one axis is stretched — so square is the optimal aspect ratio, and that is why chip designers push mesh layouts toward square.
Neighbours (adjacent nodes): $H = 1$ on both. The best case is identical; the mesh's advantage is entirely about far pairs.
Bandwidth, not just latency: cut the network in half. A ring has only 2 links crossing the cut; a $N \times N$ mesh has $N$ (for $N = 16$ , that's 4) — 2× the bisection bandwidth. So the mesh wins on throughput and delay.

Solve

\frac{N}{4} = \frac{2 N}{3}

(approximate forms) for the crossover

N

N = \frac{8}{3}

, so

N \approx 7.1

— beyond this the mesh's average hops win.

The one-picture summary

Everything above collapses into one chart: ring latency ( $N /4$ ) rising as a straight ramp, mesh latency ( $2 N /3$ ) crawling along a gentle curve, crossing near $N \approx 7$ , and separating forever after. That gap is why chiplet fabrics and meshes exist.

Recall Feynman retelling — say it back in plain words

Data inside a chip has to walk from one core to another, and every step through a router costs time. We measure "typical distance" by averaging over every source-and-destination pair — including the boring pair that goes nowhere (distance $0$ ) — so both shapes are judged on the same footing. On a ring, cores sit on a loop; adding up the distances (two nodes at each step, plus one lone node on the far side) gives a total of $N^{2} /4$ , and dividing by all $N$ destinations gives exactly $N /4$ — pile on cores and the loop just gets longer. On a grid (mesh), you can only walk along blocks, so distance is "columns apart plus rows apart" (Manhattan). Counting how many pairs sit a gap $d$ apart — $k - d$ going right, $k - d$ going left, so $2 (k - d)$ — and averaging gives $\frac{k ^{2} - 1}{3 k}$ per axis, roughly a third of the side length; two axes make about $\frac{2}{3} N$ . Because $N$ grows much slower than $N$ , the grid keeps latency low as chips get huge. For tiny chips ( $N = 4$ ) they tie, and at $N = 1$ both give $0$ , which is why rings ruled until ~10 cores and meshes/fabrics took over above that. Keep the grid square — stretch it into a long rectangle and the average distance grows. And the grid carries more lanes across its middle too, so it wins on bandwidth as well as speed.

Recall Related vault threads

Coherency traffic riding this fabric: 6.3.5-Cache-Coherency-Protocols
Why distant nodes = slower memory: 6.2.8-NUMA-Architecture
Turning bytes/cycle into GB/s: 5.1.7-Memory-BandwidthCalculation
Where buses came from: 6.3.1-Bus-ArchitectureBasics
Off-chip cousins: 6.4.3-PCIe-Topology, 7.2.4-Network-Topologies