Visual walkthrough — Infinity Fabric - mesh interconnects

6.3.11 · D2 · Hardware › Interconnects, Buses & SoC › Infinity Fabric - mesh interconnects

Yeh page ek hi result ko zero se build karti hai: kyun 2D mesh of cores, ring of cores ko beat karta hai jab core counts badhte hain. Hum assume nahi karenge ki aap jaante hain "hop" kya hota hai, "average distance" ka matlab kya hai, ya $\frac{2\sqrt{N}}{3}$ number kahaan se aata hai. Har symbol earn kiya jayega. End tak aap poori story ek napkin par draw kar paoge.

Parent: Infinity Fabric & mesh interconnects (dekho Hinglish version bhi).

---

Step 1 — "Hop" kya hota hai? (poori story ka atom)

KYA. Ek chip nodes ka ek set hai. Ek node network par ek stop hai: socho "ek core plus uska chhota router". Ek router woh tiny traffic-cop circuit hai jo ek data packet receive karta hai aur use next node ki taraf push karta hai. Jab bhi koi packet ek router se guzarke next node mein jaata hai, hum one hop count karte hain.

YEH PEHLE KYUN DEFINE KAREIN. Latency — data aane se pehle ki delay — kitne routers se packet guzra is par depend karti hai. Toh agar hum hops count kar sakein, toh koi chip banane se pehle designs compare kar sakte hain. Hops ruler hain.

PICTURE. Neeche, ek packet node A se nikalta hai aur B aur C se guzarkar node D tak pahunchta hai. Teen arrows = teen hops.

---

Step 2 — Ring, aur kyun hum shorter way lete hain

KYA. $N$ nodes ko ek circle mein arrange karo. Yahaan ==$N$== simply "kitne nodes (cores) hain" hai. Har node ka apne left aur right neighbour se link hai — bas yahi pura ring hai.

SHORTER WAY KYUN. Ek real ring bidirectional hoti hai (do counter-rotating lanes). Node 0 se, node 5 ya toh 5 hops clockwise hai ya $N-5$ hops doosri taraf. Router hamesha chhota choose karta hai. Yahi woh key rule hai jo average ko roughly ring ka ek quarter banata hai, aadha nahi.

PICTURE. Red node se, green destination short way (solid) se reach hoti hai, long way (dashed) se nahi.

Exact discrete count, term by term build karke. Ek node ko source fix karo (symmetry ki wajah se har node identical hai, toh ek source sabka stand-in hai). Ab har node tak uski shortest distance list karo, apne aap tak bhi. Concreteness ke liye $N$ even lete hain:

• distance $0$: khud — 1 node
• distance $1$: ek neighbour clockwise, ek counter-clockwise — 2 nodes (yahi woh "$1,1$" hai)
• distance $2$: 2 nodes (woh "$2,2$")
• … har distance par do nodes ka yeh pattern $\frac{N}{2}-1$ distance tak continue hota hai…
• distance $\frac{N}{2}$: exactly opposite node — dono taraf se equally reachable, toh yeh 1 node hai, ek baar count kiya, do baar nahi. Yeh akela far-side node hi wajah hai ki do branches use evenly split nahi karte.

Toh source se saari shortest distances ka sum hai

$$S=\underset{\text{self}}{\underbrace{0}}+2\sum _{d=1}^{\frac{N}{2}-1}d+\underset{\text{far node}}{\underbrace{\frac{N}{2}}}=2\cdot \frac{(\frac{N}{2}-1)\frac{N}{2}}{2}+\frac{N}{2}=\frac{N^2}{4}.$$

• $2{\sum }_{d=1}^{N/2-1}d$ ::: har distance $1,2,\dots ,\frac{N}{2}-1$ par do nodes, ${\sum }_{d=1}^{m}d=\frac{m(m+1)}{2}$ use karke $m=\frac{N}{2}-1$ ke saath
• $+\frac{N}{2}$ ::: single far-side node, ek baar add kiya
• $=\frac{N^2}{4}$ ::: algebra collapse hone ke baad neat total

Ab average nikaalo. Hum saare $N$ destinations se divide karte hain (self included, hamare page rule ke mutabiq):

$$\mathbb{E}[H_{\text{ring}}]=\frac{S}{N}=\frac{N^2/4}{N}=\frac{N}{4}.$$

• $N$ se divide karna ::: kyunki fixed source ke liye $N$ possible destinations hain, aur hum "kahin nahi jaana" ($0$) ko unhe mein se ek count karte hain
• $=\frac{N}{4}$ ::: exact even-$N$ result — clean, bina kisi correction ke jab self-pair include ho

$N=16$ ke liye: $\frac{16}{4}=4$ exactly. (Pehle hand-wavy "excluding self" counts ne $4.27$ diya tha; self-pair include karke — hamara fair rule — clean $4$ milta hai.)

---

Step 3 — Mesh, aur kyun hum distance $|\Delta x|+|\Delta y|$ se measure karte hain

KYA. Ab usi $N$ nodes ko ek square grid par rakho, $k$ rows aur $k$ columns, toh $N=k^2$ aur $k=\sqrt{N}$. Yahaan ==$k$== grid ki side length hai. Har node apne North, South, East, West neighbour se link hai. (Real chips often rectangular grids $k_x\times k_y$ use karte hain — hum Step 7 mein woh cover karenge.)

YEH DISTANCE KYUN. Grid par aap diagonally move nahi kar sakte — packets sirf links ke along jaate hain. Toh column $x_1$ se $x_2$ jaane ke liye $|x_2-x_1|$ horizontal hops karne padte hain, aur row $y_1$ se $y_2$ jaane ke liye $|y_2-y_1|$ vertical hops. Bars $|\cdot |$ ka matlab hai "absolute value" — gap ki size, yeh ignore karte hue ki aap left jaate ho ya right (distance kabhi negative nahi hoti). Inhe add karne par total hops milte hain. Is grid-distance ko Manhattan distance kaha jaata hai (jaise city blocks mein walk karna).

PICTURE. $(0,0)$ se $(3,2)$ tak: teen East hops, phir do North hops.

---

Step 4 — Ek axis par average gap: exact discrete sum

KYA. Hum saare random source–destination pairs (self-pairs included, hamare page rule ke mutabiq) par typical Manhattan distance chahte hain. Symmetry se X-average aur Y-average equal hain, toh humein sirf ek axis chahiye: $\mathbb{E}[|x_2-x_1|]$ — average column-gap jab $x_1$ aur $x_2$ mein se har ek equally likely hai koi bhi position $0,1,\dots ,k-1$.

AXES MEIN KYUN SPLIT KAREIN. Kyunki $H=|\Delta x|+|\Delta y|$ ek sum hai, aur sum ka average averages ka sum hota hai, hum ek mushkil 2D average ko do aasaan 1D averages mein split kar sakte hain. Yahi pura trick hai — isiliye grid tractable hai.

PICTURE. Columns ke har ordered pair aur unke beech ka gap; hum literally saare gaps add karte hain aur divide karte hain.

Exactly $2(k-d)$ pairs mein gap $d$ kyun hota hai. Ek ordered pair hai $(x_1,x_2)$ — start column phir end column, aur $(2,5)$ ko $(5,2)$ se alag count kiya jaata hai. Ek fixed gap size $d\ge 1$ lo. Kitne ordered pairs exactly $d$ apart hain?

• Right jaane wale pairs count karo, $x_2=x_1+d$. Start $x_1$ ho sakta hai $0,1,\dots ,k-1-d$ (aur aage right gaye toh $x_2$ grid se bahar gir jaata hai). Yeh $k-d$ starting positions hain → $k-d$ pairs.
• Left jaane wale pairs count karo, $x_2=x_1-d$. Mirror argument se, aur $k-d$ pairs.
• Total: $(k-d)+(k-d)=2(k-d)$ ordered pairs gap $d$ ke saath.

Upar ki picture mein yeh do staircases (rightward aur leftward) dikhti hain, har ek length $k-d$ ki, $d$ badhne par shrink hoti hain (bade gaps rare hain — unke liye bahut kam positions available hain). $d=0$ (self-pairs) ke liye exactly $k$ hain ($x_1=x_2$), sum mein $0$ contribute karte hain lekin denominator mein phir bhi count hote hain. Ab $d\times (\text{count})$ sum karo aur saare $k^2$ ordered pairs se divide karo:

$$\mathbb{E}[|x_2-x_1|]=\frac{1}{k^2}\sum _{d=1}^{k-1}d\cdot 2(k-d)=\frac{k^2-1}{3k}$$

• ${\sum }_{d=1}^{k-1}$ ::: har possible gap size $d\ge 1$ par add karo ($d=0$ term $0$ hai, toh numerator se drop ho jaata hai lekin $k^2$ denominator mein rehta hai)
• $2(k-d)$ ::: abhi derive ki gayi count — do directions, $d$ badhne par fewer starting spots
• $\frac{k^2-1}{3k}$ ::: algebra ke baad exact answer — woh number jo truly correct hai

$k=4$ ke liye: exact $=\frac{15}{12}=1.25$, versus tidy $\frac{4}{3}=1.33$ — 6% overestimate, $k$ badhne par shrink hota hai.

---

Step 5 — Do axes add karo: mesh average hops

KYA. X-average aur Y-average combine karo.

ADDITION KYUN KAAM KARTA HAI. Sum ka average averages ke sum ke barabar hota hai — hamesha, chahe X aur Y related bhi hote (yahaan hain nahi). Toh hum simply do per-axis averages add karte hain.

PICTURE. Do per-axis averages milkar ek L-shaped path length banate hain.

$$\mathbb{E}[H_{\text{mesh}}]=\underset{\text{X hops (exact)}}{\underbrace{\frac{k^2-1}{3k}}}+\underset{\text{Y hops (exact)}}{\underbrace{\frac{k^2-1}{3k}}}=\frac{2(k^2-1)}{3k}\approx \frac{2k}{3}=\frac{2\sqrt{N}}{3}$$

• $\frac{k^2-1}{3k}$ (do baar) ::: Step 4 se exact per-axis average
• $\frac{2(k^2-1)}{3k}$ ::: unka exact sum
• $\frac{2k}{3}$ ::: clean approximation, $k=\sqrt{N}$ use karke taaki ring ke $N$ se directly compare kar sakein

---

Step 6 — $N=16$ plug in karo aur jeet dekho

KYA. Real numbers daalo. $N=16$ use karo (ek $4\times 4$ mesh vs ek 16-node ring) aur $T_{\text{hop}}=2$ cycles.

$N=16$ KYUN. Yeh sabse chhota size hai jahaan gap obvious hai aur yeh parent note se match karta hai, toh aap cross-check kar sakte ho.

PICTURE. Do bars: ring latency mesh latency ke upar towering karta hua.

$$\mathbb{E}[H_{\text{ring}}]=\frac{16}{4}=4\Rightarrow T_{\text{ring}}=4\cdot 2=8\text{ cycles}$$ $$\mathbb{E}[H_{\text{mesh}}]\approx \frac{2\sqrt{16}}{3}=\frac{2\cdot 4}{3}\approx 2.67\Rightarrow T_{\text{mesh}}\approx 2.67\cdot 2=5.3\text{ cycles}$$

(Exact discrete mesh value, $\frac{2(16-1)}{3\cdot 4}=\frac{30}{12}=2.5$, mesh ko aur bhi bada edge deta hai — approximation conservative hai.)

---

Step 7 — Edge, non-square & degenerate cases (reader ko kabhi map se giraana nahi)

KYA. Argument ke woh corners check karo jahaan neat formulas galat ho sakti hain.

KYUN. Ek formula jo sirf "on average, large square grids ke liye" kaam karta hai, usse uske extremes par test karna zaroori hai, warna aap use wahaan trust karoge jahaan woh fail karta hai.

PICTURE. Chaar tiny scenarios.

• Single node ($N=1$) — dono networks: sirf ek node hai, toh single pair hai (self, self) distance $0$ ke saath. Ring: $\frac{N}{4}=\frac{1}{4}$ tidy form se, lekin exact self-included sum hai $S=0$, average $\frac{0}{1}=0$ ✓ — formula ka tidy version yahaan distort karta hai, exact wala correct hai. Mesh: exact $\frac{2(k^2-1)}{3k}$ at $k=1$ gives $0$ ✓; tidy $\frac{2\sqrt{1}}{3}\approx 0.67$ over-state karta hai. Dono networks correctly $H=0$ dete hain ek node ke liye — koi nahi hai jisko bhejein.
• Bade network mein same node (source = dest): $H=0$. Koi hops nahi, latency bas local access hai. Yahi woh pair hai jo hum deliberately include karte hain har average mein (Step 0 rule).
• $N=4$ ($2\times 2$ mesh vs 4-ring), exact self-included forms: ring $=\frac{S}{N}$ with $S=0+1+2+1=4$ → $\frac{4}{4}=1.0$; mesh $=\frac{2(4-1)}{3\cdot 2}=1.0$. $N=4$ par tie — mesh abhi itne small size par aage nahi nikla hai, yahi woh exact reason hai ki rings low core counts par rule karte the.
• Non-square (rectangular) mesh $k_x\times k_y$: axes bas apni own lengths use karti hain — X-average $=\frac{k_x^2-1}{3k_x}$, Y-average $=\frac{k_y^2-1}{3k_y}$, add. Ek long thin grid ($k_x\ne k_y$) ka same node count ke square se bada average distance hota hai, kyunki ek axis stretched hai — toh square optimal aspect ratio hai, aur isiliye chip designers mesh layouts ko square ki taraf push karte hain.
• Neighbours (adjacent nodes): $H=1$ dono par. Best case identical hai; mesh ka advantage poori tarah far pairs ke baare mein hai.
• Bandwidth, sirf latency nahi: network ko aadha kaat do. Ring mein sirf 2 links cut cross karte hain; $\sqrt{N}\times \sqrt{N}$ mesh mein $\sqrt{N}$ hain ($N=16$ ke liye, woh 4 hain) — 2× bisection bandwidth. Toh mesh throughput aur delay dono par jeet ta hai.

---

Ek-picture summary

Upar ki sab cheez ek chart mein collapse hoti hai: ring latency ($N/4$) ek seedhi ramp par uthti hai, mesh latency ($2\sqrt{N}/3$) ek gentle curve par creep karti hai, $N\approx 7$ ke paas cross karti hai, aur phir hamesha ke liye alag ho jaati hai. Wahi gap kyun chiplet fabrics aur meshes exist karte hain.

Recall Feynman retelling — plain words mein wapas bolao

Chip ke andar data ko ek core se doosre core tak walk karna padta hai, aur har step ek router se guzarne mein time lagta hai. Hum "typical distance" measure karte hain har source-and-destination pair par average lekar — us boring pair ko bhi include karke jo kahin nahi jaata (distance $0$) — taaki dono shapes ko same footing par judge kiya ja sake. Ring par, cores ek loop par baithte hain; distances add karo (har step par do nodes, plus far side par ek akela node) toh total $N^2/4$ milta hai, aur saare $N$ destinations se divide karne par exactly $N/4$ milta hai — cores pile karo aur loop bas lamba hota jaata hai. Grid (mesh) par, aap sirf blocks ke along walk kar sakte ho, toh distance "columns apart plus rows apart" hai (Manhattan). Count karo kitne pairs gap $d$ par hain — $k-d$ right jaate hain, $k-d$ left jaate hain, toh $2(k-d)$ — aur average karne par milta hai $\frac{k^2-1}{3k}$ per axis, roughly side length ka ek third; do axes milake roughly $\frac{2}{3}\sqrt{N}$ banate hain. Kyunki $\sqrt{N}$, $N$ se bahut dheere grow karta hai, grid chips ke bare hone par latency low rakhta hai. Tiny chips ($N=4$) ke liye woh tie karte hain, aur $N=1$ par dono $0$ dete hain, isiliye rings ~10 cores tak rule karte the aur meshes/fabrics uske upar takeover karte hain. Grid ko square rakho — use lamba rectangle mein stretch karo aur average distance grow kar jaata hai. Aur grid apne middle ke across zyaada lanes bhi carry karta hai, toh woh bandwidth par bhi aur speed par bhi jeet ta hai.

Recall Related vault threads

• Coherency traffic is fabric par ride karta hai: 6.3.5-Cache-Coherency-Protocols
• Kyun distant nodes = slower memory: 6.2.8-NUMA-Architecture
• Bytes/cycle ko GB/s mein turn karna: 5.1.7-Memory-BandwidthCalculation
• Buses kahaan se aaye: 6.3.1-Bus-ArchitectureBasics
• Off-chip cousins: 6.4.3-PCIe-Topology, 7.2.4-Network-Topologies