Worked examples — Infinity Fabric - mesh interconnects

6.3.11 · D3 · Hardware › Interconnects, Buses & SoC › Infinity Fabric - mesh interconnects

Yeh parent topic on Infinity Fabric & mesh interconnects ka "worked examples" deep-dive hai. Parent note ne tumhe kya hai fabric yeh bataya aur do starter examples diye. Yahan hum har case class dhundte hain — har woh shape ka question jo exam ya real design review phenk sakta hai — aur har ek ko zero se solve karte hain.

Koi bhi number aane se pehle, teen quantities yaad rakho jo hum baar baar use karte hain, plain words mein:

Do clock names baar baar neeche aate hain; inhe abhi hi jaano taaki koi bhi symbol kabhi surprise na ho:

Bandwidth examples mein baar baar use hone wala ek aur term:

---

The scenario matrix

Is topic ke baare mein har question in cells mein se kisi ek mein aata hai. Hamara kaam hai ki koi bhi cell chhutti na rahe.

Cell	Kya cheez ise alag banati hai	Covered by
A. Bandwidth — one direction	frequency × width, single lane	Ex 1
B. Bandwidth — bidirectional / ratio	double karo, phir FCLK:MCLK ratio check karo	Ex 2
C. Routing — generic hop count	Manhattan distance on a mesh	Ex 3
D. Routing — degenerate (same row / same node)	ek axis mein zero, ya zero total	Ex 4
E. Ring vs mesh — the crossover	mesh actually kab jeet ta hai? difference ka sign	Ex 5
F. Bisection bandwidth — the "cut"	ek slice cross karne wale links gino	Ex 6
G. Chiplet latency — the multi-hop word problem	ns aur cycles ko clock domains ke paar mix karo	Ex 7
H. Coherency traffic — O(N) vs O(k)	probes ka limiting/scaling behaviour	Ex 8
I. Exam twist — mismatched ratio bottleneck	fabric memory se slower: kaun jeet ta hai?	Ex 9

Hum A→I order mein chalenge.

---

Cell A — Bandwidth, one direction

Forecast: pehle order of magnitude guess karo — tens of GB/s? Hundreds? Apna guess likh lo.

1. Frequency ko ticks per second mein convert karo. $1.8\text{ GHz}=1.8\times 10^9$ cycles/s. Yeh step kyun? Bandwidth bytes per second hai, isliye denominator mein seconds chahiye. GHz already "per second" deta hai.
2. Width ko ticks se multiply karo. Har tick $32$ bytes move karta hai, isliye $B{W}_{\text{1-dir}}=32\frac{\text{B}}{\text{cycle}}\times 1.8\times 10^9\frac{\text{cycle}}{\text{s}}=57.6\times 10^9\frac{\text{B}}{\text{s}}.$ Yeh step kyun? Units cancel ho jaate hain: cycle over cycle disappear ho jaata hai, bytes/second bachta hai — exactly yahi "bandwidth" ka matlab hai.
3. Naam do. $57.6\times 10^9\text{ B/s}=57.6\text{ GB/s}$. Yeh step kyun? $10^9\text{ B/s}$ by definition ek gigabyte per second (GB/s) hai, isliye number ko us unit mein likhte hain jo har jagah datasheets mein quote hoti hai — same value, human-readable naam.

Verify: Units: $\frac{\text{B}}{\text{cycle}}\cdot \frac{\text{cycle}}{\text{s}}=\frac{\text{B}}{\text{s}}$ ✓. Sanity: 32 bytes ek 64-byte cache line ka aadha hai, isliye almost 2 billion half-lines per second move ho rahi hain — comfortably "tens of GB/s" range mein.

---

Cell B — Bidirectional & the clock ratio

Forecast: kya bidirectional simply double ho jaata hai? Aur kya fabric is memory ke against $1:1$ ya $1:2$ geared hai?

1. Double karo. Ek link ke har direction ke liye alag wire hoti hai, isliye $B{W}_{\text{bi}}=2\times 57.6=115.2\text{ GB/s}.$ Yeh step kyun? Send aur receive ek saath independent lanes par chalte hain — share nahi karte, isliye hum unhe add karte hain.
2. DDR label se MCLK nikalo. "DDR4-3600" matlab $3600\text{ MT/s}$ (mega-transfers/s). DDR double data rate hai — har clock mein do transfers — isliye memory clock hai $\text{MCLK}=\frac{3600}{2}=1800\text{ MHz}=1.8\text{ GHz}$. Yeh step kyun? Ratio clock se clock compare karta hai, isliye actual MCLK recover karne ke liye transfer rate se "double" nikaalna padta hai.
3. True ratio batao. $\text{FCLK}=1.8\text{ GHz}$ aur $\text{MCLK}=1.8\text{ GHz}$, isliye gearing $1:1$ hai — $2:1$ nahi. Dono lockstep mein tick karte hain. Yeh step kyun? Ratio ko correctly naam dena Cell B ka poora point hai; ek $1:1$ fabric har memory tick mein ek packet move karta hai, jo AMD ka ideal tuning target hai.
4. Memory bandwidth compute karo (ek direction). Is platform mein do channels hain (dual-channel — upar Memory channel definition dekho), har ek $8\text{ B}$ wide, $3.6\times 10^9$ transfers/s par: $B{W}_{\text{mem}}=2\text{ch}\times 3.6\times 10^9\times 8=57.6\text{ GB/s (one direction)}.$ Yeh step kyun? Fabric ko memory se same direction mein compare karna zaroori hai taaki dekh sakein ki fabric ek read stream ke saath chal sakta hai ya nahi — aur channel count platform se padha jaata hai, assume nahi kiya jaata.

Verify: $2\times 57.6=115.2$ ✓; $\frac{3600}{2}=1800\text{ MHz}$ isliye ratio $=1:1$ ✓. Fabric-per-direction ($57.6$) $=$ memory-per-direction ($57.6$), isliye ek read burst fabric mein bina pile up kiye drain ho jaata hai. Cell I neeche dikhata hai kya hota hai jab yeh ratio toot ta hai.

---

Cell C — Generic routing hop count

Forecast: kya yeh straight-line distance hai, ya grid-walk distance?

Figure 1 — XY routing on a 4×4 mesh. Alt: 16 dots ki ek grid; ek teal source dot (1,0) par aur ek plum destination (3,3) par, ek orange arrowed path se jude hue jo pehle 2 steps east phir 3 steps north jaata hai, exactly ek baar turn karta hai. Yeh picture yeh point clear karta hai ki packet grid par chalta hai, kabhi diagonally nahi.

1. Woh distance define karo jo chahiye. Grid par tum diagonally nahi ud sakte; links ke along chalna padta hai. Sahi measure hai Manhattan distance:
2. Pehle X ("XY routing" ka "X"). $x=1$ se $x=3$ tak: $|3-1|=2$ hops east. X pehle kyun? Dimension-ordered routing ek order fix karta hai (pehle X, phir Y) taaki packets ek doosre ko loop mein na chase karein — yeh deadlock prevent karta hai (parent note, virtual channels).
3. Phir Y. $y=0$ se $y=3$ tak: $|3-0|=3$ hops north.
4. Add karo. Total $=2+3=5$ hops. Path: $(1,0)\to (2,0)\to (3,0)\to (3,1)\to (3,2)\to (3,3)$ — Figure 1 mein orange path dekho: yeh exactly ek baar turn karta hai.

Verify: Manhattan $=|3-1|+|3-0|=2+3=5$ ✓. Ek single turn XY routing confirm karta hai (X leg, phir Y leg).

---

Cell D — Degenerate routing (zeros)

Degenerate cases se pehle, average-hop formula ko scratch se banate hain taaki yeh page apne aap par khada ho sake.

Socho ki tum $\sqrt{N}$ dots lambi ek line ke ek axis par randomly do nodes pick kar rahe ho. Average par dono picks lagbhag ek tehayi dur land karti hain — expected gap $\frac{\sqrt{N}}{3}$ hai. (Intuition: agar tum $L$ length ki ek stick par do points uniformly drop karo, unke beech average distance $\frac{L}{3}$ hai — points zyada baar opposite ends ki bajaye ek doosre ke paas hote hain.) Ek mesh mein ek X axis aur ek Y axis hai, aur do gaps simply add ho jaate hain:

$$\text{avg hops}=\underset{\text{X gap}}{\underbrace{\frac{\sqrt{N}}{3}}}+\underset{\text{Y gap}}{\underbrace{\frac{\sqrt{N}}{3}}}=\frac{2\sqrt{N}}{3}.$$

Yaad rakho: yeh sabhi pairs par average hai, kisi ek specific route ki distance nahi. Yahi distinction exactly woh hai jo neeche degenerate cases expose karte hain.

Forecast: "average hops $=\frac{2\sqrt{N}}{3}$" yahan kya deta hai, aur kya yeh galat hai?

1. Case (a): same X column. $|2-2|+|3-1|=0+2=2$ hops (purely Y). Ise kyun point out karte hain? Averaged formula $\frac{2\sqrt{N}}{3}\approx 2.67$ predict karega $\approx 2.67$ — lekin yeh sabhi pairs ka average hai, is specific pair ka nahi. Ek aligned pair ke liye X term exactly zero hai. Average ko kabhi ek single deterministic route par apply mat karo.
2. Case (b): source = destination. $|2-2|+|2-2|=0$ hops. Message kabhi local port nahi chhodta. Yeh kyun matter karta hai: zero hops matlab zero router-to-router latency; sirf local agent handoff ka cost hai. Degenerate lekin real (ek core apna L2 slice hit kar raha hai).

Verify: (a) $0+2=2$ ✓; (b) $0+0=0$ ✓. Dono maximum possible hop count $2\times (4-1)=6$ se $\le$ hain, aur $\ge 0$ — valid bounds ke andar.

---

Cell E — Ring vs mesh crossover (win ka sign)

Pehle, do symbols jinpar yeh example jeeta hai — inhe clearly samjhe bina aage mat padho:

Forecast: tiny $N$ (maan lo 4 cores) par kaun lower hai? Solve karne se pehle guess karo.

Figure 2 — hop-count crossover. Alt: warm-paper plot par do curves; ek teal straight line N/4 (ring) aur ek orange curve 2·sqrt(N)/3 (mesh) jo N=7.1 ke paas cross karti hain, plum dashed line se mark ki gayi; crossing ke left mein ring lower hai, right mein mesh lower hai. Yeh picture exactly woh core count dikhata hai jahan mesh jeetna shuru karta hai.

1. Inequality set up karo. Mesh jeet ta hai jab uska hop count chhota ho: $\frac{2\sqrt{N}}{3}<\frac{N}{4}.$ Yeh step kyun? $T_{\text{hop}}$ dono ke liye same hai, isliye cancel ho jaata hai — winner purely hop count par depend karta hai.
2. $N$ ke liye solve karo. Dono sides ko 12 se multiply karo: $8\sqrt{N}<3N$, yani $8<3\sqrt{N}$, isliye $\sqrt{N}>\frac{8}{3}\approx 2.67$, deta hai $N>{\left(\frac{8}{3}\right)}^2=\frac{64}{9}\approx 7.1$. Numbers plug in kyun nahi karte? Hum crossover chahte hain, ek data point nahi — inequality solve karne se boundary directly milti hai.
3. Real mesh tak round karo. Sabse chhota perfect square jahan $N>7.1$ hai woh $N=9$ ($3\times 3$) hai.
4. Boundary par sign check karo. $N=4$ par: ring $=1.0$, mesh $\approx 1.33$ hops → ring jeet ta hai (mesh tiny core counts par haarta hai!). $N=16$ par: ring $=4$, mesh $\approx 2.67$ → mesh jeet ta hai. $(\frac{N}{4}-\frac{2\sqrt{N}}{3})$ ka sign unke beech flip karta hai.

Verify: $N=9$: mesh $\frac{2\cdot 3}{3}=2.0$, ring $\frac{9}{4}=2.25$ → mesh smaller ✓. $N=4$: mesh $1.333>$ ring $1.0$ ✓ (ring jeet ta hai, parent ke "~10–12 core" real-world crossover se match karta hai jab router overhead add kiya jaaye). General topology trade-off ke liye 7.2.4-Network-Topologies bhi dekho.

---

Cell F — Bisection bandwidth (the cut)

Forecast: kaun si topology ke paas beech span karne wale zyada wires hain?

Figure 3 — the bisecting cut. Alt: do side-by-side diagrams; left mein, ek 16-node ring jisme ek teal dashed horizontal cut exactly 2 links kaat ti hai; right mein, ek 4×4 mesh jisme ek teal dashed vertical cut 4 links kaat ta hai (ek column). Mesh ke paas clearly do gune zyada links crossing hain. Isliye mesh bisection bandwidth $\sqrt{N}$ ke saath scale karta hai.

1. Bisection bandwidth define karo. Network ko do equal halves mein kaato; bisection bandwidth us cut dwara severed har link ki total bandwidth hai. Yeh metric kyun? Agar dono halves poori takat se ek doosre se baat karein, toh yeh unke beech traffic ki ceiling hai — real "kitna flow ho sakta hai" number.
2. Ring cut. Ring ek single loop hai; use aadha karne se exactly 2 links cut hote hain (do arcs). Figure 3 mein teal dashed cut dekho.
3. Mesh cut. Ek $4\times 4$ mesh ko beech mein cut karne se ek column of vertical links cut hote hain: 4 links ($\sqrt{N}=4$). $\sqrt{N}$ kyun? Cut grid ki ek puri row/column ke across chalta hai, aur ek $\sqrt{N}\times \sqrt{N}$ grid mein $\sqrt{N}$ links hain jo ise span karte hain.
4. Bandwidth tak. Ring $=2\times 115.2=230.4\text{ GB/s}$; mesh $=4\times 115.2=460.8\text{ GB/s}$ → mesh ke paas $2\times$ bisection bandwidth hai.

Verify: ratio $\frac{4}{2}=2$ ✓; $4\times 115.2=460.8$ ✓; $2\times 115.2=230.4$ ✓. Isliye mesh bandwidth ko $\sqrt{N}$ ke saath scale karta hai jabki ring flat rehta hai.

---

Cell G — Chiplet latency word problem (mixed units)

Forecast: kaun sa leg dominate karta hai — on-chip cycles ya die-crossing ns?

1. Cycles ko ns mein convert karo. $3.5\text{ GHz}$ par ek cycle $\frac{1}{3.5\times 10^9}\text{ s}=0.2857\text{ ns}$ lasti hai. Isliye $5$ cycles $=5\times 0.2857=1.43\text{ ns}$. Pehle convert kyun? Tum cycles aur nanoseconds directly add nahi kar sakte — woh alag clock domains mein rehte hain. Sabko ek unit (ns) mein lao.
2. Die crossings add karo. $20+20=40\text{ ns}$ do fabric hops ke liye. Sirf add kyun karte hain? Ek single serial path ke along latencies accumulate hoti hain — packet ek leg karta hai, phir doosra, isliye total time sum hai.
3. Total. $1.43+40=41.43\text{ ns}$. Yeh answer kyun hai? Ab sab kuch nanoseconds mein hai, isliye teen serial legs end-to-end latency tak add ho jaate hain: $41.43\text{ns}$.

Verify: $\frac{5}{3.5}=1.4286\text{ ns}$ ✓; total $=1.4286+40\approx 41.43\text{ ns}$ ✓. Die-crossing ke $40\text{ ns}$ on-chip ke $1.4\text{ ns}$ ko dwarf karte hain — yahi "chiplet penalty" hai jo parent mention karta hai, aur isliye 6.2.8-NUMA-Architecture matter karta hai: distant memory zyada door feel karta hai.

---

Cell H — Coherency traffic scaling (limiting behaviour)

Forecast: kya directory traffic total cores ke saath badhta hai ya sirf sharers ke saath?

1. Broadcast: sab ko probe karo. Har write sabhi doosre cores ko probe bhejta hai: $N-1$ messages. $N=8$ par: $7$; $N=64$ par: $63$. $N-1$ kyun? Broadcast ko nahi pata kisne line rakhi hai, isliye writer ke alawa sab cores se poochna padta hai — yahi $O(N)$ scaling hai jiske baare mein parent warn karta hai.
2. Directory: sirf sharers ko probe karo. Home L3 slice ko sharer list pata hai, isliye yeh $k$ probes bhejta hai $N$ se independent. $N=8$ par: $3$; $N=64$ par: $3$. Constant kyun? Directory sharers ko explicitly track karta hai, isliye traffic $O(k)$ hai — chip ke badhne par flat. Isliye mesh CPUs directories use karte hain (dekho 6.3.5-Cache-Coherency-Protocols).
3. $N=64$ par ratio. Broadcast/directory $=\frac{63}{3}=21\times$ — directory ke saath $21$ gune kam messages. Ratio kyun lete hain? Yeh saving ko size-free way mein express karta hai — "21 times less traffic" woh headline hai jo ek designer care karta hai, units se independent.

Verify: $N-1$: $7$ aur $63$ ✓; directory: $3$ aur $3$ ✓; ratio $\frac{63}{3}=21$ ✓. Broadcast linearly blow up karta hai; directory flat rehta hai — woh limiting behaviour jo 64-core fabrics ko feasible banata hai.

---

Cell I — Exam twist: broken ratio bottleneck

Forecast: faster memory ke saath, kya tumhe faster real bandwidth milti hai — ya koi cheez upstream cap karti hai?

1. Fabric one-direction bandwidth. $32\times 2.0\times 10^9=64\text{ GB/s}$. Ek direction kyun? Read stream memory→core ek single direction mein flow karta hai; like se like compare karo.
2. Memory one-direction bandwidth. Given hai $96\text{ GB/s}$ (do channels, $8\text{ B}$ har ek, $6000\text{ MT/s}$).
3. Bottleneck minimum hai. Data dono se guzarna chahiye; slower stage throughput cap karta hai: $B{W}_{\text{eff}}=\min (64,96)=64\text{ GB/s}.$ Min kyun, average kyun nahi? Ek pipeline apni slowest stage ki speed par chalta hai — fabric ek hard wall hai jisse memory bytes squeeze hokar guzarni padti hain.
4. Lesson. Lagging FCLK ke saath faster RAM $96-64=32\text{ GB/s}$ memory bandwidth waste karta hai. Fix: FCLK badhao (ya $1:2$ penalty accept karo). 6.4.3-PCIe-Topology se compare karo jahan ek slow root link similarly device throughput cap karta hai.

Verify: fabric $32\times 2.0=64$ ✓; $\min (64,96)=64$ ✓; wasted $96-64=32$ ✓. Exam trap: bada memory number matlab nahi bada real bandwidth.

---

Recall Quick self-test

Ek $4\times 4$ mesh, $(0,1)\to (2,1)$, XY routing — kitne hops? ::: $|2-0|+|1-1|=2$ hops (Cell D, X-only) Fabric $32\text{ B}$ at $1.8\text{ GHz}$, bidirectional? ::: $2\times 32\times 1.8=115.2\text{ GB/s}$ (Cell B) 64-core chip par $k=3$ sharers ke liye directory probes? ::: $3$, N se independent (Cell H) Ring vs mesh at $N=4$ — hops par kaun jeet ta hai? ::: Ring ($1.0<1.33$) — mesh tiny N par haarta hai (Cell E) Fabric $64\text{ GB/s}$, memory $96\text{ GB/s}$ — effective? ::: $\min =64\text{ GB/s}$ (Cell I)