Infinity Fabric - mesh interconnects

6.3.11 · Hardware › Interconnects, Buses & SoC

Overview

Modern multi-core processors mein cores, caches, memory controllers, aur I/O ko ek doosre se communicate karne ka tarika chahiye. Infinity Fabric (AMD ki implementation) aur mesh interconnects (Intel ki) scalable, distributed networks-on-chip hain jo purani bus aur ring topologies ko replace karte hain jab core count ~10-12 cores se zyada ho jaata hai.

Fundamental problem: Ek shared bus saturate ho jaati hai (saare cores ek wire ke liye compete karte hain). Ring mein latency badh jaati hai (message hop-by-hop travel karta hai). Ek mesh ya fabric har node ko multiple paths deta hai kisi bhi doosre node tak pahunchne ke liye, latency aur bandwidth ko balance karte hue.

Core Concepts

Infinity Fabric kya hai?

Key properties:

• Cache coherent: distributed, scalable coherency protocol use karta hai (loosely MOESI par based, lekin custom "Scalable Coherent Protocol")
• Point-to-point links: koi shared medium nahi; har connection dedicated hai
• Packet-switched: data flits (flow-control units) ke roop mein travel karta hai
• Topology-agnostic: design ke hisaab se rings, meshes, ya hybrid topologies bana sakta hai

Mesh Interconnect kya hai?

Intel ka mesh (Xeon Scalable, Core X): Skylake-X mein ring bus replace karne ke liye introduce kiya gaya. Har router mein hota hai:

• 5 ports: 4 cardinal directions + 1 local agent (core/cache)
• Buffers aur arbitration logic
• Routing tables (usually dimension-ordered: pehle X, phir Y)

Rings aur Buses Ko Inhone Kyun Replace Kiya

Ring Bus ki Limit

Intel ka ring (Nehalem se Broadwell tak): cores ek ring mein arranged, har stop ek core + L3 slice hai. Bandwidth ke liye do counter-rotating rings.

Problem:

• Latency core count ke saath scale karti hai: Core 0 se core 15 tak 16-core ring par ek message ~8 stops hop karta hai (average hops = N/4).
• Bandwidth saturation: Ring bandwidth shared hai. 10+ cores ke memory requests issue karne par ring bottleneck ban jaati hai.

Breaking point: ~10-12 cores. Iske baad, ring latency aur contention performance degrade kar dete hain.

Shared Bus ki Limit

Classic front-side bus (FSB): saare cores memory/chipset ke liye ek bus share karte hain.

Problem:

• Electrical loading: har added core capacitance add karta hai, clock speed limit ho jaati hai.
• Arbitration overhead: ek waqt mein sirf ek transaction.
• Bandwidth wall: FSB ke liye max ~10 GB/s.

Solution: Point-to-point. Har link independent hai, full speed chalti hai, aur multiple transactions simultaneously hoti hain.

Infinity Fabric Kaise Kaam Karta Hai

Architecture

Components:

1. Coherent Master (CM): CPU core ka L1/L2 interface. Read/write requests issue karta hai.
2. Coherent Slave (CS): L3 cache slice ya memory controller. Requests ka response deta hai.
3. I/O Master (IOM): PCIe/SATA/USB controllers. Non-coherent traffic.
4. Router/Switch: packet router with multiple virtual channels (VCs) deadlock prevent karne ke liye.

Packet structure:

• Header: source ID, dest ID, command (read/write/probe/snoop), address
• Payload: cache line (64 B) ya chhota
• Credit-based flow control: sender receiver ke buffer space ko track karta hai

Topology Examples

Zen 2 (Ryzen 3000, EPYC Rome):

• Har chiplet (8 cores, 32 MB L3) ka internal Infinity Fabric cores ko L3 slices se connect karta hai.
• Central I/O die mein memory controllers aur inter-chiplet fabric hoti hai.
• Chiplets I/O die se Infinity Fabric IF/xGMI links (32 B/cycle/direction at ~1.8 GHz FCLK = ~115 GB/s per link) ke zariye connect hote hain.

MI300X (CDNA 3):

• 8 GPU chiplets + 4 I/O dies.
• Full mesh: har die multiple neighbors se connect hoti hai.
• 144 GB/s per xGMI link (900 GB/s aggregate per GPU die).

Ryzen 7000 (Zen 4):

• Desktop ke liye monolithic die (1 CD), isliye internal fabric simpler hai.
• FCLK (Fabric Clock) memory clock (MCLK) ke saath 1:1 ya 1:2 ratio par chalta hai.

Coherency Protocol

Challenge: 8 chiplets par 64 cores ke saath, L1/L2/L3 caches ko coherent kaise rakho?

Solution: Scalable Coherent Protocol (details proprietary hain, lekin principles):

1. Directory-based: Har L3 slice track karti hai ki kin cores ke paas ek line ki copy hai (sharer list).
2. Snoops targeted hote hain: Sirf sharers ko invalidation probes milte hain, saare cores ko nahi (broadcast snooping ke unlike).
3. Home node: Woh L3 slice jiske address range mein ek cache line aati hai woh "home" hai. Saare requests home se route hote hain.
4. Three-hop protocol (typical):
   • Core A line X request karta hai → Home L3 slice
   • Home directory check karta hai, zaroorat padne par sharers ko probes bhejta hai
   • Sharers respond karte hain → Home Core A ko reply karta hai

Directory kyun scale karta hai: Broadcast snooping har request har core ko bhejta hai (O(N) traffic). Directory sirf sharers ko probes bhejta hai (O(k), k = sharers, typically 1-3).

Derivation: Mesh Latency vs Ring

Ring Latency

Maano N cores ek ring mein hain, uniform traffic.

Average hops: Random source-destination pair ke liye, expected hops = N/4 (ring ka quarter on average, shorter direction use karke).

Latency per hop: ~1-2 cycles (router se pass-through).

N = 16 ke liye, $T_{\text{hop}}=2$ cycles:

Mesh Latency

2D mesh: $\sqrt{N}\times \sqrt{N}$ grid.

Manhattan distance: Random (x₁, y₁) se (x₂, y₂) ke liye:

$k\times k$ grid par uniform distribution ke liye:

N = 16 (4×4 mesh) ke liye:

$T_{\text{hop}}=2$ cycles ke saath:

Speedup: $\frac{8}{5.3}\approx 1.5\times$ kam latency.

Bandwidth: Mesh mein $2N$ links hain (N nodes mein se har ek ke 4 links, shared edges ek baar count karke → $4N/2=2N$). Ring mein $N$ links hain. Bisection bandwidth (network ko aadha kaat do):

• Ring: 2 links cut cross karte hain (do ring segments).
• Mesh: $\sqrt{N}$ links cut cross karte hain (vertical links ki ek row).

N = 16 ke liye:

• Ring bisection: 2 links
• Mesh bisection: 4 links → 2× zyada.

Infinity Fabric ki Latency Competitive Kyun Hai

EPYC Rome: 8-chiplet design, har chiplet mein 8 cores local ring/mesh mein hain, phir chiplets ke beech Infinity Fabric hops.

Worst case: Chiplet 0 par core → Chiplet 7 par L3 slice.

• Intra-chiplet: ~5 cycles
• Chiplet-to-I/O die: ~20 ns (35 cycles at 1.8 GHz FCLK)
• I/O-to-chiplet: ~20 ns
• Total: ~75 cycles (~40 ns at 3.5 GHz core clock)

Monolithic die se compare karein: Ek 64-core monolithic 8×8 mesh mein ~5.3 avg hops × 2 cycles = 10.6 cycles intra-die hote, lekin memory access phir bhi ~100 cycles add karta. Chiplet penalty (~30 cycles extra) yield/cost benefits ke saath acceptable hai.

Worked Examples

Solution:

• FCLK = 1.8 GHz → 1.8 × 10⁹ cycles/s
• Data width = 32 B per direction per cycle
• Bandwidth (ek direction) = 1.8 × 32 = 57.6 GB/s
• Bidirectional = 2 × 57.6 = 115.2 GB/s

Yeh step kyun? Har cycle mein fabric 32 bytes transfer karta hai. Bytes/second paane ke liye frequency se multiply karo.

Reality check: DDR4-3600 dual-channel = 2 × 3.6 GHz × 8 B = 57.6 GB/s. Fabric bandwidth memory bandwidth se match karti hai (1:1 ratio koi bottleneck nahi rehne deta).

Solution:

• Dimension-ordered routing (XY): pehle X mein move karo, phir Y.
• X distance: 3 - 0 = 3 hops (east)
• Y distance: 2 - 0 = 2 hops (north)
• Total hops = 3 + 2 = 5 hops

Path: (0,0) → (1,0) → (2,0) → (3,0) → (3,1) → (3,2)

XY routing kyun? Turn restrictions fix karke deadlock prevent karta hai. Saare packets pehle east/west jaate hain, phir north/south—resource dependency graph mein koi cycles nahi → no deadlock.

Solution:

1. Core A write issue karta hai → request chiplet 1 L3 home ko route hoti hai (2 hops via I/O die = 40 ns).
2. Home directory check karta hai → dekhta hai Core B ek sharer hai.
3. Home Core B ko probe bhejta hai → "Invalidate X" (2 hops = 40 ns).
4. Core B respond karta hai → "ACK, line invalidated" (2 hops = 40 ns).
5. Home Core A ko reply karta hai → "Write complete" (2 hops = 40 ns).

Total latency: 4 × 40 = 160 ns (~560 cycles at 3.5 GHz).

Yeh kyun matter karta hai: Shared data par writes NUMA mein expensive hote hain. Software ko false sharing minimize karni chahiye (do cores ek hi 64 B cache line mein alag variables par write kar rahe hain).

Common Mistakes

Fix:

• FCLK (Fabric Clock): internal Infinity Fabric frequency.
• MCLK (Memory Clock): DDR frequency (MT/s rating ki aadhi).
• UCLK (Unified Memory Controller Clock): memory controller frequency.

Zen 2/3 par: FCLK = UCLK = MCLK 1:1 mode mein (sweet spot). DDR4-3600 ke baad, FCLK 1:2 ratio par aa jaata hai → zyada memory bandwidth lekin worse latency (fabric keep up nahi kar sakta).

Steel-man: Confusing hai kyunki AMD ne "Infinity Fabric" ko overload kiya hai—iska matlab inter-die fabric bhi hai (design par fixed) aur configurable FCLK bhi. Inter-chiplet links core clock ke fixed ratio par chalte hain, jabki FCLK dynamic hai.

Fix: ≤8 cores ke liye, ring simpler aur lower latency wala hai. Ring mein hota hai:

• Simpler router design (sirf 2 directions, 5 ports nahi)
• Lower die area overhead
• 8 cores ke liye avg hops: 8/4 = 2, same as a 3×3 mesh

Intel ne Broadwell tak (max 18 cores) ring use kiya kyunki dual rings + caching latency mask kar deta tha. Sirf 16+ cores par mesh decisively jeetta hai.

Fix: Yield aur cost dominate karte hain. 7 nm par ek 64-core monolithic die:

• Die size ~800 mm² (vs ~80 mm² per chiplet)
• Yield: $(1-\text{defect density}\times \text{area}{)}^{\#criticallayers}$
• 0.09 defects/cm² (TSMC 7 nm) par, 800 mm² ~10% yield karta hai, 80 mm² ~80% yield karta hai.
• Chiplets: 8 high-yield chhote dies + 14 nm par sasta I/O die use karo.

Result: Chiplets monolithic ka ~1/3 cost karte hain, aur 30-cycle penalty real workloads mein ≪ 1% hai (zyaatar time compute mein jaata hai, cross-chiplet traffic mein nahi).

Steel-man: Gamers "latency penalty" dekhte hain aur worried ho jaate hain. Lekin games inter-core communication par stress nahi dalte—woh single-thread performance aur GPU se limited hote hain.

Active Recall

#flashcards/hardware

Mesh interconnects kya problem solve karte hain jo rings nahi kar sakti? :: Rings ki latency O(N) hai aur bisection bandwidth limited hai. Meshes high core counts tak O(√N) latency aur parallel paths se zyada bandwidth ke saath scale karti hain.

Purana tarika (bus): Sab log ek hallway mein chillate hain. Ek waqt mein sirf ek insaan chilla sakta hai. 64 logon ke saath, sab kuch chaos ho jaata hai.

Behtar tarika (ring): Tum notes ek circle mein pass karte ho. Agar tum desk 1 par ho aur desk 50 se baat karni hai, note 49 desks se guzarta hai. Slow!

Best tarika (mesh): Building mein hallways ka ek grid hai—jaise kisi shehar ki streets. Tum note north, south, east, ya west pass kar sakte ho. Ab desk 1 se desk 50 sirf ~10 "blocks" door hai (shortcuts lo!). Multiple notes alag streets par ek saath travel kar sakti hain.

Infinity Fabric AMD ka is grid ka version hai. Har "desk" ek CPU core hai. "Notes" data hain. Grid ensure karta hai ki sabko apna data jaldi mile, 64 desks ke saath bhi. Isliye AMD ke bade CPUs (EPYC) mein itne saare cores ho sakte hain bina traffic jam ke.

Ya: "Fabric = Freeway (bus nahi, circle nahi)"—multiple lanes, direct routes.

Connections

• 6.3.1-Bus-ArchitectureBasics — Infinity Fabric bus limitations se evolve hua
• 6.3.5-Cache-Coherency-Protocols — MOESI/directory protocols Fabric coherency ka aadhar hain
• 6.2.8-NUMA-Architecture — Chiplets NUMA domains create karte hain; Fabric inter-node traffic manage karta hai
• 6.4.3-PCIe-Topology — I/O die PCIe ko Infinity Fabric ke zariye cores tak route karta hai
• 5.1.7-Memory-BandwidthCalculation — FCLK:MCLK ratio effective bandwidth ko affect karta hai
• 7.2.4-Network-Topologies — HPC interconnects ke mesh/torus concepts on-chip fabrics par bhi apply hote hain

Concept Map