Infinity Fabric - mesh interconnects

Overview

Modern multi-core processors need a way for cores,aches, memory controllers, and I/O to talk to each other. Infinity Fabric (AMD's implementation) and mesh interconnects (Intel's) are scalable, distributed networks-on-chip that replace older bus and ring topologies when core counts exceed ~10-12 cores.

The fundamental problem: A shared bus saturates (all cores compete for one wire). A ring adds latency (message travels hop-by-hop). A mesh or fabric gives each node multiple paths to reach any other node, balancing latency and bandwidth.

Core Concepts

What is Infinity Fabric?

Key properties:

• Cache coherent: uses a distributed, scalable coherency protocol (loosely based on MOESI, but custom "Scalable Coherent Protocol")
• Point-to-point links: no shared medium; each connection is dedicated
• Packet-switched: data travels as flits (flow-control units)
• Topology-agnostic: can form rings, meshes, or hybrid topologies depending on the design

What is a Mesh Interconnect?

Intel's mesh (Xeon Scalable, Core X): introduced in Skylake-X to replace the ring bus. Each router has:

• 5 ports: 4 cardinal directions + 1 local agent (core/cache)
• Bufers and arbitration logic
• Routing tables (usually dimension-ordered: X first, then Y)

Why These Replaced Rings and Buses

The Ring Bus Limit

Intel's ring (Nehalem through Broadwell): cores arranged in a ring, each stop is a core + L3 slice. Two counter-rotating rings for bandwidth.

Problem:

• Latency scales with core count: A message from core 0 to core 15 on a16-core ring must hop through ~8 stops (average hops = N/4).
• Bandwidth saturation: Ring bandwidth is shared. With 10+ cores issuing memory requests, the ring becomes a bottleneck.

Breaking point: ~10-12 cores. Beyond that, ring latency and contention degrade performance.

Shared Bus Limit

Classic front-side bus (FSB): all cores share one bus to memory/chipset.

Problem:

• Electrical loading: each added core adds capacitance, limiting clock speed.
• Arbitration overhead: only one transaction at a time.
• Bandwidth wall: ~10 GB/s max for FSB.

Solution: Point-to-point. Each link is independent, runs full speed, and multiple transactions occur simultaneously.

How Infinity Fabric Works

Architecture

Components:

1. Coherent Master (CM): CPU core's L1/L2 interface. Issues read/write requests.
2. Coherent Slave (CS): L3 cache slice or memory controller. Responds to requests.
3. I/O Master (IOM): PCIe/SATA/USB controllers. Non-coherent traffic.
4. Router/Switch: packet router with multiple virtual channels (VCs) to prevent deadlock.

Packet structure:

• Header: source ID, dest ID, command (read/write/probe/snoop), address
• Payload: cache line (64 B) or smaller
• Credit-based flow control: sender tracks receiver's buffer space

Topology Examples

Zen 2 (Ryzen 3000, EPYC Rome):

• Each chiplet (8 cores, 32 MB L3) has internal Infinity Fabric connecting cores to L3 slices.
• Central I/O die has the memory controllers and inter-chiplet fabric.
• Chiplets connect to I/O die via Infinity Fabric IF/xGMI links (32 B/cycle/direction at ~1.8 GHz FCLK = ~115 GB/s per link).

MI300X (CDNA 3):

• 8 GPU chiplets + 4 I/O dies.
• Full mesh: each die connects to multiple neighbors.
• 144 GB/s per xGMI link (900 GB/s aggregate per GPU die).

Ryzen 7000 (Zen 4):

• Monolithic die for desktop (1 CD), so internal fabric is simpler.
• FCLK (Fabric Clock) runs at 1:1 or 1:2 ratio with memory clock (MCLK).

Coherency Protocol

Challenge: With 64 cores across 8 chiplets, how do you keep L1/L2/L3 caches coherent?

Solution: Scalable Coherent Protocol (details proprietary, but principles):

1. Directory-based: Each L3 slice tracks which cores have a copy of a line (sharer list).
2. Snoops are targeted: Only the sharers receive invalidation probes, not all cores (unlike broadcast snooping).
3. Home node: The L3 slice whose address range includes a cache line is the "home". All requests route through the home.
4. Three-hop protocol (typical):
   • Core A requests line X → Home L3 slice
   • Home checks directory, sends probes to sharers if needed
   • Sharers respond → Home replies to Core A

Why directory scales: Broadcast snooping sends every request to every core (O(N) traffic). Directory sends probes only to sharers (O(k), k = sharers, typically 1-3).

Derivation: Mesh Latency vs Ring

Ring Latency

Assume N cores in a ring, uniform traffic.

Average hops: For a random source-destination pair, expected hops = N/4 (quarter of the ring on average, using the shorter direction).

Latency per hop: ~1-2 cycles (pass-through the router).

For N = 16, $T_{\text{hop}} = 2$ cycles:

Mesh Latency

2D mesh: $\sqrt{N} \times \sqrt{N}$ grid.

Manhattan distance: For random (x₁, y₁) to (x₂, y₂):

For uniform distribution on a $k \times k$ grid:

For N = 16 (4×4 mesh):

With $T_{\text{hop}} = 2$ cycles:

Speedup: $\frac{8}{5.3} \approx 1.5 \times$ lower latency.

Bandwidth: Mesh has $2N$ links (each of N nodes has 4 links, shared edges counted once → $4N/2 = 2N$). Ring has $N$ links. Bisection bandwidth (cut the network in half):

• Ring: 2 links cross the cut (the two ring segments).
• Mesh: $\sqrt{N}$ links cross the cut (one row of vertical links).

For N = 16:

• Ring bisection: 2 links
• Mesh bisection: 4 links → 2× higher.

Why Infinity Fabric's Latency is Competitive

EPYC Rome: 8-chiplet design, each chiplet is 8cores in a local ring/mesh, then Infinity Fabric hops between chiplets.

Worst case: Core on chiplet 0 → L3 slice on chiplet 7.

• Intra-chiplet: ~5 cycles
• Chiplet-to-I/O die: ~20 ns (35 cycles at 1.8 GHz FCLK)
• I/O-to-chiplet: ~20 ns
• Total: ~75 cycles (~40 ns at 3.5 GHz core clock)

Compared to monolithic die: A 64-core monolithic 8×8 mesh would have ~5.3 avg hops × 2 cycles = 10.6 cycles intra-die, but memory access still adds ~100 cycles. The chiplet penalty (~30 cycles extra) is acceptable given the yield/cost benefits.

Worked Examples

Solution:

• FCLK = 1.8 GHz →.8 × 10⁹ cycles/s
• Data width = 32 B per direction per cycle
• Bandwidth (one direction) = 1.8 × 32 = 57.6 GB/s
• Bidirectional = 2 × 57.6 = 115.2 GB/s

Why this step? Each cycle, the fabric transfers 32 bytes. Multiply by frequency to get bytes/second.

Reality check: DDR4-3600 dual-channel = 2 × 3.6 GHz × 8 B = 57.6 GB/s. Fabric bandwidth matches memory bandwidth (1:1 ratio ensures no bottleneck).

Solution:

• Dimension-ordered routing (XY): move in X first, then Y.
• X distance: 3 - 0 = 3 hops (east)
• Y distance: 2 - 0 = 2 hops (north)
• Total hops = 3 + 2 = 5 hops

Path: (0,0) → (1,0) → (2,0) → (3,0) → (3,1) → (3,2)

Why XY routing? Prevents deadlock by fixing turn restrictions. All packets go east/west first, then north/south—no cycles in the resource dependency graph.

Solution:

1. Core A issues write → request routes to chiplet 1 L3 home (2 hops via I/O die = 40 ns).
2. Home checks directory → sees Core B is a sharer.
3. Home sends probe to Core B → "Invalidate X" (2 hops = 40 ns).
4. Core B responds → "ACK, line invalidated" (2 hops = 40 ns).
5. Home replies to Core A → "Write complete" (2 hops = 40 ns).

Total latency: 4 × 40 = 160 ns (~560 cycles at 3.5 GHz).

Why this matters: Writes to shared data are expensive in NUMA. Software should minimize false sharing (two cores writing different variables in the same 64 B cache line).

Common Mistakes

The fix:

• FCLK (Fabric Clock): internal Infinity Fabric frequency.
• MCLK (Memory Clock): DDR frequency (half of MT/s rating).
• UCLK (Unified Memory Controller Clock): memory controller frequency.

On Zen 2/3: FCLK = UCLK = MCLK in1:1 mode (sweet spot). Beyond DDR4-3600, FCLK drops to 1:2 ratio → higher memory bandwidth but worse latency (fabric can't keep up).

Steel-man: It's confusing because AMD overloaded "Infinity Fabric" to mean both the inter-die fabric (fixed at design) and the configurable FCLK. The inter-chiplet links run at a fixed ratio of core clock, while FCLK is dynamic.

The fix: For ≤8 cores, ring is simpler and lower latency. Ring has:

• Simpler router design (only 2 directions, not 5 ports)
• Lower die area overhead
• Avg hops for8 cores: 8/4 = 2, same as a3×3 mesh

Intel used ring through Broadwell (18 cores max) because dual rings + caching masks the latency. Only at 16+ cores does mesh win decisively.

The fix: Yield and cost dominate. A 64-core monolithic die on7 nm:

• Die size ~800 mm² (vs ~80 mm² per chiplet)
• Yield: $(1 - \text{defect density} \times \text{area})^{\# critical layers}$
• At 0.09 defects/cm² (TSMC 7 nm), 800 mm² yields ~10%, 80 mm² yields ~80%.
• Chiplets: use 8 high-yield small dies + cheap I/O die on 14 nm.

Result: Chiplets cost ~1/3 of monolithic, and 30-cycle penalty is ≪ 1% in real workloads (most time is compute, not cross-chiplet traffic).

Steel-man: Gamers see the "latency penalty" and worry. But games rarely stress inter-core communication—they're limited by single-thread perf and GPU.

Active Recall

#flashcards/hardware

What problem do mesh interconnects solve that rings cannot? :: Rings have O(N) latency and limited bisection bandwidth. Meshes scale to high core counts with O(√N) latency and higher bandwidth via parallel paths.

Old way (bus): Everyone shouts into one hallway. Only one person can shout at a time. With 64 people, it's chaos.

Better way (ring): You pass notes in a circle. If you're at desk1 and need to talk to desk 50, the note goes through49 desks. Slow!

Best way (mesh): The building has a grid of hallways—like streets in a city. You can pass note north, south, east, or west. Now desk 1 to desk 50 is only ~10 "blocks" away (take shortcuts!). Multiple notes can travel at once on different streets.

Infinity Fabric is AMD's version of this grid. Each "desk" is a CPU core. The "notes" are data. The grid makes sure everyone gets their data fast, even with64 desks. That's why AMD big CPUs (EPYC) can have so many cores without turning into a traffic jam.

Or: "Fabric = Freway (not a bus, not a circle)"—multiple lanes, direct routes.

Connections

• 6.3.1-Bus-ArchitectureBasics — Infinity Fabric evolved from bus limitations
• 6.3.5-Cache-Coherency-Protocols — MOESI/directory protocols underpin Fabric coherency
• 6.2.8-NUMA-Architecture — Chiplets create NUMA domains; Fabric manages inter-node traffic
• 6.4.3-PCIe-Topology — I/O die routes PCIe through Infinity Fabric to cores
• 5.1.7-Memory-BandwidthCalculation — FCLK:MCLK ratio affects effective bandwidth
• 7.2.4-Network-Topologies — Mesh/torus concepts from HPC interconnects apply to on-chip fabrics

Concept Map

Hinglish (regional understanding)