CXL (Compute Express Link)

What Problem Does CXL Solve?

Traditional systems have memory coherency only within a processor's domain (CPU cores share L3 cache coherently). But:

A GPU's memory is isolated from CPU memory
An FPGA accelerator can't see CPU cache state
Adding more RAM requires pluging DIMMs into the CPU's memory controller (limited slots)

The pain points:

Data duplication: Copy data from CPU→GPU, process, copy back. Wastes bandwidth and time.
Stale data: CPU updates a value in its cache; GPU reads the old value from its own memory. Cache incoherency.
Stranded memory: GPU has80 GB VRAM, but CPU needs 128 GB for a dataset—can't share.

CXL fixes this by extending the CPU's cache-coherent memory protocol over PCIe's physical layer.

CXL Architecture: Three Protocol Types

CXL defines three sub-protocols that run on top of PCIe 5.0/6.0 physical/electrical layers:

1. CXL.io (I/O Protocol)

Analogy: This is the "dial-up" fallback mode. Every CXL device must support CXL.io so it works in legacy systems.

2. CXL.cache (Host-Accelerator Cache Coherency)

Key opcodes:

ReadCur (read current, no intent to modify)
ReadOwn (read with intent to modify—like CPU's "RFO")
SnpData, SnpInv (snoop requests from host to device)

3. CXL.mem (CPU-Memory Device Coherency)

Use cases:

Memory pooling: Multiple servers share a CXL memory box over a switch.
Storage-class memory: CXL-attached persistent memory (like Intel Optane on steroids).
Tiered memory: Hot data in DRAM, warm data in CXL memory, cold in SD.

CXL vs. PCIe: Physical Layer Reuse

Formula Why CXL Builds on PCIe

PCIe 5.0 provides 32 GT/s per lane (gigatransfers/sec). With 128b/130b encoding, effective bandwidth per lane:

B = \frac{32 \times 10^9 \, \text{transfers/s} \times 128 \, \text{bits}}{130 \, \text{bits}} = 31.5 \, \text{Gb/s} \approx 3.94 \, \text{GB/s}

For a ×16 link (16 lanes):

B_{x16} = 16 \times 3.94 = 63 \, \text{GB/s bidirectional (32 GB/s each direction)}

CXL's twist: It time-multiplexes CXL.io, CXL.cache, and CXL.mem traffic on the same PCIe lanes using Flex Bus protocol. The link arbitrates between protocol types dynamically.

Derivation steps:

PCIe 5.0 spec: 32 GT/s raw rate.
Encoding overhead: 128b/130b (2 bits per130 for error detection).
Net rate: $32 \times (128/130) = 31.5$ Gb/s per lane.
Scale by number of lanes for aggregate bandwidth.

Why this matters: CXL doesn't need new cables or connectors—it's a software/firmware upgrade to PCIe. Hyperscalers can retrofit existing PCIe 5.0 infrastructure.

Imagine you and your friend are editing the same Google Doc. Without "auto-sync," you might both have different versions—you add a sentence, but your friend doesn't see it yet. That's incoherent. Google Docs solves this by syncing every change instantly (like "cache coherency").

CXL does the same for computer chips. Your CPU and GPU both want to read/write the same memory address. CXL makes sure if the CPU changes a number, the GPU instantly knows about it (or vice versa). No confusion, no stale data. The "sync" happens in hardware, at nanosecond speeds.

The MESI protocol (Modified, Exclusive, Shared, Invalid) is extended for CXL:

| State | CPU Cache | Accelerator Cache | Who Can Write? | |----------|-----------|-------------------| | Modified | Has dirty data | Invalid | CPU only | | Exclusive| Has clean data | Invalid | CPU (can upgrade to M) | | Shared | Has clean data | Has clean data | Neither (must request ownership) | | Invalid | Stale/absent | Stale/absent | Neither

CXL.cache flow (Accelerator-initiated):

GPU wants to read address A.
GPU sends SnpData(A) to CPU.
CPU checks its cache:
- If Modified: Write back to memory, send data to GPU, downgrade to Shared.
- If Shared/Exclusive: Send data, downgrade to Shared.
- If Invalid: Fetch from memory, send to GPU.
GPU caches data in Shared state.

Why the downgrades? To maintain the single-writer or multiple-reader invariant. If CPU is Modified (dirty writer), GPU can't also write until CPU relinquishes ownership.

CXL Generations and Bandwidth

Version	Base Spec	Bandwidth (×16)	Key Feature
CXL 1.0	PCIe 5.0	64 GB/s (32+32)	CXL.io, .cache, .mem
CXL 2.0	PCIe 5.0	64 GB/s	Memory pooling, switches
CXL 3.0	PCIe 6.0	128 GB/s (64+64)	Peer-to-peer, fabric

CXL 2.0 innovation: Multi-level switching. You can have a CXL switch that connects:

4 CPUs (hosts)
8 CXL memory devices
All hosts can access all memory devices (shared pool).

CXL 3.0 innovation: Devices can talk directly to each other without CPU involvement. GPU-to-GPU over CXL fabric (like NVLink but open standard).

Worked Example: AI Inference with CXL

Problem: Run a70B parameter LM. Model weights = 140 GB (fp16). Single GPU VRAM = 80 GB. Traditional solution: split model across 2 GPUs (slow, complex).

CXL solution:

Plug in a CXL memory expander with 256 GB capacity.
Load model weights into CXL memory.
GPU reads weights on-demand via CXL.cache:
- GPU issues ReadShared(weight_addr).
- CXL controller fetches 64-byte cache line from CXL memory.
- GPU caches it locally for the current batch.
Latency: CXL read ~200 ns (vs. PCIe DMA at ~2 µs for setup + transfer).
Bandwidth: 32 GB/s read from CXL memory (enough for inference at ~50 tokens/sec).

Why this works:

Coherency: If the CPU updates a model parameter (fine-tuning?), GPU sees the update immediately.
No copy: Weights stay in CXL memory. GPU treats it as an extension of its address space.
Scalability: Add more CXL memory boxes as models grow to 500B parameters.

Common Mistakes

Memory Coherency Math

Formula Coherency Lat Breakdown

Assume a CXL.cache read from CPU memory:

Accelerator request: Send SnpData packet. Time = PCIe TLP latency = $t_{\text{TLP}} \approx 50 \, \text{ns}$ .
CPU cache lookup: CPU checks L3 cache tags. Time = $t_{\text{L3}} \approx 20 \, \text{ns}$ .
Snoop hit (data in L3): Read cache line, serialize, send back. Time = $t_{\text{L3}} + t_{\text{TLP}} = 70 \, \text{ns}$ .
Snoop miss (data in DRAM): Fetch from DRAM ( $t_{\text{DRAM}} \approx 80 \, \text{ns}$ ), send back. Time = $80 + 50 = 130 \, \text{ns}$ . Total latency (L3 hit):

T_{\text{CXL.cache}} = t_{\text{req}} + t_{\text{L3}} + t_{\text{resp}} = 50 + 20 + 50 = 120 \, \text{ns}

Compare to PCIe DMA (non-coherent):

T_{\text{DMA}} = t_{\text{setup}} + \frac{64 \, \text{B}}{32 \, \text{GB/s}} \approx 2000 \, \text{ns (setup dominates)}

CXL is 16× faster for small random reads because it skips DMA setup overhead and directly snoops the CPU cache.

Mnemonic

Or: "Coherent Xecution Layer"—emphasizes the shared-memory execution model.

Connections

PCIe-(PCI-Express): Physical layer reuse, TLP format, enumeration
Cache-CoherencyProtocols: MESI/MOESI extended for CXL.cache
NUMA-(Non-Uniform-Memory-Access): CXL memory is NUMA-like (farther = higher latency)
DMA-(Direct-Memory-Access): CXL reduces need for DMA by making memory coherent
Memory-Hierarchy: CXL adds a tier between DRAM and storage
HBM-(High-Bandwidth-Memory): CXL.mem can use HBM as backing store
NVLink: Nvidia's proprietary coherent link (similar goals, closed ecosystem)
Gen-Z-and-CCIX: Competing/defunct coherent interconnect standards

Further Exploration

CXL switching topologies: How does a CXL 2.0 switch route coherency traffic between4 hosts and 8 memory devices?
Bias in memory pooling: If 4 servers share CXL memory, who gets priority when bandwidth is saturated?
CXL 3.0 fabric: How do peer-to-peer accelerator transfers avoid CPU bottlenecks?
Security: Can a malicious CXL device snoop another device's memory? (Integrity and Data Encryption—IDE—in CXL 3.0)

#flashcards/hardware

What problem does CXL solve that PCIe alone cannot? :: CXL provides cache coherency between CPU and attached devices (GPUs, accelerators, memory). PCIe only supports non-coherent DMA, leading to stale data and explicit copy overhead.

What are the three CXL sub-protocols?

CXL.io (standard PCIe I/O), CXL.cache (device caches host memory coherently), CXL.mem (host accesses device memory as system RAM).

How does CXL achieve lower latency than PCIe DMA for small reads?

CXL.cache directly snoops the CPU cache (~120 ns) instead of setting up a DMA transfer (~2 µs for setup). Skips the CPU→memory→DMA→device copy chain.

What is Flex Bus in CXL?

A time-multiplexing protocol that dynamically shares PCIe lanes between CXL.io, CXL.cache, and CXL.mem traffic on the same physical link.

If a CPU cache line is Modified and a CXL device requests it viaSnpData, what happens? :: CPU writes the dirty line back to memory, sends data to the device, and downgrades to Shared state. Both now have coherent Shared copies.

Why is CXL.mem useful for AI workloads?

AI models (e.g., 70B LM = 140 GB) exceed GPU VRAM. CXL.mem lets the GPU treat a CXL memory expander as extended address space without explicit copies, with ~200 ns latency.

What bandwidth does CXL 3.0 provide over a×16 link?

128 GB/s bidirectional (64 GB/s each direction), based on PCIe 6.0's 64 GT/s rate.

Why can't CXL replace PCIe for all devices?

CXL is for coherent memory/compute devices. Non-coherent I/O (SSDs, NICs) doesn't benefit from cache coherency and works fine with standard PCIe, which has lower protocol overhead.

What is memory pooling in CXL 2.0?

Multiple hosts (CPUs) share access to multiple CXL memory devices via a CXL switch, creating a disagregated memory pool. Any host can read/write any memory device.

How does CXL 3.0 enable peer-to-peer accelerator communication?

Devices can send CXL transactions directly to each other over a CXL fabric, bypassing the CPU for GPU↔GPU or GPU↔FPGA data transfers.

Concept Map

Hinglish (regional understanding)

Intuition Hinglish mein samjho

CXL ka main kaam hai CPU aur baki devices (GPU, memory, accelerators) ke bech cache coherent memory sharing provide karna. Socho, tumhare CPU ne koi value apne cache mein update kari, lekin GPU ko wo purani value mil rahi hai kyunki wo apne alag memoryein dekh raha hai—yeh incoherency problem hai. PCIe mein tumhe explicitly data copy karna padta hai CPU se GPU, phir wapas—bahut slow aur error-prone.

CXL yeh problem solve karta hai by extending CPU ka coherency protocol over PCIe lanes. Teen types ke protocols hain: CXL.io (normal PCIe jaisa I/O), CXL.cache (device CPU ke memory ko cache kar sakta hai coherently, jaise CPU cores apne bech karte hain),

What Problem Does CXL Solve?

CXL Architecture: Three Protocol Types

1. CXL.io (I/O Protocol)

2. CXL.cache (Host-Accelerator Cache Coherency)

3. CXL.mem (CPU-Memory Device Coherency)

CXL vs. PCIe: Physical Layer Reuse

Cache Coherency Deep Dive

CXL Generations and Bandwidth

Worked Example: AI Inference with CXL

Common Mistakes

Memory Coherency Math

Mnemonic

Connections

Further Exploration

Concept Map

Hinglish (regional understanding)

Go deeper — visual, from zero

Test yourself — Interconnects, Buses & SoC

Connections