CXL (Compute Express Link)

6.3.5 · Hardware › Interconnects, Buses & SoC

CXL Kaunsi Problem Solve Karta Hai?

Traditional systems mein memory coherency sirf ek processor ke domain ke andar hoti hai (CPU cores L3 cache coherently share karte hain). Lekin:

• GPU ki memory CPU memory se isolated hoti hai
• Ek FPGA accelerator CPU cache state nahi dekh sakta
• Zyada RAM add karne ke liye CPU ke memory controller mein DIMMs plug karne padte hain (limited slots)

Takleef ke points:

1. Data duplication: Data CPU→GPU copy karo, process karo, wapas copy karo. Bandwidth aur time waste hota hai.
2. Stale data: CPU apne cache mein ek value update karta hai; GPU apni memory se purani value padh leta hai. Cache incoherency.
3. Stranded memory: GPU ke paas 80 GB VRAM hai, lekin CPU ko ek dataset ke liye 128 GB chahiye—share nahi ho sakta.

CXL isko PCIe ke physical layer ke upar CPU ke cache-coherent memory protocol ko extend karke fix karta hai.

CXL Architecture: Teen Protocol Types

CXL teen sub-protocols define karta hai jo PCIe 5.0/6.0 physical/electrical layers ke upar chalte hain:

1. CXL.io (I/O Protocol)

Analogy: Ye "dial-up" fallback mode hai. Har CXL device ko zaroori hai CXL.io support karna taaki ye legacy systems mein kaam kare.

2. CXL.cache (Host-Accelerator Cache Coherency)

Key opcodes:

• ReadCur (current padhna, modify karne ki niyat nahi)
• ReadOwn (modify karne ki niyat se padhna—CPU ke "RFO" jaisa)
• SnpData, SnpInv (host se device ko snoop requests)

3. CXL.mem (CPU-Memory Device Coherency)

Use cases:

• Memory pooling: Multiple servers ek CXL memory box switch ke zariye share karte hain.
• Storage-class memory: CXL-attached persistent memory (Intel Optane on steroids ki tarah).
• Tiered memory: Hot data DRAM mein, warm data CXL memory mein, cold storage mein.

CXL vs. PCIe: Physical Layer Reuse

Derivation steps:

1. PCIe 5.0 spec: 32 GT/s raw rate.
2. Encoding overhead: 128b/130b (2 bits per 130 error detection ke liye).
3. Net rate: $32\times (128/130)=31.5$ Gb/s per lane.
4. Aggregate bandwidth ke liye lanes ki sankhya se scale karo.

Kyun matter karta hai: CXL ko naye cables ya connectors ki zaroorat nahi—ye PCIe ka software/firmware upgrade hai. Hyperscalers existing PCIe 5.0 infrastructure retrofit kar sakte hain.

Cache Coherency Deep Dive

MESI protocol (Modified, Exclusive, Shared, Invalid) CXL ke liye extend ki gayi hai:

CXL.cache flow (Accelerator-initiated):

1. GPU address A read karna chahta hai.
2. GPU CPU ko SnpData(A) bhejta hai.
3. CPU apna cache check karta hai:
   • Agar Modified hai: Memory mein write back karo, GPU ko data bhejo, Shared mein downgrade karo.
   • Agar Shared/Exclusive hai: Data bhejo, Shared mein downgrade karo.
   • Agar Invalid hai: Memory se fetch karo, GPU ko bhejo.
4. GPU data Shared state mein cache karta hai.

Downgrades kyun? Single-writer ya multiple-reader invariant maintain karne ke liye. Agar CPU Modified (dirty writer) hai, GPU tab tak nahi likh sakta jab tak CPU ownership na chhhode.

CXL Generations aur Bandwidth

CXL 2.0 innovation: Multi-level switching. Tumhare paas ek CXL switch ho sakta hai jo connect kare:

• 4 CPUs (hosts)
• 8 CXL memory devices
• Sab hosts sab memory devices access kar sakte hain (shared pool).

CXL 3.0 innovation: Devices seedha ek doosre se baat kar sakte hain CPU involvement ke bina. GPU-to-GPU over CXL fabric (NVLink jaisa lekin open standard).

Worked Example: AI Inference with CXL

Problem: 70B parameter LM chalao. Model weights = 140 GB (fp16). Single GPU VRAM = 80 GB. Traditional solution: model 2 GPUs mein split karo (slow, complex).

CXL solution:

1. 256 GB capacity ke saath ek CXL memory expander lagao.
2. Model weights CXL memory mein load karo.
3. GPU on-demand weights CXL.cache ke zariye padhe:
   • GPU ReadShared(weight_addr) issue karta hai.
   • CXL controller CXL memory se 64-byte cache line fetch karta hai.
   • GPU ise current batch ke liye locally cache karta hai.
4. Latency: CXL read ~200 ns (PCIe DMA ke ~2 µs setup + transfer ke muqable mein).
5. Bandwidth: CXL memory se 32 GB/s read (~50 tokens/sec par inference ke liye kaafi).

Ye kyun kaam karta hai:

• Coherency: Agar CPU ek model parameter update kare (fine-tuning?), GPU update turant dekhta hai.
• No copy: Weights CXL memory mein rehte hain. GPU ise apne address space ka extension maanta hai.
• Scalability: Jaise model 500B parameters tak badhen, aur CXL memory boxes add karo.

Common Mistakes

Memory Coherency Math

Mnemonic

Ya: "Coherent Xecution Layer"—shared-memory execution model par zor deta hai.

Connections

• PCIe-(PCI-Express): Physical layer reuse, TLP format, enumeration
• Cache-CoherencyProtocols: MESI/MOESI CXL.cache ke liye extend ki gayi
• NUMA-(Non-Uniform-Memory-Access): CXL memory NUMA-jaisi hai (door = zyada latency)
• DMA-(Direct-Memory-Access): CXL memory coherent banakar DMA ki zaroorat kam karta hai
• Memory-Hierarchy: CXL DRAM aur storage ke beech ek tier add karta hai
• HBM-(High-Bandwidth-Memory): CXL.mem backing store ke roop mein HBM use kar sakta hai
• NVLink: Nvidia ka proprietary coherent link (similar goals, closed ecosystem)
• Gen-Z-and-CCIX: Competing/defunct coherent interconnect standards

Further Exploration

1. CXL switching topologies: CXL 2.0 switch 4 hosts aur 8 memory devices ke beech coherency traffic kaise route karta hai?
2. Bias in memory pooling: Agar 4 servers CXL memory share karein, to bandwidth saturate hone par priority kise milti hai?
3. CXL 3.0 fabric: Peer-to-peer accelerator transfers CPU bottlenecks kaise avoid karte hain?
4. Security: Kya ek malicious CXL device doosre device ki memory snoop kar sakta hai? (CXL 3.0 mein Integrity and Data Encryption—IDE)

#flashcards/hardware

CXL kaunsi problem solve karta hai jo akela PCIe nahi kar sakta? :: CXL CPU aur attached devices (GPUs, accelerators, memory) ke beech cache coherency provide karta hai. PCIe sirf non-coherent DMA support karta hai, jisse stale data aur explicit copy overhead hota hai.

Agar CPU cache line Modified hai aur ek CXL device SnpData ke zariye request karta hai, to kya hota hai? :: CPU dirty line memory mein write back karta hai, device ko data bhejta hai, aur Shared state mein downgrade ho jaata hai. Dono ke paas ab coherent Shared copies hain.

Concept Map