CXL (Compute Express Link): Exercises — Hinglish deep dive

6.3.5 · D4 · Hardware › Interconnects, Buses & SoC › CXL (Compute Express Link)

Shuru karne se pehle, ek picture hai jo neeche use hone wale har term ko fix kar deti hai. Jab bhi confused ho, ise dekho.

Host CPU hai. Ek accelerator ek GPU/FPGA hai jo host memory ko cache kar sakta hai (yahi CXL.cache hai). Ek memory device RAM ka ek box hai jise CPU wire ke upar se access karta hai (yahi CXL.mem hai). Teen arrows teen sub-protocols hain jo ek PCIe cable par ride karte hain.

Level 1 — Recognition

Exercise 1.1 — Protocol ko Match karo

Neeche diye gaye har zaroorat ke liye, naam batao ki teen CXL sub-protocols (CXL.io, CXL.cache, CXL.mem) mein se kaun sa kaam karta hai.

(a) Ek legacy driver ko device ko plain PCIe config reads se enumerate karna hai. (b) Ek GPU ek CPU cache line ki coherent copy hold karna chahta hai. (c) OS ek plug-in RAM box ko extra system memory ke roop mein dekhna chahta hai.

Recall Solution

(a) CXL.io — yeh ordinary PCIe hai (config, MMIO, DMA). Har CXL device ko ise support karna padta hai taaki yeh non-CXL slots mein boot ho sake. (b) CXL.cache — accelerator host memory ko cache karta hai aur coherency protocol mein join karta hai. (c) CXL.mem — host device memory ko apne physical address space ke part ke roop mein access karta hai.

Direction hi hint hai: .cache = device host ka RAM cache karta hai; .mem = host device ka RAM access karta hai.

Exercise 1.2 — MESI State Naming

CPU ke paas ek cache line hai jo dirty hai (uski value main memory se alag hai), aur accelerator ki copy absent hai. Dono states ka naam batao.

Recall Solution

CPU = Modified (M): iske paas ek hi up-to-date, dirty copy hai, isliye koi aur read kare us se pehle ise write back karna padega. Accelerator = Invalid (I): iske paas koi valid copy nahi hai. Yeh exactly parent table ki row hai "Modified | Has dirty data | Invalid | CPU only".

Level 2 — Application

Exercise 2.1 — Per-lane Bandwidth Raw Rate se

PCIe 5.0 $32 GT/s$ (giga-transfers per second) per lane pe chalta hai aur 128b/130b encoding use karta hai — wire par har 130 bits mein se sirf 128 tumhara data carry karte hain (2 bits overhead hain). Effective data rate per lane $Gb/s$ mein compute karo, phir $GB/s$ mein convert karo (8 bits = 1 byte).

Recall Solution

Hum kya karte hain: raw rate ko bits ke useful fraction se scale karte hain. Kyun: har transfer wire par 1 bit move karta hai, lekin un bits mein se sirf $128/130$ tumhara payload hain. $B_{lane} = 32 \times 1 0^{9} \times \frac{128}{130} = 31.507 Gb/s$ Bytes mein convert karo (8 se divide karo): $B_{lane} = \frac{31.507}{8} = 3.938 GB/s \approx 3.94 GB/s$ Yeh parent note ke $\approx 3.94 GB/s$ per lane se match karta hai.

Exercise 2.2 — ×16 Link tak Scale karo

Ek CXL 1.0 link $\times 16$ (16 lanes) hai aur full-duplex hai (dono directions mein simultaneously same rate). (a) one-direction bandwidth aur (b) aggregate bidirectional figure do.

Recall Solution

(a) Ek direction: per-lane ko 16 se multiply karo. $B_{x16, one dir} = 16 \times 3.938 = 63.0 GB/s$ Hmm — parent 32 GB/s each direction quote karta hai. Difference isliye hai kyunki parent usable protocol rate ko (encoding ke upar framing/flit overhead ki wajah se) ek clean " $32 + 32$ " tak round down karta hai. Toh: (b) Aggregate = == $64 GB/s$ == bidirectional, yaani practice mein $32 GB/s$ har taraf.

Do numbers, do meanings: $\approx 63 GB/s$ encoding-only ceiling hai; $64 GB/s$ ( $32 + 32$ ) rounded protocol figure hai jo spec advertise karta hai. Dono parent mein aate hain — yeh contradictions nahi hain.

Level 3 — Analysis

Exercise 3.1 — Ek Coherent Read Trace karo

CPU ke paas address A par line Modified state mein hai (dirty). GPU ReadShared(A) issue karta hai. Exact ordered steps likho aur dono caches ki final MESI state do. Explain karo kyun write-back hona zaroori hai.

Recall Solution

Neeche diye figure mein arrows follow karo.

GPU ReadShared(A) CXL.cache ke zariye host ko bhejta hai.
Host apna cache snoop karta hai, A ko Modified mein paata hai → iske paas ek hi correct value hai.
Host dirty line ko main memory mein write back karta hai. Kyun: memory abhi stale value hold karti hai; agar hum yeh skip karte, toh baad mein koi reader jo directly memory jaata hai use garbage dikhta. Coherency require karta hai ki jab line Shared ho jaye toh memory ke paas truth ho.
Host GPU ko fresh data bhejta hai aur apni khud ki copy M → Shared kar leta hai.
GPU line ko Shared mein install karta hai. Final states: CPU = Shared, GPU = Shared. Ab koi bhi ownership request kiye bina write nahi kar sakta.

Exercise 3.2 — Dono ko Write kyun nahi karne dete?

Single-writer / multiple-reader invariant state karo aur Exercise 3.1 ki line use karke explain karo ki agar GPU ko write karne diya jaye jabki CPU Modified raha, toh kya toot jaata hai.

Recall Solution

Invariant: kisi bhi instant par ek cache line ke paas ya toh ek writer hota hai (Modified/Exclusive) ya koi bhi number ke readers (Shared) — kabhi bhi ek writer plus koi aur accessor nahi. Agar CPU Modified raha (writing A = 5) jabki GPU ne bhi write kiya (A = 9), toh ab do divergent "truths" hain. Baad mein ek reader 5 ya 9 dekh sakta hai depending on kaun sa cache snoop kiya — classic incoherency jo parent ke Google-Docs analogy mein warn kiya gaya hai. Shared mein downgrade exactly ek authoritative value enforce karta hai. Yahi coherency discipline Cache-CoherencyProtocols mein describe ki gayi hai.

Level 4 — Synthesis

Exercise 4.1 — Memory Expansion Budget

Ek 8-socket server mein $2 TB$ DRAM per socket hai. Ek in-memory database ko $32 TB$ chahiye. Tum CXL.mem ke zariye $4 TB$ each ke CXL memory expanders add karte ho. (a) On-board kitna DRAM hai? (b) Kitne expanders chahiye? (c) OS kitna total address-space size dekhta hai?

Recall Solution

(a) On-board DRAM $= 8 \times 2 = 16 TB$ . (b) Shortfall $= 32 - 16 = 16 TB$ ; har expander $4 TB$ ka hai, toh $16/4 === 4 ==$ expanders. (c) OS $16 + 4 \times 4 = 32 TB$ ka unified physical address space dekhta hai. CPU ka memory controller har physical address check karta hai: agar DRAM range mein hai toh DDR command issue karta hai; agar usse bahar hai toh CXL.mem transaction format karta hai aur expander ko route karta hai. Yeh tiering idea NUMA-(Non-Uniform-Memory-Access) mein hai — paas wali memory fast hai, CXL memory ek door wala "NUMA node" hai.

Exercise 4.2 — Latency-aware Access Mix

Local DRAM latency $= 80 ns$ ; CXL memory latency $= 150 ns$ . Ek workload apne $70%$ accesses hot data ke liye DRAM mein karta hai aur $30%$ warm data ke liye CXL memory mein. Average access latency compute karo. Phir all-DRAM ideal se compare karo.

Recall Solution

Kyun weighted average: har access wahan latency pay karta hai jahan uska data rehta hai; expected latency probability-weighted sum hai. $\overset{ˉ}{t} = 0.70 \times 80 + 0.30 \times 150 = 56 + 45 = 101 ns$ All-DRAM ideal hota $80 ns$ , toh CXL tier $101 - 80 = 21 ns$ ( $\approx 26%$ ) extra latency cost karta hai — lekin tumhe memory capacity milti hai jo physically DIMM slots mein fit nahi hoti. NVMe se compare karo jo tens of microseconds ka hai: CXL memory abhi bhi flash se $\sim 100 \times$ faster hai.

Level 5 — Mastery

Exercise 5.1 — Model-serving Capacity

Ek 70B-parameter language model fp16 mein store hai ( $2$ bytes per parameter) aur serve karna hai. GPU VRAM $= 80 GB$ . (a) Model weight size? (b) Kya yeh ek GPU mein fit hota hai? (c) Ek $256 GB$ CXL memory expander ke saath, kya yeh fit hota hai, aur weights GPU ko stream karne ke liye kaun sa protocol use hota hai?

Recall Solution

(a) $70 \times 1 0^{9} params \times 2 bytes = 140 \times 1 0^{9} bytes === 140 GB ==$ . (b) $140 > 80$ → nahi, yeh ek GPU ke VRAM mein fit nahi hota. (c) $140 \leq 256$ → haan, yeh CXL expander mein fit hota hai. GPU ReadShared(weight_addr) CXL.cache ke zariye issue karta hai; CXL controller demand par expander memory se 64-byte cache lines fetch karta hai aur GPU unhe current layer ke liye coherently cache karta hai. Yeh slow manual copies ko coherent on-demand streaming se replace karta hai (compare karo DMA-(Direct-Memory-Access) se, jo blocks coherency ke bina copy karta hai).

Exercise 5.2 — Weight Streaming par Bandwidth Ceiling

Maano har token ke forward pass ko CXL 3.0 $\times 16$ link par $64 GB/s$ per direction se stream karke saare $140 GB$ weights ek baar touch karne padte hain. (a) Sirf bandwidth se minimum time per token. (b) Peak tokens/second yeh ceiling allow karta hai.

Recall Solution

(a) Kyun bytes ko bandwidth se divide karo: $D$ bytes ko $R$ bytes/s rate ke link par move karne ka time $t = D / R$ hota hai. $t = \frac{140 GB}{64 GB/s} = 2.1875 s per token$ (b) Tokens/s $= 1/ t = 1/2.1875 === 0.457 tok/s ==$ . Interpretation: har weight ko token per stream karna brutally slow hai — isliye real systems hot weights VRAM mein resident rakhte hain (HBM-(High-Bandwidth-Memory) $\sim$ TB/s deta hai) aur CXL memory ko ek overflow tier ke roop mein use karte hain, primary path ke roop mein nahi. Yeh exercise reveal karta hai kyun HBM, CXL ke saath exist karta hai: hot weights ke liye bottleneck capacity nahi, bandwidth hai.

Exercise 5.3 — Generation Reasoning

CXL 3.0 ek fabric par peer-to-peer device-to-device transfers add karta hai, "NVLink jaisa lekin open." Do sentences mein explain karo kyun GPU→GPU transfer se CPU ko hataana matter karta hai, aur un earlier open interconnects ka naam batao jo CXL ne absorb kiye.

Recall Solution

CPU ko hatana host memory aur CPU ke coherency engine ke zariye ek round-trip eliminate karta hai, isliye GPU→GPU data seedha fabric speed par fabric ke across move hota hai jisme lower latency aur koi host-bandwidth tax nahi hota. Jin earlier efforts ko CXL ne effectively absorb kiya woh hain Gen-Z-and-CCIX — unke memory-semantic aur coherency ideas CXL 2.0/3.0 fabric model mein fold kar diye gaye.

Recall One-line self-test recap

.cache = device caches host RAM; .mem = host reaches device RAM ::: right PCIe 5.0 per-lane effective rate ::: ~3.94 GB/s CXL 1.0 x16 aggregate ::: 64 GB/s (32+32) Read of a Modified line ends in state ::: Shared for both 70B fp16 model size ::: 140 GB Invariant CXL enforces ::: single-writer or multiple-reader