Exercises — CXL (Compute Express Link)

(a) CXL.io — this is ordinary PCIe (config, MMIO, DMA). Every CXL device must support it so it boots in non-CXL slots. (b) CXL.cache — the accelerator caches host memory and joins the coherency protocol. (c) CXL.mem — the host reaches device memory as part of its physical address space.

Direction is the tell: .cache = device caching host's RAM; .mem = host reaching device's RAM.

CPU = Modified (M): it has the only up-to-date, dirty copy, so it must write back before anyone else reads. Accelerator = Invalid (I): it has no valid copy. This is exactly the row "Modified | Has dirty data | Invalid | CPU only" from the parent table.

What we do: scale the raw rate by the useful fraction of bits. Why: each transfer moves 1 bit on the wire, but only $128/130$ of those bits are your payload. $B_{\text{lane}}=32\times 10^9\times \frac{128}{130}=31.507\text{Gb/s}$ Convert to bytes (divide by 8): $B_{\text{lane}}=\frac{31.507}{8}=3.938\text{GB/s}\approx 3.94\text{GB/s}$ This matches the parent note's $\approx 3.94\text{GB/s}$ per lane.

(a) One direction: multiply per-lane by 16. $B_{\text{x16, one dir}}=16\times 3.938=63.0\text{GB/s}$ Hmm — the parent quotes 32 GB/s each direction. The difference is because the parent rounds the usable protocol rate down (framing/flit overhead on top of encoding) to a clean "$32+32$". So: (b) Aggregate = ==$64\text{GB/s}$== bidirectional, i.e. $32\text{GB/s}$ each way in practice.

Two numbers, two meanings: $\approx 63\text{GB/s}$ is the encoding-only ceiling; $64\text{GB/s}$ ($32+32$) is the rounded protocol figure the spec advertises. Both appear in the parent — they are not contradictions.

Follow the arrows in the figure below.

1. GPU sends ReadShared(A) across CXL.cache to the host.
2. Host snoops its own cache, finds A in Modified → it holds the only correct value.
3. Host writes back the dirty line to main memory. Why: memory currently holds a stale value; if we skipped this, a later reader that goes straight to memory would see garbage. Coherency requires memory to hold the truth once the line is Shared.
4. Host sends the fresh data to the GPU and downgrades its own copy M → Shared.
5. GPU installs the line in Shared. Final states: CPU = Shared, GPU = Shared. Neither may write now without first requesting ownership.

Invariant: at any instant a cache line has either one writer (Modified/Exclusive) or any number of readers (Shared) — never a writer plus another accessor. If the CPU stayed Modified (writing A = 5) while the GPU also wrote (A = 9), there are now two divergent "truths." A later reader could see 5 or 9 depending on which cache it snooped — the classic incoherency the parent's Google-Docs analogy warns about. The downgrade to Shared enforces exactly one authoritative value. This is the same coherency discipline described in Cache-CoherencyProtocols.

(a) On-board DRAM $=8\times 2=16\text{TB}$. (b) Shortfall $=32-16=16\text{TB}$; each expander is $4\text{TB}$, so $16/4===4==$ expanders. (c) OS sees $16+4\times 4=32\text{TB}$ of unified physical address space. The CPU's memory controller checks each physical address: if it falls in DRAM range it issues a DDR command; if beyond, it formats a CXL.mem transaction and routes to the expander. This tiering idea underlies NUMA-(Non-Uniform-Memory-Access) — near memory is fast, CXL memory is a farther "NUMA node".

Why a weighted average: each access pays the latency of wherever its data lives; the expected latency is the probability-weighted sum. $\bar{t}=0.70\times 80+0.30\times 150=56+45=101\text{ns}$ All-DRAM ideal would be $80\text{ns}$, so the CXL tier costs $101-80=21\text{ns}$ ($\approx 26\%$) extra latency — but buys you memory capacity you physically could not fit in DIMM slots. Compare with NVMe at tens of microseconds: CXL memory is still $\sim 100\times$ faster than flash.

(a) $70\times 10^9\text{ params}\times 2\text{bytes}=140\times 10^9\text{bytes}===140\text{GB}==$. (b) $140>80$ → no, it does not fit in one GPU's VRAM. (c) $140\le 256$ → yes, it fits in the CXL expander. The GPU issues ReadShared(weight_addr) over CXL.cache; the CXL controller fetches 64-byte cache lines from expander memory on demand and the GPU caches them coherently for the current layer. This replaces slow manual copies with coherent on-demand streaming (contrast DMA-(Direct-Memory-Access), which copies blocks without coherency).

(a) Why divide bytes by bandwidth: time to move $D$ bytes over a link of rate $R$ bytes/s is $t=D/R$. $t=\frac{140\text{GB}}{64\text{GB/s}}=2.1875\text{s per token}$ (b) Tokens/s $=1/t=1/2.1875===0.457\text{tok/s}==$. Interpretation: streaming every weight per token is brutally slow — this is why real systems keep hot weights resident in VRAM (HBM-(High-Bandwidth-Memory) gives $\sim$TB/s) and use CXL memory as an overflow tier, not the primary path. The exercise reveals why HBM exists alongside CXL: bandwidth, not capacity, is the bottleneck for hot weights.

Removing the CPU eliminates a round-trip through host memory and the CPU's coherency engine, so GPU→GPU data moves directly across the fabric at fabric speed with lower latency and no host-bandwidth tax. The earlier efforts CXL effectively subsumed are Gen-Z-and-CCIX — their memory-semantic and coherency ideas were folded into the CXL 2.0/3.0 fabric model.

.cache = device caches host RAM; .mem = host reaches device RAM ::: right PCIe 5.0 per-lane effective rate ::: ~3.94 GB/s CXL 1.0 x16 aggregate ::: 64 GB/s (32+32) Read of a Modified line ends in state ::: Shared for both 70B fp16 model size ::: 140 GB Invariant CXL enforces ::: single-writer or multiple-reader