CXL (Compute Express Link): Visual walkthrough — Hinglish deep dive

6.3.5 · D2 · Hardware › Interconnects, Buses & SoC › CXL (Compute Express Link)

Hum ideas lenge CXL (Compute Express Link) (the parent) se, aur Cache-CoherencyProtocols, DMA-(Direct-Memory-Access), HBM-(High-Bandwidth-Memory), NUMA-(Non-Uniform-Memory-Access) aur Gen-Z-and-CCIX ko jahaan zaroorat ho wahaan touch karenge.

Step 1 — Ek wire jo flip karti hai: "transfer" ka matlab kya hai

WHAT. CXL link, bilkul bottom pe, copper wires ka ek pair hai — ek lane. Ek wire ek voltage carry karti hai jise sender "high" aur "low" ke beech flip karta hai. Receiver, ek fixed doori par, uss voltage ko ek clock tick par sample karta hai aur use 1 ya 0 kehta hai.

WHY yahan se shuru karein. Parent table mein har bandwidth number bas itna hai ki "ye kitne flips per second hain, minus wo flips jo hum error-checking par waste karte hain." Agar tum ek wire ko flipping samajh lo, tum poora table samajh jaate ho. Kuch aur chahiye hi nahi.

PICTURE. Neeche square wave dekho. Har vertical jump wire ka state change karna hai. Neeche chhote clock ticks wo moments hain jab receiver dekhta hai.

Step 2 — PCIe 5.0 ki raw rate: 32 GT/s per lane

WHAT. PCIe 5.0 electrical spec wire ki flip rate ko $32 GT/s$ par pin karta hai. Yahi raw number hai jahan se har CXL 1.0/2.0 link shuru hoti hai.

WHY yeh number aur "bas aur tez" kyun nahi. Rate physics ke wajah se capped hai: flips ko aur tez karo toh copper sharp edges ko blur kar deta hai (signal blur ho jaata hai, high aur low mil jaate hain). $32 GT/s$ wo sabse tez rate hai jo standard ko trust hai ki receiver ek real motherboard trace ke upar ab bhi high aur low mein farq kar sake. CXL is exact electrical layer ko reuse karta hai — poora point yehi hai, yeh PCIe ki wire par ride karta hai (compare karo Gen-Z-and-CCIX se, jisne nayi fabrics define karne ki koshish ki).

PICTURE. Step 1 jaisi wire, lekin ab ticks $32$ billion per second packed hain. Edges par blur isliye hai kyun hum aur upar nahi ja sakte.

Step 3 — Tax: 128b/130b encoding

WHAT. Sender tumhare 128 bits akele nahi bhejta. Woh har 128 bits ke liye 2 extra bits ("sync header") jod deta hai, jo ek 130-bit chunk banaata hai. Toh har 130 transfers mein se sirf 128 tumhara data carry karte hain.

WHY yeh 2 bits. Receiver ko jaanna chahiye kahan ek chunk shuru hoti hai aur noise se bit-flips pakadne chahiye. Woh 2 header bits boundary mark karte hain aur receiver ko apni clock re-lock karne dete hain. Yeh ek tax hai jo tum tab pay karte ho taki link reliable rahe — purane PCIe 3.0 ke "8b/10b" scheme se sasta, jis mein har 10 mein se 2 bits ka tax tha (20%!).

PICTURE. Neeche ka bar ek 130-bit chunk hai. Chhodi pale-yellow part tumhara payload hai; ant mein patla pink sliver 2 overhead bits hai.

Step 4 — Ek lane ki real data rate

WHAT. Raw rate ko efficiency se multiply karo taaki ek lane ki useful data rate mile.

WHY multiply karein. Bandwidth hai "useful bits per second" = (transfers per second) × (fraction jo useful hain). Woh literally $R_{raw} \times η$ hai. Koi aur operation sensible nahi hai — hum bas raw rate ko tax se discount kar rahe hain.

PICTURE. Do stacked bars: ucha wala $32$ raw hai, neeche wala jo tax ke baad bachta hai, $31.5$ .

Step 5 — Road ko wide karo: ×16 lanes

WHAT. Ek CXL link kai lanes ko side by side bundle karta hai. Ek "×16" link 16 identical lanes hain jo ek direction mein parallel mein data carry karti hain. Inhe jodo.

WHY bas addition hai. Lanes independent wires hain jo ek hi time par flip ho rahi hain — koi shared bottleneck nahi — toh total data rate simply 16 copies hai. Yahi "parallel lanes" trick hai jo HBM-(High-Bandwidth-Memory) internally use karta hai, bas silicon interposer ki jagah copper par.

PICTURE. Solah parallel arrows, har ek $3.94 GB/s$ , ek mote highway mein merge hote hue jispe $63 GB/s$ likha hai.

Step 6 — Dono directions: parent ka "64 GB/s"

WHAT. CXL wires full-duplex hain: har lane actually do wire-pairs hain, ek send ke liye, ek receive ke liye, ek saath chalte hain. Toh tumhe ×16 rate har taraf, simultaneously milti hai.

WHY dono count karein. Send aur receive compete nahi karte — alag copper. Marketing tables aksar sum quote karti hain ("aggregate"), aur yehi hai jahan parent ka " $64 GB/s$ ( $32 + 32$ )" aata hai — har direction roughly $\approx 32 GB/s$ rounded.

PICTURE. Do mote highways side by side, opposite arrows, har ek ~ $32 GB/s$ , milake ~ $64$ .

Step 7 — Degenerate cases (kabhi surprised mat hona)

WHAT. Edges par kya hota hai — ek narrow link, ek idle link, ek link jahan CXL coherency ke liye lanes borrow karta hai?

WHY inhe cover karein. Formula $B = N_{lanes} \times R_{raw} \times η$ tab bhi sahi answer dena chahiye jab inputs chhhote ya weird ho jaayein, varna tum ek spec sheet galat padh loge.

×1 link ( $N = 1$ ): $B = 3.94 GB/s$ ek taraf. Formula smoothly degrade hota hai — ek lane base unit hai.
×0 / no lanes trained: link come up karne mein fail ho jaati hai, $B = 0$ . "Aadhi lane" jaisi koi cheez nahi; lane counts sirf $1, 2, 4, 8, 16$ hain.
Idle link: wire phir bhi flip karti rehti hai (woh clock locked rakhne ke liye "idle" filler symbols bhejti hai), toh raw transfers $32 GT/s$ par continue karte hain, lekin useful bandwidth $0$ hai — ek yaad dilana ki GT/s ≠ delivered data.
Coherency traffic (CXL.cache/.mem) wahi lanes share karta hai: Flex Bus protocols ko time-multiplex karta hai, toh agar link ke time slots ka aadha hissa snoop/coherency messages carry karta hai, tumhara data throughput roughly aadha ho jaata hai chahe $B_{agg}$ unchanged ho. Pipe ki width fixed hai; usme kya flow karta hai yeh ek scheduling choice hai.

PICTURE. Ek tick-strip jo wahi fixed-width link dikhata hai teen fills ke saath: full data, idle filler, aur data + coherency snoops ka mix.

Ek-picture summary

Poora derivation ek board par: ek wire → 32 GT/s → 128/130 se tax karo → 3.94 GB/s per lane → ×16 lanes → 63 GB/s ek taraf → ×2 directions → 64 GB/s aggregate, coherency tax us flow ka kuch hissa lekar.

Recall Feynman retelling — zor se bolo

Ek akeli copper wire ki imagine karo jo ek light on aur off kar rahi hai. PCIe 5.0 use 32 billion times per second flick karta hai — yahi "raw" speed hai, ek transfer per flick. Lekin har 130 flicks mein, 2 ek chhote "kya tum abhi bhi wahaan ho?" signal par waste ho jaate hain, toh sirf 128/130 ≈ 98.5% real data hai. Woh lagbhag 31.5 gigabits chhod jaata hai, aur kyunki 8 bits ek byte banaate hain, ek wire lagbhag 3.94 gigabytes per second carry karti hai. Ab 16 aisi wires side by side rakho (ek ×16 link): 16 × 3.94 ≈ 63 GB/s ek taraf ja raha hai. Aur kyunki CXL mein alag send aur receive wires hain, tumhe woh speed dono taraf ek saath milti hai — use jodo aur round karo, aur tum famous ~64 GB/s par pahuncho. Yahi poora parent-note formula hai, ek wiggling wire se bana. Catch yeh hai: CXL cache-coherency chatter un wahi lanes par run karta hai, toh us 64 ka kuch hissa CPU aur accelerator ko sync mein rakhne mein kharcha hota hai — jo exactly woh magic hai jo PCIe akela nahi kar sakta tha.

Recall Quick self-test

Raw PCIe 5.0 per-lane transfer rate? ::: 32 GT/s Encoding efficiency fraction? ::: 128/130 ≈ 0.985 Ek lane ki usable data rate GB/s mein? ::: lagbhag 3.94 GB/s 130/128 ki jagah 128/130 se kyun multiply karte hain? ::: har 130 transfers mein se sirf 128 tumhara data carry karte hain, toh tum ek fraction less than 1 rakhte ho CXL 3.0 ka 128 GB/s tak jump kahaan se aata hai? ::: yeh PCIe 6.0 par ride karta hai, jo har transfer mein ~2× data pack karta hai (PAM4 signalling) Kya CXL PCIe 5.0 se zyada raw GB/s deta hai? ::: nahi — wahi wire, wahi bandwidth; CXL ki jeet coherency aur lower latency hai