6.3.11 · D5 · HinglishInterconnects, Buses & SoC

Question bankInfinity Fabric - mesh interconnects

3,597 words16 min read↑ Read in English

6.3.11 · D5 · Hardware › Interconnects, Buses & SoC › Infinity Fabric - mesh interconnects

Yeh parent topic ka "gotcha" companion hai. Neeche har line mein ek trap chupi hai — ek aisi jagah jahan ek reasonable-lagte belief actually galat hoti hai. Answer ko cover karo, apna guess commit karo, phir reveal karo. Har answer tumhe reason deta hai, sirf verdict nahi.

Is page par use hone wale symbols aur words

Traps se pehle, har symbol ko earn karna zaroori hai. Yeh page sirf inhi ka use karta hai:

Figure — Infinity Fabric  -  mesh interconnects

Upar ke chaar pictures dekho. Ek bus (sabse baayi taraf) ek shared black wire hai — sab apni baari ka intezaar karte hain. Ek ring nodes ko ek loop mein jodata hai. Ek mesh ek grid hai jahan inner nodes ke chaar neighbours hote hain. Ek torus (sabse daayein, red wrap-around links) ek mesh hai jiske edges opposite side se wrap around ho jaate hain, isliye har node ke chaar neighbours hote hain — koi edge tiles nahi. Har question ke liye yeh picture dimag mein rakho.

Traps se pehle do aur mechanisms jo tumhe jaanne chahiye

Kuch answers neeche do hardware tricks par lean karte hain. Pehle inhe yahaan earn karo.

Prerequisite pages ki ek-nazar reminder

Tumhe yeh page chhodni nahi hai, lekin har linked topic kya contribute karta hai yeh raha:

Recall Prerequisite links tumhe kya dete hain
  • 6.3.1-Bus-ArchitectureBasics ::: kyun ek single shared wire traffic ko serialize karta hai aur electrical-loading wall se takraata hai.
  • 6.3.5-Cache-Coherency-Protocols ::: MOESI-style states aur broadcast snooping aur directory tracking ke beech ka farq.
  • 6.2.8-NUMA-Architecture ::: kyun "near memory" aur "far memory" ki latencies alag hoti hain — wahi effect jo chiplets create karte hain.
  • 7.2.4-Network-Topologies ::: ring / mesh / torus / hop-count aur bisection-bandwidth vocabulary.
  • 6.4.3-PCIe-Topology aur 5.1.7-Memory-BandwidthCalculation ::: jahaan fabric bandwidth I/O aur DRAM limits se milti hai.

"Grows like" ka matlab — ek paragraph mein Big-O

Kuch answers compare karte hain ki machine bade hone par traffic kaise grow karta hai. Hum yeh Big-O notation se likhte hain: matlab "amount ke direct proportion mein grow karta hai" (cores double karo, kaam double ho jaata hai), aur matlab "amount ke proportion mein grow karta hai" aur jo grow nahi karta use ignore karta hai. Yeh kahe ka ek tarika hai ki cheez kaise scale karti hai, exact count nahi. Isliye "broadcast snooping hai" matlab uska kaam poore core count ke saath barhta hai, jabki "ek directory hai" matlab uska kaam sirf (chote) sharer count ke saath barhta hai.


Do hop-count formulas derive karna (taaki recall lines earned ho)

Neeche ke traps aur quote karte hain. Step by step, picture ke saath, yeh hai kyun woh averages aate hain.

Figure — Infinity Fabric  -  mesh interconnects

"16 tiles ⇒ 16 links" error ka apna picture deserve karta hai, kyunki real count bhi nahi hai — woh formula missing edge links ko ignore karta hai.

Figure — Infinity Fabric  -  mesh interconnects

True or false — justify karo

Ek shared bus har core ko ek saath communicate karne deta hai jab tak wire fast ho
False. Ek bus ek shared medium hai — clock chahe kitni bhi ho ek waqt mein sirf ek transaction hoti hai; arbitration cores ko baari-baari lene par majboor karti hai, isliye cores add karne se parallel traffic nahi, waiting add hoti hai.
Infinity Fabric ek single fixed topology hai jo har AMD chip mein baked in hai
False. Yeh topology-agnostic hai: wahi protocol chiplet ke andar rings banata hai, ek central I/O die ki taraf star-like spokes, ya GPU dies ke beech full meshes — product par depend karta hai.
Mesh hamesha ring se kisi bhi core count par latency mein beat karta hai
False. Chote number of cores ke liye (jaise 4–8) ring ki short circumference uske average hops comparable bana deti hai, aur mesh ke extra router ports thode gain ke liye area aur power cost karte hain — isliye rings ~10–12 cores tak survive karti raheen.
Ek torus aur ek plain mesh ki worst-case latency same hoti hai
False. Ek torus apne edges wrap karta hai, isliye opposite corners wrap link se neighbours hote hain — uski worst-case distance roughly half ho jaati hai plain mesh ke mukable, lekin physically longer wires ki cost par jo chip par route karne mushkil hote hain.
Cache coherency aur data transport fabric par same kaam hai
False. Transport (flits ko point-to-point move karna) aur coherency (decide karna ki kaun ek line hold kar sakta hai) alag layers hain; fabric coherency messages carry karta hai lekin Scalable Coherent Protocol woh logic hai jo decide karta hai woh messages kya hain.
Fabric Clock (FCLK) aur memory clock (MCLK) hamesha equal hone chahiye
False. Woh ek ratio par chalte hain — 1:1, 1:2, etc. 1:1 ratio (coupled mode) synchronizer crossing se bachata hai aur latency kam karta hai; 2:1 memory ko fabric se faster chalne deta hai, latency cost ke saath.
Directory-based protocol har write par har core ko probe bhejta hai
False. Yeh broadcast snooping describe karta hai. Ek directory per line sharer list track karta hai aur sirf un cores ko probe karta hai jo actually ek copy hold kar rahe hain — typically 1–3, sab nahi.
Point-to-point links ka matlab hai data kabhi buffer nahi hoga
False. Packet-switched links credit-based flow control aur per-router buffers use karte hain; ek sender tab hi flit transmit kar sakta hai jab uske paas ek free receiver slot ka credit ho, warna ruk jaata hai.
Bisection bandwidth measure karta hai ki ek link kitna fast hai
False. Yeh measure karta hai ki network ko do halves mein split karne ke liye kitne links kaatne padte hain — ek network-wide capacity, single-link speed nahi. Yeh worst-case cross-chip traffic limits reveal karta hai.
EPYC Rome par, ek core jo doosre chiplet ki L3 reach karta hai utni hi fast hai jitni apni L3 reach karna
False. Cross-chiplet access central I/O die tak hop out karni padti hai aur wapas aana padta hai, isliye yeh designs exactly NUMA-like distance effects expose karte hain (dekho 6.2.8-NUMA-Architecture).
Mesh mein zyada links automatically kisi bhi single destination par zyada usable bandwidth deta hai
False. Ek single source-destination flow phir bhi ek waqt mein ek path use karta hai; extra links aggregate traffic help karte hain aur alternate routes dete hain, koi wider single stream nahi.

Error pakdo

"Average ring hops hain, kyunki ek message poore ring ke around travel kar sakta hai."
Error: tum hamesha ek bidirectional ring par shorter direction lete ho, trip ko par cap karte ho aur average lagbhag hota hai. assume karna long way double-count karta hai jo tum kabhi use nahi karte.
"Ek 4×4 mesh mein 16 links hain kyunki usmein 16 tiles hain."
Error: links aur tiles alag count hote hain. Direction se count karo — 12 horizontal 12 vertical links, yaani . "" shortcut un edges ko over-count karta hai jo corner tiles ke paas nahi hain (upar link-count figure dekho).
"Directory coherency saari snoop traffic hata deta hai, isliye yeh free hai."
Error: yeh broadcast snoop traffic hata deta hai, sab nahi. Home node phir bhi sharers ko targeted probes bhejta hai aur har access par look up karna padta hai — cost sharer count ke saath scale hoti hai ( = sharers), zero nahi.
"I/O die sirf ek passive wiring hub hai jismein koi logic nahi hai."
Error: Zen 2+ par I/O die memory controllers aur inter-chiplet fabric switching host karta hai; yeh ek active routing aur coherency participant hai, passive copper nahi.
"Kyunki links dedicated point-to-point hain, fabric kabhi deadlock nahi kar sakta."
Error: dedicated links cyclic buffer dependencies ko prevent nahi karte. Routers multiple virtual channels specifically un cycles ko todne ke liye use karte hain — exactly kaise, yeh neeche deadlock-cycle figure mein dekho.
"FSB (Front-Side Bus) bandwidth kam hai kyunki wire patla hai; wider bus sab kuch fix kar deta hai."
Error: real limit electrical loading hai — har attached core capacitance add karta hai, clock cap kar deta hai — plus single-transaction arbitration. Bits wide karna koi bhi problem nahi hata.
"XY dimension-ordered routing kabhi kabhi long way leta hai, isliye yeh hops waste karta hai."
Error: XY routing hamesha ek minimal (Manhattan) path leta hai — X-distance plus Y-distance. Yeh restrict karta hai ki tum kaun sa minimal path lete ho (deadlock-free), length kabhi nahi — neeche routing figure dekho.

Do subtle mechanisms visualize karna

Deadlock kyun banta hai, aur virtual channels use kyun todte hain

Figure — Infinity Fabric  -  mesh interconnects

XY routing hops kabhi waste kyun nahi karta, aur kaun se turns yeh forbid karta hai

Figure — Infinity Fabric  -  mesh interconnects

Why questions

Directory broadcast snooping se behtar kyun scale karta hai jab core count barhta hai
Broadcast har request har core ko bhejta hai, isliye traffic ke roop mein grow karta hai (poore core count ke proportional). Ek directory sirf actual sharers ko probe karta hai, isliye traffic ke roop mein grow karta hai jahan = sharers (1–3) roughly constant rehta hai se regardless.
Intel ne Skylake-X ke aas-paas ring bus kyun abandon kiya
Ek ring par, latency core count ke saath badti hai (average hops) aur ring ki shared bandwidth ~10–12 cores ke baad saturate ho jaati hai; ek 2D mesh har node ko multiple shorter paths deta hai, latency aur bisection bandwidth ko control mein rakhte hue.
Ryzen desktop ke liye 1:1 FCLK-to-MCLK ratio usually kyun recommend kiya jaata hai
1:1 (coupled) ratio fabric aur memory ke beech ek asynchronous clock-domain crossing remove kar deta hai, round-trip latency cut karta hai; memory ko fabric se faster chalana (2:1) woh crossing penalty wapas lata hai.
5800X par fabric bandwidth (≈57.6 GB/s ek direction) dual-channel DDR4-3600 se match kyun karta hai
Design se. Agar fabric memory se slower hota toh yeh memory reads bottleneck karta; unhe match karna (1:1 ratio) ensure karta hai ki interconnect kabhi DIMMs aur cores ke beech limiting stage na bane (dekho 5.1.7-Memory-BandwidthCalculation).
Coherency protocol mein "home node" ki zaroorat kyun hai
Home node (woh L3 slice jo ek line ki address range ki owner hai) woh single serialization point hai jo directory hold karta hai, isliye us line ke sab requests ek hi jagah se route hote hain aur ek consistent order paate hain — do cores ko line ki state ke baare mein silently disagree karne se rok ta hai.
Chiplet designs ek bade die banana ki jagah extra cross-die latency kyun accept karte hain
Chote chiplets mein manufacturing yield kaafi zyada hoti hai aur cost kam hoti hai; ek single 64-core monolithic die bahut bada aur defect-prone hoga. Latency penalty cheaper, more scalable silicon ke liye pay ki gayi keemat hai.
Packet-switching poori cache line ek unit mein bhejne ki jagah flits kyun use karta hai
Flits (flow-control units) mein split karne se flow control aur buffering chote pieces par operate karte hain, isliye ek router ko sirf chote buffers chahiye aur woh ek message ko kai hops par pipeline kar sakta hai, har stop par use poora store karne ki jagah.

Edge cases

Jab source aur destination same tile par hote hain toh mesh par average hop count kya hota hai
Zero hops — ek core jo apna local L2 slice reach karta hai router network mein kabhi enter nahi karta. average mein yeh zero-distance case shamil hai, isliye average maximum se neeche hai.
4×4 mesh par worst-case (average nahi) hop count kya hai, aur yeh kahaan hota hai
6 hops, ek corner se opposite corner tak: Manhattan distance . Average () is maximum ko hide karta hai, jo tail latency ke liye matter karta hai.
Us 4×4 mesh ko torus mein convert karne se worst case kaise change hota hai
Wrap-around links aur ko sirf hops apart bana dete hain (X mein ek wrap step aur Y mein ek), isliye worst case 6 se ghat kar lagbhag 4 hops ho jaata hai — torus ka edge-case advantage jo ek plain mesh match nahi kar sakta.
Single-chiplet (monolithic) desktop Zen 4 part ki inter-chiplet latency kya hoti hai
Koi nahi hoti — sirf ek core die hone se, sab traffic internal fabric par rehti hai aur kabhi xGMI link cross nahi karta, isliye us part ke liye cross-chiplet NUMA penalty simply exist hi nahi karta.
Agar do cores ek saath same line request karein, toh tie kaun tota hai
Home node unhe serialize karta hai: jis bhi request ko home ki directory logic pehle order karta hai use pehle handle kiya jaata hai, aur doosre ko baad mein aaya hua treat kiya jaata hai — kabhi bhi ek sachi simultaneous grant nahi hoti.
ke degenerate case par ring aur mesh ki bisection bandwidth kya hai
Dono collapse ho jaate hain: ek 2-node "ring" sirf ek bidirectional link hai, aur ek mesh well-defined bhi nahi hai. Mesh ka advantage tabhi appear hota hai jab itna bada ho ki ek real grid form ho sake.
Agar do cores ek saath same line request karein, toh tie kaun tota hai
Home node unhe serialize karta hai: jis bhi request ko home ki directory logic pehle order karta hai woh jeetta hai, aur doosre ko baad mein aaya hua treat kiya jaata hai — kabhi bhi ek sachi simultaneous grant nahi hoti.
Strict XY routing ke under mesh ke alternate-path advantage ka kya hota hai
Ek fixed source-destination pair ke liye zero benefit milta hai — aisi har packet wahi fixed path leti hai, isliye us path par ek hotspot ko route nahi kiya ja sakta chahe doosre links idle kyun na padhe hon.

Recall Quick self-test

aur ka matlab kya hai ::: = nodes ki sankhya; = mesh side length isliye (aur alag se = coherency baat mein sharers ki sankhya). Ring average hops formula aur kyun ::: — shorter-arc distances ka average, jinka midpoint hai. grid par mesh average hops aur kyun ::: — expected gap per axis (from scaled by ), do axes add kiye. 4×4 mesh ka real link count ::: 24, se; ek torus wrap links add karke reach karta hai. Credit kya hai ::: ek token jiska matlab "ek free receiver buffer slot"; sender ek per flit spend karta hai aur khatam hone par ruk jaata hai. Virtual channels kyun exist karte hain ::: ek wire par separate buffer lanes jo blocked flits ko alternate lane lene deti hain, deadlock cycles todte hue. Big-O ka matlab ::: shorthand ki koi quantity kaise scale karti hai — poore node count ke saath grow karta hai, sirf chote sharer count ke saath.