NUMA architectures: Visual walkthrough — Hinglish deep dive

6.1.7 · D2 · Hardware › Parallelism & Multicore › NUMA architectures

Yeh page parent note ka headline result — NUMA factor aur average-latency formula $t_{a v g} = t_{l oc a l} [(1 - f) + f \cdot F_{N U M A}]$ — rebuild karta hai, ek single memory request aur ek stopwatch se shuru karke. Hum assume karte hain ki aapko sirf itna pata hai ki ek processor kabhi-kabhi memory se koi number maangta hai. Baaki sab kuch draw kiya gaya hai.

Agar koi word yahan heavy lage, toh parent NUMA architectures note mein wider context hai; caches aur coherence ke prerequisite ideas memory consistency models aur multicore synchronization mein hain.

Step 1 — "Memory access time" ka matlab kya hai

KYA HAI. Ek processor ek aisi machine hai jo, har second mein kayi billions baar kehti hai "mujhe address $A$ par stored number do." Maangne aur milne ke beech ka time ek memory access hai. Hum us duration ko ek naam dete hain: $t$ , nanoseconds (ns) mein measure kiya jaata hai — second ka arb-waaan hissa.

YE KYUN shuru karein. Baad ke har symbol ko stopwatch readings se banaya gaya hai. Agar hum kabhi nail nahi karte "hum kya time kar rahe hain," toh $F_{N U M A}$ jaisi ratios meaningless hain. Toh sabse pehla object ek single arrow hai ek core se ek memory bank tak, uspe clock laga hua.

PICTURE. Ek core, ek memory bank, ek request jaati hui aur data wapas aata hua. Green bar stopwatch par elapsed time hai.

Step 2 — Do tarah ke requests: local aur remote

KYA HAI. Ek NUMA machine mein memory ko nodes mein kaata jaata hai. Har node ek core-group hai jo apni nearby memory bank se chipka hua hai. Ek request aapke apne node mein ja sakti hai (chhota safar) ya kisi aur ke node mein (interconnect wire ke paar lamba safar). Same operation, do bahut alag distances.

DO naam kyun. Ek single symbol $t$ ab reality describe nahi kar sakta, kyunki same core alag-alag times dekhta hai is baat par depend karte hue ki data kahaan rakha hai. Jis moment ek number do outcomes mein split ho jaata hai, hum har outcome ko apna naam dete hain — warna hum unhe compare nahi kar sakte.

$t_{l oc a l} = core se apne memory controller tak t_{p r oc \to co n t r o l l er} + memory chips padhna t_{D R A M}$

$t_{p r oc \to co n t r o l l er}$ — core se memory controller tak chhoti hop, jo same node mein baitha hai.
$t_{D R A M}$ — DRAM chips se bits nikalne ka physical time. Unavoidable, har jagah same.

$t_{r e m o t e} = upar wala sab, ab bhi hota hai t_{l oc a l} + doosre node tak wire cross karna t_{in t er co nn ec t}$

$t_{in t er co nn ec t}$ — apna node chhodne ka extra toll: interconnect par (QPI / Infinity Fabric), door node ke controller tak, aur data ka return trip. Yeh term local access ke liye zero hai aur remote ke liye positive.

PICTURE. Same core se do arrows: ek chhota green waala apne bank mein, ek lamba red waala interconnect ke paar neighbour ke bank mein. Red arrow visibly lamba hai — woh length hi $t_{in t er co nn ec t}$ hai.

Step 3 — "Kitna slow hai" ko ek clean number mein badalna: $F_{N U M A}$

KYA HAI. Hum raw nanoseconds ki utni parwah nahi karte jitni ratio ki: remote trip local se kitne times slow hai? $t_{r e m o t e}$ ko $t_{l oc a l}$ se divide karo aur result ko ek naam do.

$F_{N U M A} = \frac{t _{r e m o t e}}{t _{l oc a l}}$

Bar ka matlab divide hai. Remote upar hai aur local neeche, taaki answer padha jaaye "remote itne-many times ek local access hai."
Agar remote $=$ local, toh fraction $\frac{80}{80} = 1$ hai. Toh $F_{N U M A} = 1$ matlab "bilkul koi penalty nahi" — ek plain UMA machine.
Agar remote double slow hai ( $160$ vs $80$ ), toh $F_{N U M A} = \frac{160}{80} = 2$ .

RATIO kyun, subtraction nahi? Hum $t_{r e m o t e} - t_{l oc a l}$ likh sakte the, lekin woh ek raw duration hai jo ek machine ki clock speed se bandi hai. Ek ratio unit-free hoti hai — woh alag-alag machines mein survive karti hai aur hume kehne deti hai "is box ka NUMA factor 1.8 hai" ek single portable quality score ke roop mein. Yahi woh exact sawaal hai jiska hume jawaab chahiye: yahan remote kitna bura hai?

PICTURE. Latencies ki ek number line jisme local mark $1 \times$ par hai aur remote marks dahine taraf push hue hain; horizontal gap, ek multiplier ke roop mein expressed, $F_{N U M A}$ hai.

Step 4 — Real programs dono ko mix karte hain: fraction $f$ introduce karo

KYA HAI. Ek running program lakhs accesses karta hai. Kuch local memory hit karte hain, kuch remote jaate hain. $f$ ko woh fraction maano jo remote jaate hain — $0$ aur $1$ ke beech ki ek number.

$f = \frac{remote accesses ki number}{total accesses}$

$f = 0$ → har access local hai (perfect locality, sapna).
$f = 1$ → har access remote hai (worst case).
$f = 0.2$ → paanch mein se ek access interconnect cross karta hai (parent note ka 80/20 example: 80% local, toh $f = 0.2$ ).

FRACTION kyun? Kyunki reality ek mixture hai, sab-local ya sab-remote nahi. Poore program ka time predict karne ke liye hume jaanna hai ki kitni baar har tarah ka hota hai. Ek single fraction $f$ "meri locality kitni leaky hai" ko ek dial mein capture karta hai jise hum ghuma sakte hain.

PICTURE. 100 tick marks ki ek bar; unka ek fraction $f$ red (remote) rang ka, baaki green (local). $f$ slide karo aur bar recolour ho jaata hai.

Step 5 — Averaging: weighted mean jo $t_{a v g}$ deta hai

KYA HAI. Agar $(1 - f)$ fraction accesses $t_{l oc a l}$ cost karte hain aur $f$ fraction $t_{r e m o t e}$ , toh typical access time unka weighted average hai — har cost ko kitni baar hota hai ussse multiply karo, phir add karo:

$t_{a v g} = jo share local hai (1 - f) t_{l oc a l} + jo share remote hai f t_{r e m o t e}$

WEIGHTING kyun, plain average nahi? Plain average $\frac{t _{l oc a l} + t _{r e m o t e}}{2}$ pretend karta ki dono equally often hote hain. Hote nahi — aapke paas 95% local aur 5% remote ho sakta hai. Har cost ko uski actual frequency se weight karna long-run mean paane ka ek hi honest tarika hai.

Ab substitute karo Step-2 ka fact $t_{r e m o t e} = t_{l oc a l} \cdot F_{N U M A}$ (jo sirf Step 3 ko rearrange kiya hai) sab kuch $t_{l oc a l}$ mein collect karne ke liye:

$t_{a v g} = (1 - f) t_{l oc a l} + f (t_{l oc a l} F_{N U M A}) = t_{l oc a l} [(1 - f) + f F_{N U M A}]$

$(1 - f)$ — local slice, har ek $1 \times$ pay karta hai.
$f F_{N U M A}$ — remote slice, har ek $F_{N U M A} \times$ pay karta hai.
$t_{l oc a l} [\dots]$ — $t_{l oc a l}$ factor out front taaki bracket ek pure multiplier ho jo bataye ki kitne local-latencies ek average access costs karta hai.

PICTURE. Ek see-saw / weighted-beam: local weight $1 \times$ par aur remote weight $F_{N U M A} \times$ par, balance point bracket ki value par land karta hua.

Step 6 — Speedup: locality aapko kitna buy karti hai

KYA HAI. Apne mixed program ko disaster baseline se compare karo jahan har access remote hai, $t_{a l l_r e m o t e} = t_{l oc a l} \cdot F_{N U M A}$ . Speedup $S$ kitne times faster mixed program hai:

$S = \frac{t _{a l l_r e m o t e}}{t _{a v g}} = \frac{t _{l oc a l} F _{N U M A}}{t _{l oc a l} [ ( 1 - f ) + f F _{N U M A} ]}$

Upar aur neeche ka $t_{l oc a l}$ cancel ho jaata hai (ek common factor divide ho jaata hai), ek clean, machine-independent expression reh jaata hai:

$S = \frac{F _{N U M A}}{( 1 - f ) + f F _{N U M A}}$

Numerator $F_{N U M A}$ — all-remote kitna bura hai.
Denominator — Step 5 ka bracket — aap actually kitne bure hain.
Bada $S$ = aapki locality ki mehnat worst case ke relative mein rang layi.

ALL-REMOTE case se kyun divide karein? Kyunki yeh locality ki value ko hi isolate karta hai. Yeh jawaab deta hai "ek NUMA-oblivious program ke comparison mein jo interconnect par bhatakta rehta hai, data ko local rakhne se mujhe kitna mila?"

PICTURE. $S$ ka curve versus $f$ : $f = 1$ par curve $1 \times$ par baitha hai (aap hi baseline hain); jaise $f \to 0$ woh $F_{N U M A}$ ki taraf climb karta hai (all-local poora factor faster hai).

Step 7 — Edge cases (reader ko kabhi stranded mat chhhodo)

KYA HAI & KYU. Ek formula jis par aap trust karte ho use apne extremes mein survive karna chahiye. $f$ aur $F_{N U M A}$ ko unki limits tak push karte hain aur confirm karte hain ki har jawaab sane hai.

Case	Plug in	Result	Meaning
Perfect UMA ( $F_{N U M A} = 1$ )	$S = \frac{1}{( 1 - f ) + f}$	$S = 1$ kisi bhi $f$ ke liye	Koi penalty exist nahi karti, toh locality help nahi kar sakti — sahi hai.
All local ( $f = 0$ )	$S = \frac{F _{N U M A}}{1 + 0}$	$S = F_{N U M A}$	Aap all-remote baseline ko poore factor se beat karte ho.
All remote ( $f = 1$ )	$S = \frac{F _{N U M A}}{0 + F _{N U M A}}$	$S = 1$	Aap hi baseline ho — koi speedup nahi, jaisa expected tha.
Extreme NUMA ( $F_{N U M A} \to \infty$ )	$S = \frac{F _{N U M A}}{( 1 - f ) + f F _{N U M A}} \to \frac{1}{f}$	$S \to 1/ f$	Speedup aapke remote fraction se cap hoti hai — ek hard Amdahl-style ceiling.

$1/ f$ ceiling punchline hai: infinitely fast local memory bhi aapko rescue nahi kar sakti agar $f$ fraction accesses ab bhi interconnect par crawl karte hain. Aapka remote traffic anchor hai.

PICTURE. Speedup curve phir se, is baar $S = 1/ f$ ceiling ek dashed line ke roop mein draw ki hui jiske against curve press karti hai jaise $F_{N U M A}$ badhta hai.

Ek-picture summary

Upar wala sab kuch, ek canvas par: ek request local (green, $\times 1$ ) aur remote (red, $\times F_{N U M A}$ ) mein split hoti hai; fraction $f$ unhe ek weighted beam par mix karke $t_{a v g}$ banata hai; all-remote baseline ko $t_{a v g}$ se divide karne par speedup curve milti hai apni $1/ f$ ceiling ke saath.

Recall Feynman retelling — plain words mein wapas bolein

Ek computer kabhi-kabhi numbers fetch karta hai memory se jo uske bilkul paas hai (fast) aur kabhi-kabhi memory se jo ek wire ke paar doosre kamre mein hai (slow). Hum ek fetch time karte hain aur use $t$ kehte hain. Kyunki same core ko do alag times dikhte hain, hum $t$ ko $t_{l oc a l}$ aur $t_{r e m o t e}$ mein split karte hain, jahan remote sirf local plus wire cross karne ka toll hai. "Remote kitna worse hai" ko ek portable number mein describe karne ke liye hum unhe divide karte hain: $F_{N U M A}$ . Ek real program mixture karta hai, toh hum $f$ track karte hain, un fetches ka share jo remote jaate hain. Typical fetch time do costs ka weighted average hai is hisaab se ki har ek kitni baar hota hai — woh weighted average, tidy karke, $t_{a v g} = t_{l oc a l} [(1 - f) + f F_{N U M A}]$ hai. Ise ek aisi program se compare karna jo har baar remote jaati hai ek speedup deta hai $S = F_{N U M A} / [(1 - f) + f F_{N U M A}]$ . Extremes test karne se pata chalta hai yeh theek behave karta hai: koi penalty nahi matlab koi speedup nahi, all-local matlab full speedup, aur — key lesson — infinitely fast local memory bhi aapko $1/ f$ par cap karta hai. Toh poora game $f$ ghatana hai: apna data wahan raho jahan aapka thread rehta hai.

Recall Quick self-test

$f$ ka kya matlab hai, aur "80% local" use kya value deta hai? ::: $f$ un accesses ka fraction hai jo remote hain; 80% local ka matlab $f = 0.2$ hai. $F_{N U M A}$ ratio kyun use karein difference $t_{r e m o t e} - t_{l oc a l}$ ki jagah? ::: Ratio unit-free aur machines mein portable hai; difference ek machine ki clock se bandi hai. $f = 0.2$ , $F_{N U M A} = 2$ ke saath speedup kya hai? ::: $S = 2/ (0.8 + 0.4) = 1.67 \times$ . Jab $F_{N U M A} \to \infty$ , speedup ceiling kya hai? ::: $S \to 1/ f$ — aapka remote fraction hard limit hai.