NUMA architectures: Question bank — Hinglish deep dive

6.1.7 · D5 · Hardware › Parallelism & Multicore › NUMA architectures

Shuru karne se pehle, ek word jis par hum baar baar lean karte hain: ek node "kuch cores + ek local memory bank + ek memory controller" ka bundle hai. "Local" matlab us core ke jaise hi node ke andar; "remote" matlab ek alag node mein, interconnect ke upar pahuncha hua. Woh picture pakde rahein — neeche har trap usi par hinge karti hai.

Har trap ke peeche ki picture

Neeche ka map ek bhi sawaal padhne se pehle dekh lein. Teen ideas yahan draw kiye gaye hain jinhe baaki page baar baar refer karta rehta hai:

Left panel — char nodes ek interconnect se jude hue. Ek local access (teal) ek hi node ke andar rehta hai; ek remote access (orange) ek ya zyada hops cross karta hai (har link jo aap traverse karte ho woh ek hop hai). Node 3 ko node 0 se node 1 ke through jaake pahuncha jaata hai: woh 2 hops hai, isliye woh slower hai.
Chhoti latency ladder woh concrete measurement hai jise yeh page throughout assume karta hai (ek 4-node machine, 8 cores + 16 GB per node): node 0 = 85 ns (1.0x), node 1 = 140 ns (1.65x, 1 hop), node 2 = 165 ns (1.94x, 2 hops), node 3 = 168 ns (1.98x, 2 hops). Notice karein ki nodes 2 aur 3 almost equal hain — yahi SE4 ka trap hai.

Do aur pictures jinhe hum point back karenge:

Yeh cache-line ping-pong (false sharing) hai. Do alag-alag nodes par do threads alag-alag words likhte rehte hain jo ek hi cache line mein hote hain. Har write poori line ko interconnect ke paar kheench laata hai — arrow aage-peechhe bounce karta hai, har bounce par saikdon nanoseconds kharach hote hain.

Yeh average-latency formula ki picture hai. Yeh jawaab deta hai "agar meri $f$ fraction accesses remote hain, toh average par mujhe kaisi latency feel hoti hai?" Hum literally accesses ki ek bar ko ek local part aur ek remote part mein split karte hain:

$t_{a v g} = the local slice (1 - f) t_{l oc a l} + the remote slice f t_{r e m o t e} .$

Ab NUMA factor $F_{N U M A} = t_{r e m o t e} / t_{l oc a l}$ substitute karein, yani $t_{r e m o t e} = F_{N U M A} t_{l oc a l}$ , aur $t_{l oc a l}$ factor out karein:

$t_{a v g} = t_{l oc a l} [(1 - f) + f F_{N U M A}] .$

Woh bracket sirf "1" (local access ki cost, $t_{l oc a l}$ ki units mein) aur " $F_{N U M A}$ " (remote ki cost) ka ek weighted average hai. Jab $f = 0$ bracket $1$ hai (sab local); jab $f = 1$ woh $F_{N U M A}$ hai (sab remote). Beech ki har cheez figure mein tilted line ke saath slide karti hai. Isliye formula exactly is shape ka hai — yeh remote fraction $f$ se control hone wala ek straight-line blend hai.

True ya false — justify karein

TF1. "Ek NUMA system mein processor dusre node ki memory nahi padh sakta."

False. Woh bilkul padh sakta hai — NUMA ek single shared address space deta hai. Bas zyada latency pay karta hai (interconnect ke upar remote path). NUMA mein "N" Non-uniform hai, No nahi.

TF2. "UMA (uniform access) hamesha NUMA se worse hota hai."

False. Chhote core counts (≈4–8) ke liye UMA aksar simpler aur faster hota hai kyunki koi interconnect hop hi nahi hota — har access ek flat cost hai. NUMA tabhi jeet'ta hai jab single shared bus bandwidth bottleneck ban jaata hai.

TF3. "ccNUMA ka matlab hai ki nodes ke beech baat karne ke liye message-passing code likhna padega."

False. ccNUMA hardware caches coherent aapke liye rakhta hai, isliye ordinary shared-memory loads aur stores nodes ke paar kaam karte hain. Message-passing non-coherent clusters par chahiye, ccNUMA par nahi.

TF4. "Do threads alag-alag nodes par same variable likhna aur do threads same cache line mein different variables likhna ek hi tarah ka slowdown cause karte hain."

Mechanism mein True, cause mein subtle. Dono cache line ko interconnect ke paar ping-pong karte hain (upar figure mein bouncing arrow). Pehla true sharing hai (avoid nahi kar sakte agar woh sach mein data share karte hain); doosra false sharing hai (layout ka ek accident jise padding fix kar sakti hai).

TF5. "Interleaved address mapping, range-based mapping se locality ke liye better hai."

False. Interleaving consecutive addresses ko aggregate bandwidth maximize karne ke liye sab nodes mein scatter karta hai; woh deliberately locality destroy karta hai. Range-based ek contiguous block ko ek node par rakhta hai, jo OS ko thread ko local memory dene deta hai.

TF6. "First-touch allocation tab hoti hai jab memory malloc ki jaati hai."

False. malloc (ya OS ke address range reserve karne se) koi node pick nahi hota. Page physically us node par place hoti hai jis core ka pehla write hota hai. Yahi "first-touch" naam ka poora point hai.

TF7. "NUMA factor

F_{N U M A} = t_{r e m o t e} / t_{l oc a l}

sirf hardware ki property hai."

True — yeh machine ka latency ratio describe karta hai. Lekin aapki achieved average latency

f

par bhi depend karti hai, jo accesses ka woh fraction hai jo remote hain, jo aapke software ki property hai.

TF8. "Agar 100% accesses remote hote, toh NUMA phir bhi UMA ko beat karta."

Depends karta hai, aur latency ke liye usually false hai.

f = 1

ke saath, average latency

= t_{l oc a l} \cdot F_{N U M A}

— strictly local access se worse. NUMA ka bandwidth advantage heavy load ke neeche help kar sakta hai, lekin purely remote workload NUMA ka main benefit throw away kar deta hai.

TF9. "Ek node ke andar cores us node ki apni memory ke liye non-uniform latency dekhte hain."

False. Ek single node ke andar cores same controller share karte hain aur local memory tak uniform latency dekhte hain — ek node effectively ek mini-SMP hai. Non-uniformity sirf nodes ke beech aati hai.

TF10. "Ek baar page first-touch se place ho jaaye, toh woh usi node par hamesha ke liye rehti hai."

False. Modern OSes runtime par nodes ke beech pages migrate kar sakti hain (jaise Linux AutoNUMA), ek hot page ko us node par move karke jiske threads use baar baar touch karte hain. First-touch initial placement set karta hai, permanent nahi.

Error dhundhein

SE1. "Code ko NUMA-friendly banane ke liye, bas har thread ko node 0 par pin kar do."

Error: sab threads ko ek node par pin karna har core ka traffic ek memory controller ke through daba deta hai — aapne UMA bottleneck recreate kar liya aur doosre nodes ki bandwidth bhook se maar di. Threads ko nodes mein spread karein aur har thread ka data uske apne node par rakhein.

SE2. "Humne worker threads spawn karne se pehle main() mein poora array initialize kar liya, toh woh ready hai."

Error: main() ek node par ek thread hai, isliye first-touch poore array ko us ek node par place karti hai. Doosre nodes ke har worker ko remote accesses karne padte hain. Parallel mein initialize karein, har thread ko woh region first-touch karne dein jise woh baad mein use karega.

SE3. "Remote latency = local latency + interconnect latency, aur bus itna hi hai."

Error: yeh remote controller ka time aur data return trip chhod deta hai, aur — modified data par coherent miss ke liye — ek third node se snoop/writeback ke liye extra hops. Ek dirty remote line ek 3-hop protocol ho sakti hai, ek interconnect crossing se kaafi zyada.

SE4. "Node 2, node 1 se door hai, isliye node 3 access karna node 2 se bhi slower hoga."

Error: latency topology (hop count) track karta hai, node numbering nahi. Is page ke top par draw ki gayi machine par nodes 2 aur 3 dono node 0 se 2 hops door hain — 165 ns aur 168 ns measure kiye gaye, essentially equal — jabki node 1 1 hop hai (140 ns). Kabhi assume mat karein higher ID = farther; actual interconnect graph padhin.

SE5. "NUMA mein cache coherence disable hai taaki fast ho, isliye ise Non-uniform kehte hain."

Error: do unrelated ideas ek saath mash kar di gayi hain. "Non-uniform" location se vary karne wali latency refer karta hai. Coherence ek alag choice hai: ccNUMA ise on rakhta hai (hardware mein); naam ka coherence off karne se koi lena dena nahi.

SE6. "Kyunki directory-based protocol mein per line ek home node hota hai, ek read hamesha home node par ek single request hai."

Error: agar line kisi doosre node ke cache mein modified hai, toh home directory ko us owner ko forward karna hoga, writeback/forward ka wait karna hoga, phir reply karna hoga — classic 3-hop path. Sirf clean lines ek round trip mein resolve hoti hain.

SE7. "Humne numactl --membind add karne ke baad 45 GB/s paaya, toh --membind akela fix hai."

Error: --membind force karta hai kahan memory rehti hai lekin kahan threads run karte hain nahi; aapko --cpunodebind (ya first-touch) bhi chahiye taaki thread aur uska data ek hi node par land karein. Memory ko node 0 par bind karna jabki thread node 3 par run kare sirf remote access guarantee karta hai.

SE8. "Page migration free real-estate hai — bas automatic NUMA balancing on karo aur tuning bhool jao."

Error: ek page migrate karne ka matlab hai use interconnect ke paar copy karna aur page tables update karna/TLBs flush karna; agar kisi page ka access pattern nodes ke beech flip-flop kare, toh OS thrash kar sakta hai, use baar baar migrate karke baar baar cost pay karta hai. Migration stable patterns ke liye help karta hai, ping-pong karne wale patterns ke liye nahi.

Why questions

WHY1. "Ek single shared bus physically ~4–8 processors se aage scale kyun nahi kar paata?"

Har processor ko ek hi electrical bus ke liye arbitrate karna padta hai, isliye requests serialize hoti hain; processors add karne se contention badhti hai jabki bus ki fixed bandwidth ab zyada tareekon se split hoti hai. Electrically, zyada taps signal integrity aur clock speed bhi degrade karte hain.

WHY2. "OS general workloads ke liye interleaving ke upar range-based mapping kyun prefer karta hai?"

Range-based ek node ki memory contiguous rakhta hai, isliye OS ek thread ko uske apne node se ek block de sakta hai (affinity). Woh first-touch aur locality enable karta hai — 80/20 win — jo interleaving pages everywhere scatter karke destroy kar deta.

WHY3. "NUMA par false sharing ek single-socket UMA machine se kaafi zyada worse kyun hoti hai?"

Ping-pong ki gayi cache line ab interconnect ke paar bounce karti hai (hundreds of ns per bounce, ping-pong figure mein) instead of ek on-die coherence fabric ke andar (tens of ns). Wahi coherence pattern jo tolerable tha woh ek severe remote-traffic storm ban jaata hai.

WHY4. "Remote accesses ka 'last 5%' bhi real performance kyun cost karta hai?"

Average latency

t_{l oc a l} [(1 - f) + f F_{N U M A}]

hai;

f = 0.05, F_{N U M A} = 2

par bracket

1.05

hai, yani 5% latency tax, aur all-remote ke muqable speedup ~1.9x hai 2.0x nahi. Remote accesses per event expensive hain, isliye chhota sa fraction bhi measurable gap chhod jaata hai.

WHY5. "Locality ko 80% se 95% push karna 0% se 80% jaane se extra speedup kyun smaller deta hai?"

Kyunki speedup

\frac{F _{N U M A}}{( 1 - f ) + f F _{N U M A}}

diminishing returns wala curve hai — zyaatar benefit jaldi capture ho jaata hai (remote fraction mein pehla bada drop), aur tail

f \to 0

ke saath flatten ho jaati hai. Analogous Amdahl-style flattening ke liye parallel algorithms dekhin.

WHY6. "Thread scheduling ka NUMA se koi lena dena kyun hota hai?"

Agar scheduler ek thread ko data first-touch hone ke baad alag node par migrate kare, toh woh saara data suddenly remote ho jaata hai. NUMA-aware scheduling ek thread ko us node par rakhne ki koshish karta hai jahan uska working set hai.

WHY7. "NUMA mere code mein ek memory-consistency bug automatically kyun fix nahi karta?"

NUMA sirf yeh change karta hai ki access kitni tez hai, yeh nahi ki kya ordering guarantees hain. Ordering memory consistency model aur aapki synchronization se govern hoti hai — aapko node placement se regardless sahi fences/locks chahiye hi chahiye.

WHY8. "OS thread migrate karne ki bajaye ek page migrate kyun karta?"

Kabhi kabhi thread pinned hoti hai (scheduler ya user ke zariye), ya kaafi threads page share karti hain — jahan accesses concentrate hain wahan ek page move karna threads aur unke poore working sets relocate karne se sasta hai. Woh ek hi locality goal ke liye reverse lever hai.

Edge cases

EC1. "Single-node NUMA system (ya BIOS mein NUMA disabled):

F_{N U M A}

kya hai?"

Sirf ek node ke saath har access local hai, isliye

t_{r e m o t e} = t_{l oc a l}

aur

F_{N U M A} = 1

. Performance model plain UMA mein degenerate ho jaata hai — machine uniformly behave karti hai.

EC2. "Agar

f = 0

ho (perfect locality)?"

Average latency

= t_{l oc a l} [(1 - 0) + 0] = t_{l oc a l}

, ideal. All-remote ke muqable speedup

= F_{N U M A}

, woh poora advantage jo hardware offer kar sakta hai — yahi target hai jo har NUMA-aware program chase karta hai.

EC3. "Parent note ek address ko node par

⌊ A / L ⌋ mod N_{n o d es}

se map karta hai, jahan

A

physical address hai (memory mein ek byte offset),

L

bytes mein cache-line size hai, aur

N_{n o d es}

nodes ki sankhya hai. Agar

N_{n o d es} = 1

ho toh?"

Tab

⌊ A / L ⌋ mod 1 = 0

har address ke liye, isliye saari memory node 0 par rehti hai. Doosre nodes ke cores hamesha remote hain — bandwidth ek controller par collapse ho jaati hai. (

⌊ \cdot ⌋

floor hai, "nearest whole number tak round down";

mod

divide karne ke baad remainder hai.)

EC4. "Ek thread jo ek huge array ko ek baar, sequentially touch karta hai, phir kabhi nahi — kya first-touch phir bhi help karta hai?"

Sirf marginally. First-touch pages us thread ke liye locally place karta hai, lekin agar data ek baar bina reuse ke read ho, toh exploit karne ke liye locality zyada nahi hai aur streaming bandwidth dominate karta hai. Locality tuning tab sabse zyada pay off karta hai jab data revisit hota hai.

EC5. "Do nodes, lekin interconnect ki latency 0 hai (idealized). NUMA distinction ka kya hota hai?"

Agar

t_{in t er co nn ec t} \to 0

aur remote controller cost bhi vanish ho jaaye, toh

t_{r e m o t e} \to t_{l oc a l}

aur

F_{N U M A} \to 1

— machine uniform dikhti hai. Real interconnects non-zero hain, isliye exactly non-uniformity exist karti hai.

EC6. "Sab threads ek read-only lookup table share karte hain. Best placement?"

Kyunki yeh read-only hai aur sab share karte hain, replicate karein har node par ek copy (page replication) taaki har thread locally padhe, ya interleave karein agar replication available nahi. Ek node par first-touch baaki sab ke liye remote kar dega.

EC7. "

F_{N U M A} < 1

— kya yeh possible hai?"

Nahi. Remote access local path ke upar ek interconnect hop aur ek second controller add karta hai, isliye

t_{r e m o t e} > t_{l oc a l}

hamesha,

F_{N U M A} > 1

deta hai. 1 se neeche ka ratio matlab hoga network cross karna ghar rehne se faster hai, jo physics forbid karti hai.

EC8. "Ek thread ka working set node 0 par first-touch hua, lekin scheduler baad mein thread ko permanently node 2 par park kar deta hai. OS phir pages node 2 par migrate karta hai. Kya first-touch waste hua?"

Waste nahi — woh us waqt sahi tha; page migration woh correction hai jab steady-state access pattern shift hoti hai. Cost ek baar interconnect ke paar copy hai; baad mein accesses phir local hain. Khatera sirf tab hai jab thread bouncing karta rahe (SE8 dekhin), jo migration thrash karta hai.

Recall Quick self-test

Average-latency formula $t_{a v g} = t_{l oc a l} [(1 - f) + f F_{N U M A}]$ — $f$ aur $F_{N U M A}$ mein se har ek kis par depend karta hai? ::: $f$ (remote fraction) aapke software par depend karta hai (placement, first-touch, scheduling); $F_{N U M A}$ (latency ratio) hardware topology par depend karta hai. Locality ke liye do ingredients ke naam batayein, aur sirf ek bind karna kyun fail karta hai. ::: Aapko thread ek node par pinned chahiye aur uska data usi node par first-touched/bound chahiye; sirf ek bind karna pairing mismatched chhod deta hai aur har access remote ho jaata hai. First-touch vs page migration — har ek kya control karta hai? ::: First-touch ek page ka initial node placement set karta hai; page migration ek already-placed page ko baad mein move karta hai jab OS dekhta hai ki access pattern doosre node par shift ho gayi hai.