Foundations — Shared memory vs distributed memory

6.1.5 · D1 · Hardware › Parallelism & Multicore › Shared memory vs distributed memory

Parent note mein ek bhi formula padhne se pehle, tumhe pata hona chahiye ki har chhota symbol kiska picture hai. Hum inhe ek-ek karke banate hain, kuch bhi nahi se, ek aisi order mein jahan har cheez pehle wali pe tiki ho.

1. Ek processor (ek worker)

Hume yeh word isliye chahiye kyunki parallelism ka matlab hai "ek saath ek se zyaada worker". Aage ki sab cheezein — sharing, messaging, bottlenecks — tabhi aati hain jab tumhare paas do ya zyaada workers hon.

Parent note mein letter $N$ ka matlab sirf kitne workers hain hai. Jab tum $O(N)$ ya $O(N^2)$ dekhte ho, ise padho jaise "jaise main workers add karta hun, cost kaise badhti hai".

2. Memory aur ek address

Parent note likhta hai "ek processor address A par data request karta hai". Yeh poora sentence tab tak meaningless hai jab tak tum memory ko numbered boxes aur address ko ek box number ki tarah na dekho. Yahi woh object hai jis par sab ladte hain.

3. Address space — shared vs local

Topic ko yeh kyun chahiye: yahi ek akela fark — ek street ya kai streets — distributed aur shared programming ke beech ka poora fork hai. Comparison table mein "Global address space" vs "Local address space" literally yahi picture hai. Dekho Parallel Programming Models ki har ek ko kaise program kiya jaata hai.

4. Cache — worker ke paas ek scratchpad

Parent note ki line "Kya A meri private L1 cache mein hai?" yeh pooch rahi hai: kya mere paas pehle se ek sticky-note copy hai? L1, L2, L3 sirf increasingly bade, thode slow notebooks hain — L1 tumhari desk par, L3 hall mein share hua.

5. Cache line — copy unit

6. Coherence — copies ko honest rakhna

Parent ka MSI / MESI / MOESI is shredding rule ke liye recipes hain. Teen MSI letters tumhari photocopy par sirf labels hain:

Topic ko coherence kyun chahiye: yeh woh hidden message passing hai jis ke baare mein parent ka final "mistake" callout warn karta hai — shared memory magically free nahi hoti; hardware quietly tumhare liye "apni copy kaato!" notes mail kar raha hai. Poori recipes Cache Coherence Protocols mein hain aur rules ki kab ek write visible hota hai woh Memory Consistency Models mein hain.

7. Bus — ek shared hallway

Yeh villain kyun hai: $N$ workers ke saath sab shared data likhte hain, bus invalidate traffic se bhar jaati hai aur bottleneck ban jaati hai — yahi reason hai ki parent kehta hai shared memory ~8–64 cores par top out hoti hai. Jab memory ko split kiya jaata hai taaki alag regions alag cores ke "closer" hon, tumhe NUMA Architecture milti hai.

8. Message passing — mailroom

Desks ke beech couriers aur corridors interconnect hain — dekho Interconnect Networks (Ethernet, InfiniBand).

9. Latency aur bandwidth — travel ki do costs

Dono terms kyun matter karte hain: ek tiny letter ke liye, sara time joote-baandhne mein hai ($T_{\text{latency}}\gg M/B$) — isliye parent kehta hai kai chhote messages ko ek bade mein batch karo. Ek bade crate ke liye, corridor width $B$ dominate karta hai.

10. Do chhote "Big-O" phrases

Yeh woh shorthand hai jo "coherence traffic worst case mein $O(N^2)$ se badhti hai" ke peeche hai — mathematical reason ki shared memory scale karna band kar deti hai.

Foundations topic ko kaise feed karte hain

Upar se neeche padho: workers memory rakhte hain (boxes); woh boxes kaise address hote hain yeh design ko fork karta hai; caches cheezein fast banate hain lekin bus par coherence chahiye; alternative ek mailroom hai jo latency aur bandwidth se price kiya jaata hai — aur Big-O batata hai ki kaun zyaada workers ke saath bachta hai.

Equipment checklist

Right side cover karo, jawab do, phir reveal karo.