6.2.7 · D1Memory hierarchy (global, shared, registers)
Intuition Is poore topic ke peeche ek hi idea haiEk GPU maths ko hazaaron guna faster kar sakta hai jitna ki apni main memory se ek number fetch karne mein lagta hai, isliye poora khel un numbers ko arithmetic units ke paas rakhne ka hai jo aapko chahiye . Memory hierarchy bas storage boxes ki ek seedhi (ladder) hai — upar tiny-and-instant, neeche huge-and-slow — aur is chapter ka har ek performance trick yahi hai ki koi data piece kis box mein rehta hai.
Parent note mein ek bhi formula padhne se pehle, aapko woh vocabulary chahiye jinse woh formulas bane hain. Is page par har ek symbol aur har ek concept listed hai jo parent assume karta hai, bilkul zero se shuru karke, aur iss tarah order kiye gaye hain ki har ek apne pehle wale par tikaa ho.
Is topic mein sab kuch ek hi image par tikaa hai: storage ko ek ladder ki tarah arrange kiya gaya hai. Upar wale rungs chhote hain par pahuunchne mein instant hain; neeche wale rungs bahut bade hain par slow. Neeche diye figure ko dekho aur apne dimag mein rakho — baad ka har idea kisi na kisi rung se juda hai.
Definition Latency aur capacity — ladder ke do axes
Latency = aap kitna wait karte ho ek piece of data ke aane tak, measured in clock cycles . Ise us arrow ki length ki tarah socho jitna tumhe ek rung tak pahuunchne ke liye chalna padta hai.Capacity = us rung par kitna fit hota hai , measured in bytes. Ise rung ki width ki tarah socho.
Poori hierarchy isliye exist karti hai kyunki ye dono ek doosre se ladte hain: fast storage mehengi transistors se bani hoti hai, isliye woh chhoti honi chahiye.
Ye topic ko kyun in do words ki zaroorat hai: parent ke summary table ki har row bas ek (latency, capacity) pair hai. Agar aap inhe distance aur width ki tarah feel nahi karte, to table sirf numbers hai.
Ek clock cycle GPU ke internal metronome ki ek tick hai — time ka sabse chhota slice jisme hardware ek basic step leta hai. Is topic ki saari latencies ticks mein count ki jaati hain, seconds mein nahi, kyunki ticks wahi hain jo hardware actually experience karta hai.
Intuition Cycles mein kyun count karein, nanoseconds mein kyun nahi?1.5 GHz par chalne wala GPU ek second mein 1.5 billion baar tick karta hai, isliye ek tick ≈ 0.67 nanoseconds. Lekin agar clock speed badh jaaye, to nanoseconds mein har latency saath mein shrink ho jaati hai. Cycles mein count karna numbers ko stable aur different GPUs ke beech comparable rakhta hai — isliye parent kehta hai "registers: 1 cycle, global: 400–800 cycles" instead of time use karne ke.
Jab parent likhta hai "memory access latency 100–1000× longer ho sakti hai computation se", iska matlab hai: ek arithmetic operation ~1 cycle cost karti hai, lekin ek global-memory fetch hundreds of cycles cost karta hai. Yahi ratio hierarchy ka poora motivation hai.
Definition Byte aur uske multiplesEk byte 8 bits hai, ek chhota number ya ek character store karne ke liye kaafi. Bade stores ko 1024 ki powers se name kiya jaata hai:
1 KB = 1024 1 MB = 1024 KB 1 GB = 1024 MB
Ek float (GPU mein ek decimal number) 4 bytes hota hai — yeh number yaad rakho, poori coalescing story isi se chalti hai. Ek byte ko ek chhoti si brick ki tarah socho; ek float chaar bricks ek saath chipki hui hain.
Ek thread ek data stream par aapka program execute karne wala ek worker hai. Ek GPU par hazaaron ek saath chalte hain. Har thread ke paas scratch values ka ek private notebook hota hai — woh registers mein rehti hain (ladder ke upar wala rung).
Agar yeh naya hai, toh ise GPU Thread Hierarchy mein poori tarah build karo. Is page ke liye aapko sirf itna chahiye: ek thread ek worker hai, aur uske paas kuch private storage hai.
Ek thread block threads ki ek team hai jo ek hi SM par chalti hai aur ek common scratchpad share kar sakti hai . Woh shared scratchpad shared memory hai (beech wala rung).
Ek warp exactly 32 threads ka ek group hai jo ek hi instruction ko lockstep mein execute karte hain . Yeh woh unit hai jo memory system actually serve karta hai — aap ek thread ke liye fetch nahi kar sakte, aap ek poore warp ke liye fetch karte hain. Dekho Warp Execution Model .
Topic ko warp kyun chahiye: coalescing aur bank conflicts dono is baare mein hain ki "jab 32 threads ek hi instant mein memory maangein toh kya hota hai". Warp nahi, coalescing nahi.
Definition Streaming Multiprocessor (SM)Ek SM GPU ki ek processing unit hai — ise ek factory floor ki tarah socho. Har SM ke andar physically apna register file aur apni shared memory hoti hai. Ek GPU mein bahut saare SMs hote hain. Kyunki fast rungs ek SM ke andar rehte hain, woh chhote hote hain aur unhe wahan chal rahe saare threads mein split karna padta hai.
Yahi poori register pressure ki kahani ki jadd hai: register file ek fixed-size cupboard hai jo SM se judi hui hai, aur har resident thread ko usme se ek drawer dena padta hai.
Ab jab pictures exist karti hain, hum letters ko naam de sakte hain. Har entry hai: symbol → simple words → jis picture mein woh rehta hai.
Definition Register-budget symbols
R t o t a l ek SM par total registers ki sankhya (jaise 65,536). Picture: SM ke cupboard mein drawers ki total sankhya.r ek thread ke dwara use kiye gaye registers . Picture: ek worker ke dwara occupy kiye gaye drawers.N t h r e a d s kitne threads ek saath SM par reh sakte hain , drawers se limited.N ma x threads per SM par hardware ceiling (jaise 2048), registers se regardless.⌊ ⌋ floor symbol: "neeche ki taraf round karo nearest whole number tak", kyunki aap ek fraction of a thread nahi chala sakte.
Occupancy thread ceiling ka woh fraction hai jo aap actually fill karte ho :
Occupancy = N ma x N t h r e a d s N ma x Memory Latency Hiding ).
Definition Banking symbols
B banks ki sankhya (typically 32). Ek bank ek narrow lane hai jo ek request per cycle serve karta hai.w bank width bytes mein (typically 4 — exactly ek float).a byte address , yaani aap kaunsa byte chahte ho.mod modulo operation: division ke baad remainder. 13 mod 5 = 3
Definition Segment aur transaction symbols
Segment = memory ka ek aligned block (32/64/128 bytes) jo global memory ek unit ke roop mein deta hai. Picture floor tiles fixed size ke; aap hamesha poori tiles fetch karte ho.N t r an s a c t i o n s ek warp ki request kitne distinct segments touch karti hai . Ek tile → 1 transaction (fast). 32 scattered tiles → 32 transactions (32× slower). Yeh hai coalescing .⌈ ⌉ ceiling symbol: "upar ki taraf round karo ". Use hota hai kyunki ek segment ka ek bhi byte touch karna poore segment ki cost lagata hai.S kisi bhi single bank par sabse bada crowd , S = i max ( threads hitting bank i ) S
latency vs capacity ladder
shared memory and banks B w
global memory transactions
Top-down padho: do units (time, size) ladder banate hain; worker hierarchy (thread → block → warp) plus SM decide karte hain kaun kaunsa rung share karta hai ; woh saath milke teen formula-families produce karte hain — registers/occupancy , bank conflicts , aur coalescing — jo is topic ko banate hain.
Worked example Register formula ek baar padhnaEk SM mein R t o t a l = 65 , 536 r = 40 N t h r e a d s = ⌊ 40 65 , 536 ⌋ = ⌊ 1638.4 ⌋ = 1638 Ise padhna: cupboard mein 65,536 drawers hain, har thread 40 chahta hai, isliye 1638 complete threads fit hote hain aur baaki 16 drawers (65536 − 1638 × 40
Common mistake Floor ka matlab "nearest tak round karna" nahi hai⌊ 1638.4 ⌋ = 1638 na ki 1638 upar ki taraf round kiya hua. Jo storage ek poora thread fill nahi kar sakti woh waste ho jaati hai, isliye capacity-limited counts ke liye hum hamesha neeche round karte hain — aur upar (⌈ ⌉
Daayein side ko cover karo aur har ek ka jawab aawaaz mein do. Agar koi fail ho, toh next deep dive se pehle uska section dobara padho.
Ek clock cycle hai GPU ki clock ki ek tick — woh unit jisme saari latencies count ki jaati hain, taaki numbers alag-alag clock speeds par comparable rahein.
Ek float kitne bytes occupy karta hai 4 bytes (aur ek bank exactly 4 bytes wide hai, w = 4
Ek thread hai ek data stream par aapka program run karne wala ek worker, apne private registers ke saath.
Ek warp hai exactly 32 threads jo lockstep mein execute karte hain — woh unit jise memory system ek saath serve karta hai.
Ek SM hai GPU ka ek processing floor jo physically register file aur shared memory rakhta hai, isliye woh chhote aur shared hote hain.
R t o t a l r ek SM par total registers, aur ek thread ke dwara use kiye gaye registers.
⌊ ⌋ nearest whole number tak neeche round karna, kyunki aap ek fraction of a thread nahi chala sakte.
Occupancy hai N t h r e a d s / N ma x
Ek bank hai shared memory ki ek narrow lane jo ek request per cycle serve karta hai; B = 32
mod division ke baad remainder — yeh ek address ko 32 banks mein se ek par map karta hai jaise hours clock par wrap hote hain.
Ek segment hai ek aligned fixed-size block (32/64/128 B) jo global memory poora deta hai; N t r an s a c t i o n s
⌈ ⌉ upar round karna — transactions ke liye use hota hai kyunki ek byte touch karna poora segment cost karta hai.
Aage: har symbol earn kar lene ke baad, ab aap parent ki three-level breakdown aur uske cousins CUDA Memory Types , Cache Architecture , aur Matrix Multiplication Optimization bina kisi undefined letter ke padh sakte ho.
