CUDA cores and execution model: Foundations — Hinglish deep dive

6.2.3 · D1 · Hardware › GPU Architecture › CUDA cores and execution model

Is page par yeh assume kiya gaya hai ki tumne parent note ka koi bhi notation pehle nahi dekha. Hum har term ek-ek karke build karte hain, aur har naya idea sirf pehle waale ideas par lean karta hai. Jab tum finish karo, CUDA cores and execution model dobara padho aur har symbol pehle se familiar lagega.

1. Clock cycle — har cheez ki heartbeat

Kisi bhi core, warp, ya memory word se pehle, time hai, jo equal ticks mein kata hua hai.

Ek drummer ki imagine karo jo ek steady beat rakh raha ho. Har beat par, factory mein har worker ko apne haath ek baar move karne ki permission hoti hai.

Yeh topic ko kyun chahiye: parent note mein har cheez cycles mein count ki jaati hai — "1 op per cycle", "200 cycles for global memory", "one warp instruction per cycle". Cycles poore page ke liye unit of time hain. Latency-hiding formula ko tum cycles ke bina nahi padh sakte.

Picture dekho: har vertical grid line ek beat hai. Ek fast operation ek beat mein fit hoti hai; ek slow operation (door waali memory ka ek trip) saikdon beats span karta hai. Yahi gap woh poori wajah hai jis wajah se GPUs us tarah build kiye jaate hain jaise hain.

2. ALU — chota calculator

Ek single pocket calculator ki imagine karo jisme exactly ek button per beat press hoti hai.

Parent note kehta hai ki ek CUDA core sirf ek ALU hai. Yeh poore page par sab se important de-mystification hai: ek "CUDA core" ek computer nahi hai, yeh ek calculator hai. Ek baar yeh maan lo, toh "16,384 CUDA cores" magical lagna band ho jaata hai — iska matlab sirf 16,384 calculators hai.

$c = a \times b + c$ padhna: $c$ ki purani value lo, usme $a \times b$ ka product add karo, result wapas $c$ mein store karo. $\times$ ka matlab multiply; $+$ ka matlab add; $=$ ka matlab "ban jaata hai".

3. Thread — ek recipe follow karta ek worker

Ek insaan ki imagine karo workstation par. Recipe (program) shared hai; notepad (unki registers, section 6) sirf unki apni hai.

Yeh topic ko kyun chahiye: poora GPU model hai "ek million threads launch karo, hardware ko decide karne do ki har ek kab run kare." Thread us story ka atom hai.

4. Warp — 32 workers lockstep mein chalte hue

Yeh woh idea hai jo GPU ko GPU banata hai, na ki sirf calculators ka ek bada pile.

Picture dekho: upar ek instruction arrow 32 ALUs tak fan out karta hai. Har ALU alag number par kaam karta hai (thread 0 A[0] par, thread 1 A[1] par, …) lekin sab identical operation perform karte hain.

32 ka number ek hardware convention hai (2 ki power, index karna easy), koi derived cheez nahi. Parent note is baare mein explicit hai.

"Ek instruction, kai threads jo principle mein diverge ho sakte hain" ka formal naam SIMT (Single Instruction, Multiple Thread) hai. Ise plain lockstep SIMD se compare karo SIMD vs SIMT mein — SIMT woh SIMD hai jo divergence ko allow karta hai (lekin penalise karta hai).

5. Blocks, grids, aur indexing formula

Ek warp 32 threads ka hai; lekin tum millions launch karte ho. Unhe kaise organize kiya jaata hai?

Ab parent ka key formula. Yeh jawab deta hai: "Main ek specific worker hoon — array ka kaunsa slot mera hai?"

Chalo har symbol earn karein:

$threadIdx.x$ — apne block ke andar tumhari seat number (0, 1, 2, …).
$blockDim.x$ — har block mein kitne threads hain (block ki size).
$blockIdx.x$ — tum kis block mein ho (block 0, block 1, …).

Yeh exact formula kyun? Socho ki seats rows mein numbered hain. Poore theatre mein apna seat number paane ke liye, tum (apna row number) × (seats per row) lete ho, phir row ke andar apni seat add karte ho. Yeh exactly $blockIdx.x \times blockDim.x + threadIdx.x$ hai.

Picture dekho: block 2, thread 3, block size 4 ke saath, global ID deta hai $2 \times 4 + 3 = 11$ — zero se count karte hue 12th slot. .x ka matlab sirf "x-direction"; blocks 2D ya 3D bhi ho sakte hain (.y, .z add karke), lekin idea samajhne ke liye 1D kaafi hai.

6. Registers — private notepad

Har worker ke elbow par notepad ki imagine karo: turant read kar sako, lekin pages sirf kuch hi hain.

Yeh topic ko kyun chahiye: registers scarce hote hain aur sare resident threads mein share out hote hain. Parent ka occupancy estimate $\frac{M}{P}$ bas yahi hai "total register slots $M$ divided by threads ki number $P$ jo tum ek saath present chahte ho". $M = 65, 536$ slots aur $P = 2048$ threads ke saath, har thread zyada se zyada $\frac{65536}{2048} = 32$ registers use kar sakta hai. Khatam ho jaayein, aur kam threads fit honge — yeh occupancy and performance ka topic hai.

7. Latency, throughput, aur yeh dono alag kyun hain

Do words jinhe parent baar baar use karta hai, aur log baar baar confuse karte hain.

Ek highway ki imagine karo. Latency = tumhari car ko road drive karne mein kitna time lagta hai. Throughput = per minute kitni cars finish line cross karti hain. Lanes add karna throughput badhata hai lekin tumhari ek car ko jaldi nahi pahunchata.

Latency hiding ka trick hai wait ko bharna: jab warp A 400 cycles memory ke liye stall ho, scheduler warps B, C, D… run karta hai, taaki koi bhi ALU idle na rahe. Parent ka formula $T_{required} = \frac{S \cdot L}{C}$ bas yahi hai "mujhe gap bharne ke liye kitne spare warps chahiye", jahan $S$ = schedulers, $L$ = stall length in cycles, $C$ = compute cycles ek warp karta hai stall hone se pehle. Yeh seedha memory hierarchy se connect karta hai, jo wajah hai ki stalls exist karte hain.

8. Efficiency symbol $η$ padhna

Parent $η$ (Greek letter "eta") introduce karta hai memory transaction efficiency ke liye.

Ek speed kyun nahi, fraction kyun? Kyunki yeh alag karta hai kitna achha tumne road use kiya se road kitni fast hai — tum $η$ ko peak bandwidth se multiply karte ho apni real effective bandwidth paane ke liye. Yeh coalescing story ke liye matter karta hai aur parallel programming patterns se connect karta hai.

Prerequisite map

Baayein taraf ki har cheez daayein taraf ke parent topic ko feed karti hai. Dhyan do ki clock cycle aur thread do sache roots hain — har doosra idea in dono se ugta hai.

Equipment checklist

Khud test karo: daayeen taraf cover karo aur zor se jawab do.

Clock cycle kya hai, ek sentence mein?

Chip ke metronome ki ek tick; har hardware unit ek tick mein ek chota step karta hai.

Kya ek CUDA core ek full processor hai?

Nahi — yeh sirf ek ALU hai, ek chota calculator jo per cycle ek arithmetic/logic op karta hai.

FMA likho.

c = a \times b + c

— multiply phir add, ek fused step mein kiya.

Ek warp mein kitne threads hote hain, aur woh number kyun?

32; ek hardware convention (2 ki power, index karna easy), core count se derive nahi.

SIMT kya allow karta hai jo plain lockstep nahi karta?

Threads branches par diverge ho sakte hain — har branch path ko serialize karne ki cost ke saath.

Global thread ID formula do aur har symbol ka naam batao.

blockIdx.x \times blockDim.x + threadIdx.x

= (kaunsa block) × (block size) + (block ke andar seat).

Kya global thread IDs 0 se start hoti hain ya 1 se?

0 — isliye kernels if (i < N) se guard karte hain.

Thread ka private 32-bit storage slot, ~1 cycle mein read/write.

Latency vs throughput ek line mein?

Latency = ek job kitna waqt leta hai; throughput = per unit time kitni jobs finish hoti hain.

Cores add karna ek single warp ko speed up kyun nahi karta?

Ek warp instruction ki minimum latency ek cycle hai; extra cores throughput badhate hain (ek saath zyada warps), latency nahi.

η

kya measure karta hai aur iske units kya hain?

Useful bytes ÷ actually moved bytes — 0 aur 1 ke beech ek unitless fraction.

Jab upar ki har line aasani se aane lage, tum CUDA cores and execution model ke liye ready ho.