CUDA cores and execution model: Question bank — Hinglish deep dive

6.2.3 · D5 · Hardware › GPU Architecture › CUDA cores and execution model

Ye ek misconception clinic hai CUDA cores aur execution model ke liye. Neeche diya gaya har item ek specific trap ko target karta hai — ek aisi jagah jahan CPUs se, raw arithmetic se, ya marketing numbers se aaya intuition aapko galat raaste pe le jaata hai. Sawaal padho, answer reveal karne se pehle guess karo, phir apni reasoning ko answer se compare karo.

Shuru karne se pehle, ek quick vocabulary refresh taaki neeche koi term aisa na ho jo aapne pin down nahi kiya ho:

Recall Wo paanch words jinpe neeche har trap depend karta hai

CUDA core ::: Ek single arithmetic-logic unit (ALU) jo ek clock mein ek FP32 ya integer op karta hai — ye koi full CPU-style core nahi hai. Warp ::: 32 threads ka ek fixed group jo ek shared scheduler ke under lockstep mein same instruction execute karta hai. SM (Streaming Multiprocessor) ::: Asli "processor" unit — 64–128 CUDA cores plus schedulers, registers, aur shared memory ka ek bundle. SIMT ::: Single Instruction, Multiple Thread — ek instruction bahut saare threads ko drive karta hai, lekin har thread apna register state rakhta hai aur diverge kar sakta hai. Occupancy ::: Kitne warps ek SM pe resident hain maximum ke relative mein — latency hiding ke liye fuel supply.

True or false — justify

Recall Ek CUDA core ek chhota CPU core hai.

False. ::: Ek CUDA core sirf ek ALU hai — iske paas koi instruction decoder nahi, koi branch predictor nahi, aur koi private cache nahi hai. Ye apni warp ke baaki 31 cores ke saath saari control logic share karta hai, isliye ye ek independent instruction stream us tarah nahi chala sakta jaise ek CPU core chala sakta hai.

Recall 16,384 CUDA cores wala GPU ek saath 16,384 fully independent programs chala sakta hai.

False. ::: Cores shared control ke under warps of 32 mein yoke kiye jaate hain, isliye aapko zyada se zyada (cores ÷ 32) independent instruction streams milte hain, aur wo bhi per SM grouped hote hain. Independence warp level pe hoti hai, core level pe nahi.

Recall Agar ek SM mein 128 cores hain aur ek warp 32 threads ka hai, toh ek warp instruction ek cycle ke quarter mein complete ho jaati hai.

False. ::: Ek warp instruction per issue atomic hoti hai — iska minimum latency ek cycle hai. Extra cores SM ko throughput dete hain (wo ek hi cycle mein kai warps ki instructions issue kar sakta hai), kisi bhi single warp ke liye latency kam nahi karte.

Recall 32 ka warp size kisi particular SM ke core count se derive kiya gaya tha.

False. ::: 32 ek fixed hardware convention hai — do ki power jo address masking aur thread indexing ko simplify karti hai, jo 8/16/32 SIMD lineage se inherit ki gayi hai. 64 ya 128 cores wale SMs bhi 32-thread warps use karte hain; wo bas per cycle zyada warps issue karte hain.

Recall SIMT mein, ek warp ke threads physically

alag branches nahi le sakte. False. ::: Wo diverge kar sakte hain — har thread apna register state rakhta hai. Lekin jab wo aisa karte hain, warp serialise karta hai: pehle if path ko chalata hai else-threads ko masked off karke, phir else path ko if-threads ko masked off karke. Divergence legal hai lekin slow hai, forbidden nahi.

Recall Resident warps ki sankhya double karna hamesha memory-latency hiding ko double kar deta hai.

False. ::: Sirf us point tak jahan latency window fully covered ho jaati hai ( $T_{required} = \frac{S \cdot L}{C}$ ). Us se aage, extra warps idle baithte hain — wo kuch add nahi karte kyunki ALUs already busy the. Dekho occupancy.

Recall Ek block hamesha exactly ek SM pe run karta hai.

True. ::: Ek block guaranteed apni poori lifetime ek single SM pe reside karta hai — isliye isiliye iske threads fast shared memory share kar sakte hain aur barriers se synchronise kar sakte hain. Grid, is se ulta, saare SMs mein spread hoti hai.

Recall Coalesced access ka matlab hai threads

same address read karte hain. False. ::: Coalescing ka matlab hai threads consecutive addresses read karte hain (thread 0 → A[0], thread 1 → A[1], …) taaki hardware unhe ek 128-byte transaction mein fuse kar sake. Sabka same address read karna ek broadcast hai, ek alag (aur bhi efficient) pattern.

Recall Transaction efficiency

$η$ gigabytes per second mein measure hoti hai. False. ::: $η = \frac{useful bytes}{transactions \times 128}$ dimensionless hai — bytes over bytes, 0 aur 1 ke beech ka fraction. Effective bandwidth $η$ times peak DRAM bandwidth hai; $η$ khud sirf wastage ratio hai.

Recall SIMT aur SIMD ek hi cheez hain sirf alag naam ke saath.

False. ::: SIMD mein ek instruction fixed vector lanes fill karta hai bina kisi per-lane control flow ke. SIMT mein har thread ka apna program counter aur register state hota hai aur wo diverge ho sakta hai — hardware per-thread independence fake karne ke liye lanes ko mask karta hai. SIMT, SIMD hai jisme per-thread freedom ka illusion hai.

Spot the error

Recall "

$S = 4$ schedulers aur $C = 10$ compute cycles per warp ke saath 200-cycle memory latency hide karne ke liye, mujhe $\frac{200}{10} = 20$ warps chahiye." Error: scheduler count drop ho gaya. ::: $T_{required} = \frac{S \cdot L}{C} = \frac{4 \times 200}{10} = 80$ warps. Har cycle SM $S$ warp-instructions issue kar sakta hai, isliye latency window ki capacity $S \cdot L$ hai, sirf $L$ nahi.

Recall "Mera SM 64 resident warps pe cap karta hai aur maine

$T_{required} = 80$ compute kiya, isliye main occupancy 80 warps tak raise karunga." Error: 64 ek hard ceiling hai. ::: Aap SM ke resident-warp cap se zyada nahi ja sakte. Kyunki $80 > 64$ , warps add karke full hiding impossible hai — aapko instead compute intensity $C$ badhani padegi (har memory op per zyada kaam karo) taaki $T_{required}$ kam ho.

Recall "Uncoalesced access zyada se zyada 2× slower hai kyunki data DRAM mein hai."

Error: transactions multiply hote hain. ::: Ek stride-32 pattern ek 128-byte transaction ko same useful bytes ke liye 32 transactions tak badal deta hai, isliye $η$ $100%$ se $3.125%$ tak gir jaata hai — 32× slower tak, 2× nahi.

Recall "Main parallelism maximise karne ke liye 2000 threads per block use karunga."

Error: block cap 1024 hai. ::: Ek single block mein zyada se zyada 1024 threads hote hain (usse ek SM pe fit hona chahiye). Massive parallelism bahut saare blocks grid mein launch karne se aata hai, oversized blocks se nahi.

Recall "Jab warp scheduler warps swap karta hai toh usse bada context-switch cost lagta hai, isliye swapping expensive hai."

Error: zero-overhead switching. ::: Saare resident warps ke registers SM ke register file mein already loaded hain. Warps switch karna free hai — scheduler bas next cycle ek alag ready warp pick karta hai. Yahi reason hai ki bohot saare warps resident rakhne ka poora point hai.

Recall "Ek CUDA core

sin(x) ek cycle mein compute karta hai bilkul multiply ki tarah." Error: galat unit. ::: CUDA cores FP32 FMA, integer, aur logic ops handle karte hain. Transcendentals jaise sin/exp SM ke alag special-function units (SFUs) pe run karte hain, kam throughput par — CUDA cores pe nahi.

Why questions

Recall Branchy, per-element-different code GPU performance kyun kill karta hai?

::: Kyunki ek warp ek instruction stream share karta hai. Jab threads diverge karte hain, hardware har path ko sequentially run karta hai baaki threads masked off karke — ek if/else split throughput halve kar sakta hai, aur deep nesting isko compound karta hai. GPUs tabhi jeette hain jab same instruction saare 32 threads pe apply ho.

Recall Threads ko blocks mein organize kyun karein ek flat pool ki jagah?

::: Blocks ek scalability abstraction dete hain: 1000 blocks ek 10-SM GPU (100 blocks/SM) ya ek 80-SM GPU (~12 blocks/SM) pe bina koi code change kiye run ho jaate hain. Ye shared memory aur barrier synchronisation ka scope bhi define karte hain. Dekho parallel patterns.

Recall Warp cap hit karne ke baad compute intensity

$C$ badhana warps add karne se zyada kyun help karta hai? ::: Kyunki $T_{required} = \frac{S \cdot L}{C}$ jaise $C$ badhta hai, shrink hota hai. Agar aap warps add nahi kar sakte (cap reach ho gayi), requirement kam karna hi ek lever bacha hai — same stall ko cover karne ke liye zyada arithmetic karo har memory fetch pe taaki kam warps chahiyein.

Recall Ek warp exactly 32 kyun hota hai aur, maano, 30 kyun nahi?

::: 32 do ki power hai, jo thread-index computation aur lane-masking ko saste bit operations (masking, shifting) ka kaam bana deta hai rather than division ke. Non-powers-of-two hardware mein har address-generation step ko complicate kar denge.

Recall Registers "per-thread" hone se kitne threads resident ho sakte hain ye limit kyun hota hai?

::: SM ka ek fixed register file hota hai (~65,536 32-bit registers bahut se recent SMs pe). Agar har thread bahut saare registers grab karta hai, toh kam threads fit hote hain: roughly $\frac{M}{P}$ registers per thread $P$ resident threads ke liye. Register-hungry kernels isliye occupancy kam karte hain aur latency hiding ko weaken karte hain.

Recall Memory coalescing

specifically warp level pe kyun matter karta hai? ::: Kyunki hardware warp ke saath mein request kiye gaye 32 addresses ko inspect karta hai aur unhe minimal 128-byte transactions mein fuse karne ki koshish karta hai. Coalescing warp ke collective access pattern ki property hai, kisi single thread ke access ki nahi.

Edge cases

Recall Ek warp launch hota hai lekin iske 32 threads mein se sirf 20

if (i < N) pass karte hain. Baaki 12 kya karte hain? ::: Unhe masked off kar diya jaata hai — wo apni lanes occupy karte rehte hain aur cycle consume karte hain, lekin unke results discard ho jaate hain. Warp 32-wide se faster nahi chal sakta; extra 12 lanes array tail par simply wasted work hain.

Recall Aap ek grid launch karte hain jahan

N block size ka multiple nahi hai. Kya ye ek bug hai? ::: Nahi — standard idiom numBlocks = (N + threadsPerBlock - 1) / threadsPerBlock upar round karta hai, kuch extra threads launch karta hai, aur if (i < N) guard overhang mask kar deta hai. Aakhri warp partly masked hota hai, jo normal hai.

Recall Ek block mein 100 threads hain. Ye kitne warps use karta hai, aur leftover lanes ka kya hota hai?

::: $⌈ 100/32 ⌉ = 4$ warps (96 + 4 threads). Chautha warp sirf 4 active lanes ke saath run karta hai aur 28 masked-off hote hain — wasted capacity. Isliye block sizes usually 32 ke multiples choose ki jaati hain.

Recall Ek warp ka har thread identical address

A[0] read karta hai. Coalesced hai ya catastrophic? ::: Na wasteful — ye ek broadcast hai. Hardware 128-byte line ek baar fetch karta hai aur value saare 32 threads ko de deta hai, isliye $η$ high rehta hai. Uniform reads efficient hain; scattered reads hurt karte hain.

Recall Ek warp ke saare 32 threads 32

alag code paths pe diverge karte hain. Worst-case slowdown? ::: 32× tak — warp ko saare 32 paths ek ek karke serialise karna padta hai, har ek ko ek single active lane ke saath run karke. Ye branch divergence ki pathological limit hai.

Recall Jab ek kernel mein sirf EK resident warp ho toh latency hiding ka kya hota hai?

::: Switch karne ke liye kuch hota hi nahi. Jis second wo warp memory pe stall karta hai (100–400 cycles), SM ke ALUs puri time idle baithte hain — bilkul bhi hiding nahi. Isliye single-warp kernels GPU ko almost completely waste karte hain, jo training kernels ke liye ek key lesson hai.

Recall Ek double-precision (FP64) op issue hoti hai. Kya har CUDA core ise full FP32 rate pe handle karta hai?

::: Nahi. FP64 SM pe alag, bahut kam double-precision units pe run karta hai. Consumer GPUs pe FP64 throughput FP32 ka ek chhota fraction ho sakta hai (e.g. 1/32) — isliye DP-heavy kernel bottleneck karta hai chahe cores "available" hon.