GPU vs CPU design philosophy: Visual walkthrough — Hinglish deep dive

6.2.1 · D2 · Hardware › GPU Architecture › GPU vs CPU design philosophy

Yeh page parent note ka central result bilkul zero se rebuild karti hai, ek waqt mein ek picture. Hum ek simple sawaal se shuru karte hain — "ek job mein kitna time lagta hai?" — aur end mein dekhte hain ki CPU aur GPU opposite bets kyun karte hain. Koi bhi symbol tab tak nahi aata jab tak hum use draw nahi karte.

Agar aapne abhi tak woh machine nahi dekhi jo ek ke baad ek instruction run karti hai, toh pehle 6.1.01-von-Neumann-architecture dekh lo; neeche ka saara kuch usi ek idea pe bana hai.

Step 1 — "Ek job mein kitna time lagta hai?" ka matlab kya hai?

KYA. Kisi bhi chip se pehle, ek word pe agree karte hain. Ek task kaam ka ek chota sa tukda hai — hamare liye, "ek pixel ka colour compute karna". Latency woh time hai ek task ke start se finish tak. Throughput yeh hai ki kitne tasks per second finish hote hain.

KYUN. Yeh do numbers opposite directions mein khinchte hain, aur poori GPU-vs-CPU story kuch nahi hai siwaaye yeh chunne ke ki kaunsa chhota karna hai (latency) ya kaunsa bada (throughput). Agar hum inhe alag nahi karte, toh hum trade-off nahi dekh sakte.

PICTURE. Do timelines dekho. Upar, ek worker har task jaldi finish karta hai lekin ek ke baad ek karta hai. Neeche, bahut saare slow workers har ek zyada time lete hain, lekin overlap karte hain — toh zyada tasks per second finish hote hain.

Yahan $t_{t a s k}$ ek task ki latency ka symbol hai. Is page ke har equation mein isi se kaam chalta hai.

Step 2 — Woh ek formula jis par sab kuch tika hai

KYA. Maano hamare paas $N$ independent tasks hain aur $P$ workers hain, har worker ek waqt mein ek task run kar sakta hai, aur har task $t_{t a s k}$ seconds leta hai. Total wall-clock time yeh hai:

$T_{t o t a l} = tasks per worker \frac{N}{P} \times seconds per task t_{t a s k}$

Term by term: $N$ kaam ka pile hai, $P$ hamare paas kitne haath hain, toh $N / P$ hai kitne tasks har haath ko karne hain; $t_{t a s k}$ (seconds per task) se multiply karo aur seconds milte hain.

KYUN. Yahi ek equation hai jo humein chahiye. Har design choice — bada cache, zyada cores, branch prediction — bas ek koshish hai ki in teen letters ( $N$ , $P$ , ya $t_{t a s k}$ ) mein se ek ko favourable direction mein push kiya jaaye. Ab hum dekhenge ki CPU aur GPU alag-alag levers kaise pull karte hain.

PICTURE. Formula ko teen dials ke set ke roop mein dekho. $t_{t a s k}$ ghataana (CPU ka move) ya $P$ badhana (GPU ka move) — dono $T_{t o t a l}$ ko shrink karte hain — lekin transistors alag jagah cost karte hain.

Step 3 — CPU ka bet: $t_{t a s k}$ ko tiny karo

KYA. CPU worst input assume karta hai: ek aisa task jahan step $I_{n}$ ko step $I_{n - 1}$ ka answer chahiye. Yeh data dependency hai — tum literally agla step jaldi shuru nahi kar sakte. Jab kaam is tarah ki chain ho, $P$ 1 pe atka rehta hai (extra workers ke paas karne ko kuch nahi), toh bachta hua ek hi lever hai $t_{t a s k}$ .

Yeh tools kyun, dusre kyun nahi? Do cheezein ek single task ko slow banati hain, toh CPU transistors spend karta hai bilkul unhi do ko khatam karne mein:

Memory door hai. DRAM read karna ~ $100 ns$ lagta hai; arithmetic karna ~ $1 ns$ . Toh CPU ek bada cache banata hai (paas wali fast memory). Ek cache hit $100 ns$ ko kuch ns tak gira deta hai. Hum cache use karte hain — zyada cores nahi — kyunki zyada cores ek aisi chain ki madad nahi kar sakte jo order mein run karni hai.
Branches surprises hain. Roughly har 5–7 instructions mein code "if…?" poochta hai aur chip ko abhi nahi pata kaunsa rasta lena hai. Galat guess karna kaam barbad karta hai. Toh CPU ek branch predictor (>95% sahi) aur speculative execution add karta hai — woh guess karta hai aur aage run karta hai. Phir se: yeh ek chain ke liye $t_{t a s k}$ shrink karta hai, jo ek hi available lever hai.

PICTURE. CPU floorplan: green ka chota patch (arithmetic units — woh part jo math karta hai) ek badi blue sea of cache aur mota orange block of control logic se ghira hua. Zyaatar silicon math nahi kar raha; woh exist karta hai taaki jo thoda math kare woh fast ho.

Step 4 — GPU ka bet: $P$ ko enormous banao

KYA. GPU best input assume karta hai: un tasks ka pahad jo ek doosre par depend nahi karte. Pixel $(x_{1}, y_{1})$ colour karne ke liye pixel $(x_{2}, y_{2})$ se kuch nahi chahiye. Yeh ek embarrassingly parallel workload hai — tum har pixel ko alag worker de sakte ho. Ab $P$ ek free lever hai, toh GPU ise transistor budget ke allow hone tak push karta hai.

KYUN. Ek transistor budget $B$ fix karo. Ek simple core $C_{s im pl e}$ transistors leta hai; ek complex CPU-style core (apne cache aur control ke saath) kareeban das guna zyada, $C_{co m pl e x} = 10 C_{s im pl e}$ . Toh:

$P_{GP U} = \frac{B}{C _{s im pl e}} = 10 \times \frac{B}{C _{co m pl e x}} = 10 \times P_{C P U}$

Padhke: same budget $B$ , lekin cache/branch-prediction per core ke liye pay na karke, har core 10× sasta hai, toh tum ~10× zyada afford kar sakte ho. GPU ek bigger $t_{t a s k}$ per core accept karta hai bahut bade $P$ ke badले mein — aur Step 2 ka formula kehta hai ki yeh ek great trade hai jab $N$ huge ho.

PICTURE. Step 3 ke side by side GPU floorplan: ab green (math units) almost poori chip fill karta hai; blue cache aur orange control patli strips mein simat jaate hain.

Yeh wahi lever-pulling logic hai jo baad mein 9.2.01-parallel-programming-models mein formalize hota hai.

Step 5 — Lekin GPUs ke paas bada cache nahi hai. Woh slow memory se kaise survive karte hain?

KYA. Humne abhi CPU ka cache throw away kar diya. Toh har core phir bhi DRAM ke liye ~ $100$ – $200 ns$ wait karta hai. Agar ek core wait karte waqt bas idle baith jaaye, toh woh saare extra cores bekar honge. GPU ka trick: bahut saare thread groups ko resident rakho, aur jis second ek group memory par stall ho, scheduler doosra group swap in kar deta hai jo compute ke liye ready hai.

Yeh tool — hardware threading — kyun, caching kyun nahi? Hazaron cores ke liye itna bada cache poora transistor budget kha jaata, Step 4 ko undo kar deta. Iske bajaaye GPU latency ko reduce karne ki jagah hide karta hai: jab group A wait kar raha hota hai, groups B, C, D… math karte hain. Tumhare paas kaafi waiting groups chahiye (dozens of warps — warp 32 threads ka bundle hota hai jo lockstep mein chalte hain) taaki baaki ke compute se stalled wale ka har gap bhar jaaye.

PICTURE. Colored bars ki ek row (warps). Warp A ek grey "memory wait" gap hit karta hai; neeche, warps B, C, D us gap ke through math units ko busy rakhte hain, toh arithmetic pipeline kabhi idle nahi hoti.

"Same instruction, bahut saare threads" idea SIMT hai, jise 6.3.01-SIMD-vs-SIMT mein explain kiya gaya hai; swapping machinery 6.2.03-memory-hierarchy-GPU mein rehti hai.

Step 6 — Payoff: jab $N ≫ P$ ho, toh sirf core-count matter karta hai

KYA. Huge- $N$ case ko Step 2 mein plug karo. Jab tasks cores se bahut zyada hon ( $N ≫ P$ ), toh per-task time do machines ke ratio se cancel ho jaata hai aur speedup core counts ki comparison mein collapse ho jaati hai:

$\frac{T _{C P U}}{T _{GP U}} = \frac{N / P _{C P U}}{N / P _{GP U}} = \frac{P _{GP U}}{P _{C P U}}$

$N$ (same pile of work) cancel ho jaata hai; $t_{t a s k}$ (per-task cost) dono mein hota hai aur cancel ho jaata hai jab tak woh itna comparable ho ki dominate na kare — aur latency-hiding (Step 5) exactly yahi khareedti hai. $P_{GP U} = 4096$ cores aur $P_{C P U} = 16$ cores ke saath:

$\frac{T _{C P U}}{T _{GP U}} = \frac{4096}{16} = 256 \times$

KYUN. Yahi parent note ka central result hai, aur ab tum dekh sakte ho ki yeh magic nahi hai — yeh Step 2 ka formula hai un do levers ke saath jo har machine ne choose kiye. GPU pure headcount se jeeetta hai sirf tab jab kaam embarrassingly parallel ho.

PICTURE. Do shrinking bars: jaise-jaise $N$ badhta hai, GPU ka total time CPU ke neeche bahut kam ho jaata hai, aur gap core counts ke ratio par settle ho jaata hai.

Step 7 — Edge case: ek aisa job jo split nahi ho sakta (GPU haarta hai)

KYA. Ab dono machines ko ek sequential job do: ek sorted array of $1 0^{9}$ elements mein binary search. Har step ko pichle step ke comparison result ki zaroorat hai — ek pure chain, toh $P$ dono machines par 1 par atka hua hai. Pile mein sirf $N = lo g_{2} (1 0^{9}) \approx 30$ tasks hain.

KYUN. $P = 1$ ke saath, Step 2 ban jaata hai $T = N \times t_{t a s k}$ — aur ab sirf $t_{t a s k}$ matter karta hai. GPU ne deliberately $t_{t a s k}$ bada kiya tha (no branch predictor, no speculation). Toh yahan iska bet ulta pad jaata hai:

CPU: branch predictor pattern seekh leta hai, speculation agla step jaldi start kar deta hai → $t_{t a s k} \approx 2 ns$ → $30 \times 2 = 60 ns$ .
GPU: slow simple core, no prediction → $t_{t a s k} \approx 5 ns$ → $30 \times 5 = 150 ns$ .

$\frac{T _{GP U}}{T _{C P U}} = \frac{150}{60} = 2.5 \times (CPU wins)$

PICTURE. 30 steps ki ek narrow staircase, har machine par ek worker. CPU worker chote strides leta hai; GPU worker lambe strides leta hai aur last mein finish karta hai. GPU ke woh hazaron extra cores idle khade hain — unhe dene ke liye kuch nahi.

Step 8 — Degenerate cases: chhota $N$ , ya ek branch jo warp ko split kar deta hai

KYA. Do aur corners jinhe reader ko kabhi miss nahi karna chahiye.

Chhota $N$ (cores se kam tasks, $N < P$ ). $P_{GP U} = 4096$ ke saath lekin sirf $N = 100$ tasks hain, toh zyaatar cores idle baithte hain; GPU job launch karne ki bhi ek fixed cost hoti hai. Step 6 ka 256× evaporate ho jaata hai — CPU bhi jeet sakta hai.
Branch divergence. Yaad karo ek warp = 32 threads jo same instruction run karte hain. Agar ek if 16 threads ko path A aur 16 ko path B par bhej deta hai, toh hardware A run karta hai jab B wait karta hai, phir B jab A wait karta hai — dono halves serialize ho jaate hain. Worst case mein ek warp half speed pe run karta hai (ya aur bura). GPU ke paas ise soft karne ke liye koi branch predictor nahi hai; yeh Step-4 bet ki kimat hai.

KYUN. Yeh exotic nahi hain — yeh decide karte hain ki tumhara real code speed up hoga ya nahi. Chhote problems aur branchy code exactly wahan hain jahan throughput machine ke paas chhabne ke liye kuch nahi hota.

PICTURE. Left: ek GPU 4096 slots ke saath lekin sirf 100 filled (bahut saare grey idle cores). Right: 32 threads ka ek warp ek if par split ho raha hai do half-warps mein jo baari-baari chalte hain, time double kar dete hain.

Recall Corners check karo

GPU ka throughput advantage kab gayab ho jaata hai? ::: Jab tasks dependent hon (chain, $P = 1$ ), jab tasks bahut kam hon ( $N < P$ , idle cores + launch overhead), ya jab ek warp ke threads branch par diverge karein (serialized execution). 30-step sequential chain par kaun jeeetta hai aur kyun? ::: CPU — $P$ ke 1 par freeze hone ke saath, sirf $t_{t a s k}$ matter karta hai, aur CPU ne apne transistors $t_{t a s k}$ chota karne mein lagaye (branch prediction, speculation).

Ek-picture summary

Upar ka sab kuch ek formula hai, $T_{t o t a l} = (N / P) \times t_{t a s k}$ , aur do opposite bets hain ki kaunsa lever pull kiya jaaye. Yeh final figure poori derivation ek canvas par rakhti hai: beech mein shared formula, left par CPU $t_{t a s k}$ shrink karta hua (big cache, few cores), right par GPU $P$ grow karta hua (many cores, tiny cache), aur do verdicts — GPU huge-independent pile jeetta hai, CPU tiny dependent chain jeetta hai.

Recall Feynman retelling — ek 12-saal ke bachche ko samjhao

Har job chhote tasks ka ek pile hota hai. Jitna time lagta hai woh hai har worker ko kitne tasks milte hain (yeh hai $N / P$ ) times ek task kitna time leta hai ( $t_{t a s k}$ ). Jaldi finish karne ke sirf do tarike hain: har task jaldi karo, ya zyada workers rakho. Ek CPU sirf kuch bahut clever workers rakhta hai aur apna zyaatar silicon unme se har ek ko lightning-fast banane mein lagata hai — perfect jab tasks ek chain banate hain jahan agle task tak nahi ja sakte jab tak pichla done na ho. Ek GPU hazaron simple workers rakhta hai aur unhe barely train karta hai — perfect jab kaam ka pahad ho jo kisi ko bhi wait nahi karaata, jaise ek million pixels. Kyunki GPU har worker ke liye mehenga helper khareedne se mana karta hai, har worker kareeban das guna sasta hota hai, toh das guna zyada fit hote hain. Jab pile gigantic hota hai, sirf worker-count matter karta hai aur GPU CPU ko crush karta hai — hamare numbers mein, 4096 vs 16 cores matlab 256× faster. Lekin job ko 30 steps ki single chain tak chhota karo aur GPU ke saste slow workers CPU ke clever fast ones se haar jaate hain, 60 ns vs 150 ns mein. Same formula, opposite bets, opposite winners.