GPU vs CPU design philosophy: Exercises — Hinglish deep dive

6.2.1 · D4 · Hardware › GPU Architecture › GPU vs CPU Design Philosophy

Ye problems धीरे-धीरे vocabulary पहचानने से लेकर पूरे design decisions synthesise करने तक चढ़ते हैं। Har solution ek collapsible callout ke andar chhupa hua hai taaki pehle tum khud try kar sako। Arithmetic paper par karo, phir reveal karo। Parent note: GPU vs CPU design philosophy।

Shuru karne se pehle, ek reminder un do words ke baare mein jinpar sab kuch depend karta hai:

Figure 1 (neeche) is poore page ke liye tumhara compass hai: left half hai CPU ki ek fast lane (choti latency), right half hai GPU ki slower lanes ka stack (huge throughput)। Har exercise asal mein yeh pooch raha hai "kya main amber lane par hoon ya cyan lanes par?" — jab bhi confused feel ho, ise dekhte rehna।

Figure 1. Latency vs throughput। Amber box = ek fast CPU worker jo ek single task jaldi finish karta hai। Cyan boxes = bahut saare slower GPU workers jo ek second mein tasks ki puri flood finish karte hain। Note karo ki koi bhi single cyan arrow amber jaisa fast nahi hai — jeet bahut saare arrows hone se aati hai।

Hum parent note se do formulae reuse karenge। Yahan inhe likha gaya hai taaki koi bhi symbol unexplained na rahe।

Level 1 — Recognition

Exercise 1.1 mein ek word use hone se pehle, ek vocabulary piece pehle exist karni chahiye:

Exercise 1.1

Har phrase ke liye batao ki yeh latency-oriented (CPU) ya throughput-oriented (GPU) design se belong karta hai: (a) branch predictor, (b) 8704 simple cores, (c) 64 MB L3 cache, (d) warp scheduler hiding memory stalls, (e) out-of-order execution।

Recall Solution

(a) branch predictor → CPU (ek thread ko fast move rakhne ke liye speculate karta hai = latency)।
(b) thousands of simple cores → GPU (bahut saare workers = throughput)।
(c) huge L3 cache → CPU (ek thread ka data 1 tick door rakhta hai = latency)।
(d) warp scheduler → GPU (stalled work ko ready work se swap karta hai = throughput high rakhta hai)।
(e) out-of-order execution → CPU (reorder karta hai ek single thread ka $I P C$ badhane ke liye = latency)। Har answer ko Figure 1 par map karo: (a),(c),(e) single amber lane ko sharpen karte hain; (b),(d) cyan stack ko feed karte hain।

Exercise 1.2

Blank fill karo: GPU par 32 threads ka ek group jo sab ek hi waqt same instruction execute karte hain use ______ kehte hain, aur is execution model ko ______ kehte hain।

Recall Solution

Ek warp; model hai SIMT (Single Instruction, Multiple Threads)। 32 kyun matter karta hai: scheduler ek instruction saare 32 ko ek saath issue karta hai, isliye usse 32 lanes ke liye sirf ek decoder chahiye — yehi exactly hai kaise ek GPU control-logic transistors bachata hai।

Level 2 — Application

Exercise 2.1

Ek CPU $N_{instructions} = 6 \times 1 0^{9}$ instructions ka ek thread $I P C = 3$ aur $f_{clock} = 4 GHz$ par run karta hai। $T_{C P U}$ compute karo।

Recall Solution

$T_{C P U} = \frac{N _{instructions}}{I P C \times f _{clock}}$ mein plug karo। Denominator: $3 \times 4 \times 1 0^{9} = 1.2 \times 1 0^{10}$ instructions/second। $T_{C P U} = \frac{6 \times 1 0 ^{9}}{1.2 \times 1 0 ^{10}} = 0.5 s .$ Matlab: 12 billion finished instructions per second par, six billion ko aadha second lagta hai।

Exercise 2.2

Ek image mein $N = 8.3 \times 1 0^{6}$ pixels hain, har ek ek independent task hai jo $t_{task} = 105 ns$ leta hai। (a) $P = 16$ cores wale CPU par, (b) $P = 8704$ cores wale GPU par (simplicity ke liye same $t_{task}$ assume karo) batch time compute karo।

Recall Solution

$T_{total} = \frac{N}{P} \times t_{task}$ use karo, $t_{task} = 105 \times 1 0^{- 9}$ s ke saath।

(a) CPU: $\frac{8.3 \times 1 0 ^{6}}{16} = 518, 750$ tasks per core। $T = 518, 750 \times 105 \times 1 0^{- 9} \approx 0.05447 s \approx 54.5 ms .$

(b) GPU: $\frac{8.3 \times 1 0 ^{6}}{8704} \approx 953.6$ tasks per core। $T = 953.6 \times 105 \times 1 0^{- 9} \approx 1.001 \times 1 0^{- 4} s \approx 0.10 ms .$

Padhna: sirf ek cheez badli hai — $P$ । Zyada workers → wohi kaam ki pile thinner split hoti hai → chhota batch time। Yahi poori throughput story ek division mein hai — exactly Figure 1 mein cyan stack amber lane ka kaam kai baar kar raha hai।

Exercise 2.3

Parent note ki simplifying assumption use karte hue ki ek batch ki speedup core-count ratio ke barabar hai, $N_{cores}^{GP U} = 4096$ , $N_{cores}^{C P U} = 16$ ke liye GPU ki CPU par speedup nikalo।

Recall Solution

$\frac{T _{C P U}}{T _{GP U}} = \frac{N _{cores}^{GP U}}{N _{cores}^{C P U}} = \frac{4096}{16} = 256 \times .$ Yeh tabhi hold karta hai jab $N_{tasks} ≫ N_{cores}$ ho taaki dono machines poori tarah busy rahein।

Exercise 2.4 — Edge case: cores se kam tasks

Ek GPU mein $P = 4096$ cores hain। Tum sirf $N = 512$ independent tasks ki ek batch launch karte ho, har ek $t_{task} = 400$ ns। (a) Kitne cores actually kaam karte hain? (b) Batch time $T_{total}$ kya hai? (c) Ek single core use karne ke muqable tumhe kya speedup mila, aur yeh 4096× se itna kam kyun hai?

Recall Solution

(a) Ek task per core, isliye sirf 512 of the 4096 cores chalte hain; baaki 3584 idle baithe hain। (b) $T_{total} = \frac{N}{P} \times t_{task}$ । Yahan $N / P = 512/4096 = 0.125$ hai, jo upar round hota hai 1 wave of work (tum pass ka fraction nahi run kar sakte), isliye $T_{total} = 1 \times 400 ns = 400 ns .$ (c) Ek core ke comparison mein jo saare 512 tasks karta hai ( $512 \times 400 = 204, 800$ ns), speedup hai $204, 800/400 = 512 \times$ — 4096× nahi। Ceiling $N$ kyun hai, $P$ nahi: tumhe kabhi bhi utna parallel speedup nahi mil sakta jitne tumhare paas independent pieces of work hain। Jab $N_{tasks} < P$ ho, effective speedup $N_{tasks}$ par cap ho jaati hai, aur $(P - N_{tasks})$ cores wasted silicon hain। Yeh throughput machine ka worst regime hai — Figure 1 mein ek giant cyan stack jisme zyada lanes empty hain। GPU ko bahut kam kaam dena sabse common tarika hai "GPU is slower!" ka surprise paane ka।

Level 3 — Analysis

Exercise se pehle, hum GPU formula ko quote karne ki jagah earn karte hain। Batch model $T_{total} = \frac{N}{P} \times t_{task}$ se shuru karo aur ek real GPU ke liye har piece ko refine karo।

Formula

$T_{total}$ se $T_{GP U}$ build karna — har unit tracked Step 1 — actually kitne workers run karte hain? Saare $N_{cores}$ har cycle compute nahi karte; ek fraction $O cc u p an cy$ (0 aur 1 ke beech ek pure number) busy hota hai। Isliye effective workers $P_{eff} = N_{cores} \times O cc u p an cy$ (phir bhi sirf ek count)।

Step 2 — task cost ko seconds mein nahi, cycles mein measure karo। Ek GPU core apna kaam ticks mein karta hai। Mano $t_{instruction}$ = ek task ko kitne clock cycles lagte hain (units: cycles/task)। Cycles ko seconds mein convert karne ke liye hum clock se divide karte hain: $f_{clock}$ ki units hain cycles/second, isliye $\frac{t _{instruction} ( cycles/task )}{f _{clock} ( cycles/second )} = seconds/task = t_{task} .$ Yahi parent note ke $t_{task}$ ka bridge hai: yeh sirf cycles-per-task hai clock se divided।

Step 3 — substitute karo। $P_{eff}$ aur $t_{task}$ ko batch model mein daalo:

= \frac{N_{\text{tasks}}}{N_{\text{cores}}\times Occupancy}\times \frac{t_{\text{instruction}}}{f_{\text{clock}}}.$$ Char factors rearrange karne par parent note ki shape milti hai: $$T_{GPU} = \frac{N_{\text{tasks}}}{N_{\text{cores}} \times Occupancy \times f_{\text{clock}}}\times t_{\text{instruction}}.$$ **Yeh shape kyun?** Denominator $N_{\text{cores}} \times Occupancy \times f_{\text{clock}}$ ki units hain (workers)$\times$(cycles/second) = **task-slots per second** — poori machine kitni tezi se work ke cycles retire karti hai। Baaki $t_{\text{instruction}}$ (cycles/task) end mein multiply karo aur cycle-units cancel ho jaate hain, seconds bachta hai। Kuch magic nahi: yeh $T_{\text{total}}$ hai jisme seconds "cycles ÷ clock" mein split hain।

Exercise 3.1

Ek GPU kernel ek chip par launch hota hai jisme $N_{cores} = 4096$ , $f_{clock} = 1.5 GHz$ , occupancy $= 0.75$ , aur $N_{tasks} = 1 0^{7}$ hai। Har task $t_{instruction} = 4$ cycles cost karta hai। $T_{GP U}$ compute karo। Phir batao ki agar occupancy $0.375$ (half) ho jaaye toh $T_{GP U}$ ka kya hoga।

Recall Solution

Effective working cores: $4096 \times 0.75 = 3072$ workers। Task-slot rate (denominator): $3072 \times 1.5 \times 1 0^{9} = 4.608 \times 1 0^{12}$ cycles/second। $T_{GP U} = \frac{1 0 ^{7}}{4.608 \times 1 0 ^{12}} \times 4 cycles = 2.170 \times 1 0^{- 6} \times 4 \approx 8.68 \times 1 0^{- 6} s \approx 8.68 μ s .$ Units check: $\frac{tasks}{cycles/s} \times \frac{cycles}{task} = seconds$ । ✓ Occupancy half karne se denominator half ho jaata hai, isliye $T_{GP U}$ double ho jaata hai $\approx 17.36 μ s$ tak। Occupancy hidden villain kyun hai: GPU memory latency chhupata hai bahut saare warps rakh ke jo swap in ke liye ready hain। Low occupancy = bahut kam ready warps = cores DRAM wait karte hue idle baithe hain। Arithmetic units exist karte hain lekin unhe koi feed nahi karta।

Exercise 3.2

$T_{C P U} = N_{instructions} / (I P C \times f_{clock})$ use karte hue explain karo ki ek CPU designer jo clock $f_{clock}$ aur aage nahi badha sakta, woh branch prediction aur out-of-order execution par transistors kyun kharach karega।

Recall Solution

$T_{C P U}$ mein exactly teen knobs hain। $N_{instructions}$ program se fixed hai। $f_{clock}$ kuch GHz ke aas-paas ek power/heat wall se takra gaya। Sirf ek remaining knob hai $I P C$ — instructions finished per tick। Branch prediction pipeline ko unknown branches par stall hone se rokti hai, aur out-of-order execution independent instructions dhoondta hai run karne ke liye jab dusre wait kar rahe hoon। Dono $I P C$ badhate hain, jo $T_{C P U}$ ghatata hai bina clock ko touch kiye। Isliye exactly woh transistors worth it hain।

Level 4 — Synthesis

Exercise 4.1 — Amdahl's ceiling

Ek program 95% parallelisable hai ( $p = 0.95$ ) aur 5% strictly sequential hai। Amdahl's law use karte hue $S (P) = \frac{1}{( 1 - p ) + \frac{p}{P}},$ jahan $P$ processors ki sankhya hai aur $S$ speedup, $P = 4096$ GPU cores ke saath speedup compute karo। Phir theoretical maximum speedup compute karo jab $P \to \infty$ ho।

Recall Solution

Sequential fraction $1 - p = 0.05$ । $P = 4096$ ke saath: $S (4096) = \frac{1}{0.05 + \frac{0.95}{4096}} = \frac{1}{0.05 + 0.0002319} = \frac{1}{0.0502319} \approx 19.91 \times .$ Jab $P \to \infty$ , term $p / P \to 0$ : $S_{m a x} = \frac{1}{1 - p} = \frac{1}{0.05} = 20 \times .$ Punchline: 4096 cores tumhe $19.9 \times$ dete hain, aur infinite cores sirf $20 \times$ dete hain। Chhota sa 5% serial part sab kuch cap kar deta hai। Isliye ek GPU ka raw core count raw speedup mein translate nahi hota jab tak workload almost embarrassingly parallel na ho।

Exercise 4.2 — Crossover point

Maan lo ek task mein $N$ independent sub-tasks hain। CPU: $P_{C P U} = 16$ cores har ek $t_{task} = 100$ ns par। GPU: $P_{GP U} = 4096$ cores har ek $t_{task} = 400$ ns par (slower cores), plus ek fixed launch overhead of $10 μ s$ । Sabse chota $N$ nikalo jiske liye GPU, CPU ko beat karta hai।

Recall Solution

CPU time: $T_{C P U} = \frac{N}{16} \times 100 ns = 6.25 N ns$ । GPU time: $T_{GP U} = 10, 000 ns + \frac{N}{4096} \times 400 ns = 10, 000 + 0.09766 N ns$ । $T_{C P U} = T_{GP U}$ set karo aur $N$ ke liye solve karo: $6.25 N = 10, 000 + 0.09766 N \Rightarrow 6.15234 N = 10, 000 \Rightarrow N \approx 1625.4.$ Isliye GPU $N \geq 1626$ sub-tasks ke liye jeetta hai। Yahi crossover Figure 2 mein mark hai। Crossover kyun exist karta hai: GPU ek fixed $10 μ s$ launch tax pay karta hai aur slower cores use karta hai। Tabhi jab kaam ka pile itna bada ho jaata hai, uska 4096-way parallelism woh tax wapas kama leta hai। Chhote jobs CPU par hone chahiye — yeh "do numbers add karne ke liye kernel launch mat karo" ka quantitative version hai।

Figure 2 dono times ko $N$ ke against plot karta hai। Ise ek race ki tarah padhna: amber CPU line origin se start hoti hai (koi launch tax nahi) aur steeply chadhti hai; cyan GPU line upar se start hoti hai ( $10 μ s$ tax) lekin almost flat chadhti hai। Jahan woh cross karte hain woh 4.2 ka answer hai — uske left mein CPU jeetta hai, right mein GPU।

Figure 2. CPU (amber) aur GPU (cyan) batch time ka problem size $N$ ke against crossover। $N \approx 1625$ par white dot exactly woh break-even point hai jo Exercise 4.2 mein solve kiya gaya। GPU ka flat slope notice karo — yeh throughput hai; CPU ka steep slope dikhata hai ki limited latency scale par kaisi dikhti hai।

Level 5 — Mastery

Exercise 5.1 — Apna budget design karo

Tumhare paas transistor budget $B = 6 \times 1 0^{10}$ hai। Ek simple GPU-style core $C_{simple} = 5 \times 1 0^{6}$ transistors cost karta hai; ek complex CPU-style core $C_{complex} = 5 \times 1 0^{7}$ transistors cost karta hai। (a) Agar tum poora budget cores par kharach karo toh har type ke kitne cores build kar sakte ho? (b) Ek latency-bound task sirf 1 core use karta hai lekin usse simple core se 4× zyada per-core performance chahiye; ek throughput-bound task mein $1 0^{7}$ independent sub-tasks hain। Har task ke liye, kaun si chip pehle finish karti hai? Numbers ke saath justify karo।

Recall Solution

(a) All-simple: $B / C_{simple} = 6 \times 1 0^{10} /5 \times 1 0^{6} = 12, 000$ cores। All-complex: $B / C_{complex} = 6 \times 1 0^{10} /5 \times 1 0^{7} = 1, 200$ cores। Ratio $10 \times$ hai, exactly jaisa parent note predict karta hai ( $C_{complex} = 10 C_{simple}$ )।

(b) Latency task 1 core use karta hai। Complex core woh 4× single-core performance deliver karta hai; simple core nahi kar sakta (yeh ek plain in-order lane hai)। Isliye 1200-complex-core chip yahan jeetti hai — 12,000 simple cores mein se 11,999 idle baithte, ek single dependent chain ke liye useless।

Throughput task mein $1 0^{7}$ independent sub-tasks hain। Kam per-core speed par bhi, 12,000-core chip kaam ko $10 \times$ zyada finely split karta hai: $1 0^{7} /12000 \approx 833$ tasks/core vs $1 0^{7} /1200 \approx 8333$ tasks/core। Maan lo simple core per task up to $9 \times$ slower bhi hai toh bhi jeetta hai, kyunki $9 \times 833 = 7497 < 8333$ । Many-simple-core chip jeetti hai।

Mastery point: same silicon budget do opposite machines produce karta hai, aur sahi choice entirely is baat se decide hoti hai ki tumhara kaam ek dependent chain hai ya ek million independent pieces। Yahi GPU-vs-CPU philosophy hai, ek budgeting problem mein reduce karke — Figure 1 ke do halves।

Exercise 5.2 — Choice ek ek sentence mein explain karo

Mechanism ko goal se match karo, phir batao ki alternative kyun galat hota us chip par: (a) GPU hardware threading use karta hai bade caches ki jagah। (b) CPU speculative execution use karta hai zyada cores ki jagah।

Recall Solution

(a) 1000s of cores ke saath, ek private cache per core transistor budget blow kar deta, isliye GPU iske badle bahut saare resident warps rakhta hai aur ek stalled warp ko ek ready wale se swap karta hai — DRAM latency memory se nahi balki parallelism se chhupata hai। Bada cache galat buy hota: yeh ek thread ki latency optimise karta, jo ek throughput machine ko parwah nahi। Dekho GPU memory hierarchy। (b) Ek CPU ka kaam ek fast dependent thread hai, isliye extra cores jo woh use nahi kar sakta wasted silicon hain; speculation instead us single thread ka $I P C$ badhata hai। Zyada cores galat buy hote un code ke liye jo parallelise nahi ho sakta — yaad karo von Neumann sequential fetch-decode-execute chain, aur kaise SIMD vs SIMT width ko alag tarike se handle karta hai।

Recall Self-test recap

GPU, CPU ko tabhi beat karta hai jab ::: kaam highly parallel ho, launch overhead pay karne ke liye itna bada ho, aur serial fraction chota ho। Clock wall ke baad CPU ka sirf ek bacha hua knob hai ::: IPC (instructions per cycle), branch prediction aur out-of-order execution se badhaya jaata hai। Occupancy $T_{GP U}$ mein enter karta hai ::: denominator mein effective core count par multiplier ki tarah (latency-hiding capacity)। Jab N_tasks, P se kam ho, speedup cap hoti hai ::: N_tasks par (cores se kam tasks hone par idle cores hoti hain)। Serial fraction $1 - p$ wale program ke liye Amdahl's ceiling hai ::: $1/ (1 - p)$ , processor count se independent।