Multicore vs manycore designs: Exercises — Hinglish deep dive

6.1.4 · D4 · Hardware › Parallelism & Multicore › Multicore vs manycore designs

Yeh page ek self-testing ladder hai. Har rung pichle se mushkil hai. Solution ko cover karo, problem try karo, phir reveal karo. Yeh page apne aap mein complete hai — har symbol jo yahan use hota hai, woh pehle define kiya gaya hai.

Definition Is poori page par use hone wale paanch symbols

Kisi bhi equation se pehle, cast se milte hain. Har ek ek plain-English quantity hai, koi mystery letter nahi.

$P$ = parallel fraction: program ke runtime ka woh hissa jo cores mein split ho sakta hai. $0$ (kuch bhi parallel nahi) se $1$ (sab kuch parallel) tak ka ek pure number. Example: "70% serial" ka matlab hai $P = 0.30$ .
$N$ = core count: kitne cores kaam share karte hain. Ek whole number ( $8$ , $512$ , $6912$ , ...).
$f$ = clock frequency: har core kitne cycles per second tick karta hai, GHz mein ( $1 0^{9}$ cycles/sec). Ek "4 GHz" core ek second mein 4 billion ticks karta hai.
$IPC$ = instructions per cycle: ek core ek tick mein kitni instructions complete karta hai. Ek fast out-of-order core $\approx 4$ karta hai; ek simple in-order core $\approx 1$ karta hai.
$U$ = utilization: cores ka woh fraction ( $0$ se $1$ tak) jo kisi bhi moment par actually useful kaam kar raha hai, stalls aur idle threads ko subtract karne ke baad. $U = 0.9$ ka matlab hai 90% busy.

Intuition Ek equation jo is poori page ko chalati hai

Neeche ke almost har problem mein asliyat mein ek hi sawaal hai, bas alag bhes mein: ek fixed transistor budget diya gaya, kya main kuch fast cores chahta hoon ya bahut saare slow cores? Do tools iska jawab dete hain:

Speedup (ek job kitni tezi se khatam hoti hai) — Amdahl's Law se: $Speedup = \frac{1}{( 1 - P ) + \frac{P}{N}}$ Ise aise padho: serial part $(1 - P)$ kabhi nahi shrink hota, lekin parallel part $P$ ko $N$ cores mein divide kiya jaata hai. Jitna bada un-shrinkable $(1 - P)$ , utna zyada woh speedup ko strangle karta hai.
Throughput (poora chip kitni operations per second karta hai): $Throughput = N \times f \times IPC \times U$ Ise aise padho: total work/sec = (kitne cores hain) × (ticks/sec har ek) × (work per tick har ek) × (fraction jo actually busy hai). Har factor upar wale box mein define kiya gaya hai.

Figure s01 — woh map jis par har problem rehti hai. Yeh picture pehle padho aur baar baar isko dekho. Horizontal axis core count $N$ hai (log scale, taaki $8$ aur $512$ ek hi plot share kar sakein), aur vertical axis aggregate throughput GIPS mein hai (billions of instructions/sec poore chip ke liye — jitna zyada utna better). Teen points marked hain:

Orange dot (low-left) multicore hai: sirf $8$ cores hain, isliye yeh far left mein hai, aur uska total throughput ( $115.2$ GIPS) modest hai — lekin un cores mein se har ek ek sprinter hai, single thread par sabse fast.
Teal dot (far right, high) manycore hai: $512$ cores ise right push karte hain aur uska aggregate throughput $537.6$ GIPS tak le jaate hain — lekin har core ek slow walker hai.
Plum diamond (middle) ek heterogeneous chip hai: kuch fast cores plus kai slow ones, deliberately dono extremes ke beech mein baitha hua. Poori page poochti hai is map par mera workload kahan rehna chahta hai? — L2.3 do labelled dots ko compute aur place karta hai, L3 unke beech crossover dhundta hai, L4 position ko transistor aur power budgets se jodta hai, aur L5 middle mein plum diamond banata hai.

Level 1 — Recognition

Kya tum philosophy ka naam le sakte ho aur definitions padh sakte ho?

Recall Solution L1.1

Manycore. Deciding features:

Bahut zyada core count (6912, multicore ke 2–16 range se bahut upar).
Simple in-order cores 1 IPC aur tiny caches / koi L2 nahi ke saath — parent ke manycore checklist se exactly match. Yeh NVIDIA A100 profile hai parent ke Example 2 se. Figure s01 ke map par, yeh chip far right par teal dot hai.

Recall Solution L1.2

Cores (aur cores add karna). Transistor budget fixed hai, isliye per-core complexity aur cores ki sankhya direct competition mein hain. Yeh parent note ka central sentence hai.

Recall Solution L1.3

(a) → Manycore. Fully data-parallel, har output element independent hai.
(b) → Multicore. Pointer chasing = unpredictable branches + latency-sensitive, simple cores ke liye bahut bura.
(c) → Multicore. Ek bada serial fraction matlab fast individual cores core count se kahin zyada matter karte hain.

Level 2 — Application

Kya tum dono models mein numbers plug kar sakte ho?

Recall Solution L2.1

Step 1 — tool choose karo. Kyun Amdahl aur throughput nahi? Sawaal poochh raha hai ek job kitni tezi se khatam hoti hai, jo ek speedup (latency) ka sawaal hai — yahi exactly Amdahl's Law measure karta hai. Throughput ek alag sawaal answer karta (total work/sec). Step 2 — plug in karo. Kyun yeh numbers waahan? $P = 0.30$ parallel slice hai, $1 - P = 0.70$ un-shrinkable serial slice hai, aur $N = 8$ hai kitne cores parallel slice ko split karte hain: $Speedup = \frac{1}{( 1 - 0.30 ) + \frac{0.30}{8}} = \frac{1}{0.70 + 0.0375} = \frac{1}{0.7375} \approx 1.36 \times$ Iska matlab: 16 cores tak double karne se bhi muskil se fark padta hai — $0.70$ serial term dominate karta hai. Isliye multicore har core ko fast rakhta hai instead of cores add karne ke.

Recall Solution L2.2

Step 1 — serial term set karo. Kyun yeh vanish hota hai? $P = 1$ matlab kuch bhi serial nahi hai, isliye $1 - P = 0$ : koi un-shrinkable part nahi hai jo speedup ko strangle kare. Step 2 — poori job ko $N$ se divide karo. Kyun denominator sirf $1/ N$ hai? Serial term gone hone ke saath, poora runtime parallel part $P = 1$ hai jo sabhi $N = 6912$ cores mein split hota hai: $Speedup = \frac{1}{0 + \frac{1}{6912}} = 6912 \times$ Yeh kaisa lagta hai: perfect parallelism speedup ko $N$ ke saath linearly badhata hai — yahi main reason hai ki manycore chips cores pile on karte hain. (Real hardware ~4000× hit karta hai kyunki memory bandwidth, cores nahi, limit ban jaati hai.)

Recall Solution L2.3

Step 1 — throughput choose karo, speedup nahi. Kyun? Sawaal poochh raha hai ki poora chip sabhi threads mein total work per second kitna karta hai — yeh throughput hai, isliye hum definition box ke chaar factors multiply karte hain. Aggregate throughput chaar factors multiply karta hai: $Multicore = 8 \times 4 \times 4 \times 0.9 = 115.2 GIPS$ $Manycore = 512 \times 1.5 \times 1 \times 0.7 = 537.6 GIPS$ Ratio: $537.6/115.2 = 4.67 \times$ . Manycore throughput mein jeetta hai ~4.7× se — yeh Figure s01 par do dots place karta hai (orange multicore neeche, teal manycore upar). Step 2 — single-thread speed: units straight rakho. Kyun $N$ drop hota hai? Ek single thread ek core par run karta hai, isliye core count uske liye irrelevant hai — sirf per-core factors survive karte hain.

Work per cycle per core sirf $IPC$ hai: multicore $4$ vs manycore $1$ ⇒ har clock mein 4× zyada work.
Throughput per core $f \times IPC$ hai: multicore $4 \times 4 = 16$ GIPS/core vs manycore $1.5 \times 1 = 1.5$ GIPS/core ⇒ $16/1.5 \approx 10.7 \times$ per core faster jab clock include ho. Toh manycore aggregate work mein 4.7× se jeetta hai lekin single-thread work mein per cycle 4× se haarta hai (≈per second 10.7×). Yeh ulta pull exactly Figure s01 mein two-axis tension hai.

Level 3 — Analysis

Kya tum break-even points dhundh sakte ho aur limits ke baare mein reason kar sakte ho?

Recall Solution L3.1

Step 1 — ceiling dhundho. Kyun $N \to \infty$ dekhte hain? Kyunki yeh runtime ke us hisse ko isolate karta hai jo koi bhi parallelism nahi chhu sakti — serial fraction. Jab $N \to \infty$ , parallel term $P / N \to 0$ (ek fixed amount of work ko ever more cores mein divide karne se woh kuch nahi ki taraf shrink ho jaata hai), sirf $(1 - P)$ bachta hai: $Speedup_{m a x} = \frac{1}{1 - P} = \frac{1}{0.05} = 20 \times$ Step 2 — 90% target set karo. Kyun 20 ka 90%, yaani 18? Hum woh core count chahte hain jahan hum achievable gain ka zyaadatar hissa capture kar lein, isliye hum poochhte hain "kaun sa $N$ speedup $= 18$ deta hai?" Full formula ko 18 ke barabar set karo: $18 = \frac{1}{0.05 + \frac{0.95}{N}}$ Step 3 — invert karo. Kyun invert? Unknown $N$ denominator ke andar phansa hua hai; dono sides flip karne se woh free hota hai. Dono sides ke reciprocals lete hain: $0.05 + \frac{0.95}{N} = \frac{1}{18} = 0.0556$ Step 4 — $N$ isolate karo. Serial term subtract karo, phir divide karo: $\frac{0.95}{N} = 0.0556 - 0.05 = 0.00556 \Rightarrow N = \frac{0.95}{0.00556} \approx 171$ Toh 20× limit ka 90% reach karne ke liye ~171 cores chahiye. Insight: us ke baad, extra cores almost waste hain — serial 5% woh wall hai.

Recall Solution L3.2

Do Amdahl speedups ko equal set karo: $\frac{1}{( 1 - P ) + \frac{P}{8}} = \frac{1}{( 1 - P ) + \frac{P}{512}}$ Numerators match karte hain, isliye equality ke liye $\frac{P}{8} = \frac{P}{512}$ chahiye, jo sirf $P = 0$ par hold karta hai. Kisi bhi $P > 0$ ke liye, manycore denominator chhota hai (kyunki $P /512 < P /8$ ), isliye manycore ka Amdahl speedup hamesha ≥ multicore ka hai har $P > 0$ ke liye, aur jab bhi $P > 0$ hota hai strictly greater hota hai. Catch: yeh pure-Amdahl view per-core speed ignore karta hai. Jab tum factor in karo ki har manycore core per clock bahut slow hai (L2.3 se), toh multicore chhote $P$ ke liye jeetta hai — jo exactly L3.3 hai.

Recall Solution L3.3

Step 1 — score banao. Kyun per-core work ko speedup se multiply karte hain? Amdahl speedup har chip ke apne baseline ke relative hai, isliye yeh hide karta hai ki ek multicore core intrinsically 16× stronger hai. Per-core weight $w$ se multiply karna dono chips ko ek common absolute yardstick par laata hai: $score = w \cdot \frac{1}{( 1 - P ) + P / N}$

Multicore: $w_{A} = 16$ , $N_{A} = 8$ .
Manycore: $w_{B} = 1$ , $N_{B} = 512$ . Step 2 — equal set karo. Kyun equal? Crossover exact woh $P$ hai jahan koi nahi jeetta — solve karo aur tum jaanoge ki har chip kis side dominate karti hai: $\frac{16}{( 1 - P ) + \frac{P}{8}} = \frac{1}{( 1 - P ) + \frac{P}{512}}$ Step 3 — cross-multiply karo. Kyun? Yeh ek move mein dono fractions clear karta hai, equation ko plain polynomial arithmetic mein convert karta hai: $16 [(1 - P) + \frac{P}{512}] = (1 - P) + \frac{P}{8}$ Step 4 — dono sides expand karo. Distribute karo aur fractions ko decimals mein convert karo ( $16/512 = 0.03125$ , $1/8 = 0.125$ ): $16 - 16 P + 0.03125 P = 1 - P + 0.125 P$ Step 5 — like terms collect karo. Constants left mein gather karo, $P$ -terms right mein: $16 - 15.96875 P = 1 - 0.875 P \Rightarrow 15 = 15.96875 P - 0.875 P = 15.09375 P$ Step 6 — solve karo. $P = \frac{15}{15.09375} \approx 0.9938$ Toh manycore tabhi overtake karta hai jab $P \approx 0.994$ — tumhe slow cores ki bheed se few fast cores ko beat karne ke liye 99% se zyada parallel code chahiye. Yeh single number chapter ki poori story hai.

Level 4 — Synthesis

Kya tum transistor budget ko performance ke saath combine kar sakte ho?

Recall Solution L4.1

$N_{complex} = \frac{400}{10} = 40, N_{simple} = \frac{400}{0.2} = 2000$ Ratio $= 2000/40 = 50$ . Tum 1 complex core ke badle 50 simple cores trade karte ho — parent ke tradeoff derivation se matching.

Recall Solution L4.2

Absolute score $= w \cdot \frac{1}{( 1 - P ) + P / N}$ with $P = 0.98$ . Complex (w=16, N=40): $denom = 0.02 + \frac{0.98}{40} = 0.02 + 0.0245 = 0.0445$ $score_{C} = 16/0.0445 = 359.6$ Simple (w=1, N=2000): $denom = 0.02 + \frac{0.98}{2000} = 0.02 + 0.00049 = 0.02049$ $score_{S} = 1/0.02049 = 48.8$ Complex layout jeetta hai ( $359.6$ vs $48.8$ , lagbhag $7.4 \times$ ). 98% parallel par bhi, 2% serial part plus 16× per-core weight fast cores ko aage rakhta hai. Flip karne ke liye ~99.4% chahiye (L3.3 logic).

Recall Solution L4.3

Complex: $40 \times 30 = 1200$ W — 200 W se bahut zyada. Max cores jo fit honge: $⌊ 200/30 ⌋ = 6$ cores. $40 - 6 = 34$ drop karo.
Simple: $2000 \times 0.4 = 800$ W — 200 W se zyada. Max: $⌊ 200/0.4 ⌋ = 500$ cores. $2000 - 500 = 1500$ drop karo. Lesson: area ab real limit nahi hai — power hai. Die mein kahin zyada cores fit hote hain jitne tum afford karke switch on kar sako. Yeh "dark silicon" ki reality hai jo Heterogeneous Computing ke peeche hai.

Level 5 — Mastery

Kya tum khud tradeoff design kar sakte ho?

Recall Solution L5.1

Normalization convention (pehle $T$ define karo). $T$ ko poori job ka runtime ek single complex core par running at per-core work 1 maano — ek reference stopwatch jo hum $T = 1$ set karte hain. Given percentages se split karte hue: serial control loop us unit ka $0.6$ leta hai aur parallel kernel $0.4$ . Neeche ka har runtime inhi units mein measured hai, toh chhota = faster. Strategy: serial loop ko ek fast complex core par daalo, aur baaki power budget parallel kernel ke liye simple cores par kharch karo — yeh Heterogeneous Computing idea hai. 1 complex core reserve karo: 30 W. Remaining power $= 200 - 30 = 170$ W. Simple cores: $⌊ 170/0.4 ⌋ = 425$ cores.

Heterogeneous chip: serial complex core par run karta hai, 16× faster ⇒ $0.6/16 = 0.0375$ . Parallel 425 simple cores par run karta hai (har ek 1× work) ⇒ $0.4/425 = 0.000941$ . Total $\approx 0.0375 + 0.000941 = 0.03844$ .
Pure complex, 6 cores: serial $0.6/16 = 0.0375$ . Parallel 6 complex cores par (har ek 16× work, 6 parallel) ⇒ $0.4/ (16 \times 6) = 0.004167$ . Total $\approx 0.0375 + 0.004167 = 0.04167$ . Heterogeneous vs pure-complex ka speedup $= 0.04167/0.03844 \approx 1.084$ , yaani ~8.4% faster, jabki dono same 200 W cap obey karte hain. Synthesis insight: serial loop ko ek fast core chahiye (16× serial work mein 425 slow cores se beat karta hai — kyunki simple cores koi serial speedup nahi dete), jabki parallel kernel swarm ko reward karta hai. Dono mix karna isliye real chips (Heterogeneous Computing, big.LITTLE, CPU+GPU) kisi bhi single philosophy se better hote hain.

Recall Solution L5.2

Ek manycore core lambi memory stalls ko thread-level parallelism se hide karta hai: jab ek thread memory ka wait karte karte stall karta hai (100+ cycles), hardware scheduler instantly doosra ready thread swap in karta hai, isliye pipeline busy rehti hai bina ek bade cache ke jo stall avoid kare ya branch predictor ke jo usse guess kare. Kyunki hamesha aur threads wait kar rahe hote hain, latency tolerate ki jaati hai rather than eliminate ki, jo transistor budget ko aur zyada cores par kharach karne ki freedom deta hai instead of deep Memory Hierarchy par. Ek multicore core bilkul ulti situation face karta hai: yeh often ek single latency-critical thread run karta hai bina koi spare threads ke swap karne ke liye, isliye ek memory stall ya ek mispredicted branch poori job stall kar deta hai. Isliye use large Memory Hierarchy (bada private L1/L2) mein invest karna padta hai data ko paas rakhne ke liye aur branch prediction mein pipeline full rakhne ke liye. Ek line mein: bahut saare threads latency tolerate karne dete hain, isliye tum core ko bare kar sakte ho; kuch threads latency eliminate karne par majboor karte hain, isliye caches aur prediction ke liye pay karna padta hai — yahi exactly reason hai ki do philosophies transistor budget ko opposite directions mein split karti hain.

Recall Self-test summary

Kaun sa single number capture karta hai "code ko kitna parallel hona chahiye manycore ke multicore per-die se beat karne se pehle"? ::: Lagbhag $P \approx 0.994$ — 99% se zyada parallel (L3.3). Modern chips mein core count kya bind karta hai, area ya power? ::: Power (L4.3) — die mein kahin zyada cores fit hote hain jitne power cap allow karta hai. Kyun manycore branch prediction skip kar sakta hai? ::: Thread-level parallelism stalls hide karta hai; ek stalled thread ko ek ready wale se swap kiya jaata hai (L5.2).