Heterogeneous computing concepts: Visual walkthrough

6.1.12 · D2 · Hardware › Parallelism & Multicore › Heterogeneous computing concepts

Step 1 — Poore kaam ko ek time bar ki tarah draw karo

KYA. Imagine karo ki poora program sirf CPU pe chal raha hai. Hume abhi yeh nahi pata ki woh kya karta hai — hume sirf yeh pata hai ki kitna time lagta hai. Us total time ko ek lamba bar banake draw karo.

KYUN. Isse pehle ki hum "cheezein speed up karna" ki baat karein, humein ek honest baseline chahiye: slow, ordinary tarike se kitna time lagta hai. Baaki sab kuch is bar ke against measure kiya jaayega, isliye hum ise ek friendly value $1$ dete hain — ek "poore kaam ke worth ka time".

PICTURE. Neeche diya bar poora kaam hai; iska length exactly $1$ hai. Horizontal axis poore kaam ke fraction ke roop mein time hai (dimensionless, $0$ se $1$ tak); koi vertical scale nahi hai — bar ki height sirf visibility ke liye hai.

Figure — Heterogeneous computing concepts

Step 2 — Bar ko is mein split karo ki GPU pe KYA ja sakta hai aur KYA NAHI

KYA. Bar ko do coloured pieces mein kaato. Ek piece woh kaam hai jo CPU pe hi rehna chahiye (files padhna, awkward if/else logic, glue code). Doosra piece woh kaam hai jo same operation hai jo laakhon numbers pe repeat hoti hai — woh piece jise ek GPU pasand karta hai.

KYUN. GPU har cheez ko speed up nahi kar sakta. Sirf repetitive, parallel part ko faayda hota hai. Isliye kisi bhi real analysis mein sabse pehla kaam yeh poochna hai ki "mere kaam ka kitna hissa eligible bhi hai accelerator ke liye?" Woh sawaal bar ko split karta hai.

Yahan har term bar ke saath ek length hai. Agar $f_{p} = 0.95$ hai toh bar ka $95%$ "GPU-eligible" colour hai aur sirf $5%$ "CPU-only" rehta hai.

PICTURE. Step 1 jaisa wahi bar aur wahi horizontal axis (poore kaam ke fraction ke roop mein time, $0$ se $1$ tak), ab do colours mein split jo milke poore $1$ mein add ho jaate hain. $f_{s}$ pe tick mark karta hai jahan serial khatam hoti hai aur parallel shuru hoti hai.

Step 3 — Sirf parallel slice ko shrink karo

KYA. Parallel slice ko GPU pe bhejon. GPU use tezi se process karta hai, isliye woh coloured slice chhoti ho jaati hai. Serial slice nahi hilti — woh apni poori length pe rehti hai.

KYUN. Yahi poore idea ka dil hai. Speedup koi magic nahi hai jo poore bar ko chota kare; woh sirf us part ko shrink karta hai jo faster machine pe chali. Humein ek number chahiye jo yeh bataye ki "GPU is slice pe kitna faster hai?"

Toh parallel slice, originally length $f_{p}$ , ban jaati hai: $\frac{f _{p}}{S _{GP U}}$

PICTURE. Do bars stacked, same time axis share karte hue (poore kaam ke fraction, $0$ se $1$ tak): upar "before" bar, neeche "after" bar. Dekho pink slice collapse hoti hai jabki yellow serial slice hath dhar ke same length pe rehti hai.

Step 4 — Woh tax add karo jiske baare mein koi baat nahi karta: data move karna

KYA. GPU ki apni memory hoti hai (VRAM). Isse use karne ke liye, CPU ko data copy in karna padta hai, aur baad mein result copy back karna padta hai. Us copying mein time lagta hai — aur yeh hota hai chahe GPU kitni bhi tezi se compute kare.

KYUN. Step 3 mein humne quietly assume kar liya tha ki GPU slice "hand over karne ke liye free" hai. Woh nahi hai. Yeh woh piece hai jo ek khoobsoorat theory ko engineering reality mein badalta hai — aur yeh wahi piece hai jo wall build karegi. Hum is cost ko naam dete hain.

Pehle, kisi bhi copy ke do ingredients ko naam do. Copying zyada time leti hai jab copy karne ke liye zyada bytes hon, aur kam time leti hai jab pipe moti ho. Isliye humein exactly do named quantities chahiye:

Kitna data hai? Yeh depend karta hai ki parallel slice kitni badi hai. Ek badi parallel slice ( $f_{p}$ jo $1$ ke closer ho) ka matlab usually zyada numbers bhejne hain, isliye $D$ actually $f_{p}$ ka ek function hai. Hum us dependence ko explicitly $D (f_{p})$ likhte hain — isse padho " $D$ $f_{p}$ ke function ke roop mein", yaani "woh bytes jinhe hume move karna hai jab parallel fraction $f_{p}$ ho". Humein sirf yeh do key facts chahiye: $D (f_{p})$ badhta hai jab $f_{p}$ badhta hai, aur $D (0) = 0$ (koi parallel slice nahi ⇒ move karne ke liye koi bytes nahi).

Definition Transfer overhead

$t_{x f er} (f_{p})$ Ab tax build karo. Ek round-trip copy ke seconds, $\frac{2 D ( f _{p} )}{B}$ , lo aur $T_{0}$ se divide karo taaki yeh original kaam ke fraction ke roop mein express ho ( $f_{s}$ aur $f_{p}$ ki tarah, jo bhi kaam ke fractions hain): $t_{x f er} (f_{p}) = \frac{1}{T _{0}} \cdot \frac{2 D ( f _{p} )}{B}$

$2$ = "copy in aur copy back" (do one-way trips),
$D (f_{p})$ = move karne ke bytes, jo parallel slice ke saath badhte hain,
$B$ = pipe width (bytes per second),
$T_{0}$ se divide karna seconds ko poore kaam ke fraction mein convert karta hai.

Hum $t_{x f er} (f_{p})$ $(f_{p})$ dikhake likhte hain precisely isliye taaki koi ise constant na samjhe. Agar copying original kaam ke time ka $2%$ costs karta hai, toh $t_{x f er} (f_{p}) = 0.02$ us particular $f_{p}$ ke liye. Important consequence: agar $f_{p} = 0$ toh $D (0) = 0$ , isliye $t_{x f er} (0) = 0$ — koi parallel kaam nahi matlab ship karne ke liye kuch nahi, hence koi tax nahi.

Ab teen pieces ko stack karo naya total time paane ke liye: $T_{h e t er o} = serial slice (unchanged) f_{s} + parallel slice (shrunk) \frac{f _{p}}{S _{GP U}} + data-move tax (grows with f_{p}) t_{x f er} (f_{p})$

Subscript hetero ka matlab hai "heterogeneous CPU+GPU system pe". Equation left to right padho aur tum literally new bar left to right padh rahe ho.

PICTURE. Same time axis pe nayi bar (poore kaam ke fraction): full yellow serial slice + tiny pink shrunk slice + ek grey tax block jiska length $t_{x f er} (f_{p})$ hai, yaani yeh depend karta hai ki humne kitna parallel data ship kiya.

Step 5 — Purana bar naaye bar se compare karo → speedup formula

KYA. Speedup simply yeh hai: "bar kitne times chhota hua?" Purana bar divided by naaya bar.

KYUN. Hum ek boss ko report karne ke liye ek single number chahte hain: "humne ise $X$ times faster banaya." Woh number un do bar lengths ka ratio hai jo humne abhi draw ki hain.

Parent note ke numbers plug karo ( $f_{s} = 0.05$ , $f_{p} = 0.95$ , $S_{GP U} = 50$ , aur is workload ke liye $t_{x f er} (0.95) = 0.02$ ): $S_{o v er a l l} = \frac{1}{0.05 + \frac{0.95}{50} + 0.02} = \frac{1}{0.05 + 0.019 + 0.02} = \frac{1}{0.089} \approx 11.2 \times$

Fifty-times-faster hardware, eleven-times-faster program. Woh do chhote constants $0.05$ aur $0.02$ usका zyaadatar hissa kha gaye.

PICTURE. Upar purana bar, neeche naaya bar, dono same time axis pe (purane kaam ke fraction, $0$ se $1$ tak), ek bracket ke saath jo "≈ 11× shorter, not 50×" dikhata hai.

Step 6 — Wall: GPU ko infinity tak push karo

KYA. Ab ek thought experiment karo. Maan lo GPU infinitely fast ho gaya — shrunk slice bilkul gayab ho jaati hai, $f_{p} / S_{GP U} \to 0$ .

KYUN. Yeh ceiling dhundne ka classic move hai. Agar ek perfect GPU bhi serial slice aur tax ko nahi hata sakta, toh woh dono milke ek hard limit hain jo koi hardware nahi tod sakta. Hum limit use karte hain kyunki yeh exact sawaal ka jawab deta hai: "sabse best possible outcome kya hai?"

Toh ek magical GPU bhi is workload ke liye $14.3 \times$ pe cap out karta hai. Jo $11.2 \times$ hum pehle se paaye woh ceiling ke surprisingly close hai — faster hardware kharidne se shayad hi kuch fark padega. Serial slice aur transfer tax hi wall hain.

PICTURE. Horizontal axis: $S_{GP U}$ (GPU kitne times faster hai, dimensionless). Vertical axis: $S_{o v er a l l}$ (overall program speedup, dimensionless). Blue curve upar jaati hai jab $S_{GP U}$ badhta hai, phir $1/ (f_{s} + t_{x f er} (f_{p}))$ pe ek dashed pink ceiling line ke against flat ho jaati hai.

Step 7 — Edge & degenerate cases (koi gap mat chhodo)

KYA. Har woh case jo ek reader hit kar sakta hai. Hum formula ko extremes pe test karte hain taaki kuch surprise na kare. Zaroori hai, hum ab respect karte hain ki $t_{x f er} (f_{p})$ $f_{p}$ pe depend karta hai: koi parallel kaam nahi ⇒ koi data move nahi ⇒ koi tax nahi.

Worked example Formula ke paanch corners

$S_{o v er a l l} = \frac{1}{f _{s} + f _{p} / S _{GP U} + t _{x f er} ( f _{p} )}$ use karte hue $f_{s} + f_{p} = 1$ , $f_{s}, f_{p} \geq 0$ ke saath:

(a) Kuch bhi parallel nahi, $f_{p} = 0$ (isliye $f_{s} = 1$ ). Ship karne ke liye koi data nahi, isliye $t_{x f er} (0) = 0$ automatically. Denominator $= 1 + 0 + 0 = 1$ . Speedup $= 1$ — bilkul CPU jaisa hi, na better, na worse. Sahi model mein tumne honestly kuch nahi paaya aur kuch nahi khoya. Lesson: GPU path setup hi mat karo.

(b) Sab kuch parallel, $f_{s} = 0$ isliye $f_{p} = 1$ , aur (idealised) koi tax nahi $t_{x f er} (1) = 0$ . Denominator $= 0 + f_{p} / S_{GP U} + 0 = 1/ S_{GP U}$ (kyunki $f_{p} = 1$ ). Speedup $= S_{GP U}$ . Formula sahi tarike se raw GPU speedup pe collapse karta hai — ideal.

(c) Fully parallel lekin tax REAL hai, $f_{s} = 0$ isliye $f_{p} = 1$ , aur $t_{x f er} (1) > 0$ . Denominator $= 0 + 1/ S_{GP U} + t_{x f er} (1)$ . Speedup $= \frac{1}{1/ S _{GP U} + t _{x f er} ( 1 )} < S_{GP U}$ . Zero serial kaam ke saath bhi, transfer tax akela tumhe raw GPU speedup se neeche kheench leta hai — wall tab bhi exist karta hai jab kuch bhi serial nahi hota. Yeh case (b) ka honest version hai.

(d) Free transfers, $t_{x f er} (f_{p}) = 0$ (unified memory, no copies). Speedup $= \frac{1}{f _{s} + f _{p} / S _{GP U}}$ — yeh classic Amdahl's Law hai. Hamara formula Amdahl plus ek data-movement tax hai; tax hataao aur Amdahl wapas aa jaata hai. Dekho Amdahl's Law and Scalability.

(e) Tiny GPU speedup, $S_{GP U} = 1$ (GPU faster nahi), real parallel kaam ke saath $f_{p} > 0$ isliye $t_{x f er} (f_{p}) > 0$ . Denominator $= f_{s} + f_{p} + t_{x f er} (f_{p}) = 1 + t_{x f er} (f_{p})$ . Speedup $= \frac{1}{1 + t _{x f er} ( f _{p} )} < 1$ — tum exactly tax ki wajah se slow ho gaye, kyunki tumne zero compute benefit ke liye data move karne ki payment ki.

PICTURE. Paanch mini-bars stacked, sab same time axis pe (purane kaam ke fraction): ek dotted vertical line purani baseline length $1$ mark karti hai, aur har case ki nayi bar sahi length par draw ki gayi hai taaki tum dekh sako ki woh baseline ke paas, neeche, ya paar kaahan jaati hai.

Ek picture summary

Sab kuch upar, compressed: length $1$ ke ek bar se shuru karo, use serial (yellow, immovable) aur parallel (pink, shrinkable) mein split karo $f_{s} + f_{p} = 1$ ke saath, pink ko $S_{GP U}$ se collapse karo, grey transfer tax $t_{x f er} (f_{p})$ jodo (jo sirf isliye wahan hai kyunki move karne ke liye pink hai), phir speedup padho old length ÷ new length ke roop mein — hamesha ke liye yellow+grey stub se bounded.

Recall Feynman retelling — ise simple words mein wapas bolo

Apne poore program ko ek chocolate bar ki tarah picture karo, aur uski length batati hai ki kitna time lagta hai.

Bar ke kuch squares "boring glue work" hain (files padhna, weird logic) — woh CPU pe rehne chahiye, aur woh kabhi chhote nahi hote. Baaki bar "ek million baar wahi math" hai — woh squares GPU pe ja sakte hain. Dono tarah ke squares hamesha milke poora bar banate hain, aur tumhare paas negative squares ya full bar se zyada nahi ho sakta.

GPU fast hai, isliye woh sirf un math squares ko shrink karta hai, unki length ko divide karta hai usse jo woh kitna faster hai. Lekin ek catch hai: tumhe data GPU tak le jaana padta hai aur jawab wapas laana padta hai, aur woh le jaana ek grey chunk costs karta hai. Woh grey chunk kitna bada hai? Woh bytes ki number ( $D$ ) divided by wire kitna fast hai ( $B$ ), times two round trip ke liye. Aur woh bytes sirf isliye exist karte hain kyunki feed karne ke liye math squares hain — agar bar ka kuch bhi GPU nahi ja sakta, toh le jaane ke liye kuch nahi aur koi grey chunk bhi nahi.

Toh nayi bar = immovable glue squares + tiny shrunken math squares + grey carrying chunk. Tumhara speedup simply yeh hai ki nayi bar purani bar se kitne times chhoti hai.

Punchline: jab tak real GPU kaam hai, ek infinitely fast GPU bhi glue squares aur grey carrying chunk chhod jaata hai. Woh ek wall set karte hain jise tum kabhi nahi tod sakte. Yahi wajah hai ki 50×-faster GPU ne sirf 11× program diya — hum pehle se wall ke almost paas the.

Yeh bhi dekho: Parallel Programming Models · Memory Hierarchy and Caching · Power and Energy Optimization · SIMD and Vector Processing · Amdahl's Law and Scalability