Heterogeneous computing concepts: Question bank — Hinglish deep dive

6.1.12 · D5 · Hardware › Parallelism & Multicore › Heterogeneous computing concepts

Shuru karne se pehle, teen words baar baar aate hain — inhe plain language mein pin karte hain taaki koi bhi reveal inhe cold use na kare:

Neeche diye gaye sabhi reveals mein use hone wale har symbol ki definition yahan ek baar di gayi hai, taaki kuch bhi cold na aaye:

Formula Symbol glossary aur do formulas jinpar ye traps based hain

Symbols (units ke saath):

$D$ ::: link ke across move kiya gaya data size, gigabytes (GB) mein.
$B$ ::: interconnect bandwidth (jaise PCIe), GB per second mein.
$W$ ::: task ka total work, floating-point operations (FLOPs) mein.
$F_{C P U}$ , $F_{GP U}$ ::: har processor kitni fast compute karta hai, FLOPs per second (FLOPS) mein.
$f_{s}$ ::: program ka woh fraction jo serial hai (CPU-only), $0 \leq f_{s} \leq 1$ .
$f_{p}$ ::: woh fraction jo parallel hai (GPU-able), jahan $f_{s} + f_{p} = 1$ .
$t_{x f er}$ ::: data-transfer overhead, original CPU-only time ke fraction ke roop mein.
$t_{i}$ ::: pipeline stage $i$ ka time (read, transfer, compute, write), seconds mein.

Break-even work (Heterogeneous computing concepts se) — woh minimum work jo offloading ko worthwhile banata hai: $W > \frac{2 D F _{C P U} F _{GP U}}{B ( F _{GP U} - F _{C P U} )}$ $2$ isliye hai kyunki data link ko do baar cross karta hai (CPU→GPU→CPU). Is $W$ se neeche, transfer cost compute savings ko beat kar deti hai.

Heterogeneous Amdahl speedup (Amdahl's Law and Scalability se): $S_{o v er a l l} = \frac{1}{f _{s} + \frac{f _{p}}{S _{GP U}} + t _{x f er}}$

Linked prerequisites jo tum khule rakhna chahte ho: Amdahl's Law and Scalability, Roofline Performance Model, Memory Hierarchy and Caching, Parallel Programming Models, SIMD and Vector Processing, DMA and I/O Controllers, Power and Energy Optimization.

True or false — justify karo

A GPU sirf "ek faster CPU" hai

False — yeh higher throughput lekin per task higher latency hai; ek single GPU thread CPU thread se slower hota hai, aur yeh tabhi jeetata hai jab thousands of threads ek saath same operation karte hain.

GPU add karna hamesha program ko fast karta hai

False — agar parallel fraction tiny hai ya data-transfer cost compute savings se zyada hai, toh heterogeneous version CPU alone se slower ho sakta hai (upar glossary mein defined break-even work

W

se neeche).

Heterogeneous matlab "ek se zyada core"

False — woh sirf multicore hai. Heterogeneous matlab different types ke cores (alag instruction sets, memory models, ya strengths), jaise CPU + GPU + NPU.

Aath identical CPU cores ek heterogeneous system banate hain

False — identical cores jo same instruction set chalate hain woh homogeneous hain; heterogeneity ke liye asymmetric strengths chahiye, sirf multiple units nahi.

Agar

S_{GP U} = \infty

ho, toh overall speedup infinite hai

False — Amdahl isko

\frac{1}{f _{s} + t _{x f er}}

par cap karta hai; serial fraction aur transfer tax fixed costs hain jinhe GPU kabhi touch nahi kar sakta.

Matrix–matrix multiply ek great GPU workload hai

True — yeh

O (n^{2})

data par

O (n^{3})

work karta hai, isliye arithmetic intensity

n

ke saath badhti hai; har byte par bahut zyada math cores ko fed rakhti hai.

Vector addition ek great GPU workload hai

False —

O (n)

data par

O (n)

work matlab ~1 FLOP/byte; yeh memory-bound hai, isliye transfer aur bandwidth dominate karte hain aur GPU data ka intezaar karte karte idle baithta hai.

Unified memory data-movement cost ko poori tarah remove kar deta hai

False — yeh code mein explicit copy step ko remove karta hai, lekin bytes physically interconnect ke across travel karte rehte hain; yeh cost ko hide karta hai, delete nahi.

FPGAs CPUs se slower hain kyunki yeh lower clock speeds par chalte hain

False — ek FPGA apne target datapath ke liye per watt aur per operation bahut faster ho sakta hai, kyunki yeh general instructions fetch/decode karne ki jagah ek custom pipeline banata hai; akela clock speed misleading hai.

Ek pipelined model ka total time har item ke saare stage times ka sum hota hai

False — pipeline fill hone ke baad, throughput slowest stage se set hota hai, isliye total time

\approx (\sum t_{i}) + (n - 1) max (t_{i})

hota hai,

n \sum t_{i}

nahi.

Error dhundho

"CPUs ke paas thousands of cores hain isliye woh parallel work mein jeette hain."

Swap ho gaya. GPUs ke paas thousands of simple cores hote hain; CPUs ke paas few powerful cores hote hain jo latency aur control flow ke liye tuned hain.

"Hume CPU ka time bachane ke liye tiny tasks GPU ko bhejna chahiye."

Wrong direction — tiny tasks mein work

W

kam hota hai, isliye transfer overhead dominate karta hai aur tum haarte ho. Tabhi offload karo jab work, data moved ke relative large ho.

T_{t o t a l} = T_{co m p u t e_GP U}

; copies ko ignore karo."

Do transfers ko ignore kar raha hai.

T_{t o t a l} = T_{C P U \to GP U} + T_{co m p u t e_GP U} + T_{GP U \to C P U}

, aur woh copies low-intensity kernels ke liye aksar dominate karti hain.

"Arithmetic intensity FLOPS mein measure hoti hai."

Nahi — FLOPS ek rate hai (ops per second). Intensity FLOP per byte hai, work aur data moved ka ratio; yeh decide karta hai ki tum compute-bound ho ya memory-bound.

"Amdahl kehta hai faster jaane ke liye serial fraction badhao."

Ulta hai — speedup tab improve hota hai jab

f_{s}

shrink kare. Ek bada serial fraction woh ceiling hai jo sab kuch cap kar deta hai.

"Bade caches GPU ko throughput king banate hain."

GPU many cores + huge bandwidth se jeetta hai, bade caches se nahi. Uske caches per core small hote hain; CPU woh hai jiske paas latency ke liye large caches hain.

"Slow write stage wale pipeline ko faster GPU add karke fix kiya ja sakta hai."

Nahi — bottleneck max-time stage hai (woh write). Non-bottleneck stage ko speed up karne se bahut kam fayda hota hai; slow stage ko balance ya overlap karo.

Why questions

CPUs branch prediction aur out-of-order execution kyun use karte hain jabki GPUs mostly nahi karte?

CPUs irregular control flow ke liye low latency chase karte hain, isliye woh stalls hide karne mein transistors spend karte hain; GPUs latency ko thousands of threads ke beech switch karke hide karte hain, isliye woh transistors zyada ALUs par spend karte hain.

Explicit data movement ko heterogeneous systems ka "bottleneck" kyun kehte hain?

Har processor ki apni memory hoti hai, isliye data ko limited-bandwidth link ke across copy karna padta hai; jab compute per byte kam hota hai, toh yeh copy time compute time ko swamp kar deta hai.

T_{h e t er o} = \frac{2 D}{B} + \frac{W}{F _{GP U}}

mein transfer term mein factor of 2 kyun hai?

Data CPU→GPU aur wapas GPU→CPU jaata hai, isliye size

D

link ko do baar cross karta hai.

Specialized accelerators (TPU, crypto units) itne energy-efficient kyun hain?

Woh ek operation hardwire kar dete hain isliye koi instruction fetch/decode/schedule overhead nahi hota — har transistor us ek task ke liye useful work karta hai, jo 10–100× better energy per op deta hai (Power and Energy Optimization).

Ek pipelined model kai items stream karne ke liye offload model ko kyun beat kar sakta hai?

Pipelining alag items ke across read, transfer, compute, aur write ko overlap karta hai, isliye full latency sirf ek baar pay karte ho aur phir slowest-stage rate par chalte ho; offload har item ke stages serially karta hai.

GPUs ke liye large caches se zyada high memory bandwidth kyun matter karta hai?

GPUs huge, regular datasets mein ek baar stream karte hain, isliye sabhi cores ko fast feed karna ek chhote working set ko reuse karne se zyada zaroori hai; caches reuse mein help karte hain, bandwidth streaming mein.

Roofline model arithmetic intensity ki kyun parwah karta hai?

Kyunki intensity ek kernel ko roof par place karti hai: low intensity bandwidth-limited slope (memory-bound) ko hit karti hai, high intensity flat peak-FLOPS ceiling (compute-bound) ko hit karti hai — dekho Roofline Performance Model.

Edge cases

Arithmetic intensity ≈ 0 (koi compute nahi, pure copy) par kya hota hai?

Kernel poori tarah transfer hai; heterogeneous execution strictly slower hai kyunki tum kisi computational benefit ke bina data copy kar rahe ho. Ise CPU par hi rakhо.

Break-even formula

W > \frac{2 D F _{C P U} F _{GP U}}{B ( F _{GP U} - F _{C P U} )}

mein agar

F_{GP U} = F_{C P U}

(GPU faster nahi) ho toh kya?

Denominator

B (F_{GP U} - F_{C P U}) \to 0

, isliye required work

W \to \infty

— koi bhi work transfer justify nahi karta jab GPU koi speed advantage nahi deta.

Agar parallel fraction

f_{p} = 0

ho toh kya?

Saara work serial hai;

S_{o v er a l l} = \frac{1}{f _{s} + t _{x f er}} \leq 1

, isliye GPU sirf transfer overhead add karta hai aur help nahi kar sakta.

Agar

f_{s} = 0

lekin

t_{x f er} > 0

ho toh kya?

Even ek perfectly parallel program

\frac{1}{t _{x f er}}

par cap hota hai — transfer tax akela ceiling set karta hai, jo unified memory ya compute ke saath copies overlap karne ki motivation deta hai.

Ek pipeline mein,

n \to \infty

hone par speedup kya limit karta hai?

Yeh

\frac{\sum t _{i}}{m a x t _{i}}

approach karta hai — slowest stage throughput cap karta hai chahe tum kitne bhi items stream karo.

Data size

D = 0

(data GPU par pehle se resident hai) ka kya matlab hai?

Transfer term vanish ho jaata hai, isliye koi bhi parallel speedup help karta hai; isliye kernels ke across data on-device rakhna (round trips avoid karna) ek key optimization hai.

Recall Le jaane wala one-line summary

Heterogeneous tabhi jeetta hai jab parallel work bada ho, arithmetic intensity high ho, serial fraction chhota ho, aur transfers hide ho jaayein — inme se koi ek miss karo aur accelerator dead weight ban jaata hai.