Domain-specific accelerators: Question bank — Hinglish deep dive

6.5.6 · D5 · Hardware › Advanced & Emerging Architectures › Domain-specific accelerators

Shuru karne se pehle, vocabulary ka ek-line refresher taaki neeche kuch bhi begana na lage. Is page mein baad mein use kiya gaya har symbol pehle yahan define kiya gaya hai.

Definition Woh words aur symbols jo jawab dene se pehle tumhare paas hone chahiye

ops/joule = ek unit energy mein kitne useful math operations milte hain. Dark silicon ke daur mein yahi — clock speed nahi — scarce currency hai.
$W$ = work: operations (FLOPs ya MACs) ki sankhya jo ek kernel ko karni hai. Ek count, unitless.
$Q$ = data moved: woh bytes ki sankhya jo kernel ko main memory se fetch karni ya usme likhni hai.
arithmetic intensity $I = W / Q$ = operations kiye gaye per byte moved. High $I$ ka matlab hai har loaded byte bahut baar reuse hota hai; low $I$ ka matlab hai baar baar re-fetch karna padta hai.
$B$ = memory bandwidth: kitne bytes per second memory deliver kar sakti hai. Chip ke memory system ki ek property.
$P_{peak}$ = peak throughput (TOPS mein measure hota hai, tera-operations-per-second): woh maximum jo compute units kar sakti thi agar perfectly fed hoti. Ek ceiling, jo rarely reach hoti hai. Jab tum "peak TOPS" words mein dekhte ho, iska matlab exactly yahi symbol $P_{peak}$ hai.
$P_{attain}$ = attainable throughput: jo tum actually paate ho, Roofline model ke hisaab se $P_{attain} = min (P_{peak}, B \cdot I)$ .
compute-bound = $B \cdot I \geq P_{peak}$ : math units bottleneck hain (tum unhe poori tarah use kar rahe ho). memory-bound = $B \cdot I < P_{peak}$ : tum data ka intezaar kar rahe ho; math units idle baithe hain.

True or false — justify

Har jawab mechanism dena chahiye, sirf verdict nahi.

DSAs mainly CPUs se zyada clock frequency par run karke jeette hain.

False. Yeh typically clock lower karte hain; jeet massive parallelism, deleted control overhead, lower precision, aur data reuse se aati hai — dekho Systolic arrays. Currency ops/joule hai, GHz nahi.

Agar kisi chip ki peak throughput

P_{peak}

double ho jaaye, toh real application throughput bhi double ho jaati hai.

False. Tabhi, jab kernel compute-bound tha. Agar woh memory-bound hai (

B \cdot I < P_{peak}

), toh extra math units idle baithe rahenge aur kuch bhi improve nahi hoga.

FP32 ki jagah INT8 use karna hamesha model ki accuracy corrupt karta hai.

False. Inference ke liye, laakhon weights rounding noise ko average kar dete hain, isliye INT8 mein negligible accuracy jaati hai. Bit-exact scientific computing ek alag domain hai jo ise tolerate nahi karta.

Ek systolic array matmul ke liye zaruri multiply-adds ki sankhya reduce karta hai.

False. Woh phir bhi

N^{3}

MACs karta hai; jo reduce hota hai woh hai memory traffic per op, kyunki har loaded value ko grid ke andar

O (N)

baar reuse kiya jaata hai.

DSAs achhe systems mein CPUs ko obsolete bana dete hain.

False. DSAs narrow hote hain — unke domain se bahar kuch bhi dheere ya bilkul nahi chalta. Real machines heterogeneous hoti hain: CPU orchestrate karta hai aur DSA hot kernel run karta hai.

Dennard scaling ka ant aur Moore's Law ka ant ek hi event hai.

False. Moore's Law (transistor count) limping on raha; Dennard scaling (transistors shrink hone par constant power density) ~2006 mein khatam hua. Humein zyada transistors mile jo sab power nahi ho sakte the — woh gap Dark silicon hai.

Software-managed scratchpad ek alag naam wala cache hi hai.

False. Cache transistors/energy tags, replacement logic, aur coherence par kharch karta hai. Scratchpad yeh sab chhod deta hai deterministic latency ke liye — software decide karta hai kya wahan rehta hai.

Lower precision energy mein help karta hai lekin throughput mein nahi.

False. Dono mein. Ek multiplier ki cost bit-width

n

ke saath roughly

n^{2}

scale karti hai, isliye bits half karne se same area mein ~4× zyada units milte hain aur ~4× kam energy per op.

Memory bandwidth

B

badhana hamesha attainable performance badhata hai.

False. Tabhi jab memory-bound ho. Jab

B \cdot I \geq P_{peak}

ho jaata hai tum compute-bound ho; zyada bandwidth tab waste hai, bilkul waise jaise zyada MACs memory-bound hone par waste hote hain.

Spot the error

Har statement mein ek flaw hai. Use identify karo. (Yaad raho $W$ = operations, $Q$ = bytes moved, $I = W / Q$ .)

"Hamare kernel ne

W

FLOPs kiye aur

Q

bytes move kiye, isliye intensity

I = Q / W

hai."

Inverted hai. Intensity operations per byte hai,

I = W / Q

. Isko flip karne se data-reuse-heavy kernels memory-bound lagenge, jo sach ke bilkul ulta hai.

"Peak throughput 90 TOPS hai aur bandwidth ceiling 15 TOPS hai, isliye attainable 90 hai kyunki hum max lete hain."

Galat operator hai. Roofline min leta hai:

P_{attain} = min (90, 15) = 15

TOPS. Tum pehle jis wall se takraate ho ussi se limited ho, zyada friendly wali se nahi.

"Ek CPU ka integer add energy waste karta hai kyunki ALU adder expensive hai."

Ulta hai. Adder khud ek picojoule ka tiny fraction hai; waste fetch + decode + register-file + control mein hai, jo op se 10–100× zyada cost kar sakta hai. Wahi overhead hai jo specialization delete karti hai.

"Kyunki ek systolic array compute-bound hai, uski arithmetic intensity

O (1)

hai."

Contradiction hai. Woh compute-bound isliye hai kyunki intensity

O (N)

hai (har value array dimension mein reuse hoti hai).

O (1)

intensity memory-bound naive case hai.

"Ek single core ke transistors double karne se speed double hoti hai — isliye hum bade cores banate hain."

Pollack's Rule violate karta hai: single-core performance sirf

area

ke roop mein badhti hai, isliye double transistors ~1.4× deta hai, 2× nahi. Wahi diminishing return specialize karne ka ek reason hai.

"Training poori tarah FP32 mein rehni chahiye warna gradients explode ho jaate hain, isliye DSAs training mein help nahi kar sakte."

Overstated hai. Mixed precision training sirf sensitive parts (jaise accumulations, master weights) ko high-precision rakhta hai jabki bulk math bf16/FP16 mein hoti hai — DSAs training ko routinely accelerate karte hain.

Why questions

Underlying reason batao, sirf "kyunki" nahi.

Kyun arithmetic intensity

I

, raw work

W

nahi, decide karta hai ki DSA help karega ya nahi?

Kyunki time

max (W / P_{peak}, Q / B)

hai; agar

Q / B

term dominate kare, toh compute add karna (

P_{peak}

badhana) kuch nahi badalta — sirf reuse badhana (

I = W / Q

) memory term ko chhota karta hai.

Kyun ek systolic array "ek baar load karo, phir stream karo"?

Ek expensive memory load ko

O (N)

saste on-chip reuses mein amortize karne ke liye, intensity ko

O (1)

O (N)

tak push karke ek memory-bound problem ko compute-bound mein badalne ke liye.

Dark-silicon era mein ops/joule binding metric kyun hai, clock speed kyun nahi?

Kyunki hum jitne transistors fit kar sakte hain utne sab ek saath power/cool nahi kar sakte, isliye limit energy budget hai; jo har joule mein zyada useful kaam kare woh chip ka zyada hissa light up kar sakta hai.

DSAs low precision kyun exploit kar sakte hain jo ek general CPU aasaani se nahi kar sakta?

Ek CPU ko arbitrary workloads ke liye bit-exact sahi rehna padta hai; ek DSA ek domain tolerance exploit karta hai (neural nets mein statistical averaging) jo guarantee karti hai ki wahan noise harmless hai.

DSAs out-of-order execution aur branch prediction kyun drop karte hain?

Unke kernels regular aur predictable hote hain (dense matmul mein koi data-dependent branches nahi hote), isliye speculation kuch nahi kharidta lekin area aur energy kharch karta hai — ise delete karna pure profit hai.

"Bahut saare specialized blocks banana" "ek huge fast core" se better kyun hai?

Pollack's Rule ke hisaab se ek huge core transistor per transistor sub-linear return deta hai, aur Dark silicon ke hisaab se tum sab kuch power nahi kar sakte vaise bhi — toh sirf wahi specialized block light up karo jo current task ko chahiye.

Ek GPU ek TPU se zyada general kyun hai?

Ek GPU bahut saare parallel workloads ke liye programmable SIMD lanes aur caches rakhta hai; ek TPU systolic matmul dataflow hard-wire karta hai, woh flexibility trade karke aur bhi zyada matmul efficiency ke liye.

Edge cases

Woh scenarios jo naive intuition todte hain.

Kya hota hai DSA ke advantage ka jab kernel tiny ho (registers mein fit ho jaaye, few ops)?

Advantage collapse ho jaata hai — Amdahl's Law dominance launch/overhead par shift ho jaati hai, aur amortize karne ke liye koi data-reuse nahi hota; ek CPU accelerator ko kaam dene mein lagte waqt se pehle finish kar sakta hai.

Kya hoga agar arithmetic intensity bahut high ho (huge reuse)?

Tum firmly compute-bound ho aur

P_{peak}

par pin ho; ab upar jaane ka ek hi raasta hai MACs add karna ya precision throughput badhana — bandwidth upgrades waste hain.

Kya hoga agar

B \cdot I

exactly

P_{peak}

ke barabar ho?

Tum bilkul roofline ke "ridge point" par ho — perfectly balanced, dono walls ka 100% use ho raha hai. Koi bhi aage optimization compute aur memory dono ko saath mein move karna chahiye.

Kya hota hai agar tum ek systolic array ko aisa data do jo grid fill na kare (ek

128 \times 128

array par

10 \times 10

matmul)?

Zyaadatar MAC cells idle rahte hain; utilization girti hai aur effective TOPS peak se bahut neeche aa jaata hai — chhhote ya oddly-shaped problems array waste karte hain, yahi ek real reason hai ki DSAs large batches pasand karte hain.

INT8 boundary par accuracy ki kya story hai — large magnitude wale outlier weights ke saath?

Uniform Quantization outliers ko clip ya coarsely bin kar sakta hai, accuracy hurt karti hai; fix hai per-channel scales ya outlier layers ko higher precision mein rakhna, yani inference par bhi mixed precision.

Jaise jaise workload ka non-accelerated ("serial") fraction badhta hai, DSA benefit ka kya hota hai?

Amdahl's Law total speedup ko cap karta hai: agar DSA fraction

f

ko touch nahi kar sakta, overall speedup

1/ f

se bound hai chahe accelerated part kitna bhi fast ho jaaye.

Zero data reuse par (har operand exactly ek baar fetch hota hai), ek systolic array se best kya milta hai?

Woh memory-bound case

I \approx O (1)

mein degenerate ho jaata hai — grid koi help nahi karta kyunki poora point (reuse) absent hai; tum purely bandwidth

B \cdot I

se limited ho.

Recall Self-check: woh ek sentence jo sab kuch ek saath baandh deta hai

Agar tum is line ko explain kar sako, tumne topic samajh liya hai ::: DSAs general-purpose overhead delete karke aur arithmetic intensity badhane ke liye data on-chip reuse karke jeette hain, taaki Roofline model bahut saare low-precision MACs ko fed rakh sake aur far more ops/joule deliver kare — lekin sirf apne narrow domain ke andar.