Domain-specific accelerators: Worked examples — Hinglish deep dive

6.5.6 · D3 · Hardware › Advanced & Emerging Architectures › Domain-specific accelerators

Yeh page drill ground hai. Parent note ne ideas banaye the (Roofline model, Systolic arrays, energy-per-op). Yahan hum un ideas ko har us situation se guzarte hain jo exam ya real chip throw kar sakti hai — inequality ka har sign, zero/degenerate cases, limits, ek word problem, aur ek nasty twist.

Kisi bhi example se pehle, hum tools aur unke symbols pe agree karte hain taaki kuch bhi undefined na rahe.

Definition Woh teen numbers jis par yeh poori page chalti hai

$P_{peak}$ = chip ki peak compute rate, operations per second mein measure hoti hai. TOPS = Tera-Ops/s = $1 0^{12}$ ops/s. Yeh hai "kitne multiply-adds silicon kar sakta hai agar kabhi starved na ho."
$B$ = memory bandwidth, bytes per second jo chip memory se pull kar sakta hai. GB/s = $1 0^{9}$ bytes/s, TB/s = $1 0^{12}$ bytes/s.
$I$ = arithmetic intensity, operations performed per byte moved. Unit: ops/byte. Yeh data reuse measure karta hai — ek bada $I$ matlab har fetched byte bahut saare math ops mein use hota hai.
$P_{attain}$ = is chip par is kernel ke liye actually achievable performance (ops/s) — woh real rate jo aapko milti hai jab aap account karte hain ki kaun sa ceiling aapko limit kar raha hai. Yeh kabhi bhi kisi bhi ceiling se bada nahi hota.

Neeche ki figure exactly yahi picture draw karti hai: blue line hai $P_{attain}$ , yellow dashed line hai rising memory ceiling $B \times I$ , red dashed line hai flat compute ceiling $P_{peak}$ , aur green dot ridge $I^{⋆}$ ko mark karta hai jahan woh cross karte hain. Isse dhyan mein rakho — neeche ka har roofline example is curve par placed ek point hai.

Scenario matrix

Har DSA numeric question inhi cells mein se ek hai. Last column us example ka naam deta hai jo use nail karta hai.

#	Case class	Jis twist ko yeh test karta hai	Example
A	Memory-bound ( $I < I^{⋆}$ )	extra MACs waste ho jaate hain	Ex 1
B	Compute-bound ( $I > I^{⋆}$ )	zyada bandwidth waste ho jaati hai	Ex 2
C	Balanced ( $I = I^{⋆}$ , boundary)	exact ridge point	Ex 3
D	Degenerate: $I \to 0$ (no reuse)	streaming, zero-reuse limit	Ex 4
E	Degenerate: $I \to \infty$ (infinite reuse)	aap $P_{peak}$ saturate karte ho	Ex 4
F	Energy-per-op comparison	ops/joule, GHz nahi	Ex 5
G	Systolic reuse & utilisation	non-square tiles, edge fill/drain waste	Ex 6
H	Word problem (real deploy)	bottleneck dhundho, phir act karo	Ex 7
I	Exam twist: precision change moves $I$	INT8 bytes halve karta hai → $I$ double hota hai	Ex 8
J	Heterogeneous / Amdahl limit	serial part total speedup cap karta hai	Ex 9

Worked examples

Worked example Example 1 — Cell A: memory-bound

Ek DSA ke paas $P_{peak} = 90$ TOPS aur $B = 300$ GB/s hai. Kernel intensity $I = 50$ ops/byte.

Forecast: Pehle ridge point $I^{⋆}$ guess karo. Kya $50$ iske left ya right mein hai?

Ridge point. $I^{⋆} = P_{peak} / B = \frac{90 \times 1 0 ^{12}}{300 \times 1 0 ^{9}} = 300$ ops/byte. Yeh step kyun? Ridge hume boundary batata hai kuch bhi compute karne se pehle — yeh problem ko ek simple left/right comparison mein convert kar deta hai.
Compare. $I = 50 < 300 = I^{⋆}$ ⇒ hum ridge ke left mein hain ⇒ memory-bound. Yeh step kyun? Ridge ka side = bottleneck. Diagnose karne ke liye ceilings par koi arithmetic zaruri nahi.
Attainable. Memory ceiling $= B \times I = 300 \times 1 0^{9} \times 50 = 15 \times 1 0^{12} = 15$ TOPS. Toh $P_{attain} = min (90, 15) = 15$ TOPS. Yeh step kyun? Hum actual usable rate report karne ke liye $min$ finish karte hain.

Verify: $15 < 90$ , memory-bound ke saath consistent. Utilisation $= 15/90 = 16.7%$ — zyaadatar MACs idle hain, exactly ridge ke left mein hone ka symptom. Units: (bytes/s)(ops/byte) = ops/s ✓.

Worked example Example 2 — Cell B: compute-bound

Same chip: $P_{peak} = 90$ TOPS, $B = 300$ GB/s. Ab ek fused kernel $I = 500$ ops/byte achieve karta hai.

Forecast: Ridge abhi bhi $300$ hai. Ab kaun si side hai?

Ridge se compare karo. $I = 500 > 300 = I^{⋆}$ ⇒ ridge ke right mein ⇒ compute-bound. Yeh step kyun? Hum Ex 1 ka ridge reuse karte hain — chip nahi bada, sirf kernel bada.
Memory ceiling. $B \times I = 300 \times 1 0^{9} \times 500 = 150 \times 1 0^{12} = 150$ TOPS. Yeh step kyun? $min$ ko feed karne ke liye memory ceiling ki actual value chahiye — diagnosis batata hai kaun sa ceiling jeetta hai, lekin sirf number batata hai kitne margin se aur ordering confirm karta hai.
Attainable. $P_{attain} = min (90, 150) = 90$ TOPS. Yeh step kyun? Compute ceiling ab jeett rahi hai; hum silicon se limited hain, memory se nahi.

Verify: $90 < 150$ ✓ compute-bound. Utilisation $= 90/90 = 100%$ — MACs fully fed. Yahan zyada bandwidth khareedna kuch nahi karta; zyada MACs khareedna help karta. Ex 1 se bilkul ulta conclusion, jaise ridge ke doosri taraf expect karte hain.

Worked example Example 3 — Cell C: exactly balanced (boundary)

Same chip. Kaun si intensity isse perfectly balanced banati hai, aur wahan kya attainable hai?

Forecast: Guess karo: kya ridge par dono ceilings agree karte hain?

Balance ke liye solve karo. Definition se $I = I^{⋆} = P_{peak} / B = 300$ ops/byte. Yeh step kyun? "Balanced" ka matlab hai koi bhi ceiling dominate nahi karti, yaani $B \times I = P_{peak}$ — us equation ko $I$ ke liye solve karna literally ridge point ki definition hai, toh balance intensity is $I^{⋆}$ .
Dono ceilings. Compute $= 90$ TOPS; memory $= 300 \times 1 0^{9} \times 300 = 90 \times 1 0^{12} = 90$ TOPS. Yeh step kyun? Ridge par dono ceilings numerically equal honi chahiye — "balanced" ka yahi poora matlab hai.
Attainable. $min (90, 90) = 90$ TOPS.

Verify: Dono ceilings $90$ TOPS ke equal hain ✓. Yeh sabse sasta intensity hai jo phir bhi chip ko saturate karti hai — isse kam MACs waste karta hai, isse zyada bandwidth waste karti hai. Yeh design target hai.

Worked example Example 4 — Cells D & E: dono limits (

$I \to 0$ aur $I \to \infty$ ) Same chip ( $P_{peak} = 90$ TOPS, $B = 300$ GB/s). Extremes explore karo.

Forecast: Jab reuse vanish hoti hai, toh kya performance $0$ ya $B$ par jaati hai? Jab reuse explode hoti hai, toh kya yeh bina bound ke badhti rehti hai?

$I \to 0$ (no reuse, pure streaming). $B \times I \to 300 \times 1 0^{9} \times 0 = 0$ , toh $P_{attain} = min (90, 0) = 0$ TOPS. Yeh step kyun? Zero reuse ka matlab hai har operation ek fresh fetch ka wait karta hai; limit mein compute engine completely starve ho jaata hai.
$I \to \infty$ (infinite reuse). $B \times I \to \infty$ , toh $P_{attain} = min (90, \infty) = 90$ TOPS — yeh $P_{peak}$ par clamp ho jaata hai. Yeh step kyun? $min$ performance cap karta hai; aap kitna bhi reuse karo, silicon ke peak se zyada nahi ja sakte.

Verify: Roofline literally ek rising line ( $B \cdot I$ ) hai jo $90$ par flatten ho jaati hai — Ex 4 ke dono limits iske dono ends hain: origin par $0$ , plateau par $90$ . Neeche ki figure dekho.

Worked example Example 5 — Cell F: energy per operation

Ek CPU ek INT32 multiply-add par $70$ pJ spend karta hai (zyaadatar fetch/decode/control mein). Ek MAC cell ek INT8 multiply-add par $0.2$ pJ spend karta hai.

Forecast: Efficiency ratio guess karo — $10 \times$ ke karib ya $100 \times$ ?

Efficiency ratio. $\frac{70 pJ}{0.2 pJ} = 350 \times$ better ops/joule. Yeh step kyun? Dark-silicon era mein binding constraint hai energy per useful op, clock nahi — toh hum energies divide karte hain, frequencies nahi.
Interpret karo. $70$ W ke fixed power budget ke liye, DSA CPU se $350 \times$ zyada multiply-adds per second kar sakta hai. Yeh step kyun? Energy/op aur power milke throughput fix karte hain: $ops/s = power / (energy per op)$ .

Verify: $0.2 \times 350 = 70$ pJ ✓. Units: pJ/pJ dimensionless hai (ek ratio) ✓. Sanity: parent note ke "10×–1000×" range se match karta hai.

Worked example Example 6 — Cell G: systolic reuse & real utilisation

Ek $128 \times 128$ MAC grid ek $256 \times 256$ matmul process karta hai.

Forecast: Peak MACs/cycle? Aur kya utilisation sach mein $100%$ hai, ya fill/drain kuch steal karta hai?

Peak MACs/cycle. $128 \times 128 = 16384$ . Yeh step kyun? Har cell ek cycle mein ek multiply-add karta hai; count sirf grid area hai.
Reuse factor. Har loaded weight apni row/column ke saare $128$ activations se stream hota hai ⇒ reuse $\approx 128$ = array dimension. Yeh step kyun? Reuse = array dimension hi hai jo arithmetic intensity ko $O (1)$ se $O (N)$ tak lift karta hai, design ko compute-bound ki taraf push karta hai.
Fill/drain overhead (the twist). Data ko pipeline fill aur drain karne mein $\sim 2 N - 1 = 255$ cycles lagte hain steady state se pehle/baad. Ek run ke upar jo $256$ activation columns stream karta hai, useful cycles $\approx 256$ , wasted $\approx 255$ , toh steady-state utilisation roughly $\frac{256}{256 + 255} \approx 50%$ hai ek small tile ke liye. Yeh step kyun? Array ke relative chhote matrices ek bada edge tax pay karte hain — isliye real chips ko bade tiles pasand hain.

Verify: $16384 = 12 8^{2}$ ✓; reuse $128 = N$ ✓; $256/ (256 + 255) = 0.5009 \approx 50%$ ✓. Lesson: peak $\neq =$ sustained; parent ka "reuse = array dimension" sirf tab hold karta hai jab pipe full ho.

Worked example Example 7 — Cell H: real-world word problem

Aap ek recommender ko DSA par deploy karte ho: $P_{peak} = 100$ TOPS, $B = 1$ TB/s. Model ke embedding-lookup stage ka $I = 2$ ops/byte hai; dense stage ka $I = 400$ ops/byte hai. Kaun sa stage bottleneck hai, aur aap kya optimise karte ho?

Forecast: Ridge hai $I^{⋆} = 100/1 = 100$ . Guess karo kaun sa stage kis side par hai.

Embedding stage. $I = 2 < 100$ ⇒ memory-bound. Attainable $= B \times I = 1 0^{12} \times 2 = 2 \times 1 0^{12} = 2$ TOPS (sirf $2%$ of peak). Yeh step kyun? Har stage ko independently same ridge ke against diagnose karo.
Dense stage. $I = 400 > 100$ ⇒ compute-bound. Attainable $= min (100, 400) = 100$ TOPS (full peak). Yeh step kyun? Dense stage ridge ke right mein hai, toh uska ceiling $P_{peak}$ hai memory nahi — isse separately re-diagnose karne se confirm hota hai ki dono stages opposite sides par hain aur inhe differently treat karna hoga.
Act karo. Embeddings lost performance dominate karta hai. Fix = wahan $I$ badhao: ops fuse karo, hot embeddings cache karo, table quantise karo (Quantization) taaki kam bytes move hon. MACs add karna us stage ke liye useless hoga. Yeh step kyun? Binding constraint optimise karo; Roofline model ne bataya ki woh kaun sa hai.

Verify: Embedding $2$ TOPS $= 2%$ of $100$ ✓; dense $100$ TOPS $= 100%$ ✓; ridge $100/1 = 100$ ✓. Dono stages ridge ke opposite sides par land hote hain — classic heterogeneous-workload picture (Heterogeneous computing).

Worked example Example 8 — Cell I: exam twist — precision

$I$ ko change karta hai Ek kernel $W$ ops karta hai aur, FP32 mein, har operand $4$ bytes ke roop mein move karta hai, $B = 300$ GB/s, $P_{peak} = 90$ TOPS wale chip par $I_{32} = 25$ ops/byte deta hai. Yahan $W$ operations ki total sankhya hai aur $Q$ bytes moved ki total sankhya hai, toh intensity hai $I = W / Q$ . Aap operands ko INT8 ( $1$ byte) par switch karte ho. Naya $I$ kya hai, aur kya bottleneck flip hota hai?

Forecast: INT8 $4 \times$ kam bytes hai. Naya intensity guess karo aur yeh bhi ki kya yeh ridge ( $I^{⋆} = 300$ ) cross karta hai.

Naya intensity. Ops $W$ unchanged; bytes moved $Q$ $4 \times$ kam ho jaate hain. Kyunki $I = W / Q$ , $Q$ quarter hone se $I$ quadruple ho jaata hai: $I_{8} = 4 \times 25 = 100$ ops/byte. Yeh step kyun? Intensity ops per byte hai; operand per kam bytes mechanically reuse-per-byte badhata hai. Isliye low precision tab bhi help karta hai jab op count identical ho.
Re-diagnose karo. $I_{8} = 100$ , ridge $I^{⋆} = 300$ . Abhi bhi $100 < 300$ ⇒ abhi bhi memory-bound, lekin karib. Yeh step kyun? Precision change karne se $I$ move hua, toh pehle ki diagnosis ab valid nahi ho sakti — kisi bhi ceiling par trust karne se pehle hume check karna hoga ki hum ridge ke kis side par land karte hain.
Attainable pehle vs baad. Pehle: $300 \times 1 0^{9} \times 25 = 7.5$ TOPS. Baad: $300 \times 1 0^{9} \times 100 = 30$ TOPS. Pure precision se $4 \times$ jump — koi naya silicon nahi. Yeh step kyun? Dikhata hai ki precision ek bandwidth lever hai, sirf compute lever nahi — yahi Mixed precision training aur INT8 inference ka point hai.

Verify: $4 \times 25 = 100$ ✓; $7.5 \times 4 = 30$ ✓; dono attainable values $90$ se neeche, toh bottleneck memory side par raha ✓. Agar hum $I_{32} = 80$ se start karte, toh INT8 $320 > 300$ deta aur flip karke compute-bound ho jaata — boundary case ke roop mein kehne layak hai.

Worked example Example 9 — Cell J: heterogeneous / Amdahl ceiling

Ek pipeline apna fraction $f = 0.9$ time ek matmul mein spend karti hai jise ek DSA $s = 50 \times$ speed up karta hai. Remaining $0.1$ (control, I/O) CPU par rehta hai, unchanged. Poore program ka speedup kya hai?

Forecast: $50 \times$ accelerator ke saath, overall speedup guess karo — $50 \times$ ke paas, ya bahut kam?

Amdahl's Law. Overall speedup $= \frac{1}{( 1 - f ) + f / s}$ . Yeh step kyun? Program ka sirf ek part accelerate hota hai; serial remainder ek hard ceiling set karta hai (Amdahl's Law).
Plug in karo. $= \frac{1}{0.1 + 0.9/50} = \frac{1}{0.1 + 0.018} = \frac{1}{0.118} \approx 8.47 \times$ . Yeh step kyun? Un-accelerated $10%$ ab runtime dominate karta hai.
Limit ( $s \to \infty$ ). $\frac{1}{0.1 + 0} = 10 \times$ — woh absolute cap jo koi bhi accelerator beat nahi kar sakta. Yeh step kyun? Dikhata hai kyun real systems heterogeneous hote hain aur kyun serial part ko shrink karna zyada matter karta hai, $s$ bada hone ke baad ek faster DSA se zyada.

Verify: $0.9/50 = 0.018$ ; $0.1 + 0.018 = 0.118$ ; $1/0.118 = 8.474 \dots$ ✓; limit $1/0.1 = 10$ ✓. Ek "perfect" DSA bhi yahan sirf $10 \times$ deta hai — Cell J ka lesson.

Recall Har example kaun sa cell hai?

Ex1 memory-bound (A) ::: ridge ke left, $I < I^{⋆}$ Ex2 compute-bound (B) ::: ridge ke right, $I > I^{⋆}$ Ex3 balanced (C) ::: ridge par, dono ceilings equal Ex4 limits (D,E) ::: $I \to 0$ gives $0$ ; $I \to \infty$ clamps to $P_{peak}$ Ex5 energy (F) ::: energies divide karo, clocks nahi → $350 \times$ Ex6 systolic (G) ::: peak $\neq =$ sustained; fill/drain tax Ex7 word problem (H) ::: har stage diagnose karo, memory-bound wala fix karo Ex8 precision twist (I) ::: kam bytes → zyada $I$ → $4 \times$ free Ex9 Amdahl (J) ::: serial part speedup cap karta hai $1/ (1 - f)$ par