Domain-specific accelerators — Hinglish notes

6.5.6 · Hardware › Advanced & Emerging Architectures

DSAs exist kyun karte hain? (First principles)

Result: hamare paas transistors hain (Moore's Law limped on karta raha) lekin unhe sab ko power nahi de sakte (Dark Silicon problem). Agar sab kuch on nahi kar sakte, toh smart move hai kai specialized blocks banana aur sirf woh wala light up karna jo abhi chahiye. Yahi DSAs ka economic engine hai.

Speedup kaise milta hai? (5 levers — the 80/20)

Canonical example: systolic array (TPU-style matmul)

Ye kyun jeet ta hai — arithmetic intensity argument (derived):

$N \times N$ matrices ke saath naive matrix multiply $C = A \cdot B$ ke liye:

Total MACs $= N^{3}$ .
Agar har operand har baar memory se aata, memory ops $= 3 N^{3}$ → arithmetic intensity $\approx \frac{1}{3}$ ops/byte. Memory-bound, terrible.

Ek $N \times N$ systolic array mein, aap $O (N^{2})$ data load karte ho aur array ke andar har value ko $O (N)$ baar reuse karte ho:

Arithmetic intensity = \frac{operations}{bytes moved} \approx \frac{N ^{3}}{N ^{2}} = O (N) .

Toh intensity array size ke saath badhti hai → aap compute-bound ho jaate ho aur MACs ko feed rakh sakte ho. Yahi poora trick hai.

Worked examples

Common mistakes (Steel-manned)

Forecast-then-Verify

Recall Reveal karne se pehle predict karo

Q: Ek kernel ki intensity $I = 2$ ops/byte hai, ek chip par jiska $B = 1$ TB/s aur $P_{peak} = 100$ TOPS hai. Compute-bound hai ya memory-bound? Attainable kya hai? Pehle predict karo... $B \cdot I = 1 0^{12} \times 2 = 2 \times 1 0^{12} = 2$ TOPS $< 100$ TOPS ⇒ memory-bound, attainable $= 2$ TOPS. Peak ka sirf 2% use ho raha hai — ops fuse karne aur reuse badhane ki classic wajah.

Flashcards

CPU per-generation speedups ka "free lunch" kya khatam kiya?

Dennard scaling ka ant (~2006): transistors shrink hone par power density constant rehna band ho gayi.

Dark-silicon problem kya hai?

Hum itne transistors fit kar sakte hain jo hum ek saath power/cool nahi kar sakte, toh humein specialize karna hoga aur sirf needed blocks light up karne honge.

Domain-specific accelerator define karo.

Ek chip jo ek class of problems ke liye specialized hai, generality trade karke performance-per-watt aur per-area mein bade gains ke liye.

DSA advantage ke 5 sources batao.

Data-level parallelism, lower/cheaper arithmetic precision, domain-specific memory (scratchpads), reduced control overhead, domain-specific language/mapping.

Systolic array kya karta hai?

MAC cells ke grid se data rhythmically stream karta hai, har loaded value ko kai baar reuse karta hai memory traffic minimize karne ke liye.

Roofline model batao.

P_{attain} = min (P_{peak}, B \times I)

jahan

B

=bandwidth,

I

=arithmetic intensity.

Arithmetic intensity kya hai?

Operations performed per byte moved from memory (FLOP/byte); zyaada intensity matlab zyaada compute-bound.

N×N systolic array arithmetic intensity ko O(N) tak kyun badhata hai?

Har loaded operand array ke across ~N baar reuse hota hai, toh ops/bytes ≈

N^{3} / N^{2}

= N.

ML inference ke liye DSAs lower precision safely kyun use kar sakte hain?

Kai weights par statistical averaging rounding noise ko chhupaati hai, toh INT8/bfloat16 negligible accuracy loss ke saath ~4× kam energy cost karta hai.

Zyaada MACs add karne se hamesha help kyun nahi milti?

Agar kernel memory-bound hai (

B \cdot I < P_{peak}

) toh extra MACs bhukhe rahenge; aapko arithmetic intensity badhani hogi.

Key DSA metric clock speed nahi, energy-per-op kyun hai?

Dark-silicon era mein binding constraint power hai; specialization fetch/decode/control energy overhead delete kar deti hai.

Recall Feynman: 12 saal ke bacche ko samjhao

Ek super-smart student imagine karo jo koi bhi homework kar sakta hai — math, art, history. Lekin har ek sawaal se pehle wo samay barbad karta hai question padhne mein, decide karne mein kya karna hai, double-check karne mein, aur pages palat ne mein. Yahi CPU hai. Ab ek factory machine imagine karo jo sirf ek kaam karti hai — maano, cookies stamp karna — lekin woh ek saath hazaron karta hai, super fast, almost koi energy use nahi karta, kyunki use kabhi "sochna" nahi padta kya karna hai. Yahi domain-specific accelerator hai. Apne ek kaam ke bahar ye dumb hai, lekin us kaam ke liye ye unbeatable hai. Kyunki hum apne chip ke saare transistors ek saath power nahi kar sakte, yeh samjhdar hai ki bahut saare yeh specialized cookie-stampers banao aur sirf woh wala switch on karo jo hume abhi chahiye.

Connections

Dennard scaling & Dark silicon — physical reasons ki DSAs exist karte hain.
Roofline model — compute vs memory bound ke liye analysis tool.
Systolic arrays — canonical matmul accelerator structure.
TPU (Tensor Processing Unit) aur GPU vs TPU — real-world DSAs.
Heterogeneous computing — CPUs specialized accelerators orchestrate karte hue.
Amdahl's Law & Pollack's Rule — general-purpose scaling ki limits.
Quantization & Mixed precision training — lower-precision arithmetic enable karna.