Yeh mathematical license hai approximate computing,
quantization, aur low-precision arithmetic ke peeche.
Hum ise yahan derive karte hain taaki tumhe kabhi bhi ise faith par lena na pade.
KYA: hum error ko number line par ek signed distance ki tarah define karte hain.
KYUN: errors ke "cancel" hone ki baat karne se pehle, unhe ek sign — ek direction — chahiye —
taaki ek + aur ek − ek doosre ko undo kar sakein. Bina sign ke distance kabhi cancel nahi ho sakta.
PICTURE: neeche, true value line par baitha hai; red arrow ε hai, right point kar raha hai
(overshoot) ya left (undershoot).
KYA: hum N operations mein se har ek ke saath ek error εi attach karte hain.
KYUN: approximate hardware ek badi galti nahi karta — woh bahut saari choti galtiyan karta hai, ek per operation (dekho approximate adders/multipliers). Total ke baare mein sochne ke liye, pehle har choti galti ko alag se naam dena zaroori hai.
PICTURE:N additions ki ek chain; har link ek red error-arrow running total mein giraa deta hai.
KYA: hum assume karte hain ki per-operation errors average mein zero hain aur ek doosre ke saath mil ke kaam nahi karte.
YEH DO HI KYUN, DOOSRE NAHI? Kyunki yahi exactly woh do ingredients hain jo cancellation trick ko chahiye.
Mean-zero matlab arrows ka koi built-in direction nahi; independence matlab woh sab agreement se same
direction mein point nahi karte. Koi bhi ek drop karo aur trick khatam (hum yeh Step 7 mein prove karte hain).
PICTURE: red error-arrows ka ek cloud zero ke aas-paas symmetrically scattered — koi preferred direction nahi.
KYA: hum expected total errorE[E] compute karte hain.
KYUN: hum jaanna chahte hain: kya sloppy sum average mein ek direction mein drift karta hai? Agar haan, toh yeh disaster hai (systematic bias). Chalte hain check karte hain.
PICTURE: saare mean-zero arrows, tip-to-tail add karo, seedha starting point par wapas aa jaate hain.
E[E]=E[∑i=1Nεi]=∑i=1N=0E[εi]=0
Term by term padhte hain: sum ka average, averages ke sum ke barabar hota hai (expectation ek + ke through "pass" ho jaata hai). Har inner term Step 3 se 0 hai. Zero ko N baar add karo toh bhi zero hi milta hai.
Yahan sab kuch ka dil hai. Hum poochte hain: typical total error kitni badi hoti hai? Woh σ of E hai,
toh pehle Var(E) compute karte hain.
KYA:σ2 ki N copies add karo.
PICTURE:N red steps of size σ ki ek random walk; shuru se crow-flies distance Nσ nahi
balki Nσ hai — steps ek doosre ko partly undo kar dete hain.
\qquad\Longrightarrow\qquad
\underbrace{\text{typical } |E|}_{\text{std.\ dev.\ of }E} \;=\; \sqrt{N\sigma^2} \;=\; \sqrt{N}\,\sigma$$
Term by term: $N$ terms mein se har ek $\sigma^2$ contribute karta hai; total variance $N\sigma^2$ hai; square root lene par (error units mein wapas jaane ke liye) typical error $\sqrt{N}\,\sigma$ milti hai.
> [!intuition] $\sqrt{N}$ kyun, ek sentence mein
> Agar aadhe errors right push karte hain aur aadhe left, toh woh *mostly* cancel ho jaate hain — jo bachta hai woh sirf
> chota sa imbalance hai, aur woh imbalance $\sqrt{N}$ ki tarah badhta hai, $N$ ki tarah nahi.
---
## Step 6 — Punchline: relative error pighal jaati hai
> [!definition] Signal vs relative error
> - **Signal** $N$ numbers ka actual sum hai. Agar har ek average $\mu$ ("mu", mean value) hai,
> toh signal kariban $N\mu$ hai — yeh $N$ mein **linearly** badhta hai.
> - **Relative error** woh hai jo user ke liye actually matter karta hai: error *answer ke fraction ke roop mein*,
> $\dfrac{\text{error}}{\text{signal}}$. $\$10$ ke note par $\$5$ ki galti badi hai, $\$1{,}000{,}000$ par invisible.
**KYA:** typical error (Step 5) ko signal size se divide karo.
**KYUN:** absolute error badhti hai, toh ek beginner ghabra jaata hai. Lekin *answer* aur tezi se badhta hai — toh fraction shrink ho jaata hai.
**PICTURE:** do curves — signal $N\mu$ line ki tarah tezi se chadh raha hai, error $\sqrt{N}\sigma$ ek gentle root ki tarah dheere chadh raha hai — unke beech ka red gap-ratio axis ki taraf collapse ho raha hai.
![[deepdives/dd-hardware-6.5.16-d2-s06.png]]
$$\frac{\text{typical error}}{\text{signal}} \;\sim\; \frac{\sqrt{N}\,\sigma}{N\mu}
\;=\; \frac{\sigma}{\mu}\cdot\frac{1}{\sqrt{N}} \;\xrightarrow[\;N\to\infty\;]{}\; 0$$
Term by term: numerator $\sqrt{N}\sigma$ hai wandering; denominator $N\mu$ hai answer; $N$'s
partly cancel ho ke $1/\sqrt{N}$ bacha lete hain, jo $N$ badhne par $0$ ki taraf slide karta hai.
> [!formula] License, stated
> **Jitna bada aggregation, utna zyada sloppiness afford kar sakte ho** — relative error
> $1/\sqrt{N}$ ki tarah girती hai. Isi liye sensors averaging karna, gradients summing karna, aur activations pooling karna approximate hardware mein survive karta hai.
---
## Step 7 — Edge & degenerate cases (kabhi koi unseen scenario mat hit karo)
> [!mistake] Case A — Sirf EK operation ($N=1$)
> $N=1$ ke saath relative error hai $\dfrac{\sigma}{\mu\sqrt{1}} = \dfrac{\sigma}{\mu}$ — **poori** error,
> kuch cancel nahi hua. **Average karne ke liye kuch nahi hai.** Approximation tumhe yahan sirf
> raw energy savings deta hai; quality safety net gayi. Chote sums *dangerous* case hain.
> [!mistake] Case B — BIASED errors (mean zero nahi, $\mathbb{E}[\varepsilon_i]=b\neq 0$)
> Ab Step 4 toot jaata hai: $\mathbb{E}[E]=\sum b = Nb$. Drift $N$ ki tarah badhti hai — **signal jitni same speed** —
> toh relative error $\to \dfrac{b}{\mu}$, ek *constant jo kabhi shrink nahi hota*. Always-round-down truncation
> exactly yahi karta hai. Parent ki mistake *"Assuming error always averages out"* dekho.
> **Fix:** unbiased rounding use karo taaki $b=0$ ho.
**PICTURE:** do random walks side by side — unbiased wala start ke paas hover karta hai ($\sqrt{N}$ spread);
biased wala steadily door jaata hai ($N$ drift).
![[deepdives/dd-hardware-6.5.16-d2-s07.png]]
> [!mistake] Case C — CORRELATED errors (independent nahi)
> Agar errors conspire karte hain (e.g. ek hot region ke har op ka same taraf flip hona), toh cross-terms aur vanish nahi hote
> aur variance $N^2$ ki tarah badh sakta hai, giving typical error $\sim N\sigma$. Cancellation gone.
> **Fix:** error sources ko decorrelate karo; isi liye [[DRAM Refresh and Memory Reliability|refresh-reduced memory]]
> *spread-out, non-critical* data ko target karta hai, aur isi liye [[Error-Correcting Codes|ECC]] correlated-critical parts ki raksha karta hai.
> [!recall]- Quick self-test
> Agar tum $N=100$ se $N=10{,}000$ (unbiased) jaate ho, toh relative error kitna shrink hogi? ::: $\sqrt{10000/100}=\sqrt{100}=10\times$ choti ho jaayegi.
> Ek truncating multiplier hamesha round down karta hai. Kya averaging uski error fix kar deti hai? ::: Nahi — yeh biased hai, drift $N$ ki tarah badhti hai, relative error $b/\mu$ par rehti hai (Case B).
> Approximate karne ke liye *sabse risky* sum kaun sa hai? ::: Bahut chota ($N$ small), kyunki kuch cancel nahi hota (Case A).
---
## Ek-picture summary
![[deepdives/dd-hardware-6.5.16-d2-s08.png]]
Poora derivation ek canvas par: har sloppy op ek red $\pm\sigma$ arrow giraa deta hai; tip-to-tail add karo toh woh
ek random walk banate hain jo sirf $\sqrt{N}\sigma$ drift karta hai ($N\sigma$ nahi) kyunki $+$'s aur $-$'s cancel ho jaate hain;
meanwhile true sum poori speed $N\mu$ par chadh raha hai; toh ratio red-wander / black-climb
$1/\sqrt{N}$ ki tarah collapse ho jaata hai. Wahi collapse *woh reason hai kyun approximate computing kaam karta hai* — aur Steps 7 precisely woh
do tarike dikhate hain jisse ise todne: **short sums** aur **biased errors**.
> [!recall]- Feynman: seedhe shabdon mein poora walkthrough
> Socho ek million bacche ek candy bag ka weight guess kar rahe hain, har ek thoda off — kuch bahut zyada, kuch bahut kam, aur koi same direction mein cheat nahi kar raha. Jab tum unke saare guesses add karte ho, toh "bahut zyada" aur "bahut kam" mostly ek doosre ko wipe out kar dete hain. Jo bachta hai woh ek tiny leftover wobble hai jo bahut dheere badhti hai — kariban bacchon ki sankhya ka square root. Lekin *total* candy weight poore bacchon ki sankhya ke saath badhta hai. Toh wobble, total ke comparison mein, ek chhotata aur chhotata sliver ban jaati hai — practically kuch nahi. Yahi wajah hai ki ek computer jab *bahut saari* cheezein add kar raha hota hai tab woh sloppily add afford kar sakta hai: sloppiness khud cancel ho jaati hai. Do tarike hain jisme yeh fail hota hai: (1) bahut kam bacche — cancel karne ke liye kuch nahi; (2) sab purposely too high guess karte hain — leftover aur cancel nahi hota aur answer jitni tezi se badhta hai.
> [!mnemonic] Ek line yaad rakhne ke liye sab kuch
> **"Noise walks ($\sqrt{N}$), signal marches ($N$) — toh relative error dies ($1/\sqrt{N}$)."**
