Responsible AI deployment practices: Visual walkthrough

6.4.15 · D2 · AI-ML › AI Safety & Alignment › Responsible AI deployment practices

Hum har symbol khud kamayenge. Aakhir mein aap yeh line poori tarah samjhoge:

$n \geq \frac{l n ( α )}{l n ( 1 - p )}$

...aur aapko pata hoga ki har letter apni jagah kyun hai.

Step 1 — "Failure rate" ka matlab kya hota hai

KYA HAI. Hum ek single number se shuru karte hain: $p$ . Yeh ek request ka failure probability hai.

KYUN. Isse pehle ki hum poochhen "kitni requests mein failure pakdenge", humein clearly samajhna hoga ki failure kitni baar hoti hai. Woh samajh $p$ hai.

PICTURE. Ek lambi strip socho chhote boxes ki, ek box per request. Zyaadatar green hain (success). Kuch rare red hain (failure). Poori strip mein red ka fraction hi $p$ hai.

Strip dekho: sirf ek chhoti si sliver red hai. Woh sliver $p$ hai. Staged rollout ka poora point yahi hai ki agar hum sirf thode boxes dekhte hain, toh shayad koi red nahi dikhega — aur galti se yeh conclude karein ki model safe hai.

Step 2 — Ek single request ke fail NA hone ki chance

KYA HAI. Agar fail hone ki probability $p$ hai, toh nahi fail hone ki probability $1 - p$ hai.

KYUN. Hamara goal kam se kam ek failure pakadna hai. Yeh seedha count karna awkward hai (ek, do, teen... failures ho sakti hain). Iska ulta count karna kaafi aasaan hai: zero failures dekhna. Toh pehle humein chahiye ek request ke succeed hone ki probability — yeh "sab succeed ho" ka building block hai.

PICTURE. Step 1 ki strip lo aur use ek single spinner mein compress karo: ek wheel jahan $p$ size ka red wedge matlab "fail" aur $1 - p$ size ka baaki green matlab "succeed" hai.

Green wedge mota hai (kyunki $p$ tiny hai), toh ek spin almost hamesha green pe girta hai. Woh "almost hamesha" hi rare failures ko dangerous banata hai — aur issi liye ek request se aapko lagbhag kuch pata nahi chalta.

Step 3 — Multiply kyun karte hain: saari n requests ke succeed hone ki chance

KYA HAI. Wheel ko $n$ baar spin karo ( $n$ requests bhejo). Probability ki har ek green pe gire:

$(1 - p) \times (1 - p) \times \dots \times (1 - p) = (1 - p)^{n}$

MULTIPLY KYUN, ADD KYUN NAHI? Kyunki requests independent hain — request 1 ka outcome request 2 ko nahi badalta. Jab aap independent events ke liye "yeh AUR yeh AUR yeh" chahte ho, probabilities multiply hoti hain. (Add karna "yeh YA yeh" ka jawab deta, jo alag sawaal hai.)

PICTURE. Ek branching tree: har request pe raasta green (success) ya red (failure) mein fork karta hai. Bilkul left edge pe neeche jaata ek all-green path hai jiska probability $(1 - p)^{n}$ hai — aur tree gehra hone ke saath yeh tezi se chhota hota jaata hai.

Note karo: ek ek spin almost zaroor green hai, lekin greens ki lambi chain maangna kam aur kam likely hota jaata hai. Woh shrinkage hamara dost hai — iska matlab hai kaafi requests ke baad "all green" rare ho jaata hai, toh all green dekhna safety ka asli evidence ban jaata hai.

Step 4 — Apni wish ko ek inequality mein badalna

KYA HAI. Hum failure miss nahi karna chahte. $α$ woh risk hai jo hum tolerate karte hain miss karne ka — woh probability jitna hum accept karte hain ki sab $n$ requests green aayein jabki model buggy ho.

Hum insist karte hain ki "all green" probability $α$ se zyada badi na ho:

$chance we see zero failures (1 - p)^{n} \leq chance we’re willing to be fooled α$

KYUN. Agar "bug hone ke bawajood all green" ki chance hamari tolerance $α$ se neeche squeeze ho jaaye, toh all green dekhna model par trust karne ke liye kaafi achha evidence hai. Yeh inequality hamara goal hai, symbols mein.

PICTURE. Ek vertical bar. Upar wala hissa $α$ hai (hamari tolerance ceiling). Curve $(1 - p)^{n}$ $n$ badhne par neeche slide karta hai. Humein $n$ itna bada chahiye ki curve ceiling ke neeche chala jaaye.

Yellow ceiling fixed hai. Blue curve zyada requests ke saath girta hai. Hum woh sabse chhota $n$ chahte hain jahan blue yellow ke neeche jaaye.

Step 5 — Logarithm kyun? $n$ ko exponent se azaad karna

KYA HAI. Humara unknown $n$ $(1 - p)^{n}$ ke exponent mein qaidi hai. Humein ek aisa tool chahiye jo exponent ko neeche ground level pe le aaye. Woh tool hai logarithm.

YEH TOOL KYUN AUR DOOSRA KYUN NAHI? Hum $(1 - p)^{n} \leq α$ ko $n$ ke liye ordinary algebra se solve nahi kar sakte — dividing ya square-rooting exponent ko free nahi karengi. Logarithm woh ek operation hai jo "kis power?" ka jawab dene ke liye bana hai, toh yeh ek locked exponent ki natural key hai. Dono sides ka $ln$ lena legal hai kyunki $ln$ increasing hai (bada input → bada output), toh yeh $\leq$ direction preserve karta hai... abhi ke liye.

Dono sides ka $ln$ lo:

$ln ((1 - p)^{n}) \leq ln (α) ⟹ ab azaad n \cdot ln (1 - p) \leq ln (α)$

PICTURE. Curve $y = ln (x)$ : yeh bade exponents ko ek straight-line playground pe le aati hai jahan $n$ sirf ek plain multiplier hai.

Step 6 — Sign trap: inequality FLIP kyun hoti hai

KYA HAI. Ab hum $n$ isolate karne ke liye dono sides ko $ln (1 - p)$ se divide karte hain. Lekin $ln (1 - p)$ ka sign dekho.

Kyunki $p$ $0$ aur $1$ ke beech hai, $1 - p$ ki value $1$ se kam hai. Aur $1$ se kam kisi bhi number ka $ln$ negative hota hai:

$ln (1 - p) < 0 (kyunki 0 < 1 - p < 1)$

Dono sides ko negative quantity $ln (1 - p)$ se divide karne par $\leq$ flip hokar $\geq$ ban jaata hai:

$n \geq \frac{ln ( α )}{ln ( 1 - p )}$

Har symbol kamaaya gaya:

$ln (α)$ — ek negative number (kyunki $α < 1$ ), "main kitna strict hun?"
$ln (1 - p)$ — ek negative number (kyunki $1 - p < 1$ ), "ek failure kitni rare hai?"
negative ÷ negative = positive, toh $n$ sensibly positive nikalta hai.
$\geq$ — humein kam se kam itna chahiye; zyada theek hai, kam theek nahi.

KYUN. $\geq$ poora punchline hai: yeh humein minimum sample size batata hai. Flip bhool jaao aur aap $n \leq$ kuch compute karoge — yeh conclude karte hue ki aap jitne strict ho utni kam requests chahiye, jo nonsense hai.

PICTURE. Ek number line jismein dono $ln$ values negative territory mein hain, aur division $n$ ko positive side pe safely land karata hai, ek arrow ke saath "tumhe itni ya zyada" mark karta hua.

Step 7 — Real numbers daalna

KYA HAI. $0.1%$ failure rate ko $95%$ confidence ke saath detect karo: $p = 0.001$ , $α = 0.05$ .

$n \geq \frac{l n ( 0.05 )}{l n ( 1 - 0.001 )} = \frac{l n ( 0.05 )}{l n ( 0.999 )} = \frac{- 2.9957}{- 0.0010005} \approx 2995.6$

Upar round karo (aap fraction mein request nahi chala sakte, aur $\geq$ neeche round karne ki ijazat nahi deta):

$n = 2996 requests.$

KYUN. Yahi number parent note quote karta hai. Iska matlab hai: naye model ko ~3000 real requests pe chalao. Agar koi fail na ho, toh aap $95%$ confidence ke saath keh sakte ho ki iska failure rate $0.1%$ se kam hai. 1000 users/day par yeh 3 din ka 1% traffic hai.

PICTURE. Girta hua curve $(1 - p)^{n}$ jisme ek vertical marker $n \approx 2996$ par hai jahan woh $α = 0.05$ ceiling cross karta hai.

Step 8 — Edge cases (reader ko koi unseen scenario nahi milna chahiye)

KYA HAI & KYUN & PICTURE, case by case:

Worked example Formula ka har corner

Case $p \to 0$ (flaw vanishingly rare hai). $ln (1 - p) \to 0^{-}$ (ek tiny negative). Zero ke kareeb kisi cheez se divide karne par $n$ infinity tak blow up ho jaata hai. Matlab: ek ultra-rare failure pakadne ke liye traffic ka pahad chahiye — kabhi kabhi sampling se practically undetectable. Yeh wahi long tail hai jiske baare mein parent note warn karta hai.

Case $p \to 1$ (model almost hamesha fail karta hai). $ln (1 - p) \to - \infty$ , toh $n \to 0$ . Matlab: ek model jo lagaataar fail karta hai woh kuch hi requests mein khud ko zaahir kar deta hai. Obvious — aur math bhi agree karta hai.

Case $α \to 1$ (mujhe koi bhi miss chalega). $ln (α) \to ln (1) = 0$ , toh $n \to 0$ . Matlab: agar aapko bug pakadne ki parwah nahi, toh koi request nahi chahiye. Practice mein useless, lekin internally consistent.

Case $α \to 0$ (main kabhi bhi fooled nahi hona chahta). $ln (α) \to - \infty$ , toh $n \to \infty$ . Matlab: absolute certainty ke liye infinite testing chahiye. Koi free lunch nahi — issi liye hum zero demand karne ki jagah ek chhota $α$ accept karte hain.

Degenerate $p = 0$ exactly. $ln (1 - 0) = ln (1) = 0$ → division by zero, undefined. Interpretation: ek truly perfect model ko kabhi "failing pakda" nahi ja sakta kyunki woh kabhi fail hi nahi karta; sawaal khud khatam ho jaata hai.

Figure $p$ ko uske range mein sweep karta hai aur dikhata hai $n$ $p \to 0$ par explode hota hai aur $p \to 1$ par collapse hota hai — poora behavioural map ek plot mein.

Ek-picture summary

Ek flow, left to right: ek spinner ( $1 - p$ green) → $n$ spins ke liye multiply ( $(1 - p)^{n}$ ) → tolerance ke neeche cap karo ( $\leq α$ ) → $n$ free karne ke liye $ln$ lo → divide karo (sign flip hota hai!) → final $n \geq \frac{ln α}{ln ( 1 - p )}$ .

Recall Feynman retelling — kisi dost ko batane jaisa bolo

Mere paas ek naya model hai. Yeh rare mein galti karta hai — us rare-mistake chance ko $p$ kaho. Toh har try mein yeh cheezein sahi karta hai $1 - p$ chance ke saath. Agar main ise $n$ baar test karun aur yeh genuinely buggy hai, toh chance ki yeh ittefaqan perfect lagta hai (all green) woh hai $1 - p$ apne aap se $n$ baar multiply, yani $(1 - p)^{n}$ . Main nahi chahta ki mujhe zyada se zyada, maano, $5%$ time fooled kiya jaaye — woh $5%$ hai $α$ . Toh main demand karta hun $(1 - p)^{n} \leq α$ . Problem: $n$ exponent mein qaidi hai. Logarithm woh ek tool hai jo exponent ko plain multiplier ke roop mein neeche le aata hai, toh main dono sides ka $ln$ leta hun aur paata hun $n ln (1 - p) \leq ln α$ . Lekin $1 - p$ $1$ se neeche hai, toh uska log negative hai — aur negative se divide karna sign flip karta hai — giving $n \geq ln α / ln (1 - p)$ . Dono logs negative hain, toh unka ratio ek achha positive number hai: minimum requests jo mujhe chahiye. $1$ -in- $1000$ bug ke liye $95%$ confidence ke saath, woh lagbhag $2996$ requests hain. Aur edges make sense karte hain: rarer bug → bahut zyada tests chahiye; stricter confidence → bahut zyada tests chahiye; kabhi koi free lunch nahi hota.

Recall

$(1 - p)^{n}$ mein exponent $n$ kya role play karta hai? ::: Yeh count karta hai ki kitni independent requests ko ek row mein succeed karna hai — kitne green spins hum demand karte hain. Logarithm kyun lete hain? ::: Yeh woh ek tool hai jo unknown $n$ ko exponent se neeche kheenchta hai, $(1 - p)^{n}$ ko $n ln (1 - p)$ mein badalta hai taaki hum $n$ ke liye solve kar sakein. Inequality $\geq$ mein flip kyun hoti hai? ::: Kyunki $ln (1 - p)$ negative hai (kyunki $1 - p < 1$ ), aur inequality ko negative number se divide karne par uski direction reverse ho jaati hai. Jab $p \to 0$ , required $n$ ka kya hota hai, aur iska kya matlab hai? ::: $n \to \infty$ — ek extremely rare failure detect karne ke liye enormous traffic chahiye; yeh long-tail problem hai. $p = 0.001, α = 0.05$ ke liye $n$ kya hai? ::: Lagbhag $2996$ requests ( $2995.6$ se upar round up).

Yeh bhi dekho: 6.3.2-MLOps-principles (jahan yeh staged rollout pipeline ko gate karta hai), 5.2.4-A-B-testing (do models compare karne ke liye wahi sampling logic), 6.4.1-Adversarial-examples (woh rare failures jinhein pakadne ki koshish yeh math karta hai), aur parent Responsible AI deployment practices.