Responsible AI deployment practices: Question bank

6.4.15 · D5 · AI-ML › AI Safety & Alignment › Responsible AI deployment practices

Shuru karne se pehle, ek plain-words refresher taaki koi bhi term bina samjhe use na ho:

Definition Vocabulary jo tumhe chahiye (expand karne ke liye click karo)

Blast radius ::: kitne users/requests ek failure touch kar sakti hai pakde jaane se pehle. 1% rollout blast radius ko traffic ke 1% tak shrink kar deta hai.
Shadow mode ::: naya model real traffic par run karta hai lekin uska output throw away kar diya jaata hai; sirf comparison ke liye log hota hai. Users ko purana model hi dikhta hai.
Concept drift ::: input aur correct answer ke beech ka relationship change ho jaata hai (e.g. "spam" ki definition evolve ho jaati hai).
Data drift ::: inputs ka distribution change ho jaata hai chahe correct rule na bada ho (e.g. zyada non-English names aane lagte hain).
OOD ::: out-of-distribution — ek aisa input jo model ne training mein kabhi nahi dekha (e.g. ek phool wala tree, jab classifier sirf bare winter trees par trained ho). Model ke paas jawab dene ka koi achha basis nahi hota, to use uncertain rehna chahiye.
Circuit breaker ::: ek automatic switch jo model par trust karna band kar deta hai aur kisi safe cheez par fallback karta hai jab errors spike karein.
HITL ::: Human-in-the-loop — ek insaan kuch predictions ko review ya approve karta hai.

Poora pipeline ek picture mein

Traps se pehle, dekho ki charon safeguards ek real-time request path mein kaise fit hote hain. Arrows follow karo: ek request enter karti hai, model jawab deta hai, monitoring dekhti hai, aur circuit breaker / HITL decide karta hai kab insaan ko control lena chahiye.

Figure — Responsible AI deployment practices

Yahi chaar safeguards, ek launch ke timeline par staged (internal → beta → 1% → 10% → 100%), defense-in-depth ladder banate hain — har layer woh pakadti hai jo pichli layer se reh gaya:

Ek symbol ek time par: sample-size formula

Neeche kai traps parent ke rollout formula par lean karte hain, to pehle har letter earn karte hain. Ek stage ko $n$ independent requests ki ek bag ki tarah socho. Har request ka ek chhota chance $p$ hai failure ka (ek bura output jo hum pakadna chahte hain). Hum probability $α$ (ek chhoti number jaise $0.05$ ) tolerate karne ke liye tayyar hain problem ko bilkul miss karne ki — yaani zero failures dekhna chahe fault real ho.

Is formula ki picture — jaise rare failures ke saath required samples explode karte hain — dekhne layak hai:

KL divergence kyun, concretely samjho

"KL divergence kyun use karein" wala trap aasaan ho jaata hai jab tum dekhte ho do distributions jo same mean par hain lekin alag shape ki hain — mean-only test kehta hai "no drift", KL kehta hai "big drift":

Sliding window vs all-time counter

Aur circuit-breaker trap ek recovering model ki picture se obvious ho jaata hai: ek all-time counter kabhi nahi bhoolta aur eventually hamesha ke liye trip kar jaata hai; ek sliding window recent health track karta hai aur reset ho sakta hai:

0.85 threshold ke liye ek toy ROC curve

Aakhir mein, "0.85 standard hai" wala trap ROC analysis ko reference karta hai. Ek ROC curve true-positive rate ko false-positive rate ke against plot karta hai jaise tum confidence threshold slide karte ho; "sahi" operating point tumhare cost trade-off par depend karta hai, kisi magic number par nahi:

True ya false — justify karo

"Staged rollout" ka matlab hai har stage ko ek fixed calendar time ke liye chalana chahe traffic kitna bhi ho.

False. Jo statistic matter karti hai woh hai number of samples seen (upar ke formula mein

n

), na din beethe. Ek low-traffic day par 1% mein ek rare failure surface nahi ho sakti, isliye gate criteria sample-based hone chahiye (ya time plus minimum-sample), sirf clock-based nahi.

Shadow mode mein zero risk hai kyunki users shadow model kabhi nahi dekhte.

User-facing risk ke liye mostly true hai, lekin zero nahi. Shadow model phir bhi compute consume karta hai aur logs/side effects se leak ho sakta hai (e.g. shared cache mein likhna). "No user-visible output" guarantee hai, "no risk at all" nahi.

Ek naya model jo purane se lower overall false-positive rate rakhta hai use promote karna hamesha safe hai.

False. Ek better average ek worse subgroup chhupaata hai: parent ke example mein overall 1.5% FPR tha lekin non-English names par 8% tha. Aggregate wins fairness regressions mask kar sakti hain.

Agar input drift (KL divergence) near zero hai, toh model accuracy guarantee se hold karegi.

False. Zero data drift concept drift rule out nahi karta — inputs same dikhte hain lekin correct label rule change ho gaya. Inputs par drift ek leading indicator hai, completeness guarantee nahi.

Rising average prediction entropy ka hamesha matlab hai model kharab ho gaya.

False. Entropy ka badhna matlab hai model less confident hai, jo genuinely ambiguous ya novel (OOD) inputs par aksar appropriate hota hai. Ye investigate karne ka signal hai, kisi bug ka proof nahi — ek galat taur par over-confident model ki entropy low bhi ho sakti hai.

Ek tripped circuit breaker matlab model toot gaya hai.

False. Iska matlab hai ki recent predictions ek reliability threshold cross kar gayi — jo ek attack, drift, ya upstream data pipeline fault ki wajah se ho sakta hai, model rot ki wajah se zaruri nahi. Breaker diagnose karne ka time deta hai, diagnose nahi karta.

Human-in-the-loop review system se bias eliminate kar deta hai.

False. Humans ke apne biases hote hain aur woh confident-looking AI outputs ko rubber-stamp kar sakte hain (automation bias). HITL risk redistribute aur cap karta hai; ise zero nahi karta.

Groups mein 5% ke andar demographic parity guarantee karti hai ki model fair hai.

False. Parity ek fairness definition hai kai mein se (equalized odds, calibration, ...) aur woh conflict kar sakte hain. Ek metric pass karna matlab doosra fail ho sakta hai, isliye "fair" ke liye metric ka naam lena zaroori hai.

Error pakdo

"Humne 10,000 examples par test kiya aur koi failure nahi mili, isliye 1-in-10,000 se rarer failures production mein nahi hongi."

Test set sirf wahi observe kar sakta hai jo usme hai; 10k samples mein rare failures ka na hona matlab nahi ki woh 10M requests mein absent hain. 0.01% rate 10k mein invisible hai lekin 10M/day par ~1,000 times/day hit karta hai — exactly isliye staged rollout exist karta hai.

"Hamare circuit breaker mein ek fixed all-time error count hai, isliye woh stable hai."

Ek cumulative all-time count sirf badhta hai aur eventually trip kar jaayega chahe model recover ho jaaye (figure s04 dekho). Breaker ko sliding window use karni chahiye (e.g. last 100 predictions) taaki woh recent health reflect kare aur reset bhi ho sake.

"Humne confidence-routing threshold 0.85 set kiya kyunki woh standard hai."

Koi universal threshold nahi hai; 0.85 ek example hai. Sahi value ROC curve par ek operating point hai (figure s05), validation analysis ke zariye choose ki gayi jo tumhare application ke liye automation rate ko error cost se trade karti hai.

"Hum overall accuracy monitor karte hain, isliye hum koi bhi degradation pakad lenge."

Overall accuracy ke liye ground-truth labels chahiye, jo production mein late (ya kabhi nahi) aate hain. Leading indicators — drift, entropy, human override rate — problems tab flag karte hain jab accuracy drop measurable bhi nahi hoti.

"Adversarial inputs training problem hain, deployment problem nahi."

Attackers deployment time par live traffic par act karte hain, test aur production distributions ke beech ki exact gap exploit karte hain (dekho 6.4.1-Adversarial-examples). Runtime monitoring aur circuit breakers deployment-side defenses hain exactly isliye kyunki akele retraining har attack anticipate nahi kar sakti.

"Ek baar validation 100% rollout par pass ho jaaye, monitoring band ki ja sakti hai."

Validation pass karna ek snapshot hai time mein; duniya drift karti rehti hai. Monitoring ongoing pillar hai — ise band karna tumhari drift, attacks, aur silent failures detect karne ki ability remove kar deta hai.

"Higher latency sirf ek UX issue hai, safety issue nahi."

Latency spikes infrastructure failure, retries, ya ek attack indicate kar sakti hai, aur time-critical settings mein (medical, fraud) slow answers unsafe answers ho sakte hain. Isliye ise safety metrics ke saath usi dashboard par monitor kiya jaata hai.

Why questions

Sample-size formula mein

ln (1 - p)

se divide karne par inequality kyun flip hoti hai?

Kyunki

ln (1 - p)

0 < p < 1

ke liye negative hai, aur ek inequality ko negative number se divide karne par uski direction reverse ho jaati hai. Isliye

(1 - p)^{n} \leq α

ban jaata hai

n \geq \frac{l n α}{l n ( 1 - p )}

Input features ke means compare karne ki bajay KL divergence kyun use karein?

Means identical ho sakte hain jabki distribution ki puri shapes alag ho sakti hain (figure s03 mein do same-mean, different-shape distributions dikhti hain). KL poori distributions compare karta hai, wo extra bits measure karta hai jo training data ko encode karne ke liye chahiye ek code se jo production data ke liye tune hai.

"Human override rate" ko dashboard par ground truth ke sabse kareeb kyun kaha jaata hai?

Baaki har metric (drift, entropy, latency) ek proxy hai; override rate ek real insaan hai jo ek real prediction ko judge kar raha hai ki woh galat-enough-to-reject hai. Badhta hua override rate direct evidence hai ki model ki usefulness slip ho rahi hai.

Defense-in-depth assume kyun karta hai ki components fail karenge?

Kyunki kisi bhi single safeguard ki ek failure probability hoti hai; independent safeguards layer karna is probability ko tiny bana deta hai ki sab simultaneously fail karein. Wahi logic hai jaise aircraft par redundant systems — tum us failure ke liye design karte ho jo tum rokh nahi sakte.

1% rollout par directly jaane ki bajay shadow mode kyun chalayein?

Shadow mode tumhe production-scale, production-distribution data deta hai zero user-visible impact ke saath, jo tumhe subgroup aur OOD failures pakadne deta hai kisi bhi real user ke affected hone se pehle. 1% rollout, contrast mein, pehle se hi real users ko naye model ke saamne expose kar deta hai.

Ek model har offline test pass kar sakta hai aur phir bhi production mein fail kyun ho sakta hai?

Offline tests ek fixed, curated distribution use karte hain; production traffic adversarial, drifting, aur long-tail inputs se bhara hota hai jo test set ne kabhi sample nahi kiye. Test distribution aur live distribution ke beech ka mismatch hi core reason hai ki teenon pillars exist karte hain.

Drift threshold ko historical data se kyon tie karein, round number pick karne ki bajay?

Normal week-to-week traffic pehle se hi kuch baseline amount fluctuate karta hai; tumhe woh natural variation jaanni chahiye taaki ek threshold set ho jo real drift par fire kare aur ordinary noise par nahi. Ek round number ya toh constant false alarms ka risk leta hai ya missed shifts ka.

Edge cases

Jis moment circuit breaker trip kare tab kya karna chahiye — saari requests block kar dein?

Use ek safe fallback par route karna chahiye (simpler model, cached answer, human handoff, ya ek graceful "unavailable" message), hard-crash nahi. Point degraded-but-safe service hai, blast radius chhota rakhna jab humans investigate karein.

Ek stage apni window ke dauran zero traffic dekhti hai (holiday, outage). Kya woh pass hoti hai?

Nahi — zero samples matlab zero evidence hai, isliye ek sample-based gate satisfy nahi hoti. Empty window par pass karna puri measurement ki purpose ko defeat kar deta.

Failure rate

p

essentially 0 hai (ek truly safe model). Sample-size formula kya kehta hai?

Jab

p \to 0

ln (1 - p) \to 0^{-}

, isliye

n \geq \frac{l n α}{l n ( 1 - p )} \to \infty

: tumhe infinitely many samples chahiye honge confident hone ke liye ki tumne ek failure dekhi. Sahi baat hai — tum sampling se akele ek arbitrarily rare failure ki absence kabhi fully prove nahi kar sakte.

HITL routing ke liye confidence threshold 0 set karo — kya hoga?

Kuch bhi kabhi humans tak route nahi hoga (saari predictions "0 exceed" karti hain), isliye system fully automated ho jaata hai bina kisi human safety net ke. Use 1 set karo aur sab kuch humans tak route hoga, system ek fully manual system mein collapse ho jaayega bina kisi automation benefit ke.

Saari classes ko equal predicted probability

p_{i} = 1/ C

milti hai. Entropy kya hai, aur iska kya matlab hai?

Entropy apne maximum par hai (

lo g_{2} C

bits, ya

ln C

nats — ek base choose karo aur consistent raho): model maximally uncertain hai, effectively guess kar raha hai. Sustained max-entropy predictions signal karti hain ki model apni depth se bahar hai (OOD) us input par.

Do fairness metrics dono hold karni chahiye lekin mathematically simultaneously nahi kar sakti. Kya karo?

Tum dono "pass" nahi kar sakte; tumhe ek explicit, documented value choice karni hogi ki is context ke liye kaun sa harm zyada matter karta hai, ideally ek governance process ke under rather than silently. Impossibility itself surface karni chahiye, chhupani nahi.

Drift detect hua lekin tumhare paas accuracy drop confirm karne ke liye abhi labels nahi hain. Retrain deploy karo ya wait karo?

Drift ko ek leading alarm treat karo: retrained model ko shadow mode / ek chhote rollout ke peechhe stage karo taaki commit karne se pehle validate kar sako, instead of blind-swap ke. Labels ka wait karna possible silent harm accept karna hai; blind-swap karna ek unvalidated fix ship karna hai.

Naya model aggregate mein better hai lekin 50 users ke ek chhote subgroup ke liye worse hai. Ship karo?

"Small group" harm ke liye license nahi hai — tum un users ke liye harm ki severity aur reversibility evaluate karte ho subgroup analysis ke zariye, sirf headcount nahi. Aksar fix targeted hota hai (e.g. underrepresented data add karo) promote karne se pehle.

Recall One-line self-test

Teen deployment pillars, order mein? ::: Pre-deployment validation → runtime monitoring → human oversight architecture. Teeno ke neeche ek single design principle? ::: Defense in depth — assume karo ki koi ek safeguard fail karega, isliye koi single failure catastrophic na ho.