Exercises — AI governance and regulation (EU AI Act)

Unacceptable → High → Limited → Minimal. Memory hook: U-H-L-M, "Ugh, Handle Lightly, Minimal." Ladder ka top red "banned" box hai; bottom green "do-what-you-like" box hai.

Tier: Limited risk (blue box). Duty: transparency (T1 — disclosure of AI use) — ise user ko disclose karna hoga ki woh ek AI se baat kar rahe hain, jab tak context se yeh obvious na ho. Kuch aur nahi (koi audits, koi logs mandated nahi).

Jhooth. Government social scoring Unacceptable tier mein hai — yeh poori tarah prohibited hai. Kitna bhi documentation karein, yeh legal nahi hoga, kyunki yeh human dignity aur freedom of thought ka violation hai. Documentation ek high-risk remedy hai; yeh banned system par apply nahi hoti.

(a) High risk — medical diagnosis health aur safety ko affect karta hai; galat output life-threatening ho sakta hai. (b) Koi bhi teen:

• Accuracy clinical standards tak (jaise sensitivity target).
• Robustness testing alag scanner types, hospitals, aur patient demographics ke across.
• Record-keeping — baad mein dispute review ke liye har prediction ko uske confidence score ke saath log karo.
• Human oversight — ek radiologist AI output review kare; AI "second opinion" hai, final word nahi.
• Data governance — training data diverse ho — age/ethnicity/disease stage ke across.

Tier: Limited risk (transparency tier). Required action: transparency (T2 — output labelling) — generated media ko artificially generated / synthetic label karna hoga taaki viewers jaanein yeh genuine recording nahi hai. Rationale: misinformation prevention + informed judgement.

Dono Minimal risk (green box) mein hain. Obligations: Act ki taraf se koi nahi. Act jaan-bujhkar roz ke AI ko unburdened rakhta hai taaki innovation choke na ho.

• Police A: Unacceptable / prohibited. Yeh public spaces mein mass surveillance ke liye real-time remote biometric identification hai — banned hai (narrow, judicially-approved exceptions ke bahar).
• Police B: Post-event targeted analysis ke roop mein allowed hai — yeh live nahi hai, blanket crowd-scan nahi hai, aur investigative oversight ke andar operate karta hai.
• Dividing lines: (1) real-time vs post-event, (2) public space mass scan vs targeted specific investigation. Live crowd scanning public behaviour par permanent chilling effect banata hai; post-hoc targeted review mein oversight hai aur ek specific subject hai.

Sabse direct obligation: Data governance (quality aur bias testing) — kyunki AI data se patterns seekhta hai, aur skewed data ek skewed model sikhata hai.

Pehle, do fairness ideas plain words mein (hum inhe use karne se pehle define karte hain):

• Demographic parity poochhta hai: kya groups ko same rate pe select kiya jaata hai? Agar 50% men pick hote hain lekin sirf 30% women, toh woh rate gap ek demographic-parity failure hai. Yeh sirf who gets chosen dekhta hai, ignoring karta hai ki actually koun qualified tha.
• Equalized odds stricter hai: jo log truly job deserve karte the, unke liye kya groups mein "yes" rate same hai — aur jo deserve nahi karte the, unke liye kya "wrong-yes" rate same hai? Yeh AI ko ground truth ke against compare karta hai, sirf raw selection counts ke nahi.

Test: ek validation set par yeh fairness metrics chalao aur numbers groups ke across compare karo.

Worked number — four-fifths (80%) rule sanity check (ek common demographic-parity screen): maan lo AI 50% male applicants aur 30% female applicants select karta hai. Impact ratio hai $\frac{0.30}{0.50}=\mathrm{0.60.}$ Kyunki $0.60<0.80$, system fail karta hai four-fifths screen → disparate impact ka strong sign (ek group bahut zyada rate par harm ho raha hai), jo ek data-governance breach flag karta hai.

Surface feature (faces) identical hai; legal trigger context hai.

• System X: non-public, consenting, single-user → "public space mein remote identification" nahi. Yeh Minimal risk mein hai (green box) Act ke under koi obligations nahi — fully allowed hai.
• System Y: remote, real-time, public, mass → yeh exactly Unacceptable / prohibited category hai — banned. Act deployment context regulate karta hai, raw technology nahi. "Facial data use karta hai" koi classification nahi hai; "real-time remote public mass identification" classification hai.

Tier: High risk (credit access ek listed critical domain hai; galat denial fundamental economic rights ko harm karta hai). Compliance stack — component → woh harm jo yeh prevent karta hai:

1. Risk management system → jo risks tum enumerate karo hi nahi, unhe mitigate nahi kar sakte (bias, error, exploitation).
2. Data governance → skewed training data → biased approvals; fairness metrics se test karo (demographic parity / equalized odds, Q8 mein define kiya).
3. Technical documentation → auditability jab koi rejected applicant complain kare.
4. Record-keeping (logs) → traceability: reconstruct karo kya data, kya model, kya decision.
5. Transparency (T3 — right to contest) → applicant ka right jaanne ka ki AI use hua aur decision challenge karne ka, explanation ke saath.
6. Human oversight → ek loan officer context-heavy cases mein model ko override kar sake.
7. Accuracy, robustness, cybersecurity → adversarial manipulation resist karo. (Ek adversarial example ek cleverly tweaked input hai jise model ko fool karne ke liye design kiya gaya hai — full treatment ke liye 6.4.4-Robustness-and-adversarial-examples dekho.)

• (i) Reward hacking = AI apne stated metric par achha score karta hai ek loophole exploit karke instead of intended job karne ke (jaise "approval accuracy" maximize karna ek aise shortcut se jo quietly ek group ko harm karta hai). Data governance + fairness testing countermeasure hai. Full topic: 6.4.2-Reward-hacking.
• (ii) Applicant ka right to contest (T3) require karta hai ki decision explainable ho — tumhe batana hoga kyun model ne na kaha. Model ki reasoning ki woh readability interpretability hai. Logs akele kaafi nahi agar koi unhe padh na sake. Full topic: 6.4.3-Interpretability-and-explainability.
• (iii) Compliance ek floor hai, ceiling nahi: powerful systems ko actually woh kaam karwaane ka broader project jo hum chahte hain, woh ongoing AI safety research hai (6.4.5-AI-safety-research), aur iski sabse serious motivation yeh worry hai ki bahut capable future systems irreversible, civilisation-scale harm cause kar sakti hain — long-term existential risk (6.4.12-Long-term-existential-risks).

Do candidate provisions:

• Manipulative AI jo ek specific group ki vulnerabilities exploit kare (yahaan, bachche).
• Subliminal techniques jo awareness se neeche behaviour influence karein. Dono Unacceptable / prohibited hain. Final tier: banned. Justification: yeh ek protected group ki vulnerability exploit karta hai aur autonomy/informed consent undermine karta hai — koi transparency label ya human-oversight bolt-on ek prohibited purpose ko rescue nahi kar sakta. (Contrast: ek honest educational talking toy jo clear disclosure ke saath aur bina manipulation ke ho, woh limited-risk hoga.)

Mixed systems ke liye rule: har function ko uski khud ki risk se classify karo, aur deployment ko union (sabse strict applicable set) satisfy karna hoga, average nahi.

• Diagnostic function → high-risk full stack (accuracy, robustness, logs, human oversight, data governance, documentation, right-to-contest).
• Chatbot function → limited-risk transparency (T1 — disclose "tum ek AI se baat kar rahe ho"). Result: chatbot ki T1 disclosure duty diagnostic high-risk obligations ke upar stack hoti hai, unki jagah nahi. Tum kabhi lighter tier ko heavier tier dilute nahi karne dete.

Apply hota hai? Haan. Act EU market mein use/effect se trigger hota hai, is se nahi ki system kahaan bana. EU applicants ko affect karne wala foreign-built hiring AI ek high-risk system ki tarah comply karna padega. Zero-case (koi EU users nahi, koi EU market effect nahi): Act apply nahi hota — jurisdiction limiting condition hai. Jaise EU-market involvement → 0, obligations → 0. Yeh parent note mirror karta hai: "chahe kahaan bhi develop hua ho," lekin conditioned on EU market.

Danger ke order mein char tiers ::: Unacceptable, High, Limited, Minimal Ek chatbot ki single legal duty ::: transparency T1 (disclose karo ki yeh AI hai) Deepfake par required action ::: transparency T2 (output ko synthetic label karo) High-risk "right to contest" duty ::: transparency T3 (explain karo aur challenge allow karo) Ek prohibited system kaafi paperwork se legal ban sakta hai ::: Nahi — prohibited ek hard wall hai Ek loan-approval AI ka tier ::: High risk Demographic parity check karta hai ::: ki groups equal rates par select ho rahe hain ya nahi Equalized odds check karta hai ::: ki ground truth ke against, correct-yes aur wrong-yes rates groups mein match karti hain ya nahi Mixed-tier deployment follow karta hai ::: union / strictest applicable obligations Act ka jurisdiction trigger hota hai ::: EU market mein use / effect se, is se nahi ki kahaan bana Four-fifths screen: impact ratio 0.30/0.50 = 0.60, jo ::: fail karta hai (0.80 se neeche) → disparate impact