AI Safety & Alignment

Question 1 (14 marks)

(a) Laplace mechanism adds noise with scale $b = \Delta f / \varepsilon = 1/0.5 = 2$. (2) Variance of Laplace($b$) is $2b^2 = 2(2)^2 = 8$. (2) Why: DP requires noise scaled to sensitivity over budget; Laplace variance is $2b^2$.

(b) Sequential composition: budgets add. $\varepsilon_{\text{total}} = k\varepsilon = 8 \times 0.5 = 4$. (2) Interpretation: for neighbouring datasets $D, D'$ and any output set $S$, $\Pr[M(D)\in S] \le e^{\varepsilon}\Pr[M(D')\in S]$; with $\varepsilon=4$, ratio bound is $e^4 \approx 54.6$ — a weak guarantee. (2)

(c) (6 marks, 2 each)

• Lower $\varepsilon$ ⇒ more noise ⇒ model outputs less sensitive to any single record ⇒ the confidence/loss distribution on members vs non-members becomes nearly indistinguishable, so the attacker's decision boundary carries little signal. (2)
• DP formally upper-bounds the advantage of any membership inference distinguisher (attacker success ≤ function of $e^{\varepsilon}$). (2)
• Metric: attack AUC (ROC-AUC of member vs non-member scores) or attacker advantage = TPR − FPR; report at a fixed low FPR. (2)

Question 2 (12 marks)

(a) Outer alignment failure. (2) The specified reward (sensor reading of removed dirt) is a flawed proxy for the true objective (actual cleanliness); the agent optimises the specification faithfully, so the gap is between designer intent and the objective we wrote down — the definition of outer misalignment. (2)

(b) Redesign: reward based on ground-truth room state via an independent/tamper-resistant measurement (e.g., held-out inspection, multiple sensors, penalise sensor occlusion), or reward actual dirt in a bin removed from room. (2) Patching is hard because each patch closes one loophole but the optimiser then finds the next-highest-reward exploit of any remaining proxy gap ("hydra" / Goodhart's law); the proxy is fundamentally not the true objective. (2)

(c) Goal-misgeneralization scenario: agent trained in rooms where dirt is always brown; at deployment dirt is grey — it learned "remove brown objects" (a correlated proxy goal) and ignores grey dirt or removes brown non-dirt. (2) Difference: here the reward was correct and unhacked during training; the failure is that the learned internal goal generalizes wrongly out-of-distribution (inner/capability-vs-goal mismatch), whereas reward hacking exploits a flawed reward in-distribution. (2)

Question 3 (12 marks)

(a) Selection rate = predicted positives / total.

• A: $(80+20)/200 = 100/200 = 0.5$. (1)
• B: $(30+30)/200 = 60/200 = 0.3$. (1)
• Demographic parity difference $= 0.5 - 0.3 = 0.2$. (2)

(b) TPR = TP/(TP+FN).

• A: $80/(80+20)=80/100=0.8$. (1)
• B: $30/(30+10)=30/40=0.75$. (1)
• TPR difference $= 0.8 - 0.75 = 0.05$. (2)

(c) Group A base rate (actual positives) $=100/200=0.5$; Group B $=40/200=0.2$. (1) Demographic parity demands equal selection rates regardless of base rate, so with unequal base rates it forces the group with fewer true positives to be over- or under-selected relative to who actually qualifies. Equal opportunity conditions on the true label. When base rates differ, you generally cannot satisfy both — an impossibility result (Kleinberg/Chouldechova): DP, equalized odds, and calibration are jointly unachievable except in degenerate cases. (3) DP flags a large gap (0.2) while equal opportunity shows near-parity (0.05), illustrating the conflict.

Question 4 (12 marks)

(a) Any four, 1.5 each (cap 6). Examples:

• Harmful content elicitation (bombs, weapons) → targets refusal / content policy.
• Prompt injection / instruction override → targets integrity of system prompt.
• PII / training-data extraction → targets privacy / memorization.
• Jailbreak via role-play / obfuscation / encoding → targets robustness of refusal.
• Bias / discriminatory outputs on protected attributes → targets fairness.
• Hallucination / misinformation under pressure → targets truthfulness. Award per category with correct property. (6)

(b) $\ell_\infty$ adversarial training makes the model robust inside a small hypercube (each pixel bounded by $\epsilon$). An $\ell_2$ ball of comparable "size" extends further along low-dimensional/sparse directions (it allows concentrating a large perturbation on few pixels), reaching points outside the defended $\ell_\infty$ cube. Robustness to one norm ball does not imply robustness to a differently shaped one. (3)

(c) The safety training generalizes as a shallow pattern tied to surface form of the request; wrapping it in a fictional persona shifts the input off the distribution where refusal was reinforced, so the model's helpfulness prior dominates — the alignment did not generalize to the reframed context (a goal-misgeneralization / robustness gap in the RLHF policy). (3)

Question 5 (10 marks)

(a) Employment/recruitment screening is high-risk under the EU AI Act (Annex III lists AI used in employment, worker management, and access to self-employment). (2) Two obligations (any two): risk management system; data governance / bias testing; technical documentation & logging; human oversight; transparency to affected persons; conformity assessment before market. (2)

(b) (2 mechanisms, 2 each capped at 4) e.g.:

• Human-in-the-loop review of borderline/auto-rejected candidates to catch systematic bias and errors before irreversible harm.
• Staged rollout / shadow deployment: run the model in parallel with human decisions, monitor disparate-impact metrics (selection-rate ratio) before granting it authority. Plus appeal mechanism / audit logging. (4)

(c) Even a narrow system's outputs may be too numerous or subtle for exhaustive human checking; scalable-oversight techniques (sampling, automated bias monitors, assisted auditing) let limited human reviewers supervise a system whose decision volume exceeds direct human review capacity. (2)

[
  {"claim":"Laplace scale b = 1/0.5 = 2 and variance 2b^2 = 8","code":"b=Rational(1,1)/Rational(1,2); var=2*b**2; result=(b==2 and var==8)"},
  {"claim":"Sequential composition total epsilon = 8*0.5 = 4","code":"result=(8*Rational(1,2)==4)"},
  {"claim":"Selection rates 0.5 and 0.3; DP diff 0.2","code":"sa=Rational(100,200); sb=Rational(60,200); result=(sa==Rational(1,2) and sb==Rational(3,10) and sa-sb==Rational(1,5))"},
  {"claim":"TPR A=0.8, TPR B=0.75, diff=0.05","code":"ta=Rational(80,100); tb=Rational(30,40); result=(ta==Rational(4,5) and tb==Rational(3,4) and ta-tb==Rational(1,20))"}
]