AI Safety & Alignment

Q1. (6 marks)

• Alignment problem: the challenge of ensuring an AI system's objectives and behaviour reliably match the intended goals/values of its human designers, especially as capability increases. (2)
• Outer alignment: the specified objective/reward function correctly captures what designers actually want (getting the right objective). (2)
• Inner alignment: the model actually learned/pursues the specified objective rather than a proxy goal that merely correlated with it during training (the learned objective matches the specified one). (2)

Q2. (5 marks)

• Reward hacking / specification gaming: an agent achieves high reward by exploiting flaws or loopholes in the reward specification without accomplishing the designer's true intent. (3)
• Example (any valid): a boat-racing agent (CoastRunners) circling to collect respawning score bonuses instead of finishing the race; or a cleaning robot covering a mess rather than removing it. (2)

Q3. (5 marks)

• Goal misgeneralization: the model learns a capable but wrong goal that was consistent with the training distribution, and pursues it competently under distribution shift, producing undesired behaviour despite a correct reward signal. (3)
• Difference: reward hacking exploits a misspecified reward (outer problem, reward is at fault); goal misgeneralization can occur even with a correct reward—the failure is that the wrong internal goal was learned (an inner/generalization problem). (2)

Q4. (9 marks) (a) Selection rate = predicted-positive / total. Group A: $(TP+FP)/N = (40+10)/100 = 0.50$. (1) Group B: $(30+30)/100 = 0.60$. (1) (b) Demographic parity requires equal selection rates. $0.50 \ne 0.60$, so demographic parity does not hold. (2) (c) TPR = TP/(TP+FN). Group A: $40/(40+20) = 40/60 = 0.667$. (1) Group B: $30/(30+10) = 30/40 = 0.75$. (1) Since $0.667 \ne 0.75$, equal opportunity does not hold. (1)

Q5. (6 marks)

• $\varepsilon$-differential privacy: A randomized mechanism $M$ is $\varepsilon$-DP if for all datasets $D, D'$ differing in one record and all output sets $S$: $\Pr[M(D)\in S] \le e^{\varepsilon}\,\Pr[M(D')\in S].$ (4)
• Membership inference attack: attempts to determine whether a particular data point was part of the model's training set. (2)

Q6. (5 marks)

• Backdoor/trojan attack: an adversary poisons training data so the model behaves normally on clean inputs but produces an attacker-chosen (malicious) output whenever a specific pattern is present. (3)
• The activating component is the trigger (trigger pattern). (2)

Q7. (4 marks)

• Laplace mechanism scale: $b = \Delta f / \varepsilon = 1/0.5 = 2$. (2)
• Variance of Laplace($b$) $= 2b^2 = 2(2)^2 = 8$. (2)

Q8. (4 marks)

• Scalable oversight: techniques that let humans supervise, evaluate, and correct AI systems even on tasks where direct human evaluation is difficult or infeasible (e.g. via debate, recursive reward modelling, AI-assisted evaluation). (2)
• Necessity: when the AI surpasses human ability, humans can no longer directly judge correctness of outputs, so decomposition/assistance is needed to provide reliable training/evaluation signals. (2)

Q9. (4 marks)

• Jailbreak: a crafted prompt that bypasses a language model's safety guardrails to elicit prohibited content; defence: RLHF safety training / input–output filtering / adversarial red-teaming. (2)
• Adversarial example: a small, often imperceptible perturbation to an input that causes misclassification; defence: adversarial training / certified robustness / input preprocessing. (2)

Q10. (6 marks)

• Four tiers (increasing risk): Minimal risk → Limited risk → High risk → Unacceptable risk. (4)
• High-risk example (any valid): CV screening / recruitment tools, credit scoring, biometric identification, critical-infrastructure or medical-device AI. (2)

[
  {"claim":"Group A selection rate 0.50, Group B 0.60, not equal (demographic parity fails)","code":"a=(40+10)/100; b=(30+30)/100; result = (a==0.5) and (b==0.6) and (a!=b)"},
  {"claim":"TPR A = 2/3, TPR B = 0.75, not equal","code":"tpra=Rational(40,40+20); tprb=Rational(30,30+10); result = (tpra==Rational(2,3)) and (tprb==Rational(3,4)) and (tpra!=tprb)"},
  {"claim":"Laplace scale b=2 and variance=8 for sensitivity 1, epsilon 0.5","code":"b=Rational(1,1)/Rational(1,2); var=2*b**2; result = (b==2) and (var==8)"}
]