Level 1 — RecognitionAI Safety & Alignment

AI Safety & Alignment

⏱ 20 minutes30 marksprintable — key stays hidden on paper

Chapter: 6.4 AI Safety & Alignment Level: 1 — Recognition (MCQ, Matching, True/False with justification) Time limit: 20 minutes Total marks: 30

Section A — Multiple Choice (1 mark each, 10 marks)

Choose the single best answer.

Q1. The alignment problem is best described as ensuring that:

(a) AI systems run faster on modern hardware
(b) An AI system's actual behaviour matches the goals its designers intend
(c) Neural networks converge during training
(d) Datasets are perfectly balanced

Q2. Outer alignment concerns:

(a) Making the specified reward/objective faithfully capture the designer's true intent
(b) Ensuring the learned model internalises the specified objective
(c) Hardening a model against jailbreaks
(d) Compressing model weights

Q3. A robotic boat that endlessly loops to collect points instead of finishing the race is a classic example of:

(a) Goal misgeneralization
(b) Membership inference
(c) Reward hacking / specification gaming
(d) Differential privacy

Q4. Goal misgeneralization occurs when a model:

(a) Achieves high reward on the specified metric during training
(b) Learns a capability but pursues an unintended goal that was correlated with reward in training, then fails under distribution shift
(c) Refuses all harmful requests
(d) Leaks training data

Q5. In differential privacy, decreasing the privacy budget $\varepsilon$ generally:

(a) Increases privacy protection but adds more noise/reduces utility
(b) Decreases privacy protection
(c) Has no effect on noise
(d) Removes the need for a training set

Q6. Scalable oversight research primarily addresses:

(a) Reducing GPU energy costs
(b) How humans can supervise AI on tasks too complex to evaluate directly
(c) Watermarking generated images
(d) Encrypting network traffic

Q7. A backdoor (data poisoning) attack is characterised by:

(a) A trigger pattern that causes targeted misbehaviour while accuracy on clean inputs stays normal
(b) Randomly corrupting all labels equally
(c) Adding differential privacy noise
(d) Fine-tuning on more clean data

Q8. Under the EU AI Act's risk-based approach, systems posing unacceptable risk are:

(a) Lightly regulated
(b) Prohibited
(c) Exempt from all rules
(d) Only subject to voluntary codes

Q9. An adversarial example is:

(a) A slow inference request
(b) An input with a small, often imperceptible perturbation crafted to cause a wrong prediction
(c) A duplicated training row
(d) A watermark embedded in text

Q10. Demographic parity as a fairness metric requires that:

(a) True positive rates are equal across groups
(b) The positive prediction rate is equal across protected groups
(c) Calibration holds within each group
(d) All errors are eliminated

Section B — Matching (1 mark each, 8 marks)

Match each term (Q11–Q18) to the correct description (A–H). Each is used once.

#	Term
Q11	Inner alignment
Q12	Red-teaming
Q13	Jailbreak
Q14	Membership inference
Q15	Watermarking / provenance
Q16	Existential risk (x-risk)
Q17	Reward hacking
Q18	Responsible deployment practice

Descriptions:

A. A prompt that bypasses a model's safety guardrails to elicit disallowed output
B. Determining whether a specific record was part of a model's training set
C. Ensuring the model's internally learned objective matches the specified training objective
D. Structured adversarial probing to discover a model's failure modes before release
E. Embedding a detectable signal to identify AI-generated content and trace origin
F. A risk that could permanently curtail humanity's potential or cause extinction
G. Exploiting flaws in a reward signal to score highly without the intended behaviour
H. Staged rollout with monitoring, model cards, and incident response

Section C — True/False with Justification (2 marks each, 12 marks)

State True or False and give a one-sentence justification. (1 mark answer + 1 mark justification.)

Q19. "A model can have perfect outer alignment yet still be inner-misaligned."

Q20. "Increasing the differential-privacy parameter $\varepsilon$ from 0.5 to 5 makes the guarantee stronger."

Q21. "Adversarial examples prove that high test accuracy alone guarantees robustness."

Q22. "Equalized odds and demographic parity are always simultaneously satisfiable for any classifier."

Q23. "Reward hacking is a symptom of outer misalignment (a flawed specification), not necessarily of the model being 'malicious'."

Q24. "Under the EU AI Act, a high-risk system is banned outright."

Answer keyMark scheme & solutions

Section A (1 mark each)

Q1 — (b). Alignment = actual behaviour matching intended goals; (a),(c),(d) are performance/data issues, not intent-matching.

Q2 — (a). Outer alignment = the specified objective faithfully encodes designer intent. (b) is inner alignment.

Q3 — (c). Looping for points instead of finishing is the canonical CoastRunners specification-gaming/reward-hacking case.

Q4 — (b). Goal misgeneralization: capable model pursues a proxy goal that was correlated with reward in-distribution and fails off-distribution. (a) alone can be fine.

Q5 — (a). Smaller $\varepsilon$ ⇒ tighter bound $e^{\varepsilon}$ ⇒ stronger privacy, requiring more noise, lowering utility.

Q6 — (b). Scalable oversight = supervising AI on tasks too hard for direct human evaluation (e.g., debate, RRM, recursive reward modelling).

Q7 — (a). Backdoor: trigger causes targeted misbehaviour; clean accuracy stays high, making it stealthy. (b) is not stealthy.

Q8 — (b). Unacceptable-risk systems (e.g., social scoring) are prohibited under the Act. High-risk are heavily regulated but allowed.

Q9 — (b). Adversarial example = small crafted perturbation flipping the prediction.

Q10 — (b). Demographic parity = equal positive-prediction rate across groups; (a) is equal opportunity, (c) is calibration.

Section B (1 mark each)

Q	Term	Match
Q11	Inner alignment	C
Q12	Red-teaming	D
Q13	Jailbreak	A
Q14	Membership inference	B
Q15	Watermarking / provenance	E
Q16	Existential risk	F
Q17	Reward hacking	G
Q18	Responsible deployment	H

Section C (2 marks each: 1 verdict + 1 justification)

Q19 — TRUE. Even with a perfectly specified objective (outer aligned), the model may internalise a different proxy goal during training, so inner alignment can still fail.

Q20 — FALSE. Larger $\varepsilon$ = weaker privacy; the multiplicative bound $e^{\varepsilon}$ grows from $e^{0.5}\approx1.65$ to $e^{5}\approx148$ , allowing far more information leakage.

Q21 — FALSE. Adversarial examples show accurate models can be fooled by tiny perturbations; high clean accuracy does not imply robustness.

Q22 — FALSE. Except in trivial/degenerate cases, fairness impossibility results show demographic parity and equalized odds generally cannot both hold when base rates differ across groups.

Q23 — TRUE. Reward hacking arises when the specification (reward) rewards unintended behaviour; the agent is simply optimising the proxy, so it's a specification (outer) flaw, not intent.

Q24 — FALSE. High-risk systems are permitted but subject to strict obligations (risk management, documentation, oversight); only unacceptable-risk systems are banned.

[
  {"claim":"DP: e^0.5 < e^5, larger epsilon weaker privacy (Q20)","code":"e05 = exp(Rational(1,2)); e5 = exp(5); result = bool((e05 < e5) and (float(e5) > 100) and (float(e05) < 2))"},
  {"claim":"Smaller epsilon gives tighter multiplicative bound (Q5)","code":"result = bool(exp(Rational(1,10)) < exp(Rational(1,2)))"},
  {"claim":"e^5 approx 148.4","code":"val = float(exp(5)); result = bool(abs(val - 148.413) < 0.01)"},
  {"claim":"Matching is a bijection over 8 items (Q11-Q18)","code":"m = {'Q11':'C','Q12':'D','Q13':'A','Q14':'B','Q15':'E','Q16':'F','Q17':'G','Q18':'H'}; result = bool(len(set(m.values()))==8 and set(m.values())==set('ABCDEFGH'))"}
]