6.4.13 · D5AI Safety & Alignment
Question bank — AI governance and regulation (EU AI Act)
Before we start, five words we will lean on constantly, defined plainly:
The tier decision picture
Before the traps, anchor the whole page to one image: risk tier is decided by what the system does and to whom, not by how big the model is. Read the flowchart top-to-bottom — the first matching condition sets the tier.
What the figure shows (for readers without the image): a single top box "AI system (any model, any size)" feeds a vertical chain of four yes/no questions. First check: is the use in the Art. 5 list? → if yes, Unacceptable / Prohibited (magenta). If no, next check: is it an Annex III use-case (hiring, credit, medical, law enforcement)? → if yes, High risk carrying the Arts. 9–15 stack (orange). If no: does it interact with or generate content for people (chatbot, deepfake)? → if yes, Limited risk, Art. 50 disclosure (violet). If no: Minimal risk, no obligations (navy). The visual lesson: model size never appears in any box — only use and impact route the system.
The Act's key hooks, so every claim below is checkable against the primary text:
Obligation mapping (which tier owes what)
What the figure shows (for readers without the image): four vertical bars, one per tier, whose height is the number of duty categories that tier owes. Minimal = height 0 (none). Limited = height 1 (just Art. 50 disclosure). High = height 7 — the tall bar — labelled with the full Arts. 9–15 stack (risk management, data governance, documentation, logs, transparency, human oversight, robustness), and remember this stack must additionally be certified through a conformity assessment (Arts. 43–45). Unacceptable = height 0 but coloured magenta with "BANNED (Art. 5)", because the point is not "few duties" but "no compliant version exists at all". The visual lesson: labelling is one short bar; high-risk is a whole ladder.
True or false — justify
A system is high-risk purely because it uses a large, powerful model like a transformer.
False. Trace it: Art. 6 says a system is high-risk only if it falls in an Annex III use-case (or is a safety component of a regulated product), so the routing question is what it does, not how big it is — a huge model powering a spam filter never enters Annex III, so it stays minimal-risk. (Foundation models have their own GPAI rulebook, Arts. 51–55 — a different ladder, not this tier.)
The EU AI Act banned facial recognition.
False. Only real-time remote biometric identification in public spaces for mass surveillance is prohibited (Art. 5(1)(h)); Face-ID phone unlock and post-event forensic review remain allowed with safeguards.
A company based outside the EU is exempt from the Act.
False. The Act is extraterritorial (Art. 2) — if the system's output is used in the EU market, obligations apply regardless of where it was built or hosted.
Minimal-risk AI still needs technical documentation filed with regulators.
False. Trace the tiers: Art. 6 pins the Arts. 9–15 documentation stack to high-risk Annex III systems; a minimal-risk system fails every routing question in the flowchart, lands in the "no obligations" box, and therefore owes no mandatory documentation.
If a high-risk system passes an accuracy benchmark, human oversight becomes optional.
False. Accuracy (Art. 15) and human oversight (Art. 14) are separate, both-required obligations — no accuracy score removes the duty for a human able to intervene or override.
A high-risk provider can always self-certify compliance.
False. Most Annex III systems allow self-assessment, but the most sensitive categories (notably remote biometric identification) require a third-party notified-body assessment (Arts. 43–45); which route applies depends on the use-case, not the provider's preference.
A deepfake used clearly as satire must still be disclosed as AI-generated.
True — this is the corrected point. Art. 50(4) provides no blanket artistic/satirical exemption; it only scales the disclosure so it does not spoil the work (e.g. a discreet notice rather than a giant watermark). The obligation to reveal synthetic origin remains.
The transparency tier requires you to explain how the AI works to every user.
False. Limited-risk transparency (Art. 50) requires disclosure that it is AI (and that content is synthetic), not a full explanation of the model's internals.
Prohibited systems can still be deployed if the vendor accepts full legal liability.
False. Art. 5 uses are banned outright; there is no "pay-to-play" or liability-waiver path, unlike high-risk which is permitted-with-conditions.
Spot the error
"Our chatbot is obviously a bot from the URL, so we skip disclosure — that satisfies the Act because disclosure only applies when it's not obvious."
Partly right, dangerously stated. Art. 50(1) waives disclosure only where AI interaction is evident to a reasonably well-informed user; assuming "the URL makes it obvious" is a weak defence — when in doubt, disclose.
"We can't audit our hiring AI's decisions, but that's fine because the model is a trade secret."
Error. Arts. 11 and 12 make documentation and logging mandatory legal obligations, and the Act's confidentiality clause (Art. 78) protects trade secrets only from public disclosure — it does not let you withhold the file from regulators or market-surveillance authorities. The law resolves the clash by keeping the audit trail intact but treating it confidentially, so "it's a trade secret" cannot defeat auditability.
"Our medical-imaging AI hit 96% sensitivity on our internal test set, so robustness is proven."
Error. One in-house test set does not show robustness across scanners, hospitals, and demographics as Art. 15 demands; robustness means testing on the varied conditions the system will actually meet.
"We labelled the AI's outputs as synthetic, so we've met all our high-risk duties."
Error. Labelling is a transparency (Art. 50) measure — the single short bar in the obligation-mapping figure. A high-risk system additionally owes the tall Arts. 9–15 stack (risk management, data governance, logs, human oversight, robustness) and a conformity assessment (Arts. 43–45) before market.
"We removed the 'gender' column from the training data, so the hiring model can't be biased."
Error. Bias hides in proxy features — innocent-looking fields that statistically stand in for a protected attribute. Example: postcode correlates with ethnicity, first name signals gender, and a "career gap" flag tracks maternity leave, so the model reconstructs the removed attribute from these. Art. 10 therefore requires testing outcomes for disparate impact, not merely deleting one column.
"Post-event facial recognition on stored CCTV to solve a crime is prohibited, same as live surveillance."
Error. The Art. 5 ban targets real-time, public-space, mass identification; targeted post-event analysis with oversight is treated differently and is not blanket-prohibited.
Why questions
Why does the Act ban some systems outright instead of just regulating them heavily?
Because certain uses (social scoring, subliminal manipulation) violate fundamental rights at their core — Recital 28 explains these practices are contrary to Union values of human dignity and autonomy, so no documentation or oversight makes them acceptable and no compliant version can exist.
Why is human oversight required even for very accurate high-risk AI?
Because Recital 73 frames oversight as preventing or minimising risks that persist despite the other safeguards: a person must be able to correctly interpret the output, stay alert to automation bias, and intervene or halt the system — accuracy metrics measure average behaviour, not the specific edge case in front of a real person, and accountability must rest with a human.
Why does the Act tie risk to the application rather than the technology?
Because the same model can be harmless (game) or dangerous (sentencing), so the tier must track potential harm to people; Recital 26's proportionality logic says regulating the tech itself would either over-burden safe uses or under-protect dangerous ones.
Why does a high-risk system need a conformity assessment before it reaches the market, not just monitoring afterwards?
Because Arts. 43–45 put the check ex-ante — the whole harm model is prevention, so the system must be shown to satisfy Arts. 9–15 before it can deny a loan or read an X-ray, with self-assessment for most cases and an independent notified body for the highest-stakes categories.
Why does record-keeping (logging) get its own Article instead of being folded into documentation?
Because Art. 12 targets ex-post traceability of live decisions: if harm occurs, authorities and affected people must reconstruct what data, what model, what decision on that specific run — a static design document (Art. 11) cannot do that, so the running log is a separate, continuous obligation.
Why does live public facial recognition get harsher treatment than storing footage?
Because real-time mass surveillance creates a permanent chilling effect on everyone's public behaviour (Recital 32), whereas targeted post-hoc review of stored footage can be gated behind case-specific oversight and judicial authorisation.
Edge cases
A game uses AI opponents but also an in-game chatbot NPC that role-plays a human. Which obligations bite?
The game logic is minimal-risk (none), but the chatbot triggers limited-risk transparency (Art. 50) — players must be able to know they are talking to AI, so a mixed system inherits the highest applicable tier per component.
A hiring tool is used only to rank candidates, and a human makes every final call — does high-risk still apply?
Yes. Employment AI (Annex III) is high-risk even as a recommender, because ranking materially shapes the human decision; human oversight (Art. 14) is an added safeguard, not an escape from the tier.
An emotion-recognition system is used in a private research lab, not on the public. Prohibited or not?
Not automatically — Art. 5 bans emotion recognition specifically in workplace and education settings, and other bans hinge on public-space mass surveillance; the same technique lands in different tiers depending on where and on whom it is used.
A free open-source model is fine-tuned by a startup into a loan-approval system. Who carries the high-risk obligations?
The party placing the high-risk system on the market or putting it into service (the deployer/provider under Art. 25) — here the startup building the loan tool; "we used open source" does not shift the duty away, and that provider is the one who must run the conformity assessment.
A high-risk system keeps learning after deployment (online updates) — do we re-check compliance?
Yes. A substantial modification that changes intended purpose or performance re-triggers conformity assessment (Art. 43(4)); dynamic systems must have their post-deployment change behaviour pre-defined in the risk-management file (Art. 9) so learned drift stays inside a tested envelope, not treated as "set once, forget".
A high-risk remote-biometric system's provider wants to self-certify to save time. Allowed?
No — this is exactly the edge case where third-party notified-body assessment is mandatory (Arts. 43–45); the self-assessment shortcut available to most Annex III systems does not extend to the most sensitive biometric categories.
A system sits in the limited-risk tier but occasionally makes automated decisions with legal effect on users. What's the trap?
The trap is treating tier as fixed by first impression — if it produces decisions affecting rights or safety it may actually be high-risk (Annex III), and transparency alone would then be insufficient.
Recall One-line summary of every trap
Tier follows use and impact (Art. 6), not model power; the Act follows the EU market (Art. 2), not the builder; GPAI foundation models sit on a separate axis (Arts. 51–55), not the four tiers; Art. 5 bans have no compliant workaround; deepfakes always need disclosure (Art. 50, no satire exemption); high-risk owes the whole Arts. 9–15 stack plus a conformity assessment (Arts. 43–45) that is self-certified for most Annex III cases but requires an independent notified body for the most sensitive categories like remote biometrics; and a substantial modification (Art. 43(4)) re-opens that assessment for systems that change after deployment.