6.4.13 · D1AI Safety & Alignment

Foundations — AI governance and regulation (EU AI Act)

1,738 words8 min readBack to topic

Before you can read the parent note on the EU AI Act, you need to understand the words and pictures hiding inside it. This page builds each one from nothing, in the order that lets the next one make sense.


1. What is "an AI system" at all?

Picture a box. On the left, an arrow going in (the input — a job application, an X-ray, a sentence you typed). On the right, an arrow coming out (the output — "hire / reject", "cancer / healthy", a reply). Inside the box is a rule nobody wrote directly; the box figured it out by looking at thousands of past examples.

Figure — AI governance and regulation (EU AI Act)

2. "Risk" — turning fear into a number-ish idea

The whole Act rests on one word: risk. Let's define it before it gets used.

Think of two knobs. One knob is severity (how much damage — a spam email is mild, a wrongful arrest is severe). The other knob is probability (how often it goes wrong). Turn either knob up and the danger light glows brighter.

Figure — AI governance and regulation (EU AI Act)

3. The four tiers — a staircase, not a switch

Once you can measure risk, you can group systems into levels. The Act uses four.

Picture a staircase seen from the side. The top step (unacceptable) has a wall — you cannot go up there. Each step down carries fewer chains. The bottom step (minimal) is open ground.

Figure — AI governance and regulation (EU AI Act)

4. "Compliance" — the checklist a system must pass

That big brace in the parent — — is not real arithmetic. The signs mean "AND", and the brace means "all of these together". Read it as a checklist where every box must be ticked, not a total you add up.


5. The vocabulary inside the checklist

Each requirement is a term the parent uses freely. Here they are, from zero.


6. "Bias" and "fairness metrics" — why data governance exists

The parent leans on two ideas without defining them: bias and fairness metrics.

Picture two piles of training examples: a tall pile from Group A, a tiny pile from Group B. The box gets lots of practice on A and barely any on B — so it guesses well for A and badly for B. That lopsidedness is the bias.


7. Rights: the "why banned" behind the red lines

The parent justifies the bans with the EU Charter of Fundamental Rights. You just need the plain idea:

Unacceptable-risk systems (social scoring, mass live facial recognition, manipulation) are banned because they attack these rights directly, not because they're merely risky. That's why they sit on the walled-off top step: no amount of paperwork can make a rights-violation acceptable. The far end of this concern — where systems threaten humanity itself — is covered in long-term existential risks, and the research to prevent it in AI safety research.


How the foundations feed the topic

AI system = learned input to output box

Risk = severity times likelihood

Four tiers staircase

Compliance checklist for high risk

Bias and fairness metrics

Fundamental rights

Banned unacceptable tier

EU AI Act governance

Read it top-down: knowing what an AI box is lets you talk about its risk; risk lets you sort into tiers; the high tier needs a compliance checklist; the top tier is banned by rights. All roads lead to the Act.


Equipment checklist

Cover the right side. If you can answer each, you're ready for the parent note.

In one sentence, what is an "AI system"?
A box that turns inputs into decisions using a rule learned from examples, not hand-written.
What two things combine to make "risk"?
Severity (how bad the harm) and likelihood (how often it happens).
Why does the Act use four tiers instead of a good/bad label?
"How much could it hurt?" is sortable; "is it good?" is not — buckets are enforceable.
Which single tier is banned outright?
Only unacceptable risk; high/limited/minimal are all allowed with conditions.
The signs in the compliance formula mean what?
Logical AND — every requirement must hold; it is a checklist, not a sum.
What is "bias" and where does it come from?
The box treats one group worse, because its training examples were lopsided.
What does a "fairness metric" give you?
A number measuring how differently two groups are treated (e.g. demographic parity).
Why are the "red line" systems banned rather than just regulated?
They attack fundamental rights directly, which no paperwork can fix.
Recall Quick self-quiz

Is high-risk AI banned? ::: No — allowed, but with the full 7-item checklist. Can extra documentation replace missing human oversight? ::: No — every box in the AND-checklist is mandatory.