System vs user mode and privilege levels: Visual walkthrough

Step 1 — The problem: two programs, one CPU

WHAT. A CPU is just a box that reads instructions from memory and does them, one after another. Right now, nothing stops one program from doing anything.

WHY start here. Before we build a lock, we must see clearly what we are locking and from whom. On the board we draw two programs — a normal app and the operating system kernel (the trusted master program that manages everything). Both feed instructions into the same CPU.

PICTURE. Look at the two boxes on the left both pointing arrows into the single CPU. Some of those instructions (drawn in pink) are dangerous: HALT (stop the machine), CLI (disable interrupts), "remap memory". Nothing yet distinguishes who is allowed to send them.

Figure — System vs user mode and privilege levels

The goal of the whole derivation, stated once:

$not trusted user program \neq ⟶ must be blocked privileged instruction$

The slash means "this arrow must be forbidden". Everything below builds the hardware that enforces that slash.

Step 2 — Invent a bit: "who is running right now?"

WHAT. We add one bit of state inside the CPU, living in a special status register (on real chips the Processor status register (PSW) / mstatus / CPSR). Call it the mode bit $M$ .

WHY. The check has to be instant and happen on every instruction, so the "who am I" answer can't sit in slow memory or be recomputed — it must be a live hardware bit the CPU already holds.

$M = {01 kernel mode (trusted, can do anything) user mode (restricted)$

Reading the equation term by term:

$M$ — the single bit inside the status register.
$M = 0$ — the CPU currently believes the kernel is running. Full power.
$M = 1$ — the CPU currently believes a user program is running. Limited.

PICTURE. The CPU now has a small yellow flag drawn on it — that's $M$ . When the kernel runs, the flag reads $0$ ; when an app runs, it reads $1$ . Nothing checks it yet; we just store it.

Step 3 — Put the check in the decoder

WHAT. Every instruction passes through the decoder — the part of the CPU that reads the raw bits of an opcode and works out what it means (see Instruction encoding and decoding). We add a rule right there.

WHY here and nowhere else. Enforcement must happen before the dangerous effect occurs. The decoder is the last gate before execution, so this is the natural checkpoint. Checking after would be too late — the machine would already be halted.

The rule, in words: if the opcode is privileged and the mode bit says user, refuse and raise a trap.

$if (opcode \in Privileged) \land (M = 1) \Rightarrow trap$

Term by term:

$opcode \in Privileged$ — this instruction is on the dangerous list.
$\land$ — logical AND; both conditions must hold.
$M = 1$ — we are in user mode.
$trap$ — abort the instruction, hand control to the kernel instead (an exception).

PICTURE. Follow the instruction flowing into the decoder. A diamond asks two questions; only when both answers are "yes" does the red TRAP path fire. A harmless instruction (blue) sails straight through to "execute".

Step 4 — Close the obvious hole: writing $M$ must itself be privileged

WHAT. A clever user program says: "Fine — I'll just set $M \leftarrow 0$ first, then run my dangerous instruction." We must forbid that.

WHY this is the linchpin. A referee you can bribe is no referee. So the instruction "write the mode bit" is placed on the privileged list itself. To become kernel, you must already be kernel. This is a deliberate fixed point: user code can never promote itself.

$"set M \leftarrow 0 " \in Privileged$

Reading it: the act of clearing the mode bit to kernel is itself a dangerous instruction, so by Step 3 a user program attempting it just... traps. The loophole seals itself.

PICTURE. A user program's arrow tries to reach in and flip the yellow flag to $0$ . The attempt hits the same decoder check from Step 3 and bounces off into a TRAP. The flag stays at $1$ .

Step 5 — But users need OS help: the one legal doorway

WHAT. Real programs must read files, print, allocate memory — all privileged underneath. So we add exactly one controlled way to enter the kernel: the system call instruction (syscall / ecall / int 0x80), the OS interface.

WHY it's safe even though it enters kernel mode. Two guarantees, both enforced by hardware:

$user (M = 1) syscall one atomic HW step kernel (M = 0), P C \leftarrow V_{OS}$

$M = 1 \to M = 0$ — yes, you do become kernel.
$P C \leftarrow V_{OS}$ — but the program counter (the address of the next instruction) is set to $V_{OS}$ , a fixed entry vector chosen by the OS, not by you.
one atomic HW step — the mode-flip and the jump happen as a single uninterruptible act.

So the user picks that they enter the kernel — never where. You slide a note through the one window; you don't choose which room behind the counter you land in.

PICTURE. The user's syscall arrow goes up through a single narrow slot in a wall. On the other side it lands only on the yellow "OS entry vector" dot — it physically cannot land anywhere else along the wall.

Step 6 — Coming back down, safely

WHAT. When the kernel finishes serving the request, it runs a return instruction (sret / iret) that drops back to user mode and resumes the app right after its syscall.

WHY it must be one atomic act too. Same reasoning mirrored: mode $0 \to 1$ and the jump back to the saved user address happen together, so no one runs at the wrong privilege in between.

$kernel (M = 0) atomic sret / iret user (M = 1), P C \leftarrow saved return address$

$M = 0 \to M = 1$ — power drops back to restricted.
saved return address — the exact instruction after the original syscall, stashed when we entered.

PICTURE. The round-trip: app → up through the slot → kernel does the privileged work → back down through the same slot → app resumes. The flag traces $1 \to 0 \to 1$ .

Step 7 — The degenerate case: a program that never calls the OS

WHAT. Consider while(1){} — an infinite loop that runs zero system calls and zero privileged instructions. Steps 1–6 never fire. Does it own the CPU forever?

WHY this case matters. If the answer were "yes", one program could freeze the machine — exactly the disaster from the parent note. The fix is external: an interrupt the app can't refuse.

Before dispatching the loop, the kernel programmed a hardware timer (arming it is privileged — the loop can't disarm it). When it fires:

$timer fires \Rightarrow same atomic trap as Step 5 \Rightarrow M = 0, P C \leftarrow V_{timer}$

The loop is interrupted mid-spin; the scheduler regains control and can switch to another process.

PICTURE. The endless loop spins (blue circle). A yellow lightning bolt — the timer — strikes it and forces the same "up through the slot" transition. The loop cannot see it coming and cannot stop it.

Step 8 — Generalise: two levels become rings

WHAT. We built a single bit $M$ with two values. Real CPUs generalise it to a small number, the Current Privilege Level (CPL), giving concentric rings.

WHY. Some code (device drivers, hypervisors) wants power between "full kernel" and "plain app". A number instead of a bit lets you order them.

$CPL = 0 (kernel, most power) \leq \dots \leq CPL = 3 (user, least power)$

Lower number = more privilege (counter-intuitive!).
x86: rings 0–3. RISC-V: M/S/U. ARM: EL0–EL3.
Our rule generalises: an instruction needing level $r$ traps whenever $CPL > r$ .

PICTURE. Concentric rings, kernel at the bullseye (ring 0), user on the outermost ring (ring 3). An arrow reminds you: inner = closer to the core = more trusted.

The one-picture summary

Everything at once: two programs, the mode bit $M$ , the decoder check, the sealed self-promotion hole, the single syscall doorway with its atomic up/down transitions, and the timer that forces even a silent loop back to the kernel.

Recall Feynman: the whole walkthrough in plain words (click to reveal)

We started with a dumb CPU where any program could do anything — including freeze the machine. Step 1: we sorted instructions into harmless and dangerous. Step 2: we glued a single flag onto the CPU that says "kernel" or "user" right now. Step 3: we taught the instruction decoder one rule — if the instruction is dangerous and the flag says user, slam on the brakes (a trap). Step 4: we noticed a cheat — a program flipping the flag itself — so we put "flip the flag" on the dangerous list too; now you can't promote yourself. Step 5: but honest programs still need help, so we cut one door in the wall — the syscall — and made it so you only choose whether to knock, never which room you land in, and the door opens in one indivisible motion so no one sneaks through the gap. Step 6: the return trip works the same in reverse. Step 7: for a program that never knocks and never misbehaves, we let the kernel arm a timer the program can't touch, so it gets yanked back anyway — that's how the OS always stays in charge. Step 8: finally, one flag becomes a small number so we can have in-between trust levels (rings), where the smaller the number, the more power you have. That single flag, one decoder rule, and one atomic doorway are the entire foundation of every secure operating system.

Recall Quick self-test

Why can't a user loop disable the timer to run forever? ::: Disabling interrupts (cli) is privileged; attempting it in user mode traps, so the timer keeps firing. What does syscall change that a normal call does not? ::: It changes the privilege level (mode bit) via an atomic hardware trap and jumps to an OS-chosen entry vector. Why is "write the mode bit" itself a privileged instruction? ::: So user code can't promote itself to kernel — it's the fixed point that makes the whole scheme unbypassable.