Visual walkthrough — OS roles — resource management, hardware abstraction, protection

---

Step 1 — What one program actually does with its time

WHAT. A running program does not use the CPU every single moment. It alternates between two states:

• Computing — doing arithmetic, running instructions on the CPU (a "CPU burst").
• Waiting — it asked the disk or the network for something and now just sits there until the data arrives (an "I/O wait"). During a wait, the CPU has nothing to do for this program.

WHY this matters. The CPU is the expensive part. If a program spends most of its life waiting, then a CPU dedicated to only that program is idle most of the time — money burning for no work. We need to measure "how much of the time is the program actually waiting?"

Let us name that fraction with a symbol we will use for the whole page:

PICTURE. Below, one program's timeline is a bar. Green chunks = computing (CPU busy), grey chunks = waiting (CPU idle for it). Eyeball it: the grey covers about 80% of the bar, so $p=0.8$.

---

Step 2 — One program alone wastes the CPU

WHAT. Put a single program on the CPU. Whenever it waits, the CPU has no other work, so the CPU waits too. The CPU is busy only during the green chunks.

WHY. With one program, "CPU busy" and "program computing" are the same event. So the fraction of time the CPU is busy is just the fraction the program computes — the grey part $p$ is pure waste.

PICTURE. Two stacked bars: the program's timeline, and directly under it the CPU's own busy/idle timeline. They match perfectly — every grey gap in the program is a red idle gap in the CPU.

The utilization (fraction of time the CPU does useful work) for one program is therefore:

$U_1=\underset{\text{all time}}{\underbrace{1}}-\underset{\begin{array}{c}\text{fraction}\\ \text{CPU sits idle}\end{array}}{\underbrace{p}}=1-p$

• The $1$ means "100% of the timeline".
• We subtract $p$ because that is exactly the grey/idle fraction.
• What's left, $1-p$, is the green/busy fraction.

With $p=0.8$: $U_1=1-0.8=0.20$. The CPU is busy only 20% of the time. Terrible.

---

Step 3 — The key idea: load a SECOND program to fill the gaps

WHAT. Keep two programs in memory at once. When program A is waiting for its disk, the OS hands the CPU to program B. The CPU only truly idles when both A and B happen to be waiting at the same moment.

WHY. This is the entire point of the OS's scheduler: A's grey gaps can be plugged with B's green work. The CPU is now idle in a smaller set of moments — only the overlap of two waits.

PICTURE. Three stacked bars: A's timeline, B's timeline, and the resulting CPU timeline. Notice the CPU bar is red (idle) only where a grey chunk in A lines up with a grey chunk in B. Most of A's old idle gaps are now filled by B's green.

---

Step 4 — Turning "both waiting" into a number

WHAT. We need the probability that at a random instant, all loaded programs are waiting at once — because that's the only time the CPU is idle.

WHY we reach for multiplication. Here is the tool and why this tool and not another: when two events are independent (one program's waiting tells you nothing about the other's, because they run different code touching different devices), the chance they both happen is the product of their chances. Addition would answer "either one waits"; we want "both wait together", which is why we multiply, not add.

PICTURE. A little 2-D "chance square". The width is program A's timeline (fraction $p$ of it is a "waiting" strip). The height is program B's timeline (again a fraction $p$ waiting). The shaded corner where both are waiting is a rectangle of area $p\times p=p^2$. Area = probability.

$P(\text{A waiting and B waiting})=\underset{\text{A waits}}{\underbrace{p}}\times \underset{\text{B waits}}{\underbrace{p}}=p^2$

With $p=0.8$: the idle corner is $0.8\times 0.8=0.64$... wait, that seems large — but read the next step: $0.64$ is the idle fraction, so busy $=1-0.64=0.36$. Better than 0.20 already.

---

Step 5 — Generalise to $n$ programs

WHAT. With $n$ programs loaded, the CPU is idle only when every single one is waiting simultaneously. Multiply the waiting fraction by itself $n$ times.

WHY. Same independence idea, extended: "all $n$ waiting" is the AND of $n$ independent waits, so we multiply $n$ copies of $p$. Repeated multiplication of the same number is what the exponent notation means — that's why an exponent appears here rather than, say, $n\cdot p$.

PICTURE. A curve of $U=1-0.8^n$ against $n$. It rises steeply then flattens toward $1$ — the visual signature of "diminishing but real returns" from adding jobs.

Plug in the parent note's numbers to confirm the curve:

---

Step 6 — The edge cases (never leave a scenario unshown)

Every corner of the formula must make sense, or we don't trust it.

Case $p=0$ (a program that never waits — pure computation). Then $p^n=0^n=0$, so $U=1-0=1$. The CPU is 100% busy with a single such program. Makes sense: there are no gaps to fill, so extra programs add nothing.

Case $p=1$ (a program that only waits, never computes). Then $p^n=1^n=1$, so $U=1-1=0$. The CPU never does useful work no matter how many you load — every program is always waiting. The formula honestly reports "hopeless".

Case $n=0$ (no programs loaded at all). By convention $p^0=1$, so $U=1-1=0$. Correct: with nothing to run, the CPU is idle 100% of the time.

Case $n\to \infty$ (infinitely many jobs). Since $0\le p<1$, repeatedly multiplying drives $p^n\to 0$, so $U\to 1$. You approach a fully-busy CPU but never quite exceed it — $U$ is capped at 1 because you cannot be more than 100% busy.

Reality-check case (why we can't just add jobs forever). The model assumes waits are truly independent and memory is free. In practice, loading too many programs makes them compete for RAM and forces paging to disk, which adds I/O and pushes $p$ back up — this is where the flat top of the Step 5 curve turns into thrashing. The formula shows the ideal ceiling; the OS's job is to stop just below it.

PICTURE. The curve again, with the four edge points marked: $p=0$ line pinned at $U=1$, $p=1$ line pinned at $U=0$, the $n=0$ start, and the flattening tail where paging would bite.

---

The one-picture summary

Everything above collapses into one image: timelines on the left (each program's grey waits, and how loading more of them fills the CPU's red idle gaps) feeding the chance-square idea in the middle ($p^n$ = all overlapping) into the utilization curve on the right ($U=1-p^n$ rising toward 1).

Recall Feynman: tell the whole walkthrough like a story

One kid on a PlayStation spends most of the time reading the menu (waiting), so the console mostly sits doing nothing — that "mostly nothing" is the fraction $p$. Give the console to a second kid whenever the first is reading, and the only wasted moments are when both are reading at once. The chance both are reading is one kid's chance ($p$) times the other's ($p$), because their menus are unrelated — that's $p^2$. Add $n$ kids and the console only rests when all of them are reading: $p$ multiplied by itself $n$ times, $p^n$. So the console is doing useful work the rest of the time: $U=1-p^n$. The more kids you add, the tinier that "everyone idle" moment gets, and the busier the console — until they start fighting over memory and you'd better stop. That is why an operating system loads many jobs: to keep the one expensive CPU from ever staring at the menu alone.

Recall

What does $p$ mean? ::: The fraction of its time a program spends waiting for I/O instead of using the CPU. Why multiply the waiting fractions instead of adding them? ::: Because "all programs waiting at once" is an AND of independent events, and the chance of independent events all occurring is the product of their chances. What is the CPU utilization formula? ::: $U=1-p^n$, where $n$ is the number of loaded programs. What happens to $U$ as $n\to \infty$ (with $p<1$)? ::: $p^n\to 0$, so $U\to 1$: the CPU approaches fully busy but is capped at 100%. Why can't we just keep adding jobs forever in reality? ::: Too many jobs compete for RAM and trigger paging, which adds I/O and pushes $p$ back up (thrashing).