Testing — unit, integration, system, acceptance, smoke, regression: Visual walkthrough

We will earn every symbol before we use it. By the end you will be able to explain, from a blank page, why smart teams pour effort into tiny cheap tests instead of a few grand ones.

Step 1 — What is a "stage"? Draw the assembly line

WHAT we do here: lay out the stages left-to-right as a conveyor belt, so "escaping a stage" becomes a concrete picture — the part rolling one station further before anyone inspects it.

WHY start here: the whole result depends on counting how many stations a bug rides through before it is caught. We can't count until we can see the stations.

PICTURE: the belt below has four stations. A defective gear (coral) sits at the first station. Each arrow to the right is one escape.

Figure — Testing — unit, integration, system, acceptance, smoke, regression

The number we will care about is: how many arrows did the gear cross before someone pulled it off the belt? Call that number $n$ .

Step 2 — What does one escape cost? The multiply, not the add

WHAT we do: attach a starting price to the freshly-born bug and ask what one escape does to that price.

WHY multiply and not add? If escaping added a flat $\$ 5$ each time, cost would be a gentle straight line. But real re-work compounds: the later stage contains everything the earlier one did, plus its own overhead — so each step multiplies the previous cost by roughly the same factor. Multiplying-by-a-constant-each-step is exactly what we draw next.

PICTURE: two ladders side by side — the left one adds a fixed rung height (a straight climb), the right one multiplies (each rung taller than the last). Real bug cost is the right ladder.

Step 3 — Build the cost after each escape, one station at a time

WHAT we do: apply the multiply rule step by step and record the running cost. This is the heart of the derivation.

WHY step by step: we want the reader to see the pattern form, not be handed a formula. Each line below is "take the previous cost, multiply by $k$ ."

PICTURE: the bug rides the belt; above each station we stack the cost as a growing bar (mint → butter → coral), each bar exactly $k$ times taller than the one before.

Now read the growing cost, station by station:

$born, n = 0 c \times k n = 1 c \cdot k \times k n = 2 c \cdot k \cdot k = c k^{2} \times k n = 3 c k^{2} \cdot k = c k^{3}$

Annotating the last term $c k^{3}$ :

$c$ — the original tiny birth-cost, still sitting there as the base.
$k^{3}$ — the multiplier applied three times, once for each of the 3 stations the bug crossed.
The exponent is literally $n$ , the arrow-count from Step 1. That is the whole trick.

Step 4 — Generalise: the formula falls out by induction

WHY it's a proof, not a guess: each transition is defined as "multiply by $k$ ." Start at $c$ . Assume after $n$ stages the cost is $c k^{n}$ (this is true for $n = 0, 1, 2, 3$ above). Then one more escape multiplies once more: $c k^{n} \cdot k = c k^{n + 1}$ — which is the same rule with $n + 1$ . Since it holds for the next $n$ whenever it holds for the current $n$ , it holds for all $n$ . That's induction.

PICTURE: the same bars from Step 3, now plotted as a smooth exponential curve — flat and cheap on the left, rocketing upward on the right. With $k = 10$ , three escapes = $1 0^{3} = 1000 \times$ the birth-cost.

Step 5 — The degenerate cases (don't skip these)

Case $n = 0$ (caught instantly): $Cost (0) = c \cdot k^{0} = c \cdot 1 = c$ . Good — a bug caught the moment it's written costs exactly its birth-price, nothing extra. The formula didn't accidentally inflate it.

Case $k = 1$ (no penalty per stage): $Cost (n) = c \cdot 1^{n} = c$ for every $n$ . This is the imaginary world where escaping a stage costs nothing extra — the curve goes flat. If that were reality, testing early wouldn't matter. The pyramid exists precisely because $k > 1$ .

Case "add instead of multiply" (the wrong model): if cost merely added $c$ each stage, $Cost (n) = c (n + 1)$ — a straight line, only $4 c$ after 3 stages, not $1000 c$ . Comparing the line to the curve shows why the multiply model is the scary (and realistic) one.

PICTURE: all three plotted together — the flat line ( $k = 1$ ), the gentle straight line (additive), and the exploding curve ( $k = 10$ ). One glance shows which world we live in.

Step 6 — Turn the curve into the pyramid shape

WHAT we do: map each test type to its $n$ (how far up the belt it catches bugs) and to its cost per test. Then let the number of tests you should write be inversely tied to cost.

WHY this produces a pyramid: cheap-to-catch ( $n = 0$ ) tests let you afford many; expensive-to-catch (high $n$ ) tests you can only afford few. Wide base, narrow top — the shape isn't a decoration, it's the cost curve flipped on its side.

PICTURE: the exponential cost curve on the left; its mirror — the classic test pyramid — on the right, wide unit base in mint, thin system/acceptance cap in coral.

Step 7 — Bonus: why the top of the pyramid must stay tiny (flakiness)

WHY multiply the probabilities? If the tests are independent — one's outcome tells you nothing about another's — then "all correct" means this one right AND that one right AND …. Independent ANDs multiply. So:

$P_{trust} = n times p \times p \times \dots \times p = p^{n}$

$p$ — per-test reliability, sitting under the product.
$n$ — how many times we multiply, i.e. suite size.
$p^{n}$ — shrinks fast because $p < 1$ (multiplying by less-than-one repeatedly heads toward zero).

PICTURE: $P_{trust} = p^{n}$ plotted against $n$ for a flaky $p = 0.999$ — it slides from near-1 down through the $0.61$ danger zone by $n = 500$ .

The one-picture summary

Everything above, compressed: the bug rides the belt (top), the cost explodes as $c k^{n}$ (middle-left), the shape flips into the pyramid (middle-right), and flakiness $p^{n}$ warns you to keep the top thin (bottom).

Recall Feynman retelling — say it back in plain words

Imagine a broken gear on a conveyor belt. Every station it slides past before someone spots it makes the fix about ten times harder, because now more people and more re-joined parts are involved. So the price isn't "a little more each time" — it multiplies: one dollar, then ten, then a hundred, then a thousand after three stations. Written down, that's $c k^{n}$ : the seed price $c$ , multiplied by the ten-fold factor $k$ once for each station $n$ it escaped. Because catching a bug at the very first station (the "unit" station) is dirt cheap, and catching it at the last station is brutally expensive, you naturally want tons of cheap early checks and only a few expensive late ones. Draw that as a stack and you get a wide-bottomed pyramid — not because someone liked triangles, but because that's the cost curve tipped on its side. And there's a second warning: big end-to-end tests occasionally lie (they're flaky). If each is right $99.9%$ of the time, running $500$ of them means the whole suite is trustworthy only $61%$ of the time, because you're multiplying $0.999$ by itself $500$ times. So keep the flaky ones few — another reason the pyramid stays skinny on top.

Recall Quick self-check

Cost of a bug caught with 0 escapes? ::: $c \cdot k^{0} = c$ — just the birth price. Why multiply by $k$ per stage instead of adding? ::: Re-work compounds; each later stage contains all earlier work plus new overhead. With $c=\$ 1,k=10 $, cos t a f t er 3 esc a p es ? :::$ $1\cdot10^{3}=$1000 $. I f$ k=1 $, w ha t ha pp e n s t o t h ec u r v e ? ::: I t g oes f l a t — n oe a r l y - c a t c ha d v an t a g e; t h e p y r ami d l oses i t sr e a so n t oe x i s t . S u i t e t r u s t f or$ n=500 $t es t se a c h$ p=0.999 $? :::$ 0.999^{500}\approx 0.61 $. W h y d oes t h ecos t c u r v e * b eco m e * t h e p y r ami d ? ::: C h e a pl o w -$ n $c a t c h es \to a f f or d man y; e x p e n s i v e hi g h -$ n$ catches → afford few. Wide base, thin top.

Prerequisites & neighbours: Defect Cost of Delay · Code Coverage · Continuous Integration · Verification and Validation · Mocks Stubs and Fakes · Test-Driven Development (TDD) · Behaviour-Driven Development (BDD) · back to the parent topic.