Foundations — Testing — unit, integration, system, acceptance, smoke, regression

Before we can talk about kinds of tests, we must be sure every word and symbol the parent note leaned on is built from the ground up. If you have never written a line of code, start here and nothing later will surprise you.

1. What is a "function"? (the atom of everything)

Picture a vending machine: you press B4 (input), a chocolate bar drops (output). The machine does not care who pressed the button — the rule is fixed.

In code we write a function like this:

def is_even(n):
    ...  # rule goes here; we build it in section 2

• def means "define a machine".
• is_even is the machine's name.
• n is the input slot (called a parameter).
• return ... is what drops out of the machine.

We have deliberately left the rule blank for now: it uses two symbols we have not earned yet. We fill it in the moment they are defined.

Why the topic needs this: the smallest thing you can test in isolation is one function. That is exactly what a unit is.

2. The % symbol and == — reading a condition

To finish our machine we need a rule for "is this number even?". Two symbols do the work; both must be earned before we write a single line using them.

Think of sorting 7 sweets into bags of 2: you fill 3 bags and have 1 sweet left over. So 7 % 2 is 1. If nothing is left over, the number divided evenly.

• 4 % 2 → 0 (even, nothing left over)
• 3 % 2 → 1 (odd, one left over)

Now — and only now — we can fill in the machine's rule from §1:

def is_even(n):
    return n % 2 == 0

It reads: "is the remainder zero?" — answer True for even, False for odd. Every symbol here has now been defined.

3. True / False — the Boolean, a light switch

The picture is a light switch: on or off. Every test ultimately collapses the world into one Boolean — did the code do the right thing, yes or no?

Why the topic needs this: a test's job is to produce ONE Boolean: pass or fail. Green or red. This is the "shout" from the intuition callout.

4. assert — the shout

If the condition is True, nothing happens and life goes on. If it is False, the test fails loudly.

def test_is_even():
    assert is_even(4) == True    # claim: 4 is even
    assert is_even(3) == False   # claim: 3 is odd

• The input we chose is 4 (and 3).
• The expected output we wrote by hand is True (and False).
• assert compares the machine's real answer to our expected answer.

Why the topic needs this: every one of the six test types — unit, integration, system, acceptance, smoke, regression — is built from this same assert shout. They differ only in what code they point it at.

5. Dependencies, and the words "mock / stub / fake"

Most real functions do not live alone. A "save user" function needs a database. A weather app needs the network. These helpers a function leans on are its dependencies.

The picture: a stunt double stands in for the actor in a dangerous scene. It looks the same on camera but is not the real star.

Why the topic needs this: a unit test replaces every dependency with a stand-in so that a failure means this function is broken, not the database. An integration test deliberately keeps the real dependency so it can test the seam between them. That single choice — real or stand-in — is the whole difference. See Mocks Stubs and Fakes for the full menu.

6. The "seam" — where two parts meet

Picture two Lego bricks: each is perfect alone, but if the studs do not line up they will not click. The seam is that clicking surface. Bugs love seams: unit A sends a date as "2024-01-31", unit B expects 31/01/2024, both are "correct" alone, together they crash.

Why the topic needs this: integration testing exists only because seams exist. If code had no seams, unit tests would be enough.

7. The stack of parts — from atom to whole tower

Now we can stack the ideas. Each layer is bigger than the one below:

Why the topic needs this: the six test types are just this ladder, each rung checking a bigger slice.

8. The two remaining types — smoke and regression (across time)

The four types in §7 differ by how big a slice they watch. The last two differ by when you run them and why, not by size.

Picture flicking the light switch when you enter a room: if no light comes on, you do not start rearranging the furniture — you fix the power first. The name comes from hardware: plug the board in; if smoke pours out, stop testing.

Picture repainting one wall of a house and then walking every other room to check you did not accidentally splash them. The change may have touched shared code, so you re-check the old rooms.

Both are still just assert shouts (§4) — the novelty is purely timing and intent.

9. Probability — reading $p$, $n$, and $p^n$

The parent's reliability formula uses two symbols and an exponent. Earn them now.

Why a product and not a sum? If test 1 must be trustworthy AND test 2 AND test 3, and each is independent, the chance they are all trustworthy is the chances multiplied:

$P_{\text{trust}}=\underset{n\text{ times}}{\underbrace{p\cdot p\cdots p}}=p^n$

Multiplying numbers below $1$ makes them shrink fast — which is the whole warning: many flaky tests destroy trust. The same "multiply per step" logic drives the cost formula $\text{Cost}=c\cdot k^n$, where $k$ is the per-stage multiplier (roughly $10$) and $c$ is the starting cost. See Defect Cost of Delay.

The figure below plots $p^n$ as $n$ grows, for three values of $p$. Read it like this: the horizontal axis is $n$ (how many tests), the vertical axis is $P_{\text{trust}}$ (how sure you are they are all fine). Follow the magenta curve ($p=0.999$, very reliable tests): even it sags below the dashed "50% trust" line once you pile up enough tests. The violet ($p=0.99$) and orange ($p=0.90$) curves collapse far sooner — a flaky test suite becomes worthless quickly. This is the whole reason the pyramid keeps the count $n$ of slow, flaky, high-level tests small.

Prerequisite map

The diagram below is a dependency map. Read each arrow → as "feeds into" or "is needed before": the box the arrow leaves must be understood before the box it points to. Every box is one foundation from this page; follow the arrows upward and they all pour into the Testing topic at the bottom.

This map feeds the parent note: the Testing topic. From here you are ready for Verification and Validation, Test-Driven Development (TDD), Code Coverage, Continuous Integration, and Behaviour-Driven Development (BDD).

Equipment checklist

Test yourself — you are ready when you can answer each without peeking.