4.5.17 · D1Software Engineering

Foundations — Property-based testing

3,788 words17 min readBack to topic

This page assumes nothing. If the parent note used a symbol like , a term like invariant, an operator like , or notation like @given, we build it here from the ground up, in the order that each piece needs the one before it.


0. The picture we keep coming back to

Before any symbol, hold this image: a box (your function) with an input arrow going in and an output arrow coming out. A normal test drops one specific ball in and checks one specific ball comes out. Property-based testing dumps a whole bucket of random balls in and checks a rule about what comes out — never a single expected ball.

Figure — Property-based testing

Keep this box in mind; every symbol below is a label on some part of this picture.


1. Function — the box itself

Picture: the box in the figure above. The input arrow is what you feed it; the output arrow is what it hands back.

Why the topic needs it: everything we test is a function — reverse, sort, average. A property is a statement about the box's behaviour, so we must first name the box.

The idea of a box whose output depends only on its input (no hidden surprises) is important enough to have its own vault note — see Pure functions. PBT works best on pure functions because the same input must always give the same output (we return to this in §9).


1b. Domain, image, codomain, and "valid input"

Before we talk about any input, we must pin down which inputs the box is even allowed to accept — that word "valid" will appear on almost every line below.

Picture: left pile = domain (allowed inputs). Right big bin = codomain (declared possible outputs). The shaded blob inside that bin = image (outputs that really occur).

Why the topic needs it: every property and every generator quietly says "for all valid inputs." Now "valid" has a precise meaning: inside the domain / satisfying the preconditions. This is exactly why §10's empty-list bug is really a domain question — was the empty list ever a valid input at all? And it's why generators must emit domain-respecting data (the parent's "generate valid inputs only" warning).


2. The symbols , , and "composition"

Why we need it: a property must be true for any valid input, so we cannot name a specific one. is how we say "for a general, unnamed thing from the domain."

Picture: two boxes chained — the output arrow of becomes the input arrow of (middle panel of the figure).

Why the topic needs it: the parent's very first property, reverse(reverse(xs)) == xs, is exactly . And the round-trip pattern is composition where undoes . You cannot read those without knowing that nested parentheses mean "chain the boxes."


3. The symbol inside a property (it is a claim, not an order)

Picture: a balance scale. Left pan holds , right pan holds . The property asserts the scale stays level for every ball you weigh.

Why the topic needs it: in code we write this as assert left == right. The word assert means "I claim this is true; blow up loudly if it isn't." That is the beating heart of every test.


3b. Comparison relations , , — "which is bigger?"

Picture: a horizontal number line. Smaller numbers are to the left, bigger to the right. means the dot for never sits to the right of the dot for .

Why the topic needs it: many properties are not equalities but bounds — "the answer must stay between two values." The parent's average property, , is a double comparison: it reads "min is average, and average is max." Without you can't state a "sits between" rule.


4. The symbol — "for all"

Picture: not one ball on the scale — an infinite bucket of balls (all from the domain), and the promise "no matter which one, the scale balances."

Why the topic needs it: this single symbol is the difference between the two styles of testing. Example-based testing checks one ball; property-based testing makes a claim. The framework can't literally try infinitely many, so it samples — which is exactly why §8's "not a proof" warning exists.


5. Invariant — the quantity that never changes

Picture: shuffle a deck of cards — the order changes, but the count (52) never does. The count is invariant under shuffling.

Why the topic needs it: the parent's property says "sorting reorders but the length is invariant." Most good properties are just "spot the thing that shouldn't change." The deep connection between invariants, contracts, and tests is its own topic — see Invariants and contracts.


6. Inverse — the box that undoes another box

Picture: = "wrap a gift", = "unwrap it." Wrap then unwrap = the original gift. The figure shows the round-trip loop returning to start.

Why the topic needs it: the round-trip property pattern is the inverse idea: decode(encode(x)) == x. Encode and decode are inverses by definition, so this equality is a mathematical certainty you can safely test — provided encode is bijective (loses no information).


7. Idempotence — doing it twice = doing it once

Picture: pressing a ground-floor elevator button. Press once, you go to the ground floor. Press it again while already there — nothing new happens.

Why the topic needs it: sort, dedupe, normalize are idempotent — sorting an already-sorted list leaves it sorted. Note this is different from round-trip: idempotence feeds the output back into the same box; round-trip uses a different, undoing box.


8. Sampling vs. proof — why "for all" is only tested, not proven

Picture: back to the infinite bucket of balls from §4 — the robot scoops out a few hundred handfuls, never the whole (infinite) bucket.

Why the topic needs it: this is the parent's third "steel-manned mistake." Understanding (a claim over all valid inputs) versus random sampling (a check over some inputs) is exactly why you still keep a few example tests for known tricky values.


8b. Shrinking — the machine hands you the smallest broken ball

Picture: a messy giant sock that fails the rule; the robot swaps it for smaller and smaller socks that still fail, and shows you only the tiniest broken one.

Why the topic needs it: shrinking is what makes a random failure debuggable. It is the parent note's "killer feature" — and it only works if your function is deterministic (§9), so the framework can replay the same input and trust that "still fails" means the same thing every time.


9. Determinism — same ball, same result, every time

Picture: a vending machine where button B4 always drops the same snack. A non-deterministic machine would sometimes drop chips, sometimes chocolate — you couldn't reproduce a failure.

Why the topic needs it: if your box secretly uses randomness, a failing test can't be replayed, so shrinking (§8b — finding the smallest failing ball) breaks down: "does this simpler input still fail?" has no stable answer. This is the parent's "flaky tests" warning, and the reason Pure functions pair so well with PBT.


9b. @given — the notation that turns a into hundreds of runs


10. Putting the vocabulary to work — a tiny read-through


The prerequisite map

Function - the box

Domain and valid input

Composition g of f of x

Variable x - any input

For all quantifier

Inverse g undoes f

Idempotence f f x = f x

Bijective one to one and onto

Round-trip property

Idempotence property

Invariant

Invariant property

Sampling not proof

Determinism

Shrinking

Reproducible counterexample

given decorator runs the for all

Property-based testing

Read it bottom-up: the box, the variable, and "for all" are the roots. They grow into the three property patterns, the shrinking machinery, and the honesty about sampling — and all of them feed the parent topic Property-based testing.


Where these lead next

  • Unit testing — the "one ball, one expected result" style PBT extends.
  • Test-driven development — writing the rule before the code.
  • Invariants and contracts — the formal home of §5's invariants.
  • Fuzzing — random-input generation without the structured rule.
  • Hypothesis (Python) and QuickCheck (Haskell) — the tools that turn a claim into hundreds of runs.
  • Pure functions — why §1 and §9 make a box safe to test this way.

Equipment checklist

Self-test — cover the right side and answer before revealing.

What does mean out loud?
" of " — the output the box gives for input .
What is the domain of a function?
The set of all inputs it is allowed to accept.
What is the codomain, and how does it differ from the image?
Codomain = the declared set of possible outputs (the labelled bin); image = the outputs that actually occur, which sits inside the codomain and may be smaller.
What makes an input "valid"?
It lives in the function's domain / satisfies its preconditions (e.g. average needs a non-empty list).
What does the arrow mean in "same input same output"?
Plain English "leads to / therefore" — a reading aid, not special function machinery.
In , which box runs first?
(the innermost parentheses) runs first, then .
Read in words and as a number-line picture.
" is less than or equal to " — 's dot never sits to the right of 's dot.
What does the double in assert?
Avg sits between min and max (min avg AND avg max).
What plain words does stand for?
"For all" / "for every."
A property written with no shown — is the "for all" still there?
Yes — every property carries an invisible ""; "for every valid input" is the whole point.
What do injective, surjective, and bijective mean?
Injective = distinct inputs give distinct outputs (no collisions); surjective = every codomain value is hit (no gaps); bijective = both, a perfect pairing.
Difference between round-trip and idempotence?
Round-trip uses an inverse to return home; idempotence reapplies the same box and stays put.
When can a function have an inverse ?
Only when is bijective (injective and surjective) — no collisions, no gaps — so the undo is unambiguous.
What is an invariant, in one sentence?
A fact that stays true no matter the input or internal steps (e.g. length is unchanged by sorting).
What is shrinking?
Automatically simplifying a failing input to the smallest one that still breaks the rule, for easy debugging.
Why is passing 1000 random cases not a proof?
It samples finitely many of infinitely many valid inputs — high confidence, not certainty.
Why must a tested function be deterministic?
So a failing input reproduces the same failure, letting shrinking find the smallest breaking case.
What does the @given(...) decorator do?
Runs the test many times, each with a fresh random input from the strategy — the executable form of .
What single symbol separates example-based from property-based testing?
— "for all valid inputs" instead of "for this one input."