Mutation testing: Exercises — Coding deep dive

This is the practice arena for Mutation testing (4.5.16). Every exercise is graded L1 → L5. Read the problem, try it yourself first, then open the solution. Every number here is machine-verified.

Before we start, one quick reminder of the counts everything below leans on. We use exactly four. The picture below is the mental model for all of them — keep it in mind for every exercise:

Common mistake Edge case — what if

$M - E = 0$ (every mutant is equivalent)? Why it feels right to plug in anyway: the formula "looks" universal, so you reach for it. The flaw: $\frac{K}{0}$ is undefined — division by zero. If all mutants are equivalent, there is nothing killable, so $K = 0$ too and you get the meaningless $\frac{0}{0}$ . Fix — the single convention this page commits to: when $M - E = 0$ we report the score as N/A (no killable mutants) — one label, used everywhere on this page, never "100%". A 100% would falsely imply a strong suite, when in truth the suite was never given anything catchable to prove itself on. Practically it means your operators produced only equivalents; regenerate mutants rather than trust any number. Every exercise below is stated with $M - E > 0$ , so the formula is always valid and this N/A case never actually fires — it exists only so you know what to do if it ever does.

Definition How do we actually

know a mutant is equivalent? This is the hardest quadrant of the topic, because deciding equivalence in general is undecidable (no algorithm settles it for all programs). In practice teams use a mix of cheap heuristics before spending human time:

Trivial-algebra check: operators like a*1 → a+0, x-0 → x, not not b → b are provably equal — flag them automatically.
Compiler / bytecode compare: if the mutant compiles to identical optimised code, it is equivalent (e.g. dead-code or unreachable-branch mutations).
Wider tests / fuzzing: throw many random and boundary inputs at the survivor. If nothing ever differs after a strong search, it is probably equivalent (a heuristic, not a proof).
Human judgement, last: an engineer reads the diff and reasons about behaviour. This is why 100% score is rare — resolving each suspected equivalent by hand is expensive.

Rule of thumb: a survivor is "guilty until proven equivalent." First try to kill it with a test; only classify it as $E$ after the checks above fail to find a distinguishing input. In the figure, that means a mutant starts in the pink $S$ region and only moves to the yellow $E$ box once you've failed to find a killer input.

Level 1 — Recognition

Goal: name the pieces. No calculation harder than counting.

L1.1

Problem. Match each event to the right word AND its symbol from the four counts: (a) a test failed on the mutated program; (b) every test passed on the mutated program; (c) the mutant does the exact same thing as the original.

Recall Solution

(a) → Killed mutant — the suite noticed the injected bug. Counted in $K$ (blue box in the figure).
(b) → Survived mutant — the suite missed it (a gap in the tests, not necessarily a bug in production code). Counted in $S$ (pink box).
(c) → Equivalent mutant — no test can ever kill it, so we exclude it from the denominator as $E$ (yellow box). It is one kind of survivor, so it also sits inside $S$ — that is why it's drawn nested against the pink region. Remember the intuition: it's unkillable by definition, so it can't be an opportunity.

L1.2

Problem. A tool changes x + y into x - y. What is the name of the rule that made this change, and what is the resulting program called?

Recall Solution

The rule is a mutation operator (here, the arithmetic-swap operator + → -).
The resulting program (with that one change) is a mutant.

L1.3

Problem. A test method runs the line return age >= 18 but contains no assert at all. What is its line coverage on that line, and how many mutants of that line can it kill?

Recall Solution

Line coverage: 100% — the line executed.
Mutants killed: 0 — with no assertion, nothing can ever fail, so no mutant is caught. This is exactly the gap Code coverage hides and mutation testing exposes.

Level 2 — Application

Goal: plug numbers into $K / (M - E)$ , and hand-simulate one mutant.

L2.1

Problem. A run gives $M = 80$ mutants, $E = 5$ equivalent, $K = 60$ killed. Compute the mutation score as a percentage (1 decimal place).

Recall Solution

Killable population $= M - E = 80 - 5 = 75$ (and $> 0$ , so the formula is valid). Why carve out the 5? Those 5 are unkillable by definition — leaving them in would pretend the suite could have caught them. $Score = \frac{60}{75} = 0.8 = 80.0%$

L2.2

Problem. Same run as L2.1, but a colleague wrongly forgets to subtract the equivalents and computes $K / M$ . What (wrong) number do they get, and by how many percentage points does it under-report the true score?

Recall Solution

Wrong value: $\frac{60}{80} = 0.75 = 75.0%$ . True value (L2.1): $80.0%$ . Under-report gap $= 80.0 - 75.0 = 5.0$ percentage points. Equivalent mutants can only be killed by nobody — that's the intuition again — so including them in the denominator drags the score down for a failure that was never the suite's fault.

L2.3

Problem. Hand-simulate. Code and test:

def sign(n):
    if n >= 0:
        return "nonneg"
    return "neg"

assert sign(4)  == "nonneg"
assert sign(-2) == "neg"

Mutant: change n >= 0 → n > 0. Does it survive or get killed?

Recall Solution

Look at the two tests against the mutant n > 0 — and see the number-line figure below, where >= and > only disagree at a single boundary point:

sign(4): 4 > 0 is true → returns "nonneg" ✓ (matches).
sign(-2): -2 > 0 is false → returns "neg" ✓ (matches).

Both tests pass → mutant SURVIVES (goes into $S$ ). The only input where >= and > differ is n = 0, and no test uses it. This is a boundary gap. Adding assert sign(0) == "nonneg" kills it, because the mutant returns "neg" for 0. Since a distinguishing input exists, this survivor is killable, not equivalent — so it stays in the denominator.

Level 3 — Analysis

Goal: reason about WHY a mutant survives, and design the test that kills it.

L3.1

Problem. Code: def in_range(x): return 1 <= x <= 10. Test suite: assert in_range(5) == True, assert in_range(50) == False. For each mutant below, decide survive/killed with the given tests, and if it survives give ONE input that kills it.

(i) 1 <= x → 1 < x

(ii) x <= 10 → x < 10

Recall Solution

Tests only probe $x = 5$ (inside) and $x = 50$ (far outside). Neither touches a boundary — look at the two boundary dots ( $x = 1$ pink, $x = 10$ yellow) in the number-line figure above.

(i) 1 < x. in_range(5): 1<5 and 5<=10 → True ✓. in_range(50) → False ✓. Both pass → SURVIVES. Killer input: x = 1. Original: 1<=1 → True; mutant: 1<1 → False. Test assert in_range(1) == True fails on mutant → killed.
(ii) x < 10. in_range(5) → True ✓, in_range(50) → False ✓. SURVIVES. Killer input: x = 10. Original: 10<=10 → True; mutant: 10<10 → False. assert in_range(10) == True kills it.

Because each survivor has a distinguishing input (found by testing the boundary), both are killable survivors, not equivalents — they belong in the denominator.

L3.2

Problem. Code: def f(a): return a * 1. A tool mutates a * 1 → a + 0. The mutant survives every possible test. Classify this mutant and explain what a survivor of this type does NOT tell you.

Recall Solution

a * 1 and a + 0 both equal a for every value of a — this is exactly the trivial-algebra case the detection checklist flags automatically. The programs are behaviourally identical, so no input can ever distinguish them → this is an equivalent mutant. What it does not tell you: it does not indicate a weak test suite. An equivalent survivor is a false positive — it counts toward $E$ (excluded from the denominator, the yellow box), not toward "add a test." Here is the intuition made concrete: there is literally no killer input, so it was never an opportunity to begin with.

L3.3

Problem. You have 30 mutants. 18 are killed by your current tests. Of the 12 survivors, careful analysis shows 3 are equivalent and 9 are genuinely killable but currently missed. (a) Current score? (b) If you add tests that kill all 9 real survivors, new score? (Both as fractions and percentages.)

Recall Solution

Here $M = 30$ , $S = 12$ (of which $E = 3$ ), $K = 18$ . Killable $= M - E = 30 - 3 = 27 > 0$ . We carve out the 3 equivalents because — one more time — they can never be killed by anyone.

(a) Now: $K = 18$ . $Score = \frac{18}{27} = \frac{2}{3} \approx 66.7%$ .
(b) After killing the 9 killable survivors: $K = 18 + 9 = 27$ . $Score = \frac{27}{27} = 1 = 100%$ . Every killable mutant is now dead — and notice the 3 equivalents were never in the denominator, which is exactly why 100% is reachable here.

Level 4 — Synthesis

Goal: combine concepts — build the killing test, connect to other techniques.

L4.1

Problem. Code: def discount(price, member): return price * 0.9 if member else price. Your only test is assert discount(100, True) == 90. Write the minimum set of additional assertions that would kill each of these three mutants, and name each killing input.

(i) member → not member

(ii) price * 0.9 → price * 1.0

(iii) else price → else price * 0.9

Recall Solution

(i) not member: with the existing test discount(100, True), mutant takes the else-branch → returns 100 ≠ 90 → already killed. No new test needed.
(ii) price * 1.0: discount(100, True) mutant returns 100 ≠ 90 → already killed by the existing test.
(iii) else price * 0.9: existing test only uses member=True, never the else-branch, so it can't see this. Need a non-member test. Killer: assert discount(100, False) == 100. Original returns 100; mutant returns 90 → fails → killed.

Minimum addition: exactly one assertion, assert discount(100, False) == 100, closing the untested else-branch. This is precisely how mutation testing grades your Unit testing and complements Test-driven development.

L4.2

Problem. A team runs 500 mutants nightly in Continuous integration. Night 1: $E = 20$ , killed $= 384$ . Night 2 (after adding tests): same 500 mutants, $E = 20$ , killed $= 432$ . (a) Compute both nights' scores (1 dp). (b) How many additional killable survivors did the new tests eliminate?

Recall Solution

Killable $= 500 - 20 = 480 > 0$ (unchanged — same code, same operators, so the same 20 equivalents are carved out both nights).

Night 1: $\frac{384}{480} = 0.8 = 80.0%$ .
Night 2: $\frac{432}{480} = 0.9 = 90.0%$ .
(b) Extra killed $= 432 - 384 = 48$ survivors newly caught by the added tests.

Level 5 — Mastery

Goal: full reasoning under trade-offs, and the 80/20 judgement call.

L5.1

Problem. Two suites are tested against the same 200 mutants (with $E = 10$ ).

Suite A: kills 180, but takes 2 minutes to run the full mutation analysis.

Suite B: kills 171, and takes 40 seconds.

(a) Compute both scores (1 dp). (b) Suite A's extra 9 kills — expressed as percentage points of score — cost an extra 80 seconds. Comment on the 80/20 trade-off.

Recall Solution

Killable $= 200 - 10 = 190 > 0$ (the 10 equivalents are carved out for both suites, since they run against the same mutants).

Suite A: $\frac{180}{190} \approx 0.9474 = 94.7%$ .
Suite B: $\frac{171}{190} = 0.9 = 90.0%$ .
(b) The gap is $94.7 - 90.0 = 4.7$ percentage points for $+ 80$ seconds of runtime. Whether that's worth it depends on what those 9 mutants touch: if they sit on critical logic (payment, auth), the 4.7 points are cheap insurance; if they're low-risk edge cases, Suite B's speed may win, especially in a fast CI loop. Chasing the last few percent hits diminishing returns — the 80/20 principle from the parent note.

L5.2

Problem. A tool reports a mutation score of $\frac{47}{50} = 94%$ , treating all 3 survivors as killable. Later, review finds 2 of the 3 survivors are equivalent and 1 is a real gap. (a) Recompute the correct score. (b) You then add one test that kills the last real survivor. New correct score? (c) Explain in one sentence why the corrected score in (a) is higher than the reported 94%.

Recall Solution

Here the tool reported $K = 47$ , "survivors" $S = 3$ , and used denominator $50$ (it assumed $E = 0$ ).

(a) Truly $E = 2$ , so killable $= 50 - 2 = 48 > 0$ , and $K = 47$ : $\frac{47}{48} \approx 0.9792 = 97.9%$ .
(b) Killing the 1 real survivor: $K = 48$ , killable $= 48$ → $\frac{48}{48} = 100%$ .
(c) Removing the 2 unkillable equivalents shrinks the denominator (48 instead of 50) while $K$ stays 47, so the same successes are now measured against a fairer, smaller pool — the ratio rises. This is the intuition once more: equivalents were never opportunities, so counting them was quietly deflating the score.

L5.3

Problem. A CI pipeline enforces a quality gate: builds fail if mutation score $< 85%$ . A run produces $M = 120$ , $E = 8$ . What is the minimum number of mutants $K$ the suite must kill to pass the gate? (Give the integer.)

Recall Solution

Killable $= 120 - 8 = 112 > 0$ . Require $\frac{K}{112} \geq 0.85$ . $K \geq 0.85 \times 112 = 95.2$ Since $K$ must be a whole number of mutants, round up: $K_{m i n} = 96$ . Check: $\frac{95}{112} \approx 84.8% < 85%$ (fails), $\frac{96}{112} \approx 85.7% \geq 85%$ (passes). So 96 kills are needed. (Note: only killable mutants count in the gate — the 8 equivalents sit outside the denominator, unkillable by definition.)

How these exercises hang together

The path is one straight climb, and each level feeds the next:

L1 Recognition — name the pieces ( $M, E, K, S$ ).
L2 Application — plug them into $K / (M - E)$ and hand-simulate one mutant.
L3 Analysis — reason about why a mutant survives (boundaries) and whether it is killable or equivalent.
L4 Synthesis — combine ideas: reuse one test to kill several mutants; connect to Unit testing and Test-driven development.
L5 Mastery — weigh trade-offs (runtime vs score, Continuous integration gates, the $M - E = 0$ → N/A edge case).

One intuition runs underneath the whole ladder and we re-earned it at every level: equivalent mutants are unkillable by definition, so they leave the denominator. Two prerequisite threads support the climb: Boundary value analysis explains almost every survivor you met (the > vs >= gaps), and detecting equivalent mutants is the hard, sometimes-undecidable quadrant handled by the checklist near the top.

Connections

Mutation testing — the parent topic these exercises drill.
Unit testing — the tests being graded in L4.1.
Code coverage — the weaker metric exposed in L1.3.
Boundary value analysis — why L2.3 and L3.1 survivors appear.
Test-driven development — the discipline mutation testing audits.
Continuous integration — where the L4.2 / L5.3 gates run.
Fault injection — the broader "break it on purpose" family.