4.5.13 · D5Software Engineering
Question bank — Testing — unit, integration, system, acceptance, smoke, regression
True or false — justify
A unit test that talks to a real database is still a unit test.
False — the moment it touches a real collaborator across a boundary it becomes an integration test; a unit test isolates the piece with mocks/stubs.
Passing every system test guarantees you shipped what the customer wanted.
False — system test is verification ("built right, meets spec"); you can perfectly meet a wrong spec, which only acceptance testing (validation) catches.
A smoke test is just a small regression test.
False — smoke is breadth-first and shallow, run first to gate the expensive suite; regression is run after a change to protect existing behaviour. Same mechanism, opposite purpose.
100% code coverage means the code is fully tested.
False — coverage only proves lines ran, not that outcomes were asserted or that edge inputs were tried; you can execute a line and check nothing meaningful about it.
If all my unit tests pass, integration tests are unnecessary.
False — units can each be correct yet break at the seam (wrong data format, wrong API contract); the pyramid keeps integration tests fewer, not zero.
Acceptance tests and system tests are the same because both run the whole app end-to-end.
False — both are black-box, but system asks "does it work correctly?" and acceptance asks "is it the right product for the user's need?" — different questions, different pass criteria.
More end-to-end tests always give more confidence than more unit tests.
False — end-to-end tests are slow and flaky; with , piling up unreliable high-level tests raises the chance of a false red, eroding trust.
Regression testing means writing brand-new tests for the bug you just fixed.
Partly false — you add a test for the fix, but regression specifically means re-running the existing suite to ensure the fix didn't break previously-working features.
A test that sometimes passes and sometimes fails with no code change is fine if it's "usually green".
False — that's a flaky test; each flaky test drops suite reliability , and collapses fast, training the team to ignore red (see Continuous Integration).
Spot the error
"I mocked the database in my DB integration test to make it fast and stable."
The error: mocking the real collaborator removes the seam you were meant to test — it degenerates into a unit test and never exercises the SQL/schema/serialization the integration test exists for.
"Our CI runs system tests first because they catch the most bugs."
The error: cheapest and fastest checks must run first (smoke → unit → integration → system); running the 20-minute system suite before a 30-second smoke wastes time when a trivially broken build could have failed in seconds.
"We hit 100% coverage, so we removed the acceptance stage."
The error: coverage is a code-execution metric answering verification; acceptance answers validation (did we build the right thing) and cannot be replaced by any line-count number.
"This unit test calls three real modules to be realistic."
The error: realism at the cost of isolation means a failure no longer pinpoints one unit — that's an integration test wearing a unit-test label, and it will give vague failures.
"A failing acceptance test proves our code has a bug."
The error: acceptance failure can mean the code is correct but the requirement was wrong or misunderstood — it's a validation signal about what was built, not necessarily a defect in how.
"Smoke passed, so we can skip the rest of the suite and release."
The error: smoke only proves the build isn't fundamentally dead ("does it turn on?"); it is a gate, not proof of correctness — the deep suite still has to run.
Why questions
Why put many tests at the bottom of the pyramid and few at the top?
Lower tests are fast, deterministic, and pinpoint the faulty function; higher tests are slow, flaky, and vague — so effort concentrates where each test is cheapest and most diagnostic.
Why does a bug get roughly 10× more expensive per stage it escapes?
Each later stage adds people, rework, and lost trust, multiplying cost by ; escaping stages gives , so over 3 stages is ~1000× (the cost-of-delay argument).
Why must integration tests deliberately use real collaborators at the boundary?
Because the whole point is the seam — data formats, API calls, schema — and a mock reproduces your assumption about the boundary, not its actual behaviour.
Why do BDD "Given–When–Then" specs live at the acceptance level?
They express the business need in stakeholder language, making them validation checks the customer can read and approve — exactly what acceptance testing verifies.
Why run smoke tests before the full suite instead of after?
To fail-fast and gate: there's no value in a 2-hour run if login crashes on startup, so a 30-second breadth check saves the whole pipeline's time.
Why does writing tests first (TDD) tend to produce well-isolated units?
Writing the test first forces you to design a small, callable, dependency-injectable unit — testability pressure pushes the design toward isolation.
Why is a warning against many flaky tests?
Independent reliabilities multiply, so even across tests gives ~0.61 trust — a 39% false-red rate — showing flakiness compounds catastrophically.
Edge cases
What kind of test is assert is_even(0) == True?
A unit test on a boundary/edge input (zero) — no dependency, pure logic, checking the degenerate value that off-by-one bugs love.
If you have exactly one function and no dependencies at all, do you still need integration tests?
No — with no seams there is nothing to integrate; integration only matters once two or more units meet at a boundary.
A test suite with tests — what is its trust probability?
Trivially , but that "certainty" is meaningless: no tests means no evidence, showing the formula only measures false reds, not actual coverage.
Can a system pass all tests and still be the wrong product?
Yes — if every test encodes a mistaken requirement, verification is green while validation fails; this is the core V&V distinction.
What happens to the cost-of-delay formula when (bug caught immediately)?
— the base cost with zero multiplier, the cheapest possible catch, which is exactly why unit tests sit at the pyramid's base.
If a regression test fails after a "fix", which is more likely wrong — the old feature or the new change?
The new change — the old feature was passing before, so a fresh red usually means your change had an unintended side effect (e.g. shared rounding logic breaking discounts).
Is a manual UAT sign-off a "test" in the automated-pyramid sense?
It's an acceptance/validation activity that often sits outside the automated pyramid — slow, human, and judgement-based — which is why it runs last and rarely, not in fast CI loops.
Recall Self-check before you leave
Cover the answers. If you can justify each in one sentence without saying a bare yes/no, you own this topic. The three deadliest confusions to re-drill: Verification vs Validation — which test, which question? ::: System = verification ("built right?"); Acceptance = validation ("right thing built?"). Smoke vs Regression — what's the difference in purpose? ::: Smoke gates the suite up front (breadth, shallow); regression protects existing behaviour after a change. Why never mock the boundary in an integration test? ::: You'd test your assumption instead of the real seam, collapsing it into a unit test.