5.5.20 · D5Embedded Systems & Real-Time Software
Question bank — Software testing in embedded — unit tests on host, HIL testing
Two words we lean on constantly, defined once so no line assumes them:
Before the traps, every acronym this page uses — expanded once, in plain words:
The one picture that ties host, target, and HIL together — refer back to it as you work through the traps:
And the timing idea the edge-case section leans on, drawn once so the symbols are earned:
True or false — justify
A host unit test can prove your firmware meets its 100μs interrupt deadline.
False — the host mocks time, so it measures the host's CPU speed, not the target's. Deadlines are only real on real hardware, which is HIL's (Hardware-in-the-Loop) job.
If a function reads a hardware register directly, it can still be unit-tested on host as-is.
False — the register address means nothing on x86, so it crashes or reads garbage. You must route it through a HAL call you can mock first.
Passing every host unit test means the firmware will work on the board.
False — host tests verify logic, not timing, ISR (Interrupt Service Routine) latency, ADC (Analog-to-Digital Converter) noise, or peripheral interaction. Green host tests + broken HIL is a completely normal state.
HIL testing makes host unit testing unnecessary.
False — HIL is slow, expensive, and hard to debug; host tests catch logic bugs in milliseconds. The two tiers cover different failure classes, not the same one twice.
Mocking a HAL function changes the behaviour of the logic under test.
False — a good mock only substitutes the hardware boundary; the pure logic (like
compute_pwm) runs unchanged. If mocking changes your logic's behaviour, your seam is in the wrong place.Achieving 100% host coverage is the goal for embedded firmware.
False — vendor drivers and thin HAL glue can't meaningfully run on host, so host coverage is naturally capped below 100%. The rough rule of thumb is to cover most of your custom logic, but the exact figure depends heavily on how much peripheral code you have — don't treat any single percentage as sacred.
A HIL test that passes once is a reliable pass.
False — real hardware has noise, timing jitter, and race conditions, so intermittent failures are the whole reason HIL exists. A flaky HIL result is a signal, not something to retry away.
Fault injection can be done in host unit tests too, not only in HIL.
True — you can make a mock return an error code or corrupt data to test error paths cheaply. Fault Injection Testing on host covers logic; HIL adds electrical faults host can't produce.
printf-style debugging is a valid substitute for a proper host test suite.
False — prints are manual, non-repeatable, and prove nothing automatically; a test suite asserts expected outcomes and re-runs in CI forever.
Peripheral and HAL glue code never needs testing because it's "just plumbing".
False — HAL glue and driver setup can hide subtle register-config and initialization bugs. Those are precisely the bugs HIL (on real hardware) is meant to catch, so "under-test the logic and under-test the glue" is a double mistake.
Spot the error
"We compiled the firmware for the target and ran it under our host unit-test runner."
You cannot execute target machine code (Cortex-M) on an x86 host. Host tests must be recompiled for the host with mocked HALs — a different build, not the same binary.
"Our mock returns a fixed ADC value, so we're confident the driver reads the ADC correctly."
The mock replaces the driver, so it tests nothing about the real driver. Only HIL (real ADC — Analog-to-Digital Converter — plus a known input voltage) can verify the actual conversion.
"The response-time test uses time.sleep() on the host to simulate the deadline."
sleep on the host measures the host's scheduler, not the target's real-time response. Timing must be measured on hardware with a logic analyzer or on-target timestamps."We put the PWM cap logic inside hal_pwm_set() so it's guaranteed applied."
That buries testable logic inside the un-mockable hardware layer, so host tests can't reach the cap. Keep the cap in pure logic; the HAL should only touch the register.
"Our HIL test writes to the DUT's internal variables to set up the scenario."
HIL can only interact with the DUT (Device Under Test) through real I/O — pins, buses, signals — not internal memory. Reaching into variables is a host-test technique; HIL must stimulate from outside the loop.
"We skipped host tests because our RTOS scheduler makes behaviour deterministic anyway."
An RTOS gives timing determinism, not logic correctness; a wrong formula is still wrong on schedule. Host tests catch the logic the scheduler faithfully runs.
"To speed up CI, we run all HIL tests on every commit and no host tests."
This is backwards — HIL is the slow, scarce resource you gate carefully, while host tests are the fast front line. CI should run host tests on every commit and HIL on a subset/nightly.
Why questions
Why route every hardware access through hal_* functions instead of calling registers directly?
Because a named seam lets you swap implementations: real registers on target, mocks on host. Without the seam there's no place to inject test inputs.
Why do embedded projects need two separate build targets?
One binary must run on the ARM MCU (cross-compiler, real HAL,
main), the other on the host (native gcc, mock HAL, test-framework entry). Same source, different compiler/HAL/entry point.Why measure interrupt latency with a logic analyzer instead of a printf timestamp?
A
printf itself takes far longer than the deadline and perturbs timing, so it hides the very thing you measure. A logic analyzer watches a GPIO edge non-intrusively at microsecond resolution.Why is the "80% of bugs in 20% of code" idea used to justify heavy host testing?
The complex logic (filters, state machines, protocol parsing) is your custom 20% and where bugs concentrate; it also happens to be hardware-free and thus host-testable. So you aim your cheapest tests exactly where risk is highest — but that's a heuristic, not a licence to skip the rest.
Why can host tests run in microseconds when HIL takes seconds per test?
Host code runs on a GHz desktop CPU with no flashing, no settling delays, and no real clock to wait on. HIL must flash firmware, let analog signals settle, and wait for actual real-time durations.
Why does HIL belong later in the pipeline rather than as the first thing you run?
It's slow, needs scarce rig hardware, and its failures are hard to debug without a scope. You want cheap host tests to have already eliminated logic bugs before spending the expensive HIL slot.
Edge cases
If a logic function has no hardware dependency at all, does it still need a HAL?
No — pure computation (a PID step, a checksum) is directly host-testable with plain inputs. Adding a HAL there is needless indirection.
What happens to a host test if you forget to link the mock HAL implementation?
The linker fails on the undefined
hal_* symbol, or worse, pulls in the real target HAL that won't run on host. The missing mock is the whole test seam, so the build simply can't complete correctly.A HIL test feeds 0V to a "throttle" ADC input — what boundary does this probe?
The zero/idle degenerate case, checking that PWM drops to its floor and no division-by-zero or underflow occurs. Zero and full-scale inputs are exactly where mapping bugs and clamps show up.
Can host tests catch integral windup in a PID controller?
Yes — windup is pure math accumulating over simulated steps, fully reproducible on host with no hardware. That's a textbook case for cheap host testing, as the parent example shows.
What if the target's ADC returns noisy values but the mock always returns a clean number?
Host tests will pass while the real system misbehaves, because the mock never reproduces noise. Only HIL feeding a real (or deliberately noisy) signal exposes filtering weaknesses.
At the exact deadline boundary — response measured at precisely — does it pass?
By the strict rule , landing exactly on the deadline fails. Real-time safety uses a strict inequality so jitter can't push you over.
If firmware works in the debugger but fails when running free (no breakpoints), what tier catches it?
HIL — the debugger alters timing and can mask race conditions that only appear at full speed. Host tests can't see it either since they don't model real timing.