Software testing in embedded — unit tests on host, HIL testing: Question bank

5.5.20 · D5 · Coding › Embedded Systems & Real-Time Software › Software testing in embedded — unit tests on host, HIL testi

Do words jinpe hum baar baar depend karte hain, ek baar define kar lete hain taaki koi bhi line inhe assume na kare:

Traps se pehle, is page mein use hone wale saare acronyms — ek baar plain words mein expand kiye gaye:

Definition Char acronyms jo aapko pehle jaanne chahiye

HIL = Hardware-in-the-Loop: firmware real target board par run hota hai jabki uske bahar ek test rig sensors ka role nibhata hai aur outputs read karta hai. Board test "loop" ke andar baitha hota hai.
DUT = Device Under Test: wo specific board jise aap test kar rahe ho — aapka firmware real silicon par run ho raha hai. HIL rig mein, DUT wo cheez hai beech mein jise baaki sab kuch poke aur measure karta hai.
ISR = Interrupt Service Routine: wo chhota function jis par chip turant jump karta hai jab koi hardware event fire hota hai (ek byte aata hai, ek timer expire hota hai). "ISR latency" matlab event aur routine ke actually start hone ke beech kitna time lagta hai.
ADC = Analog-to-Digital Converter: wo peripheral jo real-world voltage (jaise ek throttle at 1.65 V) ko ek aisa number banata hai jo code padh sake (jaise 2048). "ADC padhna" matlab wo converted number grab karna.

Wo ek picture jo host, target, aur HIL ko saath jorti hai — traps solve karte waqt isse refer karte rehna:

Figure — Software testing in embedded — unit tests on host, HIL testing

Aur timing ka wo idea jis par edge-case section depend karta hai, ek baar draw kar lete hain taaki symbols earn ho jayein:

True or false — justify

A host unit test can prove your firmware meets its 100μs interrupt deadline.

False — host time ko mock karta hai, isliye ye target ki CPU speed nahi balki host ki CPU speed measure karta hai. Deadlines sirf real hardware par real hoti hain, jo HIL (Hardware-in-the-Loop) ka kaam hai.

Agar ek function directly hardware register read karta hai, toh bhi use host par as-is unit-test kiya ja sakta hai.

False — register address x86 par kuch bhi mean nahi karta, toh ya toh crash hoga ya garbage read hoga. Pehle use ek HAL call ke through route karna padega jo mock ho sake.

Har host unit test pass karna matlab firmware board par kaam karega.

False — host tests sirf logic verify karte hain, timing, ISR (Interrupt Service Routine) latency, ADC (Analog-to-Digital Converter) noise, ya peripheral interaction nahi. Green host tests + broken HIL bilkul normal state hai.

HIL testing, host unit testing ko unnecessary bana deta hai.

False — HIL slow, expensive, aur debug karna mushkil hai; host tests milliseconds mein logic bugs pakad lete hain. Ye dono tiers alag failure classes cover karte hain, ek hi cheez do baar nahi.

HAL function ko mock karna logic ke behaviour ko change kar deta hai jo test ho raha hai.

False — ek accha mock sirf hardware boundary substitute karta hai; pure logic (jaise compute_pwm) unchanged run hoti hai. Agar mocking aapki logic ka behaviour change kar raha hai, toh aapka seam galat jagah par hai.

Host par 100% coverage achieve karna embedded firmware ka goal hai.

False — vendor drivers aur thin HAL glue host par meaningfully run nahi ho sakti, isliye host coverage naturally 100% se neeche cap hoti hai. Thumb rule roughly hai ki apni custom logic ka zyaada hissa cover karo, lekin exact figure heavily depend karta hai aapke paas kitna peripheral code hai — kisi single percentage ko sacred mat maano.

HIL test ek baar pass ho gaya toh wo reliable pass hai.

False — real hardware mein noise, timing jitter, aur race conditions hoti hain, isliye intermittent failures hi poora reason hai HIL exist karne ka. Ek flaky HIL result ek signal hai, koi cheez nahi jo retry karke hatani ho.

Fault injection sirf HIL mein nahi, host unit tests mein bhi ho sakta hai.

True — aap ek mock ko error code return karne ya data corrupt karne de sakte ho taaki error paths sasti tarah test ho sakein. Fault Injection Testing on host logic cover karta hai; HIL electrical faults add karta hai jo host produce nahi kar sakta.

printf-style debugging ek proper host test suite ka valid substitute hai.

False — prints manual, non-repeatable hote hain, aur automatically kuch prove nahi karte; ek test suite expected outcomes assert karta hai aur CI mein hamesha re-run hota hai.

Peripheral aur HAL glue code ko kabhi test nahi karna chahiye kyunki ye "sirf plumbing" hai.

False — HAL glue aur driver setup subtle register-config aur initialization bugs hide kar sakti hai. Ye precisely wahi bugs hain jo HIL (real hardware par) pakadne ke liye meant hai, isliye "logic ko under-test karo aur glue ko under-test karo" dono galtiyan hain.

Spot the error

"Humne firmware ko target ke liye compile kiya aur use apne host unit-test runner ke neeche run kiya."

Aap target machine code (Cortex-M) ko x86 host par execute nahi kar sakte. Host tests host ke liye recompile hone chahiye mocked HALs ke saath — ek alag build, wahi binary nahi.

"Hamara mock ek fixed ADC value return karta hai, toh hum confident hain ki driver ADC sahi se read kar raha hai."

Mock driver ko replace kar deta hai, toh real driver ke baare mein kuch bhi test nahi ho raha. Sirf HIL (real ADC — Analog-to-Digital Converter — plus ek known input voltage) actual conversion verify kar sakta hai.

"Response-time test host par deadline simulate karne ke liye time.sleep() use karta hai."

sleep host par target ki real-time response nahi, host ke scheduler ko measure karta hai. Timing hardware par ek logic analyzer se ya on-target timestamps se measure honi chahiye.

"Humne PWM cap logic hal_pwm_set() ke andar rakha taaki guaranteed apply ho."

Ye testable logic ko un-mockable hardware layer ke andar bury kar deta hai, toh host tests cap tak reach nahi kar sakte. Cap ko pure logic mein rakho; HAL ko sirf register touch karna chahiye.

"Hamara HIL test scenario set up karne ke liye DUT ke internal variables mein write karta hai."

HIL sirf real I/O ke through DUT (Device Under Test) ke saath interact kar sakta hai — pins, buses, signals — internal memory se nahi. Variables mein reach karna ek host-test technique hai; HIL ko loop ke bahar se stimulate karna padta hai.

"Humne host tests skip kar diye kyunki hamara RTOS scheduler behaviour ko deterministic banata hai."

RTOS timing determinism deta hai, logic correctness nahi; ek galat formula schedule par bhi galat hi rehta hai. Host tests woh logic pakad lete hain jise scheduler faithfully run karta hai.

"CI speed up karne ke liye, hum har commit par saare HIL tests run karte hain aur koi host tests nahi."

Ye ulta hai — HIL slow, scarce resource hai jise aap carefully gate karte ho, jabki host tests fast front line hain. CI ko har commit par host tests run karne chahiye aur HIL ek subset/nightly par.

Why questions

Har hardware access ko directly registers call karne ki jagah hal_* functions ke through kyun route karein?

Kyunki ek named seam aapko implementations swap karne deta hai: target par real registers, host par mocks. Seam ke bina test inputs inject karne ki koi jagah hi nahi hai.

Embedded projects ko do separate build targets kyun chahiye?

Ek binary ARM MCU par run honi chahiye (cross-compiler, real HAL, main), doosri host par (native gcc, mock HAL, test-framework entry). Same source, alag compiler/HAL/entry point.

Interrupt latency printf timestamp ki jagah logic analyzer se kyun measure karein?

printf khud deadline se kaafi zyada time leta hai aur timing perturb karta hai, toh jis cheez ko measure karna hai use hi hide kar deta hai. Ek logic analyzer microsecond resolution par ek GPIO edge ko non-intrusively dekhta hai.

"80% bugs 20% code mein hote hain" wala idea heavy host testing justify karne ke liye kyun use hota hai?

Complex logic (filters, state machines, protocol parsing) aapka custom 20% hai aur wahin bugs concentrate hote hain; aur ye hardware-free bhi hota hai, isliye host-testable. Toh aap apne sabse saste tests exactly wahin aim karte ho jahan risk sabse zyada hai — lekin ye ek heuristic hai, baki sab skip karne ka licence nahi.

Host tests microseconds mein run kyun ho jaate hain jabki HIL ko test ke liye seconds lagte hain?

Host code ek GHz desktop CPU par run hota hai bina flashing, settling delays, aur koi real clock wait kiye. HIL ko firmware flash karna padta hai, analog signals settle hone deni padti hain, aur actual real-time durations ka intezaar karna padta hai.

HIL pipeline mein pehli cheez ki jagah baad mein kyun aata hai?

Ye slow hai, scarce rig hardware chahiye, aur bina scope ke iski failures debug karna mushkil hai. Aap chahte ho ki saste host tests pehle logic bugs eliminate kar chuke hon expensive HIL slot spend karne se pehle.

Edge cases

Agar ek logic function ki koi bhi hardware dependency nahi hai, toh bhi use HAL chahiye?

Nahi — pure computation (ek PID step, ek checksum) directly plain inputs ke saath host-testable hai. Waahan HAL add karna needless indirection hai.

Agar aap mock HAL implementation link karna bhool jaao toh host test ka kya hoga?

Linker undefined hal_* symbol par fail ho jaata hai, ya worse, real target HAL pull in ho jaata hai jo host par run nahi karega. Missing mock poora test seam hai, toh build correctly complete hi nahi ho sakti.

Ek HIL test "throttle" ADC input ko 0V feed karta hai — ye kaun sa boundary probe karta hai?

Zero/idle degenerate case, check karta hai ki PWM apni floor tak girti hai aur koi division-by-zero ya underflow nahi hota. Zero aur full-scale inputs exactly wahin hain jahan mapping bugs aur clamps dikhte hain.

Kya host tests PID controller mein integral windup pakad sakte hain?

Haan — windup pure math hai jo simulated steps par accumulate hoti hai, host par bina kisi hardware ke fully reproducible. Ye saste host testing ka textbook case hai, jaisa parent example dikhata hai.

Agar target ka ADC noisy values return kare lekin mock hamesha clean number return kare toh kya hoga?

Host tests pass ho jaayenge jabki real system misbehave karega, kyunki mock kabhi noise reproduce nahi karta. Sirf HIL jo real (ya deliberately noisy) signal feed kare, filtering weaknesses expose karta hai.

Exactly deadline boundary par — response exactly

T_{deadline}

measure ho — kya ye pass hai?

Strict rule

T_{measured} < T_{deadline}

ke hisaab se, exactly deadline par land karna fail hai. Real-time safety strict inequality use karta hai taaki jitter aapko uske upar push na kar sake.

Agar firmware debugger mein kaam kare lekin free run karne par fail ho (no breakpoints), toh kaun sa tier ise pakadta hai?

HIL — debugger timing alter kar deta hai aur race conditions mask kar sakta hai jo sirf full speed par dikhti hain. Host tests bhi ise nahi dekh sakte kyunki wo real timing model nahi karte.