Worked examples — Fault tolerance — fail-safe vs fail-operational

5.5.26 · D3 · Coding › Embedded Systems & Real-Time Software › Fault tolerance — fail-safe vs fail-operational

Yeh page ek drill sheet hai. Parent note ne tumhe formulas diye the; yahan hum har tarah ke input ko hit karte hain jo woh formulas face kar sakte hain — tiny, huge, "kya ho agar koi number zero ho", aur trick question. Har ek ko pehle khud solve karo phir steps padho.

Har example bilkul teen numbered steps mein todi gayi hai (setup → core computation → finish), aur har step ka apna "Why this step?" hai — taaki rhythm kabhi na badle.

Two names you must not confuse

Matrix se pehle, do acronyms ko pin down karo jo yeh page side by side use karta hai:

The scenario matrix

Kisi bhi number se pehle, chalte hain kya vary ho sakta hai ek fault-tolerance problem mein — taaki koi case aisa na rahe jo humne rehearse na kiya ho.

Is poore topic ke do main engines hain:

Yahan har symbol parent note mein samjhaya gaya tha. Agar koi unfamiliar lage, ruko aur use phir padho — hum ab uske upar build kar rahe hain.

Yeh raha case classes ka grid jo yeh topic tumhare saamne rakh sakta hai:

Ab hum A → H chalenge, ek example per cell. Related machinery Redundancy patterns (N+1, TMR, DMR), Safety Integrity Levels (SIL), aur Watchdog timers and health monitoring mein hai — jab koi step unpar lean kare toh link out karo.

[!example] Example 1 — Cell A: near-perfect detection ($\beta \to 1$)

Statement. Ek pressure sensor $\lambda =10^{-5}$ per hour ki rate se fail hota hai. Diagnostics $\beta =0.99$ failures ko catch karke valve ko safe (closed) state mein drive kar deti hain. MTTDf nikalo aur batao yeh kitne saal hai.

Forecast: Pehle guess karo — kya 99% failures detect karna tumhe thodi safety dega ya bahut zyada? Woh number dimag mein rakho.

1. Failure rate ko split karo. Dangerous slice hai ${\lambda }_d=(1-\beta )\lambda$. ${\lambda }_d=(1-0.99)(10^{-5})=0.01\times 10^{-5}=10^{-7}{\text{h}}^{-1}$ Why this step? Sirf undetected failures dangerous hoti hain — detected ones ne tumhe already safe steer kar diya. Toh hum $\lambda$ ka 99% throw away kar dete hain shuru karne se pehle.

2. Invert karke mean time nikalo. MTTDf $=1/{\lambda }_d$. $\text{MTTDf}=\frac{1}{10^{-7}}=10^7\text{hours}$ Why this step? Constant-rate process ke liye, mean-time-to-event exactly rate ka reciprocal hota hai (yeh wahi ${\int }_0^{\infty }{e}^{-{\lambda }_{d}t}dt=1/{\lambda }_d$ hai jo parent note se aaya hai).

3. Years mein convert karo. $8760$ hours/year se divide karo. $10^7/8760\approx 1141\text{years}$ Why this step? "Per hour" mein rate hoti hai toh time hours mein milta hai; humans saalon mein sochte hain, isliye hum $8760$ hours in a year se rescale karte hain taaki "1141-year" safety claim samajh aaye. Unit conversion physics nahi badlata, sirf readout badlata hai.

Verify: Undetected fraction $100\%$ se $1\%$ gayi — ${\lambda }_d$ mein $\times 100$ shrink — isliye MTTDf "detect nothing" baseline $10^5$ h se $\times 100$ badha. Units: $\frac{1}{{\text{h}}^{-1}}=\text{h}$. ✓ Detection bahut powerful hai — opposite extreme ke liye Ex 2 dekho.

[!example] Example 2 — Cell B: the degenerate case $\beta =0$

Statement. Same sensor, $\lambda =10^{-5}$/h, lekin diagnostics disconnected hain: $\beta =0$. Ab MTTDf kya hai?

Forecast: Zero detection ke saath, kya formula blow up karega, break karega, ya sirf plain number dega?

1. $\beta =0$ plug karo. ${\lambda }_d=(1-0)\lambda =\lambda =10^{-5}$. Why this step? Ab har failure dangerous hai — koi safe branch bilkul nahi hai. System ab "fail-safe" nahi raha; yeh sirf ek bare component hai, isliye MTTDf plain MTTF par collapse ho jaata hai.

2. Invert karo. $\text{MTTDf}=\frac{1}{10^{-5}}=10^5\text{hours}$ Why this step? Ex 1 jaisa hi reciprocal-of-rate rule — lekin ab rate poora $\lambda$ hai, kyunki kuch bhi subtract nahi kiya gaya.

3. Years mein convert karo. $10^5/8760\approx 11.4\text{years}$. Why this step? Formula divide by zero nahi karta — dangerous $\lambda$ sirf tabhi infinite hoti hai jab $\beta \to 1$ ho, $\beta \to 0$ par nahi. $\beta =0$ par MTTDf simply raw MTTF $=1/\lambda$ ke barabar hota hai, jo ek finite, legible number of years hai.

Verify: Ex 1 se compare karo: detection off karne se MTTDf $10^7$ se $10^5$ ho gayi — exactly woh factor $(1-\beta {)}^{-1}=100$ jo pehle gain kiya tha, ab surrender kiya. ✓ Asli blow-up ($\beta \to 1$) ka matlab hai "ek system jo kabhi dangerously fail nahi hota" — ek idealisation, isliye math correctly MTTDf ko $\infty$ sirf wahan bhejta hai.

[!example] Example 3 — Cell C: realistic middling detection

Statement. Ek industrial actuator: $\lambda =4\times 10^{-6}$/h, aur typical self-test coverage $\beta =0.6$. ${\lambda }_d$ aur MTTDf years mein nikalo.

Forecast: 60% coverage decent lagta hai — kya yeh Ex 1 ka "1000-year" bar clear karega?

1. Dangerous rate. ${\lambda }_d=(1-0.6)(4\times 10^{-6})=0.4\times 4\times 10^{-6}=1.6\times 10^{-6}$. Why this step? $40\%$ failures undetected slip hoti hain, isliye hum $\lambda$ ka sirf woh hissa rakhte hain.

2. MTTDf in hours. $\frac{1}{1.6\times 10^{-6}}=625,000\text{hours}$. Why this step? Phir reciprocal-of-rate — woh single move jo kisi bhi dangerous rate ko mean time mein badal deta hai.

3. Years mein convert karo. $625000/8760\approx 71.3\text{years}$. Why this step? Hours ko years mein rescale karo human-readable safety horizon ke liye, exactly Ex 1 ki tarah.

Verify: Sirf $71$ saal — Ex 1 ke $1141$ se bahut neeche. Lesson: $\beta \approx 0.9$ se neeche tum safe game mein barely ho; Safety Integrity Levels (SIL) tables exactly isi liye bahut high $\beta$ demand karte hain. Units check: h. ✓

[!example] Example 4 — Cell D: TMR early life (redundancy ka home turf)

Statement. Teen identical flight computers, $\lambda =10^{-3}$/h. $\lambda t=0.1$ par evaluate karo (mission ke early life mein). Single unit ki survival ko TMR survival se compare karo.

Forecast: Ek ki jagah teen copies — failure chance kitna chhota hoga?

1. Single unit. $R_1=e^{-\lambda t}=e^{-0.1}\approx 0.9048$. Why this step? Ek component bare exponential se survive karta hai — yeh figure mein chalk-blue baseline curve hai, humara yardstick jo beat karna hai.

2. TMR exact. Master formula use karo $\lambda t=0.1$ ke saath: $R_{\text{sys}}=e^{-0.3}+3e^{-0.2}(1-e^{-0.1})$ $=0.7408+3(0.8187)(0.0952)\approx 0.7408+0.2338=0.9746$ Why this step? Pehla term = teeno alive; doosra term = exactly ek dead lekin do survivors usse abhi bhi out-vote kar rahe hain. Yeh chalk-pink curve hai jo dashed line par blue se upar baithti hai — woh height difference redundancy ne kharida.

3. Failure chances read off karo. Single $\approx 1-0.9048=9.5\%$ fail hota hai; TMR $\approx 1-0.9746=2.5\%$ fail hota hai. Why this step? "Reliability" survival probability hai, isliye failure chance ek minus woh hota hai — woh number jo safety case actually care karta hai. Hum dono compute karte hain taaki win us language mein stated ho ("box lose hone ka chance") jo engineers argue karte hain.

Verify: $1-0.9048=0.0952$ aur $1-0.9746=0.0254$; ratio $0.0952/0.0254\approx 3.7\times$ improvement. ✓ Yahi to Redundancy patterns (N+1, TMR, DMR) ka pura point hai — bade wins jahan $\lambda t$ chhota ho.

[!example] Example 5 — Cell E: the long-run limit (the surprise)

Statement. Usi TMR system ke liye, MTTF (poore voted system ki mean time to failure) $1/\lambda$ ke multiple ke roop mein compute karo. Kya yeh single unit se bada hai ya chhota?

Forecast: Teen components — surely average lifetime ek se zyada hoga? Bet lagao, phir padho.

1. Reliability integrate karo. MTTF $R_{\text{sys}}(t)$ ke neeche ka area hai; pehle product ko clean exponentials mein expand karo: $3e^{-2\lambda t}(1-e^{-\lambda t})=3e^{-2\lambda t}-3e^{-3\lambda t}$ ${\text{MTTF}}_{\text{TMR}}={\int }_0^{\infty }\left[e^{-3\lambda t}+3e^{-2\lambda t}-3e^{-3\lambda t}\right]dt$ Why this step? Mean lifetime = survival curve ke neeche ka area (wahi identity jo MTTDf ke liye Ex 1 mein use hui). Pehle expand karte hain taaki har term ek pure exponential ho jise hum ek rule se integrate kar sakein.

2. Term by term integrate karo. ${\int }_0^{\infty }{e}^{-kt}dt=\frac{1}{k}$ use karo kisi bhi positive constant $k$ ke liye (yahan $k=3\lambda ,2\lambda ,3\lambda$): $=\frac{1}{3\lambda }+\frac{3}{2\lambda }-\frac{3}{3\lambda }=\frac{1}{3\lambda }+\frac{3}{2\lambda }-\frac{1}{\lambda }$ Why this step? Har exponential zero tak decay karta hai, isliye uska total area bas uski decay rate $k$ ka reciprocal hai — woh single fact teeno integrals ek saath kar deta hai.

3. $6\lambda$ par combine karo. Teeno fractions ko common denominator $6\lambda$ par rakh ke add karo: $=\frac{2}{6\lambda }+\frac{9}{6\lambda }-\frac{6}{6\lambda }=\frac{2+9-6}{6\lambda }=\frac{5}{6\lambda }\approx 0.833\frac{1}{\lambda }$ Why this step? Teen pieces ke denominators $3\lambda$, $2\lambda$, $\lambda$ hain; $6\lambda$ unka least common multiple hai, isliye har ek ko $6\lambda$ par rewrite karne se numerators directly add ho jaate hain aur ek single clean fraction mil jaata hai.

Verify: $\frac{5}{6}\approx 0.833<1$ — TMR ka mean lifetime single unit ke $1/\lambda$ se chhota hai! Kyun? Bina repair ke, TMR mein zyada components hain jo do failures accumulate kar sakte hain jab tak woh use kill na kar dein, aur voter khud late life mein unforgiving hota hai. Redundancy early mein help karta hai (Ex 4), average lifetime par nahi. Crossover point ke liye Ex 6 dekho. ✓

[!example] Example 6 — Cell F: where does TMR stop helping?

Statement. Woh time $t_{\star }$ nikalo (as a multiple of $1/\lambda$) jahan TMR reliability single unit ke barabar hoti hai — jiske baad redundancy actually worse ho jaati hai.

Forecast: "Early" aur "forever" ke beech kahin dono curves cross karengi. Guess karo: $t=1/\lambda$ se pehle ya baad?

1. Substitution ke saath equal set karo. $x=e^{-\lambda t}$ let karo (toh $x\in (0,1]$, $t$ mein decreasing), deta hai: $x^3+3x^2(1-x)=x$ Why this step? $x$ substitute karne se exponentials ek plain polynomial mein badal jaate hain jo hum haath se solve kar sakte hain — koi calculator nahi chahiye.

2. Quadratic tak reduce karo. $x^3+3x^2-3x^3=x\Rightarrow -2x^3+3x^2-x=0$ simplify karo, phir $x$ se divide karo: $2x^2-3x+1=0$ Why this step? Har term mein $x$ hai, aur $x=e^{-\lambda t}\ne 0$ kisi bhi finite time ke liye, isliye $x$ se divide karna legal hai aur cubic ko quadratic tak drop karta hai jo hum recognise karte hain.

3. Solve karo aur back-substitute karo. $x=\frac{3\pm \sqrt{9-8}}{4}=\frac{3\pm 1}{4}$ ⇒ $x=1$ ya $x=\frac{1}{2}$. $x=1$ discard karo (woh $t=0$ hai, trivially equal); $x=\frac{1}{2}$ lo: $e^{-\lambda t_{\star }}=\frac{1}{2}\Rightarrow \lambda t_{\star }=\ln 2\approx 0.693\Rightarrow t_{\star }\approx \frac{0.693}{\lambda }$ Why this step? Quadratic dono intersection heights deta hai; sirf $x=\frac{1}{2}$ meaningful crossover hai, aur natural log ke saath substitution undo karne se actual time milta hai.

Verify: $t_{\star }$ se just pehle (Ex 4 $\lambda t=0.1<0.693$ par tha) TMR aage tha — consistent. Figure mein curves exactly $\lambda t=\ln 2$ par milti hain, phir blue pink ko overtake karta hai. ✓ Real systems "worse" region ko repair/replacement (Real-time scheduling with fault recovery) add karke avoid karte hain taaki woh kabhi $t_{\star }$ ke baad na baithe.

[!example] Example 7 — Cell G: strategy choose karo (word problem)

Statement. Tum ek shipping canal par drawbridge ke liye controller design kar rahe ho. Agar controller fault kare, toh kaunsi strategy — fail-safe ya fail-operational — aur kyun?

Forecast: Socho: kuch na karne vs kuch galat karne ka worst outcome kya hai?

1. Poochho: kya loss-of-function safe hai ya dangerous? Agar bridge simply move karna band kar de aur wahan ruke, toh na koi ship crush hoga aur na koi car gap mein drive karegi. Why this step? Yeh is poore topic ka single decision rule hai — kisi bhi redundancy math (Common-mode failures and diversity ke alawa) se pehle, tum pehle classify karte ho ki kuch na karna tumhe safe state mein land karta hai ya nahi.

2. Dono failure costs compare karo. Kuch na karna (bridge frozen) = traffic delay. Kuch galat karna (bridge ek car ke neeche khulja / ek ship par band ho jaye) = catastrophe. Why this step? Fail-safe tab correct hota hai jab no-action wrong-action se sasta ho; yahan dono costs wildly asymmetric hain, jo design decide karta hai.

3. Fail-safe par commit karo aur mechanism spell out karo. Kisi bhi detected fault par → bridge ko uski current position mein clamp karo, road barriers ko RED kar do, alarm bajao, aur motion resume karne se pehle human reset require karo. Why this step? Hum judgement ko hardware mein turn karte hain: spring-loaded brakes jo power loss par engage hote hain (bilkul parent note ke Example 1 ke railway signal relay ki tarah) guarantee karte hain safe state tab bhi jab software dead ho, isliye koi code defect bridge ko unsafely move nahi kar sakta. Woh physical default hi choice ko trustworthy banata hai, sirf reasoning nahi.

Verify: Aircraft se contrast karo jahan controls freeze karna sab ko maar deta hai — woh must be fail-operational. Same decision rule, opposite answer, kyunki wahan "kuch na karna" catastrophe hai. ✓ Yahan koi arithmetic nahi hai, lekin judgement hi exam ka real target hai.

[!example] Example 8 — Cell H: exam twist (counting galat ki gayi)

Statement. Ek student likhta hai: "TMR tab fail hota hai jab teeno units fail ho jaayein, isliye $P_{\text{fail}}=p^3$." Per-unit failure probability $p=10^{-3}$ over the mission ke saath, sahi $P_{\text{fail}}$ nikalo aur batao student ka shortcut risk ko kitna galat judge karta hai.

Forecast: Kya "all three" sahi count hai, ya "two or more"? Yeh classic slip hai — aage padhne se pehle decide karo.

1. True failure condition state karo. Ek majority voter tab haarta hai jab woh majority form nahi kar paata — yeh tab hota hai jab 2 ya zyada units fail ho jaayein, 3 nahi. $P_{\text{fail}}=\left(\genfrac{}{}{0ex}{}{3}{2}\right)p^2(1-p)+p^3$ Why this step? Do bad units already ek good unit ko out-vote kar dete hain, isliye system doosri failure land hote hi dead ho jaata hai. Sirf "all 3 bad" count karna "exactly 2 bad" ke har arrangement ko ignore karta hai aur isliye failures ko undercount karta hai.

2. $p=10^{-3}$ ke saath evaluate karo. $=3(10^{-6})(0.999)+10^{-9}\approx 2.997\times 10^{-6}+10^{-9}\approx 3.0\times 10^{-6}$ Why this step? Number plug karne se pata chalta hai ki "exactly 2" term ($\approx 3\times 10^{-6}$) "all 3" term ($10^{-9}$) se teen orders of magnitude bada hai — woh piece jo student ne throw away kiya woh essentially poora answer hai.

3. Student ke shortcut se compare karo. Student ka $p^3=(10^{-3}{)}^3=10^{-9}$, isliye unka estimate hai: $\frac{3.0\times 10^{-6}}{10^{-9}}\approx 3000\times \text{ too optimistic}.$ Why this step? Error ko quantify karna ek subtle miscount ko ek safety-critical number mein badal deta hai: ek design jo $10^{-9}$ par sign off hua jo actually $3\times 10^{-6}$ hai usne apna risk roughly $3000\times$ under-budget kiya hai.

Verify: Dominant term $3p^2=3\times 10^{-6}$ swamps $p^3=10^{-9}$, isliye "exactly 2" case ignore karna real risk ka 99.97% throw away kar deta hai. Yeh parent note ke A320 figure ($\approx 3\times 10^{-6}$ per flight) se match karta hai. ✓

[!recall]- Rapid self-test