Visual walkthrough — Fault tolerance — fail-safe vs fail-operational

This page takes ONE promise from the parent note — "redundancy lets a system keep flying after a part breaks" — and rebuilds the maths from nothing. By the end you will have drawn the curve that tells an aircraft engineer: how likely is my triple-redundant flight computer to still be alive after 10 hours?

We build every symbol before we use it. If you have never seen $e$, a probability, or an exponent, you start on line one and finish at the summary.

Prerequisites you can lean on: Redundancy patterns (N+1, TMR, DMR), and this is a child of the parent topic.

---

Step 1 — What does "reliability" even mean? Draw one component as a coin that flips over time

WHAT. We start with a single component — one flight-control computer. We invent a number called reliability, written $R(t)$, that answers exactly one question:

"If I switch this on at time $0$, what is the chance it is still working at time $t$?"

WHY this number and not another. Engineers cannot promise "it never breaks." Everything breaks eventually. So instead of a yes/no promise, we use a probability — a number between $0$ (impossible) and $1$ (certain). $R(t)=1$ means definitely alive; $R(t)=0$ means definitely dead. We track how this number slides from 1 down toward 0 as time passes.

PICTURE. Look at the horizontal track: time flows left to right. The cyan bar is the chance the component is still alive. At $t=0$ it fills the whole bar (brand new, $R=1$). As time flows the bar shrinks — the amber slice is the chance it has already died.

---

Step 2 — The "hazard rate" $\lambda$: how fast the bar shrinks

WHAT. We need to say how quickly the alive-bar shrinks. We introduce one symbol, ==$\lambda$ (Greek letter "lambda"), the hazard rate== — failures per hour.

$\lambda =\underset{\text{a small number}}{\underbrace{10^{-4}}}\underset{\text{how many}}{\underbrace{\text{failures}}}/\underset{\text{per this much time}}{\underbrace{\text{hour}}}$

Reading it: in each hour of running, there is a $10^{-4}$ (one in ten thousand) chance this component picks that hour to die.

WHY a single constant. For electronics in their normal life (past infant-mortality, before wear-out), the chance of dying "this hour" does not depend on age — a 3-hour-old chip is no more tired than a 1-hour-old one. That memoryless property is what lets $\lambda$ be one fixed number. See Watchdog timers and health monitoring for how we actually measure failures in the field.

PICTURE. Each hour we chop the same fraction $\lambda$ off whatever alive-bar remains. Not a fixed amount — a fixed fraction. Watch the steps: each drop is a slice of the current bar, so the drops get physically smaller as the bar shrinks.

---

Step 3 — Why $e^{-\lambda t}$ appears (and what $e$ is doing here)

WHAT. Chopping the same fraction off, over and over, forever-more-finely, produces exactly one curve:

$R(t)=e^{-\lambda t}$

Let us earn every symbol.

$R(t)=\underset{\stackrel{\begin{array}{c}\text{how much decay}\\ \text{has piled up}\end{array}}{\overbrace{-\lambda t}}}{\underbrace{e}}$

• $\lambda t$ = rate $\times$ time = the total amount of decay accumulated by time $t$. If $\lambda =10^{-4}$/h and $t=10$ h, then $\lambda t=10^{-3}$.
• The minus sign says decay shrinks the bar.
• $e$ (Euler's number, $\approx 2.718$) is the natural base that appears whenever something changes by a fixed proportion of its current size. That is precisely Step 2's picture.

WHY $e$ and not, say, $\frac{1}{2}$? Because the decay is continuous — the coin isn't flipped once per hour, it's "flipping" every instant. If you take "lose fraction $\lambda$" and apply it in ever-smaller time slices and let the slices go to zero, the limit is $e^{-\lambda t}$. It is the smooth version of repeated fractional loss.

PICTURE. The staircase of Step 2, smoothed into a curve. It starts at height $1$, plunges fastest at the start, and flattens as it hugs (but never touches) zero. The amber dot marks our example: at $\lambda t=10^{-3}$, the height is $\approx 0.999$.

---

Step 4 — Three computers: draw all the ways they can live or die

WHAT. Triple Modular Redundancy (TMR) uses three identical units and a voter — a small circuit that takes all three answers and outputs whatever the majority says. The system is "alive" as long as at least 2 of 3 units still work (the voter needs two agreeing votes).

Let $p=1-e^{-\lambda t}$ be one unit's chance of being dead at time $t$, and $q=e^{-\lambda t}$ its chance of being alive (note $p+q=1$).

WHY 2-of-3. One healthy unit can't be trusted alone — if it disagrees with a broken one, the voter can't tell which is right. Two agreeing units out-vote one liar. So we need $\ge 2$ alive.

PICTURE. A tree of every outcome for 3 units — each branch is alive (cyan) or dead (amber). Eight leaves total. The framed leaves (all-3-alive, and each of the three 2-alive cases) are the ones where the system survives.

---

Step 5 — Count the survivor branches with the binomial

WHAT. We add up the framed branches. The number of ways to choose which units are the survivors is written $\left(\genfrac{}{}{0ex}{}{3}{k}\right)$ ("3 choose $k$") — how many distinct leaves have exactly $k$ alive.

$\left(\genfrac{}{}{0ex}{}{3}{3}\right)=1\text{(one way: all alive)}\left(\genfrac{}{}{0ex}{}{3}{2}\right)=3\text{(three ways: pick which one died)}$

System survives if 3 alive or exactly 2 alive:

$R_{\text{TMR}}(t)=\underset{\text{all 3 alive}}{\underbrace{\left(\genfrac{}{}{0ex}{}{3}{3}\right)q^3}}+\underset{\begin{array}{c}\text{exactly 2 alive,}\\ \text{1 dead}\end{array}}{\underbrace{\left(\genfrac{}{}{0ex}{}{3}{2}\right)q^{2}p}}$

Substitute $q=e^{-\lambda t}$ and $p=1-e^{-\lambda t}$:

$R_{\text{TMR}}(t)=\underset{\text{all three alive}}{\underbrace{e^{-3\lambda t}}}+\underset{\text{one specific unit dead, other two alive, }\times 3}{\underbrace{3e^{-2\lambda t}(1-e^{-\lambda t})}}$

WHY multiply the probabilities. The three units fail independently (different chips, different power lines — see Common-mode failures and diversity for when this assumption breaks). Independent events' joint probability is the product: $q\cdot q\cdot q=q^3$.

PICTURE. Two stacked reliability curves: the single unit (thin cyan) and TMR (thick cyan). Notice TMR sits above the single curve early on, then dips below it far to the right.

---

Step 6 — The early-life win: plug in real numbers

WHAT. Aircraft care about the flight, which is early in the component's life ($\lambda t$ small). Take $\lambda t=0.1$ (i.e. $t=0.1/\lambda$):

$R_{\text{single}}=e^{-0.1}=0.905$

$R_{\text{TMR}}=e^{-0.3}+3e^{-0.2}(1-e^{-0.1})=0.7408+3(0.8187)(0.0952)=0.9746$

Term by term for TMR:

• $e^{-0.3}=0.7408$ → chance all three survived.
• $3\times e^{-0.2}\times (1-e^{-0.1})=3\times 0.8187\times 0.0952=0.2338$ → chance exactly one died.
• Sum $=0.9746$.

Failure chance drops from $9.5\%$ (single) to $2.5\%$ (TMR) — nearly 4× safer right where it counts.

WHY it works here but not later. Early on, a second failure is rare (both need to die), so tolerating the first failure is almost a free pass.

PICTURE. Zoom into the left of Step 5's plot. Two amber gaps to the top line $R=1$: the single-unit gap is tall ($9.5\%$), the TMR gap is short ($2.5\%$).

---

Step 7 — The surprising long-run twist (the degenerate case)

WHAT. Integrate $R_{\text{TMR}}$ over all time to get the Mean Time To Failure (average lifetime):

${\text{MTTF}}_{\text{TMR}}={\int }_0^{\infty }{R}_{\text{TMR}}(t)dt=\frac{1}{3\lambda }\cdot 1+3\left(\frac{1}{2\lambda }-\frac{1}{3\lambda }\right)=\frac{5}{6\lambda }$

But a single unit's MTTF is $\frac{1}{\lambda }$. So:

${\text{MTTF}}_{\text{TMR}}=\frac{5}{6\lambda }\approx 0.83\times \frac{1}{\lambda }<{\text{MTTF}}_{\text{single}}$

Three units have a SHORTER average life than one!

WHY this is not a contradiction. Averaged over all time, TMR loses — because far out, needing 2-of-3 to survive is harder than 1-of-1 (there's just more stuff to have already broken, and no repair). But we don't fly for infinity. On the finite mission window that matters, TMR is far safer (Step 6). This is the crossing point of the two curves in Step 5.

PICTURE. The two curves crossing. Left of the crossing (short missions): TMR on top = safer. Right of it (long unattended life): single on top. Amber vertical line marks the crossover.

---

The one-picture summary

This final drawing compresses all seven steps: from one shrinking bar ($e^{-\lambda t}$), to the 2-of-3 branch count, to the two crossing curves and the early-life safety gain.

Recall Feynman retelling — say it back in plain words

A single computer's chance of still being alive starts at 1 and slides down a smooth curve, $e^{-\lambda t}$, where $\lambda$ is how many failures happen per hour and $\lambda t$ is how much "decay" has piled up. The curve is $e$-shaped because the thing loses the same fraction of what's left every instant — like a savings account bleeding a fixed percent.

Now put three identical computers behind a voter that trusts the majority. The system lives as long as two of three live. I list all eight ways three coins can land, keep the four where at least two are alive, and add their probabilities — multiplying because the units fail independently. That gives $R_{\text{TMR}}=e^{-3\lambda t}+3e^{-2\lambda t}(1-e^{-\lambda t})$.

Early in life this triple system is much safer: at $\lambda t=0.1$ the failure chance falls from $9.5\%$ to $2.5\%$. But integrate over all time and TMR's average lifetime is only $\frac{5}{6\lambda }$ — shorter than a single unit's $\frac{1}{\lambda }$! No paradox: redundancy trades long-run average life for short-run safety. On a 10-hour flight — where safety per mission is everything — that's exactly the trade we want.

---

Related deep prerequisites & extensions: Byzantine fault tolerance (when a faulty unit lies instead of dying silently), Real-time scheduling with fault recovery, Hardware safety mechanisms, and Formal verification of safety properties.