Kalman filter derivation — predict step, update step: Visual walkthrough

We only need one running story: "Where is my drone, in one dimension?" A number line, marked in metres. That's it.

Step 1 — A guess is a hill, not a point

WHAT. I think my drone is at $10$ m. But I am not certain. So instead of one dot on the number line, I draw a hill — tall where I think it is, low where I doubt it. This hill is a Gaussian (a bell curve).

WHY a hill and not a dot? Because a dot claims perfect knowledge, which I don't have. The hill carries two facts at once: its peak (my best guess) and its width (my doubt). We will name these.

PICTURE. Look at figure s01. The peak sits at $a = 10$ m — that's my best guess (the mean). The width is set by $σ_{a}$ — the standard deviation, literally "how many metres of wiggle room." A fat hill = unsure; a thin spike = confident.

Figure — Kalman filter derivation — predict step, update step

Why the exponential and not, say, a triangle? Because when you combine two independent pieces of evidence, probabilities multiply, and the exponential is the one shape that stays the same shape after multiplying (a hill times a hill is still a hill). We lean on that in Step 3. See Covariance matrices and Gaussian distributions.

Step 2 — A measurement is a second hill

WHAT. My GPS now reports $b = 14$ m, and GPS is also fuzzy, with its own width $σ_{b}$ . So I draw a second hill, peaked at $14$ .

WHY draw it the same way? So the two pieces of evidence live on the same picture and can be compared fairly — both as hills of position, both with an honest width.

PICTURE. Figure s02: the cyan hill is my prior guess (peak $10$ ), the amber hill is the measurement (peak $14$ ). Notice they overlap. The true position is probably somewhere in that overlap — near neither peak exactly.

Step 3 — Multiply the hills; the answer is another hill

WHAT. Two independent pieces of evidence combine by multiplying their hills, point by point along the number line. Where both are tall, the product is tall; where either is near zero, the product is squashed to zero.

WHY multiply? This is Bayesian inference in one move: independent evidence multiplies. It's the reason the fused hill sits between the two peaks and is thinner than either.

PICTURE. Figure s03: cyan hill × amber hill = the white hill. Three things to read off it:

Its peak sits between $10$ and $14$ — a compromise.
It leans toward whichever original hill was thinner (more trusted).
It is narrower than both parents — combining evidence makes you more sure.

Multiplying two exponentials means adding their exponents: $- \frac{( x - a ) ^{2}}{2 σ _{a}^{2}} - \frac{( x - b ) ^{2}}{2 σ _{b}^{2}} .$

Each term :: a hill's "penalty" for a candidate $x$ being far from that hill's peak.
The $+$ :: because $exp (P) \cdot exp (Q) = exp (P + Q)$ — multiplying hills = adding exponents.

Step 4 — Find the new peak (that's the fused estimate)

WHAT. The fused hill's peak is where the sum of penalties above is smallest (least penalty = tallest hill). Slide $x$ until the slope is flat — that's the top of the hill.

WHY use the derivative? The derivative is the tool that measures slope. "Peak" means "slope $= 0$ ." So we differentiate the exponent and set it to zero — that's the one place the hill stops rising and starts falling. This is the only calculus on the whole page, and it's here for exactly this reason: to locate a maximum.

PICTURE. Figure s04: the white fused hill with a flat tangent line at its peak, marked $\overset{x}{^}$ . The two grey slopes on either side show the derivative is positive on the left, negative on the right, zero exactly at the top.

Differentiate the exponent, set to $0$ : $\frac{x - a}{σ _{a}^{2}} + \frac{x - b}{σ _{b}^{2}} = 0.$

$\frac{x - a}{σ _{a}^{2}}$ :: how hard the prior hill "pulls" $x$ back toward $a$ ; a thin hill (small $σ_{a}^{2}$ ) pulls harder.
$\frac{x - b}{σ _{b}^{2}}$ :: same pull toward the measurement $b$ .
$= 0$ :: the two pulls balance — the peak sits where the tug-of-war is even.

Solve for $x$ (call it $\overset{x}{^}$ , the fused estimate): $\overset{x}{^} = \frac{σ _{b}^{2} a + σ _{a}^{2} b}{σ _{a}^{2} + σ _{b}^{2}} .$

Notice the cross-weighting: $a$ is multiplied by the other hill's variance $σ_{b}^{2}$ . Small $σ_{a}^{2}$ (I'm confident) → $a$ 's weight $σ_{b}^{2}$ dominates → answer leans to $a$ . Correct: the thinner hill wins.

Step 5 — Rewrite as "old guess + a nudge": meet the gain $K$

WHAT. The formula above is symmetric and pretty, but a filter runs online: it has an old guess and wants to nudge it toward the new data. So we algebraically rearrange the same $\overset{x}{^}$ into $\overset{x}{^} = a + K (b - a), K = \frac{σ _{a}^{2}}{σ _{a}^{2} + σ _{b}^{2}} .$

WHY this form? Because $b - a$ is the surprise (how far the measurement lands from my guess), and $K$ is the fraction of that surprise I actually act on. This is the seed of the real Kalman gain.

PICTURE. Figure s05: number line with $a$ at $10$ , $b$ at $14$ ; a full arrow from $a$ to $b$ of length $(b - a) = 4$ ; a shorter amber arrow of length $K (b - a)$ landing on $\overset{x}{^}$ . The gain $K$ literally is the fraction of the arrow you travel.

This $K$ is exactly $K_{k} = P_{k}^{-} H^{⊤} (H P_{k}^{-} H^{⊤} + R)^{- 1}$ from the parent, with $σ_{a}^{2} \to P^{-}$ , $σ_{b}^{2} \to R$ , and $H = 1$ . Same idea, dressed for vectors — see State-space representation.

Step 6 — The new width: why confidence always grows

WHAT. The fused hill is thinner. Its variance: $σ^{2} = (1 - K) σ_{a}^{2}, equivalently \frac{1}{σ ^{2}} = \frac{1}{σ _{a}^{2}} + \frac{1}{σ _{b}^{2}} .$

WHY does the width shrink? Because $K \in [0, 1]$ , the factor $(1 - K)$ is between $0$ and $1$ , so $σ^{2} < σ_{a}^{2}$ always. The second form is the punchline: precisions (1/variance) add. Each hill contributes its own sharpness, and sharpnesses stack.

PICTURE. Figure s06: the parent hills faint in the background, the fused white hill drawn solid and visibly taller and narrower than either. Its half-width $σ$ is marked and is shorter than both $σ_{a}$ and $σ_{b}$ .

Step 7 — The two degenerate cases (never left unshown)

WHAT. Push the dials to their extremes so no reader is ever surprised.

Case A — a perfect sensor ( $σ_{b}^{2} \to 0$ ). The measurement hill becomes an infinitely thin spike. $K \to 1$ . The fused hill collapses onto $b$ : $\overset{x}{^} \to b$ , $σ^{2} \to 0$ . We fully believe the sensor.

Case B — a useless sensor ( $σ_{b}^{2} \to \infty$ ). The measurement hill flattens into a pancake carrying no information. $K \to 0$ . The fused hill stays exactly where the prior was: $\overset{x}{^} \to a$ , $σ^{2} \to σ_{a}^{2}$ . We ignore garbage data. (This is why setting $Q = 0$ in the parent is a trap: it makes the prior the infinitely-thin one, so the filter ignores every real measurement forever.)

WHY show these? Because a formula you can't stress-test at its limits is a formula you don't understand. Both extremes agree with common sense — that's our proof it's right.

PICTURE. Figure s07: two side-by-side panels. Left: a spike measurement (amber) drags the fused peak onto it. Right: a flat measurement (amber) leaves the fused hill sitting on the prior.

The one-picture summary

Figure s08 stacks the whole derivation into one frame: prior hill (cyan) at $a$ , measurement hill (amber) at $b$ , and the fused hill (white) sitting between them — thinner than both — with the gain-fraction arrow $K (b - a)$ carrying the peak from $a$ to $\overset{x}{^}$ . Every symbol on the page lives somewhere in this one picture.

Recall Feynman retelling — say it out loud

Imagine you're guessing where your drone is. Your first guess is a hill on a number line: tall where you think it is, wide if you're unsure. Then GPS gives you a second hill somewhere else. To combine two honest opinions, you multiply the hills — and a hill times a hill is another, skinnier hill sitting between the two. Its peak is your new best guess; it leans toward whichever hill was skinnier (the one you trusted more). To find that peak you slide along until the slope is flat — that's calculus doing one small job: locate the top. Rearranged, the answer says "start at your old guess, then step a fraction $K$ of the way toward the measurement." That fraction $K$ is your own doubt divided by the total doubt: unsure of yourself → step far; confident → barely budge. And the new hill is always thinner than both — because looking at data can only reduce your ignorance. That's the entire Kalman update. The matrix version just does this in many dimensions at once.

Recall Quick self-test

Why does multiplying two Gaussians give a Gaussian? ::: Because multiplying exponentials adds their exponents, and the sum of two quadratics in $x$ is still a single quadratic — the signature of one Gaussian. In $\overset{x}{^} = a + K (b - a)$ , what does $b - a$ represent? ::: The innovation / surprise — how far the measurement lands from what I predicted. If $K = 0$ , what happened? ::: The sensor was useless (or the prior was perfect); the estimate ignores the measurement and stays at $a$ . Why is the fused variance always smaller than $σ_{a}^{2}$ ? ::: Because $σ^{2} = (1 - K) σ_{a}^{2}$ and $K \in [0, 1]$ , so $(1 - K) < 1$ — combining evidence can only sharpen.

Where this goes next: the same three-hill picture, made non-linear, becomes the Extended Kalman Filter (EKF) and Unscented Kalman Filter (UKF); the "precisions add" idea is the engine of Recursive Least Squares; and fusing hills from different instruments is exactly IMU and GPS sensor fusion.