Visual walkthrough — Kalman gain — minimizes trace of covariance

---

Step 1 — Two blurry guesses on one number line

WHAT. Imagine we want to know one number: call it $x$ (say, a rocket's altitude in metres). We are never told $x$ exactly. Instead we have two foggy opinions about it.

• A prediction ${\hat{x}}^{-}$ — the little minus sign means "before we look at the sensor." It came from a physics model.
• A measurement $z$ — a number a sensor reported.

WHY start here. Everything in the Kalman filter is about combining these two foggy opinions into one sharper opinion. Before we can combine them, we have to draw them honestly: not as sharp points, but as bell-shaped fog. A wide fog means "I'm not sure"; a narrow fog means "I'm confident."

PICTURE. Below, the blue fog is the prediction and the pink fog is the measurement. The horizontal axis is the value of $x$. The width of each hill is its uncertainty.

---

Step 2 — The error is what we actually want to shrink

WHAT. We do not get to see $x$, so we cannot measure "how wrong am I?" directly. But we can reason about it. Define the error $e=\underset{\text{true value}}{\underbrace{x}}-\underset{\text{our guess}}{\underbrace{\hat{x}}}.$ If our guess is perfect, $e=0$. If we guess too high, $e$ is negative; too low, positive.

In particular, the pre-update error — the error of the prediction, before we touch the sensor — gets its own name and its own minus sign to match ${\hat{x}}^{-}$: $e^{-}=x-{\hat{x}}^{-}.$ Its average squared size is exactly the prediction fog $P^{-}$ from Step 1: $P^{-}=\mathbb{E}[e^{-}{e}^{-\top }]$.

WHY this quantity. We cannot minimise "distance from the truth" if we insist on a single guaranteed number — the noise makes every trial different. So we minimise it on average, over many imagined trials. The honest score is the mean squared error: $P=\mathbb{E}[e{e}^{\top }],\mathrm{tr}(P)=\mathbb{E}[\Vert e{\Vert }^2].$ Here $\mathbb{E}[\cdot ]$ means "average over all the ways the noise could have played out," and $\mathrm{tr}$ (the trace) means "add up the diagonal" — for one number it is just the variance itself. We square the error so that being wrong by $-3$ is exactly as bad as being wrong by $+3$.

PICTURE. The error $e$ is the gap between the true spike (yellow) and our guess. Averaging the squared gap over the fog gives the number $P$ we are trying to make small.

---

Step 3 — The blend rule: a slider between the two guesses

WHAT. We build our corrected guess ${\hat{x}}^{+}$ (the plus means "after using the sensor") as: ${\hat{x}}^{+}=\underset{\text{start at prediction}}{\underbrace{{\hat{x}}^{-}}}+\underset{\text{slider}}{\underbrace{K}}\left(\underset{\text{the surprise}}{\underbrace{z-H{\hat{x}}^{-}}}\right).$

Let us read every piece where it sits:

• ${\hat{x}}^{-}$ — we begin at the prediction.
• $H{\hat{x}}^{-}$ — $H$ turns a state into "what the sensor should read." For altitude with an altimeter, $H=1$: state and reading are the same units. So $H{\hat{x}}^{-}$ is "the reading we expected." ($H$ is the measurement map; keep this name, we use it again in Step 6.)
• $z-H{\hat{x}}^{-}$ — the innovation: measured reading minus expected reading. This is the surprise. If it is zero, the sensor told us nothing new.
• $K$ — the gain, our slider. $K=0$ means "ignore the surprise, keep the prediction." $K=1$ (for $H=1$) means "jump all the way to the measurement."

WHY this exact form. It is the simplest rule that is unbiased: if both fogs are centred on the truth on average, this blend is too, for any $K$. It also has a beautiful geometric meaning — see the picture.

PICTURE. The corrected guess lives on the line segment between prediction and measurement. $K$ picks where on that segment we land.

---

Step 4 — How the fog transforms: the Joseph form

WHAT. We ask: if we blend with slider $K$, how wide is the new fog $P^{+}$? Track the error through the blend. Start from the post-update error $e^{+}=x-{\hat{x}}^{+}$, substitute the blend rule from Step 3 and the sensor model $z=Hx+v$ (where $v$ is the sensor's random noise, with spread $R$). Using the pre-update error $e^{-}=x-{\hat{x}}^{-}$ we defined in Step 2, this collapses to: $e^{+}=(I-KH)e^{-}-Kv.$

Here $I$ is the identity — the "do-nothing" element, the number $1$ in the scalar case (and the identity matrix in general), so that $I{e}^{-}=e^{-}$. Term by term:

• $(I-KH)e^{-}$ — the leftover prediction error. The factor $(I-KH)$ shrinks toward zero as $K$ grows: leaning on the sensor erases the old error.
• $-Kv$ — the freshly imported sensor noise. This grows with $K$: the more we listen to the sensor, the more of its noise we swallow.

WHY the cross-terms vanish. To get the new variance $P^{+}=\mathbb{E}[e^{+}{e}^{+\top }]$ we multiply $e^{+}$ by its own transpose and average. Multiplying out gives four blocks; two of them are the "cross" blocks $-(I-KH)\mathbb{E}[e^{-}{v}^{\top }]K^{\top }$ and its mirror. Here is the key fact we must state, because the whole formula rests on it:

With those cross blocks gone, only the two "self" blocks survive: $\boxed{P^{+}=(I-KH)P^{-}(I-KH{)}^{\top }+KR{K}^{\top }.}$ This is the Joseph form (see Joseph Form Covariance Update). It is honest for any slider position $K$.

PICTURE. Two contributions, pulling opposite ways: the blue "prediction-error" term falls as $K$ rises; the pink "noise" term climbs.

---

Step 5 — The uncertainty is a U-shaped valley

WHAT. Write $P^{+}$ as a function of the single number $K$ (scalar case, $H=1$, so $I=1$): $P^{+}(K)=(1-K{)}^2{P}^{-}+K^{2}R.$ Expand: $P^{+}(K)=P^{-}-2K{P}^{-}+K^2(P^{-}+R)$. The last term $K^2(P^{-}+R)$ is a parabola opening upward — because $P^{-}+R>0$ can never be negative.

WHY a parabola matters. An upward parabola has exactly one lowest point. So there is exactly one best slider setting — no ambiguity, no second competing minimum. This is why the Kalman gain is unique.

PICTURE. Plot total uncertainty against $K$. It is a smiling U. Too-small $K$ (left) clings to a bad model; too-large $K$ (right) drowns in sensor noise; the bottom is the sweet spot.

---

Step 6 — Find the bottom: set the slope to zero (scalar first, then the matrix)

WHAT (scalar). The bottom of any smooth valley is where the ground is momentarily flat — the slope is zero. Take the derivative of $P^{+}(K)$ with respect to $K$ and set it to $0$: $\frac{d{P}^{+}}{dK}=-2P^{-}+2K(P^{-}+R)=0.$

Read it term by term:

• $-2P^{-}$ — the downhill pull from erasing prediction error.
• $+2K(P^{-}+R)$ — the uphill push from imported noise, growing with $K$.

Balance them: $K(P^{-}+R)=P^{-}⟹\boxed{K=\frac{P^{-}}{P^{-}+R}}.$

WHY a derivative and not guessing. The derivative is the tool that answers "at which $K$ does the valley stop falling and start rising?" — precisely the flat bottom. Nothing else pins down the exact minimum in one clean step.

WHAT (matrix — and why it is not just "put $H$ back"). In real GNC the state is a list of numbers (position, velocity, ...), so $x$ is a vector, $P^{-}$ is a symmetric matrix, and the gain $K$ is a matrix: it has as many rows as there are states and as many columns as there are measurements. The slider is no longer one knob but a whole grid of knobs, so "slope = 0" must become "the trace stops changing in every direction of $K$ at once." We minimise the honest scalar score $J(K)=\mathrm{tr}(P^{+})$. Expanding the Joseph form and abbreviating $S=H{P}^{-}{H}^{\top }+R$: $J(K)=\mathrm{tr}(P^{-})-2\mathrm{tr}(KH{P}^{-})+\mathrm{tr}(KS{K}^{\top }),$ where the two mixed traces merged into one factor of $2$ because $\mathrm{tr}(M)=\mathrm{tr}(M^{\top })$. Now use two standard matrix-calculus facts — the direct generalisation of "$\frac{d}{dK}(-2K{P}^{-})=-2P^{-}$" and "$\frac{d}{dK}(K^{2}S)=2KS$": $\frac{\partial }{\partial K}\mathrm{tr}(KA)=A^{\top },\frac{\partial }{\partial K}\mathrm{tr}(KS{K}^{\top })=2KS(S\text{ symmetric}).$ So the "slope in every direction is zero" condition reads $\frac{\partial J}{\partial K}=-2P^{-}{H}^{\top }+2KS=0⟹KS=P^{-}{H}^{\top }.$ The scalar $K=P^{-}/(P^{-}+R)$ is exactly this with $H=1$ and everything a number — so the matrix result contains the scalar one, it does not merely decorate it.

WHY $S$ can be inverted. To solve $KS=P^{-}{H}^{\top }$ for $K$ we need to undo $S$, i.e. multiply by $S^{-1}$ — which only exists if $S$ is invertible. It is, because $R$ (any real sensor has nonzero noise) is positive-definite and $H{P}^{-}{H}^{\top }$ is positive-semidefinite (it is a covariance mapped through $H$), and (positive-definite) + (positive-semidefinite) is positive-definite, hence invertible. So we are entitled to write: $\boxed{K=P^{-}{H}^{\top }{S}^{-1},S=H{P}^{-}{H}^{\top }+R.}$

PICTURE. The tangent line to the U is flat exactly at $K=P^{-}/(P^{-}+R)$; we mark that point. In the matrix case, "flat" means flat along every axis of the $K$-grid at once.

---

Step 7 — The two extreme sliders (degenerate cases)

WHAT. Push the sensor to its two extremes and watch the slider snap to an end.

Case A — perfect sensor, $R\to 0$ (pink fog collapses to a spike). $K=\frac{P^{-}}{P^{-}+0}=1.$ Slider all the way right: ${\hat{x}}^{+}=z$. We copy the measurement outright. Makes sense — why trust a foggy model over a flawless sensor?

Case B — useless sensor, $R\to \infty$ (pink fog spreads to a flat sheet). $K=\frac{P^{-}}{P^{-}+\infty }\to 0.$ Slider all the way left: ${\hat{x}}^{+}={\hat{x}}^{-}$. We ignore the sensor entirely and keep the prediction.

WHY show both. A reader must never meet a scenario we skipped. These two limits are the guardrails of the slider: it can never demand more than "all sensor" or less than "all model." Everything real lives strictly between.

PICTURE. Two mini-boards: on the left the pink spike drags $K\to 1$; on the right the flat pink sheet lets $K\to 0$.

---

The one-picture summary

Everything above, on a single board: two fogs go in, the slider $K$ picks the blend, the total uncertainty traces a U-shaped valley, and its flat bottom is $K=P^{-}{H}^{\top }{S}^{-1}$, producing an output fog narrower than both inputs.

Recall Feynman: the whole walkthrough in plain words

You have two guesses about one number: a model's guess and a sensor's reading. Draw each as a hill of fog — wide fog means unsure, narrow fog means confident. You want a final guess that sits between them, and a slider called $K$ decides where. Slide toward the sensor and you erase the model's error but drink in the sensor's noise; slide toward the model and you keep the sensor's noise out but trust a shaky model. So the total "how-wrong-am-I" score dips down and then climbs back up — a valley. The best slider sits at the very bottom of that valley, and doing the arithmetic gives $K=P^{-}/(P^{-}+R)$ (or with the sensor map, $K=P^{-}{H}^{\top }{S}^{-1}$). When the state is a list of numbers the slider becomes a grid of knobs, but the same idea holds: turn every knob until the total uncertainty stops falling in every direction — that flat bottom is the gain. We are allowed to divide by $S$ because a real sensor always has some noise, so $S$ is never zero. At the optimum the final fog is narrower than both starting fogs — you now know the number better than either source could tell you alone. Perfect sensor? Slider snaps to the sensor. Useless sensor? Slider snaps to the model. Nothing in between is ever a surprise.

---

Connections

• Parent: 3.5.22 Kalman gain — minimizes trace of covariance (Hinglish)
• Kalman Filter — Predict Step
• Kalman Filter — Update Step
• Covariance Matrices and Uncertainty
• Least Squares Estimation
• Innovation and Residuals
• Joseph Form Covariance Update
• Extended Kalman Filter (EKF) in GNC
• Sensor Fusion in Navigation