Kalman gain — minimizes trace of covariance: Question bank

Reminders of the symbols, so nothing is used before it is meant:

$x$ — the (unknown) true state. $\overset{x}{^}$ — our estimate of it.
$e = x - \overset{x}{^}$ — the estimation error, the vector pointing from our guess to the truth. Its "before-measurement" version is $e^{-} = x - \overset{x}{^}^{-}$ and its "after" version is $e^{+} = x - \overset{x}{^}^{+}$ .
$\overset{x}{^}^{-}, P^{-}$ — the prediction (prior estimate) and its uncertainty (covariance) before seeing the sensor. Here $P^{-} = E [e^{-} (e^{-})^{⊤}]$ .
$\overset{x}{^}^{+}, P^{+}$ — the corrected estimate and its uncertainty after folding in the measurement, $P^{+} = E [e^{+} (e^{+})^{⊤}]$ .
$z = H x + v$ — the measurement; $H$ maps state → measurement space; $v$ is sensor noise with covariance $R$ .
$K$ — the Kalman gain, the "trust dial."
$S = H P^{-} H^{⊤} + R$ — the innovation covariance (spread of the surprise).
$tr (\cdot)$ — the trace, sum of diagonal entries of a matrix.
$J = tr (P^{+})$ — the cost function we minimize over $K$ : the total mean-squared error we want as small as possible.

Figure — Kalman gain — minimizes trace of covariance

True or false — justify

More trust in the measurement always shrinks the final uncertainty.

False. Uncertainty

J = tr (P^{+})

is a U-shaped (parabolic) function of

K

; past the optimum, extra gain lets in more sensor noise than prediction error it removes, so uncertainty grows again (see the figure).

The fused estimate can be more certain than both the prediction and the measurement individually.

True. Combining two independent noisy sources yields

P^{+}

smaller than either

P^{-}

R

— e.g.

P^{-} = 4, R = 1

gives

P^{+} = 0.8

, below both. That gain is the entire reason to fuse.

The simplified update

P^{+} = (I - K H) P^{-}

is valid for any gain

K

you choose.

False. That short form only holds at the optimal $K$ , where the noise term collapses. For an arbitrary

K

you must use the Joseph form

(I - K H) P^{-} (I - K H)^{⊤} + K R K^{⊤}

If the sensor is perfect (

R \to 0

) and it observes the state fully (square, invertible

H

), the filter ignores its own prediction entirely.

True. With

R \to 0

and

H

invertible,

K \to H^{- 1}

(scalar

H = 1

K \to 1

), so

\overset{x}{^}^{+} = H^{- 1} z = x

up to noise — the estimate becomes the measurement and the prior is overridden. If

H

is not invertible, the sensor only pins down the observed part of the state (see Edge cases).

A useless sensor (

R \to \infty

) makes the estimate diverge.

False. As

R \to \infty

S \to \infty

and

K \to 0

, so

\overset{x}{^}^{+} = \overset{x}{^}^{-}

— the filter simply keeps its prediction, calmly, with no harm done.

The Kalman gain minimizes the largest error component of the state.

False. It minimizes the trace — the sum of all diagonal variances (total mean-squared error), not the maximum entry. It optimizes the whole bundle, not the worst axis alone.

The trace of

P

equals the expected squared distance between estimate and truth.

True (given the estimate is unbiased).

tr (P) = E [e^{⊤} e] = E [∥ x - \overset{x}{^} ∥^{2}]

, since summing the diagonal variances is summing

E [e_{i}^{2}]

— exactly the total mean-squared error when

E [e] = 0

A larger prior uncertainty

P^{-}

pushes the gain

K

higher.

True. With less confidence in the prediction, the filter leans on the measurement more;

K = P^{-} H^{⊤} S^{- 1}

grows with

P^{-}

(up to the ceiling set by

R

The Joseph form can accidentally produce a covariance with negative variances.

False. The Joseph form

(I - K H) P^{-} (I - K H)^{⊤} + K R K^{⊤}

is a sum of two positive-semidefinite pieces, so it stays symmetric and PSD — that's why implementers keep it for numerical safety.

The innovation

z - H \overset{x}{^}^{-}

being zero means the estimate is exactly correct.

False. Zero innovation only means the sensor agrees with the prediction; both could still be wrong together. It signals "no surprise," not "no error."

Spot the error

"Since

R

is the sensor noise, the gain is

K = P^{-} H^{⊤} / R

The denominator must be

S = H P^{-} H^{⊤} + R

, not

R

alone. It also includes the prior uncertainty mapped into measurement space — both sources' confidence set the gain.

"The optimum is where

\partial J / \partial K = 0

, so it could be a maximum of the trace."

It cannot. In the scalar case the second derivative is literally

2 S

with

S = H P^{-} H^{⊤} + R > 0

, so it's a genuine minimum. In the matrix case

K

is matrix-valued, so there is no single "

2 S

" number — instead the Hessian (the quadratic form

δ K \mapsto tr (δ K S δ K^{⊤})

) is positive-definite because

S ≻ 0

, which is exactly what guarantees the stationary point is a minimum.

"We can drop the

K R K^{⊤}

term when deriving

P^{+}

because sensor noise is small."

Never drop it for a general

K

— it's what makes the trade-off a parabola. Only at the optimal

K

does it collapse: there

K S = P^{-} H^{⊤}

, so

K R K^{⊤} = K (S - H P^{-} H^{⊤}) K^{⊤}

cancels the extra pieces, leaving

P^{+} = (I - K H) P^{-}

"Cross terms like

E [e^{-} v^{⊤}]

should appear in

P^{+}

They vanish because the prior error

e^{-} = x - \overset{x}{^}^{-}

and the sensor noise

v

are assumed uncorrelated (and both zero-mean), so

E [e^{-} v^{⊤}] = 0

. That is exactly what lets the covariance split into the two clean terms.

"The update

\overset{x}{^}^{+} = \overset{x}{^}^{-} + K (z - H \overset{x}{^}^{-})

is nonlinear in the measurement, so it's optimal only by luck."

It's linear in

z

— the simplest unbiased blend — and for Gaussian noise this linear form is provably optimal, not lucky.

tr (K H P^{-})

and

tr (P^{-} H^{⊤} K^{⊤})

are different terms, so no factor of 2 appears."

They're equal, because the trace of a matrix equals the trace of its transpose; adding two equal terms gives the factor of 2 in the derivative.

Why questions

Why minimize the trace rather than, say, the determinant of

P

Because

tr (P) = E [∥ x - \overset{x}{^} ∥^{2}]

is the total mean-squared error — the natural "how wrong am I" number — and it yields a clean linear-algebra minimization giving the standard Kalman gain.

Why does

tr (P)

equal the sum of the state components' variances?

The diagonal entry

P_{ii} = E [e_{i}^{2}]

is the variance of the

i

-th error component (using

E [e_{i}] = 0

); the trace sums the diagonal, so it sums those variances into one total-error scalar.

Why is the linear update form chosen instead of a general nonlinear correction?

Linear is the simplest unbiased blend, and under Gaussian noise it is exactly optimal, so there's no accuracy left to gain from complicating it.

Why does the trace behave like a parabola (U-shape) in

K

The

K R K^{⊤}

term is quadratic and positive-definite in

K

, while the prediction term shrinks smoothly — their sum is a positive quadratic with one minimum, as the figure's green curve shows.

Why does the

K R K^{⊤}

term collapse exactly at the optimal gain?

At the optimum

K S = P^{-} H^{⊤}

, so substituting back makes the noise term cancel the extra prediction pieces, leaving the tidy

P^{+} = (I - K H) P^{-}

— a coincidence only at the minimizing

K

Why keep the Joseph form in real code if

P^{+} = (I - K H) P^{-}

is simpler?

The Joseph form stays symmetric and positive-semidefinite even with rounding error and slightly non-optimal gains, guarding against numerically "impossible" covariances.

Why does the innovation covariance

S

combine both

H P^{-} H^{⊤}

and

R

The surprise

z - H \overset{x}{^}^{-}

carries uncertainty from the prediction (mapped through

H

) and from the sensor; you must weight the innovation by the total spread of what could cause it.

Why does raising

R

(noisier sensor) lower the gain?

A noisier sensor is less trustworthy, so

S

grows and

K = P^{-} H^{⊤} S^{- 1}

shrinks, pulling the estimate back toward the prediction.

Why must the estimate be unbiased for the trace to mean "total error"?

E [e] \neq = 0

, the covariance

P

measures only the spread around the (wrong) average, not the average miss itself; minimizing

tr (P)

would then leave a systematic bias untouched. Zero-mean error makes

tr (P) = E [∥ e ∥^{2}]

exactly.

Edge cases

What is

K

and

\overset{x}{^}^{+}

when the prediction is perfectly certain,

P^{-} \to 0

K \to 0

, so

\overset{x}{^}^{+} = \overset{x}{^}^{-}

— a flawless prior overrides the sensor entirely, mirror-image of the perfect-sensor case.

If prediction and measurement are equally uncertain (scalar

P^{-} = R

H = 1

), where does the gain land?

K = P^{-} / (P^{-} + R) = 1/2

— the filter splits the difference exactly halfway, as symmetry demands.

What if

H

is non-square (fewer measurements than states, so it cannot be inverted)?

The gain still exists via

K = P^{-} H^{⊤} S^{- 1}

, since only

S = H P^{-} H^{⊤} + R

(a smaller matrix) is inverted, never

H

. The sensor sharpens only the observed directions of the state; unobserved directions keep their prior uncertainty.

Why does the formula use

S^{- 1}

instead of

H^{- 1}

, given

H

may not be invertible?

Because we never need to invert

H

— the derivation minimizes over

K

and inverts only the always-square, positive-definite

S

. The "

K \to H^{- 1}

" statement is a special case that requires a square invertible

H

; the general formula sidesteps it entirely.

What happens to

P^{+}

when

K

is set to zero deliberately?

The Joseph form gives

P^{+} = P^{-}

: no measurement is used, so uncertainty is unchanged — the filter coasts on prediction alone.

Can

P^{+}

ever exceed

P^{-}

for the optimal gain?

No — at the optimum

P^{+} = (I - K H) P^{-}

can only equal or shrink

P^{-}

; measurement information never increases uncertainty when weighted correctly.

What does zero measurement noise and zero prediction uncertainty (

R \to 0, P^{-} \to 0

) imply?

Both sources claim perfect certainty;

S \to 0

makes

K

ill-defined (

0/0

), signalling a degenerate, over-confident setup that must be regularized in practice.

Connections

Kalman gain — minimizes trace of covariance (parent)
Kalman Filter — Update Step
Covariance Matrices and Uncertainty
Innovation and Residuals
Joseph Form Covariance Update
Sensor Fusion in Navigation
Least Squares Estimation
Extended Kalman Filter (EKF) in GNC