Extended Kalman Filter (EKF) — linearization, Jacobians: Question bank

Before we start, here is every symbol this page uses, in plain words, so nothing appears unearned:

$f$ is the nonlinear motion function: it takes your current state guess and returns the next one.
$h$ is the nonlinear measurement function: it takes a state and returns what your sensor should read.
$F$ and $H$ are the Jacobians — the matrices of slopes (partial derivatives) of $f$ and $h$ . They are the local "straight-line stand-ins" for the curves, valid only right next to the point where you compute them.
$P$ is the covariance: a blob describing how uncertain (and how correlated) your state estimate is.
$\hat{x}_{k}^{-}$ is the predicted state (after motion, before the measurement); $\hat{x}_{k - 1}$ is the previous estimate.
$Q$ is the process-noise covariance: how much extra uncertainty the world injects each step because your motion model $f$ is imperfect. It is added to the predicted covariance.
$R$ is the measurement-noise covariance: how noisy your sensor is. Big $R$ means "don't trust this reading much."
$S$ is the innovation covariance: the total spread of the "surprise" between what you measured and what you predicted, built from the state spread pushed into sensor space plus the sensor noise, $S = H P^{-} H^{⊤} + R$ .
$K$ is the Kalman gain: the fraction of the surprise you actually apply to correct your estimate, $K = P^{-} H^{⊤} S^{- 1}$ . It is recomputed every step and balances trust between prediction and sensor.

True or false — justify

True: mean propagates through nonlinear f, covariance through the Jacobian

True — the best guess of the next state is

f

of the current best guess, but spread transforms by the local sensitivity, so

P^{-} = F P F^{⊤} + Q

. Mean uses the curve, covariance uses the tangent.

True: if f is already linear, the EKF reduces exactly to the linear Kalman filter

True — the Jacobian of a linear map

F x

is the constant matrix

F

itself, so there is no approximation left and the machinery is identical to the Kalman Filter (linear).

True: the Jacobian is the best possible constant-matrix approximation of a curve at a point

True — it is the first-order Taylor term, and no constant matrix matches the local slopes better. That is precisely why the EKF chooses it over any other matrix.

True: two different states can produce the same Jacobian

True — the slope of a curve can repeat (e.g. the pendulum's

- \frac{g}{L} cos θ

is equal at

θ

and

- θ

). The Jacobian describes local behaviour, not identity of the state.

False: because the EKF linearizes, it can only handle mildly nonlinear systems

Half-false — it handles strong nonlinearity fine as long as the curve is nearly straight over one uncertainty blob. Trouble comes from large curvature-times-uncertainty, not from nonlinearity per se.

False: making the process-noise covariance

Q

zero makes the filter more accurate

False — with

Q = 0

the covariance shrinks and the filter grows over-confident, ignores new data, and diverges.

Q

deliberately keeps the estimate humble about model errors.

True: the EKF covariance can underestimate the true uncertainty

True — the linear tangent throws away the curvature that actually spreads a Gaussian out, so

P

often comes out optimistically small. This is a known EKF failure mode the UKF mitigates.

False: the innovation

y_{k} = z_{k} - H \hat{x}_{k}^{-}

False — the innovation must use the nonlinear prediction

h (\hat{x}_{k}^{-})

H

appears only inside the innovation covariance

S

and the gain

K

; using

H \hat{x}^{-}

would linearize the mean, which we specifically refuse to do.

Spot the error

"Compute

H

at the previous estimate

\hat{x}_{k - 1}

to reuse the point where you found

F

Wrong evaluation point —

h

must be linearized at the predicted state

\hat{x}_{k}^{-}

, because that is the state the measurement is being compared against. This is the classic #1 EKF bug.

"Since the mean uses

f

, the covariance should be

f (P)

for symmetry."

There is no meaningful "

f (P)

" —

P

is a second-moment object, not a state, so it transforms by the linear sensitivity

F P F^{⊤}

, never by feeding a matrix into

f

"A radar bearing residual of

35 9^{\circ} - 1^{\circ} = 35 8^{\circ}

means the filter is way off."

The residual was never wrapped — the true angular error is

- 2^{\circ}

. Angle residuals must be wrapped to

(- π, π]

; see atan2 & Angle Wrapping.

"My initial guess is rough but a filter always converges, so I'll trust it."

The tangent-line approximation is only valid near

\hat{x}

; a bad linearization point yields garbage

F, H

and the filter can diverge outright. Seed a good

\hat{x}_{0}

"The Kalman gain

K

is a fixed tuning constant I set once."

K

is recomputed every step from

P^{-}, H

and the measurement-noise covariance

R

(via

K = P^{-} H^{⊤} S^{- 1}

); it automatically shrinks when the sensor is noisy and grows when the sensor is trusted. Hard-coding it defeats the whole filter.

"I'll use

arctan (y / x)

for the bearing measurement to keep it simple."

arctan

collapses quadrants II and III onto I and IV, giving the wrong angle for half the plane. Use

atan2 (y, x)

, which reads both signs and covers all four quadrants.

P_{k} = (I - K_{k} H_{k}) P_{k}^{-}

came out slightly non-symmetric, that's fine."

Numerically it drifts and can lose positive-definiteness, breaking the filter silently. Use the Joseph-form update or force symmetry — covariance must stay symmetric and positive-definite.

Why questions

Why does the mean go through

f

but the covariance through

F

The single most-likely next state is literally

f

of the most-likely current state; but uncertainty is about how nearby points spread, and to first order they spread by the slope matrix

F

. Different questions, different tools.

Why is

F

evaluated at

\hat{x}_{k - 1}

but

H

\hat{x}_{k}^{-}

F

linearizes the motion you are about to apply, so it uses the state before motion;

H

linearizes the sensor around the state you arrived at, so it uses the state after motion. Each tangent is taken where its curve is actually used.

Why can't we just push the whole Gaussian through the curve exactly?

A Gaussian pushed through a curve comes out skewed and non-Gaussian, and the KF equations only speak the language of means and covariances. Linearizing keeps the output Gaussian so the math stays valid.

Why does the pendulum Jacobian contain

cos θ

and why does that matter?

With

g

the gravitational acceleration and

L

the pendulum length, it is the derivative of the restoring term

- \frac{g}{L} sin θ

. Near

θ = 0

it equals

- \frac{g}{L}

(stiff, linear oscillator), but near

θ = π /2

it vanishes — the linearized stiffness disappears, which the state-dependent Jacobian faithfully reports.

Why prefer the UKF when nonlinearity is severe?

The EKF keeps only the first Taylor term and discards curvature; the UKF instead pushes a set of sample points through the true

f, h

, capturing curvature without ever forming a Jacobian.

Why is the range partial

\partial r / \partial x = x / r

equal to a cosine?

x / r

is the cosine of the target's bearing angle, so it says "range grows fastest when you move straight toward the sensor and not at all when you move sideways" — a geometric sanity check baked into the Jacobian.

Why does a too-small measurement-noise covariance

R

sometimes make the filter jittery?

Small

R

means "the sensor is nearly perfect," so the gain

K

grows large and the estimate chases every noisy reading. The filter overtrusts noise and rattles instead of smoothing.

Edge cases

What is the range Jacobian at the origin

(0, 0)

Undefined —

r = 0

puts

r

and

r^{2}

in denominators, so the bearing is meaningless and the covariance update blows up. A target sitting on the sensor is a genuine singularity to guard against.

What happens to the bearing partials as the target moves far away (large

r

\partial θ / \partial x = - y / r^{2} \to 0

: distant targets barely change bearing when they move, so angle measurements become weak at long range and the filter leans on range instead.

If the Jacobian

F

turns out to have no state-dependence, what does it tell you?

Your dynamics are actually linear and you never needed an EKF at all — the plain Kalman Filter (linear) is exact and cheaper.

What if

\hat{x}_{k}^{-}

lands exactly where

h

is non-differentiable (e.g. a

∣ \cdot ∣

kink)?

The Jacobian is undefined there; the tangent-line premise fails outright. You must use a sub-gradient, smooth the model, or switch to a derivative-free filter.

At the pendulum's inverted point

θ = π

, what does the Jacobian say?

- \frac{g}{L} cos π = + \frac{g}{L}

, a positive stiffness meaning the equilibrium is unstable — the linearization correctly flips sign and predicts runaway growth away from the top.

What if the innovation covariance

S_{k}

is nearly singular?

The gain

K = P^{-} H^{⊤} S^{- 1}

needs

S^{- 1}

; a near-singular

S

(from tiny

R

and a degenerate

H

) makes the gain explode. Regularize

R

or check that

H

isn't rank-deficient.