Kalman gain — minimizes trace of covariance

3.5.22 · Physics › Guidance, Navigation & Control (GNC)

Set-up: the update step

Measurement step pe hamare paas hai:

Prior (predicted) estimate $\overset{x}{^}^{-}$ with covariance $P^{-}$ .
Measurement $z = H x + v$ , jahan $v$ sensor noise hai, $E [v v^{⊤}] = R$ , state error se uncorrelated.
$H$ state ko → measurement space mein map karta hai.

Hum ek linear corrected estimate banate hain: $\overset{x}{^}^{+} = \overset{x}{^}^{-} + K (z - H \overset{x}{^}^{-}) .$

Derivation from first principles

Step 1 — Error propagate karo. Prior error define karo $e^{-} = x - \overset{x}{^}^{-}$ . Tab $e^{+} = x - \overset{x}{^}^{+} = x - \overset{x}{^}^{-} - K (z - H \overset{x}{^}^{-}) .$ Yeh step kyun? Bas update rule substitute karo — hum track karte hain ki error kaise transform hoti hai.

$z = H x + v$ substitute karo: $e^{+} = e^{-} - K (H x + v - H \overset{x}{^}^{-}) = e^{-} - K H e^{-} - K v = (I - K H) e^{-} - K v .$ Yeh step kyun? $H x - H \overset{x}{^}^{-} = H e^{-}$ group karne se error explicitly expose hota hai; $v$ injected sensor noise hai.

Step 2 — Posterior covariance compute karo. $P^{+} = E [e^{+} e^{+ ⊤}] .$ Kyunki $e^{-}$ aur $v$ uncorrelated hain ( $E [e^{-} v^{⊤}] = 0$ ), cross terms vanish ho jaate hain: $P^{+} = (I - K H) P^{-} (I - K H)^{⊤} + K R K^{⊤} .$ Yeh step kyun? Yeh Joseph form hai — kisi bhi gain $K$ ke liye valid hai, sirf optimal ke liye nahi. Yeh guaranteed symmetric aur positive-semidefinite hai. Yeh "prediction contribution" ko "noise contribution" se cleanly alag karta hai.

Step 3 — Trace minimize karo. Hum $J (K) = tr (P^{+})$ minimize karte hain. Expand karo: $P^{+} = P^{-} - K H P^{-} - P^{-} H^{⊤} K^{⊤} + K (H P^{-} H^{⊤} + R) K^{⊤} .$ Maano $S = H P^{-} H^{⊤} + R$ (innovation covariance). Trace lo aur do matrix-calculus identities use karo:

\frac{\partial}{\partial K}\operatorname{tr}(KBK^\top)=2KB\ (B\text{ symmetric}).$$ Tab $$\frac{\partial J}{\partial K} = -2(HP^-)^\top + 2KS = -2P^-H^\top + 2KS.$$ *Yeh step kyun?* Do mixed terms $\operatorname{tr}(KHP^-)$ aur $\operatorname{tr}(P^-H^\top K^\top)$ equal hain (ek matrix ka trace uske transpose ke trace ke barabar hota hai), isliye factor 2 aata hai. Derivative ko zero set karo: $$KS = P^-H^\top \implies$$ > [!formula] Optimal Kalman gain > $$\boxed{K = P^-H^\top\,(HP^-H^\top + R)^{-1} = P^-H^\top S^{-1}}$$ > Yeh $K$, $\operatorname{tr}(P^+)$ minimize karta hai. Second derivative $2S$ positive-definite hai, isliye yeh genuine **minimum** hai. **Step 4 — Optimum pe simplified posterior.** Optimal $K$ ko $P^+$ mein plug karo. Kyunki $KS=P^-H^\top$ hai, $K R K^\top$ term collapse ho jaata hai aur: $$\boxed{P^+ = (I-KH)P^-.}$$ *Yeh step kyun?* Yeh nice hai, lekin sirf *optimal* $K$ pe hi true hai. Numerical safety ke liye implementers Joseph form rakhte hain. ![[3.5.22-Kalman-gain-—-minimizes-trace-of-covariance.png]] --- ## Limiting cases (Forecast-then-Verify) > [!example] Perfect sensor, $R\to 0$ > **Forecast:** hume sensor pe completely trust karna chahiye. > **Verify:** $K = P^-H^\top(HP^-H^\top)^{-1}$. Scalar $H=1$ mein: $K = P^-/P^- = 1$. Tab $\hat{x}^+ = z$ — exactly measurement. ✅ > [!example] Useless sensor, $R\to\infty$ > **Forecast:** measurement ko ignore karo. > **Verify:** $S\to\infty$ toh $K = P^-H^\top S^{-1}\to 0$. Tab $\hat{x}^+=\hat{x}^-$. ✅ > [!example] Scalar numeric > $P^-=4,\ H=1,\ R=1$. > $S = 4+1=5$, $K = 4/5 = 0.8$. > Nayi covariance $P^+ = (1-0.8)\cdot 4 = 0.8$. > *Yeh step kyun?* Kyunki $P^+=0.8 < 4 = P^-$ aur $<1=R$ hai, fused estimate **dono sources mein se kisi ek se bhi zyada certain hai** — yahi fusion ka poora point hai. --- ## Common mistakes > [!mistake] "Zyada gain always = better estimate." > **Kyun aisa lagta hai:** bada $K$ measurement zyada use karta hai, aur measurements "real data" ki tarah lagte hain. > **Fix:** trace $\operatorname{tr}(P^+)$ $K$ mein ek *parabola* hai ($KRK^\top$ term quadratic aur positive hai). Optimum ke baad, extra $K$ se zyada sensor noise inject hoti hai jitna prediction error remove hota hai. Ek unique sweet spot hota hai. > [!mistake] $P^+$ derive karte waqt noise term bhool jaana. > **Kyun aisa lagta hai:** log $P^+=(I-KH)P^-$ likhte hain aur ise *kisi bhi* $K$ ke liye use karte hain. > **Fix:** woh shortcut sirf **optimal $K$ pe hi valid hai**. Arbitrary $K$ ke liye Joseph form $(I-KH)P^-(I-KH)^\top + KRK^\top$ use karna zaroori hai. > [!mistake] $S^{-1}$ ko $1/R$ treat karna. > **Kyun aisa lagta hai:** $R$ sensor noise hai, toh lagta hai jaise "isi cheez se divide karna hai." > **Fix:** $S = HP^-H^\top + R$ mein propagated prior uncertainty bhi shamil hai. Dono sources ki confidence matter karti hai. --- > [!recall]- Feynman: explain to a 12-year-old > Tum guess kar rahe ho ki ek ball kahan land karegi. Tumhara dost (model) ek jagah predict karta hai; thodi blurry photo (sensor) doosri jagah dikhati hai. Dono exact nahi hain. Kalman gain ek **slider from 0 to 1** hai jo batata hai ki apna guess model ki jagah se kitna photo ki jagah ki taraf move karo. Agar model usually galat hota hai, slider photo ki taraf slide karo; agar photo blurry hai, model ke paas raho. Hum slider ki position wahan choose karte hain jahan hamaara final guess jitna tight ho sake — sabse chota "kitna galat hoon mein" number, jo uncertainty ka **trace** hai. > [!mnemonic] > **"PH over S"** — $K = \dfrac{PH}{S}$. Socho *"measurement ki taraf Push karo Prior-times-H se, Innovation Spread S se roka jaata hai."* Aur trace ek **U-shaped valley** hai: bahut kam $K$ = bure model pe trust; bahut zyada $K$ = noisy sensor pe trust; U ka bottom = optimal. --- ## #flashcards/physics Optimal Kalman gain kaunsi quantity minimize karta hai? ::: Posterior error covariance ka trace $\operatorname{tr}(P^+)$, yaani total mean-squared estimation error. Trace minimize kyun karna sahi hai? ::: $\operatorname{tr}(P)=\mathbb{E}[\|x-\hat x\|^2]$, yeh sab state components ka total mean-squared error hai. Joseph-form posterior covariance batao (kisi bhi $K$ ke liye valid). ::: $P^+=(I-KH)P^-(I-KH)^\top+KRK^\top$. Optimal Kalman gain formula do. ::: $K=P^-H^\top(HP^-H^\top+R)^{-1}=P^-H^\top S^{-1}$. Innovation covariance $S$ kya hai? ::: $S=HP^-H^\top+R$: predicted uncertainty measurement space mein mapped plus sensor noise. Optimal gain pe simplified $P^+$? ::: $P^+=(I-KH)P^-$ — sirf optimum pe valid hai. $R\to0$ (perfect sensor) hone par $K$ ka kya hota hai? ::: $K\to H^{-1}$ (scalar: 1); estimate measurement ke equal ho jaata hai. $R\to\infty$ (useless sensor) hone par $K$ ka kya hota hai? ::: $K\to0$; estimate prediction ke equal ho jaata hai. Trace mein $K$ mein unique minimum kyun hota hai? ::: $KRK^\top$ term ise $K$ mein positive-definite quadratic banata hai; second derivative $2S\succ0$. Trace minimize karne ke liye kaun si derivative identity use hoti hai? ::: $\partial_K\operatorname{tr}(KBK^\top)=2KB$ aur $\partial_K\operatorname{tr}(KA)=A^\top$. --- ## Connections - [[Kalman Filter — Predict Step]] - [[Kalman Filter — Update Step]] - [[Covariance Matrices and Uncertainty]] - [[Least Squares Estimation]] (Kalman = recursive weighted least squares) - [[Innovation and Residuals]] - [[Joseph Form Covariance Update]] - [[Extended Kalman Filter (EKF) in GNC]] - [[Sensor Fusion in Navigation]] ## 🖼️ Concept Map ```mermaid flowchart TD PRED[Prediction x-hat-minus, P-minus] -->|blended by K| EST[Corrected estimate x-hat-plus] MEAS[Measurement z = Hx + v] -->|forms innovation| INNOV[Innovation z - H x-hat-minus] INNOV -->|scaled by K| EST K[Kalman gain K] -->|controls trust| EST EST -->|error e = x - x-hat| ERR[Estimation error] ERR -->|expected outer product| P[Error covariance P] P -->|sum of diagonal| TR[Trace tr of P] TR -->|equals total mean-squared error| MSE[Total MSE] P -->|Joseph form| JOS[P-plus = I-KH P-minus I-KH transpose + K R K transpose] JOS -->|larger K shrinks prediction term| TRADE[Trade-off] JOS -->|larger K grows K R K transpose| TRADE TRADE -->|minimize trace| K ```

WHAT are we minimizing?

Set-up: the update step

Derivation from first principles