Kalman gain — minimizes trace of covariance: Exercises

3.5.22 · D4 · Physics › Guidance, Navigation & Control (GNC) › Kalman gain — minimizes trace of covariance

Shuru karne se pehle, yahan ek toolbox hai, ek hi jagah par, taaki koi bhi symbol unexplained na rahe.

Yahan, $\overset{x}{^}^{-}$ hamari prediction hai aur $P^{-}$ iska uncertainty (variance) hai; $z$ sensor reading hai; $H$ ek state ko woh banata hai jo sensor ko padhna chahiye; $R$ sensor noise variance hai; $K$ gain hai, $[0, 1]$ mein ek dial (scalar case mein) jo batata hai ki prediction se measurement ki taraf kitna slide karna hai.

Neeche di gayi figure Problem 2.2 ke numbers use karke "slider" idea ko concrete banati hai: $10$ par ek prediction, $15$ par ek measurement, aur fused estimate $14$ par land karta hai kyunki gain $K = 0.8$ hai. Dekho ki orange diamond kaise dono sources ke beech baitha hai, measurement ki taraf khicha hua.

Figure — Kalman gain — minimizes trace of covariance

L1 — Recognition

Problem 1.1

Scalar update mein, inme se kaun sa innovation hai, aur kaun sa innovation covariance hai? (a) $z - H \overset{x}{^}^{-}$ (b) $H P^{-} H^{⊤} + R$ (c) $P^{-} H^{⊤} S^{- 1}$

Recall Solution 1.1

WHAT: names ko formulas se match karo.

(a) $z - H \overset{x}{^}^{-}$ innovation $y$ hai: measurement minus jo hum measure hone ki expect karte the.
(b) $H P^{-} H^{⊤} + R$ innovation covariance $S$ hai: prior uncertainty measurement space mein push ki gayi, plus sensor noise.
(c) $P^{-} H^{⊤} S^{- 1}$ gain $K$ hai — pucha nahi gaya, par pehchanna accha hai. WHY it matters: $y$ batata hai ki tum kitne surprised ho; $S$ batata hai ki uss size ka surprise kitna surprising hai. Tum hamesha pehle wale ko doosre se divide karte ho ( $K$ ke zariye).

Problem 1.2

Ek scalar filter mein $P^{-} = 9$ , $H = 1$ , $R = 0$ (ek perfect sensor) hai. Derivatives compute kiye bina, $K$ kya hona chahiye, aur $\overset{x}{^}^{+}$ kya hoga?

Recall Solution 1.2

WHAT: ek perfect sensor ( $R = 0$ ) ka koi noise nahi, isliye hume use poora trust karna chahiye. $S = H P^{-} H^{⊤} + R = 1 \cdot 9 \cdot 1 + 0 = 9$ . Phir $K = P^{-} H^{⊤} S^{- 1} = 9 \cdot 1 \cdot \frac{1}{9} = 1$ . $K = 1$ ke saath: $\overset{x}{^}^{+} = \overset{x}{^}^{-} + 1 \cdot (z - \overset{x}{^}^{-}) = z$ . Estimate measurement ke barabar ho jaata hai. ✅ WHY: $K = 1$ ka matlab hai ki slider poori tarah measurement ki taraf push ho gayi hai.

L2 — Application

Problem 2.1

Scalar: $P^{-} = 4$ , $H = 1$ , $R = 1$ . $S$ , $K$ , aur $P^{+}$ compute karo.

Recall Solution 2.1

Step 1 ( $S$ banao): $S = 1 \cdot 4 \cdot 1 + 1 = 5$ . Step 2 (gain): $K = P^{-} H^{⊤} S^{- 1} = 4 \cdot 1 \cdot \frac{1}{5} = 0.8$ . Step 3 (posterior): optimum par, $P^{+} = (1 - K H) P^{-} = (1 - 0.8) \cdot 4 = 0.8$ . WHY the sanity check: $P^{+} = 0.8$ $P^{-} = 4$ aur $R = 1$ dono se chhota hai. Fusion dono sources mein se kisi ek se bhi behtar hai — yahi poora point hai.

Problem 2.2

Same numbers jaise 2.1 ( $K = 0.8$ ). Prediction $\overset{x}{^}^{-} = 10$ thi, sensor $z = 15$ read karta hai. Corrected estimate $\overset{x}{^}^{+}$ nikalo.

Recall Solution 2.2

Step 1 (innovation): $y = z - H \overset{x}{^}^{-} = 15 - 10 = 5$ . Step 2 (correct): $\overset{x}{^}^{+} = \overset{x}{^}^{-} + K y = 10 + 0.8 \cdot 5 = 14$ . WHY it lands where it does: answer $14$ prediction $10$ aur measurement $15$ ke beech baitha hai, sensor ke zyada paas kyunki $K = 0.8 > 0.5$ ka matlab hai hum yahan sensor par zyada trust karte hain. Yeh exactly upar wali slider figure hai.

Problem 2.3

Verify karo ki Joseph form wohi $P^{+}$ deta hai jo Problem 2.1 mein optimal $K = 0.8$ ke liye shortcut ne diya.

Recall Solution 2.3

Joseph: $P^{+} = (1 - K H)^{2} P^{-} + K^{2} R = (0.2)^{2} \cdot 4 + (0.8)^{2} \cdot 1 = 0.16 + 0.64 = 0.8$ . ✅ Shortcut: $(1 - K H) P^{-} = 0.2 \cdot 4 = 0.8$ . ✅ Dono agree karte hain — kyunki $K$ optimal hai.

L3 — Analysis

Problem 3.1

$P^{-} = 4$ , $R = 1$ , $H = 1$ wale scalar case ke liye, $P^{+} (K) = (1 - K)^{2} \cdot 4 + K^{2} \cdot 1$ ko $K$ ke function ke roop mein likho. Dikhao ki yeh ek parabola hai, differentiate karke iska minimum nikalo, aur confirm karo ki $K = 0.8$ milta hai.

Recall Solution 3.1

Step 1 (expand): $P^{+} (K) = 4 (1 - 2 K + K^{2}) + K^{2} = 4 - 8 K + 4 K^{2} + K^{2} = 5 K^{2} - 8 K + 4$ . WHAT this is: $K$ mein ek parabola jo upar ki taraf khulti hai (coefficient $5 > 0$ ), isliye iska ek unique minimum hai — yeh parent note ka "U-shaped valley" hai. Step 2 (differentiate): $\frac{d P ^{+}}{d K} = 10 K - 8$ . Zero set karo: $10 K = 8 \Rightarrow K = 0.8$ . ✅ Step 3 (minimum par value): $P^{+} (0.8) = 5 (0.64) - 8 (0.8) + 4 = 3.2 - 6.4 + 4 = 0.8$ . ✅ WHY differentiate here: trace ko minimize karna is is parabola ki derivative ko zero set karna hai — general matrix derivation scalar case mein exactly yahi collapse hoti hai. Coefficient $10 = 2 S$ positive second derivative hai jo guarantee karta hai ki yeh minimum hai, maximum nahi.

Neeche di gayi figure exactly yahi parabola plot karti hai. Violet curve trace karo: yeh orange dot ( $K = 0.8$ , $P^{+} = 0.8$ ) par bottom out karta hai, $K = 0$ par $P^{+} = 4$ tak uthta hai (model par over-trusting, navy dot) aur $K = 1$ par $P^{+} = 1$ tak wapas uthta hai (sensor par over-trusting, magenta dot). Ek valley, ek optimum.

Problem 3.2

Maano ek engineer $K = 1$ (sensor par poora trust) hard-code karta hai chahe $R = 1$ , $P^{-} = 4$ , $H = 1$ ho. Joseph form use karke actual resulting $P^{+}$ compute karo aur optimal $0.8$ se compare karo.

Recall Solution 3.2

Joseph zaroori hai ( $K = 1$ optimal nahi hai): $P^{+} = (1 - 1)^{2} \cdot 4 + 1^{2} \cdot 1 = 0 + 1 = 1$ . Compare karo: optimal ne $0.8$ diya tha. Toh sensor par over-trusting se $P^{+} = 1 > 0.8$ milta hai — worse, kyunki humne sensor noise $R = 1$ sab absorb kar liya aur prediction ki help throw away kar di. WHY: $K = 1$ par parabola $5 K^{2} - 8 K + 4$ deta hai $5 - 8 + 4 = 1$ , valley ke bottom se aage (figure mein magenta dot). Extra gain ab prediction error remove karne se zyada tezi se noise add karta hai.

L4 — Synthesis

Problem 4.1

Do sensors same scalar state read karte hain, independently: $z_{1}$ variance $R_{1} = 2$ ke saath aur $z_{2}$ variance $R_{2} = 6$ ke saath. Ek "flat" prior se shuru karke (infinite $P^{-}$ , matlab koi useful prediction nahi), unhe dono baar sequentially Kalman update apply karke fuse karo aur final variance nikalo. Phir classic inverse-variance-weighting result $P_{final} = (\frac{1}{R _{1}} + \frac{1}{R _{2}})^{- 1}$ se compare karo.

Recall Solution 4.1

Step 1 — infinite prior ke saath pehla measurement. Savdhan rehna hoga: $P^{-} \to \infty$ ke saath shortcut $(1 - K H) P^{-}$ indeterminate form $0 \cdot \infty$ hai, isliye hum Joseph form $P_{1} = (1 - K)^{2} P^{-} + K^{2} R_{1}$ use karte hain (kisi bhi $K$ ke liye valid, koi cancellation assumed nahi). Optimal gain $K = \frac{P ^{-}}{P ^{-} + R _{1}}$ hai. Substitute karo: $P_{1} = (\frac{R _{1}}{P ^{-} + R _{1}})^{2} P^{-} + (\frac{P ^{-}}{P ^{-} + R _{1}})^{2} R_{1} = \frac{R _{1}^{2} P ^{-} + P ^{- 2} R _{1}}{( P ^{-} + R _{1} ) ^{2}} = \frac{R _{1} P ^{-} ( R _{1} + P ^{-} )}{( P ^{-} + R _{1} ) ^{2}} = \frac{R _{1} P ^{-}}{P ^{-} + R _{1}} .$ Ab $P^{-} \to \infty$ lo: top aur bottom ko $P^{-}$ se divide karo, milta hai $\frac{R _{1}}{1 + R _{1} / P ^{-}} \to R_{1}$ . Toh step 1 ke baad, $P_{1} = R_{1} = 2$ — pehla measurement simply estimate ban jaata hai, par Joseph form ne hume $0 \cdot \infty$ ki koi bhi trickery ke bina wahan pahuncha diya. Step 2 — $z_{2}$ fuse karo: ab $P^{-} = P_{1} = 2$ , $R = R_{2} = 6$ , $H = 1$ use karo. $S = 2 + 6 = 8$ , $K = 2/8 = 0.25$ , $P_{2} = (1 - 0.25) \cdot 2 = 1.5$ . Inverse-variance weighting se cross-check: $(\frac{1}{2} + \frac{1}{6})^{- 1} = (\frac{3}{6} + \frac{1}{6})^{- 1} = (\frac{4}{6})^{- 1} = \frac{6}{4} = 1.5$ . ✅ WHY this is beautiful: Kalman filter, recursively apply kiya, optimal least-squares / sensor fusion weighting reproduce karta hai. Har measurement ek ek karke fold in hoti hai, phir bhi answer unhe ek saath process karne se match karta hai.

Problem 4.2

Ek 2-state filter position aur velocity track karta hai. Prior covariance aur measurement model hain $P^{-} = [4001], H = [10], R = 2.$ Sirf position measure ki jaati hai (toh $R$ yahan ek single scalar hai). $S$ , gain $K$ (ek $2 \times 1$ vector), aur posterior $P^{+}$ compute karo.

Recall Solution 4.2

Step 1 ( $H P^{-}$ ): $H P^{-} = [10] [4001] = [40]$ . Step 2 ( $S$ ): $S = H P^{-} H^{⊤} + R = [40] [10] + 2 = 4 + 2 = 6$ . Step 3 ( $K = P^{-} H^{⊤} S^{- 1}$ ): $P^{-} H^{⊤} = [40]$ , toh $K = \frac{1}{6} [40] = [0.667 0]$ . Step 4 ( $P^{+} = (I - K H) P^{-}$ ): yahan $I$ $2 \times 2$ identity hai. $K H = [0.667 0] [10] = [0.667 0 00]$ , toh $I - K H = [0.333 0 01]$ . $P^{+} = [0.333 0 01] [4001] = [1.333 0 01]$ . WHY velocity untouched rehti hai: $K$ ki doosri entry $0$ hai, toh velocity variance $1$ par rehti hai. Ek position sensor velocity ko directly improve nahi kar sakta jab dono $P^{-}$ mein uncorrelated hain (off-diagonals zero hain). Position variance $4 \to 1.333$ tak girti hai. Dekho Covariance Matrices and Uncertainty.

L5 — Mastery

Problem 5.1

Scalar case ke liye prove karo ki optimal $K = \frac{P ^{-}}{P ^{-} + R}$ ( $H = 1$ lete hue) hamesha finite positive $R$ ke liye $0 \leq K < 1$ satisfy karta hai, aur ki $P^{+} < min (P^{-}, R)$ . Dono facts interpret karo.

Recall Solution 5.1

Bound on $K$ : $H = 1$ ke saath, $K = \frac{P ^{-}}{P ^{-} + R}$ . Kyunki $P^{-} > 0$ aur $R > 0$ , numerator positive hai aur strictly denominator se chhota hai, toh $0 < K < 1$ . Jaise $R \to 0$ , $K \to 1$ ; jaise $R \to \infty$ , $K \to 0$ . Dial strictly $[0, 1]$ ke andar rehta hai — kabhi bhi kisi ek source par over-trust nahi karta. ✅ Posterior: $P^{+} = (1 - K) P^{-} = \frac{R}{P ^{-} + R} P^{-} = \frac{P ^{-} R}{P ^{-} + R}$ . Yeh harmonic-style combination hai (product over sum). Kyunki $\frac{P ^{-} R}{P ^{-} + R} < P^{-}$ (numerator aur denominator insight multiply karo: $\frac{R}{P ^{-} + R} < 1$ ) aur symmetrically $< R$ , hume $P^{+} < min (P^{-}, R)$ milta hai. ✅ Interpretation: do independent estimates fuse karna hamesha strictly akele best single ek se behtar hai — certainty parallel resistors ki tarah add hoti hai: $\frac{1}{P ^{+}} = \frac{1}{P ^{-}} + \frac{1}{R}$ .

Problem 5.2

$P^{-} = 4$ , $R = 1$ use karke, numerically confirm karo ki parallel-resistor identity $\frac{1}{P ^{+}} = \frac{1}{P ^{-}} + \frac{1}{R}$ holds karta hai, aur batao ki yeh kaafi saare sensors stack karne ke liye kya matlab rakhta hai.

Recall Solution 5.2

$\frac{1}{P ^{-}} + \frac{1}{R} = \frac{1}{4} + \frac{1}{1} = \frac{5}{4} = 1.25$ . Toh $P^{+} = \frac{1}{1.25} = 0.8$ . ✅ Problem 2.1 ke $P^{+} = 0.8$ se match karta hai. Meaning: har naya independent measurement apni precision $\frac{1}{R}$ (information kehlaati hai) add karta hai. $n$ identical sensors of noise $R$ stack karo aur fused precision $\frac{1}{P ^{-}} + \frac{n}{R}$ ho jaati hai — uncertainty $n$ badhne ke saath zero ki taraf shrink hoti hai. Yeh "information adds" view EKF aur saari recursive fusion ke neeche hai.

Recall Poori ladder ka Feynman recap

L1: parts ( $y$ , $S$ , $K$ ) ko naam do taaki koi bhi symbol stranger na rahe. L2: numbers plug in karo aur dekho $\overset{x}{^}^{+}$ kaise dono sources ke beech land karta hai, gain ki size se khicha hua. L3: dekho $P^{+}$ ka trace ek U-shaped parabola hai jiska exactly ek bottom hai — optimum ek balance hai, extreme nahi. L4: measurements ek ek karke fold in karo aur classic inverse-variance (least-squares) weighting recover karo. L5: prove karo ki fusion hamesha ek win hai kyunki precisions (information) parallel resistors ki tarah add hoti hain, toh $P^{+}$ sirf shrink ho sakta hai. Same slider, paanch depths — "pieces pehchaano" se "prove karo ki yeh kabhi hurt nahi kar sakta" tak.

Connections

Kalman gain — minimizes trace of covariance (parent)
Kalman Filter — Update Step
Kalman Filter — Predict Step
Covariance Matrices and Uncertainty
Least Squares Estimation
Innovation and Residuals
Joseph Form Covariance Update
Extended Kalman Filter (EKF) in GNC
Sensor Fusion in Navigation