Visual walkthrough — Kalman gain — minimizes trace of covariance

3.5.22 · D2 · Physics › Guidance, Navigation & Control (GNC) › Kalman gain — minimizes trace of covariance

---

Step 1 — Ek number line par do blurry guesses

KYA HAI. Socho hum ek number jaanna chahte hain: usse $x$ bolte hain (maan lo, rocket ki altitude metres mein). Humein $x$ kabhi exactly nahi bataya jaata. Isliye hamare paas uske baare mein do dhundli raayein hain.

• Ek prediction ${\hat{x}}^{-}$ — chhota minus sign matlab "sensor dekhne se pehle." Yeh physics model se aayi.
• Ek measurement $z$ — ek number jo sensor ne report kiya.

YE KYUN shuru karte hain. Kalman filter mein sab kuch in do dhundli raayion ko combine karke ek teez raayi banane ke baare mein hai. Combine karne se pehle hume inhe imaandaari se draw karna hoga: sharp points ki tarah nahi, balki bell-shaped fog ki tarah. Chauda fog matlab "mujhe pata nahi"; teeka fog matlab "mujhe confidence hai."

PICTURE. Niche, blue fog prediction hai aur pink fog measurement hai. Horizontal axis $x$ ki value hai. Har hill ki chauraayi uski uncertainty hai.

---

Step 2 — Error wahi hai jo hum actually shrink karna chahte hain

KYA HAI. Hume $x$ nahi dikhta, toh hum seedha "main kitna galat hoon?" measure nahi kar sakte. Lekin hum iske baare mein soch sakte hain. Error define karo: $e=\underset{\text{true value}}{\underbrace{x}}-\underset{\text{our guess}}{\underbrace{\hat{x}}}.$ Agar guess perfect hai, $e=0$. Zyada guess kiya toh $e$ negative hai; kum kiya toh positive.

Khaaskar, pre-update error — yaani prediction ka error, sensor ko touch karne se pehle — ka apna naam hai aur ${\hat{x}}^{-}$ se match karta minus sign bhi: $e^{-}=x-{\hat{x}}^{-}.$ Iska average squared size exactly Step 1 wala prediction fog $P^{-}$ hai: $P^{-}=\mathbb{E}[e^{-}{e}^{-\top }]$.

YEH QUANTITY KYUN. Agar hum ek guaranteed single number par zor dein toh "truth se door" minimise nahi kar sakte — noise har trial ko alag banata hai. Isliye hum isse average mein minimise karte hain, kaafi imagined trials par. Imaandaar score hai mean squared error: $P=\mathbb{E}[e{e}^{\top }],\mathrm{tr}(P)=\mathbb{E}[\Vert e{\Vert }^2].$ Yahan $\mathbb{E}[\cdot ]$ matlab "noise kitne bhi tarike se khelti, sab par average," aur $\mathrm{tr}$ (matlab trace) matlab "diagonal add karo" — ek number ke liye yeh sirf variance hi hai. Error ko square karte hain taaki $-3$ se galat hona aur $+3$ se galat hona equally bura lage.

PICTURE. Error $e$ true spike (yellow) aur hamare guess ke beech ka gap hai. Us squared gap ka fog par average karne se number $P$ milta hai jise hum chhota karna chahte hain.

---

Step 3 — Blend rule: do guesses ke beech ek slider

KYA HAI. Hum apna corrected guess ${\hat{x}}^{+}$ (plus matlab "sensor use karne ke baad") is tarah banate hain: ${\hat{x}}^{+}=\underset{\text{start at prediction}}{\underbrace{{\hat{x}}^{-}}}+\underset{\text{slider}}{\underbrace{K}}\left(\underset{\text{the surprise}}{\underbrace{z-H{\hat{x}}^{-}}}\right).$

Har piece ko jahan wo baithe wahan padhte hain:

• ${\hat{x}}^{-}$ — hum prediction se shuru karte hain.
• $H{\hat{x}}^{-}$ — $H$ ek state ko "sensor ko kya padhna chahiye" mein badalta hai. Altitude ke liye altimeter ke saath, $H=1$: state aur reading same units mein hain. Toh $H{\hat{x}}^{-}$ hai "jo reading hum expect karte the." ($H$ measurement map hai; yeh naam yaad rakho, Step 6 mein phir aayega.)
• $z-H{\hat{x}}^{-}$ — innovation: measured reading minus expected reading. Yahi surprise hai. Agar yeh zero hai, sensor ne kuch naya nahi bataya.
• $K$ — gain, haara slider. $K=0$ matlab "surprise ignore karo, prediction rakho." $K=1$ (jab $H=1$) matlab "seedha measurement par jaao."

YEH EXACT FORM KYUN. Yeh sabse simple rule hai jo unbiased hai: agar dono fogs average mein truth par centered hain, toh yeh blend bhi hai, kisi bhi $K$ ke liye. Iska ek sundar geometric meaning bhi hai — picture dekho.

PICTURE. Corrected guess prediction aur measurement ke beech line segment par rehta hai. $K$ decide karta hai ki us segment par hum kahan land karte hain.

---

Step 4 — Fog kaise transform hoti hai: the Joseph form

KYA HAI. Hum poochhte hain: agar slider $K$ se blend karein, toh nayi fog $P^{+}$ kitni chaudi hogi? Error ko blend ke through track karo. Post-update error $e^{+}=x-{\hat{x}}^{+}$ se shuru karo, Step 3 ka blend rule aur sensor model $z=Hx+v$ substitute karo (jahan $v$ sensor ka random noise hai, spread $R$ ke saath). Step 2 mein define kiye pre-update error $e^{-}=x-{\hat{x}}^{-}$ ka use karte hue, yeh collapse ho jaata hai: $e^{+}=(I-KH)e^{-}-Kv.$

Yahan $I$ identity hai — "kuch-na-karo" element, scalar case mein number $1$ (aur general mein identity matrix), taaki $I{e}^{-}=e^{-}$. Term by term:

• $(I-KH)e^{-}$ — bacha hua prediction error. Factor $(I-KH)$ zero ki taraf shrink hota hai jaise $K$ badhta hai: sensor par lean karna purana error mitaata hai.
• $-Kv$ — naya import hua sensor noise. Yeh $K$ ke saath badhta hai: sensor jitna zyada sunenge, uska utna zyada noise swallow karenge.

CROSS-TERMS KYUN GAYAB HO JAATE HAIN. Nayi variance $P^{+}=\mathbb{E}[e^{+}{e}^{+\top }]$ paane ke liye $e^{+}$ ko uske apne transpose se multiply karke average lete hain. Multiply karne par chaar blocks aate hain; unme se do "cross" blocks hain $-(I-KH)\mathbb{E}[e^{-}{v}^{\top }]K^{\top }$ aur uska mirror. Yahan ek key fact hai jo hume batana padega, kyunki poora formula issi par tikaa hai:

Un cross blocks ke jaate hi, sirf do "self" blocks bachte hain: $\boxed{P^{+}=(I-KH)P^{-}(I-KH{)}^{\top }+KR{K}^{\top }.}$ Yeh Joseph form hai (dekho Joseph Form Covariance Update). Kisi bhi slider position $K$ ke liye yeh honest hai.

PICTURE. Do contributions, ulti direction mein khichte hain: blue "prediction-error" term $K$ badhne par girta hai; pink "noise" term chadhta hai.

---

Step 5 — Uncertainty ek U-shaped valley hai

KYA HAI. $P^{+}$ ko single number $K$ ke function ki tarah likho (scalar case, $H=1$, toh $I=1$): $P^{+}(K)=(1-K{)}^2{P}^{-}+K^{2}R.$ Expand karo: $P^{+}(K)=P^{-}-2K{P}^{-}+K^2(P^{-}+R)$. Aakhiri term $K^2(P^{-}+R)$ ek parabola hai jo upar khulti hai — kyunki $P^{-}+R>0$ kabhi negative nahi ho sakta.

PARABOLA KYUN MATTER KARTA HAI. Upar khulne wali parabola ka exactly ek lowest point hota hai. Isliye exactly ek best slider setting hai — koi ambiguity nahi, koi doosra competing minimum nahi. Isliye Kalman gain unique hai.

PICTURE. $K$ ke against total uncertainty plot karo. Yeh ek smiling U hai. Bohot chota $K$ (left) ek bekar model se chipka rehta hai; bohot bada $K$ (right) sensor noise mein dub jaata hai; bottom sweet spot hai.

---

Step 6 — Bottom dhundho: slope ko zero karo (pehle scalar, phir matrix)

KYA HAI (scalar). Kisi bhi smooth valley ka bottom wahan hota hai jahan zameen momentarily flat ho — slope zero ho. $P^{+}(K)$ ka $K$ ke respect mein derivative lo aur $0$ se equal karo: $\frac{d{P}^{+}}{dK}=-2P^{-}+2K(P^{-}+R)=0.$

Term by term padho:

• $-2P^{-}$ — prediction error mitaane ka downhill pull.
• $+2K(P^{-}+R)$ — imported noise ka uphill push, $K$ ke saath badhta hai.

Balance karo: $K(P^{-}+R)=P^{-}⟹\boxed{K=\frac{P^{-}}{P^{-}+R}}.$

DERIVATIVE KYUN, GUESSING KYUN NAHI. Derivative woh tool hai jo jawab deta hai "kis $K$ par valley girna band karke chadhna shuru karta hai?" — precisely woh flat bottom. Iske alawa kuch bhi exact minimum ko ek clean step mein pin down nahi kar sakta.

KYA HAI (matrix — aur kyun sirf "$H$ wapas daal do" nahi hai). Real GNC mein state numbers ki list hoti hai (position, velocity, ...), isliye $x$ ek vector hai, $P^{-}$ ek symmetric matrix hai, aur gain $K$ ek matrix hai: jitne states hain utne rows hain aur jitne measurements hain utne columns. Slider ab ek knob nahi balki knobs ka poora grid hai, isliye "slope = 0" ka matlab hona chahiye "trace har direction of $K$ mein ek saath girna band ho." Hum honest scalar score $J(K)=\mathrm{tr}(P^{+})$ minimise karte hain. Joseph form expand karke aur $S=H{P}^{-}{H}^{\top }+R$ abbreviate karke: $J(K)=\mathrm{tr}(P^{-})-2\mathrm{tr}(KH{P}^{-})+\mathrm{tr}(KS{K}^{\top }),$ jahan do mixed traces ek $2$ ke factor mein merge ho gaye kyunki $\mathrm{tr}(M)=\mathrm{tr}(M^{\top })$. Ab do standard matrix-calculus facts use karo — "$\frac{d}{dK}(-2K{P}^{-})=-2P^{-}$" aur "$\frac{d}{dK}(K^{2}S)=2KS$" ka direct generalisation: $\frac{\partial }{\partial K}\mathrm{tr}(KA)=A^{\top },\frac{\partial }{\partial K}\mathrm{tr}(KS{K}^{\top })=2KS(S\text{ symmetric}).$ Toh "har direction mein slope zero hai" condition yeh ban jaati hai: $\frac{\partial J}{\partial K}=-2P^{-}{H}^{\top }+2KS=0⟹KS=P^{-}{H}^{\top }.$ Scalar $K=P^{-}/(P^{-}+R)$ exactly yahi hai jab $H=1$ aur sab kuch number ho — isliye matrix result scalar wale ko contain karta hai, sirf decorate nahi karta.

$S$ ko invert kyun kar sakte hain. $K$ ke liye $KS=P^{-}{H}^{\top }$ solve karne ke liye $S$ ko undo karna padega, matlab $S^{-1}$ se multiply karo — jo exist karta hai tabhi jab $S$ invertible ho. Hai, kyunki $R$ (kisi bhi real sensor mein nonzero noise hoti hai) positive-definite hai aur $H{P}^{-}{H}^{\top }$ positive-semidefinite hai (yeh $H$ se map ki gayi covariance hai), aur (positive-definite) + (positive-semidefinite) positive-definite hoti hai, isliye invertible. Toh hum likh sakte hain: $\boxed{K=P^{-}{H}^{\top }{S}^{-1},S=H{P}^{-}{H}^{\top }+R.}$

PICTURE. U ka tangent line exactly $K=P^{-}/(P^{-}+R)$ par flat hai; hum woh point mark karte hain. Matrix case mein, "flat" matlab $K$-grid ke har axis par ek saath flat.

---

Step 7 — Do extreme sliders (degenerate cases)

KYA HAI. Sensor ko uske do extremes par push karo aur slider ko ek end par snap hote dekho.

Case A — perfect sensor, $R\to 0$ (pink fog ek spike mein collapse hoti hai). $K=\frac{P^{-}}{P^{-}+0}=1.$ Slider bilkul right: ${\hat{x}}^{+}=z$. Hum measurement directly copy karte hain. Sahi lagta hai — flawless sensor ke hote foggy model par kyun trust karein?

Case B — useless sensor, $R\to \infty$ (pink fog flat sheet mein phail jaati hai). $K=\frac{P^{-}}{P^{-}+\infty }\to 0.$ Slider bilkul left: ${\hat{x}}^{+}={\hat{x}}^{-}$. Hum sensor ko bilkul ignore karte hain aur prediction rakhte hain.

DONO KYUN DIKHAAYEN. Reader ko koi aisa scenario kabhi nahi milna chahiye jo humne skip kiya. Ye do limits slider ke guardrails hain: yeh kabhi "all sensor" se zyada ya "all model" se kum demand nahi kar sakta. Jo real hai woh strictly beech mein hota hai.

PICTURE. Do mini-boards: left par pink spike $K\to 1$ ki taraf kheenchta hai; right par flat pink sheet $K\to 0$ hone deta hai.

---

Ek picture mein poora summary

Upar sab kuch, ek hi board par: do fogs andar jaati hain, slider $K$ blend choose karta hai, total uncertainty ek U-shaped valley trace karta hai, aur uska flat bottom $K=P^{-}{H}^{\top }{S}^{-1}$ hai, jo dono inputs se narrow output fog deta hai.

Recall Feynman: poora walkthrough plain words mein

Tumhare paas ek number ke baare mein do guesses hain: ek model ki guess aur ek sensor ki reading. Har ek ko fog ki hill ki tarah draw karo — chauda fog matlab unsure, narrow fog matlab confident. Tumhe ek final guess chahiye jo beech mein baithe, aur $K$ naam ka slider decide karta hai kahan. Sensor ki taraf slide karo aur model ka error mitaate ho lekin sensor ka noise pee lete ho; model ki taraf slide karo aur sensor ka noise bahar rakhte ho lekin ek shaky model par trust karte ho. Isliye total "main kitna galat hoon" score girti hai phir wapas chadhti hai — ek valley. Best slider us valley ke bilkul bottom par baithe, aur calculation karne par $K=P^{-}/(P^{-}+R)$ milta hai (ya sensor map ke saath, $K=P^{-}{H}^{\top }{S}^{-1}$). Jab state numbers ki list ho toh slider knobs ka grid ban jaata hai, lekin same idea lagti hai: har knob tab tak ghumi jab tak total uncertainty har direction mein girna band na ho — wahi flat bottom gain hai. $S$ se divide karna allowed hai kyunki real sensor mein hamesha kuch noise hoti hai, isliye $S$ kabhi zero nahi hota. Optimum par final fog dono starting fogs se zyada narrow hoti hai — ab tum woh number dono sources se better jaante ho. Perfect sensor? Slider sensor par snap. Useless sensor? Slider model par snap. Beech mein kuch bhi kabhi surprise nahi hona chahiye.

---

Connections

• Parent: 3.5.22 Kalman gain — minimizes trace of covariance (Hinglish)
• Kalman Filter — Predict Step
• Kalman Filter — Update Step
• Covariance Matrices and Uncertainty
• Least Squares Estimation
• Innovation and Residuals
• Joseph Form Covariance Update
• Extended Kalman Filter (EKF) in GNC
• Sensor Fusion in Navigation