LQG — LQR + Kalman filter, separation principle

3.5.36 · Physics › Guidance, Navigation & Control (GNC)

---

LQG HAI KYA?

YEH KYUN MATTER KARTA HAI (GNC context): Ek spacecraft/aircraft kabhi apna exact state nahi jaanta — gyros drift karte hain, star-trackers noisy hote hain, thrust uncertain hoti hai. LQG un conditions mein steer karne ka provably optimal tarika deta hai, isliye yeh real attitude & trajectory controllers ka base hai.

---

Part 1 — LQR ko scratch se derive karna

Cost jo minimize karni hai (infinite horizon): $J={\int }_0^{\infty }\left(x^{\top }Qx+u^{\top }Ru\right)dt$

• $Q⪰0$: state error ko penalize karta hai. $R\succ 0$: control effort ko penalize karta hai.
• Tuning knob: bada $Q$ → aggressive (fast track karo); bada $R$ → gentle (fuel bachao).

Value function se derivation (dynamic programming)

KYUN: Optimal control Bellman's principle follow karta hai — kisi bhi state $x$ se cost-to-go ek function $V(x)$ hai. Linear-quadratic problems ke liye hum guess karte hain $V(x)=x^{\top }Px$ (ek quadratic bowl) aur verify karte hain ki yeh kaam karta hai.

HOW — Hamilton–Jacobi–Bellman (HJB) equation. Optimal cost-to-go ko satisfy karna hoga: $0={\min }_u\left[x^{\top }Qx+u^{\top }Ru+\nabla V^{\top }(Ax+Bu)\right]$

$V=x^{\top }Px$ ke saath, hamare paas $\nabla V=2Px$ hai. Substitute karo: $0={\min }_u\left[x^{\top }Qx+u^{\top }Ru+2x^{\top }P(Ax+Bu)\right]$

$u$ ke upar minimize karo — $u$ ke w.r.t. derivative lo aur $0$ set karo:

\;\Rightarrow\; \boxed{u = -R^{-1}B^\top P\, x = -Kx}$$ *Yeh step kyun?* Yahaan **state feedback** paida hota hai: optimal control state mein *linear* hai, gain $K=R^{-1}B^\top P$ ke saath. **$u=-Kx$ wapas plug karo** $P$ dhundhne ke liye. $x^\top P A x = \tfrac12 x^\top(PA+A^\top P)x$ use karte hue: $$0 = x^\top\!\left( Q + A^\top P + P A - P B R^{-1} B^\top P \right)x \quad \forall x$$ Kyunki yeh sab $x$ ke liye hold karta hai, bracket zero hai: > [!formula] Continuous Algebraic Riccati Equation (CARE) > $$A^\top P + P A - P B R^{-1} B^\top P + Q = 0$$ > Symmetric $P\succeq 0$ ke liye solve karo, phir **optimal gain** hai > $$K = R^{-1} B^\top P, \qquad u^\star = -Kx.$$ --- ## Part 2 — Kalman filter (dual problem) > [!intuition] Estimator ki zaroorat kyun hai > Hum sirf $y=Cx+v$ measure karte hain — noisy aur aksar incomplete. Hume $\hat x$ *reconstruct* karna hai. Kalman filter ek **model hai jo predict karta hai, phir correct karta hai**: dynamics use karo predict karne ke liye ki $x$ kahan hona chahiye, phir measurement ki taraf itna nudge karo jo *model pe trust* aur *sensor pe trust* ke beech trade-off kare. **HOW — estimator structure (Luenberger-style, optimal gain):** $$\dot{\hat x} = \underbrace{A\hat x + Bu}_{\text{predict}} + \underbrace{L\,(y - C\hat x)}_{\text{correct}}$$ Term $(y-C\hat x)$ **innovation** hai — jo humne measure kiya aur jo hum expect karte the, unke beech ka "surprise". **Estimation error define karo** $e = x - \hat x$. Equations subtract karo (aur $\dot x = Ax+Bu+w$ use karte hue): $$\dot e = (A - LC)\,e + w - L v$$ > [!intuition] Khoobsoorat duality > Error dynamics matrix $A-LC$ ka **bilkul wahi algebraic form** hai jaise LQR closed loop $A-BK$, lekin $A\to A^\top$, $B\to C^\top$ ke saath. Toh optimal $L$ ek Riccati equation solve karta hai jo LQR wali ka dual hai — swap karo $(A,B,Q,R)\to(A^\top,C^\top,W,V)$. > [!formula] (Filter) Riccati equation se Kalman gain > $$A\Sigma + \Sigma A^\top - \Sigma C^\top V^{-1} C\, \Sigma + W = 0$$ > $$L = \Sigma C^\top V^{-1}$$ > jahaan $\Sigma$ steady-state **error covariance** $\mathbb E[e e^\top]$ hai. Zyada sensor noise $V$ → chhota $L$ (model pe trust karo); zyada process noise $W$ → bada $L$ (sensor pe trust karo). --- ## Part 3 — Separation Principle (crown jewel) > [!intuition] Yeh claim kya karta hai > $K$ (LQR) aur $L$ (Kalman) **bilkul independently** design karo, phir $u=-K\hat x$ set karo. Result mein jo LQG controller banega woh **optimal** hoga, aur closed-loop poles **union** honge controller poles (eig of $A-BK$) aur estimator poles (eig of $A-LC$) ka. **KYUN yeh sach hai — joint system likho.** $u=-K\hat x$ aur $\hat x = x - e$ ke saath: $$\dot x = Ax - BK\hat x + w = Ax - BK(x-e) + w = (A-BK)x + BK\,e + w$$ $$\dot e = (A-LC)e + w - Lv$$ $\begin{bmatrix} x \\ e\end{bmatrix}$ stack karo: $$\frac{d}{dt}\begin{bmatrix} x \\ e\end{bmatrix} = \underbrace{\begin{bmatrix} A-BK & BK \\ 0 & A-LC \end{bmatrix}}_{\text{block upper-triangular!}}\begin{bmatrix} x \\ e\end{bmatrix} + (\text{noise})$$ > [!formula] Separation Principle > Kyunki system matrix **block upper-triangular** hai, iske eigenvalues hain: > $$\operatorname{eig}\!\begin{bmatrix} A-BK & BK \\ 0 & A-LC\end{bmatrix} = \operatorname{eig}(A-BK)\;\cup\;\operatorname{eig}(A-LC)$$ > Dono designs **interact nahi karte**. Regulator aur estimator alag-alag tune kiye ja sakte hain. *Zero block kyun?* Estimation error $e$ state ko drive karta hai, lekin **controller error dynamics corrupt nahi kar sakta** — $e$ pure physics se evolve karta hai ($A-LC$), $K$ se independent. Woh akela $0$ block separation ka poora mathematical content hai. ![[3.5.36-LQG-—-LQR-+-Kalman-filter,-separation-principle.png]] --- ## Worked Example 1 — Scalar LQR **System:** $\dot x = x + u$ ($A=1,B=1$), cost $Q=1, R=1$. **CARE:** $A^\top P + PA - PB R^{-1}B^\top P + Q = 0 \Rightarrow 2P - P^2 + 1 = 0$. *Yeh step kyun?* Scalars commute karte hain, toh $A^\top P + PA = 2P$; $PBR^{-1}B^\top P=P^2$. Solve karo $P^2 - 2P - 1 = 0 \Rightarrow P = 1+\sqrt2$ ($P>0$ lo). **Gain:** $K = R^{-1}B^\top P = P = 1+\sqrt2 \approx 2.414$. **Closed loop:** $A-BK = 1 - 2.414 = -1.414 < 0$ ✅ stable (jaisa hona chahiye). ## Worked Example 2 — Scalar Kalman **Same $A,C=1$**, noises $W=1$ (process), $V=1$ (sensor). **Filter Riccati:** $2\Sigma - \Sigma^2 + 1 = 0 \Rightarrow \Sigma = 1+\sqrt2$. *Yeh step kyun?* Yeh Example 1 ka **dual** hai bilkul same numbers ke saath. **Kalman gain:** $L = \Sigma C^\top V^{-1} = 1+\sqrt2 \approx 2.414$. **Estimator pole:** $A-LC = -1.414 < 0$ ✅. ## Worked Example 3 — Separation in action Poore LQG closed-loop poles = $\{-1.414,\; -1.414\}$ = (controller pole) ∪ (estimator pole). Dhyan do ki humein kabhi coupled 2×2 optimization solve nahi karni padi — yahi practical payoff hai. --- ## Common Mistakes (Steel-manned) > [!mistake] "Bas noisy measurement $y$ ko $-Ky$ mein feed kar do." > **Kyun sahi lagta hai:** $y$ *hi* haara data hai; ise directly feed karna seedha aur simple lagta hai. > **Kyun galat hai:** $y$ mein sensor noise $v$ *bhi* hai aur sirf ek projection $Cx$ (partial state). Raw $y$ feed karne se noise amplify hoti hai aur unmeasured states reconstruct nahi ho sakti. > **Fix:** Pehle filter karo — Kalman filter se $\hat x$ use karo, phir $u=-K\hat x$. > [!mistake] "Separation matlab noise performance affect nahi karta." > **Kyun sahi lagta hai:** *Poles* cleanly separate hote hain, toh decoupled lagta hai. > **Kyun galat hai:** Separation **pole placement / gains ki optimality** ke baare mein hai, zero cost ke baare mein nahi. $BK\,e$ coupling term ka matlab hai ki estimation error *state mein inject hoti hai* — LQG cost $> $ ideal LQR cost. Noise phir bhi cost karti hai. > **Fix:** Separation ⇒ *design* decoupled hai; *performance* phir bhi noise ke saath kharab hoti hai. > [!mistake] "LQG guaranteed robust hai jaise LQR." > **Kyun sahi lagta hai:** LQR ke famous ≥60° phase / infinite gain margins hain. > **Kyun galat hai:** Doyle (1978): **"LQG ke koi guaranteed margins nahi hain."** Estimator LQR ki robustness destroy kar sakta hai. > **Fix:** **LQG/LTR (Loop Transfer Recovery)** use karo margins recover karne ke liye. --- ## #flashcards/physics LQG mein har letter kya stand karta hai? ::: Linear system, Quadratic cost, Gaussian noise. Separation principle ek line mein bolo. ::: LQR gain $K$ aur Kalman gain $L$ independently design karo; $u=-K\hat x$ use karo; closed-loop poles = eig$(A-BK)\cup$eig$(A-LC)$. Separation principle mathematically kyun sach hai? ::: Stacked $[x,e]$ system matrix block upper-triangular hai (bottom-left mein zero block), toh eigenvalues split ho jaate hain. LQR ke liye CARE likho. ::: $A^\top P + PA - PBR^{-1}B^\top P + Q = 0$; phir $K=R^{-1}B^\top P$. Optimal LQR control law kya hai? ::: $u^\star=-Kx$ with $K=R^{-1}B^\top P$ — linear state feedback. Kalman filter mein innovation kya hota hai? ::: $y - C\hat x$: measurement minus uski prediction (woh "surprise"). Noise ke saath Kalman gain kaise change hota hai? ::: Zyada sensor noise $V$ → chhota $L$ (model pe trust karo); zyada process noise $W$ → bada $L$ (sensor pe trust karo). LQR ko Kalman filter se kaunsi duality map karti hai? ::: Swap karo $(A,B,Q,R)\to(A^\top,C^\top,W,V)$; Riccati equations ek doosre ka mirror hain. Kya LQG robustness margins guarantee karta hai? ::: Nahi — Doyle ne dikhaya ki LQG ke koi guaranteed margins nahi hain; unhe recover karne ke liye LQG/LTR use karo. $Q$ aur $R$ badhane se kya hota hai? ::: Bada $Q$ → aggressive tracking; bada $R$ → gentle, low-effort control. Quadratic cost hi kyun specifically? ::: Yeh hamesha positive hai, bade errors ko disproportionately penalize karta hai, aur ek *linear* optimal feedback law deta hai. --- > [!recall]- Feynman: 12-saal ke bacche ko samjhao > Socho ek toy car ko target pe drive kar rahe ho, lekin tumhara chashma foggy hai (kharab sensors) aur hawa chal rahi hai (noise). Do kaam hain: **(1)** *Steering rule* — agar tumhe pata hota car exactly kahan hai, toh wheel kitna ghumate? (Yahi LQR hai.) **(2)** *Car kahan hai guess karna* — tumhare dynamics jo batate hain aur tumhari foggy aankhein jo dikhati hain, dono combine karo. (Yahi Kalman filter hai — prediction ko noisy glimpse ke saath blend karta hai.) Amazing trick (**separation**): perfect steering rule aur perfect guessing rule **alag-alag** figure out karo, phir apne best guess ke basis pe steer karo — aur yeh phir bhi tumhara best possible hai. Dono puzzles ek saath solve karne ki zaroorat nahi. > [!mnemonic] "**K**eep **L**ooking, then **K**eep steering — **L** finds it, **K** fixes it." > $L$ = **L**ooker (Kalman, estimation). $K$ = **K**icker (LQR, control). Yeh alag lanes mein kaam karte hain — isliye *separation*. Alphabet order design order se match karta hai: pehle $L$ ka estimate banao, phir $K$ apply karo. ## Connections - [[LQR — Linear Quadratic Regulator]] - [[Kalman Filter]] - [[Riccati Equation]] - [[State-space Representation]] - [[Controllability and Observability]] ($K$ aur $L$ ke exist karne ke liye zaroori) - [[LQG-LTR Loop Transfer Recovery]] - [[Dynamic Programming and Bellman Equation]] - [[H-infinity Control]] (LQG ka robust alternative) ## 🖼️ Concept Map ```mermaid flowchart TD P[Real problem: noisy sensors, hidden state] -->|solved by| LQG[LQG control] LQG -->|fuses| LQR[LQR state feedback] LQG -->|fuses| KF[Kalman filter estimator] SP[Separation principle] -->|lets us design separately| LQR SP -->|lets us design separately| KF KF -->|provides estimate x-hat| LQR LQR -->|minimizes| J[Quadratic cost J] J -->|solved via| HJB[HJB equation with V equals x P x] HJB -->|yields| Riccati[Algebraic Riccati equation] Riccati -->|gives gain| K[Feedback gain K equals R inv B P] K -->|forms law| U[Control u equals minus K x] Model[Linear model with Gaussian noise] -->|assumed by| LQG ``` ## 🔬 Deep Dive > [!intuition] Aur gehraai mein jao — visual, zero se > Is topic ke step-by-step 3Blue1Brown-style breakdowns. - [[3.5.36 D1 Foundations|D1 · Foundations — har symbol zero se]] - [[3.5.36 D2 Visual Walkthrough|D2 · Visual walkthrough — derivation pictures mein]] - [[3.5.36 D3 Worked Examples|D3 · Worked examples — har scenario]] - [[3.5.36 D4 Exercises|D4 · Exercises — graded, full solutions]] - [[3.5.36 D5 Question Bank|D5 · Question bank — concept traps]]