LQG — LQR + Kalman filter, separation principle

---

WHAT is LQG?

WHY this matters (GNC context): A spacecraft/aircraft never knows its exact state — gyros drift, star-trackers are noisy, thrust is uncertain. LQG gives the provably optimal way to steer under those conditions, which is why it underlies real attitude & trajectory controllers.

---

Part 1 — Deriving LQR from scratch

The cost to minimize (infinite horizon): $J = \int_0^\infty \left( x^\top Q\, x + u^\top R\, u \right)\, dt$

• $Q \succeq 0$: penalizes state error. $R \succ 0$: penalizes control effort.
• Tuning knob: big $Q$ → aggressive (track fast); big $R$ → gentle (save fuel).

Derivation via the value function (dynamic programming)

WHY: Optimal control obeys Bellman's principle — the cost-to-go from any state $x$ is a function $V(x)$. For linear-quadratic problems we guess $V(x)=x^\top P x$ (a quadratic bowl) and verify it works.

HOW — the Hamilton–Jacobi–Bellman (HJB) equation. The optimal cost-to-go must satisfy: $0 = \min_u \left[ x^\top Q x + u^\top R u + \nabla V^\top(Ax+Bu) \right]$

With $V=x^\top P x$, we have $\nabla V = 2Px$. Substitute: $0 = \min_u \left[ x^\top Q x + u^\top R u + 2x^\top P (Ax+Bu) \right]$

Minimize over $u$ — take derivative w.r.t. $u$ and set to $0$:

\;\Rightarrow\; \boxed{u = -R^{-1}B^\top P\, x = -Kx}$$ *Why this step?* This is where **state feedback** is born: the optimal control is *linear in the state*, with gain $K=R^{-1}B^\top P$. **Plug $u=-Kx$ back in** to find $P$. Using $x^\top P A x = \tfrac12 x^\top(PA+A^\top P)x$: $$0 = x^\top\!\left( Q + A^\top P + P A - P B R^{-1} B^\top P \right)x \quad \forall x$$ Since it holds for all $x$, the bracket is zero: > [!formula] Continuous Algebraic Riccati Equation (CARE) > $$A^\top P + P A - P B R^{-1} B^\top P + Q = 0$$ > Solve for symmetric $P\succeq 0$, then the **optimal gain** is > $$K = R^{-1} B^\top P, \qquad u^\star = -Kx.$$ --- ## Part 2 — The Kalman filter (the dual problem) > [!intuition] Why we need an estimator > We only measure $y=Cx+v$ — noisy and often incomplete. We must *reconstruct* $\hat x$. The Kalman filter is a **model that predicts, then corrects**: use dynamics to predict where $x$ should be, then nudge toward the measurement by an amount that trades off *trust in model* vs *trust in sensor*. **HOW — the estimator structure (Luenberger-style, optimal gain):** $$\dot{\hat x} = \underbrace{A\hat x + Bu}_{\text{predict}} + \underbrace{L\,(y - C\hat x)}_{\text{correct}}$$ The term $(y-C\hat x)$ is the **innovation** — the surprise between what we measured and what we expected. **Define the estimation error** $e = x - \hat x$. Subtracting equations (and using $\dot x = Ax+Bu+w$): $$\dot e = (A - LC)\,e + w - L v$$ > [!intuition] The beautiful duality > The error dynamics matrix $A-LC$ has **exactly the same algebraic form** as the LQR closed loop $A-BK$, but with $A\to A^\top$, $B\to C^\top$. So the optimal $L$ solves a Riccati equation dual to the LQR one — swap $(A,B,Q,R)\to(A^\top,C^\top,W,V)$. > [!formula] Kalman gain via the (filter) Riccati equation > $$A\Sigma + \Sigma A^\top - \Sigma C^\top V^{-1} C\, \Sigma + W = 0$$ > $$L = \Sigma C^\top V^{-1}$$ > where $\Sigma$ is the steady-state **error covariance** $\mathbb E[e e^\top]$. Large sensor noise $V$ → small $L$ (trust the model); large process noise $W$ → large $L$ (trust the sensor). --- ## Part 3 — The Separation Principle (the crown jewel) > [!intuition] What it claims > Design $K$ (LQR) and $L$ (Kalman) **completely independently**, then set $u=-K\hat x$. The resulting LQG controller is **optimal**, and the closed-loop poles are the **union** of the controller poles (eig of $A-BK$) and the estimator poles (eig of $A-LC$). **WHY it's true — write the joint system.** With $u=-K\hat x$ and $\hat x = x - e$: $$\dot x = Ax - BK\hat x + w = Ax - BK(x-e) + w = (A-BK)x + BK\,e + w$$ $$\dot e = (A-LC)e + w - Lv$$ Stack $\begin{bmatrix} x \\ e\end{bmatrix}$: $$\frac{d}{dt}\begin{bmatrix} x \\ e\end{bmatrix} = \underbrace{\begin{bmatrix} A-BK & BK \\ 0 & A-LC \end{bmatrix}}_{\text{block upper-triangular!}}\begin{bmatrix} x \\ e\end{bmatrix} + (\text{noise})$$ > [!formula] Separation Principle > Because the system matrix is **block upper-triangular**, its eigenvalues are: > $$\operatorname{eig}\!\begin{bmatrix} A-BK & BK \\ 0 & A-LC\end{bmatrix} = \operatorname{eig}(A-BK)\;\cup\;\operatorname{eig}(A-LC)$$ > The two designs **do not interact**. Regulator and estimator can be tuned separately. *Why the zero block?* The estimation error $e$ drives the state, but the **controller cannot corrupt the error dynamics** — $e$ evolves by pure physics ($A-LC$), independent of $K$. That single $0$ block is the entire mathematical content of separation. ![[3.5.36-LQG-—-LQR-+-Kalman-filter,-separation-principle.png]] --- ## Worked Example 1 — Scalar LQR **System:** $\dot x = x + u$ ($A=1,B=1$), cost $Q=1, R=1$. **CARE:** $A^\top P + PA - PB R^{-1}B^\top P + Q = 0 \Rightarrow 2P - P^2 + 1 = 0$. *Why this step?* Scalars commute, so $A^\top P + PA = 2P$; $PBR^{-1}B^\top P=P^2$. Solve $P^2 - 2P - 1 = 0 \Rightarrow P = 1+\sqrt2$ (take $P>0$). **Gain:** $K = R^{-1}B^\top P = P = 1+\sqrt2 \approx 2.414$. **Closed loop:** $A-BK = 1 - 2.414 = -1.414 < 0$ ✅ stable (as it must be). ## Worked Example 2 — Scalar Kalman **Same $A,C=1$**, noises $W=1$ (process), $V=1$ (sensor). **Filter Riccati:** $2\Sigma - \Sigma^2 + 1 = 0 \Rightarrow \Sigma = 1+\sqrt2$. *Why this step?* It's the **dual** of Example 1 with the exact same numbers. **Kalman gain:** $L = \Sigma C^\top V^{-1} = 1+\sqrt2 \approx 2.414$. **Estimator pole:** $A-LC = -1.414 < 0$ ✅. ## Worked Example 3 — Separation in action The full LQG closed-loop poles = $\{-1.414,\; -1.414\}$ = (controller pole) ∪ (estimator pole). Notice we never had to solve a coupled 2×2 optimization — that's the practical payoff. --- ## Common Mistakes (Steel-manned) > [!mistake] "Just feed the noisy measurement $y$ into $-Ky$." > **Why it feels right:** $y$ *is* our data; feeding it back seems direct and simple. > **Why it's wrong:** $y$ contains sensor noise $v$ *and* only a projection $Cx$ (partial state). Feeding raw $y$ back amplifies noise and can't reconstruct unmeasured states. > **Fix:** Filter first — use $\hat x$ from the Kalman filter, then $u=-K\hat x$. > [!mistake] "Separation means the noise doesn't affect performance." > **Why it feels right:** The *poles* separate cleanly, so it looks decoupled. > **Why it's wrong:** Separation is about **pole placement / optimality of gains**, not zero cost. The $BK\,e$ coupling term means estimation error *does* inject into the state — LQG cost $> $ ideal LQR cost. Noise still costs you. > **Fix:** Separation ⇒ *design* is decoupled; *performance* still degrades with noise. > [!mistake] "LQG is guaranteed robust like LQR." > **Why it feels right:** LQR has famous ≥60° phase / infinite gain margins. > **Why it's wrong:** Doyle (1978): **"LQG has no guaranteed margins."** The estimator can destroy LQR's robustness. > **Fix:** Use **LQG/LTR (Loop Transfer Recovery)** to recover margins. --- ## #flashcards/physics What does each letter in LQG stand for? ::: Linear system, Quadratic cost, Gaussian noise. State the separation principle in one line. ::: Design LQR gain $K$ and Kalman gain $L$ independently; use $u=-K\hat x$; closed-loop poles = eig$(A-BK)\cup$eig$(A-LC)$. Why does the separation principle hold mathematically? ::: The stacked $[x,e]$ system matrix is block upper-triangular (zero block bottom-left), so eigenvalues split. Write the CARE for LQR. ::: $A^\top P + PA - PBR^{-1}B^\top P + Q = 0$; then $K=R^{-1}B^\top P$. What is the optimal LQR control law? ::: $u^\star=-Kx$ with $K=R^{-1}B^\top P$ — linear state feedback. What is the innovation in a Kalman filter? ::: $y - C\hat x$: measurement minus its prediction (the "surprise"). How does the Kalman gain change with noise? ::: Large sensor noise $V$ → small $L$ (trust model); large process noise $W$ → large $L$ (trust sensor). What duality maps LQR to the Kalman filter? ::: Swap $(A,B,Q,R)\to(A^\top,C^\top,W,V)$; the Riccati equations mirror each other. Does LQG guarantee robustness margins? ::: No — Doyle showed LQG has no guaranteed margins; use LQG/LTR to recover them. What does increasing $Q$ vs $R$ do? ::: Larger $Q$ → aggressive tracking; larger $R$ → gentle, low-effort control. Why quadratic cost specifically? ::: It's always positive, penalizes big errors disproportionately, and yields a *linear* optimal feedback law. --- > [!recall]- Feynman: explain to a 12-year-old > Imagine driving a toy car to a target, but your glasses are foggy (bad sensors) and there's wind (noise). Two jobs: **(1)** *Steering rule* — if you knew exactly where the car is, how much would you turn the wheel? (That's LQR.) **(2)** *Guessing where the car is* — combine what your dynamics tells you with what your foggy eyes see. (That's the Kalman filter — it blends a prediction with a noisy peek.) The amazing trick (**separation**): you can figure out the perfect steering rule and the perfect guessing rule **separately**, then just steer based on your best guess — and it's still the best you can do. You don't have to solve both puzzles at once. > [!mnemonic] "**K**eep **L**ooking, then **K**eep steering — **L** finds it, **K** fixes it." > $L$ = the **Look**er (Kalman, estimation). $K$ = the **K**icker (LQR, control). They work in different lanes — hence *separation*. Alphabet order matches design order: build $L$'s estimate, then apply $K$. ## Connections - [[LQR — Linear Quadratic Regulator]] - [[Kalman Filter]] - [[Riccati Equation]] - [[State-space Representation]] - [[Controllability and Observability]] (needed for $K$ and $L$ to exist) - [[LQG-LTR Loop Transfer Recovery]] - [[Dynamic Programming and Bellman Equation]] - [[H-infinity Control]] (robust alternative to LQG) ## 🖼️ Concept Map ```mermaid flowchart TD P[Real problem: noisy sensors, hidden state] -->|solved by| LQG[LQG control] LQG -->|fuses| LQR[LQR state feedback] LQG -->|fuses| KF[Kalman filter estimator] SP[Separation principle] -->|lets us design separately| LQR SP -->|lets us design separately| KF KF -->|provides estimate x-hat| LQR LQR -->|minimizes| J[Quadratic cost J] J -->|solved via| HJB[HJB equation with V equals x P x] HJB -->|yields| Riccati[Algebraic Riccati equation] Riccati -->|gives gain| K[Feedback gain K equals R inv B P] K -->|forms law| U[Control u equals minus K x] Model[Linear model with Gaussian noise] -->|assumed by| LQG ``` ## 🔊 Hinglish (regional understanding) > [!intuition]- Hinglish mein samjho > Dekho, LQG ka matlab hai real-world control jahan tumhare paas do problem hain: sensor noisy hai aur tum poora state dekh nahi paate. Iska solution do tukdon me todo. Pehla — **LQR**: maan lo tumhe exact state pata hai, to sabse optimal steering kya hoga? Wo quadratic cost $J=\int (x^\top Q x + u^\top R u)dt$ minimize karke nikalta hai, aur answer hamesha ek simple linear rule hota hai: $u=-Kx$. $Q$ badhao to aggressive control, $R$ badhao to fuel-saving gentle control. $K$ nikalne ke liye Riccati equation solve karni padti hai. > > Dusra tukda — **Kalman filter**: kyunki tumhe exact $x$ nahi pata, tum use estimate karte ho. Filter pehle dynamics se predict karta hai, phir measurement se compare karke correct karta hai — is difference $(y-C\hat x)$ ko **innovation** kehte hain. Agar sensor bahut noisy hai (bada $V$), to filter apne model pe zyada bharosa karta hai (chhota $L$); agar process me disturbance zyada hai (bada $W$), to sensor pe zyada bharosa (bada $L$). > > Ab asli jaadu — **Separation Principle**. Yeh kehta hai ki $K$ aur $L$ ko tum **alag-alag** design kar sakte ho, jaise dono kabhi mile hi nahi, phir bas $u=-K\hat x$ laga do — aur poora system phir bhi optimal rahega. Iska proof ek chhoti si baat me hai: jab tum state $x$ aur estimation error $e$ ko stack karte ho, matrix **upper-triangular** ban jaata hai (neeche-baaye zero block), isliye poles do alag sets me split ho jaate hain: controller ke ($A-BK$) aur estimator ke ($A-LC$). > > Ek warning zaroor yaad rakhna: LQR ki jo mashhoor robustness margins hoti hain, wo LQG me **guarantee nahi** hoti (Doyle ne 1978 me dikhaya). Isliye practical GNC me **LQG/LTR** use karte hain margins wapas laane ke liye. Aur haan — separation ka matlab "noise ka koi asar nahi" nahi hai; design decouple hota hai, performance abhi bhi noise se degrade hoti hai. ![[audio/3.5.36-LQG-—-LQR-+-Kalman-filter,-separation-principle.mp3]]