Exercises — LQG — LQR + Kalman filter, separation principle

• $w$ = process noise: random pushes on the physics itself — wind gusts, unmodelled dynamics, thrust jitter.
• $v$ = measurement noise: random error inside the sensor reading.
• $W$ = the strength (covariance) of $w$; $V$ = the strength of $v$.
• Since $L=\Sigma C^{\top }{V}^{-1}$ and $\Sigma$ grows with $W$: large $W$ → large $L$ (trust the sensor more, because the model is being kicked around). Large $V$ → small $L$ (trust the model, sensor is bad).

Increase $R$ (penalise effort more). Larger $R$ means $R^{-1}$ is smaller, so $K=R^{-1}{B}^{\top }P$ shrinks → smaller control moves → gentler, slower response. Increasing $Q$ would do the opposite (aggressive tracking).

The bottom-left $0$ block. It says the estimation error $e$ evolves by its own physics ($A-LC$) and is not affected by the controller gain $K$. Because the matrix is block-upper-triangular, its eigenvalues are just the union $\text{eig}(A-BK)\cup \text{eig}(A-LC)$ — so the two designs decouple.

Start from the scalar CARE $2AP-\frac{B^2{P}^2}{R}+Q=0$ (derived above: $A^{\top }P+PA=2AP$ because scalars commute, and $PB{R}^{-1}{B}^{\top }P=B^2{P}^2/R$). Substitute $A=2,B=1,R=1,Q=3$: $2(2)P-\frac{1^2{P}^2}{1}+3=0\Rightarrow 4P-P^2+3=0\Rightarrow P^2-4P-3=0.$ Quadratic formula: $P=\frac{4+\sqrt{16+12}}{2}=\frac{4+\sqrt{28}}{2}=2+\sqrt{7}\approx 4.6458$ (keep the positive root — $P$ must be $>0$). $K=R^{-1}{B}^{\top }P=1\cdot 1\cdot P=2+\sqrt{7}\approx \mathrm{4.6458.}$ Closed loop: $A-BK=2-(2+\sqrt{7})=-\sqrt{7}\approx -2.6458<0$ ✅ stable. Why negative? The whole point of LQR is to make the closed loop decay; a positive $A$ (unstable open loop) is pulled negative by feedback.

This is the dual of L2.1 with identical numbers ($A\to A^{\top }=2$, $C^{\top }=1$, $W=3$, $V=1$). The scalar FARE is $2A\Sigma -\frac{C^2{\Sigma }^2}{V}+W=0$ (same collapse: $A\Sigma +\Sigma A^{\top }=2A\Sigma$, and $\Sigma C^{\top }{V}^{-1}C\Sigma =C^2{\Sigma }^2/V$). Substitute $A=2,C=1,V=1,W=3$: $2(2)\Sigma -\frac{1^2{\Sigma }^2}{1}+3=0\Rightarrow 4\Sigma -{\Sigma }^2+3=0\Rightarrow \Sigma =2+\sqrt{7}\approx \mathrm{4.6458.}$ $L=\Sigma C^{\top }{V}^{-1}=2+\sqrt{7}\approx \mathrm{4.6458.}$ Estimator pole $A-LC=2-(2+\sqrt{7})=-\sqrt{7}\approx -2.6458<0$ ✅.

Scalar FARE: $2A\Sigma -\frac{C^2{\Sigma }^2}{V}+W=0$. With $A=0,C=1$ this becomes $-\frac{{\Sigma }^2}{V}+W=0\Rightarrow {\Sigma }^2=WV\Rightarrow \Sigma =\sqrt{WV}(\text{positive root}).$

• $V=4$: $\Sigma =\sqrt{1\cdot 4}=2$, so $L=\Sigma C^{\top }{V}^{-1}=2/4=\mathrm{0.5.}$
• $V=\frac{1}{4}$: $\Sigma =\sqrt{1\cdot \frac{1}{4}}=0.5$, so $L=0.5/0.25=2.$ Bad sensor → small $L=0.5$ (trust model); good sensor → large $L=2$ (trust sensor). Exactly the intuition.

$A-BK=-\sqrt{7},A-LC=-\sqrt{7},BK=2+\sqrt{7}.$ Matrix $M=\left[\begin{array}{cc}-\sqrt{7}& 2+\sqrt{7}\\ 0& -\sqrt{7}\end{array}\right]$. A block-upper-triangular (here just upper-triangular) matrix has eigenvalues on its diagonal: $\{-\sqrt{7},-\sqrt{7}\}\approx \{-2.6458,-2.6458\}$. These are exactly $\text{eig}(A-BK)\cup \text{eig}(A-LC)$. The off-diagonal $BK=2+\sqrt{7}$ never entered the eigenvalues — the $0$ block guarantees that. ✅

The state equation carries an extra driving term $BKe$: the estimation error $e$ leaks into the true state through the controller. Even though $e\to 0$ in the mean, its random fluctuations continuously inject disturbance into $x$, so the achieved cost $J_{\text{LQG}}>J_{\text{LQR,ideal}}$. Separation decouples the gains you compute (design), not the cost you pay (performance). Noise still costs you.

Stability needs $A-BK=5-K<0\Rightarrow K>5$. LQR always returns such a $K$ (it guarantees stability whenever the pair is controllable). The smallest integer is $K=6$, giving pole $5-6=-1<0$ ✅. Any LQR-optimal $K$ will actually exceed $5$; the exact value depends on $Q,R$, but stability is guaranteed regardless.

Here $A=-1,B=1,C=1$. Controller — scalar CARE $2AP-\frac{B^2{P}^2}{R}+Q=0$. Substitute $A=-1,B=1,R=1,Q=4$: $2(-1)P-P^2+4=0\Rightarrow -2P-P^2+4=0\Rightarrow P^2+2P-4=0.$ $P=\frac{-2+\sqrt{4+16}}{2}=-1+\sqrt{5}\approx 1.2361$ (positive root). $K=R^{-1}{B}^{\top }P=P=-1+\sqrt{5}\approx \mathrm{1.2361.}$ Controller pole $A-BK=-1-(-1+\sqrt{5})=-\sqrt{5}\approx -\mathrm{2.2361.}$ Filter — scalar FARE $2A\Sigma -\frac{C^2{\Sigma }^2}{V}+W=0$. Substitute $A=-1,C=1,V=1,W=9$: $-2\Sigma -{\Sigma }^2+9=0\Rightarrow {\Sigma }^2+2\Sigma -9=0\Rightarrow \Sigma =-1+\sqrt{10}\approx \mathrm{2.1623.}$ $L=\Sigma C^{\top }{V}^{-1}=-1+\sqrt{10}\approx \mathrm{2.1623.}$ Estimator pole $A-LC=-1-(-1+\sqrt{10})=-\sqrt{10}\approx -\mathrm{3.1623.}$ Controller (clean form): with numbers plugged into $\dot{\hat{x}}=A\hat{x}+Bu+L(y-C\hat{x})$, $u=-K\hat{x}=-(\sqrt{5}-1)\hat{x},\dot{\hat{x}}=A\hat{x}+Bu+L(y-C\hat{x})$ where $A=-1,B=1,C=1,K=\sqrt{5}-1\approx 1.2361,L=\sqrt{10}-1\approx \mathrm{2.1623.}$ Closed-loop poles: $\{-\sqrt{5},-\sqrt{10}\}\approx \{-2.2361,-3.1623\}$ — the union, by separation. ✅

Scalar CARE $2AP-\frac{B^2{P}^2}{R}+Q=0$. With $A=1,B=1$: $2P-\frac{P^2}{R}+Q=0.$ (a) $Q=100,R=1$: $2P-P^2+100=0\Rightarrow P^2-2P-100=0\Rightarrow P=1+\sqrt{101}\approx \mathrm{11.0499.}$ $K=R^{-1}P=\mathrm{11.0499.}$ Pole $A-BK=1-11.0499=-10.0499$ — very fast. (b) $Q=1,R=100$: $2P-\frac{P^2}{100}+1=0$. Multiply by $100$: $200P-P^2+100=0\Rightarrow P^2-200P-100=0\Rightarrow P=100+\sqrt{10100}\approx \mathrm{200.4988.}$ $K=P/100=\mathrm{2.00499.}$ Pole $A-BK=1-2.00499=-1.00499$ — barely stable, gentle. Comment: heavy $Q$ / light $R$ ⇒ big gain, fast (far-left) pole, lots of effort. Heavy $R$ ⇒ small gain, sluggish pole near the imaginary axis, low effort.

• Controllability of $(A,B)$ (or at least stabilisability): the input $u$ must be able to influence every unstable direction of the state — otherwise no $K$ can stabilise $A-BK$.
• Observability of $(A,C)$ (or at least detectability): the sensor $y$ must reveal every unstable direction — otherwise no $L$ can stabilise $A-LC$, and $\hat{x}$ can't track hidden unstable modes. Both come from the Riccati Equation having a stabilising positive-(semi)definite solution.

(a) The margin guarantees are a property of the full-state feedback loop $A-BK$ measured at the plant input. The Kalman filter inserts extra dynamics ($A-LC$) between the measurement and the control, reshaping the loop transfer function; that reshaping can shrink the phase/gain margins to arbitrarily small values. Separation preserves optimality of the gains, not robustness of the loop. (b) LQG/LTR: you deliberately push the process-noise weight $W$ toward infinity (or design at the plant output), which drags the Kalman gain $L$ so large that the estimator becomes nearly transparent — the LQG loop transfer function then recovers the full-state LQR loop shape and gets its margins back. (Alternatively, H-infinity Control optimises a robustness bound directly.)

Characteristic polynomial: $\det (M-\lambda I)=\det \left[\begin{array}{cc}A-BK-\lambda & BK\\ 0& A-LC-\lambda \end{array}\right].$ For a triangular matrix the determinant is the product of the diagonal: $\det (M-\lambda I)=(A-BK-\lambda )(A-LC-\lambda ).$ Setting this to $0$ gives roots $\lambda =A-BK$ and $\lambda =A-LC$ — the off-diagonal $BK$ never appears. The $0$ in the lower-left is exactly what makes the determinant factor this way. This is the separation principle in two lines. $■$

L1 Recognition ::: Know what each symbol/piece is and which noise grows $L$. L2 Application ::: Solve a scalar Riccati and read off $K$, $L$, poles. L3 Analysis ::: Explain why poles split yet performance still degrades. L4 Synthesis ::: Design a complete scalar LQG from a blank page. L5 Mastery ::: State existence conditions, the robustness footgun, and prove separation.