LQG — LQR + Kalman filter, separation principle: Visual walkthrough

This is the pictures-first companion to the parent LQG note. We assume you have seen the ideas of State-space Representation and LQR — Linear Quadratic Regulator only in passing; we rebuild what we need.

Step 1 — What "state" and "control" even look like

Let us read this term by term, right where each symbol sits:

$\overset{x}{˙}$ — the velocity of the dot (how fast $x$ changes each second). The overdot means "rate of change".
$A x$ — the drift: the dot's natural tendency. If $A > 0$ the dot runs away from zero on its own (unstable — it needs help).
$B u$ — our influence: how strongly the knob $u$ moves the dot. $B$ is the "leverage" of the knob.

WHY start here? Everything below is about controlling and watching this dot. If you can see the dot, the arrows, and where we want it to go (the origin), you can follow the entire derivation.

PICTURE: the dot at position $x$ , a red drift arrow pushing it outward, a lavender control arrow we get to aim.

Figure — LQG — LQR + Kalman filter, separation principle

Step 2 — Why we measure "badness" with a bowl

Term by term:

$Q x^{2}$ — squared position error. Squaring makes it always positive and punishes big errors disproportionately. $Q$ is how much we care about accuracy.
$R u^{2}$ — squared effort. $R$ is how much we care about saving fuel/energy.

WHY squares and not, say, $∣ x ∣$ ? Two reasons, both visual. (1) A square is a smooth bowl — it has a single lowest point and no sharp corners, so calculus can find the bottom by setting a slope to zero. (2) The bottom of a quadratic bowl is reached by a straight-line feedback rule — the optimal knob turns out to be simply proportional to $x$ . Any other penalty shape breaks that clean linear answer. This choice of tool (a quadratic) is earned by the answer it gives.

PICTURE: two parabolic bowls — a position bowl in coral, an effort bowl in mint — and their sum, one wider bowl we must sit at the bottom of.

Step 3 — The cost-to-go: guessing the shape of the future

WHY guess it is a bowl too? The running penalty is a bowl ( $Q x^{2}$ ), the physics is linear, so it is a very reasonable guess that the accumulated penalty is also a bowl: $V (x) = P x^{2}, P > 0.$

$P$ — a single positive number that sets how steep the future-cost bowl is. Bigger $P$ = starting off-target costs you more.

This "guess a form, then verify it satisfies the equation" is exactly the Bellman method. The slope of this bowl is $\frac{d V}{d x} = 2 P x$ — we will need that slope in one line.

PICTURE: the cost-to-go bowl $V = P x^{2}$ , with the local slope $2 P x$ drawn as a tangent arrow at the current dot.

Step 4 — Balancing the future: the Bellman/HJB condition

Reading the new piece: $2 P x$ is the slope of the future-cost bowl (Step 3), and $(A x + B u)$ is the velocity of the dot (Step 1). Their product is how fast your remaining future cost is dropping as the dot moves. Balance = the two effects cancel at the optimum.

Minimize over the knob — treat everything as a bowl in $u$ and set its slope to zero: $\frac{d}{d u} [R u^{2} + 2 P x B u] = 2 R u + 2 P B x = 0 ⟹ u^{⋆} = - \frac{P B}{R} x = - K x$

$K = \frac{P B}{R}$ (scalar; in general $R^{- 1} B^{⊤} P$ ) — the feedback gain, born right here.
The minus sign is the whole point: if the dot drifts right, push left. This is negative feedback.

PICTURE: the effort-versus-future bowl in $u$ , its minimum marked, and the resulting rule "u equals minus K times x" shown as a slanted feedback line.

Step 5 — Closing the loop, and the Riccati equation

Term by term (scalar CARE):

$2 P A$ — the drift's contribution to future cost (in general $A^{⊤} P + P A$ ).
$- P^{2} B^{2} / R$ — how much the optimal push buys back (in general $- P B R^{- 1} B^{⊤} P$ ). It is subtracted because control reduces cost.
$+ Q$ — the unavoidable running penalty.

Solve for $P > 0$ , read off $K = P B / R$ , and the controlled dot now obeys $\overset{x}{˙} = (A - B K) x .$ If $A - B K < 0$ the dot decays to zero — stable. That is the whole LQR result.

PICTURE: the unstable open-loop trajectory (running away) vs the closed-loop trajectory ( $A - B K$ , decaying), same starting dot.

Step 6 — But we can't see the dot: predict + correct

$\overset{x}{^}$ — the estimate; a second dot that chases the real one.
$y - C \overset{x}{^}$ — the innovation, the surprise: measured minus expected. Zero surprise = no correction needed.
$L$ — the Kalman gain: how far to jump toward the surprise. Trusting sensors more ⇒ larger $L$ .

PICTURE: true dot (coral) hidden behind a fog band; estimate dot (lavender) predicting forward, then a correction arrow pulling it toward the noisy measurement tick.

Step 7 — The error dot and the beautiful duality

Read it: the gap $e$ shrinks by itself if $A - L C < 0$ , but is constantly kicked by process noise $w$ and by filtered sensor noise $Lv$ .

Now stare at the shape $A - L C$ . Compare it to the controller's $A - B K$ :

Regulator	Estimator
$A - B K$	$A - L C$
$A \to A^{⊤}$	$B \to C^{⊤}$ , $K \to L^{⊤}$

They are the same algebra with the substitution $(A, B, Q, R) \to (A^{⊤}, C^{⊤}, W, V)$ . So $L$ comes from a dual Riccati Equation, and the whole Kalman Filter design is the LQR design wearing a mirror. (This is the observability side mirroring the controllability side.)

PICTURE: the error dot $e$ decaying under $A - L C$ , side-by-side mirrored against the state dot decaying under $A - B K$ .

Step 8 — Gluing them together: the block-triangular matrix

Read the four blocks:

top-left $A - B K$ — the controller dynamics (Step 5).
bottom-right $A - L C$ — the estimator dynamics (Step 7).
top-right $B K$ — the leak: estimation error does disturb the true state. (Noise still costs you — separation is about design, not about zero cost.)
bottom-left $0$ — the hero. The error evolves by pure physics; the controller gain $K$ cannot corrupt it.

WHY the zero matters: a block matrix with a zero below the diagonal is upper-triangular, and the eigenvalues of an upper-triangular block matrix are just the eigenvalues of the diagonal blocks: $eig [A - B K 0 B K A - L C] = eig (A - B K) \cup eig (A - L C)$ The controller poles and the estimator poles never mix. Design each alone; the union just works. That is the Separation Principle.

PICTURE: the $2 \times 2$ block matrix drawn as four tiles, the bottom-left tile a glowing zero, arrows showing "error → state" (leak) but no arrow "state → error".

Step 9 — The edge cases you must never trip on

Degenerate limits (read them on the earlier figures):

$V \to 0$ (perfect sensor): $L \to$ large, estimate snaps onto truth, LQG $\to$ LQR.
$V \to \infty$ (useless sensor): $L \to 0$ , the filter ignores $y$ and trusts the model alone.
$W \to 0$ (perfect model): filter also leans on the model; error decays fast.
$Q \to 0$ (accuracy free): $K \to 0$ , do nothing — nothing to correct.
$R \to \infty$ (fuel priceless): $K \to 0$ again — never push.

PICTURE: a 2-D map, horizontal axis = trust-in-sensor (small $V$ ), vertical = trust-in-model (small $W$ ), with the four corner behaviours labelled.

The one-picture summary

Two independent bowls (one for control, one for estimation) produce two gains ( $K$ , $L$ ); the block-triangular matrix with its corner zero fuses them without interference.

Recall Feynman retelling — say it like a story

There is a dot on a line that drifts away on its own, and I have a knob to shove it back. I decide "badness" is far-off-squared plus push-squared — a bowl — because bowls have one bottom and calculus finds it. Rolling to the bottom tells me: push in proportion to how far off you are, with strength $K$ . That is the LQR half.

But I can't see the dot; I see a foggy, noisy shadow of it. So I run a second dot, my guess, that predicts with physics and then jumps toward whatever the fog shows me, jump-size $L$ . Choosing $L$ is the exact same bowl problem viewed in a mirror — that's the Kalman half.

Now the trick: I feed my guess into the knob. When I stack the real dot and the guessing-error side by side, the equation has a zero in the bottom-left corner — because my knob can shove the dot but it can never mess up how the error heals itself. A matrix with that zero has poles equal to just the controller's plus the estimator's, never tangled. So I design the two halves alone and glue them — optimal. The only price: the error still leaks into the dot (top-right block), so noise still costs me, and I got no free robustness — that's why LQG/LTR exists.

Recall Quick self-test

Why is the optimal control linear in the state? ::: Because the cost and cost-to-go are quadratic bowls, and the bottom of a quadratic in $u$ is a linear function of $x$ . What single feature of the stacked matrix proves separation? ::: The zero in the bottom-left block (upper-triangular ⇒ eigenvalues are the union of the diagonal blocks). Does separation mean noise doesn't hurt performance? ::: No — the $B K$ leak injects error into the state, so LQG cost exceeds ideal LQR cost. As sensor noise $V \to 0$ , what happens to LQG? ::: The Kalman gain $L$ grows, the estimate snaps to truth, and LQG reduces to LQR.

See also: LQG — LQR + Kalman filter, separation principle · LQR — Linear Quadratic Regulator · Kalman Filter · Riccati Equation · Dynamic Programming and Bellman Equation · Controllability and Observability · LQG-LTR Loop Transfer Recovery · H-infinity Control