Linear Quadratic Regulator (LQR) — Riccati equation, optimal gains: Exercises

Before we start, one picture fixes the whole vocabulary you need.

Level 1 — Recognition

L1.1

State the continuous-time Algebraic Riccati Equation (CARE) and identify which single term makes it nonlinear in $P$ .

Recall Solution

$A^{⊤} P + P A - P B R^{- 1} B^{⊤} P + Q = 0.$ The term $- P B R^{- 1} B^{⊤} P$ is quadratic in $P$ (it contains $P$ twice), so the equation is nonlinear. Every other term ( $A^{⊤} P$ , $P A$ , $Q$ ) is linear or constant in $P$ .

L1.2

You are told $R = [2008]$ , $B = [01]$ , and the solved $P = [3114]$ . Write the formula for $K$ and identify the size (rows × columns) of $K$ .

Recall Solution

$K = R^{- 1} B^{⊤} P$ . Here $R$ is $2 \times 2$ , $B^{⊤}$ is $1 \times 2$ , $P$ is $2 \times 2$ . But wait — $B$ is $2 \times 1$ so there is one control input, so $R$ must be $1 \times 1$ to be conformable. The stated $R$ is inconsistent; the correct recognition is: with $m = 1$ control and $n = 2$ states, $K$ is $m \times n = 1 \times 2$ .

The point of the exercise: always check dimensions. $u \in R^{m}$ , $x \in R^{n}$ , so $u = - K x$ forces $K$ to be $m \times n$ . Here $1 \times 2$ .

L1.3

Which of these is the optimal LQR control law? (a) $u = - K x^{2}$ (b) $u = - K x$ (c) $u = - K \overset{x}{˙}$ (d) $u = K x$ .

Recall Solution

(b) $u = - K x$ . It is linear in $x$ (rules out (a)), uses the state not its derivative (rules out (c)), and the sign is negative — you push against the offset to drive $x \to 0$ (rules out (d), which would push you away and destabilize).

Level 2 — Application

L2.1 (scalar CARE)

Plant $\overset{x}{˙} = 2 x + u$ (so $a = 2, b = 1$ ), weights $q = 4, r = 1$ . Find $p > 0$ , the gain $K$ , and the closed-loop pole $a - b K$ .

Recall Solution

Scalar CARE: $2 a p - \frac{b ^{2}}{r} p^{2} + q = 0 \Rightarrow 4 p - p^{2} + 4 = 0 \Rightarrow p^{2} - 4 p - 4 = 0.$ $p = \frac{4 + 16 + 16}{2} = \frac{4 + 32}{2} = 2 + 22 \approx 4.828 (positive root) .$ $K = \frac{b}{r} p = p = 2 + 22 \approx 4.828.$ Closed loop: $a - b K = 2 - (2 + 22) = - 22 \approx - 2.828 < 0.$ Stable — even though the open loop ( $a = 2$ ) was unstable.

L2.2 (double integrator gain)

Using the parent's double-integrator result $K = [\frac{1}{ρ}, \frac{2}{ρ ^{1/4}}]$ , compute $K$ for $ρ = 16$ .

Recall Solution

$\frac{1}{16} = \frac{1}{4} = 0.25.$ And $ρ^{1/4} = 1 6^{1/4} = 2$ , so $\frac{2}{2} = \frac{1}{2} \approx 0.707.$ $K = [0.25, 0.7071] .$ Notice: bigger $ρ$ (heavier control penalty) shrank both gains — softer pushing.

L2.3 (cost-to-go value)

For L2.1 you found $p \approx 4.828$ . If the satellite starts at $x (0) = 3$ , what is the total optimal cost $J = V (x_{0}) = p x_{0}^{2}$ ?

Recall Solution

$V = p x_{0}^{2} = (2 + 22) (9) = 18 + 182 \approx 43.46.$ The value function $V = p x^{2}$ is the remaining cost if you steer optimally from here on — no integral needed once $p$ is known.

Level 3 — Analysis

L3.1 (only the ratio matters)

Show that scaling both weights, $Q \to α Q$ and $R \to α R$ for $α > 0$ , leaves the gain $K$ unchanged.

Recall Solution

CARE with scaled weights: $A^{⊤} P + P A - P B (α R)^{- 1} B^{⊤} P + α Q = 0.$ Try $P \to α P^{'}$ (guess the solution scales too). Substitute: $α A^{⊤} P^{'} + α P^{'} A - α^{2} P^{'} B \frac{1}{α} R^{- 1} B^{⊤} P^{'} + α Q = α (A^{⊤} P^{'} + P^{'} A - P^{'} B R^{- 1} B^{⊤} P^{'} + Q) = 0.$ The bracket is the original CARE, so $P^{'}$ is the original solution and $P = α P^{'}$ . Then $K = (α R)^{- 1} B^{⊤} (α P^{'}) = \frac{1}{α} R^{- 1} B^{⊤} α P^{'} = R^{- 1} B^{⊤} P^{'} .$ The $α$ 's cancel — $K$ is unchanged. Only the ratio $Q / R$ sets the controller.

L3.2 (which knob makes it aggressive?)

In the scalar case $K = \frac{a r + a ^{2} r ^{2} + q r b ^{2}}{r b}$ (rewrite of $K = \frac{b}{r} p$ ). Analyse: does the pole $a - b K$ move left (faster) as $q$ grows or as $r$ grows?

Recall Solution

$a - b K = a - \frac{b ^{2}}{r} p$ . As $q ↑$ , $p ↑$ (larger $q$ in the CARE forces larger $p$ ), so $\frac{b ^{2}}{r} p ↑$ , so $a - b K$ becomes more negative → pole moves left → faster/aggressive. As $r ↑$ with $q$ fixed: the penalty on control grows, $K$ shrinks, $a - b K$ moves toward $a$ (right) → slower/gentler. Verdict: big $Q$ = aggressive, big $R$ = lazy.

L3.3 (closed-loop poles of the double integrator)

For $ρ = 1$ , $K = [1, 2]$ . Find the closed-loop matrix $A - B K$ and its eigenvalues; state the damping ratio.

Recall Solution

$A - B K = [0010] - [01] [12] = [0 - 1 1 - 2] .$ Characteristic polynomial: $det (s I - (A - B K)) = s^{2} + 2 s + 1 = 0.$ $s = \frac{- 2 \pm 2 - 4}{2} = - \frac{1}{2} \pm j \frac{1}{2} .$ Comparing $s^{2} + 2 ζ ω_{n} s + ω_{n}^{2}$ with $ω_{n} = 1$ : $2 ζ = 2 \Rightarrow ζ = \frac{1}{2} \approx 0.707.$ Nicely damped — no overshoot ringing to speak of.

The poles for varying $ρ$ trace a curve — see the figure.

Level 4 — Synthesis

L4.1 (LQR is PD control)

The double-integrator gain $K = [k_{p}, k_{d}]$ gives $u = - k_{p} x_{1} - k_{d} x_{2}$ . Explain, term by term, why this is exactly a spring–damper (PD) controller, and for $ρ = 1$ give the $(k_{p}, k_{d})$ and hence natural frequency and damping.

Recall Solution

$x_{1}$ = position, $x_{2}$ = velocity. So $u = - k_{p} x_{1} - k_{d} x_{2}$ :

$- k_{p} x_{1}$ = a spring pulling position back to zero (proportional term, the "P").
$- k_{d} x_{2}$ = a damper opposing velocity (derivative term, the "D"). That is precisely a PD controller — see PID Controllers. LQR derived it as optimal. For $ρ = 1$ : $k_{p} = 1, k_{d} = 2$ . Closed loop $\overset{x}{¨}_{1} + k_{d} \overset{x}{˙}_{1} + k_{p} x_{1} = 0$ , i.e. $ω_{n} = k_{p} = 1$ , $ζ = k_{d} / (2 ω_{n}) = 2 /2 = 1/ 2 \approx 0.707.$

L4.2 (connect to stability certificate)

The value function $V = x^{⊤} P x$ with $P ≻ 0$ is also a Lyapunov function for the closed loop. Verify $\dot{V} < 0$ along $\overset{x}{˙} = (A - B K) x$ using the CARE, and name what this proves.

Recall Solution

$\dot{V} = \overset{x}{˙}^{⊤} P x + x^{⊤} P \overset{x}{˙} = x^{⊤} [(A - B K)^{⊤} P + P (A - B K)] x .$ With $K = R^{- 1} B^{⊤} P$ , expand $(A - B K)^{⊤} P + P (A - B K) = A^{⊤} P + P A - 2 P B R^{- 1} B^{⊤} P$ . Use CARE ( $A^{⊤} P + P A = P B R^{- 1} B^{⊤} P - Q$ ): $= P B R^{- 1} B^{⊤} P - Q - 2 P B R^{- 1} B^{⊤} P = - (Q + P B R^{- 1} B^{⊤} P) .$ So $\dot{V} = - x^{⊤} (Q + P B R^{- 1} B^{⊤} P) x \leq 0$ , and it is $< 0$ whenever $Q ≻ 0$ . This proves asymptotic stability of the closed loop — see Lyapunov Stability. The optimal cost-to-go is a ready-made Lyapunov function.

L4.3 (why controllability is required)

The parent lists $(A, B)$ controllable as a condition for a stabilizing LQR solution. Give a $2 \times 2$ example where an uncontrollable, unstable mode makes LQR fail, and explain why.

Recall Solution

Take $A = [10 0 - 1]$ , $B = [01]$ . The controllability matrix is $[B A B] = [01 0 - 1]$ — rank 1, so the system is uncontrollable (see Controllability and Observability). The mode with eigenvalue $+ 1$ (first state) has no $B$ -coupling — control $u$ never touches $x_{1}$ . So $\overset{x}{˙}_{1} = x_{1}$ grows unboundedly no matter what $u$ does. No $K$ can stabilize it; LQR's stabilizing solution does not exist. Fix requires: every unstable mode must be reachable from $u$ (that is exactly stabilizability).

Level 5 — Mastery

L5.1 (full design from scratch)

A reaction wheel drives an attitude angle: $\ddot{θ} = u$ . Design an LQR with $Q = [9000]$ (penalize angle heavily, ignore rate), $R = [1]$ . Find $K$ , the closed-loop poles, and the settling behaviour.

Recall Solution

This is the double integrator with $x_{1} = θ, x_{2} = \dot{θ}$ . Redo CARE with $Q_{11} = 9$ , $ρ = 1$ :

$(1, 1) : 9 - p_{2}^{2} = 0 \Rightarrow p_{2} = 3.$
$(2, 2) : 2 p_{2} - p_{3}^{2} = 0 \Rightarrow p_{3} = 2 p_{2} = 6 \approx 2.449.$
$K = \frac{1}{ρ} [p_{2}, p_{3}] = [3, 6] .$ Closed loop $s^{2} + 6 s + 3 = 0 \Rightarrow s = \frac{- 6 \pm 6 - 12}{2} = - \frac{6}{2} \pm j \frac{6}{2} .$ Numerically $s \approx - 1.225 \pm 1.225 j$ . $ω_{n} = 3 \approx 1.732$ , $ζ = 6 / (23) = 1/ 2 \approx 0.707$ . Reading it: heavier $Q$ (9 vs 1) pushed the poles further left ( $ω_{n}$ up from 1 to $3$ ) — faster response, same damping. Exactly the " $Q$ = aggressive" story from L3.

L5.2 (prove the closed loop is always stable in the scalar case)

Prove that for any real $a$ (including unstable $a > 0$ ), the scalar LQR closed-loop pole $a - b K$ is strictly negative, given $b \neq = 0, q > 0, r > 0$ .

Recall Solution

$p = \frac{a r + a ^{2} r ^{2} + q r b ^{2}}{b ^{2}}$ and $K = \frac{b}{r} p$ , so $a - b K = a - \frac{b ^{2}}{r} p = a - \frac{1}{r} \cdot \frac{a r + a ^{2} r ^{2} + q r b ^{2}}{1} = a - a - \frac{a ^{2} r ^{2} + q r b ^{2}}{r} = - \frac{a ^{2} r ^{2} + q r b ^{2}}{r} .$ Since $q > 0, r > 0, b \neq = 0$ , the radicand $a^{2} r^{2} + q r b^{2} > 0$ , so the whole expression is strictly negative for every $a$ . LQR always stabilizes a controllable ( $b \neq = 0$ ) scalar plant. $■$

L5.3 (synthesis with the estimator — pointer)

You can only measure position, not velocity. Explain in words how you still run this LQR, and name the combined design.

Recall Solution

Feed the measurement into a Kalman Filter to produce an estimate $\overset{x}{^}$ of the full state (position and velocity), then apply $u = - K \overset{x}{^}$ . The separation principle says you may design $K$ (LQR) and the estimator independently; the combination is LQG Control (Linear-Quadratic-Gaussian). The LQR gain $K$ from this page is used unchanged — you just feed it $\overset{x}{^}$ instead of $x$ .

Recall One-line recap of the whole page

Riccati $\to P ⪰ 0 \to K = R^{- 1} B^{⊤} P \to u = - K x$ ; $Q$ = aggressive, $R$ = lazy, only ratio matters; the same $K$ is a PD controller (L4.1), a Lyapunov certificate (L4.2), and slots into LQG Control with an estimator (L5.3).