Linear Quadratic Regulator (LQR) — Riccati equation, optimal gains: Exercises

3.5.35 · D4 · Physics › Guidance, Navigation & Control (GNC) › Linear Quadratic Regulator (LQR) — Riccati equation, optimal

Shuru karne se pehle, ek picture poori vocabulary fix kar deti hai jo tumhe chahiye.

Level 1 — Recognition

L1.1

Continuous-time Algebraic Riccati Equation (CARE) state karo aur identify karo kaunsa single term ise $P$ mein nonlinear banata hai.

Recall Solution

$A^{⊤} P + P A - P B R^{- 1} B^{⊤} P + Q = 0.$ Term $- P B R^{- 1} B^{⊤} P$ $P$ mein quadratic hai (ismein $P$ do baar hai), isliye equation nonlinear hai. Baaki har term ( $A^{⊤} P$ , $P A$ , $Q$ ) $P$ mein linear ya constant hai.

L1.2

Tumhe bataya gaya hai $R = [2008]$ , $B = [01]$ , aur solved $P = [3114]$ . $K$ ka formula likho aur $K$ ka size (rows × columns) identify karo.

Recall Solution

$K = R^{- 1} B^{⊤} P$ . Yahan $R$ hai $2 \times 2$ , $B^{⊤}$ hai $1 \times 2$ , $P$ hai $2 \times 2$ . Ruko — $B$ hai $2 \times 1$ toh ek control input hai, isliye $R$ conformable hone ke liye $1 \times 1$ hona chahiye. Stated $R$ inconsistent hai; sahi recognition yeh hai: $m = 1$ control aur $n = 2$ states ke saath, $K$ hai $m \times n = 1 \times 2$ .

Exercise ka point: hamesha dimensions check karo. $u \in R^{m}$ , $x \in R^{n}$ , toh $u = - K x$ force karta hai $K$ ko $m \times n$ hone ke liye. Yahan $1 \times 2$ .

L1.3

In mein se optimal LQR control law kaun si hai? (a) $u = - K x^{2}$ (b) $u = - K x$ (c) $u = - K \overset{x}{˙}$ (d) $u = K x$ .

Recall Solution

(b) $u = - K x$ . Yeh $x$ mein linear hai ((a) rule out), state use karta hai derivative nahi ((c) rule out), aur sign negative hai — tum offset ke khilaaf push karte ho $x \to 0$ drive karne ke liye ((d) rule out, jo tumhe door push karta aur destabilize karta).

Level 2 — Application

L2.1 (scalar CARE)

Plant $\overset{x}{˙} = 2 x + u$ (toh $a = 2, b = 1$ ), weights $q = 4, r = 1$ . $p > 0$ , gain $K$ , aur closed-loop pole $a - b K$ nikalo.

Recall Solution

Scalar CARE: $2 a p - \frac{b ^{2}}{r} p^{2} + q = 0 \Rightarrow 4 p - p^{2} + 4 = 0 \Rightarrow p^{2} - 4 p - 4 = 0.$ $p = \frac{4 + 16 + 16}{2} = \frac{4 + 32}{2} = 2 + 22 \approx 4.828 (positive root) .$ $K = \frac{b}{r} p = p = 2 + 22 \approx 4.828.$ Closed loop: $a - b K = 2 - (2 + 22) = - 22 \approx - 2.828 < 0.$ Stable — jabki open loop ( $a = 2$ ) unstable tha.

L2.2 (double integrator gain)

Parent ke double-integrator result $K = [\frac{1}{ρ}, \frac{2}{ρ ^{1/4}}]$ ka use karke, $ρ = 16$ ke liye $K$ compute karo.

Recall Solution

$\frac{1}{16} = \frac{1}{4} = 0.25.$ Aur $ρ^{1/4} = 1 6^{1/4} = 2$ , toh $\frac{2}{2} = \frac{1}{2} \approx 0.707.$ $K = [0.25, 0.7071] .$ Notice karo: bada $ρ$ (heavier control penalty) ne dono gains shrink kar diye — softer pushing.

L2.3 (cost-to-go value)

L2.1 mein tumne $p \approx 4.828$ nikala. Agar satellite $x (0) = 3$ se start kare, toh total optimal cost $J = V (x_{0}) = p x_{0}^{2}$ kya hai?

Recall Solution

$V = p x_{0}^{2} = (2 + 22) (9) = 18 + 182 \approx 43.46.$ Value function $V = p x^{2}$ remaining cost hai agar tum yahan se optimally steer karo — ek baar $p$ pata chal jaaye toh koi integral nahi chahiye.

Level 3 — Analysis

L3.1 (sirf ratio matter karta hai)

Dikhao ki dono weights scale karne par, $Q \to α Q$ aur $R \to α R$ for $α > 0$ , gain $K$ unchanged rehta hai.

Recall Solution

Scaled weights ke saath CARE: $A^{⊤} P + P A - P B (α R)^{- 1} B^{⊤} P + α Q = 0.$ Try $P \to α P^{'}$ (guess ki solution bhi scale hoti hai). Substitute karo: $α A^{⊤} P^{'} + α P^{'} A - α^{2} P^{'} B \frac{1}{α} R^{- 1} B^{⊤} P^{'} + α Q = α (A^{⊤} P^{'} + P^{'} A - P^{'} B R^{- 1} B^{⊤} P^{'} + Q) = 0.$ Bracket original CARE hai, toh $P^{'}$ original solution hai aur $P = α P^{'}$ . Phir $K = (α R)^{- 1} B^{⊤} (α P^{'}) = \frac{1}{α} R^{- 1} B^{⊤} α P^{'} = R^{- 1} B^{⊤} P^{'} .$ $α$ 's cancel ho jaate hain — $K$ unchanged hai. Sirf ratio $Q / R$ controller set karta hai.

L3.2 (kaunsa knob aggressive banata hai?)

Scalar case mein $K = \frac{a r + a ^{2} r ^{2} + q r b ^{2}}{r b}$ ( $K = \frac{b}{r} p$ ka rewrite). Analyse karo: kya pole $a - b K$ left (faster) move karta hai jab $q$ badhta hai ya jab $r$ badhta hai?

Recall Solution

$a - b K = a - \frac{b ^{2}}{r} p$ . Jab $q ↑$ , $p ↑$ (CARE mein bada $q$ bada $p$ force karta hai), toh $\frac{b ^{2}}{r} p ↑$ , toh $a - b K$ zyaada negative ho jaata hai → pole left move karta hai → faster/aggressive. Jab $r ↑$ with $q$ fixed: control par penalty badhti hai, $K$ shrink hota hai, $a - b K$ $a$ ki taraf (right) move karta hai → slower/gentler. Verdict: bada $Q$ = aggressive, bada $R$ = lazy.

L3.3 (double integrator ke closed-loop poles)

$ρ = 1$ ke liye, $K = [1, 2]$ . Closed-loop matrix $A - B K$ aur uske eigenvalues nikalo; damping ratio state karo.

Recall Solution

$A - B K = [0010] - [01] [12] = [0 - 1 1 - 2] .$ Characteristic polynomial: $det (s I - (A - B K)) = s^{2} + 2 s + 1 = 0.$ $s = \frac{- 2 \pm 2 - 4}{2} = - \frac{1}{2} \pm j \frac{1}{2} .$ $s^{2} + 2 ζ ω_{n} s + ω_{n}^{2}$ se compare karo with $ω_{n} = 1$ : $2 ζ = 2 \Rightarrow ζ = \frac{1}{2} \approx 0.707.$ Achha damped — overshoot ringing practically nahi.

Varying $ρ$ ke liye poles ek curve trace karte hain — figure dekho.

Level 4 — Synthesis

L4.1 (LQR is PD control)

Double-integrator gain $K = [k_{p}, k_{d}]$ deta hai $u = - k_{p} x_{1} - k_{d} x_{2}$ . Term by term explain karo kyun yeh exactly ek spring–damper (PD) controller hai, aur $ρ = 1$ ke liye $(k_{p}, k_{d})$ do aur hence natural frequency aur damping.

Recall Solution

$x_{1}$ = position, $x_{2}$ = velocity. Toh $u = - k_{p} x_{1} - k_{d} x_{2}$ :

$- k_{p} x_{1}$ = ek spring jo position ko zero pe wapas kheenchti hai (proportional term, "P").
$- k_{d} x_{2}$ = ek damper jo velocity ko oppose karta hai (derivative term, "D"). Yeh precisely ek PD controller hai — dekho PID Controllers. LQR ne ise optimal ke roop mein derive kiya. $ρ = 1$ ke liye: $k_{p} = 1, k_{d} = 2$ . Closed loop $\overset{x}{¨}_{1} + k_{d} \overset{x}{˙}_{1} + k_{p} x_{1} = 0$ , yaani $ω_{n} = k_{p} = 1$ , $ζ = k_{d} / (2 ω_{n}) = 2 /2 = 1/ 2 \approx 0.707.$

L4.2 (stability certificate se connect karo)

Value function $V = x^{⊤} P x$ with $P ≻ 0$ closed loop ke liye ek Lyapunov function bhi hai. CARE use karke $\overset{x}{˙} = (A - B K) x$ along $\dot{V} < 0$ verify karo, aur batao yeh kya prove karta hai.

Recall Solution

$\dot{V} = \overset{x}{˙}^{⊤} P x + x^{⊤} P \overset{x}{˙} = x^{⊤} [(A - B K)^{⊤} P + P (A - B K)] x .$ $K = R^{- 1} B^{⊤} P$ ke saath, expand karo $(A - B K)^{⊤} P + P (A - B K) = A^{⊤} P + P A - 2 P B R^{- 1} B^{⊤} P$ . CARE use karo ( $A^{⊤} P + P A = P B R^{- 1} B^{⊤} P - Q$ ): $= P B R^{- 1} B^{⊤} P - Q - 2 P B R^{- 1} B^{⊤} P = - (Q + P B R^{- 1} B^{⊤} P) .$ Toh $\dot{V} = - x^{⊤} (Q + P B R^{- 1} B^{⊤} P) x \leq 0$ , aur yeh $< 0$ hai jab bhi $Q ≻ 0$ . Yeh closed loop ki asymptotic stability prove karta hai — dekho Lyapunov Stability. Optimal cost-to-go khud ek ready-made Lyapunov function hai.

L4.3 (kyun controllability required hai)

Parent $(A, B)$ controllable ko stabilizing LQR solution ke liye condition list karta hai. Ek $2 \times 2$ example do jahan ek uncontrollable, unstable mode LQR fail kara de, aur explain karo kyun.

Recall Solution

Lo $A = [10 0 - 1]$ , $B = [01]$ . Controllability matrix hai $[B A B] = [01 0 - 1]$ — rank 1, toh system uncontrollable hai (dekho Controllability and Observability). Eigenvalue $+ 1$ wala mode (first state) ka $B$ -coupling nahi hai — control $u$ kabhi $x_{1}$ ko touch nahi karta. Toh $\overset{x}{˙}_{1} = x_{1}$ unboundedly grow karta hai chahe $u$ kuch bhi kare. Koi $K$ ise stabilize nahi kar sakta; LQR ka stabilizing solution exist nahi karta. Fix ke liye: har unstable mode $u$ se reachable hona chahiye (yahi exactly stabilizability hai).

Level 5 — Mastery

L5.1 (scratch se full design)

Ek reaction wheel attitude angle drive karta hai: $\ddot{θ} = u$ . LQR design karo $Q = [9000]$ (angle heavily penalize karo, rate ignore karo), $R = [1]$ ke saath. $K$ , closed-loop poles, aur settling behaviour nikalo.

Recall Solution

Yeh double integrator hai $x_{1} = θ, x_{2} = \dot{θ}$ ke saath. $Q_{11} = 9$ , $ρ = 1$ ke saath CARE redo karo:

$(1, 1) : 9 - p_{2}^{2} = 0 \Rightarrow p_{2} = 3.$
$(2, 2) : 2 p_{2} - p_{3}^{2} = 0 \Rightarrow p_{3} = 2 p_{2} = 6 \approx 2.449.$
$K = \frac{1}{ρ} [p_{2}, p_{3}] = [3, 6] .$ Closed loop $s^{2} + 6 s + 3 = 0 \Rightarrow s = \frac{- 6 \pm 6 - 12}{2} = - \frac{6}{2} \pm j \frac{6}{2} .$ Numerically $s \approx - 1.225 \pm 1.225 j$ . $ω_{n} = 3 \approx 1.732$ , $ζ = 6 / (23) = 1/ 2 \approx 0.707$ . Padho: heavier $Q$ (9 vs 1) ne poles ko further left push kiya ( $ω_{n}$ 1 se $3$ tak up) — faster response, same damping. Exactly L3 ki " $Q$ = aggressive" story.

L5.2 (prove karo ki scalar case mein closed loop hamesha stable hai)

Prove karo ki kisi bhi real $a$ ke liye (unstable $a > 0$ bhi), scalar LQR closed-loop pole $a - b K$ strictly negative hai, given $b \neq = 0, q > 0, r > 0$ .

Recall Solution

$p = \frac{a r + a ^{2} r ^{2} + q r b ^{2}}{b ^{2}}$ aur $K = \frac{b}{r} p$ , toh $a - b K = a - \frac{b ^{2}}{r} p = a - \frac{1}{r} \cdot \frac{a r + a ^{2} r ^{2} + q r b ^{2}}{1} = a - a - \frac{a ^{2} r ^{2} + q r b ^{2}}{r} = - \frac{a ^{2} r ^{2} + q r b ^{2}}{r} .$ Kyunki $q > 0, r > 0, b \neq = 0$ , radicand $a^{2} r^{2} + q r b^{2} > 0$ , toh poora expression har $a$ ke liye strictly negative hai. LQR hamesha ek controllable ( $b \neq = 0$ ) scalar plant ko stabilize karta hai. $■$

L5.3 (estimator ke saath synthesis — pointer)

Tum sirf position measure kar sakte ho, velocity nahi. Explain karo words mein ki tum phir bhi yeh LQR kaise run karte ho, aur combined design ka naam batao.

Recall Solution

Measurement ko ek Kalman Filter mein feed karo taaki full state (position aur velocity) ka estimate $\overset{x}{^}$ mile, phir $u = - K \overset{x}{^}$ apply karo. Separation principle kehta hai ki tum $K$ (LQR) aur estimator independently design kar sakte ho; combination hai LQG Control (Linear-Quadratic-Gaussian). Is page ka LQR gain $K$ unchanged use hota hai — bas $x$ ki jagah $\overset{x}{^}$ feed karo.

Recall Poore page ka one-line recap

Riccati $\to P ⪰ 0 \to K = R^{- 1} B^{⊤} P \to u = - K x$ ; $Q$ = aggressive, $R$ = lazy, sirf ratio matter karta hai; wahi $K$ ek PD controller hai (L4.1), ek Lyapunov certificate hai (L4.2), aur ek estimator ke saath LQG Control mein fit hota hai (L5.3).