Linear Quadratic Regulator (LQR) — Riccati equation, optimal gains

3.5.35 · Physics › Guidance, Navigation & Control (GNC)

KYA hai problem?

YE cost kyun? Ye engineering trade-offs ka steel-man hai. Error ko instantly zero nahi kar sakte — uske liye infinite control chahiye. Saara fuel bhi nahi bacha sakte — system drift karega. Quadratic form ka integral in dono ke beech trade karne ka sabse smooth tarika hai, aur quadratics math ko closed form mein solvable banate hain.

Quadratic kyun ( $∣ x ∣$ ya $x^{4}$ nahi)? Kyunki linear plant + quadratic cost ke saath, optimal control state mein linear nikalta hai ( $u = - K x$ ). Yahi magical combination hai: Linear + Quadratic → Linear controller.

KAISE: Optimal control first principles se derive karo

Hum dynamic programming / value function (Hamilton–Jacobi–Bellman) use karte hain.

Step 1 — Value function guess karo

State $x$ se optimal "cost-to-go" define karo: $V (x) = min_{u (\cdot)} \int_{t}^{\infty} (x^{⊤} Q x + u^{⊤} R u) d t .$

$V (x) = x^{⊤} P x$ (jahan $P = P^{⊤} ⪰ 0$ ) guess kyun karte hain? Kyunki cost quadratic hai aur dynamics linear, isliye cost-to-go bhi ek quadratic form hona chahiye — koi linear ya constant term nahi hoga ( $x = 0$ pe cost 0 hai), aur linear dynamics mein koi higher power survive nahi karti.

Step 2 — HJB equation likho

Bellman principle kehta hai: optimum pe, instantaneous cost plus cost-to-go ki rate of change zero hoti hai: $0 = min_{u} [x^{⊤} Q x + u^{⊤} R u + \nabla V^{⊤} (A x + B u)] .$ $V = x^{⊤} P x$ ke saath, $\nabla V = 2 P x$ hai, toh: $0 = min_{u} [x^{⊤} Q x + u^{⊤} R u + 2 x^{⊤} P (A x + B u)] .$

Step 3 — $u$ ke upar minimize karo (isse gain milta hai)

$\partial / \partial u$ lo aur zero set karo: $\frac{\partial}{\partial u} [u^{⊤} R u + 2 x^{⊤} P B u] = 2 R u + 2 B^{⊤} P x = 0.$

Ye step kyun? Bracket $u$ mein convex hai (kyunki $R ≻ 0$ ), isliye stationary point global minimum hai. Solve karne pe: $u^{*} = - R^{- 1} B^{⊤} P x = - K x, K = R^{- 1} B^{⊤} P$

Step 4 — $u^{*}$ wapas substitute karo → Riccati equation

$u^{*} = - R^{- 1} B^{⊤} P x$ ko HJB bracket mein daalo. Note karo $u^{* ⊤} R u^{*} = x^{⊤} P B R^{- 1} B^{⊤} P x$ aur $2 x^{⊤} P B u^{*} = - 2 x^{⊤} P B R^{- 1} B^{⊤} P x$ . Ye combine hokar $- x^{⊤} P B R^{- 1} B^{⊤} P x$ dete hain: $0 = x^{⊤} (Q + A^{⊤} P + P A - P B R^{- 1} B^{⊤} P) x .$ Ye sabhi $x$ ke liye hold karna chahiye, isliye brackets mein matrix zero hai:

"Algebraic" kyun hai? Infinite-horizon steady state mein $\dot{P} = 0$ hota hai, isliye differential Riccati equation collapse hokar is algebraic wale mein aa jaati hai. Nonlinear kyun? Term $- P B R^{- 1} B^{⊤} P$ $P$ mein quadratic hai.

Worked Example 1 — Scalar system (poora haath se karo)

Plant: $\overset{x}{˙} = a x + b u$ jahan $a, b$ scalars hain; cost weights $q > 0, r > 0$ .

Riccati (scalars): $2 a p - \frac{b ^{2}}{r} p^{2} + q = 0.$

Kyun? $A^{⊤} P + P A = 2 a p$ ; $P B R^{- 1} B^{⊤} P = \frac{b ^{2}}{r} p^{2}$ ; $Q = q$ add karo.

$p$ ke liye quadratic solve karo (positive root lo, kyunki $P ⪰ 0$ ): $p = \frac{a r + a ^{2} r ^{2} + q r b ^{2}}{b ^{2}} .$ Positive root kyun? $V = p x^{2}$ ek cost hai — non-negative hona chahiye; sirf $+$ root $p > 0$ deta hai.

Gain: $K = \frac{b}{r} p$ . Closed loop: $\overset{x}{˙} = (a - b K) x$ .

Stability check: $a - b K = a - \frac{b ^{2}}{r} p < 0$ hamesha, toh controller hamesha stabilize karta hai chahe open-loop plant unstable ho ( $a > 0$ ).

Numbers: $a = 1$ (unstable!), $b = 1$ , $q = 1$ , $r = 1$ : $p = 1 + 1 + 1 = 1 + 2 \approx 2.414, K = p \approx 2.414, a - b K \approx - 1.414 < 0.$ Stable. ✔

Worked Example 2 — Double integrator (ek real GNC case)

Ek satellite axis: position $x_{1}$ , velocity $x_{2}$ , control acceleration $u$ : $A = [0010], B = [01], Q = [1000], R = [ρ] .$

Maano $P = [p_{1} p_{2} p_{2} p_{3}]$ . CARE expand karo. Key equations:

$(1, 1) : 1 - p_{2}^{2} / ρ = 0 \Rightarrow p_{2} = ρ .$
$(1, 2) : p_{1} - p_{2} p_{3} / ρ = 0 \Rightarrow p_{1} = p_{2} p_{3} / ρ .$
$(2, 2) : 2 p_{2} - p_{3}^{2} / ρ = 0 \Rightarrow p_{3} = 2 ρ p_{2} = 2 ρ^{3/4} .$

Gain: $K = R^{- 1} B^{⊤} P = \frac{1}{ρ} [p_{2}, p_{3}] = [\frac{1}{ρ}, \frac{2}{ρ ^{1/4}}] .$

Ye kyun important hai: $K = [k_{p}, k_{d}]$ exactly ek PD controller hai! Pehla gain position pe act karta hai (spring), doosra velocity pe (damper). LQR ne PD control optimally derive kar diya. $ρ = 1$ ke saath: $K = [1, 2]$ , closed-loop poles $- \frac{1}{2} \pm j \frac{1}{2}$ pe aate hain — ek achha damped response (damping ratio $= 1/ 2$ ).

Common mistake Steel-manned common errors

1. "Bada $R$ = faster response." Sahi lagta hai: $R$ ek knob hai, zyada karna aggressive lagta hai. Galat: bada $R$ control ko zyada penalize karta hai, toh controller kam effort use karta hai → slower, gentler response. Bada $Q$ aggressive banata hai. Fix: $Q$ / $R$ ratio aggressiveness set karta hai; sirf ratio matter karta hai (dono scale karne se kuch nahi badalta).

2. Galat Riccati root lena. $P \neq ⪰ 0$ wala root lene se gain milta hai jo destabilize kar deta hai. Fix: hamesha stabilizing solution lo — unique $P ⪰ 0$ jo $A - B K$ ko Hurwitz banata hai ( $(A, B)$ controllable aur $(A, Q)$ observable hona chahiye).

3. $R^{- 1}$ bhool jaana. Log likhte hain $K = B^{⊤} P$ . Fix: $R ≻ 0$ isliye required hai taaki $R^{- 1}$ exist kare — yahi reason hai ki $R$ strictly positive definite hona chahiye (sirf $⪰ 0$ nahi).

4. Riccati mein sign. $+ P B R^{- 1} B^{⊤} P$ likhna. Fix: quadratic term negative hai: $- P B R^{- 1} B^{⊤} P$ . Yahi woh cheez hai jo optimizer act karke "bachata" hai.

Recall Feynman: 12-saal ke bachche ko explain karo

Socho ek hilaate-huye shopping cart ko kisi jagah steer karna. Jagah se door hona irritating hai (yahi $Q$ hai), lekin cart ko bahut zyada push karna tiring aur jerky hai (yahi $R$ hai). LQR har situation ke liye perfect amount of push figure out karta hai taaki tum smoothly pahuncho bina khud ko thakaye. Aur amazing baat ye hai: perfect rule simply ye hai ki "jitna door ho utna hi push karo" — ek recipe ( $u = - K x$ ) jo har jagah kaam aati hai. Riccati equation woh puzzle hai jo tum ek baar solve karke woh recipe paate ho.

Active Recall

LQR kaunsi cost minimize karta hai?

J = \int_{0}^{\infty} (x^{⊤} Q x + u^{⊤} R u) d t

— weighted state error + control effort.

Optimal LQR control law kya hai?

u^{*} = - K x

jahan

K = R^{- 1} B^{⊤} P

Continuous-time Algebraic Riccati Equation batao.

A^{⊤} P + P A - P B R^{- 1} B^{⊤} P + Q = 0

Optimal controller x mein linear kyun hai?

Linear plant + quadratic cost ⇒ quadratic value function

V = x^{⊤} P x

⇒ minimize karne se linear

u = - K x

milta hai.

Kaunsa Riccati root lete hain?

Unique symmetric positive-semidefinite (stabilizing)

P

A - B K

ko Hurwitz banata hai.

R badhane ka effect?

Control ko zyada penalize karta hai ⇒ kam effort ⇒ slower, gentler response.

Q badhane ka effect?

Error ko zyada penalize karta hai ⇒ zyada aggressive, faster response.

R positive definite kyun hona chahiye (sirf PSD nahi)?

Taaki

K = R^{- 1} B^{⊤} P

mein

R^{- 1}

exist kare (aur cost

u

mein strictly convex ho).

Stabilizing LQR solution guarantee karne ki conditions?

(A, B)

controllable aur

(A, Q)

observable.

Double integrator ke liye LQR kaunsa classical controller reproduce karta hai?

PD controller: position aur velocity pe

K = [k_{p}, k_{d}]

Optimal value (cost-to-go) function kis form mein hoti hai?

V (x) = x^{⊤} P x

Riccati equation kahaan se aati hai?

Optimal

u^{*}

ko HJB equation mein substitute karke aur require karo ki ye sabhi

x

ke liye hold kare.

Connections

Hamilton-Jacobi-Bellman Equation — LQR iski sabse clean closed-form solution hai.
Controllability and Observability — valid stabilizing $P$ ke liye zaroori hai.
Pole Placement — LQR ka alternative; LQR optimally poles place karta hai.
Kalman Filter — dual Riccati equation; combine karo → LQG Control.
PID Controllers — LQR optimally PD gains generalize/derive karta hai.
Lyapunov Stability — $V = x^{⊤} P x$ closed loop ke liye Lyapunov function hai.
State-Space Representation — $\overset{x}{˙} = A x + B u$ ki foundation.

Linear Quadratic Regulator (LQR) — Riccati equation, optimal gains

KYA hai problem?

KAISE: Optimal control first principles se derive karo

Step 1 — Value function guess karo

Step 2 — HJB equation likho

Step 3 — $u$ ke upar minimize karo (isse gain milta hai)

Step 4 — $u^{*}$ wapas substitute karo → Riccati equation

Worked Example 1 — Scalar system (poora haath se karo)

Worked Example 2 — Double integrator (ek real GNC case)

Active Recall

Connections

Concept Map

Deep Dive