80/20 core: neeche sab kuch ek hi move par reduce hota hai — unknown dynamics ko (kuch) × (unknown parameters) ke roop mein likho, phir least squares solve karo. Yeh master kar lo toh system ID ka 80% ho gaya.
Sabse simple, sabse zyada use hone wala case karte hain: ek linear discrete-time model
xk+1=Axk+Buk.
Step 1 — Unknowns stack karo.Kyun?A,B woh hain jo chahiye; unhe group karo:
Θ=[AB],zk=[xkuk].
Phir model simply xk+1=Θzk ho jaata hai. Yeh hai key trick: dynamics = matrix × features.
Step 2 — Data ko matrices mein collect karo.Kyun? Har time step par ek equation; unhe stack karo taaki sab ek saath solve ho sakein.
X+=[x1x2⋯xN],Z=[z0z1⋯zN−1].
Saare data par model padhta hai X+≈ΘZ.
Step 3 — Cost define karo.Kyun? Data noisy hai, isliye exactly hit nahi kar sakte; squared mismatch minimize karo (Gaussian-noise MLE):
J(Θ)=∥X+−ΘZ∥F2=tr[(X+−ΘZ)(X+−ΘZ)⊤].
Step 4 — Derivative zero karo.Kyun? Convex quadratic ka minimum.
∂Θ∂J=−2(X+−ΘZ)Z⊤=0.
Rearrange karo: X+Z⊤=ΘZZ⊤.
Woh Z† hi ek symbol mein poora subject hai: yeh least-squares wala "data se divide karna" hai.
Real dynamics linear nahi hoti. Lekin hum linear-in-parameters form rakh sakte hain:
x˙=Θϕ(x,u),ϕ=[1,x,u,x2,xu,sinx,…]⊤.
Data par evaluate kiye gaye candidate functions ki ek badi matrix Φ banao, phir wohi least-squares solve karo — optionally sparsity ke saath (chhote coefficients drop karo) taaki recovered model ke terms kam aur interpretable hon. Yahi SINDy hai (Sparse Identification of Nonlinear Dynamics).
\Theta = X_+Z^{-1}=[0.9\;\;1.1].$$
Toh $a=0.9,\ b=1.1$. **Check:** row 0: $0.9(1)+1.1(1)=2.0$ ✓; row 1: $0.9(2)+1.1(0)=1.8$ ✓.
## Worked Example 2 — kyun excitation matter karta hai
Maano poori flight mein har $u_k=0$ raha. Toh $Z$ ki $u$-row sab zeros hai, isliye $ZZ^\top$ singular hai → $b$ **unidentifiable** hai. *Yeh step kyun matter karta hai:* tumne literally input channel ko kabhi poke nahi kiya, isliye koi data $b$ nahi bata sakta. **Fix:** ek rich signal inject karo (chirp / multisine / PRBS) — *persistent excitation*.
## Worked Example 3 — library se nonlinear
Pendulum-jaisa data suspect hai: $\dot\theta_2 = -c\sin\theta_1$. Use karo $\phi=[1,\ \theta_1,\ \sin\theta_1]$. Least squares coefficients $\approx[0,\ 0,\ -c]$ return karta hai; sparsity pehle do zero kar deta hai. **Library form kyun?** Yeh fit ko *unknowns mein linear* rakhta hai, chahe model state mein nonlinear ho — isliye wohi $\Theta=\dot X\,\Phi^\dagger$ formula kaam karta hai.
---
> [!recall]- Feynman: 12-saal ke bacche ko samjhao
> Socho tumhare paas ek toy car hai aur tumhe nahi pata woh kaise chalti hai. Tum usse bahut baar film karte ho: "jab maine joystick *itna* push kiya, woh *wahan* pahunch gayi." Phir tum ek chhota math machine banate ho jo current jagah aur tumhare push se agla spot guess karta hai. Tum machine ko tab tak tweak karte ho jab tak uske guesses tumhari films se match nahi karne lagte. Ab tum predict kar sakte ho car push karne se *pehle* — yahi system identification hai! Ek rule: car ko *bahut tarike se* drive karna padega, warna tum kabhii nahi seekhoge ki woh buttons jo tumne kabhi press nahi kiye woh actually kya karte hain.
---
## Common mistakes (steel-manned)
> [!mistake] "Zyada candidate features → better model."
> **Sahi lagta hai:** zyada functions = zyada flexibility. **Reality:** tum noise ko overfit karte ho aur $ZZ^\top$ ill-conditioned ho jaata hai. **Fix:** sparsity/regularization ($L_1$, ya ridge $\Theta = X_{+}Z^\top(ZZ^\top+\lambda I)^{-1}$) aur *held-out* data par validate karo.
> [!mistake] "Low training error = achha dynamics model."
> **Sahi lagta hai:** data match kiya. **Fix:** hamesha **multi-step rollout** test karo — predictions ko wापस feed karo. Chhote one-step errors ek trajectory mein compound hote hain; ek achhe ID model ko *simulate forward* karne par bhi stable rehna chahiye.
> [!mistake] "Main raw sensor data differentiate karke $\dot{x}$ le sakta hun."
> **Sahi lagta hai:** derivative sirf ek finite difference hai. **Fix:** finite differences high-frequency noise amplify karte hain; pehle smooth karo (Savitzky–Golay, TV-regularized differentiation).
---
## Forecast-then-Verify
> [!example] Compute karne se pehle predict karo
> **Q:** Example 1 mein, agar mere paas ek *teesra* data row hota jo pehle do ka exact linear combo hai, toh kya $\Theta$ change hoga? **Forecast:** Nahi — yeh koi nayi information nahi deta; least squares already span fit kar chuka hai. **Verify:** $ZZ^\top$ rank unchanged, solution identical. ✓
---
## #flashcards/coding
System identification ka goal kya hai? ::: Measured state/input data se prediction error minimize karke ek predictive dynamics model $x_{k+1}=f_\theta(x_k,u_k)$ (ya $\dot x=f_\theta$) recover karna.
Linear model $X_+ = \Theta Z$ ke liye least-squares solution batao. ::: $\Theta = X_+Z^\top(ZZ^\top)^{-1} = X_+Z^\dagger$ (Moore–Penrose pseudoinverse).
$ZZ^\top$ singular kyun ho sakta hai, aur fix kya hai? ::: Jab inputs/states sab directions excite nahi karte (poor excitation). Fix: rich inputs use karo (persistent excitation: chirp, PRBS, multisine).
Woh key property kya hai jo nonlinear SINDy ko phir bhi linear least squares use karne deti hai? ::: Model ko **parameters mein linear** banaya jaata hai: $\dot x = \Theta\,\phi(x,u)$ ek fixed feature/candidate library $\phi$ ke saath.
Practice mein continuous-time ID ke bajaye discrete-time ID kyun prefer karein? ::: Continuous ko noisy data se $\dot x$ chahiye (numerical differentiation noise amplify karta hai); discrete-time differentiation avoid karta hai.
One-step training error ek misleading metric kyun hai? ::: Multi-step rollout mein errors compound hote hain; simulated trajectories validate karni chahiye, sirf next-step predictions nahi.
$ZZ^\top$ se divide karna physically kya represent karta hai? ::: Input/state excitation "energy" (ek covariance) se down-weighting; jinhe barely explore kiya gaya woh directions down-weight hoti hain.
Ridge-regularized dynamics fit batao. ::: $\Theta = X_{+}Z^\top(ZZ^\top+\lambda I)^{-1}$ with $\lambda>0$ (aur sparsity ke liye $L_1$/thresholding, jaise SINDy mein).
---
## Connections
- [[Least Squares and the Pseudoinverse]]
- [[Kalman Filter — State Estimation]]
- [[Model Predictive Control (MPC)]]
- [[SINDy — Sparse Identification of Nonlinear Dynamics]]
- [[Persistent Excitation and Experiment Design]]
- [[Regularization — Ridge and Lasso]]
- [[Digital Twins in Aerospace]]
- [[Numerical Differentiation of Noisy Signals]]
## 🖼️ Concept Map
```mermaid
flowchart TD
HIDDEN[Hidden EOM xdot = f x u] -->|recovered by| SYSID[System Identification]
DATA[Flight-test data x u] -->|feeds| SYSID
SYSID -->|minimizes| PRED[Prediction error]
SYSID -->|two flavours| DISC[Discrete-time model]
SYSID -->|two flavours| CONT[Continuous-time model]
CONT -->|needs| DERIV[Estimate xdot from noisy x]
DERIV -->|amplifies| NOISE[Noise problem]
DISC -->|core move| LLS[Linear least squares]
LLS -->|key trick| FEAT[Dynamics = matrix x features]
FEAT -->|stack unknowns| THETA[Theta = A B]
SYSID -->|model feeds| APPS[MPC / Kalman / Digital twin]
```
