ML project lifecycle — Hinglish notes

5.3.1 · AI-ML › MLOps & Deployment

Lifecycle exist kyu karta hai?

Canonical stages (Google/Andrew Ng framing):

Scoping — problem define karo, metrics, feasibility, value.
Data — collect, label, clean karo, baseline establish karo.
Modeling — train karo, error-analyze karo, iterate karo.
Deployment — real users/systems ko predictions serve karo.
Monitoring & Maintenance — drift detect karo, retrain karo, loop back karo.

Stage 1 — Scoping (sabse zyada leverage wala 20%)

HOW (checklist):

Business/user problem aur us decision ko identify karo jo woh drive karta hai.
Ek north-star metric aur ek satisficing metric chuno (jaise maximize accuracy subject to latency $< 100$ ms).
Feasibility estimate karo (kya human-level performance possible hai? kya data mein signal hai?) aur value (kya 2% lift matter karta hai?).

Stage 2 — Data

Formula

Avoidable bias vs variance decomposition (isliye baselines matter karti hain): Woh typical, well-behaved ordering jo hum diagnostic ke taur par use karte hain woh hai $best achievable Bayes/HLP ≲ DevErr, TrainErr \leq DevErr$

Avoidable bias $= TrainErr - HLP$ → model/features par focus karo.
Variance $= DevErr - TrainErr$ → zyada/cleaner data, regularization par focus karo.

Important caveat (kyun $Bayes \leq TrainErr$ ek law NAHI hai): ek model overfit kar sakta hai aur training error ko Bayes error / human-level performance ke neeche le ja sakta hai (woh noise memorize karta hai). Isliye HLP sirf best achievable generalization (dev/test) error ke liye ek floor hai, training error par ek strict lower bound nahi. Agar kabhi $TrainErr < HLP$ dikhe, toh yeh overfitting ka ek red flag hai, aur "avoidable bias" negative/meaningless ho jaata hai — tab real signal poori tarah variance term mein rehta hai.

Kyun is tarah derive kiya: honest decomposition generalization error par hai: $DevErr \approx irreducible Bayes + bias, may be negative if overfit (TrainErr - Bayes) + variance / generalization gap (DevErr - TrainErr)$ . Gap ko split karna batata hai ki kaunsa lever kheencho, taaki tum galat cheez tune karne mein weeks waste na karo.

Stage 3 — Modeling

Model-centric ("fixed data par model improve karo") vs data-centric ("fixed model ke liye data improve karo") — modern practice data-centric ki taraf jhukti hai kyunki clean data better generalize karta hai.

Stage 4 — Deployment

Deployment patterns (har ek blast radius reduce karta hai):

Shadow deployment — model live traffic par run karta hai lekin uska output use nahi hota; tum silently compare karte ho.
Canary deployment — traffic ka ek chhota % (jaise 5%) route karo, dekho, phir ramp up karo.
Blue-green deployment — do full environments, traffic instantly switch karo, instantly roll back karo.

Automation ke degrees: Human only → Shadow → AI assist → Partial autonomy → Full autonomy.

Stage 5 — Monitoring & Maintenance

Worked Examples

Common Mistakes (Steel-manned)

Active Recall

Recall Quick self-test (answers chhupaao, pehle forecast karo)

5 stages ko order mein name karo.
Data drift aur concept drift mein kya fark hai?
Given HLP 1%, Train 5%, Dev 6% — tum kahan focus karte ho?
Pehle rollout ke liye shadow deployment canary se safer kyun hai?
Expected-value-of-project formula likho aur har term name karo.
Iska kya matlab hai agar TrainErr HLP se neeche aaye?

Recall Feynman: ek 12-saal ke bacche ko explain karo

Imagine karo tum ek puppy (model) ko pichhli garmi ke treats use karke train karte ho. Tum define karte ho ki "accha behavior" matlab kya hai (scoping), treats collect karte ho aur examples dikhate ho (data), puppy ko sikhate ho (modeling), phir use ek party mein loose chhor dete ho (deployment). Lekin party mein log practice se alag behave karte hain, isliye tum dekhte rehte ho aur jab woh garbad karta hai toh dobara sikhane jaate ho (monitoring). Tum kabhi bhi dekhna band nahi karte — isliye yeh ek loop hai, ek baar ka trick nahi.

Connections

Data drift and concept drift
Model monitoring and observability
Deployment strategies (shadow, canary, blue-green)
Bias-variance decomposition
Error analysis
Data-centric vs model-centric AI
CI-CD for ML pipelines
Human-level performance and Bayes error
Overfitting and generalization gap

ML project lifecycle ke 5 stages order mein kya hain?

Scoping → Data → Modeling → Deployment → Monitoring & Maintenance.

ML lifecycle ek loop kyun hai, line kyun nahi?

Kyunki deployment drift aur errors reveal karta hai jo earlier stages (data/modeling) mein retraining ke liye feed back hote hain.

Data drift aur concept drift mein kya fark hai?

Data drift = P(x) changes; concept drift = P(y|x) changes (same input, alag correct label).

Avoidable bias define karo aur yeh kaise compute hoti hai.

Avoidable bias = TrainErr − Human/Bayes level; bada positive value → model/features improve karo.

Error-decomposition sense mein variance define karo.

Variance = DevErr − TrainErr; badi value → zyada/cleaner data lo ya regularize karo.

Kya HLP/Bayes error TRAINING error par ek strict lower bound hai?

Nahi. Ek overfit model HLP/Bayes se neeche training error achieve kar sakta hai; HLP sirf best achievable generalization (dev/test) error ko floor karta hai.

Agar TrainErr < HLP ho toh yeh kya signal karta hai?

Overfitting — model noise memorize karta hai; avoidable bias negative/meaningless ho jaati hai, isliye variance (generalization) gap par focus karo.

Shadow deployment kya karta hai?

Model ko live traffic par run karta hai lekin uske outputs ignore karta hai, tumhe current system se silently compare karne deta hai.

Canary deployment kya karta hai?

Traffic ka ek chhota fraction new model ki taraf route karta hai, monitor karta hai, phir gradually ramp up karta hai.

Expected-value-of-project formula likho.

E[Value] = p_success · V_success − C_build − C_maintain.

Ek lower-accuracy model better deployment choice kyun ho sakta hai?

Kyunki north-star constraints ke saath ek business metric hai (latency, cost, fairness), raw accuracy nahi.

Baseline kya hai aur ise jaldi establish kyun karte hain?

Ek simple reference (human level / heuristic / trivial model) jo dikhata hai ki real model value add karta hai ya nahi aur kitna headroom exist karta hai.

Data-centric vs model-centric AI?

Data-centric fixed model ke liye data improve karta hai; model-centric fixed data par model improve karta hai.