CI - CD pipelines for ML — Hinglish notes

5.3.11 · AI-ML › MLOps & Deployment

KIYA KYA HAIN pieces?

PIPELINE ka flow KAISE hota hai

Stages, order mein:

Trigger — ek git push, ek schedule (cron), ya ek drift alarm.
CI: validate — lint code, unit tests run karo, data schema validate karo (kya columns/types/ranges theek hain?), ek tiny training run karo crashes pakdne ke liye.
Train — pipeline pe full training (laptop pe nahi), ek model artifact produce karta hai.
Evaluate (gate) — held-out set pe metrics compute karo; current production model se compare karo.
Register — agar pass ho jaaye, artifact + metadata ko ek model registry mein push karo (versioned).
CD: deploy — package karo (Docker), staging pe roll out karo, integration tests chalaao, phir production.
Monitor — live metrics dekho; drift/degradation Trigger ko feed back karta hai → loop close hoti hai.

Formula Promotion gate (WHY ek formula, feeling nahi)

Hume deploy karne ka yes/no decision automate karna padega. Naye model ka score $s_{new}$ aur incumbent ka $s_{prod}$ define karo same held-out set pe. Promote tab hi karo jab woh strictly, meaningfully better ho: $promote ⟺ s_{new} \geq s_{prod} + δ$

$δ$ (ek margin) kyun? Metrics mein noise hoti hai. Margin ke bina tum random fluctuations pe models churn karte rehoge. $δ$ tumhara "kya yeh real hai?" threshold hai.
Same test set kyun? Taaki comparison apples-to-apples ho — warna tum skill differences ko dataset differences se compare kar rahe hote.

Noise se ek safe $δ$ nikalna: agar metric ek proportion (accuracy) hai $n$ test points pe, toh uska standard error hai $S E = \frac{p ( 1 - p )}{n} .$ ~95% sure hone ke liye ki gain real hai, require karo ki gain roughly $2 S E$ se zyada ho: $δ ≳ 2 \frac{p ( 1 - p )}{n} .$ Kaise padhe: zyada test data ( $n ↑$ ) → chhota needed margin → tum chhoti improvements pe bhi trust kar sakte ho.

Worked examples

Common mistakes

Recall Feynman: 12-saal ke bacche ko explain karo

Socho ek toy factory hai jisme ek conveyor belt hai. Har naya toy design (code), plastic ka naya batch (data), aur taiyaar toy (model) belt pe sawaar hota hai. Har checkpoint pe ek robot inspector check karta hai: Kya design safe hai? Kya plastic saaf hai? Kya toy actually dukaan wale se zyada achha kaam karta hai? Sirf woh toys jo har checkpoint pass karte hain, unhe box karke dukaan bheja jaata hai. Aur ek special sensor dukaan pe nazar rakhta hai — agar bacchon ko toy pasand aana band ho jaaye, toh woh ek ghanti bajaata hai jo automatically naya banana shuru kar deta hai. Belt + inspectors + ghanti sab milke ML CI/CD pipeline hain.

Active recall

Ek ML system mein teen cheezein kaunsi hain jo independently change hoti hain (normal software ke unlike)?

Code, data, aur model.

CT (Continuous Training) CI/CD ke beyond kya add karta hai?

Automatic retraining jo naye data ya performance drift se trigger hoti hai, model decay se ladne ke liye.

Promotion gate old aur new models ke liye same held-out test set kyun use karta hai?

Taaki score difference model skill reflect kare, evaluation data ke differences nahi.

Kisi bhi improvement pe promote karne ki bajaye margin

δ

kyun require karte hain?

Metrics noisy hoti hain; ek chhota gain random luck ho sakta hai.

δ

ensure karta hai ki improvement noise floor exceed kare.

Accuracy pe safe promotion margin ka rule-of-thumb formula batao.

δ ≳ 2 p (1 - p) / n

(≈ do standard errors).

Training/serving skew kya hai aur CI/CD isse kaise prevent karta hai?

Jab training aur production environments differ karte hain, different behaviour dete hain; prevent hota hai prod mein use hone wali same containerized pipeline ke andar training se.

Chhe ordered pipeline stages batao.

Validate, Train, Evaluate, Register, Deploy, Monitor.

Model registry kya hota hai?

Versioned store of trained model artifacts plus metadata (metrics, data version, lineage) jo promotion aur rollback ke liye use hoti hai.

Kaunsa metric commonly CT retrain trigger karta hai, aur rough threshold kya hai?

Population Stability Index (PSI); PSI > 0.2 significant drift signal karta hai.

"Sab tests pass" ML model ship karne ke liye kaafi kyun nahi hai?

Tests code bugs pakdte hain lekin statistical failure nahi; tumhe ek metric-based evaluation gate bhi chahiye.

Connections

Model Registry — jahan Register stage versioned artifacts store karta hai.
Data Versioning (DVC) — triangle ke "data" leg ko version karna.
Data & Concept Drift — jo CT ko trigger karta hai.
Model Monitoring — pipeline close karne wala feedback loop.
Docker & Containerization — training/serving skew se bachne ke liye packaging.
A-B Testing & Canary Deployment — Deploy stage pe safer rollout strategies.
Statistical Significance — promotion margin $δ$ ko justify karta hai.