Kubernetes for ML workloads — Hinglish notes

5.3.9 · AI-ML › MLOps & Deployment

ML ko Kubernetes ki ZAROORAT kyun hai?

Core idea KYA hai? Tum end state YAML mein describe karte ho. K8s ek reconciliation loop chalata hai: continuously desired state vs actual state compare karo, aur gap close karne ke liye actions lo.

$action = reconcile (desired, observed)$

Yeh exactly ek control system hai — jaise ek thermostat. Woh analogy hi poori design hai.

Object hierarchy (scratch se banao)

Yeh kaise nest hote hain: Deployment → ReplicaSet → Pods → Containers, sab Nodes par schedule hote hain, Service ke through expose hote hain.

Resource requests derive karna: scheduler ki arithmetic

Derivation — kya ek Pod fit hoga? Maano ek Node ka allocatable resource $C$ hai (jaise 16 GB RAM). Existing Pods $\sum_{i} r_{i}$ request karte hain. Ek naya Pod request $r$ ke saath fit hoga agar:

$r \leq C - \sum_{i} r_{i}$

Derivation — ek cluster mein kitne replicas fit honge? $N$ identical nodes ke saath, har node ki capacity $C$ hai, aur har replica $r$ request karta hai:

$max replicas = N ⌊ \frac{C}{r} ⌋$

Hum $⌊ \cdot ⌋$ per node use karte hain, globally nahi, kyunki ek single Pod do machines mein split nahi ho sakta — resources node-by-node pack hote hain.

Autoscaling: HPA formula derive karna

Derivation ek conservation idea se. Maano current replicas $R_{c u r}$ collectively metric value $M_{c u r}$ dikhate hain (jaise average CPU% ya requests/sec), aur tum chahte ho ki har replica target $M_{t a r}$ par ho. Maano load evenly shared hai aur replicas ke saath linearly scale karta hai. Total load $L = R_{c u r} \cdot M_{c u r}$ . Per-replica load $M_{t a r}$ ke equal banane ke liye:

$R_{c u r} \cdot M_{c u r} = R_{d es} \cdot M_{t a r} \Rightarrow R_{d es} = R_{c u r} \cdot \frac{M _{c u r}}{M _{t a r}}$

Kubernetes up round karta hai (ceil) taaki tum kabhi under-provision na karo:

ML-specific patterns

Worked example Minimal GPU inference Deployment (YAML skeleton)

apiVersion: apps/v1
kind: Deployment
spec:
  replicas: 3
  template:
    spec:
      containers:
      - name: model-server
        image: myrepo/model:v2
        resources:
          requests: { cpu: "2", memory: "8Gi", nvidia.com/gpu: 1 }
          limits:   { cpu: "4", memory: "8Gi", nvidia.com/gpu: 1 }

GPU/memory ke liye request == limit kyun? GPUs over-commit nahi ho sakte, aur memory over-commit se OOM-kills ka risk hai — unhe match karna predictable, "Guaranteed" QoS deta hai.

Common mistakes (Steel-manned)

Active recall

Recall Khud test karo (answers chhupao)

K8s kin do states ko reconcile karta hai? → desired vs actual/observed.
Scheduling kaunsa field drive karta hai, request ya limit? → request.
Desired replicas ka formula? → $⌈ R_{c u r} M_{c u r} / M_{t a r} ⌉$ .
Memory limit exceed karne par kya hota hai? → Pod OOM-killed ho jata hai.
Batch training ko completion tak kaunsa object run karta hai? → Job.
Service labels kyun use karta hai, IPs kyun nahi? → Pod IPs constantly change hoti hain.

Recall Feynman: 12-saal ke bachche ko explain karo

Socho tum ek pizza shop chalate ho jisme kaafi saare identical chefs (Pods) hain. Tum manager (Kubernetes) ko kehte ho: "hamesha 3 chefs kaam karte rahen, har ek ko ek oven chahiye." Agar ek chef quit kar de, manager turant naya hire kar leta hai. Agar badi rush aaye, manager aur chefs add kar deta hai (autoscaling). Customers ek number (Service) par phone karte hain aur manager unhe jo bhi chef free ho use route kar deta hai — unhe kabhi nahi pata kaunsa chef hai. Tum kabhi chefs ko haath se manage nahi karte; tum sirf manager ko goal batate ho, aur woh cheezein sahi rakhta hai.

Connections

Docker & Containerization — Pods containers run karte hain; K8s unhe orchestrate karta hai.
Model Serving (TF-Serving, TorchServe, Triton) — inference Pod ke andar kya hota hai.
Horizontal vs Vertical Scaling — HPA horizontal hai.
CI-CD for ML — rolling updates naye model versions deploy karte hain.
Distributed Training — multi-worker ranks ke liye StatefulSets + PVCs.
Cost Optimization in Cloud ML — bin-packing & request-sizing.
Control Systems - Feedback Loops — reconcile loop ek controller hai.

Kubernetes ko describe karne ke liye best mental model kya hai?

Ek cluster OS jo ek reconciliation loop chalata hai jo actual state ko declared desired state ke equal rakhta hai.

Scheduler Pod ko place karne ke liye kaunsa resource field use karta hai?

request (guaranteed reserved minimum), limit nahi.

Jab ek Pod memory limit aur CPU limit exceed kare toh kya hota hai?

Memory over-limit → Pod OOM-killed ho jata hai; CPU over-limit → Pod throttled ho jata hai (killed nahi).

HPA desired-replica formula do.

R_des = ceil(R_cur * M_cur / M_tar).

N nodes of capacity C par per-Pod request r ke saath max replicas kitne?

N * floor(C / r), kyunki Pods per-node pack hote hain aur split nahi ho sakte.

Service kaise jaanta hai ki kaunse Pods ko route karna hai?

Label selector se, dynamically — Pod IPs change hoti hain isliye yeh unhe kabhi hardcode nahi karta.

Batch training job completion tak kaunsa K8s object run karta hai?

Ek Job (ya scheduled runs ke liye CronJob); yeh tab tak restart karta hai jab tak ek success na mil jaye.

Distributed training ke liye StatefulSet kyun use karte hain?

Yeh stable network identity (worker-0, worker-1) aur stable persistent volumes deta hai jo har rank ko chahiye.

GPU inference Pods ke liye request=limit kyun set karte hain?

GPUs over-commit nahi ho sakte aur memory over-commit se OOM ka risk hai; unhe match karna predictable Guaranteed QoS deta hai.

HPA ko tiny metric changes par oscillate karne se kya rokta hai?

Ek tolerance dead-band (default ~10%) jiske neeche koi scaling nahi hoti.