Model serving (REST APIs, FastAPI) — Hinglish notes

5.3.6 · AI-ML › MLOps & Deployment

WHY serve models at all?

WHY not just import the model in every app?

Model files bahut badi ho sakti hain (GBs) aur special hardware (GPU) chahiye hota hai → aap har jagah copy nahi chahte.
Aap ek source of truth chahte hain: ek jagah model update karo, sab ko benefit milega.
Aap language independence chahte hain: ek JavaScript frontend pickle.load se PyTorch model nahi load kar sakta, lekin woh ek HTTP request zaroor bhej sakta hai.

Toh hum model ko ek network boundary ke peeche rakhte hain aur ek standard protocol se baat karte hain.

WHAT is a REST API?

Model serving ke liye, key pieces:

Piece	Meaning	Example
Endpoint	URL path jahan client hit karta hai	`POST /predict`
Request body	Input features as JSON	`{"x1": 2.0, "x2": 5.1}`
Response	Prediction as JSON	`{"pred": 1, "prob": 0.87}`
Status code	Outcome signal	`200 OK`, `422 Unprocessable`, `500 Error`

Figure — Model serving (REST APIs, FastAPI)

HOW FastAPI works (derive the flow from scratch)

Hum server ko piece by piece build karte hain, har layer pe why poochte hue.

Step 1 — Model ko ONCE startup pe load karo.

from fastapi import FastAPI
import joblib
 
app = FastAPI()
model = joblib.load("model.pkl")   # loaded at import time → stays in RAM

Why this step? Model load karna expensive hai. Agar hum ise request handler ke andar load karte, toh har request file ko dobara padhti → latency explode ho jaati. Ek baar load karo, requests ke across reuse karo.

Step 2 — Pydantic se input schema define karo.

from pydantic import BaseModel
 
class Features(BaseModel):
    x1: float
    x2: float

Why this step? Bahar ki duniya untrusted JSON bhejti hai. Pydantic usse validate aur parse karta hai: agar ek client x1: "hello" bheje, toh FastAPI automatically 422 ke saath reject kar deta hai, aapke model ko garbage dekhne se pehle. Yeh aapka contract hai.

Step 3 — Endpoint define karo.

@app.post("/predict")
def predict(f: Features):
    x = [[f.x1, f.x2]]
    y = model.predict(x)[0]
    p = model.predict_proba(x)[0].max()
    return {"pred": int(y), "prob": float(p)}

Why this step? Decorator @app.post("/predict") URL + method ko function se bind karta hai. FastAPI type hint f: Features padhta hai, toh woh jaanta hai ki body ko us schema mein automatically parse karna hai. Hum ek dict return karte hain → FastAPI use JSON mein serialize karta hai.

Step 4 — Ise ASGI server se run karo.

uvicorn main:app --host 0.0.0.0 --port 8000

Why this step? FastAPI sirf app logic hai; Uvicorn asli web server hai jo ek socket pe listen karta hai aur HTTP bolata hai. FastAPI ASGI (Asynchronous Server Gateway Interface) pe built hai, jo ise blocking ke bina kai concurrent requests handle karne deta hai.

Worked examples

Worked example Example 3 — Batch endpoint for throughput

class Batch(BaseModel):
    rows: list[list[float]]
@app.post("/predict_batch")
def predict_batch(b: Batch):
    preds = model.predict(b.rows)
    return {"preds": preds.tolist()}

Why this step? 1000 rows pe ek baar model.predict call karna 1000 alag HTTP calls se kaafi sasta hai — vectorized inference + ek round-trip. Throughput $↑$ dramatically.

Common mistakes (Steel-man → Fix)

Active recall

Recall Quick self-test (hide and answer)

Model ko per request nahi, startup pe kyun load karte hain?
/predict ke liye kaun sa HTTP method aur kyun?
Pydantic aapke liye kya karta hai?
FastAPI aur Uvicorn mein kya fark hai?
Model change kiye bina throughput kaise badhate hain?

Recall Feynman: explain to a 12-year-old

Socho tum akele ek mushkil math puzzle solve kar sakte ho. Har koi tumhari desk pe bheedhne ki jagah, tum apne room ke bahar ek mailbox rakhte ho. Log apne numbers ke saath ek note daalte hain, tum solve karte ho, aur mailbox mein answer wapas rakh dete ho. Mailbox REST API hai. FastAPI woh rules ka set hai ki notes kaise likhe jaane chahiye, aur woh koi bhi note jo galat likha gaya ho wapas phenk deta hai. Uvicorn woh mail carrier hai jo actually notes tumhare door tak le jaata aur laata hai. Tum puzzle-solving book (model) ko apni desk pe din bhar khula rakhte ho taaki har baar uthani na pade.

Forecast-then-Verify

Recall Predict before you run

Tumhare paas 4 Uvicorn workers hain, har ek 50 ms mein ek request handle karta hai. Forecast karo max throughput. ... Verify: $Throughput = N / T = 4/0.050 = 80$ requests/sec. Agar tum inference ko aadha karke 25 ms karo, throughput double hokar 160 req/s ho jaata hai — confirm karta hai ki latency aur throughput inversely linked hain.

Flashcards

What is model serving?

Ek trained model ko network ke upar inference ke liye available karna taaki clients inputs bhej sakein aur predictions paa sakein bina model/compute ke khud ke paas rakhe.

Why load the model at startup instead of per request?

Model loading expensive hai (disk I/O + deserialization); ek baar load karke reuse karne se har request mein woh cost add nahi hoti.

Which HTTP method should /predict use and why?

POST — tum data processing ke liye submit kar rahe ho; POST JSON body allow karta hai, cache nahi hota, aur data URL mein expose nahi karta.

What does Pydantic provide in FastAPI?

Request bodies ki automatic parsing + validation ek typed schema ke against; kharaab input ko 422 ke saath reject karta hai model run hone se pehle.

What is the difference between FastAPI and Uvicorn?

FastAPI app framework hai (routing, validation, docs); Uvicorn ASGI web server hai jo actually socket pe listen karta hai aur HTTP bolata hai.

Formula for serving latency?

T_total = T_network + T_deserialize + T_preprocess + T_inference + T_serialize.

Formula for throughput?

Throughput ≈ N_workers / T_per_request.

Why is a batch endpoint faster than many single requests?

Kai rows pe ek vectorized model.predict + ek single HTTP round-trip, per-row inference aur per-request network overhead se behtar hai.

Why cast predictions to native Python types before returning?

JSON numpy types (np.int64/np.float32) serialize nahi kar sakta; int/float mein cast karo ya .tolist() use karo.

What HTTP status code should a malformed input return, and why not 500?

422 (client error) — galti client ke input ki hai, server ki nahi; 500 galat tarike se ek server bug imply karta hai.

Connections

Docker & Containerization — FastAPI app + model ko ek portable image mein package karo.
Model Monitoring & Drift — served endpoint wahin hai jahan tum drift ke liye inputs/outputs log karte ho.
Batching & Inference Optimization — ek single worker se aage throughput techniques.
gRPC vs REST — internal services ke liye REST ka ek lower-latency binary alternative.
CI-CD for ML — automatically nayi model versions ko same endpoint ke peeche test aur deploy karo.
Load Balancing & Autoscaling — throughput formula se $N_{workers}$ ko scale karna.