Level 3 — ProductionMLOps & Deployment

MLOps & Deployment

⏱ 45 minutes60 marksprintable — key stays hidden on paper

Level 3 (Production): Derivations, Code-from-Memory, Explain-Out-Loud Time limit: 45 minutes Total marks: 60

Question 1 — Model Serving from Memory (12 marks)

(a) From memory, write a minimal FastAPI service that loads a scikit-learn model at startup and exposes a POST /predict endpoint accepting JSON {"features": [...]} and returning {"prediction": ...}. Include the Pydantic request model and startup loading. (8)

(b) Explain out loud (2–3 sentences) why you load the model once at module/startup scope rather than inside the request handler. (2)

(c) State the difference between using def vs async def for the endpoint when your prediction call is CPU-bound and synchronous. (2)

Question 2 — Latency & Cost Derivation (12 marks)

A real-time inference service must meet an SLA of p99 latency ≤ 150 ms. Single-request model inference takes 40 ms on CPU. You consider dynamic batching.

(a) If batching up to $B$ requests adds a fixed queue wait of up to $w$ ms and inference time grows as $t(B) = 40 + 8(B-1)$ ms, write the worst-case per-request latency as a function of $B$ and $w$ , assuming a request may wait the full window $w$ . (3)

(b) Given a max wait window $w = 30$ ms, find the largest integer batch size $B$ that still satisfies the 150 ms SLA. (4)

(c) At that $B$ , if each machine sustains throughput of $\frac{1000 \cdot B}{t(B)}$ requests/sec, and you must serve 8000 req/s, how many machines are needed? (3)

(d) Name two techniques (other than batching) to reduce inference latency. (2)

Question 3 — Data Drift Detection Derivation (12 marks)

You monitor a feature and use the Population Stability Index (PSI) to detect drift between a reference distribution and current traffic, binned into $k$ bins:

$\text{PSI} = \sum_{i=1}^{k} (a_i - e_i)\ln\!\left(\frac{a_i}{e_i}\right)$

where $e_i$ = expected (reference) proportion and $a_i$ = actual (current) proportion in bin $i$ .

(a) Compute PSI for 3 bins with expected $e = [0.4, 0.4, 0.2]$ and actual $a = [0.3, 0.5, 0.2]$ . Give the value to 4 decimal places. (6)

(b) Given the common thresholds (< 0.1 no shift, 0.1–0.25 moderate shift, > 0.25 major shift), state your conclusion. (2)

(c) Explain the difference between data drift and concept drift, and give one detection signal appropriate for concept drift that PSI cannot capture. (4)

Question 4 — Containerization & CI/CD (12 marks)

(a) From memory, write a multi-stage Dockerfile for a Python ML service that (i) installs dependencies in a build stage, (ii) copies only what's needed into a slim runtime stage, and (iii) runs the app with uvicorn. (6)

(b) Explain out loud why multi-stage builds reduce final image size and attack surface. (2)

(c) List, in order, four stages you'd put in a CI/CD pipeline for an ML model that differ from a standard software pipeline, and say what each guards against. (4)

Question 5 — A/B Testing for Models (7 marks)

You run an A/B test comparing a new model (B) vs current (A) on conversion rate. Group A: 2000 users, 200 conversions. Group B: 2000 users, 240 conversions.

(a) Compute the observed conversion rates and the absolute lift. (3)

(b) Compute the pooled standard error of the difference in proportions using $SE = \sqrt{\hat{p}(1-\hat{p})\left(\tfrac{1}{n_A}+\tfrac{1}{n_B}\right)}$ and the z-statistic. Is the result significant at the 5% level ( $z_{crit}=1.96$ )? (4)

Question 6 — Explain-Out-Loud: LLM Serving & Edge (5 marks)

(a) Explain how PagedAttention (as in vLLM) improves throughput for LLM serving versus naive contiguous KV-cache allocation. (3)

(b) State why exporting to ONNX helps edge deployment, and one accuracy risk introduced by INT8 quantization. (2)

Answer keyMark scheme & solutions

Question 1 (12 marks)

(a) (8 marks) — 2 for imports, 2 for Pydantic model, 2 for startup load, 2 for endpoint.

from fastapi import FastAPI
from pydantic import BaseModel
import joblib
 
app = FastAPI()
model = joblib.load("model.pkl")   # loaded once at import/startup
 
class PredictRequest(BaseModel):
    features: list[float]
 
@app.post("/predict")
def predict(req: PredictRequest):
    pred = model.predict([req.features])[0]
    return {"prediction": float(pred)}

Why: Pydantic validates/parses input; the decorator maps HTTP POST to the handler; [req.features] shapes to 2D for sklearn.

(b) (2 marks) Loading once avoids re-reading and deserializing the model from disk on every request (expensive, adds latency and I/O), and keeps a single warm object in memory reused across requests. (2)

(c) (2 marks) For a CPU-bound synchronous call, use def — FastAPI runs it in a threadpool so it doesn't block the event loop. Using async def would run the blocking call directly on the event loop, stalling concurrency. (2)

Question 2 (12 marks)

(a) (3 marks) Worst-case per-request latency: $L(B, w) = w + t(B) = w + 40 + 8(B-1)$

(b) (4 marks) With $w=30$ : $L = 30 + 40 + 8(B-1) = 70 + 8(B-1) \le 150$ $8(B-1) \le 80 \Rightarrow B-1 \le 10 \Rightarrow B \le 11$ Largest integer $B = 11$ . (Check: $L = 70 + 80 = 150 \le 150$ ✓)

(c) (3 marks) At $B=11$ : $t(11) = 40 + 8(10) = 120$ ms. Throughput/machine $= \frac{1000 \cdot 11}{120} = 91.67$ req/s. Machines $= \lceil 8000 / 91.67 \rceil = \lceil 87.27 \rceil = 88$ machines.

(d) (2 marks) Any two: quantization (INT8/FP16), model distillation/smaller model, ONNX/TensorRT graph optimization, caching frequent inputs, GPU acceleration, operator fusion. (1 each)

Question 3 (12 marks)

(a) (6 marks) Term by term:

Bin 1: $(0.3-0.4)\ln(0.3/0.4) = (-0.1)\ln(0.75) = (-0.1)(-0.287682) = 0.0287682$
Bin 2: $(0.5-0.4)\ln(0.5/0.4) = (0.1)\ln(1.25) = (0.1)(0.223144) = 0.0223144$
Bin 3: $(0.2-0.2)\ln(1) = 0$

$\text{PSI} = 0.0287682 + 0.0223144 + 0 = 0.0510826 \approx \mathbf{0.0511}$

(b) (2 marks) PSI $= 0.0511 < 0.1$ → no significant population shift; model retraining not triggered by this feature.

(c) (4 marks) Data drift = change in the input distribution $P(X)$ (covariate shift); PSI on features detects this. Concept drift = change in the relationship $P(Y|X)$ — the mapping from inputs to labels changes even if inputs look the same (2). PSI on features cannot see this since inputs may be stable; you need signals tied to labels/performance, e.g. monitoring rolling model accuracy / error rate against delayed ground-truth labels, or drift in the residual/prediction-vs-outcome distribution (2).

Question 4 (12 marks)

(a) (6 marks)

# ---- build stage ----
FROM python:3.11 AS builder
WORKDIR /app
COPY requirements.txt .
RUN pip install --prefix=/install --no-cache-dir -r requirements.txt
 
# ---- runtime stage ----
FROM python:3.11-slim
WORKDIR /app
COPY --from=builder /install /usr/local
COPY . .
EXPOSE 8000
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]

Marks: build stage (2), copy artifacts to slim runtime (2), uvicorn CMD + EXPOSE (2).

(b) (2 marks) The build stage's compilers, caches, and dev headers are discarded; only installed packages + app code land in the final slim image → smaller size, faster pulls, fewer packages = smaller attack surface.

(c) (4 marks) Any four, in order (0.5 each stage + 0.5 guard = 1 each):

Data validation — guards against schema/range drift in training data.
Model training + eval gate — guards against shipping a model below a metric threshold.
Model + artifact versioning/registry push — guards against unreproducible/untraceable deployments.
Model tests (bias, performance, invariance) + shadow/canary deploy — guards against silent regressions in production behavior.

Question 5 (7 marks)

(a) (3 marks)

$p_A = 200/2000 = 0.10$
$p_B = 240/2000 = 0.12$
Absolute lift $= 0.12 - 0.10 = 0.02$ (2 percentage points).

(b) (4 marks) Pooled proportion: $\hat{p} = \frac{200+240}{2000+2000} = \frac{440}{4000} = 0.11$ $SE = \sqrt{0.11 \cdot 0.89 \cdot (1/2000 + 1/2000)} = \sqrt{0.0979 \cdot 0.001} = \sqrt{0.0000979} = 0.009894$ $z = \frac{0.02}{0.009894} = 2.021$ Since $2.021 > 1.96$ , the difference is statistically significant at the 5% level. (Marginal — borderline.)

Question 6 (5 marks)

(a) (3 marks) Naive serving allocates a contiguous KV-cache block sized to max sequence length per request → heavy internal/external fragmentation and wasted memory, limiting batch size. PagedAttention splits the KV-cache into fixed-size blocks allocated on demand (like OS virtual-memory paging), so memory is used only as tokens are generated. This near-eliminates fragmentation, allows more concurrent sequences per GPU, enables sharing blocks (e.g., across beams/prefixes), and thus raises throughput.

(b) (2 marks) ONNX gives a portable, framework-agnostic graph that edge runtimes (ONNX Runtime, TensorRT) can optimize/execute across hardware without the training framework (1). INT8 quantization reduces numeric precision, which can introduce accuracy degradation from rounding/clipping — outlier activations may be poorly represented, hurting accuracy unless calibrated/QAT is used (1).

[
  {"claim":"Q2b largest batch B=11 satisfies 150ms SLA and B=12 does not",
   "code":"L=lambda B: 70+8*(B-1); result=(L(11)<=150 and L(12)>150)"},
  {"claim":"Q2c machines needed = 88",
   "code":"import math; t=40+8*10; thr=1000*11/t; result=math.ceil(8000/thr)==88"},
  {"claim":"Q3a PSI approx 0.0511",
   "code":"import sympy as sp; e=[sp.Rational(4,10),sp.Rational(4,10),sp.Rational(2,10)]; a=[sp.Rational(3,10),sp.Rational(5,10),sp.Rational(2,10)]; psi=sum((a[i]-e[i])*sp.log(a[i]/e[i]) for i in range(3)); result=abs(float(psi)-0.0511)<0.001"},
  {"claim":"Q5 z-statistic approx 2.021 and significant at 5%",
   "code":"import sympy as sp; p=sp.Rational(440,4000); SE=sp.sqrt(p*(1-p)*(sp.Rational(1,2000)+sp.Rational(1,2000))); z=float(sp.Rational(2,100)/SE); result=(abs(z-2.021)<0.01 and z>1.96)"}
]