Intuition The one core ideaA trained model is a machine that turns numbers into an answer , and serving means putting a
mailbox in front of that machine so anyone on the network can drop in numbers and get the
answer back. Everything on this page is just the vocabulary you need to describe that mailbox
precisely — one word at a time, each earned before it is used.
The parent note (Model serving ) throws a lot of words at
you at once: function , inference , request , REST , HTTP method , JSON , endpoint ,
Pydantic , ASGI , latency , throughput . If any of those made you blink, this page is for you.
We define every single one from nothing, in the order that lets each rest on the previous.
A rule that takes an input and always gives back one output . Write it f ( x ) = y x y x y
The parent writes f ( x ) → y "produces" . Here:
x input , also called features — the numbers describing one thing you want a
prediction about (e.g. a house's size and age).y output , the prediction (e.g. the predicted price, or a class label like 1).
Why the topic needs this: the whole point of serving is to let other programs call this f f.x1, f.x2 for how individual features x 1 , x 2 x
x list of numbers , not one number
Real inputs have several features at once, so x = [ x 1 , x 2 , … ] x 1 = 2.0 x 2 = 5.1 2.0, 5.1 ). The double brackets [[2.0, 5.1]] in the
code mean "one row inside a table of rows" — the machine expects a grid , even for a single item.
The act of feeding a real input into an already-trained model to get its prediction . It is
the using phase, as opposed to training (the earlier phase where the rule f
Common mistake "Inference means the model is still learning."Why it feels right: the word sounds like "figuring out," which sounds like learning.
Why it's wrong: during inference the rule is frozen . It only computes y x
Why the topic needs this: serving is inference-as-a-service . Every request that arrives triggers
exactly one inference.
Before "REST" or "HTTP" mean anything, you need the cast of characters.
Definition The four words
Network := wires (or wifi) connecting computers so they can send each other messages .Server := the computer (and program) that holds the model and waits for messages .Client := any program that sends a message asking the server to do something (a phone
app, a website, another service).Request / Response := the message sent to the server / the message sent back .
Picture a mailbox on a door (the parent's Feynman image): the client drops a note (request), the
server reads it, solves it, and puts the answer back (response).
Why the topic needs this: "model serving" literally means being the server for a model . The
word serve = "answer requests." Every later term (endpoint, status code, latency) describes some
part of this send/answer loop.
The set of rules computers use to phrase requests and responses on the web . Think of it as
the grammar of the note in the mailbox: how to write the address, the message, and the reply.
Every HTTP request carries a method — a single verb saying what kind of action you want:
Method
Plain meaning
Mailbox picture
GET "Read me something"
Peeking at a poster already on the wall
POST "Here is data, please process it"
Dropping a filled-in form to be worked on
PUT "Replace this thing"
Swapping the poster for a new one
DELETE "Remove this thing"
Tearing the poster down
/predict uses POST, not GET
Asking for a prediction means handing over fresh data to be processed — that is exactly what
POST is for. GET is meant for reading a fixed thing and should carry no big body; stuffing
features into a GET URL means they get logged, cached, and size-limited. POST keeps the data in a
clean body where it belongs. (Parent's "Why POST" box.)
Why the topic needs this: the method is the first word the server reads; it decides which handler
runs. Pick the wrong verb and you fight the whole web's assumptions.
A plain-text way to write structured data as key–value pairs that every programming language
can read and write. It looks like a labelled box:
{ "x1" : 2.0 , "x2" : 5.1 } Read it as "the field named x1 holds 2.0, the field named x2 holds 5.1."
Intuition Why JSON and not just raw numbers?A bare 2.0, 5.1 is ambiguous — which is which? JSON attaches a name to each value, so the
server can't confuse x1 with x2. And because JSON is just text , a JavaScript webpage and a
Python server agree on it without either understanding the other's internals. This is the
"language independence" the parent mentions.
Why the topic needs this: both the request body ({"x1":..., "x2":...}) and the response
({"pred":1, "prob":0.87}) are JSON. It is the common tongue of the mailbox.
These two words appear in the parent's latency formula, so we define them now.
Definition Serialize & deserialize
Serialize := turn a program's in-memory object into flat text/bytes so it can travel
over the network. (Python dict → JSON string.)Deserialize := the reverse: read flat text back into a usable object . (JSON string →
Python object.)
Serial ize = spread into a serial line of characters to ship; de serialize
= de construct that line back into structure.
Why the topic needs this: a network can only carry bytes, never live Python objects. So every
request is deserialized on arrival and every response is serialized on the way out — two of the five
stages in the parent's latency budget.
Now that request/response/HTTP/JSON exist, "REST" is easy.
A style of designing web APIs where the client acts on resources (things named by a URL)
using standard HTTP methods, and each request is stateless .
The server keeps no memory of your previous requests . Every request must carry everything
the server needs to answer it. Like a mailbox that never remembers who wrote yesterday — each note
stands alone.
Intuition Why stateless helps servingIf the server remembers nothing, you can run ten identical copies of it and it doesn't matter
which copy answers your note — they're interchangeable. That is what makes scaling (adding
workers) possible. (Foreshadows Load Balancing & Autoscaling .)
Why the topic needs this: REST is the convention FastAPI implements. "Endpoint," "status code,"
and "JSON body" are all REST vocabulary.
A specific URL path plus method that the server answers , e.g. POST /predict. It's the
address on the mailbox for one particular service.
Intuition Why the client-vs-server split mattersIf bad input (x1: "cat") returns 500, it looks like your model crashed — a false alarm at
3am. Returning 422 correctly says "your note was scribbled wrong, fix it." The number is a
blame-assignment signal, and getting it right saves debugging hours. (Parent Example 2.)
The parent's tooling — each defined plainly.
Definition The tool stack
FastAPI := a Python library for writing REST APIs quickly . It maps your functions to
endpoints and handles JSON in/out. It is the rulebook for the mailbox .Pydantic := the part that checks incoming JSON matches the shape you declared . Declare
x1: float; if a client sends text, Pydantic rejects it with 422 before your model runs. Your
contract with the outside world.Uvicorn := the actual web server that owns the network socket and speaks HTTP. FastAPI
is only logic; Uvicorn is the mail carrier walking notes to and from the door.ASGI := the standard interface letting a Python app handle many requests at once without
each one blocking the others. It's why one server can juggle many mailbox notes concurrently.
Why the topic needs this: these four are the moving parts of the parent's HOW section. Knowing
which does what stops you from, say, expecting FastAPI to open a socket (that's Uvicorn's job).
How long ONE request takes, start to finish . The parent breaks it into a pipeline sum:
T total = T network + T deserialize + T preprocess + T inference + T serialize
How MANY requests you finish per second across the whole system:
Throughput ≈ T total per request N workers T N T N N / T
Intuition Latency vs throughput — not the same!Latency is one car's trip time ; throughput is cars-per-minute through the toll plaza . Add
more toll booths (workers) and throughput rises even though each car's trip is unchanged. Speed up
the road (lower latency) and both improve. Deepen this in
Batching & Inference Optimization and Load Balancing & Autoscaling .
Worked check (parent's numbers): 4 workers, 50 ms each →
Throughput = 4/0.050 = 80 4/0.025 = 160
Network with client and server
HTTP and methods GET POST
Serialize and Deserialize
REST style stateless resources
Endpoint and Status codes
Throughput equals workers over latency
FastAPI Pydantic Uvicorn ASGI
Hide the answers; if you can state each without peeking, you're ready for the parent note and for
Docker & Containerization , gRPC vs REST , CI-CD for ML , and Model Monitoring & Drift .
A model is fundamentally a … function f ( x ) → y
Inference means … running a frozen, already-trained model to get a prediction (not learning).
A client is … any program that sends a request asking the server to do something.
A server is … the computer/program holding the model and waiting to answer requests.
A request/response pair is … the message sent to the server and the answer sent back.
HTTP is … the grammar computers use to phrase web requests and responses.
GET vs POST: … GET reads a fixed thing (no big body); POST submits data to be processed.
JSON is … plain text of key–value pairs every language can read, e.g. {"x1":2.0}.
Serialize / deserialize means … turn an object into text to ship / turn text back into an object.
REST is … a style using resources + HTTP methods + stateless requests.
Stateless means … the server remembers nothing between requests; each carries all it needs.
An endpoint is … a specific URL path + method the server answers, like POST /predict.
Status codes 2xx / 4xx / 5xx mean … success / client's fault / server's fault.
FastAPI does … maps Python functions to REST endpoints and handles JSON in/out.
Pydantic does … validates incoming JSON against your declared schema, rejecting bad input.
Uvicorn does … owns the network socket and actually speaks HTTP (the web server).
ASGI is … the interface letting one Python app handle many requests concurrently.
Latency is … time for ONE request (a pipeline sum of its stages).
Throughput is … requests finished per second = N workers / T
