Kubernetes for ML workloads: Foundations — AI-ML deep dive

Before you can read the Kubernetes for ML workloads parent note without tripping, you must be able to see every symbol it throws at you. This page builds each one from nothing, in the order that each depends on the one before it.

0. The word "declarative" (everything rests on this)

Why does the topic need this idea first? Because every symbol below is a way of writing down a goal, not a command. If you read them as commands you will misunderstand all of them.

1. Container — the sealed lunchbox

The picture: a lunchbox. Whatever you pack at home is exactly what you get at school — the outside world can't change the contents.

Why the topic needs it: Kubernetes never runs "a program." It only ever runs containers. So the container is the true atom underneath everything. If this word is fuzzy, review Docker & Containerization first — that is where containers are built.

2. Pod — the lunchbox tray

The picture: a cafeteria tray holding one lunchbox (sometimes two that must always travel together, like a main dish + its sauce). K8s moves the tray, never a loose lunchbox.

Why the topic needs it: every formula on the parent page counts Pods, schedules Pods, restarts Pods. The Pod is the unit of counting. When the parent says "3 replicas," it means 3 Pods.

3. Node — the shelf the trays sit on

The picture: a shelf with limited space. You can only stack so many trays before it is full.

Why the topic needs it: the whole "will this Pod fit?" question is just "is there room on a shelf?" You cannot understand scheduling without a mental image of a bounded shelf.

4. request and limit — the two numbers on every tray

This is the pair beginners get wrong most often, so we build both slowly.

The picture: a hotel room.

The request is the room you book — nobody else can have it even while you're out sightseeing.
The limit is the room's physical walls — you cannot spill past them.

Why the topic needs it: the scheduler (next section) uses request to decide placement, and the kernel uses limit to decide punishment. Two different numbers doing two different jobs — mixing them up is the source of two of the parent's four "mistakes."

5. The symbols $\sum$ , $⌊ ⌋$ , and $⌈ ⌉$

The parent uses three math symbols that a smart 12-year-old may not have met. Here they are, each with a picture, before you ever see them in a formula.

So $\sum_{i} r_{i} = r_{1} + r_{2} + r_{3} + \dots$ — nothing more mysterious than adding a grocery bill.

6. The fit inequality — putting the symbols together

Now every piece of the parent's placement rule is defined. Read it as a full sentence:

In words: a new Pod's request $r$ (space I want to book) must be less than or equal to the Node's capacity $C$ (shelf size) minus the space already booked by everyone else $\sum_{i} r_{i}$ . That right side is simply "leftover room."

And the packing count: $max replicas = N ⌊ \frac{C}{r} ⌋$ "Per shelf, how many whole trays fit? ( $⌊ C / r ⌋$ ) Then multiply by number of shelves ( $N$ )." We floor per Node because a single Pod cannot be sawn in half across two machines.

7. The scaling symbols — $R$ , $M$ , and the reconciliation loop

The autoscaling section introduces four labelled quantities. Each is just a labelled number:

The picture: waiters in a restaurant. If 4 waiters are each running at 90% effort and you want them at a comfy 50%, you need more waiters to spread the work.

The ratio $M_{c u r} / M_{t a r}$ says "how far off target are we?" Multiply by current count, round up (ceiling — never under-staff). If $M_{c u r} = 90$ , $M_{t a r} = 50$ , $R_{c u r} = 4$ : $⌈ 4 \times 1.8 ⌉ = ⌈ 7.2 ⌉ = 8$ .

8. Deployment, Service, and the label idea

Two more objects, each a picture:

The prerequisite map

Read top to bottom: the declarative mindset feeds containers, containers feed Pods, Pods sit on Nodes, Nodes force the request/limit numbers, those feed the fit-and-pack math, which feeds autoscaling — and the reconciliation loop wires it all into the feedback picture that is Kubernetes.

Equipment checklist

Cover the right side; can you answer each before reading on?

Declarative vs imperative in one line ::: Declarative = describe the goal; imperative = list the steps. K8s is declarative.
What a container packages ::: The program plus every dependency, sealed so it runs identically anywhere.
Smallest schedulable unit in K8s ::: The Pod (one or more containers sharing network + storage).
What the letter $C$ means ::: A Node's allocatable capacity (free space on the shelf), e.g. 16 GB.
What the letter $N$ means ::: The number of Nodes (shelves) in the cluster.
request vs limit ::: request = guaranteed reserved amount (drives placement); limit = hard ceiling (drives throttle / OOM-kill).
Meaning of $\sum_{i} r_{i}$ ::: Add up the requests of all Pods already on the Node.
$⌊ 7.2 ⌋$ equals ::: 7 (floor rounds down).
$⌈ 7.2 ⌉$ equals ::: 8 (ceiling rounds up).
Why floor for packing, ceiling for scaling ::: Floor = whole Pods that fit (never overpromise space); ceiling = whole Pods needed (never underprovide capacity).
The fit inequality in words ::: New request $r$ ≤ capacity $C$ minus already-reserved $\sum_{i} r_{i}$ .
The four HPA letters ::: $R_{c u r}$ , $R_{d es}$ = current/desired replicas; $M_{c u r}$ , $M_{t a r}$ = current/target metric.
What OOM-killed means ::: A Pod exceeding its memory limit is terminated by the kernel (Out Of Memory).
Why a Service uses labels not IPs ::: Pod IPs change constantly as Pods die and respawn; labels are stable stickers to match against.
The reconciliation loop analogy ::: A thermostat: measure reality, compare to goal, nudge, repeat forever (Control Systems - Feedback Loops).

Connections

Kubernetes for ML workloads — the parent this page prepares you for.
Docker & Containerization — where the container (Section 1) is actually built.
Model Serving (TF-Serving, TorchServe, Triton) — the servers that sit behind a Service.
Horizontal vs Vertical Scaling — adding Pods (horizontal) is exactly what the HPA formula does.
Control Systems - Feedback Loops — the reconciliation-loop / thermostat math.
Cost Optimization in Cloud ML — why stranded capacity and over-requesting waste money.