Foundations — ML project lifecycle

This page assumes you have seen none of the notation in the parent note. We build every symbol from scratch, anchor it to a picture, and say why the lifecycle needs it. Read top to bottom — each block uses only symbols defined above it.

0. The one picture that holds everything

Before any symbol, look at the shape of the problem.

The black curve is the world: the true relationship between an input and its correct answer. The red curve (this section's spotlight) is your model: a guess at that relationship built from a handful of past dots (training data). Notice the model hugs the dots it saw but drifts away from the true curve where it saw nothing. Every number in this topic is a way of measuring one of those gaps.

1. What is an "input" and an "output"? — the symbols $x$ and $y$

Why the topic needs them: the entire Stage 2 (Data) is about pairing each $x$ with the right $y$ consistently. Without a symbol for "input" and "answer" we cannot talk about labelling, drift, or error at all.

2. What does "predict" mean? — the symbol $\hat{y}$ and the function

The picture: the red curve in §0 is the function. Feed it an $x$ (a spot on the horizontal axis), read off the red height — that height is $\hat{y}$.

Why the hat matters: the topic constantly compares $\hat{y}$ (what the model said) against $y$ (what was true). "Error" is literally how far $\hat{y}$ is from $y$. If you can't tell those two apart notationally, nothing in Stages 3–5 makes sense.

3. Random variables vs. their values — capital $X$ vs. little $x$

One tiny convention prevents a lot of confusion later, so we fix it before distributions.

Picture: $X$ is the dice still spinning in the air; $x$ is the 4 it landed on. $P(x)$ means "the chance the random variable $X$ takes the specific value $x$", often written in full as $P(X=x)$.

Why bother: every expectation below averages a random variable ($\mathbb{E}[\ell ]$) but is computed from realizations ($x_i,y_i$ in a dataset). Keeping capital = "the spinning thing" and little = "the landed value" makes the empirical formulas in §8 read correctly.

4. What is a "distribution"? — the symbol $P(x)$

Here is the single most important — and most skipped — idea in the whole parent note.

Look at the figure: two bell-shaped humps. The red hump (this section's spotlight) is live traffic today; the black hump is training data from last year. They no longer sit on top of each other. That horizontal shift is data drift: $P(x)$ changed.

5. The conditional $P(y\mid x)$ and the joint $P(x,y)$

Picture: stand at one spot $x$ on the horizontal axis. Look straight up. $P(y\mid x)$ describes how the true answers pile up above that single spot.

Why two different $P$'s? Because two different things can rot a model:

• $P(x)$ changes → the inputs moved (=data drift).
• $P(y\mid x)$ changes → the inputs look the same but the right answer changed (=concept drift: a photo of a "phone" meant a landline in 1990, a smartphone today).

You cannot separate these two failure modes without two separate symbols. That is why the parent's Stage 5 definition uses both.

6. The expectation symbol $\mathbb{E}[\cdot ]$ — "average, weighted by how likely"

The scoping formula — the project's expected-value equation, written out in full at the end of this section — uses $\mathbb{E}[\text{Value}]$. Let's earn $\mathbb{E}$ first.

Picture (figure): a see-saw. Outcomes sit at their payoff positions along a plank; each is a weight whose size is its probability. The red marker (this section's spotlight) is $\mathbb{E}[X]$, the balance point — heavier (more likely) outcomes pull the balance toward themselves.

The scoping formula (two outcomes, $i\in \{\text{success},\text{fail}\}$): $\mathbb{E}[\text{Value}]=p_{\text{success}}\cdot V_{\text{success}}+(1-p_{\text{success}})\cdot 0-C_{\text{build}}-C_{\text{maintain}}$

• $p_{\text{success}}$ = chance it works (feasibility); the fail probability is $1-p_{\text{success}}$.
• $V_{\text{success}}$ = payoff if it works. The failure payoff is taken as $0$, so its term vanishes.
• $C_{\text{build}},C_{\text{maintain}}$ = money spent regardless of success, so they are subtracted flat.

That collapses to the parent's $\mathbb{E}[\text{Value}]=p_{\text{success}}{V}_{\text{success}}-C_{\text{build}}-C_{\text{maintain}}$.

7. What "error" actually means — the loss function

Section 2 called error "how far $\hat{y}$ is from $y$." Let's make that one honest symbol before we start comparing error numbers, because "distance" means different things for numbers vs. categories.

8. Population vs. empirical — how we actually compute an error rate

The formula in §7 integrates over the true $P(x,y)$, which we never fully know. In practice we only have a finite pile of examples, so we replace the integral by a plain average over the pile.

9. Error numbers — TrainErr, DevErr, HLP, Bayes

The figure stacks the numbers on one axis so the two gaps are visible distances, with the red ${\epsilon }_{\text{Bayes}}$ (this section's spotlight) as the true floor beneath HLP. The lower bracket (HLP→TrainErr) is avoidable bias; the upper bracket (TrainErr→DevErr) is variance.

10. Percent and the $\lesssim$ / $\le$ symbols

Tiny but assumed:

• $\le$ means "less than or equal to" — a firm ordering.
• $\lesssim$ ("curvy less-than") means "less than in the usual, well-behaved case, but not guaranteed" — it flags the caveat above.
• A percent like $8\%$ means $8$ wrong guesses per $100$; as a fraction it's $0.08$.

Why the topic bothers with the curvy symbol: it is a warning label that overfitting can break the ordering. Notation itself is carrying the caveat.

11. How it all feeds the topic

Every arrow says "you need the left box before the right box makes sense." All roads converge on the ML project lifecycle — which then loops back through monitoring to Model monitoring and observability and Deployment strategies (shadow, canary, blue-green).

Equipment checklist

Cover the right side and answer out loud before revealing.