Foundations — K-fold cross-validation

Before you can read the parent note K-fold Cross-Validation, you need to earn every symbol it throws at you. This page builds each one from nothing — plain words first, then a picture, then why the topic needs it. Read top to bottom; each idea leans on the one above it.

0. The scene: a dataset is a stack of cards

Look at the figure: the tall stack on the left is $D$, and $N$ is the count written beside it.

Why the topic needs this: cross-validation is entirely about how we cut up this stack. If you don't picture $D$ as a physical stack of $N$ cards, none of the cutting will make sense.

1. A "sample" and why we care about unseen ones

Here is the whole reason this topic exists. When we build a model, we want to know:

"How well will it answer cards it has never studied?"

If you test a student using the exact questions they revised from, they'll look brilliant — but you've learned nothing about the real world. Same with models.

Why the topic needs this: the entire fold machinery below is a device to keep "seen" and "unseen" strictly separate — so the score stays honest.

2. Folds: cutting the stack into $K$ equal piles

The little subscript number (the tiny $1$ in $F_1$) is just a label — "pile number one". It carries no maths; it's a name tag.

Why "equal-sized"? If one pile were tiny and another huge, the grades wouldn't be comparable — a score from 2 cards is far shakier than from 40. Equal piles make every turn a fair fight.

3. Set language: $\cup$, $\setminus$, $\ne$, "disjoint"

The parent note writes things like $D=F_1\cup F_2\cup \dots \cup F_K$ and $T_i=D\setminus F_i$. These are just shorthand for combining and removing piles. Let's earn each squiggle.

Why the topic needs this: the split "train on all folds but one, validate on the one held out" is $T_i={\bigcup }_{j\ne i}{F}_j$ and $V_i=F_i$. That one line of set-notation is the whole algorithm.

4. Subscripts and the loop index $i$

Think of $i$ as which chapter of the exam you're currently marking. When $i=1$ you hide pile 1; when $i=2$ you hide pile 2; and so on until every pile has had its turn as the hidden test.

5. The model, its parameters $\theta$, and training $T$

You don't need to know what's inside $\theta$ — for this topic $\theta$ is just a box that came out of training and can now answer questions.

Why theta and not a plain letter? Convention: across machine learning, $\theta$ almost always means "the learned parameters". Using it here connects this note to everything else you'll read.

6. The metric $M$ and a "score"

Why the topic needs this: every round produces one score. The whole point of $K$ rounds is to get $K$ scores we can then combine.

7. Summation $\sum$ — the averaging machine

The final estimate uses $\sum$. Let's earn it before using it.

8. Spread: the standard error

Prerequisite map

Each box on the left must feel obvious before the box on its right makes sense. When the whole chain feels natural, you're ready for the parent note.

Equipment checklist

Cover the right-hand side and test yourself. Say the answer out loud before revealing.