Foundations — Train - validation - test splitting

Everything in the parent note is machinery built around that single idea. Below we unpack every symbol and word it uses, from absolute zero, in the order they build on each other. If the parent note wrote a symbol without explaining it, we explain it here.

0. What is "data" — the picture behind $D$, $X$, $y$, $N$

Before any splitting, we need to agree on what we are splitting.

Picture a spreadsheet. Each row is one example (one customer, one photo, one house). Each column is one measured number about that example (age, price, pixel brightness).

Why do we need these letters? Because the entire topic is about cutting rows of $D$ into groups. To talk about cutting, we first need a name for the thing being cut ($D$), the inputs ($X$), the answers ($y$), and the total count ($N$).

1. Model, parameters, hyperparameters — three words the parent leans on

The parent note says the training set fits parameters and the validation set tunes hyperparameters. These two words look alike but mean opposite things, so we pin them down.

Why the topic needs this split of words: the training set turns the front knobs, and the validation set helps you pick the back switches. Different jobs → different data. That is the whole reason a second hidden set exists.

2. Generalization and $P_{\text{unseen}}$ — the quantity we actually care about

The parent writes $P_{\text{unseen}}$: performance on unseen data.

So the entire splitting ritual is an attempt to estimate a number we cannot see ($P_{\text{unseen}}$) using data we can see, by pretending some of our own data is "the future."

• Generalization = doing well on new data, i.e. a high $P_{\text{unseen}}$.
• Memorizing / overfitting = doing well on practice, badly on new data. (See 3.2.1-Overfitting-underfitting.)

3. A set and mutually exclusive — what "split" literally means

The parent says the three subsets are mutually exclusive. Two symbols make this exact.

Why the topic needs this: if the same row sat in both the training bucket and the test bucket, the "exam" would contain a question the model already practised. Mutual exclusivity is the guarantee that the exam is honest.

In symbols the parent implies: $D=\text{Train}\cup \text{Val}\cup \text{Test},\text{Train}\cap \text{Val}=\varnothing ,\dots$

• $\cup$ (union) means "pour all buckets together" — you get back the whole pile.
• $\cap$ (intersection) means "rows shared by two buckets."
• $\varnothing$ (empty set) means "no rows at all."

So $\text{Train}\cap \text{Val}=\varnothing$ reads: train and validation share nothing.

4. Ratios and the $60:20:20$ notation

The parent writes splits like $70:15:15$. This is a ratio — a recipe for slicing.

So for the counts: $n_{\text{train}}=0.70\times N,n_{\text{val}}=0.15\times N,n_{\text{test}}=0.15\times N.$ Here $n_{\text{train}}$ just means "the number of rows in the training bucket," and $n_{\text{train}}+n_{\text{val}}+n_{\text{test}}=N$.

5. Proportion $p_c$ and the floor $\lfloor \rfloor$ — the stratification symbols

The parent's stratification math uses $p_c$ and those funny brackets $\lfloor \rfloor$.

Why floor appears: you cannot put $6.9$ images in a bucket — rows are whole things. So $n_c^{\text{train}}=\lfloor p_c\times n_{\text{train}}\rfloor$ says: "take this class's share of the training bucket, then round down to a whole count."

Why the topic needs $p_c$ at all: stratification means "keep each class's proportion the same in every bucket." Without a name for that proportion, we couldn't state the rule. (Contrast with the danger in 6.3.2-Data-leakage when proportions or statistics cross bucket boundaries.)

6. Shuffle, index, and the fit / transform pair

The last three symbols the parent assumes are procedural.

How these foundations feed the topic

Each box is one symbol group above. Follow the arrows and you have rebuilt the whole parent note from zero. Next steps that use this foundation: 2.1.13-Cross-validation (reuse hidden data cleverly) and 4.1.5-Early-stopping (use the validation bucket during training).

Equipment checklist

Cover the answer and say it out loud before revealing.