K-fold cross-validation: Worked examples — AI-ML deep dive

This page is the worked-example gym for K-fold Cross-Validation. The parent note built the machinery; here we run it against every kind of input a real problem can throw at you — clean data, messy data, tiny data, imbalanced data, and the traps that make the number lie to you.

Before we touch numbers, one promise: we will never write a symbol you haven't met. Let us re-anchor every symbol this page uses.

Definition Every symbol we reuse everywhere

$N$ = the total number of samples (rows of data) we have in hand. If a spreadsheet has $100$ rows, $N = 100$ .
$D$ = the whole dataset — the bag holding all $N$ samples.
$K$ = the number of folds, i.e. how many equal piles we chop $D$ into.
$F_{i}$ = the == $i$ -th fold==, the $i$ -th pile. The folds are disjoint and together rebuild the whole set: $D = F_{1} \cup F_{2} \cup \dots \cup F_{K}$ .
$n_{i}$ = the ==number of samples in fold $F_{i}$ ==. When $N$ divides evenly by $K$ , every $n_{i} = N / K$ ; otherwise some folds are one bigger (see Ex 2).
$r$ = the remainder of $N$ divided by $K$ , written $r = N mod K$ . It counts how many folds get one extra sample when the split isn't even. If $N = 103, K = 5$ then $r = 3$ .
$D ∖ F_{i}$ = " $D$ with fold $F_{i}$ removed" — the training set for iteration $i$ (everything except the exam pile).
$score_{i}$ = the performance number (accuracy, error, ...) the model earns when fold $F_{i}$ is the exam and the rest is study material.
$CV_{K}$ = the average of those $K$ scores. It is our single best guess at how the model behaves on data it has never seen.

$CV_{K} = \frac{1}{K} \sum_{i = 1}^{K} score_{i}$

Read the $\sum$ (capital Greek "sigma") as "add up": walk $i$ from $1$ to $K$ and total the scores.

Figure 1 — The K-fold layout ( $K = 5$ ). The full bar is the dataset $D$ of $N$ samples, chopped into $5$ equal folds $F_{1} \dots F_{5}$ . In iteration $3$ (shown), the red pile $F_{3}$ is the exam (validation) and every black pile together forms the training set $D ∖ F_{3}$ . Slide the red pile along and you get all $5$ iterations.

And the spread measure:

Figure 2 — What $s$ measures. Each red arrow is one fold's distance $∣ score_{i} - CV_{K} ∣$ from the dashed mean line. Long arrows on the right (Model Q) mean a big $s$ ; the arrows collapsing to nothing on the left (Model P) mean $s = 0$ .

The scenario matrix

Every K-fold problem lives in one of these cells. The examples below are chosen so each row gets touched at least once.

Cell	What makes it special	Covered by
A. Clean divisible case	$N$ divides evenly by $K$	Ex 1
B. Non-divisible $N$	$N / K$ is not a whole number → uneven folds	Ex 2
C. Degenerate $K = N$	Leave-One-Out (each fold = 1 sample)	Ex 3
D. Degenerate $K = 1$ / $K$ too big	invalid / trivial edges	Ex 3 (twist)
E. Zero-spread folds	all scores identical → $s = 0$	Ex 4
F. High-spread folds	scores swing wildly → large $s$	Ex 4
G. Imbalanced classes	stratification needed or a fold breaks	Ex 5
H. Regression metric	metric is an error (RMSE), lower = better	Ex 6
I. Real-world word problem	pick $K$ under a time budget	Ex 7
J. Exam-style twist	leakage / nested-CV trap	Ex 8

The worked examples

Worked example Example 1 — Cell A: the clean divisible case

Statement: $N = 100$ samples, $K = 5$ , metric = accuracy. The five fold-accuracies come out $0.85, 0.82, 0.88, 0.84, 0.86$ . Report $CV_{5}$ and $s$ .

Forecast: Guess the average and whether $s$ is small (folds agree) or large. Jot it down.

Check the split is legal. $100/5 = 20$ samples per fold, whole number. Why this step? If it weren't whole, folds would be uneven (that's Cell B) — always verify divisibility first.
Average the scores. $CV_{5} = \frac{0.85 + 0.82 + 0.88 + 0.84 + 0.86}{5} = \frac{4.25}{5} = 0.850$ . Why this step? The definition of $CV_{K}$ is literally the mean; the mean reduces the luck of any one split.
Deviations from the mean. $0, - 0.03, + 0.03, - 0.01, + 0.01$ . Why this step? $s$ measures scatter around the average, so we need each gap first.
Square, average, root. Squares: $0, 0.0009, 0.0009, 0.0001, 0.0001$ . Sum $= 0.0020$ ; divide by $5$ $= 0.0004$ ; $0.0004 = 0.02$ . Why this step? Squaring stops $+$ and $-$ deviations cancelling; the final root restores accuracy-units.

Answer: $85.0% \pm 2.0%$ .

Verify: All five scores lie inside $0.85 \pm 0.03$ , so a spread of $0.02$ is sensible (not bigger than the largest deviation $0.03$ , not zero). Units: accuracy in, accuracy out. ✓

Worked example Example 2 — Cell B:

$N$ not divisible by $K$ Statement: $N = 103$ samples, $K = 5$ . How big is each fold, and — since the folds are now different sizes — how should the averaging in $CV_{K}$ handle that?

Forecast: Guess the fold sizes, and guess whether the two smaller folds should count "less" in the average.

Divide. $103/5 = 20.6$ — not a whole number. Why this step? You cannot put $0.6$ of a sample anywhere; the ideal "equal folds" is impossible.
Find the remainder $r$ . $103 = 5 \times 20 + 3$ , so $r = 103 mod 5 = 3$ . The first $r = 3$ folds get one extra sample. Sizes: $n_{1}, \dots, n_{5} = 21, 21, 21, 20, 20$ . Why this step? This is the standard rule (scikit-learn's): spread the remainder one-per-fold so no fold is ever more than one sample bigger than another.
Decide how to average. Because the folds differ by at most one sample, the plain unweighted mean $CV_{K} = \frac{1}{K} \sum score_{i}$ is what scikit-learn's cross_val_score returns, and the tiny size gap ( $21$ vs $20$ ) barely changes the answer. If you wanted to be exact, a sample-weighted mean $CV_{K}^{w} = \frac{\sum _{i} n _{i} score _{i}}{\sum _{i} n _{i}}$ counts each fold in proportion to its size $n_{i}$ . Why this step? A fold that judged the model on $21$ points carries slightly more evidence than one that used $20$ ; weighting by $n_{i}$ honours that, though for near-equal folds both means agree to within rounding.
Sanity total. $n_{1} + \dots + n_{5} = 21 + 21 + 21 + 20 + 20 = 103$ . Why this step? Every sample must be validated exactly once; the sizes must re-sum to $N$ .

Answer: fold sizes $21, 21, 21, 20, 20$ ; use the plain mean (or the $n_{i}$ -weighted mean for exactness — they nearly coincide here).

Verify: $3$ folds of size $21$ and $2$ of size $20$ ; $3 \cdot 21 + 2 \cdot 20 = 63 + 40 = 103 = N$ . ✓ Largest fold minus smallest $= 21 - 20 = 1$ ✓ (never more than one apart). No sample is skipped or duplicated.

Worked example Example 3 — Cells C & D: the degenerate extremes (

$K = N$ and bad $K$ ) Statement: $N = 8$ tiny dataset. (a) What is $K = N$ called and how many models train? (b) Why are $K = 1$ and $K = 9$ both illegal?

Forecast: Guess the name of $K = N$ and the two failure reasons.

Set $K = N = 8$ . Each fold holds $8/8 = 1$ sample. This is Leave-One-Out CV (LOOCV). Why this step? When a fold shrinks to a single point, validation is "hide one, train on the rest" — the extreme of the K-fold idea.
Count models. You train $K = 8$ separate models. Why this step? K-fold always trains exactly $K$ models — one per held-out fold.
Try $K = 1$ . With $K = 1$ there is one fold $F_{1}$ = the whole dataset $D$ , so the training set $D ∖ F_{1} = \emptyset$ (the empty set). Why this step? A model trained on nothing is meaningless — $K \geq 2$ is required.
Try $K = 9 > N = 8$ . You can make at most $8$ non-empty folds from $8$ points; the $9$ th fold has $0$ samples. Why this step? $K$ can never exceed $N$ ; the hard ceiling is $K = N$ (LOOCV).

Answer: (a) LOOCV, $8$ models. (b) $K = 1$ empties the training set; $K = 9$ empties a fold. Legal range: $2 \leq K \leq N$ .

Verify: models trained $= K = 8$ ✓. Ceiling $K = N = 8$ ✓. Floor $K = 2$ ✓.

Worked example Example 4 — Cells E & F: zero spread vs. huge spread

Statement: Two models, both $K = 4$ .

Model P scores $0.90, 0.90, 0.90, 0.90$ .
Model Q scores $0.60, 0.99, 0.70, 0.95$ . Their averages differ ( $0.90$ vs $0.81$ ), but the real story is in the spread — which do you trust?

Forecast: Guess each $CV_{4}$ and each $s$ before computing.

Means. $CV_{4}^{P} = \frac{3.60}{4} = 0.900$ . $CV_{4}^{Q} = \frac{0.60 + 0.99 + 0.70 + 0.95}{4} = \frac{3.24}{4} = 0.810$ . Why this step? Compare like-for-like: the headline number first.
P's spread. Every deviation is $0$ , so $s_{P} = 0 = 0$ . Why this step? Identical scores mean zero disagreement — the degenerate zero-variance case (Cell E).
Q's spread. Deviations from $0.81$ : $- 0.21, + 0.18, - 0.11, + 0.14$ . Squares: $0.0441, 0.0324, 0.0121, 0.0196$ ; sum $= 0.1082$ ; $/4 = 0.02705$ ; $\approx 0.1645$ . Why this step? Wild scores → large $s$ → the estimate is fragile (Cell F).

Answer: P: $0.900 \pm 0.000$ (rock solid). Q: $0.810 \pm 0.164$ (untrustworthy — some folds nearly failed).

Verify: $s_{P} = 0$ matches identical scores ✓. $s_{Q} \approx 0.16$ is smaller than Q's biggest deviation $0.21$ and bigger than its smallest $0.11$ ✓ — a valid "typical distance". Figure 2 above draws exactly these two cases.

Worked example Example 5 — Cell G: imbalanced classes need stratification

Statement: $N = 100$ with $90$ of class A and $10$ of class B. You run plain random $K = 5$ . What can go wrong, and what does stratified K-fold guarantee?

Forecast: Guess how many class-B samples should land in each fold, and what happens if they don't.

Ideal per-fold class-B count. $10$ B-samples $\div 5$ folds $= 2$ per fold. Why this step? A representative fold must mirror the whole set's $90 : 10$ ratio.
The random-split danger. Nothing forces the spread — a random shuffle could dump all $10$ B-samples into folds $1$ and $2$ , leaving folds $3, 4, 5$ with zero B-samples in validation. Why this step? Accuracy on a B-free fold tells you nothing about B → the score is a lie for those folds.
Stratified fix. Split each class separately, then merge: each fold gets $90/5 = 18$ A-samples $+ 10/5 = 2$ B-samples $= 20$ total. Why this step? Splitting within each class forces every fold to hold the exact $9 : 1$ ratio.
Ratio check. $18 : 2 = 9 : 1$ , same as $90 : 10$ . Why this step? Confirms the fold is a faithful miniature of the dataset.

Answer: Each stratified fold = $18$ A + $2$ B, matching $9 : 1$ .

Verify: $5 \times 18 = 90$ A-samples and $5 \times 2 = 10$ B-samples, total $100 = N$ ✓, and every fold's ratio $18/2 = 9 = 90/10$ ✓.

Figure 3 — Left: a naive random split can starve some folds of the rare red class-B samples (fold 4 has none — its accuracy is meaningless). Right: stratified splitting places exactly $2$ red B-samples in every fold, so each pile mirrors the $9 : 1$ dataset.

Worked example Example 6 — Cell H: a regression metric (lower is better)

Statement: $K = 4$ regression, metric = RMSE (root-mean-square error, in the target's units, e.g. dollars). Fold RMSEs: $3.0, 5.0, 4.0, 4.0$ . Report $CV_{4}$ and $s$ , and say what "best" means.

Forecast: With RMSE, is a bigger or smaller $CV_{K}$ better? Guess the mean.

Direction of the metric. RMSE is an error: $0$ is perfect, big is bad — the opposite of accuracy. Why this step? When you later compare models, you now pick the lowest $CV_{K}$ , not the highest. Getting this backwards flips your conclusion.
Mean. $CV_{4} = \frac{3.0 + 5.0 + 4.0 + 4.0}{4} = \frac{16.0}{4} = 4.0$ . Why this step? Same averaging rule as accuracy — the estimator doesn't care what the metric means.
Spread. Deviations: $- 1, + 1, 0, 0$ . Squares: $1, 1, 0, 0$ ; sum $2$ ; $/4 = 0.5$ ; $0.5 \approx 0.7071$ . Why this step? $s$ formula is identical; only interpretation of the mean changes.

Answer: $CV_{4} = 4.0 \pm 0.71$ (dollars). Lower would be better.

Verify: Mean $4.0$ sits between the min $3.0$ and max $5.0$ ✓. $s \approx 0.71 <$ the largest deviation $1.0$ ✓. Units: RMSE in dollars → $s$ in dollars ✓.

Worked example Example 7 — Cell I: real-world word problem (time budget)

Statement: You have $N = 1000$ samples and a deep model that takes 6 minutes to train once. Your total compute budget is 1 hour. Between $K = 3$ , $K = 5$ , $K = 10$ , which fit the budget, and which do you pick?

Forecast: Guess which $K$ values fit under 60 minutes.

Cost model. K-fold trains $K$ models, so time $= K \times 6$ minutes. Why this step? Each fold means one full re-train from scratch.
Compute each. $K = 3 \to 18$ min. $K = 5 \to 30$ min. $K = 10 \to 60$ min. Why this step? Compare directly against the $60$ -minute ceiling.
Filter by budget. $18 \leq 60$ ✓, $30 \leq 60$ ✓, $60 \leq 60$ ✓ — all three fit, but $K = 10$ uses the entire hour with no margin. Why this step? You want headroom for setup/failures.
Pick on bias-variance. $K = 5$ trains each model on $\frac{4}{5} \cdot 1000 = 800$ samples (close to the final $1000$ , so low bias) and costs only $30$ min. Why this step? $K = 5$ is the standard sweet spot: near- $K = 10$ quality at half the cost and comfortable margin.

Answer: All fit; choose $K = 5$ ( $30$ min, $800$ -sample training folds).

Verify: $5 \times 6 = 30 \leq 60$ ✓. Training-fold size $\frac{4}{5} \cdot 1000 = 800$ ✓. $K = 10$ cost $= 60 =$ budget exactly ✓ (no slack).

Worked example Example 8 — Cell J: exam-style twist (the leakage trap)

Statement: A student normalizes the whole dataset and then runs $5$ -fold CV, reporting $95%$ . A second student normalizes inside each fold (statistics learned from the training folds only) and reports $91%$ . Which number is honest, and by how much was the first inflated?

First, two quick definitions so the trap is precise:

The global mean $μ$ is the average of a feature computed over all $N$ samples: $μ = \frac{1}{N} \sum_{j = 1}^{N} x_{j}$ , where $x_{j}$ is that feature's value for sample $j$ .
The global std (standard deviation) $σ$ measures the typical spread of that feature around $μ$ , again over all $N$ samples: $σ = \frac{1}{N} \sum_{j = 1}^{N} (x_{j} - μ)^{2}$ .

Normalizing means replacing each value by $(x_{j} - μ) / σ$ . The danger is which samples go into $μ$ and $σ$ .

Forecast: Guess which student is right and the size of the gap.

Spot the leak. Student 1's $μ$ and $σ$ are computed over all $N$ samples — including the validation folds. So when the model "studies" the training folds, statistics from the exam folds have already bled in. Why this step? Cross-validation is only valid if validation data is untouched during any training-side computation.
Direction of the bias. Leakage makes the validation fold look easier (it was used to set the scaling), so the reported score is optimistically inflated. Therefore Student 1's $95%$ is too high; Student 2's $91%$ is the honest estimate. Why this step? The inflated number over-promises on truly unseen data — the whole point of CV is to prevent that.
Quantify the inflation. Gap $= 0.95 - 0.91 = 0.04 = 4$ percentage points. Why this step? Naming the size shows leakage is not cosmetic — it can hide $4$ points of real overfitting.

Answer: Student 2's $91%$ is honest; Student 1's number is inflated by $4$ percentage points because the global $μ, σ$ leaked validation information.

Verify: $0.95 - 0.91 = 0.04$ ✓; and the leak direction is optimistic, so the leaked score $\geq$ the clean score ( $0.95 \geq 0.91$ ) ✓. Correct fix: compute $μ, σ$ from each iteration's training set $D ∖ F_{i}$ only, then apply to $F_{i}$ .

Recall Quick self-test

Fold sizes for $N = 103$ , $K = 5$ ? ::: $21, 21, 21, 20, 20$ (the first $r = 3$ folds get the extra sample). How should CV average uneven folds exactly? ::: Sample-weighted mean $\frac{\sum n _{i} score _{i}}{\sum n _{i}}$ ; the plain mean is fine when folds differ by at most one sample. Legal range of $K$ for $N = 8$ ? ::: $2 \leq K \leq 8$ ; $K = 8$ is LOOCV. Two models, same-ish story, one has $s = 0$ and one $s = 0.16$ — trust which? ::: The $s = 0$ model; its folds agree. For RMSE, is bigger $CV_{K}$ better or worse? ::: Worse — RMSE is an error, lower is better. Whole-dataset normalization (global $μ, σ$ ) before splitting causes what? ::: Data leakage → optimistically inflated scores. Does our $s$ divide by $K$ or by $K$ then $K$ again? ::: By $K$ inside the root — it's the population standard deviation, not the standard error of the mean.

Go deeper on the mechanics in the parent: K-fold Cross-Validation · or read it in Hinglish.