Underfitting vs overfitting diagnosis: Question bank

This is a trap bank. Every item below targets a place where beginners confidently say the wrong thing about diagnosing underfitting vs overfitting. Read the prompt, answer out loud with a reason, then reveal. If your reason was "it just is", you failed the item even if the yes/no matched.

Before we start, three anchor pictures so every symbol below is earned.

Definition The three numbers you keep comparing

$J_{t r ain}$ = the average mistake your model makes on the examples it studied. Think "score on the practice test."
$J_{c v}$ = the average mistake on a held-out set it never studied (cross-validation / validation set). Think "score on the surprise exam." See 2.6.04-Cross-validation-techniques for how this set is carved out.
$ϵ^{*}$ = the best score anyone or anything could get on this data (human-level or Bayes error). The floor you cannot go below.
$m$ = the number of training examples you actually feed the model. Think "how many practice questions you studied." When we say "as $m$ grows" we mean you hand the model more and more training examples.

Two distances matter: avoidable bias $= J_{t r ain} - ϵ^{*}$ (how far practice score is from perfect) and variance $= J_{c v} - J_{t r ain}$ (how much worse the surprise exam is than practice). Underfitting = big avoidable bias. Overfitting = big variance.

Picture 1 — the two distances on a single number line. Below, one horizontal axis of "error". The floor $ϵ^{*}$ sits on the left; $J_{t r ain}$ and $J_{c v}$ are two marks to its right. The arrow from $ϵ^{*}$ to $J_{t r ain}$ is the avoidable bias; the arrow from $J_{t r ain}$ to $J_{c v}$ is the variance. Read the whole page against this picture.

Figure — Underfitting vs overfitting diagnosis

Picture 2 — the four fits as learning curves. Each little plot shows the two curves ( $J_{t r ain}$ rising as $m$ grows, $J_{c v}$ falling as $m$ grows). Underfitting: both flatten high, tiny gap. Overfitting: $J_{t r ain}$ hugs the floor, $J_{c v}$ stays high, big gap that closes only slowly. Just-right: both meet low near $ϵ^{*}$ . Every "learning curve" trap below refers here.

Picture 3 — the bias/variance quadrant. Two axes: bias (left→right) and variance (bottom→top). The four corners are the four diagnoses. When a trap says "low-bias/high-variance quadrant", find that corner here.

True or false — justify

A model with 99% training accuracy is a good model

False. High training accuracy alone tells you nothing about generalization; if

J_{c v}

is much worse it is textbook overfitting (memorizing, not learning).

Underfitting means the model has too much data

False. Underfitting is a capacity problem — the model is too simple to represent the pattern. More data does not fix it; the errors plateau high regardless of

m

(the number of training examples).

If training and validation error are both high, you are overfitting

False. Both high with a small gap is the signature of high bias (underfitting). Overfitting needs low

J_{t r ain}

but high

J_{c v}

A small generalization gap always means a good model

False. A model that underfits also has a small gap (both errors sit high together). Small gap is only "good" when both errors are also low.

Getting more training data can cure overfitting

True. More examples constrain a high-capacity model so it can no longer twist through noise, shrinking the variance gap — though the gap closes slowly. See 4.1.02-Data-augmentation-strategies when real data is scarce.

Getting more training data can cure underfitting

False. A model that lacks the capacity to fit the pattern will stay stuck no matter how many examples you feed it — the plateau does not move.

Reducing model capacity always reduces test error

False. It reduces variance but raises bias. If you were already underfitting, cutting capacity makes test error worse. It only helps when variance was the dominant problem.

Zero training error is the goal of training

False. Zero

J_{t r ain}

usually means you have memorized noise; the honest target is low

J_{c v}

, which often coexists with a small but nonzero

J_{t r ain}

near

ϵ^{*}

J_{c v} < J_{t r ain}

, something is broken

Usually true. Validation should be at least as hard as training; if it looks easier, suspect a data leak, a validation set that is too small/lucky, or a train/val distribution mismatch.

Spot the error

"My validation accuracy is 78%, so my model overfits."

Incomplete — 78% alone is meaningless. Overfitting is diagnosed by the gap to training accuracy, not by any single number. 78% with 79% training is fine; with 99% training it overfits.

"Training error is 15%, validation is 18%, gap is only 3%, so we should add regularization."

Wrong action. A tiny gap with high errors is underfitting; regularization makes a simple model even simpler and worse. You should increase capacity instead.

"Bias is the difference

J_{c v} - J_{t r ain}

Swapped definitions.

J_{c v} - J_{t r ain}

is variance (the gap). Avoidable bias is

J_{t r ain} - ϵ^{*}

, the distance from practice score to the floor.

"Human-level error is 0, so avoidable bias equals training error."

Dangerous assumption. Only true if the task truly has zero irreducible error. On noisy tasks

ϵ^{*} > 0

, so using

ϵ^{*} = 0

makes you chase bias that is physically unremovable.

"The learning curves cross, so my model is just right."

J_{t r ain}

and

J_{c v}

never truly cross —

J_{c v} \geq J_{t r ain}

in expectation. What "just right" looks like is the two curves converging close together at a low value (Picture 2, right panel), not intersecting.

"I added dropout and training accuracy dropped, so dropout hurt the model."

Misread. Dropout should lower training accuracy — that is regularization at work. Judge it by whether

J_{c v}

improved and the gap shrank, not by

J_{t r ain}

. See 2.6.05-Regularization-methods.

"My degree-15 polynomial has low training error, so it captured the true trend."

Low

J_{t r ain}

from a flexible model is exactly what memorizing noise looks like. The true trend is confirmed only when

J_{c v}

is also low. See 2.6.01-Bias-variance-tradeoff.

Why questions

Why does the train–validation gap approximate variance rather than bias?

Bias shows up as error even on the training set, so it cancels in the subtraction. What survives

J_{c v} - J_{t r ain}

is the part that comes from the model's sensitivity to which examples it saw — that is variance.

Why do both learning curves plateau high when underfitting?

The model's capacity is the ceiling. Extra data cannot raise a ceiling, so both training and validation errors flatten out at the same high level, gap and all (Picture 2, left panel).

Why does the overfitting gap shrink slowly as

m

(the number of training examples) grows rather than instantly?

Each new example removes only one degree of freedom the model was exploiting to memorize noise. A very flexible model has many, so you need a lot of data to pin them all down.

Why compare errors to

ϵ^{*}

instead of to zero?

Because some error is irreducible noise. Measuring against zero would blame the model for randomness it cannot remove; measuring against

ϵ^{*}

isolates the error you can actually fix.

Why can the same model be underfitting on one dataset and overfitting on another?

Fit is a relationship, not a property. Capacity relative to pattern complexity and data size decides it; change the data and the same architecture can flip diagnoses. See 3.2.01-Neural-network-capacity.

Why does increasing capacity trade bias for variance rather than removing error outright?

A more flexible model can represent the true pattern (lower bias) but also has more freedom to chase noise (higher variance). You move along the tradeoff curve; you do not escape it.

Why is looking only at test-set accuracy a bad diagnostic habit?

A single number cannot tell you which failure mode you have, so it cannot tell you what to do next. You need training vs validation and the

ϵ^{*}

floor to decode the cause.

Edge cases

What does the diagnosis say when

J_{t r ain} \approx ϵ^{*}

and

J_{c v} \approx ϵ^{*}

Both near the floor with a tiny gap — this is "just right." Further gains require better data or features, not architecture changes.

What if

J_{t r ain}

is below

ϵ^{*}

(training error under human-level)?

The model has out-memorized humans on the training set — a strong overfitting warning. Real skill is capped near

ϵ^{*}

, so beating it on training is suspicious, not impressive.

What does a huge avoidable bias and a huge variance at the same time mean?

You have both problems: too simple to fit and somehow unstable — often a broken pipeline, mismatched train/val distributions, or a badly tuned setup. Fix the larger component first, then re-diagnose.

With only 5 training examples and perfect training fit, why is the diagnosis unreliable?

Both

J_{t r ain}

and

J_{c v}

estimates are extremely noisy with tiny sets. A perfect fit on 5 points is expected even for correct models, so you cannot conclude overfitting yet — grow the set first.

What if validation error is lower than training error throughout the learning curve?

Suspect a leak or a validation set easier than training (e.g., regularization/dropout only active during training inflates

J_{t r ain}

). Investigate the split before trusting any diagnosis.

What happens to the diagnosis when the validation set is drawn from a different distribution than training?

The gap now mixes true variance with distribution shift, so a large gap no longer cleanly means overfitting. Separate a "train-dev" set (same distribution as training) to tell the two apart.

At the extreme of infinite training data, where do the curves land for a correctly-specified model?

J_{t r ain}

and

J_{c v}

both converge to

ϵ^{*}

with a vanishing gap — variance is driven to zero and only irreducible error remains.

Recall One-line self-check before you act

Say the phrase: "Both high, small gap ⇒ add capacity; low train, big gap ⇒ regularize or add data." If your numbers don't fit either phrase, re-measure — you probably have a pipeline bug, not a fit problem.