Handling missing values (deletion, imputation strategies): Exercises

This is a self-testing page. Read each problem, try it on paper, THEN open the collapsible solution. The problems climb a ladder: L1 Recognition → L2 Application → L3 Analysis → L4 Synthesis → L5 Mastery. Each level ends with a [!mistake] callout that steel-mans the trap most people fall into there.

Parent: Handling missing values. Related tools you may lean on: 2.1.01-Data-quality-issues-and-detection, 2.1.03-Outlier-detection-and-treatment, 2.2.01-Feature-scaling-and-normalization, 3.1.02-Training-validation-test-split, 4.3.01-Bias-variance-tradeoff, 5.2.03-Data-leakage-prevention.

Intuition What every symbol here means (read once)

NaN = "Not a Number" = the empty cell, the hole in the table.
A row = one record (one patient, one day, one house). A column / feature = one measured quantity (Age, Income...).
$x_{i}$ = the value in the $i$ -th slot of a list of numbers. The little subscript $i$ is just a counter that walks through the list: $x_{1}$ is the first value, $x_{2}$ the second, and so on up to $x_{n}$ (the last), where $n$ = how many values there are.
$\overset{x}{ˉ}$ (read "x-bar") = the mean = add up the observed numbers, divide by how many there are.
Median = the middle value when you sort the numbers from smallest to largest. If there is an odd count, it is the single middle one; if there is an even count, it is the average of the two middle ones. Unlike the mean, one giant outlier barely moves the median, because the median only cares about position in the sorted order, not how far out a value sits.
Variance $σ^{2}$ = the average of the squared distances of each number from the mean. Big variance = numbers spread far apart; small variance = numbers huddled near the mean. We square the distances so that being below the mean (a negative gap) does not cancel being above it (a positive gap).
MCAR / MAR / MNAR = the three reasons data can go missing.

Definition The three missingness mechanisms (stated once, up front)

Let a cell's value be either observed or missing. The mechanism is about what the chance of being missing depends on.

MCAR — Missing Completely At Random: the probability of being missing depends on nothing at all — not the observed columns, not the hidden value. Example: a sensor drops readings by pure chance. Effect: deletion loses samples but stays unbiased.
MAR — Missing At Random: the probability of being missing depends only on other observed columns, not on the hidden value itself. Example: younger people (observed Age) skip the income box more often, but among people of the same age, high and low earners skip equally. Effect: deletion biases; a model that uses the observed columns can recover the truth.
MNAR — Missing Not At Random: the probability of being missing depends on the missing value itself. Example: high earners hide their income because it is high. Effect: no standard formula corrects this.

One decision question ties them together: "Does the chance of a blank depend on any value?" No → MCAR. Yes, on an observed column → MAR. Yes, on the missing value → MNAR.

Definition Why we use the population variance (denominator

$n$ ) on this page There are two variance formulas and they differ only in the divisor. In both, $i$ is the counter that runs over every value in the list ( $i = 1, 2, \dots, n$ ), $x_{i}$ is the $i$ -th value, and $n$ is how many values there are: $σ^{2} = \frac{1}{n} \sum_{i = 1}^{n} (x_{i} - \overset{x}{ˉ})^{2} (population), s^{2} = \frac{1}{n - 1} \sum_{i = 1}^{n} (x_{i} - \overset{x}{ˉ})^{2} (sample) .$

The $n - 1$ version ("sample variance", Bessel's correction) is used when you have a sample and want an unbiased estimate of an unknown population's variance — dividing by $n - 1$ compensates for the fact that the sample mean $\overset{x}{ˉ}$ is itself estimated from the same data and hugs the points a little too closely.
The $n$ version ("population variance") is the plain average squared distance of the numbers you actually have.

We deliberately use the $n$ version throughout because our claim is about the spread of the concrete filled vector the model will see, not about estimating a hidden super-population. Crucially, the mean-imputation shrinkage we prove in L5.1 happens under either divisor — swapping $n$ for $n - 1$ scales every variance by the same constant $\frac{n}{n - 1}$ and cannot rescue the lost spread. So the choice is for clarity, not to hide anything.

Level 1 — Recognition

Can you name the mechanism and pick the safe reflex?

Exercise L1.1

A temperature sensor logs a reading every minute. Due to a hardware glitch it randomly drops exactly 3% of readings, and the glitch has nothing to do with the temperature, the time, or anything else. Name the missingness mechanism.

Recall Solution

MCAR — Missing Completely At Random. The definition: missingness has zero relationship to any variable, observed or unobserved. The glitch is a coin flip independent of everything. Because it is unrelated to the data, deleting those rows loses information (fewer samples) but introduces no bias — the rows that survive are a fair random sample of all rows.

Exercise L1.2

In a survey, people with very high salaries deliberately leave the Income box blank because they don't want to reveal it. The blank depends on the salary value that is itself missing. Name the mechanism, and say whether mean-imputation will fix the bias.

Recall Solution

MNAR — Missing Not At Random. The reason a value is missing depends on the unobserved value itself (the high salary). Mean imputation will not fix it. You only observe the lower and middle salaries, so their mean is below the true population mean. Filling the blanks with that too-low mean drags your estimate even further from the truth. MNAR needs domain knowledge or sensitivity analysis, not a standard formula.

Level 2 — Application

Plug into the formulas.

Exercise L2.1 — Mean imputation and its variance dent

Data (one feature): $[4, 8, NaN, 12, 16]$ . (a) Impute the NaN with the mean of the observed values. (b) Compute the population variance $σ^{2}$ after imputation.

Recall Solution

(a) Mean of observed ${4, 8, 12, 16}$ : $\overset{x}{ˉ} = \frac{4 + 8 + 12 + 16}{4} = \frac{40}{4} = 10.$ Filled series: $[4, 8, 10, 12, 16]$ .

(b) Population variance = average squared distance from the mean of the filled series. The filled mean is still $10$ (imputing at the mean never moves the mean). $σ^{2} = \frac{( 4 - 10 ) ^{2} + ( 8 - 10 ) ^{2} + ( 10 - 10 ) ^{2} + ( 12 - 10 ) ^{2} + ( 16 - 10 ) ^{2}}{5}$ $= \frac{36 + 4 + 0 + 4 + 36}{5} = \frac{80}{5} = 16.$ Notice the imputed point sits exactly on the mean, contributing $0$ to the spread — so it silently shrinks the variance. The figure below makes this visible.

Figure — Handling missing values (deletion, imputation strategies)

Exercise L2.2 — Forward fill (including the leading-gap edge case)

A stock's daily close: $[50, 52, NaN, NaN, 58]$ . (a) Apply forward fill. (b) Now suppose instead the series starts with a blank: $[NaN, 52, 54, NaN, 58]$ . What happens to the leading NaN under forward fill, and how do you fix it?

Recall Solution

(a) Forward fill carries the last observed value into each gap: $x_{t} = x_{t - 1} when x_{t} is NaN .$ The last value before the gap is $52$ . Both blanks inherit it: $[50, 52, 52, 52, 58] .$ It assumes prices change slowly day-to-day (temporal autocorrelation). If a real jump to $58$ happened gradually across the gap, forward fill underestimates the intermediate days.

(b) The leading-gap edge case. Forward fill needs a previous value to carry. But the very first slot has nothing before it — there is no $x_{0}$ . So the rule $x_{t} = x_{t - 1}$ is undefined at $t = 1$ , and the leading NaN stays NaN: $[NaN, 52, 54, 54, 58] (slot 4 filled from 54, but slot 1 unfilled) .$ Fixes, in order of preference:

Backward fill for the leading positions: use the next observed value, $x_{t} = x_{t + 1}$ , so the first slot inherits $52$ → $[52, 52, 54, 54, 58]$ . This is the standard companion to forward fill precisely for edges.
A global fallback (mean/median of the observed series) for any slot that neither a previous nor a next value can reach.
Drop the leading rows if the series must not begin with a fabricated value (e.g. a strict time index where invented starting prices would mislead). Moral: forward fill alone cannot fill a gap at the very start (nor can backward fill at the very end) — always pair them or set an explicit fallback for the boundaries.

Exercise L2.3 — Median vs mean under an outlier

Data: $[3, 4, 5, NaN, 100]$ . Compute both the mean-imputation value and the median-imputation value for the NaN.

Recall Solution

Mean of observed ${3, 4, 5, 100}$ : $\overset{x}{ˉ} = \frac{3 + 4 + 5 + 100}{4} = \frac{112}{4} = 28.$ Median of observed = middle of the sorted list ${3, 4, 5, 100}$ . There are four numbers (an even count), so the median is the average of the two middle ones (the 2nd and 3rd after sorting): $median = \frac{4 + 5}{2} = 4.5.$ The single outlier $100$ yanks the mean to $28$ but barely touches the median (it only sits at the far end of the sorted order and never becomes a "middle" value). That is why median is the safer imputation for skewed / outlier-heavy features (it minimises $\sum ∣ x_{i} - c ∣$ , which is robust, instead of $\sum (x_{i} - c)^{2}$ , which is dominated by the far point).

Level 3 — Analysis

Compare strategies and reason about consequences.

Exercise L3.1 — How much data does listwise deletion cost?

You have $8$ features, each independently missing $10%$ of the time (MCAR). Under listwise deletion (drop a row if any feature is NaN), what fraction of rows survives, and what fraction is lost?

Recall Solution

A row survives only if all 8 features are present. Each is present with probability $0.90$ , and they are independent, so we multiply: $P (survive) = 0.9 0^{8} \approx 0.4305.$ Fraction lost: $1 - 0.9 0^{8} \approx 0.5695 \approx 57%.$ Even with a modest $10%$ per column, listwise deletion throws away more than half the data. This is the "loss of power" cost of MCAR: no bias, but brutal shrinkage of your sample.

Exercise L3.2 — KNN imputation, worked distances

Impute Weight for patient A using $K = 2$ nearest neighbours and uniform weights.

Patient	Height	Age	Weight
A	160	40	NaN
B	162	41	60
C	158	38	55
D	175	55	82

Distance uses every feature except Weight: $d (A, i) = (H_{A} - H_{i})^{2} + (Age_{A} - Age_{i})^{2}$ .

Recall Solution

Step 1 — distances (Weight is excluded because it's what we're predicting; using it would be circular): $d (A, B) = (160 - 162)^{2} + (40 - 41)^{2} = 4 + 1 = 5 \approx 2.236.$ $d (A, C) = (160 - 158)^{2} + (40 - 38)^{2} = 4 + 4 = 8 \approx 2.828.$ $d (A, D) = (160 - 175)^{2} + (40 - 55)^{2} = 225 + 225 = 450 \approx 21.213.$ Step 2 — pick $K = 2$ nearest: the two smallest are B ( $2.236$ ) and C ( $2.828$ ). D is far away and is excluded. Step 3 — impute (uniform weights = plain average of neighbours' Weight): $Weight_{A} = \frac{60 + 55}{2} = 57.5 kg .$ This beats global mean imputation (which would be $\frac{60 + 55 + 82}{3} \approx 65.7$ ) because it ignores the dissimilar patient D and trusts the locally similar B and C — the local-smoothness assumption at work.

Worked example Why you must scale features first (the WHY behind the side-note)

Distances add up squared differences across columns, so a column with larger raw units silently dominates. Suppose Height were recorded in millimetres instead of centimetres, and Age in years:

A vs B in mm: Height gap $= (1600 - 1620) = - 20$ mm, Age gap $= - 1$ yr. $d = (- 20)^{2} + (- 1)^{2} = 400 + 1 = 401 \approx 20.02.$ Height's $400$ utterly swamps Age's $1$ — the Age column has effectively zero say in who the neighbours are, purely because of its unit. Contrast the cm version from the solution where Height and Age contributed comparable amounts. The fix — put every feature on a common footing first (e.g. standardise each to mean $0$ , spread $1$ ) so no column wins by unit alone. This is exactly the job of 2.2.01-Feature-scaling-and-normalization; run it before any distance-based imputer.

Level 4 — Synthesis

Combine mechanism + strategy + pipeline discipline.

Exercise L4.1 — Choose and justify

A dataset of loan applicants has Income missing for 35% of rows, and investigation shows younger applicants skip it far more often (missingness depends on the observed Age column). You must build a model. State the mechanism, reject two strategies with reasons, and pick one.

Recall Solution

Mechanism: missingness depends on an observed variable (Age) but not on the hidden income itself → MAR (see the up-front definition). Formally: among applicants of the same age, high- and low-income people skip equally; the skipping is explained entirely by the observed Age column. Reject listwise deletion: dropping $35%$ of rows that are disproportionately young skews the survivors toward older applicants → biased model, not just weaker. (Under MAR, deletion biases; under MCAR it would only weaken.) Reject plain mean imputation: it ignores Age entirely, flattening the true Income-varies-with-Age structure, shrinking variance, and breaking the Age–Income correlation. Pick model-based / conditional imputation (e.g. MICE, or KNN on Age): because missingness depends on observed Age, a model that predicts Income from Age can recover the true conditional distribution. That is exactly the situation MAR is designed for.

Exercise L4.2 — Where in the pipeline does imputation belong?

You split data into train/validation/test. You compute the imputation mean. From which subset must the mean be computed, and why? Then say what goes wrong if you compute it from the whole dataset.

Recall Solution

Compute the imputation statistic (mean, KNN neighbours, MICE models) using the training set only, then apply that frozen statistic to fill validation and test. Why: the validation/test sets simulate unseen future data. If the imputation mean is computed over all rows, information from test rows leaks into the training pipeline — this is data leakage (5.2.03-Data-leakage-prevention). Your reported accuracy becomes optimistic and won't hold in production. Concrete failure: a test row's true value silently influenced the fill values used during training and validation, so your model "peeked." The fix is to fit the imputer on the train fold and transform the others — align this with your 3.1.02-Training-validation-test-split discipline.

Level 5 — Mastery

Prove a claim from first principles and handle a degenerate edge case.

Exercise L5.1 — Prove mean imputation cannot increase variance

Let observed values be $x_{1}, \dots, x_{m}$ with mean $\overset{x}{ˉ}$ and observed variance $σ_{obs}^{2} = \frac{1}{m} \sum (x_{i} - \overset{x}{ˉ})^{2}$ (here $i$ counts over the $m$ observed values). Now append $r$ copies of $\overset{x}{ˉ}$ (the imputed points), giving $n = m + r$ values. Show the new variance $σ_{new}^{2} = \frac{m}{n} σ_{obs}^{2} \leq σ_{obs}^{2}$ .

Recall Solution

Step 1 — the mean is unchanged. Sum of all $n$ values $= \sum x_{i} + r \overset{x}{ˉ} = m \overset{x}{ˉ} + r \overset{x}{ˉ} = (m + r) \overset{x}{ˉ} = n \overset{x}{ˉ}$ . Divide by $n$ : mean $= \overset{x}{ˉ}$ . (This is why imputing at the mean "doesn't change the mean.") Step 2 — squared deviations of the new points are zero. Each appended value equals $\overset{x}{ˉ}$ , so $(\overset{x}{ˉ} - \overset{x}{ˉ})^{2} = 0$ . They contribute nothing to spread. Step 3 — total squared deviation is unchanged, but we divide by more. $σ_{new}^{2} = \frac{1}{n} [= m σ_{obs}^{2} i = 1 \sum m (x_{i} - \overset{x}{ˉ})^{2} + imputed r \cdot 0] = \frac{m σ _{obs}^{2}}{n} = \frac{m}{m + r} σ_{obs}^{2} .$ Step 4 — conclude. Since $0 \leq \frac{m}{m + r} \leq 1$ (with $r \geq 0$ ), we get $σ_{new}^{2} \leq σ_{obs}^{2}$ , equality only when $r = 0$ (nothing imputed). Mean imputation always shrinks (never grows) the variance — a proof, not a slogan. (The same shrink factor $\frac{m}{n}$ appears whether you use the population or sample divisor, so the effect is not an artefact of our choosing the $n$ version.)

Sanity check with L2.1: there $m = 4, r = 1, σ_{obs}^{2} = \frac{( 4 - 10 ) ^{2} + ( 8 - 10 ) ^{2} + ( 12 - 10 ) ^{2} + ( 16 - 10 ) ^{2}}{4} = \frac{80}{4} = 20$ , so $σ_{new}^{2} = \frac{4}{5} \cdot 20 = 16$ . ✓ Matches the $16$ we computed directly.

Exercise L5.2 — Degenerate KNN

For patient A you want to impute Weight, but every other patient in the dataset also has Weight = NaN. What does KNN imputation return, and what is the principled fallback?

Recall Solution

What KNN returns: nothing valid. KNN imputation fills a hole by averaging the target values of A's nearest neighbours: $x_{A j} = \frac{\sum _{i^{'} \in N_{K}} w _{A i^{'}} x _{i^{'} j}}{\sum _{i^{'} \in N_{K}} w _{A i^{'}}} .$ Every candidate neighbour has $x_{i^{'} j} = NaN$ , so there is no neighbour with an observed Weight to average. There are zero valid terms in the numerator — the expression collapses to $\frac{0}{0}$ , which is undefined. In practice a library either raises an error or silently returns NaN, i.e. it cannot impute this feature by KNN. The reason is structural: KNN borrows information that must actually exist somewhere, and here it exists nowhere.

The principled fallback (in order of preference):

If the entire column is empty (Weight NaN for everyone): the column carries no signal at all — there is nothing to learn or borrow from it. Drop the column. Keeping it only invites a fabricated fill that pretends to be data.
If only A's local neighbourhood happens to be empty (some rows elsewhere do have Weight): widen the search — increase $K$ or the radius until you catch neighbours that possess an observed Weight, or fall back to a global statistic (mean/median of the observed Weights) for that one row. You trade locality for the ability to fill at all.
Flag and investigate. A whole-column gap almost never happens by accident; it usually signals an upstream data-quality failure (a broken join, a field that was never collected). Log it and trace it back through 2.1.01-Data-quality-issues-and-detection before trusting any downstream model.

The moral: imputation can only redistribute existing information — it can never manufacture it. When the information is truly absent (empty column), the honest move is to remove the column, not to invent numbers.