Handling imbalanced datasets (SMOTE, undersampling): Question bank

This page is a workout for your judgement, not your calculator. Each line below is a Question ::: Answer reveal. Read the question, say your answer out loud, THEN reveal. If your reasoning differs from the answer's reasoning — even if you got the right verdict — that gap is the misconception to fix.

Everything here builds on the parent topic. Where a term first appears we anchor it in one plain sentence so you never meet a symbol cold.

Before the questions, three symbols and one formula appear everywhere below — meet them here so you never hit them cold:

The next figure shows the other big idea these questions probe — why the learned boundary drifts into minority land when one class swamps the other.

True or false — justify

95% accuracy on a 95:5 imbalanced problem proves the model is good.

False — a model that blindly predicts the majority class scores exactly 95% and catches zero minority cases; accuracy alone is blind here, use recall/precision from 3.2.4-Precision-recall-and-F1-score.

SMOTE increases the total number of samples in the dataset.

True — it adds synthetic minority rows and deletes nothing, so the dataset grows; contrast with undersampling which shrinks it.

Random undersampling can throw away information you can never recover.

True — deleted majority rows are gone, including rare-but-real majority subgroups, so any pattern living only in those rows is permanently lost.

SMOTE creates brand-new information about the minority class.

False — it only interpolates between existing minority points, so it reshapes and thickens the region you already have but invents no genuinely new evidence.

Perfect 1:1 balance is always the right target ratio

r

False — the ratio

r

(majority:minority you aim for) is a knob; sometimes 2:1 or 4:1 matches the real-world base rate better and avoids over-correcting the decision boundary.

You should apply SMOTE to the entire dataset before splitting into train and test.

False — synthetic points would leak into the test set, so a synthetic near-copy of a train point could be "tested," inflating your score; always resample inside the training fold, see 2.1.8-Cross-validation-and-holdout-methods.

Cross-entropy loss treats a minority error and a majority error as equally important.

True per-sample — but because majority errors are far more numerous their sum dominates the gradient, so the model effectively optimizes for the majority.

Feature scaling is irrelevant to SMOTE.

False — SMOTE uses Euclidean distance to find neighbors, and unscaled features let large-range features dominate the distance, so scaling first (2.1.3-Feature-scaling-and-normalization) is essential.

Undersampling changes the model's estimate of how common the minority class is.

True — after balancing, the training base rate no longer matches reality, so predicted probabilities are skewed and may need recalibration before deployment.

Cost-sensitive learning and resampling are two names for the same thing.

False — resampling changes the data that the model sees, while 3.4.7-Cost-sensitive-learning changes the loss by weighting errors; you can use either, or both.

Spot the error

"I set n_samples equal to the imbalance ratio (majority count divided by minority count, here 99) in resample, so 99:1 keeps 99 majority rows."

Wrong — n_samples is the count of majority rows to keep, which is

r \cdot N_{min}

(with

r = 1

, that's

N_{min}

rows); the imbalance ratio is a different quantity and using it as a keep-count is a category error.

"I ran SMOTE, then dropped rows with missing values."

Wrong order — SMOTE needs a complete numeric feature space to interpolate; handle missing data first (2.1.10-Dealing-with-missing-data), otherwise the distances are undefined or misleading.

"SMOTE searches for the k nearest neighbors among all samples."

Wrong — it searches only among minority samples, so the synthetic point stays inside minority territory; using all classes would pull synthetics across the boundary into majority land.

"My synthetic sample used

λ = 1.5

x_{new} = x_{i} + λ (x_{neighbor} - x_{i})

to make it more extreme."

Wrong — the interpolation factor

λ

must lie in

[0, 1]

so the new point sits on the segment between two real minorities;

λ > 1

walks past the neighbor and may land on impossible feature values.

"I balanced first, evaluated on the balanced set, and reported 90% recall."

Wrong — the balanced set is artificial, so the score doesn't reflect real deployment where the minority is rare; evaluate on the original untouched distribution.

"I used accuracy to pick between undersampling and SMOTE."

Wrong metric — accuracy rewards majority-heavy predictions; compare using minority recall, precision, or F1 (3.2.4-Precision-recall-and-F1-score) instead.

"I applied SMOTE to my categorical one-hot columns and got fractional values like 0.4."

Wrong tool — plain SMOTE interpolates continuous features, producing meaningless in-between categories; use a categorical-aware variant (e.g. SMOTE-NC) for mixed data.

Why questions

Why does the decision boundary drift toward the minority region under imbalance?

Because the learner minimizes total error and there are so few minority points that ceding their territory costs almost nothing overall, so the boundary shrinks the minority's decision region (see the second figure above).

Why does SMOTE use interpolation rather than just copying minority rows many times?

Exact duplicates stack identical points, which lets a model memorize them and overfit; interpolation spreads new points along the minority's shape, widening the region the model must learn.

Why is Euclidean distance the default choice inside SMOTE?

It assumes locality — that points close together in feature space share class behavior — so a synthetic placed between neighbors is a plausible member of the same class; other metrics work but need tuning.

Why can undersampling actually help a decision-tree-based model on huge datasets?

With millions of majority rows, dropping most still leaves plenty to learn from, and the balanced set lets trees (4.1.2-Decision-trees-and-random-forests) split on minority-relevant features that overall accuracy would otherwise suppress.

Why does resampling belong inside the cross-validation loop, not before it?

If you resample first, synthetic or duplicated points appear on both sides of every fold split, leaking test information into training and giving over-optimistic scores.

Why might combining undersampling and SMOTE beat either alone?

Undersampling trims dense, redundant majority mass and SMOTE thickens the sparse minority, so together they balance the ratio without discarding as much information or inventing as many synthetics.

Edge cases

What does SMOTE do when the minority class has only ONE sample?

It fails — with a single point there are no minority neighbors to interpolate toward, so the k-nearest-neighbor step is undefined; you need at least

k + 1

minority samples.

What happens if a minority point is actually a labeling error (an outlier)?

SMOTE will happily interpolate around it, generating a cluster of synthetic points reinforcing the noise, which can carve a spurious minority region into the boundary.

What is the loss fraction if you undersample a 100:1 dataset to perfect 1:1 balance?

You keep

1 \cdot N_{min}

majority rows out of

N_{maj} = 100 \cdot N_{min}

, so

Loss = 1 - \frac{1 \cdot N _{min}}{100 \cdot N _{min}} = 1 - \frac{1}{100} = 99%

of the majority data is discarded.

What if the two classes are already nearly balanced (say 55:45)?

Resampling is largely unnecessary and may hurt — the mild skew rarely breaks a model, and injecting synthetics or deleting rows just adds noise or wastes data.

What happens to two minority clusters that are far apart in feature space?

If a randomly chosen point

x_{i}

and its neighbor

x_{neighbor}

happen to straddle the gap, interpolation can drop a synthetic

x_{new}

in the empty valley between clusters — a location no real fraud ever occupied.

λ = 0

, what is the synthetic point

x_{new}

It equals the original

x_{i}

exactly (a duplicate), since

x_{i} + 0 \cdot (x_{neighbor} - x_{i}) = x_{i}

; at

λ = 1

it lands exactly on the neighbor.

What if the minority region overlaps heavily with the majority region?

SMOTE synthetics land inside contested territory and blur the boundary further, often lowering precision; here cost-sensitive weighting (3.4.7-Cost-sensitive-learning) is usually safer than generating more overlap.

Recall Quick self-check

Cover the answers above. If you can justify "95% accuracy is worthless here" and "resample only inside the training fold" from scratch, you own the two deadliest traps. Accuracy on a 95:5 split when always predicting majority ::: exactly 95%, and zero minority caught. Where does resampling go relative to the train/test split ::: strictly inside the training portion, after splitting.