Worked examples — Handling imbalanced datasets (SMOTE, undersampling)

This page is the drill hall. The parent note gave you the machinery — undersampling loss, SMOTE interpolation. Here we run that machinery through every kind of input it can meet, so no exam or real dataset surprises you.

Before any numbers, one promise: every symbol here was earned in the parent. If you forgot one, here is the pocket dictionary.

Recall Pocket dictionary (tap to open)

• $N_{\text{maj}}$ ::: how many samples the majority (big) class has.
• $N_{\text{min}}$ ::: how many samples the minority (rare) class has.
• imbalance ratio ::: $N_{\text{maj}}:N_{\text{min}}$, e.g. 99:1 means 99 big-class samples per 1 rare one.
• $r$ ::: the majority:minority ratio you want after undersampling. $r=1$ means perfectly balanced.
• $N_{\text{maj}}^{\prime }$ ::: majority samples you keep after undersampling $=r\cdot N_{\text{min}}$.
• $\lambda$ (lambda) ::: a random dial between 0 and 1 that says how far along the line between two minority points a new synthetic point sits.
• ${\mathbf{x}}_i$ ::: one minority data point (a list of feature numbers, like [2.5, 0.8, 0.6]).

---

The scenario matrix

Every problem this topic throws is one of these cells. The worked examples below each carry a tag like (Cell C) so you can see the coverage is complete.

Cell	Scenario class	The trap it hides
A	Mild imbalance (ratio < 10:1)	Do we even need to resample?
B	Extreme imbalance (100:1, 1000:1)	Undersampling loss explodes
C	Undersampling with $r\ne 1$ (not full balance)	People confuse "keep" vs "throw away"
D	SMOTE single-point interpolation	Getting $\lambda$ direction right
E	Degenerate: minority has 1 sample	KNN needs neighbours — it breaks
F	Limiting case: $\lambda =0$ and $\lambda =1$	Do we get duplicates?
G	Real-world word problem (fraud/medical)	Choosing the right tool
H	Exam twist: combine SMOTE + undersample, or resample-then-CV leak	The subtle mistake

We now walk all eight cells. Where geometry lives (SMOTE, decision regions), a figure carries it.

---

Cell A — Do we even resample?

Forecast: Guess the ratio and whether it's "dangerous" before reading on.

1. Compute the ratio. $\frac{N_{\text{maj}}}{N_{\text{min}}}=\frac{7000}{3000}\approx 2.33$, i.e. about 2.33 : 1. Why this step? The ratio is the single number that decides urgency. The parent flagged "danger" at > 10:1.
2. Compare to the danger line. $2.33<10$, so this is mild. A well-tuned model plus precision/recall monitoring usually handles it — no synthetic data needed. Why this step? Resampling isn't free; it distorts the true distribution. You only pay that cost when the imbalance genuinely starves the model.

Verify: Sanity check the fractions add up: $7000+3000=10000$ ✓. Minority is $3000/10000=30\%$ of data — plenty to learn from.

---

Cell B — Extreme imbalance and the loss explosion

Forecast: Will you keep more or fewer than 1% of the normal readings?

1. Target keep count. $N_{\text{maj}}^{\prime }=r\cdot N_{\text{min}}=1\times 1000=1000$. Why this step? $r=1$ means the kept majority must equal the minority count.
2. Apply the loss formula from the parent, $\text{Loss}=1-\frac{r{N}_{\text{min}}}{N_{\text{maj}}}$: $\text{Loss}=1-\frac{1000}{999000}\approx 1-0.001001=0.998999\approx 99.9\%.$ Why this step? This quantifies the pain: you discard ~99.9% of normal readings — nearly every learned "normal" pattern vanishes.
3. Read the verdict. At 1000:1, pure undersampling is reckless. Prefer SMOTE (Cell D) or cost-sensitive learning.

Verify: Kept $+$ thrown $=$ all: kept fraction $=1000/999000\approx 0.001001$; loss $\approx 0.998999$; they sum to $1.0$ ✓.

---

Cell C — Undersampling with a target that is NOT full balance

Forecast: More or fewer than the 100 you'd keep at $r=1$?

1. Keep count. $N_{\text{maj}}^{\prime }=r\cdot N_{\text{min}}=3\times 100=300$ legit kept. Why this step? $r=3$ deliberately leaves 3 legit per fraud — closer to a realistic prior, which keeps the model from over-crying "fraud".
2. Thrown-away count. $N_{\text{maj}}-N_{\text{maj}}^{\prime }=9900-300=9600$ discarded. Why this step? This is the raw information sacrifice.
3. Loss fraction. $\text{Loss}=1-\frac{300}{9900}=1-0.0303=0.9697\approx 96.97\%$. Why this step? Even a "gentle" 3:1 still tosses ~97% of majority data at this severe imbalance — the lever barely moves the loss.

Verify: Kept 300 + thrown 9600 = 9900 = $N_{\text{maj}}$ ✓. And $r=3$ is bigger than $r=1$, so we keep 300 > 100 ✓ (more kept when we tolerate more imbalance).

---

Cell D — SMOTE single-point interpolation (the core move)

Forecast: Will the new size land between 2.5 and 2.7, and where exactly?

1. Direction vector. ${\mathbf{x}}_{\text{neighbor}}-{\mathbf{x}}_i=[2.7-2.5,0.82-0.8,0.58-0.6]=[0.2,0.02,-0.02]$. Why this step? This arrow (yellow in the figure) points from ${\mathbf{x}}_i$ toward its neighbour. Note the third component is negative — irregularity decreases toward the neighbour, which is fine.
2. Scale by $\lambda$. $0.4\times [0.2,0.02,-0.02]=[0.08,0.008,-0.008]$. Why this step? $\lambda =0.4$ means "walk 40% of the way along that arrow". Look at the pink dot on the figure sitting 40% along.
3. Add to ${\mathbf{x}}_i$. ${\mathbf{x}}_{\text{new}}=[2.5,0.8,0.6]+[0.08,0.008,-0.008]=[2.58,0.808,0.592]$. Why this step? The synthetic sample is a plausible malignant point — every coordinate lies between the two real ones.

Verify: Each coordinate must sit between the endpoints. $2.5\le 2.58\le 2.7$ ✓, $0.8\le 0.808\le 0.82$ ✓, and for the decreasing feature $0.58\le 0.592\le 0.6$ ✓. All in-between → valid.

---

Cell E — Degenerate: the minority has only ONE sample

Forecast: Does it interpolate, duplicate, or crash?

1. Count available neighbours. SMOTE searches within the minority class for neighbours. With 1 point, the only "neighbour" is itself — there are 0 other minority points. Why this step? Interpolation needs a second distinct point to draw a line to. There is none.
2. What the formula gives. ${\mathbf{x}}_{\text{new}}={\mathbf{x}}_i+\lambda ({\mathbf{x}}_i-{\mathbf{x}}_i)={\mathbf{x}}_i$ — a pure duplicate for any $\lambda$. Why this step? The direction vector is the zero vector, so no new location can be created. Real libraries instead raise an error ($k$ exceeds available samples).
3. The fix. Duplicate the point a few times first (random oversampling) or gather more real minority data; SMOTE cannot invent variety from a single point.

Verify: $\lambda ({\mathbf{x}}_i-{\mathbf{x}}_i)=\lambda \cdot 0=0$ for every $\lambda$, so ${\mathbf{x}}_{\text{new}}={\mathbf{x}}_i$ ✓ — mathematically no new information.

---

Cell F — Limiting values of $\lambda$

Forecast: Which one lands on the original, which on the neighbour?

1. $\lambda =0$. ${\mathbf{x}}_{\text{new}}={\mathbf{x}}_i+0\cdot (\dots )=[2.5,0.8,0.6]$ — exactly ${\mathbf{x}}_i$. Why this step? Zero steps along the arrow → you never left home.
2. $\lambda =1$. ${\mathbf{x}}_{\text{new}}={\mathbf{x}}_i+1\cdot ({\mathbf{x}}_{\text{neighbor}}-{\mathbf{x}}_i)={\mathbf{x}}_{\text{neighbor}}=[2.7,0.82,0.58]$. Why this step? A full step lands you on the neighbour.
3. Consequence. Both endpoints are duplicates of real points, not new information. That is why SMOTE draws $\lambda$ from the open-ish interval $(0,1)$ — the interesting samples live strictly in between.

Verify: At $\lambda =0$ output $={\mathbf{x}}_i$ ✓; at $\lambda =1$ output $={\mathbf{x}}_{\text{neighbor}}$ ✓. Endpoints reproduce originals, confirming the interpolation is a straight line segment.

---

Cell G — Real-world word problem: pick the tool

Forecast: Would you delete healthy scans or synthesise diseased ones?

1. Read the constraint. Minority (50 diseased) is tiny and irreplaceable; majority is plentiful. Why this step? The scarce resource decides the strategy. You never want to lose the precious 50.
2. Rule out pure undersampling. Balancing to $r=1$ by cutting healthy to 50 discards $1150-50=1100$ scans and, worse, leaves only 100 total — too few to train. Why this step? Undersampling throws away data; here that starves the model (Cell B logic).
3. Choose SMOTE (optionally + mild undersample). Synthesise diseased scans up toward the healthy count using scaled features (SMOTE uses Euclidean distance, so scale first!). Synthetic count to reach balance: $N_{\text{synthetic}}=N_{\text{maj}}-N_{\text{min}}=1150-50=1100.$ Why this step? We grow the precious class without deleting anything, and scaling stops the largest-range feature from dominating the distance.

Verify: After SMOTE the diseased class $=50+1100=1150$, matching the 1150 healthy → 1:1 balance ✓, and zero real scans lost ✓.

---

Cell H — Exam twist: the resample-before-split leak

Forecast: Is the high recall real or fake?

1. Trace the data flow. SMOTE created synthetic points by interpolating all minority samples — including ones that will later fall into the validation fold. Why this step? A synthetic training point can sit between a real point and a validation point, effectively memorising it.
2. Name the leak. This is data leakage: validation information bled into training via synthesis. The recall is inflated and won't hold on truly unseen data. Why this step? CV is supposed to estimate performance on unseen data; the leak breaks that guarantee.
3. The fix. Resample inside each fold: split first, then SMOTE only the training portion of each fold (use an imbalanced-learn Pipeline). Real minority points in the validation fold stay untouched. Why this step? Now no synthetic point ever "sees" validation data — the score is honest.

Verify (conceptual sanity): Leaked-CV recall $\ge$ correct-CV recall almost always, and the gap disappears on a held-out test set the pipeline never touched — the classic symptom of leakage.

---

Coverage recap

Recall Did we hit every cell?

A mild ratio ::: Cell A (churn 2.33:1, no resample) B extreme loss ::: Cell B (1000:1 → 99.9% loss) C custom $r$ ::: Cell C ($r=3$ keeps 300, loss 96.97%) D SMOTE interp ::: Cell D (${\mathbf{x}}_{\text{new}}=[2.58,0.808,0.592]$) E degenerate ::: Cell E (1 minority sample → no neighbours) F limits ::: Cell F ($\lambda =0,1$ give duplicates) G word problem ::: Cell G (grow the rare via SMOTE) H exam twist ::: Cell H (resample-before-split leak)

Every scenario the topic can produce now has a worked, verified answer. When a real dataset lands on your desk, find its cell above and reuse the recipe. For the tree-based models that are unusually robust to imbalance, see 4.1.2-Decision-trees-and-random-forests.