Foundations — Handling imbalanced datasets (SMOTE, undersampling)

This page assumes you have never seen a single symbol from the parent note. We build each one from a picture before it is ever used. Read top to bottom; nothing appears before it is earned.

0. What is a "dataset" and a "class"?

Before any math, fix the picture. Imagine a table. Each row is one thing you observed — one transaction, one medical scan, one email. Each column is a measured number about that thing — the amount of money, the tumor size, the number of links in the email.

Look at the figure: the black rows are samples, the black columns are features, and the single red column on the right is the label. That red column is the whole reason this topic exists — everything is about predicting it.

1. The symbol $\mathbf{x}$ — a bold letter for a whole row

The parent note writes things like ${\mathbf{x}}_i$ and ${\mathbf{x}}_{\text{new}}$. What is that?

Why bold? Plain $x$ means a single number. Bold $\mathbf{x}$ warns you: "I am not one number, I am a whole list of numbers travelling together." The parent note needs this because the oversampling technique on that page (which we will meet later and whose name spells out to Synthetic Minority Oversampling Technique) moves entire rows around in space, not one number at a time.

The little subscript is just a name-tag:

• ${\mathbf{x}}_i$ = "the $i$-th sample" (sample number $i$).
• ${\mathbf{x}}_j$ = "some other sample, number $j$".
• ${\mathbf{x}}_{\text{neighbor}}$ = "the sample sitting nearby".
• ${\mathbf{x}}_{\text{new}}$ = "the brand-new synthetic sample we invent".

The double subscript $x_{i,f}$ means "sample $i$, feature $f$" — pick a row ($i$ from 1 up), then pick a column ($f$ from 1 up to $F$), and you get back one plain (non-bold) number.

2. Counting the classes — $N_{\text{maj}}$ and $N_{\text{min}}$

Now we count rows by label — that is, by the value of $y_i$.

In the figure, the tall black bar is $N_{\text{maj}}$ and the tiny red bar is $N_{\text{min}}$. That gap between bar heights is the imbalance. Every technique tries to make those two bars the same height.

Why a ratio and not a difference? Because a difference of "9800 rows" means nothing on its own — 9800 out of ten billion is balanced, but 9800 out of 9900 is disaster. Dividing cancels the overall size and leaves the pure skew. This is exactly the reasoning behind why raw accuracy misleads (see 3.2.4-Precision-recall-and-F1-score).

3. The target ratio $r$ — the dial you turn

The parent uses $r$ in $N_{\text{maj}}^{\prime }=r\cdot N_{\text{min}}$.

Why the prime mark? $N_{\text{maj}}^{\prime }$ (read "N-maj-prime") means "the majority count after we edited it". The prime ${}^{\prime }$ is a bookkeeping tick: unprimed = before, primed = after. So $N_{\text{maj}}^{\prime }=r\cdot N_{\text{min}}$ reads: "after editing, keep $r$ times the minority count of majority samples."

4. Fractions and percentages — the "Loss" symbol

The parent's undersampling "Loss" is: $\text{Loss}=1-\frac{r\cdot N_{\text{min}}}{N_{\text{maj}}}$

Read it piece by piece.

Why subtract from 1? Because "kept" and "lost" together must make the whole (100%). One is $1$ minus the other. This tiny move — flip a keep fraction into a loss fraction — is worth memorising; it appears everywhere.

5. Absolute value and distance — the symbol $d({\mathbf{x}}_i,{\mathbf{x}}_j)$

The oversampling technique needs to know which dots are "nearby". That needs a ruler. First, one tiny tool.

Look at the figure. The red line is the direct distance $d$. The two black legs are the differences in each feature. Squaring makes every difference positive (no cancelling), adding combines them, and the square root undoes the squaring to give back a real length in the same units as the axes.

So the general formula the parent writes, $d({\mathbf{x}}_i,{\mathbf{x}}_j)=\sqrt{{\sum }_{f=1}^F(x_{i,f}-x_{j,f}{)}^2},$ reads in words: "for every feature, take the gap, square it, add all the squares, then square-root the total."

6. Nearest neighbours and the letter $k$

Why "same class"? The oversampling technique only wants neighbours that are also minority ($y_i=1$), so the invented point stays inside minority territory and never drifts into majority land.

7. Making a point "between" — $\lambda$ and interpolation

The oversampling technique's heart is: ${\mathbf{x}}_{\text{new}}={\mathbf{x}}_i+\lambda ({\mathbf{x}}_{\text{neighbor}}-{\mathbf{x}}_i)$

Read the figure with the formula:

• $({\mathbf{x}}_{\text{neighbor}}-{\mathbf{x}}_i)$ is the arrow from the start dot to the neighbour dot (the black arrow).
• $\lambda$ shrinks that arrow: $\lambda =0$ gives no movement, $\lambda =1$ gives the whole arrow.
• Adding it to ${\mathbf{x}}_i$ slides you from the start dot part-way toward the neighbour.

So $\lambda =0.4$ lands the red synthetic dot 40% of the way along the segment. It is guaranteed to sit between two real minority dots — a plausible new example, never a wild outlier.

The name SMOTE

Now that every ingredient exists, we can safely read the parent's headline acronym:

Prerequisite map

Every foundation above feeds one of the two parent techniques, which feed the topic. Undersampling needs only counting and fractions; SMOTE additionally needs distance, neighbours, and interpolation.

Related vault topics

• 2.1.3-Feature-scaling-and-normalization — run before distance-based SMOTE.
• 3.2.4-Precision-recall-and-F1-score — the metrics that expose imbalance.
• 3.4.7-Cost-sensitive-learning — an alternative fix that reweights instead of resampling.
• 2.1.8-Cross-validation-and-holdout-methods — how to test fairly after resampling.
• 4.1.2-Decision-trees-and-random-forests — models often paired with these fixes.
• 2.1.10-Dealing-with-missing-data — a sibling preprocessing step.

Equipment checklist