Binning and discretization: Visual walkthrough

Before any symbol, three plain-word promises:

A data point is one number we measured. We will write the whole collection as $x$ .
A bin is one shelf — a stretch of the number line with a left wall and a right wall.
A bin edge is one wall. If you have $k$ shelves in a row, you need $k + 1$ walls.

Step 1 — Lay the numbers on a line

WHAT. Take the raw dataset and place every measurement as a dot on a horizontal number line.

WHY. Binning is fundamentally a geometric act: we are about to draw vertical walls between dots. You cannot decide where walls go until you can see where the dots are. Every strategy that follows is just a different rule for placing those walls.

PICTURE. Look at the figure. Each black dot is one house price from our running dataset $x = [100, 150, 200, 250, 300, 350, 900] (thousands $) .$ Here $x$ (read "the data") is just the whole list of dots. Notice the lonely red dot far to the right at $900$ — that is the outlier that will haunt Step 3.

Step 2 — Equal-width: cut the ruler into equal pieces

WHAT. Ignore where the dots sit. Just chop the range $R$ into $k$ pieces of identical length.

WHY. This is the simplest possible rule — the same rule you'd use to mark a ruler. It keeps the original scale honest: every shelf is " $267$ k wide," a sentence a human can report. We pick this first because it needs nothing but $min$ , $max$ and a count $k$ .

PICTURE. The red segment in the figure is one bin width $w$ . We stamp that same width along the line to plant the walls.

The membership rule — which shelf a dot lands on — is $f (x) = i if b_{i - 1} \leq x < b_{i} .$ Read it as: " $x$ belongs to shelf $i$ when it sits at-or-past the left wall $b_{i - 1}$ but before the right wall $b_{i}$ ." The last bin includes its right wall so $max$ isn't homeless.

Step 3 — Watch equal-width break on a skewed dataset

WHAT. Count how many dots actually land in each equal-width shelf.

WHY. A rule is only good if it balances the data. This step is the whole reason other strategies exist — so we must see the failure, not just be told about it.

PICTURE. In the figure the three shelves are shaded. The left shelf is a black traffic-jam of six dots; the middle shelf is a red empty box; the right shelf holds the single outlier.

Bin 1 $[100, 367)$ : ${100, 150, 200, 250, 300, 350}$ → 6 dots
Bin 2 $[367, 633)$ : ${}$ → 0 dots
Bin 3 $[633, 900]$ : ${900}$ → 1 dot

Step 4 — Equal-frequency: let the dots decide the walls

WHAT. Flip the logic. Instead of fixing wall spacing, fix how many dots each shelf must hold, then slide the walls until that count is met.

WHY. We want balance (Step 3 showed why). If every shelf must contain the same number of dots, no shelf can ever be empty. The walls now hug dense regions tightly and stretch across sparse gaps.

PICTURE. In the figure the dots are still black, but now the red walls sit between dots, chosen so two dots live on each side. The middle gap that killed equal-width is simply spanned by one wide shelf.

Resulting shelves — no empties:

$[100, 167)$ : ${100, 150}$ → 2
$[167, 283)$ : ${200, 250}$ → 2
$[283, 900]$ : ${300, 350, 900}$ → 3

The price: shelf widths are now $67$ , $116$ , $617$ — wildly uneven. Balanced counts, ugly labels. That is the trade the parent note names.

Step 5 — Custom edges: put walls where the world has walls

WHAT. Throw away both automatic rules and place walls by hand, at values that mean something in the real world.

WHY. Sometimes the data density is irrelevant — the cause lives at a fixed threshold (a poverty line, a legal age, a tax bracket). A wall placed there aligns the shelf with the mechanism that actually moves your target.

PICTURE. The figure shows the same income line, but the red walls are labelled words ("poverty line", "middle class") rather than computed numbers. The walls land on human meaning, not on dot counts.

$edges = [0, 15 k, 50 k, 100 k, \infty)$ Each wall is a known economic stress point. A model then learns a readable rule like "IF income $< 15$ k THEN high risk." This is also where feature interactions shine — a custom "low-income $\times$ high-debt" bucket can encode a rule no single continuous feature captures.

Step 6 — The degenerate cases nobody shows you

WHAT. Test the recipes on broken inputs so no surprise can ambush you later.

WHY. Contract rule: cover every case, including the ugly ones.

PICTURE. Three tiny panels: all-equal dots, a single dot, and a missing value.

Step 7 — The trade-off, drawn as two curves

WHAT. Show why the number of bins $k$ has a sweet spot.

WHY. Every earlier step chose $k = 3$ by hand. Here we see the force that decides $k$ .

PICTURE. One axis is $k$ . Two curves: bias (error from squashing dots into flat shelves) falls as $k$ grows; variance (jitteriness from tiny noisy shelves) rises as $k$ grows. Their sum is a U — the red dot marks its bottom.

Few bins → each shelf averages over a wide range → you lose detail → high bias.
Many bins → each shelf sees few, noisy dots → overfits noise → high variance.

This is the bias–variance tradeoff wearing a binning costume. A starting guess for $k$ : $k \approx n or k = ⌊ 1 + lo g_{2} (n) ⌋ (Sturges) .$ For $n = 100$ : Sturges gives $k = ⌊ 1 + lo g_{2} 100 ⌋ = ⌊ 1 + 6.64 ⌋ = 7$ . Always confirm with cross-validation — and bin inside each fold, never on the full dataset, or the test fold leaks its quantiles into training.

After binning, each shelf label is a category. To feed it to a linear model you usually pass it through one-hot encoding so the model treats shelves as unordered slots, not as numbers to add.

The one-picture summary

One number line, three treatments stacked: equal-width (evenly spaced walls, one empty shelf), equal-frequency (walls hugging dots, equal counts, uneven widths), and custom (walls on meaningful thresholds). Same dots, three philosophies of where to build the walls.

Recall Feynman retelling — say it back in plain words

Imagine dots on a ruler. Binning means dropping vertical walls to make shelves. The lazy way (equal-width) chops the ruler into equal chunks — great until one weird faraway dot stretches the ruler and leaves a shelf standing over empty air. The fair way (equal-frequency) slides the walls until each shelf holds the same number of dots — no empties ever, but the shelves come out lopsided in width. The wise way (custom) puts walls where the real world already has them — poverty lines, legal ages — so a shelf means something a human can name. Then the only real decision is how many shelves: too few and you blur out the story (bias), too many and you're just tracing noise (variance). The good number sits at the bottom of that U — and you find it by testing, binning fresh inside every fold so the test data never whispers its answers into training.

Recall Quick self-test

Why does equal-width leave an empty bin on skewed data? ::: Because it spaces walls by distance, not by data count — a wide empty gap between clusters gets its own shelf with zero dots. What does the quantile $Q (1/3)$ return? ::: The value with one third of the data below it — the cutoff that puts $n /3$ dots on its left. Equal-frequency guarantees equal ___ but sacrifices equal ___. ::: counts ; widths What breaks equal-width when all values are identical? ::: The range $R = max - min = 0$ , so width $w = 0$ and every wall collapses onto one point. Why bin inside each CV fold rather than once upfront? ::: Otherwise quantile edges are learned from test data too — that leakage inflates your score.