Visual walkthrough — Gini impurity

Everything here is one idea per step, one picture per step. Follow the red object in each figure — red always marks the thing this step is about.

Step 1 — What is a "node"? A bag of coloured balls

WHAT. A decision-tree node is just a group of items, each carrying a class label. We draw it as a bag of coloured balls. Here: 3 red balls and 2 black balls, 5 total.

WHY start here. Before any formula, we need a concrete object to count on. Every quantity below is a count or fraction of balls in this bag — nothing else.

PICTURE. Look at the bag. Red = class 1, black = class 2. The whole formula is a statement about this bag.

Step 2 — The game: draw, hide, guess by a second draw

WHAT. We play a game. Draw one ball and hide its colour (this is the true item). Then draw a second ball independently and copy its colour as our guess. We win if the guess matches the truth, lose if it differs.

WHY this game. Impurity should be small when the bag is one colour (easy, always win) and large when it's mixed (hard, often lose). This "guess by a random second draw" game has exactly that behaviour — so its losing rate is a perfect impurity meter.

PICTURE. Two draws from the same bag: the left draw is the hidden truth, the right draw is our guess. "Independent" means the second draw doesn't know the first.

Here "independent" means: each draw is a fresh random grab from the same proportions $p_i$. So the guess is red with probability $p_{\text{red}}$, black with probability $p_{\text{black}}$ — regardless of what the truth was.

Step 3 — Lay out all outcomes as a square

WHAT. Two draws, each red-or-black, give $2\times 2=4$ possible outcome pairs: (true, guess). We draw them as a unit square split into strips.

WHY a square. Two independent choices multiply. A square of side 1 lets area = probability: the truth splits the square left-to-right (widths $p_{\text{red}},p_{\text{black}}$), the guess splits it top-to-bottom (heights $p_{\text{red}},p_{\text{black}}$). Every little rectangle's area is its probability because area = width × height = $p_i\times p_j$.

PICTURE. Four tiles. The two diagonal tiles (red–red, black–black) are matches — we win. The two off-diagonal tiles are mismatches — we lose. The red-shaded region is the total losing area.

$\underset{\text{true is class }i}{\underbrace{p_i}}\times \underset{\text{guess is class }j}{\underbrace{p_j}}=\underset{\text{area of one tile}}{\underbrace{p_i{p}_j}}$

• $p_i$ — the width of the truth-strip for class $i$.
• $p_j$ — the height of the guess-strip for class $j$.
• $p_i{p}_j$ — the tile's area = probability of "truth $=i$ and guess $=j$".

Step 4 — Winning area = sum of squares $\sum p_i^2$

WHAT. Add up the areas of the match tiles (the diagonal). A match on class $i$ needs truth $=i$ and guess $=i$: area $p_i\cdot p_i=p_i^2$.

WHY square. The same index $i$ appears twice — one draw for truth, one for guess, both landing on $i$. Two identical independent probabilities multiplied gives $p_i\times p_i=p_i^2$. The square is literally "same colour twice."

PICTURE. Only the two diagonal (match) tiles are highlighted red now. Their areas are $p_{\text{red}}^2$ and $p_{\text{black}}^2$; together they form the total win area.

$P(\text{win})={\sum }_{i=1}^K{p}_i^2$

• ${\sum }_{i=1}^K$ — "add over every class $i$, from 1 to $K$" (here $K=2$ classes).
• $p_i^2$ — the diagonal tile area for class $i$ (a match).

For our bag: $P(\text{win})=0.6^2+0.4^2=0.36+0.16=0.52$.

Step 5 — Gini = losing area = $1-\sum p_i^2$

WHAT. The whole square has area $1$ (all probabilities together). Wins fill $\sum p_i^2$. Everything left over is losses — and losses are exactly what impurity means.

WHY "1 minus." Total area is 1, and win + lose covers everything with no overlap. So the losing area is simply $1$ minus the winning area — no extra work needed. This is the Gini impurity.

PICTURE. The full square (area 1) with the diagonal win-region carved out; the remaining red L-shape is $G$.

We can also read $G$ off the off-diagonal tiles directly — the two mismatch tiles have equal areas $p_{\text{red}}{p}_{\text{black}}$ each: $G=p_{\text{red}}{p}_{\text{black}}+p_{\text{black}}{p}_{\text{red}}=2(0.6)(0.4)=\mathrm{0.48.}✓$ Same number, two ways to see it — because "lose" and "not win" are the same region.

Step 6 — The "per-class" rewrite $\sum p_i(1-p_i)$

WHAT. Split the losing area a different way: group all mismatch tiles by their truth colour. All tiles where truth is class $i$ have total width $p_i$; the losing fraction of that column is everything except the matching height $p_i$, i.e. $(1-p_i)$.

WHY. Sometimes we want to know "how much does class $i$ contribute to the mistakes?" This rewrite gives each class its own error term. It's the same $G$, re-sliced by column instead of by diagonal.

PICTURE. One column (truth = red, width $p_{\text{red}}$) is highlighted; inside it, the losing part has height $(1-p_{\text{red}})$. Its area $p_{\text{red}}(1-p_{\text{red}})$ is red.

$G={\sum }_{i=1}^K\underset{\text{truth is }i}{\underbrace{p_i}}\underset{\text{guess is NOT }i}{\underbrace{(1-p_i)}}$

• $p_i$ — probability the true ball is class $i$ (column width).
• $(1-p_i)$ — probability the guess ball is any other class (the losing height in that column).

Check: $0.6(0.4)+0.4(0.6)=0.24+0.24=0.48$. $✓$ Identical to Step 5 — algebra confirms geometry: ${\sum }_i{p}_i(1-p_i)={\sum }_i{p}_i-{\sum }_i{p}_i^2=1-{\sum }_i{p}_i^2.$

Step 7 — Edge cases: pure, 50/50, and uniform

WHAT. Test the picture at its extremes so no scenario surprises you.

Case A — pure bag ($p=(1,0)$): the square is all one colour, the win-diagonal fills it entirely, losing area $=0$. So $G=0$. Nothing to misclassify.

Case B — fair 2-class mix ($p=(0.5,0.5)$): each strip is half-width. Diagonal wins cover $0.25+0.25=0.5$; losses $=0.5$. This is the maximum for 2 classes — the mix fights you hardest.

Case C — 3-class uniform ($p=(\frac{1}{3},\frac{1}{3},\frac{1}{3})$): now a $3\times 3$ square, three diagonal tiles of area $\frac{1}{9}$ each, wins $=\frac{1}{3}$, so $G=1-\frac{1}{3}=\frac{2}{3}$. This is $1-\frac{1}{K}$ with $K=3$.

WHY uniform is worst. Squaring rewards concentration — one big $p$ gives a big $p^2$, shrinking $G$. Spreading probability evenly makes every $p^2$ tiny, so $\sum p_i^2$ is as small as it can be and $G$ as large as it can be.

PICTURE. Three squares side by side: pure (no red), 50/50 (half red), 3-uniform (two-thirds red).

The one-picture summary

This single figure stacks the whole story: the bag → the outcome square → the win-diagonal → the losing L-shape that is $G$ → the two equivalent readings.

Active Recall

Connections

• Parent: Gini impurity — the formula this page derives visually.
• Decision Trees — where these bags become tree nodes.
• Entropy and Information Gain — the same square story with $-\log$ instead of squares.
• Classification vs Regression Trees — regression swaps this misclassification area for variance.
• Random Forests — many Gini-grown bags averaged.
• Gini Coefficient (Economics) — same name, different picture (Lorenz curve, not this square).

Concept Map