Gini impurity: Worked examples — AI-ML deep dive

Read the Hinglish version if that's easier.

The scenario matrix

Before working examples, let's list every distinct situation Gini can face. Each row is a "cell". Our examples below will each be tagged with the cell it fills.

Cell	Situation	What makes it special
C1	Perfectly pure node ( $p = (1, 0)$ )	$G$ hits its floor: exactly $0$
C2	Balanced 2-class ( $p = (0.5, 0.5)$ )	$G$ hits its 2-class ceiling: $0.5$
C3	Skewed 2-class ( $p = (0.9, 0.1)$ )	Between floor and ceiling — the common case
C4	Uniform $K$ -class ( $p = (1/ K, \dots)$ )	$G$ hits its ceiling $1 - 1/ K$ for that $K$
C5	Skewed multi-class	Concentration lowers $G$ below the ceiling
C6	Degenerate: one sample, or empty child	Edge input — does the formula even work?
C7	Split decision (compare two splits)	Weighted Gini + gain choose the winner
C8	Real-world word problem	Translate reality → proportions → $G$
C9	Exam twist: recover $p$ from a given $G$	Run the formula backwards

Recall Why enumerate cells at all?

Because a "smallest possible" or "largest possible" input often behaves differently from a typical one. A test that never checks the empty child (C6) will crash in real code. Covering the matrix is how you make sure your understanding has no holes.

Let's define the three things we'll use over and over. Two describe a single node; the third scores a whole split.

Whenever we evaluate a split (a cut that sends the parent's items into several child nodes), we need one more formula. It is worth stating on its own, because Cells C6–C8 all lean on it.

Everything below is just these formulas, applied honestly to every cell.

Cells C1–C4: the four "corner" nodes

The picture below plots $G$ for a 2-class node ( $K = 2$ ) as the first class's proportion $p$ slides from $0$ to $1$ . Watch how the corners of our matrix land on this curve.

Worked example Example 1 — C1 (pure) and C2 (balanced), read off the curve

Node P: 20 spam, 0 ham. Node Q: 10 spam, 10 ham. (Both have $K = 2$ .)

Forecast: guess each $G$ before reading — one should be the smallest possible, one the largest possible for 2 classes.

Node P proportions: $p = (20/20, 0/20) = (1, 0)$ . Why this step? Gini only ever eats proportions, so convert counts first.
$\sum p_{i}^{2} = 1^{2} + 0^{2} = 1$ , so $G_{P} = 1 - 1 = 0$ . Why subtract from 1? $\sum p_{i}^{2}$ is the probability of a correct match; $1$ minus it is the mistake rate. Here matching is certain, so mistakes $= 0$ .
Node Q proportions: $p = (10/20, 10/20) = (0.5, 0.5)$ .
$\sum p_{i}^{2} = 0. 5^{2} + 0. 5^{2} = 0.5$ , so $G_{Q} = 1 - 0.5 = 0.5$ . Why subtract from 1? Same reason: a $0.5$ chance of matching leaves a $0.5$ chance of erring — the peak of the red curve, the worst mix for 2 classes.

Verify: $G$ for $K = 2$ can never exceed $1 - 1/ K = 1 - 1/2 = 0.5$ . Node Q sits exactly on that ceiling ✓; Node P sits on the $0$ floor ✓.

The next figure shows how the ceiling $1 - 1/ K$ climbs as classes multiply — and why more classes can be "more confusing".

Cell C5: concentration beats spreading

Cell C6: degenerate & edge inputs

Real code hands Gini weird nodes: a single sample, or an empty child after a split. Does the formula survive?

Worked example Example 5 — C6, single-sample node and empty child

Node S: exactly 1 sample, class "yes" (binary problem, $K = 2$ : classes "yes" and "no"). Child E: 0 samples (nothing routed here).

Forecast: a 1-item node can't be mixed, so guess $G = 0$ . An empty child has no misclassification to make — but watch the weight.

Node S: the one item is "yes", so $p = (1, 0)$ (proportion "yes" $= 1/1 = 1$ , proportion "no" $= 0/1 = 0$ ). Why write both entries? In a binary problem there are always $K = 2$ proportions; showing the explicit $0$ for the absent class avoids ambiguity.
$\sum p_{i}^{2} = 1^{2} + 0^{2} = 1$ , so $G_{S} = 1 - 1 = 0$ . Why subtract from 1? A lone item makes matching certain, so the mistake rate is $0$ — trivially pure.
Child E: proportions are undefined ( $0/0$ ). But in the weighted-split formula its weight is $N_{c} / N = 0/ N = 0$ . Why this saves us? From the [!formula] box, $G_{split} = \sum_{c} \frac{N _{c}}{N} G_{c}$ . An empty child multiplies whatever-garbage- $G_{E}$ -would-be by $0$ , contributing nothing. We simply skip empty children in practice.
Consider a split that routes all items to Node S and leaves Child E empty. Then $G_{split} = \frac{N _{S}}{N} G_{S} + \frac{0}{N} G_{E} = 1 \cdot G_{S} + 0 = G_{S},$ so gain $= G_{parent} - G_{split} = G_{parent} - G_{S} = 0$ (since $S$ is the parent here). Why does gain vanish? The empty term contributes $0 \cdot G_{E}$ , so $G_{split}$ equals the single non-empty child's impurity, which equals the parent's — no impurity was removed. The tree therefore refuses such a "split", exactly as it should.

Verify: $G_{S} = 1 - 1 = 0$ ✓. And $\frac{0}{N} \cdot (anything) = 0$ , so the empty child term vanishes ✓ — the formula is safe at the edge, and a one-sided "split" earns zero gain ✓.

Cell C7: the split decision, fully worked

This is the whole point of Gini: comparing splits with the weighted-split formula. The figure shows a parent node being cut two different ways.

Worked example Example 6 — C7, choosing between three candidate splits

Parent: 6 positive (+), 6 negative (−), total $N = 12$ , $K = 2$ .

Forecast: the split that produces the purest children (children nearest $G = 0$ ) should win the largest gain.

Parent impurity: $p = (6/12, 6/12) = (0.5, 0.5)$ , so $G_{parent} = 1 - (0. 5^{2} + 0. 5^{2}) = 1 - 0.5 = 0.5.$ Why first? Gain is measured relative to the parent, so we need this baseline. (Subtract from 1 for the usual reason: $0.5$ correct-match chance leaves a $0.5$ mistake rate.)
Split A → Left $(6 +, 0 -) = 6$ , Right $(0 +, 6 -) = 6$ (a perfect split!).
- $G_{L} = 1 - 1^{2} - 0^{2} = 0$ ; $G_{R} = 1 - 0^{2} - 1^{2} = 0$ . Why 0? Each child is one pure class.
- $G_{A} = \frac{6}{12} (0) + \frac{6}{12} (0) = 0$ . Why weight by $6/12$ ? Each child holds half the data, so each gets half the say (the $N_{c} / N$ from the formula box).
Split B → Left $(5 +, 1 -) = 6$ , Right $(1 +, 5 -) = 6$ .
- $G_{L} = 1 - (5/6)^{2} - (1/6)^{2} = 1 - 25/36 - 1/36 = 10/36 \approx 0.278$ . Why subtract both squares? One term per class inside the child.
- $G_{R} =$ same by symmetry $\approx 0.278$ .
- $G_{B} = \frac{6}{12} (0.278) + \frac{6}{12} (0.278) = 0.278$ .
Split C → Left $(4 +, 2 -) = 6$ , Right $(2 +, 4 -) = 6$ .
- $G_{L} = 1 - (4/6)^{2} - (2/6)^{2} = 1 - 16/36 - 4/36 = 16/36 \approx 0.444$ .
- $G_{R} =$ same $\approx 0.444$ .
- $G_{C} = 0.444$ .
Gains (bigger = better), each $= G_{parent} - G_{split}$ :
- Split A: $0.5 - 0 = 0.5$ .
- Split B: $0.5 - 0.278 = 0.222$ .
- Split C: $0.5 - 0.444 = 0.056$ . Why subtract? Gain measures how much impurity the split removed from the parent.

Decision: Split A wins — it fully separates the classes.

Verify: the purest children (A, both $G = 0$ ) gave the largest gain ( $0.5$ ); the muddiest (C) gave the smallest ( $0.056$ ) ✓, exactly as forecast. Gains are ordered $A > B > C$ , matching child purity.

Cell C8: a real-world word problem

Worked example Example 7 — C8, a loan-approval node

A bank's decision-tree node holds 200 loan applicants in $K = 3$ classes: 150 who repaid ("good"), 40 who defaulted ("bad"), and 10 still "pending". A junior analyst asks: "how impure is this node, and would splitting on 'income > 50k' help?" The income split gives:

High-income child: 120 good, 5 bad, 5 pending (130 people).
Low-income child: 30 good, 35 bad, 5 pending (70 people).

Forecast: the high-income child is dominated by "good", so it should be much purer than the low-income child, which is nearly good/bad balanced.

Parent proportions ( $K = 3$ ): $p = (150/200, 40/200, 10/200) = (0.75, 0.20, 0.05)$ .
$G_{parent} = 1 - (0.7 5^{2} + 0.2 0^{2} + 0.0 5^{2}) = 1 - (0.5625 + 0.04 + 0.0025) = 1 - 0.605 = 0.395$ . Why subtract from 1? $0.605$ is the correct-match chance; $0.395$ is the mistake rate. One dominant class keeps impurity moderate.
High-income child: $p = (120/130, 5/130, 5/130)$ . $G_{H} = 1 - (\frac{120}{130})^{2} - 2 (\frac{5}{130})^{2} = 1 - 0.8521 - 0.0030 = 0.1450.$ Why the $2 \times$ ? Two classes ("bad", "pending") share the identical proportion $5/130$ .
Low-income child: $p = (30/70, 35/70, 5/70)$ . $G_{L} = 1 - (\frac{30}{70})^{2} - (\frac{35}{70})^{2} - (\frac{5}{70})^{2} = 1 - 0.1837 - 0.25 - 0.0051 = 0.5612.$
Weighted split impurity (the [!formula] box, $N_{c} / N = 130/200$ and $70/200$ ): $G_{split} = \frac{130}{200} (0.1450) + \frac{70}{200} (0.5612) = 0.0943 + 0.1964 = 0.2907.$ Why weight? The larger, purer high-income child deserves more say than the smaller messy one.
Gain $= G_{parent} - G_{split} = 0.395 - 0.2907 = 0.1043$ . Why subtract? It reports how much impurity the income split stripped away.

Verify (with units of "probability"): all $G$ values lie in $[0, 1 - 1/3] = [0, 0.667]$ ✓. The high-income child ( $0.145$ ) is far purer than the low-income child ( $0.561$ ) as forecast ✓, and the split earns a positive gain $\approx 0.104$ ✓ ( $G_{split} = 0.2907 < G_{parent} = 0.395$ ), so yes, splitting on income helps ✓.

Cell C9: the exam twist — run Gini backwards

Worked example Example 8 — C9, recover the split from a given impurity

Exam question: "A 2-class node ( $K = 2$ ) has Gini impurity $G = 0.32$ . What are the two class proportions?"

Forecast: Gini is symmetric in $p$ and $1 - p$ , so expect a matched pair like $(p, 1 - p)$ with $p$ somewhere between $0.5$ and $1$ (since $G < 0.5$ means it's skewed, not balanced).

Let the proportions be $p$ and $1 - p$ . Then $G = 1 - p^{2} - (1 - p)^{2} = 2 p - 2 p^{2} = 2 p (1 - p) .$ Why this form? For $K = 2$ , expanding $1 - p^{2} - (1 - p)^{2}$ collapses to $2 p (1 - p)$ — a clean quadratic in one unknown.
Set equal to $0.32$ : $2 p (1 - p) = 0.32 \Rightarrow p - p^{2} = 0.16 \Rightarrow p^{2} - p + 0.16 = 0$ . Why rearrange? To put it in standard quadratic form so the quadratic formula applies.
$p = \frac{1 \pm 1 - 4 ( 0.16 )}{2} = \frac{1 \pm 0.36}{2} = \frac{1 \pm 0.6}{2}$ . Why the quadratic formula? It's the tool that inverts any $a x^{2} + b x + c = 0$ — exactly what "recover $p$ " asks.
So $p = 0.8$ or $p = 0.2$ . Both describe the same node: $(0.8, 0.2)$ . Why the same node? The two roots are the two class shares — relabelling which class is "first" cannot change the node.

Verify: $G = 1 - 0. 8^{2} - 0. 2^{2} = 1 - 0.64 - 0.04 = 0.32$ ✓. The two roots are mirror images (as forecast) because relabelling the classes can't change impurity.

Active Recall

Recall

A node has 3 classes with counts $(80, 15, 5)$ out of 100. Gini? ::: $1 - (0. 8^{2} + 0.1 5^{2} + 0.0 5^{2}) = 1 - (0.64 + 0.0225 + 0.0025) = 0.335$ Two children with sizes 40 and 60 have Ginis $0.1$ and $0.5$ . Weighted split Gini? ::: $\frac{40}{100} (0.1) + \frac{60}{100} (0.5) = 0.04 + 0.30 = 0.34$ Which cell does an empty child fall in, and why doesn't it break the formula? ::: C6 (degenerate); its weight $N_{c} / N = 0$ zeroes the term before $G_{c}$ is ever evaluated. A 2-class node has $G = 0.5$ . What are its proportions? ::: $(0.5, 0.5)$ — the unique 2-class maximum. Why does a skewed 3-class node have lower $G$ than a uniform 3-class node? ::: Squaring rewards concentration, so a dominant class raises $\sum p_{i}^{2}$ , lowering $G = 1 - \sum p_{i}^{2}$ .

Connections

Gini impurity — parent note (formula, derivation, mnemonics).
Decision Trees — where these split comparisons (C7) actually run.
Entropy and Information Gain — the same examples with $- \sum p lo g p$ instead.
Overfitting and Pruning — why chasing the pure children of C1/Split A can backfire.
Random Forests — many Gini-grown trees averaged.
Classification vs Regression Trees — regression swaps Gini for variance.
Gini Coefficient (Economics) — different concept, same name (see parent's mistake box).

Worked examples — Gini impurity