Choosing K (elbow method, silhouette score): Exercises

Two symbols carry this whole page, so let us fix them in plain words before anything else. First, one letter you will meet everywhere:

Figure — Choosing K (elbow method, silhouette score)

Alt-text: a point $i$ (amber star) inside a cyan "home" cluster on the left and a white "rival" cluster on the right. Short amber lines connect $i$ to its home neighbours — their average is $a (i)$ . Longer dashed cyan lines connect $i$ to the rival points — their average is $b (i)$ .

Look at the figure: $a (i)$ is the short amber spread inside the home pile, $b (i)$ is the long cyan reach to the nearest rival pile. When cyan is much longer than amber, $s (i)$ is near $+ 1$ .

Level 1 — Recognition

Exercise 1.1 (L1)

Below are four inertia readings from running K-Means Clustering with $K = 1, 2, 3, 4$ . Fill in the "drop" (how much inertia fell) for each step.

K	Inertia
1	12000
2	8000
3	3600
4	3200

Recall Solution 1.1

The drop is just the previous inertia minus the current inertia.

$K = 1 \to 2$ : $12000 - 8000 = 4000$
$K = 2 \to 3$ : $8000 - 3600 = 4400$
$K = 3 \to 4$ : $3600 - 3200 = 400$

Notice inertia only ever falls as $K$ grows — that is a mathematical guarantee, because more centres means every point can sit closer to some centre.

Exercise 1.2 (L1)

A point has $a (i) = 0.4$ and $b (i) = 1.6$ . Without computing, is this point well-clustered, ambiguous, or misclassified? Then compute $s (i)$ .

Recall Solution 1.2

Home distance ( $0.4$ ) is much smaller than rival distance ( $1.6$ ), so it is well-clustered ( $s$ near $+ 1$ ). $s (i) = \frac{1.6 - 0.4}{m a x ( 0.4 , 1.6 )} = \frac{1.2}{1.6} = 0.75$

Level 2 — Application

Exercise 2.1 (L2)

Using the table from Exercise 1.1, identify the elbow $K$ and justify it in one sentence.

Recall Solution 2.1

Drops: $4000, 4400, 400$ . The drop collapses from $4400$ to $400$ between $K = 3$ and $K = 4$ , so the elbow is at $K = 3$ — beyond it, extra clusters barely tighten the piles.

Exercise 2.2 (L2)

A point $B$ sits near a border: $a (B) = 0.9$ , $b (B) = 1.0$ . Compute $s (B)$ and state in words what its value tells you.

Recall Solution 2.2

$s (B) = \frac{1.0 - 0.9}{m a x ( 0.9 , 1.0 )} = \frac{0.1}{1.0} = 0.1$ Close to $0$ means ambiguous — $B$ is almost equally close to its home cluster and to the neighbour, so it sits on the overlap boundary. (Had $a$ and $b$ been exactly equal, $s$ would be precisely $0$ — a perfect tie.)

Exercise 2.3 (L2)

Three points have silhouettes $s = 0.8, 0.2, - 0.1$ . Compute the overall silhouette score for this tiny dataset.

Recall Solution 2.3

$Silhouette = \frac{0.8 + 0.2 + ( - 0.1 )}{3} = \frac{0.9}{3} = 0.3$ One negative point ( $- 0.1$ , likely misplaced) drags the average down from what the good point alone would give.

Level 3 — Analysis

Exercise 3.1 (L3)

You get these silhouette averages:

K	Avg silhouette
2	0.70
3	0.66
4	0.48

Silhouette says $K = 2$ . The elbow curve (same data) bends clearly at $K = 3$ . Which $K$ do you report, and why is the silhouette peak here not decisive?

Recall Solution 3.1

Report $K = 3$ . Silhouette is biased toward fewer clusters: cutting data into two big blobs is the easiest way to make every point far from "the other side", so $K = 2$ often wins the raw number. The elbow's honest bend at $K = 3$ plus a still-healthy silhouette of $0.66$ (only $0.04$ below the peak) means $K = 3$ captures real structure without the coarseness of $K = 2$ . See Bias-Variance Tradeoff: $K = 2$ is high-bias (too coarse).

Exercise 3.2 (L3)

Prove in words why inertia can never increase when you raise $K$ from $k$ to $k + 1$ .

Recall Solution 3.2

Take the best solution for $K = k$ . Now add one more centre placed exactly on top of an existing one. Every point can keep its old assignment, so inertia is unchanged. But K-means will only move a point to a new centre if that lowers its squared distance. Therefore the optimal $K = k + 1$ inertia is $\leq$ the $K = k$ inertia. Since it can only stay equal or fall, inertia is monotonically non-increasing in $K$ — this is why raw inertia can never pick $K$ by itself.

Exercise 3.3 (L3)

A cluster $C_{I}$ contains exactly one point. What is $a (i)$ for that lonely point, and why does this break the cohesion formula?

Recall Solution 3.3

The cohesion formula divides by $∣ C_{I} ∣ - 1$ . With $∣ C_{I} ∣ = 1$ , that is $\frac{1}{0}$ — undefined. There are no "other points in the same cluster" to measure closeness to. By convention silhouette sets $s (i) = 0$ for a singleton cluster: with no home neighbours, we cannot say the point fits well or badly.

Level 4 — Synthesis

Exercise 4.1 (L4)

Two clusters, each with two points on a line:

$C_{1} = {0, 2}$ with centroid $μ_{1} = 1$
$C_{2} = {9, 11}$ with centroid $μ_{2} = 10$

(a) Compute total inertia. (b) Compute $a$ and $b$ for the point at $x = 0$ , then $s (0)$ . Distances are absolute differences on the line.

Recall Solution 4.1

(a) Inertia. Squared distance of each point to its centroid:

$C_{1}$ : $(0 - 1)^{2} + (2 - 1)^{2} = 1 + 1 = 2$
$C_{2}$ : $(9 - 10)^{2} + (11 - 10)^{2} = 1 + 1 = 2$ $Inertia = 2 + 2 = 4$

(b) Silhouette of $x = 0$ . $a (0)$ = average distance to other points in $C_{1}$ . Only one other point ( $x = 2$ ), so we divide by $∣ C_{1} ∣ - 1 = 1$ : $a (0) = ∣0 - 2∣ = 2$ . $b (0)$ = average distance to points in the nearest other cluster $C_{2} = {9, 11}$ , dividing by the full $∣ C_{2} ∣ = 2$ : $\frac{∣0 - 9∣ + ∣0 - 11∣}{2} = \frac{9 + 11}{2} = 10$ . $s (0) = \frac{b - a}{m a x ( a , b )} = \frac{10 - 2}{m a x ( 2 , 10 )} = \frac{8}{10} = 0.8$ Strongly well-clustered, as expected — the two clusters are far apart on the line.

Exercise 4.2 (L4)

Same two clusters as 4.1, but now suppose someone forces $K = 1$ : one big cluster ${0, 2, 9, 11}$ with centroid $μ = 5.5$ . Compute the new inertia and comment on what the elbow between $K = 1$ and $K = 2$ looks like.

Recall Solution 4.2

Squared distances to $μ = 5.5$ : $(0 - 5.5)^{2} = 30.25$ , $(2 - 5.5)^{2} = 12.25$ , $(9 - 5.5)^{2} = 12.25$ , $(11 - 5.5)^{2} = 30.25$ . $Inertia_{K = 1} = 30.25 + 12.25 + 12.25 + 30.25 = 85$ From $K = 1$ to $K = 2$ inertia crashes from $85$ to $4$ — a drop of $81$ . Any further split ( $K = 3$ ) can only shave off a tiny bit more. That huge drop then flat line is the elbow: it screams $K = 2$ .

Level 5 — Mastery

Exercise 5.1 (L5)

You must pick $K$ for a customer dataset. You have three signals:

Elbow bends at $K = 4$ .
Silhouette: $s (2) = 0.71, s (3) = 0.63, s (4) = 0.55$ .
Business team insists there are exactly 3 meaningful segments and will act on them.

Design a defensible decision and justify each vote. Then say what one extra diagnostic you would run before committing.

Recall Solution 5.1

Voting system (the parent's best-practice workflow):

Elbow → votes $K = 4$ .
Silhouette → raw max is $K = 2$ , but that is the fewer-clusters bias; $K = 3$ at $0.63$ is only $0.08$ below and still solidly positive.
Domain → votes $K = 3$ , and it is actionable.

Decision: $K = 3$ . It sits between the elbow ( $4$ ) and the silhouette peak ( $2$ ), keeps a healthy silhouette, and is the only value the business can operationalise — a tie-breaker that matters because clustering exists to serve a downstream task (see Hyperparameter Tuning).

Extra diagnostic: plot the per-point silhouette diagram for $K = 3$ . If only a handful of points go negative and the three clusters have comparable sizes, $K = 3$ is confirmed. Many negatives or one tiny cluster → reconsider (perhaps two segments truly overlap, hinting DBSCAN).

Exercise 5.2 (L5)

Given per-point silhouettes for a $K = 3$ run — cluster sizes ${4, 3, 3}$ with values $C_{1} = {0.9, 0.8, 0.7, 0.6}$ , $C_{2} = {0.5, 0.4, 0.6}$ , $C_{3} = {0.2, - 0.1, - 0.3}$ — compute (a) each cluster's mean silhouette, (b) the overall silhouette, and (c) diagnose which cluster is problematic and what you would do.

Recall Solution 5.2

(a) Per-cluster means.

$C_{1} = \frac{0.9 + 0.8 + 0.7 + 0.6}{4} = \frac{3.0}{4} = 0.75$
$C_{2} = \frac{0.5 + 0.4 + 0.6}{3} = \frac{1.5}{3} = 0.50$
$C_{3} = \frac{0.2 - 0.1 - 0.3}{3} = \frac{- 0.2}{3} \approx - 0.067$

(b) Overall (average over all $10$ points, so here $N = 10$ ): $\frac{3.0 + 1.5 + ( - 0.2 )}{10} = \frac{4.3}{10} = 0.43$

(c) Diagnosis. $C_{3}$ has a negative mean — its points are, on average, closer to a rival cluster than to home. This cluster is not real: its members are probably absorbed into $C_{2}$ or scattered noise. Action: try $K = 2$ , or move to a density method like DBSCAN that can label those three points as noise instead of forcing a cluster.

Recall Explain to a 12-Year-Old

Elbow method asks "when does adding another toy box stop making my room tidier?" Silhouette asks "does each toy feel like it's in the right box, or does it wish it were in the box next door?" You trust neither alone — you let the graph, the score, and common sense vote.