Choosing K (elbow method, silhouette score): Question bank

Before we start, three words we lean on constantly, all from the parent note:

Inertia = total squared distance from every point to its own cluster centre (lower = tighter clusters).
==Cohesion $a (i)$ == = how close point $i$ sits to its own cluster.
==Separation $b (i)$ == = how close point $i$ sits to the nearest other cluster.

True or false — justify

As K rises, inertia can go up if the clusters get worse

False. K-means minimises inertia at each K, and any (K+1)-solution can copy the K-solution plus one extra singleton centroid, so inertia is monotonically non-increasing in K — it never rises.

The elbow is the K where inertia becomes zero

False. Inertia only reaches zero at K = N (every point its own centroid). The elbow is where the rate of decrease sharply slows, long before zero.

A higher silhouette score always means a better clustering for your problem

False. Silhouette measures geometric tightness under Euclidean distance; it is systematically biased toward small K (splitting data in half scores easily) and ignores whether the split is meaningful for your task.

Silhouette score can be negative

True. When

a (i) > b (i)

— a point is closer on average to a neighbouring cluster than to its own —

s (i) < 0

, signalling likely misassignment. The overall average can dip below zero if many such points exist.

The elbow method needs you to specify a distance metric, just like silhouette

True in spirit. Inertia is built from squared Euclidean distances, so both methods assume Euclidean geometry and spherical-ish clusters; neither is metric-free.

If two K values give nearly equal silhouette scores, the lower K is the safer default

True, usually. Fewer clusters means a simpler, more interpretable model with lower variance, echoing the Bias-Variance Tradeoff — but only if domain knowledge doesn't demand the finer split.

Silhouette can be computed for K = 1

False. With one cluster there is no other cluster, so

b (i)

is undefined. Silhouette starts at K = 2; the elbow method starts at K = 1.

A perfectly smooth inertia curve means K-means found no structure

False. It means no single K stands out sharply — the data may have hierarchical, overlapping, or non-spherical structure. Reach for silhouette, the Gap Statistic, or a different algorithm like DBSCAN.

Spot the error

"I picked K by finding the minimum inertia across all K."

Inertia is minimised at K = N (zero), so this always picks the maximum K — useless. You want the bend, not the minimum.

"My silhouette peaked at K=2, so K=2 is the true number of clusters."

Peaking at K=2 is the method's known bias, not proof of truth. Cross-check with the elbow, the per-point silhouette diagram, and domain meaning before trusting it.

"The elbow curve had no bend, so K-means failed."

A smooth curve means K-means found no dominant K, not that it failed. The algorithm still ran; the diagnostic is just inconclusive — combine tools rather than concluding failure.

"I used silhouette to choose K for DBSCAN, since it worked for K-Means Clustering."

DBSCAN doesn't take K as input; it discovers cluster count from density (eps, minPts). Choosing-K tooling here is category-mismatched — tune DBSCAN's density parameters instead.

"a(i) averages distance to ALL points in my cluster, including the point itself."

You must exclude the point itself, so you divide by

∣ C_{I} ∣ - 1

, not

∣ C_{I} ∣

. Including a point's zero distance to itself would wrongly shrink

a (i)

"b(i) is the average distance to ALL other clusters combined."

No —

b (i)

takes the minimum over other clusters: only the single nearest neighbouring cluster matters, because that's the strongest competing home for the point.

"Silhouette is unbounded, so 5.2 is a great score."

The

max (a, b)

denominator forces

s (i) \in [- 1, 1]

; any average outside that range means a computation bug.

"Elbow said 4, silhouette said 3, so both methods disagree and are useless."

Disagreement is normal and informative — it flags a candidate range (3–4). Treat the methods as voters plus domain knowledge, not oracles that must agree.

Why questions

Why does inertia use squared distance instead of plain distance?

Squaring gives a closed-form optimal centroid (the mean) and penalises far points heavily (2× farther ⇒ 4× cost), which is exactly what Lloyd's algorithm exploits when it re-sets centroids to cluster means.

Why does silhouette divide by

max (a (i), b (i))

rather than by a fixed constant?

To normalise into

[- 1, 1]

regardless of the data's absolute distance scale, so scores are comparable across datasets and K values.

Why does the elbow method plot inertia against K rather than just reporting one number?

The useful signal is the shape — the point of diminishing returns — which only appears as a bend across many K values; a single inertia number is meaningless without its neighbours.

Why is silhouette biased toward small K?

Fewer clusters are easier to keep far apart, so

b (i)

tends to stay large while

a (i)

stays modest, inflating

s (i)

— halving data almost always looks "clean" geometrically.

Why combine elbow + silhouette + domain knowledge instead of trusting one?

Each has a blind spot: elbow is subjective, silhouette is small-K-biased, domain intuition can be wrong. Treating them as a voting system turns three weak signals into one robust decision — the workflow relates to disciplined Hyperparameter Tuning.

Why might Hierarchical Clustering sidestep the "choose K first" problem entirely?

It builds a full merge tree (dendrogram) without committing to K upfront; you cut the tree at any height afterward, so K becomes a post-hoc choice rather than a required input.

Why does a point on a cluster boundary get a silhouette near 0?

On the boundary its own-cluster distance

a (i)

and nearest-other distance

b (i)

are nearly equal, so the numerator

b (i) - a (i) \approx 0

— the point genuinely belongs equally to both.

Edge cases

What is the silhouette of a point that is the only member of its cluster?

a (i)

is undefined (no same-cluster neighbours), so by convention

s (i) = 0

— a singleton is treated as neither well- nor poorly-clustered.

What happens to inertia and silhouette when K = N (every point alone)?

Inertia = 0 (perfectly tight), but silhouette is 0 for every singleton — the two diagnostics correctly refuse to reward this degenerate over-clustering.

If all your data forms one true blob, what will the elbow show?

No sharp bend — inertia declines smoothly because splitting one blob is always somewhat helpful geometrically. The absence of an elbow is itself the answer: K = 1.

Two clusters have very different densities (tight vs loose). What can silhouette get wrong?

A point in the loose cluster can have large

a (i)

purely from natural spread and score poorly despite being correctly assigned; silhouette assumes comparable, spherical densities and misjudges here — a case for DBSCAN.

What K does silhouette recommend if the data is pure uniform noise with no clusters?

Often a small K like 2 with a mediocre score, because there's no real structure to reward; low peak silhouette across all K is the tell that no clustering is meaningful.

If every method points to the same K, is the answer guaranteed correct?

No — agreement gives confidence, not proof. All methods share the Euclidean-spherical assumption, so they can be jointly fooled by non-convex or manifold-shaped clusters.

Recall One-line survival summary

Elbow finds diminishing returns in inertia; silhouette scores fit vs. neighbours in $[- 1, 1]$ ; both assume round clusters, both can be fooled, so vote — never obey a single number.