Foundations — Choosing K (elbow method, silhouette score)

Before you can read the parent note on Choosing K, you meet a pile of symbols: $\mathbf{x}$, $\mathbf{\mu }$, $\Vert \cdot {\Vert }^2$, $C_i$, $\sum$, $a(i)$, $b(i)$, $\min$, $\max$. This page builds each one from nothing, in the order they need each other. A smart 12-year-old should finish able to read every formula on the parent page out loud and know what it means.

1. A data point — the dot on the page

Everything starts with a single piece of data.

The bold letter $\mathbf{x}$ is a signal: "this is not one number, it is a little bundle of numbers." Each number is called a feature or a coordinate.

Look at the figure: two features means we can draw each data point as a dot on a flat page, spending along the bottom, frequency up the side. This picture — dots scattered on a plane — is the mental image behind every symbol that follows. Clustering is just: draw loops around dots that huddle together.

2. Distance — how far apart two dots are

To say dots "huddle together" we need to measure the gap between two dots.

The little subscripts $i$ and $j$ are just name tags for different dots ("dot number $i$", "dot number $j$"). They are not maths operations — they only tell dots apart.

WHY straight-line distance and not something fancier? Because on a flat page, "close together" means short ruler-length. This is the ordinary distance from the Pythagorean theorem:

The silhouette score on the parent page uses $d({\mathbf{x}}_i,{\mathbf{x}}_j)$ directly, so this symbol is essential.

3. Norm and squared distance — the $\Vert \cdot {\Vert }^2$ symbol

The parent's inertia formula uses $\Vert \mathbf{x}-{\mathbf{\mu }}_i{\Vert }^2$. Let's earn every mark.

So $\Vert \mathbf{x}-\mathbf{\mu }\Vert$ is: "make the arrow from the centre $\mathbf{\mu }$ to the point $\mathbf{x}$, then take its length." Subtracting two points gives the arrow between them; the bars give that arrow's length.

The little ${}^2$ on the outside means square that length (multiply it by itself).

In the figure, each dashed line is one $\Vert \mathbf{x}-\mathbf{\mu }\Vert$; the little shaded square hanging off it is that distance squared. Bigger square = bigger penalty.

4. Centroid — the balance point of a cluster, ${\mathbf{\mu }}_i$

Bold again → it is a point with coordinates, not a single number. To find it you average the spending of all dots in the cluster (that's its horizontal position) and average their frequency (its vertical position). This is the "mean" the squaring in §3 rewards.

5. A cluster and its size — $C_i$ and $|C_i|$

Notice the bars mean two different things depending on what's inside:

• $\Vert \mathbf{v}\Vert$ around an arrow/point → its length.
• $|C_i|$ around a set → its count.

Context tells you which. The parent uses $|C_I|-1$ in cohesion because when you average distances from a point to its cluster-mates, the point doesn't count itself, so there are $|C_I|-1$ neighbours.

6. The summation sign $\sum$ — "add up all of these"

The scariest-looking symbol is the friendliest.

${\sum }_{\mathbf{x}\in C_i}\Vert \mathbf{x}-{\mathbf{\mu }}_i{\Vert }^2$

Read it slowly, right to left:

• $\Vert \mathbf{x}-{\mathbf{\mu }}_i{\Vert }^2$ — the squared distance from one dot to its centre.
• ${\sum }_{\mathbf{x}\in C_i}$ — "for every dot $\mathbf{x}$ in ($\in$) cluster $C_i$, do that and add them all together."

The symbol $\in$ means "is a member of." So the whole thing is: walk through every dot in this cluster, square its distance to the centre, and total them.

A $\sum$ with numbers, like ${\sum }_{i=1}^K$, means "let $i$ be $1$, then $2$, ... up to $K$, and add a copy for each." Nested sums (one inside another) just mean do all clusters, and inside each, do all its points — a loop inside a loop.

7. Inertia — one number for "how messy are the clusters"

Now every piece of the inertia formula is defined, so we can read it whole:

This single number is what the elbow method plots against $K$.

8. $\min$ and $\max$ — the smallest and biggest of a list

The silhouette formulas use two more words dressed as symbols.

• In separation, $b(i)={\min }_{J\ne I}(\dots )$ means: look at the average distance to each other cluster, and keep the smallest — the nearest rival cluster. ($J\ne I$ = "over all clusters $J$ that are not the home cluster $I$".)
• In the silhouette coefficient, dividing by $\max (a(i),b(i))$ keeps the answer between $-1$ and $1$ by dividing by whichever of the two numbers is larger.

9. Cohesion $a(i)$ and separation $b(i)$ — the silhouette pair

With distance, $\sum$, size, and $\min$ in hand, both are now readable:

The silhouette of one dot then just compares the two: $s(i)=\frac{b(i)-a(i)}{\max (a(i),b(i))}$ Close to home and far from rivals ⇒ $b>a$ ⇒ $s$ near $+1$. That's the whole story.

Prerequisite map

The whole page fans into two measuring sticks — inertia for the elbow, silhouette for the fit score — and those two feed the decision of Choosing K. Related methods worth meeting later: K-Means Clustering (which produces the clusters), DBSCAN and Hierarchical Clustering (which need no $K$), the Gap Statistic (a fancier elbow), and the general idea of Hyperparameter Tuning and Bias-Variance Tradeoff that "picking $K$" is one example of.

Equipment checklist

Cover the right side and answer each; only move on when all reveal correctly.