Worked examples — Explained variance and choosing components

This page is the "no surprises" drill for Explained variance and choosing components. The parent note gave you the formulas; here we hit every kind of input the topic can throw at you — clean cases, weird cases, and the traps — and work each one to the last decimal.

Before we start, one promise: every symbol below is already earned in the parent note, but let us re-anchor the three we lean on constantly, in plain words.

The scenario matrix

Every explained-variance problem is really one of these cells. The rest of the page fills each one.

[!example] Example 1 — Case A: clean drop-off, obvious elbow

Statement. Eigenvalues (already sorted): $\lambda =[9,3,0.6,0.4]$. Find every ${\text{EVR}}_i$, the CEV curve, and the elbow.

Forecast: guess now — does one component already carry more than half? Where does the curve flatten?

1. Total variance. $\sum \lambda =9+3+0.6+0.4=13$. Why this step? The denominator is the whole spread; every fraction is measured against it.
2. EVRs. $\frac{9}{13}=0.6923$, $\frac{3}{13}=0.2308$, $\frac{0.6}{13}=0.0462$, $\frac{0.4}{13}=0.0308$ (each rounded to 4 decimals). Why this step? Turns raw variances into comparable fractions.
3. CEV. $0.6923,0.9231,0.9692,1.0000$. Why this step? Running total answers "keep first $k$?".
4. Elbow. The jump from comp 2 to comp 3 collapses ($0.2308\to 0.0462$). Keep $k=2$: 92.3% retained, half the dimensions gone. Why this step? The elbow is where each extra component stops paying its way — the classic stopping rule.

The figure below shows exactly this: cyan bars are the four EVRs (notice bar 1 is more than half the height of the whole chart), and the amber curve is the CEV climbing to $1$. The amber arrow marks the elbow — the point after which each new bar barely lifts the curve. That flattening is the visual cue to stop at $k=2$.

Verify: EVRs sum to $1$; CEV is non-decreasing and ends at $1.0000$. Sanity: comp 1 alone > 50%, matching a clear dominant direction. ✓

[!example] Example 2 — Case B: flat spectrum, nothing to compress

Statement. $\lambda =[1,1,1,1,1]$ (five equal variances). What does explained variance tell us?

Forecast: if all directions are equally spread, how many can you drop for free?

1. Total. $\sum \lambda =5$. Why this step? Baseline — the whole spread we measure fractions against.
2. EVR each. $\frac{1}{5}=0.20$ for every component. Identical. Why this step? Equal EVRs are the fingerprint of a data cloud with no preferred direction.
3. CEV. $0.2,0.4,0.6,0.8,1.0$ — a perfectly straight line. Why this step? A straight CEV line is the visual signature of "no free lunch."
4. Reading it. To keep 90% you need $k$ with $0.2k\ge 0.9\Rightarrow k\ge 4.5\Rightarrow k=5$. You must keep all of them. PCA cannot help here. Why this step? It turns the abstract "flat spectrum" into the concrete verdict: no compression is possible at this threshold.

In the figure below the cyan bars are all the same height (each EVR $=0.2$) and the amber CEV climbs as a perfectly straight ramp to $1$ — contrast this with Example 1's cliff-then-plateau. A straight ramp with no elbow means there is no natural place to stop early. The x-axis is component number; the left y-axis is EVR, the right y-axis (amber) is CEV.

Verify: $0.2\times 5=1.0$; CEV straight-line slope $=0.2$ per step. This is the worst case for dimensionality reduction — the data is an isotropic ball. ✓

[!example] Example 3 — Case C: a zero eigenvalue (degenerate direction)

Statement. $\lambda =[6,4,0]$. Interpret the zero.

Forecast: what happened in the data for a variance to be exactly zero?

1. Total. $6+4+0=10$. Why this step? Baseline denominator; note the zero contributes nothing to it.
2. EVRs. $0.6,0.4,0.0$. Why this step? The third component captures no spread at all — the EVR of a dead direction is exactly zero.
3. What a zero means. ${\lambda }_3=0\Rightarrow$ the data has no variation along that eigenvector — the points lie flat in a 2-D plane inside 3-D space (one feature is a linear combination of the others). Why this step? Zero variance = redundant direction = it can be dropped with zero information loss.
4. CEV. $0.6,1.0,1.0$. Keeping $k=2$ already gives 100%. Why this step? Once the CEV reaches exactly $1$ before the last component, every remaining component is provably disposable.

In the figure below the third cyan bar has height zero (the amber arrow points to the empty slot on the axis), and the CEV curve reaches $1.0$ already at $k=2$ then runs flat — the visual signature of a rank-deficient dataset. The x-axis is component number; the left y-axis is EVR, the right (amber) is CEV.

Verify: ${\text{CEV}}_2=\frac{6+4}{10}=1.0$ exactly, and ${\text{CEV}}_3$ adds nothing. A true rank-deficient dataset. This is the degenerate input every treatment must cover. ✓

[!example] Example 4 — Case D: threshold hunt with an off-grid cutoff

Statement. $\lambda =[10,8,5,3,2,1,0.7,0.3]$ (total $30$). Smallest $k$ for $\ge 95\%$?

Forecast: the 95% line rarely lands on a component boundary — guess whether it's 5, 6, or 7.

1. Target variance. $0.95\times 30=28.5$. Why this step? Convert the fraction into an absolute cumulative-variance goal so we can just add.
2. Cumulative sums. $10,18,23,26,28,29,29.7,30$. Why this step? We walk components in importance order, accumulating.
3. Cross the line. $28<28.5$ at $k=5$; $29\ge 28.5$ at $k=6$. First crossing is $k=6$. Why this step? We take the smallest $k$ that meets or beats the target.
4. Report CEV. ${\text{CEV}}_6=\frac{29}{30}\approx 0.9667$ (96.67%). Why this step? We report the actual retained fraction, not just the count, so the reader knows exactly how much information the chosen $k$ keeps.

In the figure below, the cyan curve is the CEV; the dashed amber line is the 95% target. Trace the curve left-to-right and watch where it first pokes above the dashed line — that is the amber dot at $k=6$. Notice the point at $k=5$ sits just under the line: this is exactly why "round the CEV up to the next component" gives 6, not 5. The x-axis is the number of components kept, the y-axis is cumulative explained variance.

Verify: ${\text{CEV}}_5=28/30\approx 0.9333<0.95$, ${\text{CEV}}_6=29/30\approx 0.9667\ge 0.95$. So $k=6$ is correct, not 5. ✓ (This mirrors the algorithm in the parent note.)

[!example] Example 5 — Case E: the scaling trap (units change the answer)

Statement. Two features: height variance $=0.01$ (metres²) and weight variance $=400$ (kg²), uncorrelated. Compute EVR on raw data, then after standardising (each feature scaled to variance 1).

Forecast: will PCA "think" weight is almost 100% of the information? Should it?

1. Raw EVRs. Total $=0.01+400=400.01$. Carrying full precision, ${\text{EVR}}_{\text{weight}}=\frac{400}{400.01}=0.99997500$ and ${\text{EVR}}_{\text{height}}=\frac{0.01}{400.01}=0.00002500$. These two are exact fractions and add to exactly $1$; the short forms $0.999975$ and $0.000025$ are just those numbers rounded to 6 decimals. Why this step? Shows raw PCA is dominated by the feature with the biggest numerical variance — an artefact of units, not importance.
2. The catch. Height carries real signal, but its tiny metric-scale variance makes it invisible. This is a units illusion, not a data fact. Why this step? Motivates standardisation.
3. Standardise. Divide each feature by its std so both have variance $1$: $\lambda =[1,1]$. Why this step? Puts features on equal footing (this is feature preprocessing before extraction).
4. New EVRs. $\frac{1}{2}=0.5$ each — now neither dominates. Why this step? Confirms standardisation removed the unit illusion: the two features are now judged by structure, each contributing equally.

In the figure below two cyan bar-pairs sit side by side: on the left, "raw" — one bar fills almost the entire height and its partner is a sliver you can barely see (the $0.000025$ height); on the right, "standardised" — two equal half-height bars. The amber arrow highlights how the sliver grows into an equal partner once units are removed. The x-axis groups the two scenarios; the y-axis is EVR.

Verify: raw fractions add exactly: $\frac{400}{400.01}+\frac{0.01}{400.01}=\frac{400.01}{400.01}=1$. Standardised EVRs sum to $1$ with each $=0.5$. Lesson confirmed: scale before PCA unless units are already comparable. ✓

[!example] Example 6 — Case F: real-world word problem

Statement. A factory logs 12 vibration sensors. PCA gives $\lambda =[50,20,15,8,4,1.5,0.8,0.4,0.2,0.06,0.03,0.01]$. Management says "keep 90% of the signal, storage is expensive." How many sensors-worth of components do you keep, and what's the compression?

Forecast: 12 sensors → guess whether you land near 4 or near 8 components.

1. Total. Sum $=50+20+15+8+4+1.5+0.8+0.4+0.2+0.06+0.03+0.01=100.00$. Why this step? Nice round total makes CEV read directly as a percentage.
2. Cumulative. $50,70,85,93,97,\dots$ Why this step? Walk down until we pass 90.
3. Cross 90%. $85<90$ at $k=3$; $93\ge 90$ at $k=4$. Keep $k=4$. Why this step? Smallest $k$ meeting the target.
4. Business answer. Store 4 components instead of 12 → $\frac{4}{12}=33.3\%$ of the storage, retaining ${\text{CEV}}_4=93\%$ of the signal. Why this step? This translates the maths into the decision management actually asked for — a storage/retention trade-off, not a bare number.

In the figure below the cyan CEV curve climbs steeply over the first four sensors then flattens into a long tail; the dashed amber line is the 90% target and the amber dot marks the first crossing at $k=4$. The steepness-then-tail shape is why 4 of 12 sensors suffice. The x-axis is the number of components kept; the y-axis is cumulative explained variance.

Verify: ${\text{CEV}}_3=0.85<0.90$, ${\text{CEV}}_4=0.93\ge 0.90$. Compression $=1-4/12=66.7\%$ storage saved. Units: fractions dimensionless, storage ratio consistent. ✓

[!example] Example 7 — Case G: exam twist, only partial info

Statement. You are told only that $d=10$, the total variance is $100$, and ${\lambda }_1=60$. The rest are unknown but sorted descending. Give tight bounds on ${\text{CEV}}_2$.

Forecast: with one component eating 60%, how big can the second be? How small?

1. Lower bound on ${\lambda }_2$. The remaining $9$ eigenvalues sum to $100-60=40$, are sorted, so ${\lambda }_2$ is the largest of nine numbers summing to $40$: it is at least the average $\frac{40}{9}=4.444$. Why this step? A max is never below the mean of its group.
2. Upper bound on ${\lambda }_2$. It cannot exceed ${\lambda }_1=60$ (descending order), and cannot exceed the whole remainder $40$. So ${\lambda }_2\le 40$. Why this step? Order + total cap the second value.
3. Translate to CEV. ${\text{CEV}}_2=\frac{60+{\lambda }_2}{100}$, so $\frac{64.444}{100}\le {\text{CEV}}_2\le \frac{100}{100}$, i.e. $0.6444\le {\text{CEV}}_2\le 1.0$. Why this step? Bounds on ${\lambda }_2$ become bounds on the retained fraction — we answer the question even without full data.

In the figure below the amber band shows the whole range ${\text{CEV}}_2$ can occupy: the lower edge at $0.6444$ (when the remaining nine are as equal as possible) and the upper edge at $1.0$ (when ${\lambda }_2$ swallows the entire remainder and the rest are zero — the Case-C degenerate extreme). The single cyan dot at ${\text{CEV}}_1=0.6$ is the one thing we know for sure. The x-axis is $k$; the y-axis is cumulative explained variance.

Verify: lower $=(60+40/9)/100=0.64\overline{4}$; upper $=1.0$ (attained only if ${\lambda }_2=40$ and the rest are 0 — the Case C degenerate extreme). Both endpoints are realisable, so the bounds are tight. ✓

[!example] Example 8 — Case H: limiting behaviour ($k=0$, $k=d$, noise floor)

Statement. Using $\lambda =[7,2,1]$ (total $10$), evaluate the endpoints ${\text{CEV}}_0$ and ${\text{CEV}}_3$, then describe what happens as more and more tiny "noise" eigenvalues are appended.

Forecast: what is CEV when you keep zero components? When you keep all?

1. $k=0$. Empty sum $=0$, so ${\text{CEV}}_0=\frac{0}{10}=0$. Why this step? Keeping nothing retains nothing — the curve must start at 0.
2. $k=d=3$. ${\text{CEV}}_3=\frac{7+2+1}{10}=1.0$. Why this step? Keeping everything retains everything — the curve must end at 1.
3. Appending noise. Add many tiny eigenvalues, say $30$ copies of $0.01$ (total extra $=0.3$). New total $=10.3$. Now ${\text{CEV}}_3=\frac{10}{10.3}=0.9709$. Why this step? Noise inflates the denominator, so the same three real components now "explain" slightly less. This is the noise floor: in real high-dimensional data the CEV curve approaches $1$ only slowly through a long flat tail.
4. Limit. As you append $m$ noise eigenvalues each $\epsilon$, ${\text{CEV}}_3=\frac{10}{10+m\epsilon }\to 0$ as $m\epsilon \to \infty$, but for realistic small totals it stays near $1$. Why this step? It pins down the asymptotic behaviour — showing that "explains almost everything" can silently erode as junk dimensions pile up, which is exactly the trap the parent note warns about.

Read the figure below left-to-right: the leftmost point is ${\text{CEV}}_0=0$ (amber arrow, "keep nothing"), the curve jumps steeply over the three real components, then crawls almost flat across the 30 tiny noise eigenvalues — the shaded band near the top is that noise floor. The amber arrow at $k=3$ marks where the real signal ends at $0.971$; every point to its right is noise dragging the total slowly toward $1$. The x-axis is components kept, the y-axis is cumulative explained variance.

Verify: ${\text{CEV}}_0=0$, ${\text{CEV}}_3=1.0$ (no noise), and with noise $10/10.3=0.970874$. Monotone, bounded in $[0,1]$, endpoints exact. ✓

[!recall]- Self-check (reveal after you answer)

[!recall]- Case-matrix coverage checklist