Worked examples — UMAP for dimensionality reduction

This page is the case-by-case drill room for UMAP for dimensionality reduction. The parent note built the machinery (the high-D weight $w_{ij}$, the low-D weight $v_{ij}$, the ${\sigma }_i$ calibration, the cross-entropy loss). Here we make sure no scenario surprises you: we enumerate every kind of input UMAP's formulas can meet, then work each one to a number, guessing first, checking at the end.

Everything below only uses symbols the parent already earned. As a lightning refresher, the two kernels are:

---

The scenario matrix

Think of every "kind of input" as a cell. A cell is degenerate if it makes a max, a division, or a log do something extreme. We must show all of them.

#	Case class	Concrete trigger	Which formula is stressed	Example
A	Nearest neighbour itself ($d_{ij}={\rho }_i$)	$\max (0,0)=0$	high-D $w$ → exactly $1$	Ex 1
B	Duplicate / zero distance ($d_{ij}=0<{\rho }_i$)	clamp $\max (0,\text{negative})$	high-D $w$ → still $1$	Ex 1
C	Ordinary far neighbour ($d_{ij}>{\rho }_i$)	positive exponent argument	high-D $w$ decay	Ex 2
D	Bandwidth solve (find ${\sigma }_i$)	sum-to-${\log }_{2}k$ constraint	binary search on ${\sigma }_i$	Ex 3
E	Small vs large $k$ limit	$k=2$ vs $k=64$	${\log }_{2}k$ target moves	Ex 4
F	Symmetrization (directed → undirected)	fuzzy union $w+w^{\prime }-w{w}^{\prime }$	edge merge	Ex 5
G	Low-D, points coincident ($\Vert y_i-y_j\Vert =0$)	denominator $=1$	low-D $v$ → $1$	Ex 6
H	Low-D, points far apart ($\Vert \cdot \Vert \to \infty$)	denominator $\to \infty$	low-D $v$ → $0$	Ex 6
I	Loss contribution per edge	plug $w,v$ into cross-entropy	attractive vs repulsive term	Ex 7
J	Real-world word problem	scRNA-seq min_dist choice	$a,b$ refit consequence	Ex 8
K	Exam twist	"does UMAP output change if you rescale all $d_{ij}$?"	scale-invariance of ${\sigma }_i$	Ex 9

The nine examples below hit every cell A–K.

---

Forecast: Guess both weights before reading. Can a weight ever exceed $1$?

1. Compute the clamped gap for the nearest neighbour. $\max (0,d_{ij}-{\rho }_i)=\max (0,45.2-45.2)=\max (0,0)=0$. Why this step? The whole point of subtracting ${\rho }_i$ is that the closest neighbour lands exactly at the start of the decay. The max guarantees we never feed a negative number into the exponential.

2. Exponentiate. $w_{ij}=\exp (-0/18.4)=\exp (0)=1$. Why this step? $\exp (0)=1$ is the ceiling of this kernel — a weight of $1$ means "fully connected". This is UMAP's local connectivity guarantee: every point has at least one neighbour it sees with certainty, so no point is ever orphaned.

3. Now the duplicate ($d_{ij}=0$). Gap $=\max (0,0-45.2)=\max (0,-45.2)=0$, so again $w_{ij}=\exp (0)=1$. Why this step? If two samples are identical, their distance is smaller than ${\rho }_i$. Without the clamp we'd get $\exp (+45.2/18.4)\approx 11.6>1$ — an impossible "probability". The max(0,·) clamp is exactly what keeps a fuzzy membership inside $[0,1]$.

Verify: Both weights equal $1$, and $1$ is the maximum a probability can be. The clamp saved case B. ✓

---

Forecast: It's the 4th-closest neighbour in the parent's example. Bigger or smaller than $0.5$?

1. Clamped gap: $\max (0,67.8-45.2)=22.6$. Why this step? $67.8>{\rho }_i$, so this neighbour lies past the nearest one — the decay is genuinely active here.

2. Divide by bandwidth: $22.6/18.4=1.228$. Why this step? ${\sigma }_i$ sets the "reach". Dividing puts the gap in units of bandwidths: this point is $1.23$ bandwidths away.

3. Exponentiate: $w_{ij}=\exp (-1.228)=0.2926\approx 0.28$. Why this step? The exponential turns "distance in bandwidths" into a smooth fade. $1.23$ bandwidths → membership $\approx 0.29$, matching the parent's listed weight of $0.28$.

Verify: Parent note lists weights $1.00,0.71,0.49,0.28,0.17,\dots$ — our $0.28$ (rounded) sits in slot 4. Units check: the exponent argument is (distance)/(distance) = dimensionless. ✓

---

Forecast: The nearest neighbour already contributes $1$. Will the other two need to add only $\approx 0.585$?

1. Fix the first term. $w_{i1}=1$ (Example 1). So we need $w_{i2}+w_{i3}={\log }_{2}3-1=0.585$. Why this step? Isolating the guaranteed $1$ reduces the unknown to two decaying terms.

2. Write the residual as a function of $\sigma$. Gaps are $52.1-45.2=6.9$ and $58.3-45.2=13.1$: $f(\sigma )=e^{-6.9/\sigma }+e^{-13.1/\sigma }-\mathrm{0.585.}$ Why this step? $f$ is monotonically increasing in $\sigma$ (bigger $\sigma$ ⇒ slower decay ⇒ bigger weights). Monotonicity is why binary search works — there is exactly one root.

3. Bracket & bisect. $\sigma =5$: $e^{-1.38}+e^{-2.62}=0.2516+0.0728=0.324<0.585$ ⇒ too small, go higher. $\sigma =10$: $e^{-0.69}+e^{-1.31}=0.5016+0.2698=0.771>0.585$ ⇒ too big, go lower. $\sigma =7.5$: $e^{-0.92}+e^{-1.747}=0.3985+0.1742=0.573<0.585$ ⇒ slightly low. $\sigma =7.7$: $e^{-0.896}+e^{-1.701}=0.4082+0.1825=0.5907>0.585$. Root ${\sigma }_i\approx 7.63$. Why this step? Each halving keeps the root bracketed; ~20 iterations reach machine precision. We show enough to trap the answer between $7.5$ and $7.7$.

Verify: At $\sigma =7.63$: $e^{-6.9/7.63}+e^{-13.1/7.63}=e^{-0.904}+e^{-1.717}=0.405+0.180=0.585$. Adding the fixed $1$ gives $1.585={\log }_{2}3$. ✓ (Numeric check in VERIFY.)

---

Forecast: Does doubling $k$ double the target?

1. $k=2$: target $={\log }_{2}2=1$. Since the nearest neighbour alone already contributes $1$, the second neighbour's weight must sum to $\approx 0$. Why this step? This forces ${\sigma }_i$ tiny, so only the single closest point matters — maximally local, easily fragmented manifolds (matches parent's "small $k$ fragments").

2. $k=64$: target $={\log }_{2}64=6$. Now $63$ far neighbours must jointly contribute $5$, forcing ${\sigma }_i$ large. Why this step? A large ${\sigma }_i$ keeps distant points at nonzero weight — the graph reaches farther, capturing global context (parent's "large $k$ blends clusters").

3. Growth rate. From $k=2$ to $k=64$ (a $32\times$ jump), the target grew $1\to 6$, only $6\times$. The ${\log }_2$ deliberately dampens the demand. Why this step? Logarithmic growth means UMAP's "effective neighbourhood" scales gently — this is what keeps behaviour smooth as you sweep n_neighbors.

Verify: ${\log }_{2}2=1$, ${\log }_{2}64=6$, and $6/1=6\ne 32$ — sub-linear, as claimed. ✓

---

Forecast: The undirected edge — closer to the small $0.28$, the large $0.90$, or above both?

1. Apply the fuzzy union. $w_{ij}^{\text{sym}}=w_{ij}+w_{ji}-w_{ij}{w}_{ji}$. Why this step? This is the probabilistic OR: "$i$ links to $j$ or $j$ links to $i$". Two directed opinions merge into one undirected fact.

2. Plug in. $0.28+0.90-(0.28)(0.90)=1.18-0.252=0.928$. Why this step? The subtraction $-w_{ij}{w}_{ji}$ removes the double-counted overlap, keeping the result $\le 1$.

3. Sanity of magnitude. $0.928>0.90$ (both) — a union is at least as strong as its strongest member, never exceeding $1$. Why this step? Confirms the OR-semantics: agreeing evidence reinforces, never cancels.

Verify: $0.928\in [0.90,1]$ ✓, and if either weight were $1$ the union would be exactly $1$ (fully connected wins). This is the cross-entropy target $w^{\text{sym}}$ used in Ex 7.

---

Forecast: At distance $0$, is $v$ exactly $1$? At distance $10$, near $0$?

1. Coincident (case G), $r=0$. $\Vert \cdot {\Vert }^{2b}=0^{1.583}=0$, so $v=\frac{1}{1+a\cdot 0}=\frac{1}{1}=1$. Why this step? This is why min_dist matters: with nothing stopping it, two points can pile up and score the maximum membership $1$. min_dist re-fits $a,b$ to forbid this and enforce spacing.

2. Far apart (case H), $r=10$. $10^{1.583}=10^{2b}$. Since ${\log }_{10}(10^{1.583})=1.583$, this is $\approx 38.3$. Then $v=\frac{1}{1+1.929\cdot 38.3}=\frac{1}{1+73.9}=\frac{1}{74.9}=0.01335$. Why this step? The heavy polynomial tail ($1+a{r}^{2b}$ instead of $\exp$) decays slowly, so far points keep a tiny but nonzero membership — this is the anti-crowding tail the parent mentioned.

3. Limit check. As $r\to \infty$, denominator $\to \infty$, so $v\to 0$; as $r\to 0$, $v\to 1$. The kernel spans the full $[0,1]$. Why this step? A valid fuzzy membership must live in $[0,1]$ at both extremes. ✓

Verify: $v(0)=1$, $v(10)\approx 0.0133$. Monotone decreasing between. ✓ See figure.

---

Forecast: How much more expensive is placing two truly-connected points far apart?

The per-edge loss is $\ell =w\ln \frac{w}{v}+(1-w)\ln \frac{1-w}{1-v}.$

1. Good layout, $v=0.90$. $0.928\ln \frac{0.928}{0.90}+0.072\ln \frac{0.072}{0.10}=0.928(0.03065)+0.072(-0.3285)=0.02844-0.02365=0.00479$. Why this step? When $v\approx w$, both logs are near $0$ — the loss nearly vanishes. Agreement is cheap.

2. Bad layout, $v=0.05$. $0.928\ln \frac{0.928}{0.05}+0.072\ln \frac{0.072}{0.95}=0.928(2.9214)+0.072(-2.5810)=2.7111-0.1858=2.5253$. Why this step? A strongly-connected pair ($w=0.928$) placed almost apart ($v=0.05$) fires the first term hard — this is the attractive force pulling them together in SGD.

3. Compare. $2.5253\gg 0.00479$ — roughly $527\times$ costlier. Why this step? The gradient of $\ell$ w.r.t. $y_i$ is what SGD follows; a huge loss ⇒ a strong pull. This is literally how UMAP learns the layout.

Verify: Good $\approx 0.0048$, bad $\approx 2.525$, ratio $\approx 527$. Both non-negative (cross-entropy is $\ge 0$). ✓

---

Forecast: Which setting lets points pack tightly enough to reveal a small dense cluster?

1. min_dist = 0.0. The $a,b$ fit lets $v\to 1$ as $r\to 0$ (exactly Ex 6a). Points may sit on top of each other. Why this step? Rare cell types are dense — a handful of near-identical expression profiles. Allowing tight packing keeps that density visible as a compact blob.

2. min_dist = 0.8. The kernel is refit so $v$ stays low until $r\gtrsim 0.8$; short-range attraction is replaced by repulsion. Why this step? This "inflates" every cluster for readability but erases local density — the rare tight blob spreads out and can be missed.

3. Decision. For discovering tight rare clusters, choose min_dist = 0.0. Why this step? The scientific goal (density-preservation) directly dictates the hyperparameter, exactly as the parent's Example 2 concluded.

Verify: Consistent with parent note: min_dist=0.0 → dense clusters, preserves density; min_dist=0.8 → inflated, loses density. ✓ (No new number.)

---

Forecast: Intuition says "distances changed, so yes." Check the algebra.

1. Scale ${\rho }_i$ too. If all $d_{ij}\to c{d}_{ij}$, then the nearest distance ${\rho }_i\to c{\rho }_i$ automatically (it's just one of the $d$'s). Why this step? ${\rho }_i$ isn't a free parameter — it is a distance, so it rescales with everything else.

2. The bandwidth solve rescales. The constraint ${\sum }_j\exp (-\frac{c{d}_{ij}-c{\rho }_i}{{\sigma }_i})={\log }_{2}k$ is identical to the original if we let ${\sigma }_i\to c{\sigma }_i$, because $\frac{c(d_{ij}-{\rho }_i)}{c{\sigma }_i}=\frac{d_{ij}-{\rho }_i}{{\sigma }_i}$. Why this step? The binary search finds a new ${\sigma }_i$ that is $c$ times the old one, leaving every exponent argument unchanged.

3. Weights are untouched. Each $w_{ij}=\exp (-\text{same argument})$ is identical. So the fuzzy graph, the symmetrization, and the loss are all unchanged ⇒ the embedding is invariant to global rescaling. Why this step? This is the deep reason UMAP cares about relative neighbourhood geometry, not absolute units — cousin to why PCA centring matters but a global scale on all axes together does not change directions.

Verify: With $c=3$: original Ex 3 had gaps $6.9,13.1$ and $\sigma =7.63$. Scaled gaps $20.7,39.3$ with $\sigma =22.89$ give $\frac{20.7}{22.89}=0.904$ and $\frac{39.3}{22.89}=1.717$ — the same exponent arguments ⇒ same weights $0.405,0.180$. ✓ (Checked in VERIFY.)

---

Recall Quick self-test

What is $w_{ij}$ when $j$ is $i$'s nearest neighbour? ::: Exactly $1$, because $\max (0,{\rho }_i-{\rho }_i)=0$ and $\exp (0)=1$. Why the max(0,·) clamp? ::: A duplicate point ($d<{\rho }_i$) would give an exponent $>0$ and a weight $>1$ — impossible for a membership. The clamp caps it at $1$. Solving ${\sigma }_i$ uses which numerical method and why does it work? ::: Binary search — the weight-sum is monotone increasing in ${\sigma }_i$, so there is a unique root. As two low-D points coincide, $v_{ij}\to$? ::: $1$ (denominator $\to 1$). As they separate, $v_{ij}\to 0$. Does scaling all distances by $c$ change the embedding? ::: No — ${\sigma }_i$ absorbs the factor, leaving every $w_{ij}$ unchanged.

Related tools worth contrasting after this drill: Nearest Neighbors Algorithms (builds the initial graph), Spectral Clustering (supplies UMAP's initial layout), Isomap and Autoencoders (rival manifold methods), and Topological Data Analysis (the fuzzy-set theory underneath). For a friendlier language pass see the Hinglish note.