Worked examples — t-SNE for visualization

This page is the worked-example workbench for t-SNE for visualization. The parent note built the machinery: the high-D neighbor probability $p_{ij}$, the low-D t-distributed $q_{ij}$, and the KL divergence cost. Here we drive that machinery through every kind of input it can meet — small numbers, big numbers, ties, zeros, degenerate cases, and real datasets — so you never hit a scenario you haven't seen.

Before anything, one reminder in plain words. A probability here is just a number between $0$ and $1$ answering "if point $i$ had to point at exactly one neighbor, how likely is it to point at $j$?" We compute these numbers from distances. The symbol $\Vert {\mathbf{x}}_i-{\mathbf{x}}_j{\Vert }^2$ means "square the straight-line distance between the two data points" — add up the squared differences of every coordinate. That's the only raw ingredient.

---

The scenario matrix

Every t-SNE question falls into one of these cells. The examples below are labelled with the cell they cover.

Cell	What makes it special	Covered by
C1 Near vs far pair	one close neighbor, one distant one → $p_{j\mid i}$ big vs tiny	Ex 1
C2 Zero distance / identical points	$\Vert {\mathbf{x}}_i-{\mathbf{x}}_j{\Vert }^2=0$ → what happens to $\exp (0)$?	Ex 2
C3 Self term $p_{i\mid i}$	the degenerate "point picks itself" case	Ex 2
C4 Perplexity = effective neighbor count	small vs large ${\sigma }_i$	Ex 3
C5 Symmetrization giving valid distribution	check ${\sum }_{ij}{p}_{ij}=1$	Ex 4
C6 Gaussian vs t-tail (crowding)	far point contribution compared	Ex 5
C7 Gradient sign: pull vs push	$p_{ij}>q_{ij}$ vs $p_{ij}<q_{ij}$	Ex 6
C8 Limiting behaviour of the gradient	distance $\to 0$ and $\to \infty$	Ex 7
C9 Real-world word problem	MNIST-style scaling / Barnes-Hut	Ex 8
C10 Exam twist: misreading the plot	global distance trap	Ex 9

---

Example 1 — C1: a near neighbor and a far neighbor

Forecast: which one gets the bigger probability, and by roughly how much — 2×? 10×? 50×?

Steps.

1. Write each unnormalized similarity $s=\exp (-\Vert {\mathbf{x}}_i-{\mathbf{x}}_{\cdot }{\Vert }^2/2{\sigma }_i^2)$. Why this step? The numerator of $p_{j\mid i}$ is exactly this Gaussian bump — it turns "distance" into "closeness", big when distance is small. $s_j=e^{-1/2}=0.6065,s_k=e^{-9/2}=0.0111$

2. Normalize by their sum so the two probabilities add to $1$. Why this step? The denominator ${\sum }_{k\ne i}$ forces the conditional distribution to be valid — point $i$ must pick some neighbor. $Z=0.6065+0.0111=0.6176$ $p_{j\mid i}=\frac{0.6065}{0.6176}=0.9820,p_{k\mid i}=\frac{0.0111}{0.6176}=0.0180$

Verify: $p_{j\mid i}+p_{k\mid i}=1$ ✓. The near point is $\approx 55\times$ more likely — the Gaussian crushes far points. This is why t-SNE is a local method.

---

Example 2 — C2 & C3: identical points and the self term

Forecast: does zero distance give probability $0$ or the maximum possible similarity?

Steps.

1. Plug distance $0$ into the Gaussian: $s_j=\exp (-0/2{\sigma }_i^2)=\exp (0)=1$. Why this step? Distance $0$ means "same location", which should mean maximum closeness — and indeed $\exp (0)=1$ is the largest value the bump reaches. Zero distance is the peak, not a hole.

2. The self term would be $s_i=\exp (-\Vert {\mathbf{x}}_i-{\mathbf{x}}_i{\Vert }^2/2{\sigma }_i^2)=\exp (0)=1$ too — the biggest of all. We set $p_{i\mid i}=0$ by hand (the sum runs over $k\ne i$). Why this step? A point picking itself as a neighbor is meaningless; if we left it in, every point would mostly point at itself and we'd learn nothing about structure.

Verify: $\exp (0)=1$ ✓, and it is the maximum of $\exp (-x)$ for $x\ge 0$ ✓. Degenerate "same point" inputs are safe: they simply get the top similarity, never a divide-by-zero.

---

Example 3 — C4: perplexity sets the neighborhood width

Forecast: with the tiny $\sigma$, will the closest neighbor grab almost all the probability, or will it be shared?

Steps.

1. Small ${\sigma }_i=0.5\Rightarrow 2{\sigma }_i^2=0.5$. Compute $s_m=e^{-d_m/0.5}$: $s=(e^{-2},e^{-8},e^{-18},e^{-32})=(0.1353,3.35\times 10^{-4},\approx 0,\approx 0)$ Normalize: $p\approx (0.9975,0.00247,\approx 0,\approx 0)$ Why this step? A narrow Gaussian only "sees" the very nearest point — this is low perplexity, hyper-local.

2. Large ${\sigma }_i=3\Rightarrow 2{\sigma }_i^2=18$. Compute $s_m=e^{-d_m/18}$: $s=(0.9460,0.8007,0.6065,0.4111)$ Sum $=2.7643$, so $p\approx (0.3422,0.2896,0.2194,0.1487)$ Why this step? A wide Gaussian spreads probability across all four — this is high perplexity, broader neighborhoods.

Verify: each distribution sums to $1$ ✓. Small $\sigma$ → one dominant neighbor; large $\sigma$ → probability spread out. This is precisely what perplexity tunes (manifold scale). Compare with the hard "assign to nearest center" rule of K-Means — t-SNE's assignment is soft.

---

Example 4 — C5: symmetrization gives a valid joint distribution

Forecast: three unordered pairs exist $\{12,13,23\}$; will their symmetrized values add to exactly $1$?

Steps.

1. Pair $(1,2)$: $p_{12}=\frac{p_{2\mid 1}+p_{1\mid 2}}{6}=\frac{0.7+0.6}{6}=0.21667$. Why this step? Averaging the two directions removes any "who chose whom" bias; dividing by $2n=6$ rescales so the whole table sums to one.

2. Pair $(1,3)$: $p_{13}=\frac{0.3+0.5}{6}=0.13333$.

3. Pair $(2,3)$: $p_{23}=\frac{0.4+0.5}{6}=0.15$.

4. The full sum counts each unordered pair twice ($p_{ij}=p_{ji}$): ${\sum }_{i\ne j}{p}_{ij}=2(0.21667+0.13333+0.15)=2(0.5)=1$

Verify: total $=1$ ✓. $P$ is a legal probability distribution — a hard prerequisite for KL divergence to even be defined.

---

Example 5 — C6: Gaussian vs t-distribution tail (the crowding fix)

Forecast: the heavy tail should keep far points "visible" — by what factor?

Steps.

1. Gaussian value: $e^{-9}=1.234\times 10^{-4}$ — essentially vanished. Why this step? A Gaussian in 2D would force moderately-far points to have almost zero similarity, so the optimizer would cram them near the center — the crowding problem.

2. t-distribution value: $(1+9{)}^{-1}=0.1$ — a thousand times larger. Why this step? The heavy tail says "you're far, but not invisible." Look at the orange curve in the figure: it stays well above the blue Gaussian for large distances. That extra weight gives clusters room to spread apart.

3. Ratio: $0.1/(1.234\times 10^{-4})\approx 810$.

Verify: ratio $\approx 810$ ✓. This single design choice is why t-SNE separates clusters cleanly where a Gaussian-Gaussian method (UMAP chooses a related but different low-D kernel) would smear them.

---

Example 6 — C7: gradient sign decides pull vs push

Forecast: $p>q$ means "should be closer than they are" — pull or push?

Steps.

1. Plug into $\frac{\partial C}{\partial {\mathbf{y}}_i}=4{\sum }_j(p_{ij}-q_{ij})({\mathbf{y}}_i-{\mathbf{y}}_j)(1+\Vert {\mathbf{y}}_i-{\mathbf{y}}_j{\Vert }^2{)}^{-1}$ for the single pair: $4(0.05-0.01)(2,0)(1+4{)}^{-1}=4(0.04)(2,0)(0.2)=(0.064,0)$ Why this step? The gradient is the direction of increasing cost; gradient descent moves opposite to it.

2. Since the gradient points in $+x$, descent moves ${\mathbf{y}}_i$ in $-x$ — toward $j$ (which lies in the $-x$ direction from $i$). Why this step? $p_{ij}>q_{ij}$ means the map has them too far apart; the update pulls them together, restoring the local neighborhood.

Verify: gradient $=(0.064,0)$, so update direction $\approx (-0.064,0)$ → toward $j$ ✓. The sign of $(p_{ij}-q_{ij})$ is the whole story: positive → attraction, negative → repulsion.

---

Example 7 — C8: limiting behaviour of the attractive force

Forecast: does the pull grow forever as points get further, or fade?

Steps.

1. As $r\to 0$: $F\approx 4(0.02)r\to 0$. Why this step? Points already on top of each other feel no force — nothing to fix. Look at the left end of the curve in the figure hugging zero.

2. As $r\to \infty$: $\frac{r}{1+r^2}\to \frac{1}{r}\to 0$. Why this step? Very distant points feel a vanishing attraction — the $1/r$ decay from the t-kernel prevents runaway pulling and keeps the optimization stable.

3. Maximum: set $\frac{d}{dr}\frac{r}{1+r^2}=\frac{1-r^2}{(1+r^2{)}^2}=0\Rightarrow r=1$. $F(1)=4(0.02)\cdot \frac{1}{2}=0.04$ Why this step? The force peaks at moderate separation — exactly where a cluster needs the most tidying.

Verify: $F(1)=0.04$, and $F\to 0$ at both ends ✓ (peak at $r=1$, the red dot in the figure). Both degenerate limits are graceful: no blow-up, no divide-by-zero.

---

Example 8 — C9: real-world scaling word problem (MNIST)

Forecast: billions of pairs — is exact even feasible?

Steps.

1. Number of unordered pairs $=\frac{n(n-1)}{2}=\frac{60000\cdot 59999}{2}=1,799,970,000$ — about $1.8$ billion. Why this step? Every pair contributes a $p_{ij}$ term; this is why exact t-SNE chokes on large $n$.

2. Speedup $=\frac{n^2}{n{\log }_{2}n}=\frac{n}{{\log }_{2}n}=\frac{60000}{{\log }_{2}60000}$. Now ${\log }_{2}60000=\frac{\ln 60000}{\ln 2}\approx \frac{11.0021}{0.6931}\approx 15.87$. $\text{speedup}\approx \frac{60000}{15.87}\approx 3781\times$ Why this step? Barnes-Hut approximates the many-far-away forces as a single lumped force, turning quadratic into near-linear work — the reason large-scale t-SNE is practical.

Verify: pairs $=1.79997\times 10^9$ ✓; speedup $\approx 3781\times$ ✓. This is the trade discussed in the parent's comparison table; if you need a parametric map for new points instead, reach for autoencoders.

---

Example 9 — C10: exam twist — reading the plot correctly

Forecast: trap or truth?

Steps.

1. Recall the cost is $\text{KL}(P\Vert Q)$ which heavily penalizes breaking local structure but is nearly blind to how far apart two separated clusters land. Why this step? The between-cluster gaps are not calibrated to any real distance; the optimizer only cared about keeping neighbors together.

2. Therefore the numbers $3$ and $10$ carry no metric meaning. The valid readings are: A, B, C are distinct clusters; within-cluster tightness and cluster existence are trustworthy; global spacing is not. Why this step? This is the single most common misuse of t-SNE, straight from the parent's "Mistake 1".

Verify: conclusion is invalid ✓ — global inter-cluster distance in a t-SNE plot is not interpretable; only local neighborhoods and cluster separation are.

---

Recall Quick self-test

Zero distance gives unnormalized similarity of what? ::: $\exp (0)=1$, the maximum. $p_{ij}>q_{ij}$ triggers which force? ::: Attraction (pull the pair together). Where does the attractive per-pair force peak (in $r$)? ::: At $r=1$. Number of pairs for $n=60000$? ::: $1,799,970,000\approx 1.8$ billion. Is inter-cluster distance in a t-SNE plot meaningful? ::: No — only local structure is.