Worked examples — Sparse autoencoders for features

This page is the "throw everything at it" companion to Sparse autoencoders for features. The parent note built the machinery: an encoder that maps input $\mathbf{x}$ to hidden code $\mathbf{h}$, a decoder that rebuilds $\hat{\mathbf{x}}$, and a sparsity penalty that forces most entries of $\mathbf{h}$ to be zero. Here we grind through every kind of number you can feed those formulas, so no situation surprises you.

Before we compute anything, let's pin down the three tools we keep reusing, each in plain words and each anchored to a picture.

The whole game: measure ${\hat{\rho }}_j$, compare it to $\rho$, and pay this KL penalty for the gap — see the figure below.

The blue curve is $\text{KL}(\rho \Vert \hat{\rho })$ as a function of the actual rate $\hat{\rho }$, with the target fixed at $\rho =0.05$. Notice: it touches zero exactly at $\hat{\rho }=0.05$ (green dot), and it climbs much faster on the right (neuron too active) than on the left (neuron too quiet). That asymmetry is the whole point — we punish chatty neurons hard.

---

The scenario matrix

Every SAE numeric question falls into one of these cells. The examples below are labelled with the cell they hit, and together they cover all of them.

Cell	What makes it special	Example
A. Under-active neuron	${\hat{\rho }}_j<\rho$ → penalty wants MORE firing	Ex. 1
B. Over-active neuron	${\hat{\rho }}_j>\rho$ → penalty wants LESS firing	Ex. 2
C. Perfectly-tuned neuron	${\hat{\rho }}_j=\rho$ → zero penalty, zero gradient	Ex. 3
D. Degenerate: dead neuron	${\hat{\rho }}_j=0$ → KL blows up (limit)	Ex. 3
E. Degenerate: always-on	${\hat{\rho }}_j\to 1$ → KL blows up (other limit)	Ex. 3
F. L1 penalty & true zeros	ReLU sparsity, count active neurons	Ex. 4
G. k-sparse hard threshold	keep top-$k$, zero the rest	Ex. 5
H. Full loss assembly	recon + KL + weight decay together	Ex. 6
I. Real-world word problem	pick $m$, expected active count	Ex. 7
J. Exam twist: L2 ≠ sparsity	why weight decay fails	Ex. 8

---

Examples

Forecast: the neuron is barely firing, so it's below target — guess the gradient is negative (push it UP).

1. Average the activations. ${\hat{\rho }}_j=\frac{1}{4}(0.02+0.00+0.01+0.01)=0.01$. Why this step? ${\hat{\rho }}_j$ is defined as the batch mean; every downstream formula needs it.
2. Plug into KL. Using the KL formula defined above: $\text{KL}(0.05\Vert 0.01)=0.05\log \frac{0.05}{0.01}+0.95\log \frac{0.95}{0.99}\approx \mathrm{0.0413.}$ Why this step? This scalar is what gets added to ${\mathcal{L}}_{\text{sparse}}$ for this neuron.
3. Gradient (per example). Differentiating ${\mathcal{L}}_{\text{sparse}}$ from the formula box with respect to one activation $h_j^{(i)}$ gives $\frac{\partial {\mathcal{L}}_{\text{sparse}}}{\partial h_j^{(i)}}=\frac{\lambda }{N}\left(-\frac{\rho }{{\hat{\rho }}_j}+\frac{1-\rho }{1-{\hat{\rho }}_j}\right)=\frac{1}{4}\left(-\frac{0.05}{0.01}+\frac{0.95}{0.99}\right)\approx -\mathrm{1.010.}$ Why this step? The sign of this gradient tells us which direction gradient descent moves the activation.

Verify: gradient is negative ⇒ descent increases $h_j$ — exactly right for a neuron that is too quiet (${\hat{\rho }}_j=0.01<0.05$). ✓

---

Forecast: this neuron is screaming — way above target — so the penalty and a positive gradient (push DOWN) should be large.

1. Average. ${\hat{\rho }}_j=\frac{1}{4}(0.9+0.8+0.7+1.0)=0.85$. Why? Same definition; note it is huge compared to $0.05$.
2. KL. $\text{KL}(0.05\Vert 0.85)=0.05\log \frac{0.05}{0.85}+0.95\log \frac{0.95}{0.15}\approx 1.611$. Why? Compare to Example 1's $0.0413$ — the same-sized gap on the over-active side costs ~40× more. That is the asymmetry from figure s01.
3. Gradient. $\frac{1}{4}\left(-\frac{0.05}{0.85}+\frac{0.95}{0.15}\right)\approx 1.568$. Why? Sign decides direction.

Verify: gradient is positive and large ⇒ descent strongly decreases $h_j$. Chatty neuron gets suppressed hard. ✓ Contrast with Ex. 1: same $\lambda ,N$, but far bigger magnitude because ${\hat{\rho }}_j$ is near $1$.

---

Forecast: perfect ⇒ zero KL and zero gradient. The two degenerate ends ⇒ infinity.

1. Perfect (C): set $\hat{\rho }=\rho =0.05$. Then $\log \frac{\rho }{\hat{\rho }}=\log 1=0$ and $\log \frac{1-\rho }{1-\hat{\rho }}=\log 1=0$, so $\text{KL}=0$. Why? KL of a distribution with itself is always zero — the neuron is already exactly where we want it, no push.
2. Zero-gradient check (C). The gradient factor is $-\frac{\rho }{\hat{\rho }}+\frac{1-\rho }{1-\hat{\rho }}$. Substitute $\hat{\rho }=\rho =0.05$: $-\frac{0.05}{0.05}+\frac{0.95}{0.95}=-1+1=0$. Why? This proves case (C) is a genuine flat spot — training neither pushes the neuron up nor down once it sits on target.
3. Dead neuron (D): as $\hat{\rho }\to 0^{+}$, the term $\rho \log \frac{\rho }{\hat{\rho }}\to \rho \log (+\infty )=+\infty$. Why care? A neuron that never fires ($h_j=0$ always) is not "maximally sparse and great" — the KL penalty explodes, forcing the network to wake it up. This is why KL-SAEs rarely leave neurons totally dead.
4. Always-on (E): as $\hat{\rho }\to 1^{-}$, the term $(1-\rho )\log \frac{1-\rho }{1-\hat{\rho }}\to (1-\rho )\log (+\infty )=+\infty$. Why? A neuron that fires on every input carries no selective information; KL blows up on this side too.

Verify: the curve in figure s01 dips to $0$ at $0.05$ and shoots up at both edges $0$ and $1$ — matching (C), (D), (E). The gradient factor is exactly $0$ at $\hat{\rho }=0.05$. Numerically, $\text{KL}(0.05\Vert 0.001)\approx 0.145$ and rises without bound as $\hat{\rho }\to 0$. ✓

---

Forecast: ReLU kills every negative, so ~half survive; the L1 term is $\lambda$ times the sum of survivors.

1. Apply ReLU. $h=\max (0,z)=(0,0.5,0,1.2,0,0.05)$. Why ReLU? It produces exact zeros (not just small numbers), which is what "sparse" literally means — see 4.1.07-L1-L2-regularization for why L1 + a corner at zero beats L2 here.
2. Count non-zeros ($L_0$). Three entries survive ⇒ $\Vert h{\Vert }_0=3$ out of $6$. Why? Interpretability cares about the count of active features, not their size.
3. L1 penalty. $\lambda {\sum }_j{h}_j=0.1(0+0.5+0+1.2+0+0.05)=0.1\times 1.75=0.175$. Why? This scalar is added to reconstruction loss; its gradient shrinks each surviving $h_j$ toward zero.

Verify: $L_0=3$, penalty $=0.175$. Sanity: the L1 penalty only touches the 3 active neurons — the zeros contribute nothing, exactly matching the sparse intuition. ✓

---

Forecast: the two biggest are $0.9$ and $0.8$; everything else becomes zero.

1. Sort by magnitude. Descending: $0.9,0.8,0.7,0.3,0.2,0.1$. Why? Top-$k$ is defined by ranking — we must know the order.
2. Keep the top 2, zero the rest. Survivors are $0.9$ (position 2) and $0.8$ (position 6). $h^{\text{sparse}}=(0,0.9,0,0,0,0.8).$ Why? This gives exact control over sparsity: precisely $k=2$ non-zeros, no penalty tuning needed.
3. Note the non-differentiability. The "pick top-2" step has zero gradient almost everywhere, so training uses a straight-through estimator (forward = hard cut, backward = identity). Why mention? Otherwise you'd wonder how gradients flow through a ranking.

Verify: $\Vert h^{\text{sparse}}{\Vert }_0=2=k$ exactly, and the kept values ($0.9,0.8$) are the two largest of the original six. ✓

---

Forecast: small recon error, one neuron perfect (zero KL), one slightly over, tiny weight-decay term.

1. Reconstruction term. $\Vert \mathbf{x}-\hat{\mathbf{x}}{\Vert }^2=(0.1{)}^2+(-0.1{)}^2+(0{)}^2=0.02$. Why? Measures information lost — the first pillar of the objective.
2. KL term. Neuron 1: $\text{KL}(0.05\Vert 0.05)=0$. Neuron 2: $\text{KL}(0.05\Vert 0.10)=0.05\log \frac{0.05}{0.10}+0.95\log \frac{0.95}{0.90}\approx 0.01670$. Sum $\times \lambda$: $0.5\times 0.01670\approx 0.00835$. Why? Second pillar — sparsity enforcement, summed over neurons.
3. Weight decay. $\alpha \Vert W{\Vert }_F^2=0.01\times 4.0=0.04$. Why? Third pillar — stops a neuron from ballooning its weights to cheat the sparsity count.
4. Add them. ${\mathcal{L}}_{\text{SAE}}=0.02+0.00835+0.04\approx 0.06835$.

Verify: total $\approx 0.0684$. Sanity: neuron 1 (perfectly tuned, Cell C) contributes nothing to KL, confirming the zero-penalty behaviour. ✓

---

Forecast: $m$ = 8×300; active count ≈ $\rho \times m$; neuron 42's rate = 80/2000 vs 0.03.

1. (a) Dictionary size. $m=8\times 300=2400$ neurons. Why $m>n$? Overcomplete bases let each real-world concept get its own dedicated neuron — the 6.3.11-Superposition-hypothesis says features get crammed together in the original 300 dims, and the SAE un-crams them.
2. (b) Expected active count. average firing = $\rho \times m=0.03\times 2400=72$ neurons. Why? $\rho$ is the average fraction active; multiply by total neurons.
3. (c) Neuron 42's rate. ${\hat{\rho }}_{42}=80/2000=0.04$. Since $0.04>0.03$, it is over-active (Cell B) — training will nudge it down. Why? Direct comparison of empirical rate to target, same logic as Example 2.

Verify: $m=2400$, expected active $=72$, ${\hat{\rho }}_{42}=0.04>0.03$ ⇒ over-active. All three consistent. ✓ (Compare with 6.3.05-Concept-activation-vectors, where a single direction plays the role our one neuron plays here.)

---

Forecast: L2 shrink keeps four non-zeros; L1 soft-threshold kills the small ones.

Here the soft-threshold operator with cutoff $t$ is defined as $S_t(h)=\text{sign}(h)\cdot \max (0,|h|-t),$ meaning: subtract $t$ from the magnitude, and if that goes negative, snap the value to exactly $0$ (keeping the original sign otherwise). This is precisely the update L1 regularization produces — it is what puts true zeros into $h$.

1. Uniform L2-style shrink. $0.5\times h=(0.20,0.15,0.30,0.05)$. Count non-zeros: still 4. Why this shows the trap? Every value got smaller but none hit zero — this is the classic mistake from the parent note: small $\ne$ sparse.
2. L1 soft-threshold $S_{0.35}(h)$: entries with $|h|\le 0.35$ (namely $0.3$ and $0.1$) snap to $0$; the rest lose $0.35$ of magnitude, giving $(0.05,0,0.25,0)$. Count non-zeros: 2. Why the difference? L1's "corner at zero" pushes any activation below the threshold exactly to zero — see 4.1.07-L1-L2-regularization. That is genuine sparsity (a lower count), which L2 can never produce.

Verify: L2 shrink ⇒ 4 non-zeros; L1 soft-threshold ⇒ 2 non-zeros. Confirms sparsity is about counts, not magnitudes. ✓ (Unlike a VAE, which regularizes a latent distribution rather than forcing hard zeros.)

---

Recall

Recall Which direction does the KL gradient push an under-active neuron?

Under-active (${\hat{\rho }}_j<\rho$) gives a negative gradient, so descent increases its activation. ::: It wakes the quiet neuron up.

Recall Why does L2 regularization fail to make a representation sparse?

L2 shrinks all activations toward small (but non-zero) values; sparsity needs exact zeros, produced by L1's corner or a top-$k$ cut. Sparsity is about counts, not magnitudes. ::: Small ≠ sparse.

Recall What happens to the KL penalty for a totally dead neuron (

${\hat{\rho }}_j\to 0$)? It blows up to $+\infty$ — KL forbids permanently silent neurons, not just permanently loud ones. ::: Both extremes are punished.