Visual walkthrough — Sparse autoencoders for features

This page rebuilds the central result of the parent note — the loss function ${\mathcal{L}}_{\text{SAE}}$ — one picture at a time, from nothing. We assume you know only what a number and a picture are. Every symbol is earned before it is used.

---

Step 1 — What is an autoencoder? (a squeeze-and-rebuild pipe)

WHAT. Imagine a number that describes something — say the brightness values of a tiny image. We call that list of numbers a vector and write it $\mathbf{x}$ (bold = "this is a whole list, not one number"). The autoencoder does two things in a row: it encodes $\mathbf{x}$ into an internal list $\mathbf{h}$, then decodes $\mathbf{h}$ back into a guess $\hat{\mathbf{x}}$ (the little hat means "my reconstruction of").

WHY. If the machine can rebuild $\mathbf{x}$ from the smaller-or-differently-shaped $\mathbf{h}$, then $\mathbf{h}$ must contain the important information. So $\mathbf{h}$ becomes a re-description of the input in the machine's own vocabulary — and that vocabulary is what we want to read.

PICTURE. Follow the arrow left to right: input $\mathbf{x}$ → encoder box → hidden list $\mathbf{h}$ → decoder box → rebuilt $\hat{\mathbf{x}}$.

The hidden list $\mathbf{h}$ lives in ${\mathbb{R}}^m$ — meaning it is a list of $m$ real numbers. The input lives in ${\mathbb{R}}^n$. When $m>n$ we call the machine overcomplete: it has more lightbulbs than input numbers, so it has room to give each pattern its own dedicated bulb. Compare this room-to-spread idea with superposition, which explains why the real network crammed many features into few neurons in the first place.

---

Step 2 — First rule: the rebuild must match (reconstruction loss)

WHAT. We need a single number that says "how wrong is $\hat{\mathbf{x}}$?" We take each rebuilt value minus its true value, square it (so over- and under-shoots both count as positive), and add them up. Averaging over many examples gives the reconstruction loss.

WHY. Squaring, not just subtracting, does two jobs: it removes the sign (a miss is a miss either way) and it punishes big misses far more than small ones. This is exactly the "distance between two points" idea — the length of the error arrow.

PICTURE. The teal dots are true pixels $\mathbf{x}$; the orange dots are the rebuild $\hat{\mathbf{x}}$. Each red segment is one error; its squared length is what we sum.

If we stopped here, the machine would happily light up every bulb a little to copy the input — a perfect photocopier that tells us nothing. That is the problem the next steps solve.

---

Step 3 — Second rule: measure how busy each bulb is (${\hat{\rho }}_j$)

WHAT. For one bulb, call it bulb $j$, look at how strongly it fires across many inputs and take the average. Call that average the activation rate ${\hat{\rho }}_j$ (the hat again = "measured from data", the Greek letter rho $\rho$ is just the name we chose).

WHY. A single input might be a fluke. We care about the bulb's habit: does it fire for almost everything (a lazy, uninformative bulb) or only for rare, specific patterns (a useful specialist)? Averaging over a batch turns a noisy signal into a stable habit-number.

PICTURE. Two bulbs shown across 12 inputs. The plum bulb fires constantly ($\hat{\rho }$ high); the teal bulb fires rarely ($\hat{\rho }$ low). We want the teal habit.

---

Step 4 — Why the bulb output sits in $[0,1]$ (the sigmoid choice)

WHAT. The bending function $f$ from Step 1 is the sigmoid $\sigma$. It takes any number and squashes it into the range 0 to 1.

WHY THIS TOOL AND NOT ANOTHER. We wanted ${\hat{\rho }}_j$ to read as "fraction of the time on." That interpretation only makes sense if each firing is itself between 0 (off) and 1 (fully on). The sigmoid guarantees this — no output can escape $[0,1]$ — so averaging them gives an honest probability-like rate. (The ReLU in the parent's word-embedding example makes a different trade: it gives exact zeros but unbounded highs, which pairs with L1 in Step 7 instead of KL.)

PICTURE. The sigmoid S-curve: huge negatives → 0, huge positives → 1, zero → 0.5. Note it never reaches 0 or 1 exactly, only approaches — this matters in Step 6.

$\sigma (z)=\frac{1}{1+e^{-z}},0<\sigma (z)<1\text{ for every finite }z.$

Here $e^{-z}$ is the exponential curve; all we need from it is: it is always positive, so the denominator is always bigger than 1 — forcing the output strictly between 0 and 1.

---

Step 5 — Comparing two "on-rates": why KL divergence

WHAT. We pick a target on-rate $\rho$ (a small chosen number, e.g. $0.05$ = "on 5% of the time"). We need a penalty that is zero when the measured ${\hat{\rho }}_j$ equals the target, and grows as it drifts away. The tool for comparing two rates like this is the KL divergence.

WHY THIS TOOL. We could just use $(\rho -{\hat{\rho }}_j{)}^2$, but that treats "twice too chatty" and "twice too quiet" symmetrically — it doesn't know these are rates bounded in $[0,1]$. KL is built for comparing two on/off probabilities: it blows up correctly near the forbidden edges (0 and 1) and asymmetrically punishes a bulb that is too active harder — exactly the direction we care about.

PICTURE. Penalty height vs measured ${\hat{\rho }}_j$, with the valley (zero penalty) sitting right at ${\hat{\rho }}_j=\rho =0.05$. Notice the wall on the right side is steep: over-firing is expensive.

---

Step 6 — Degenerate cases: what happens at the edges

WHAT / WHY. A good derivation must survive its extremes. Look at the two dangerous values of ${\hat{\rho }}_j$:

• ${\hat{\rho }}_j\to 0$ (a bulb that never fires): the term $\rho \log \frac{\rho }{{\hat{\rho }}_j}$ has ${\hat{\rho }}_j$ in a denominator inside a log → it shoots to $+\infty$. Good: a totally dead bulb is punished, so the network can't cheat by killing bulbs entirely — it must keep them available at rate $\rho$.
• ${\hat{\rho }}_j\to 1$ (a bulb always on): now $(1-\rho )\log \frac{1-\rho }{1-{\hat{\rho }}_j}$ has $1-{\hat{\rho }}_j\to 0$ in its denominator → $+\infty$ again. The always-on lazy bulb is also punished hard.

This is precisely why Step 4's sigmoid never actually reaches 0 or 1: if it could, the penalty would be literally infinite and gradients would explode. Staying strictly inside $(0,1)$ keeps everything finite.

PICTURE. The same KL curve as Step 5 but zoomed to show both walls rising to infinity at 0 and 1, with the safe valley in between.

---

Step 7 — Introducing the sparsity dial $\lambda$ and the L1 alternative

WHAT. So far we have two competing wishes: rebuild faithfully (Step 2) and stay sparse (Steps 5–6). We need a single knob that says how much we care about sparsity relative to reconstruction. That knob is the sparsity weight $\lambda$: a chosen non-negative number we multiply the sparsity penalty by before adding it to the reconstruction loss. $\lambda =0$ means "ignore sparsity, just copy"; large $\lambda$ means "sparsity matters more than accuracy". It is dimensionless in spirit — its only job is to trade off two loss numbers, so its right value is found by trying a few and watching the features.

WHY a dial at all. The two wishes are in tension: perfect reconstruction wants many bulbs on, sparsity wants few. $\lambda$ sets where on that tug-of-war we settle. Without it, the two terms would be added blindly with whatever accidental scale each happens to have.

The L1 alternative. Instead of matching a target rate, we can just add up the raw activation sizes and penalise the total: $\lambda {\sum }_j|h_j|$. This is L1 regularization on the activations — the same dial $\lambda$ now scaling a simpler penalty.

WHY THIS TOOL. The truly ideal penalty counts non-zero bulbs — the $L_0$ "count" — but counting is a step function: flat, no slope, so gradient descent gets no signal. L1 is the closest smooth-enough stand-in. Its graph is a "V" with a sharp corner at zero; that corner keeps pushing small activations all the way to exactly 0, unlike L2's rounded bowl which only shrinks them toward 0 but never quite there.

PICTURE. Three penalty shapes side by side over one activation value: L0 (flat steps, no gradient), L1 (sharp V — pulls to a true zero), L2 (smooth U — only shrinks). See the L1 corner pinning the value at 0.

Compare with PCA/ICA (which find dense directions) and variational autoencoders (which regularise the code's distribution instead of its sparsity).

---

Step 8 — Assemble the full loss (add the three rules)

WHAT. Stack the three ingredients into one number to minimise: rebuild faithfully (Step 2), stay sparse (Steps 5–7, dialed by $\lambda$), and don't let weights explode.

WHY the third term. Without a leash on weights, the machine could make a few bulbs enormous to compensate for the silenced ones — technically sparse, but numerically unstable. A gentle L2 penalty $\alpha \Vert \mathbf{W}{\Vert }_F^2$ (its own dial $\alpha$, the "weight-decay strength") keeps them tame.

WHY the term-scaling matters. Notice the reconstruction term is averaged over the $N$ examples but the sparsity term is a sum over the $m$ bulbs, and ${\hat{\rho }}_j$ inside it is already a per-batch average over $N$. So neither $\lambda$ nor $\alpha$ carry a hidden $\frac{1}{N}$ — but this means the right $\lambda$ depends on your batch/dataset size and on $m$. If you double the batch, ${\hat{\rho }}_j$ barely moves (it's already an average), so $\lambda$ is stable in $N$; but if you change $m$ (more bulbs), the summed penalty grows, so you must re-tune $\lambda$ (or divide the sparsity sum by $m$ to make it a per-bulb average). Keeping every term on a comparable "per-unit" scale is what lets one $\lambda$ transfer across runs.

PICTURE. Three colored tanks feeding one output valve labelled ${\mathcal{L}}_{\text{SAE}}$ — the loss the optimiser drives down.

---

Step 9 — Which way does each bulb get pushed? (deriving the gradient)

WHAT. During training we adjust each bulb using the slope of the sparsity penalty with respect to its firing $h_j^{(i)}$. Let us actually derive that slope, step by step, rather than quote it.

WHY. The sign of this slope tells the optimiser the direction: turn this bulb down, or up. That single sign is the whole training story — so it is worth seeing exactly where it comes from.

The chain, one link at a time. The firing $h_j^{(i)}$ affects the loss only through the average ${\hat{\rho }}_j$, which in turn sits inside the KL. So we walk two links: first $\text{KL}$ depends on ${\hat{\rho }}_j$, then ${\hat{\rho }}_j$ depends on $h_j^{(i)}$.

$\text{Link A:}\frac{\partial \text{KL}(\rho \Vert {\hat{\rho }}_j)}{\partial {\hat{\rho }}_j}=\underset{\text{from }\rho \log \frac{\rho }{{\hat{\rho }}_j}}{\underbrace{-\frac{\rho }{{\hat{\rho }}_j}}}+\underset{\text{from }(1-\rho )\log \frac{1-\rho }{1-{\hat{\rho }}_j}}{\underbrace{\frac{1-\rho }{1-{\hat{\rho }}_j}}}$

Each piece is just "derivative of $\log$": $\frac{d}{du}\log \frac{c}{u}=-\frac{1}{u}$ applied with $u={\hat{\rho }}_j$ (first term, times the constant $\rho$) and with $u=1-{\hat{\rho }}_j$ (second term — the extra minus from the inner $1-{\hat{\rho }}_j$ flips the sign back to positive, times $1-\rho$).

$\text{Link B:}{\hat{\rho }}_j=\frac{1}{N}{\sum }_{i=1}^N{h}_j^{(i)}\Rightarrow \frac{\partial {\hat{\rho }}_j}{\partial h_j^{(i)}}=\frac{1}{N}$

because $h_j^{(i)}$ appears in the sum exactly once, divided by $N$.

Multiply the links (chain rule) and carry the overall dial $\lambda$ from Step 7:

PICTURE. Number line of ${\hat{\rho }}_j$: to the right of $\rho$ the arrow points "reduce firing"; to the left it points "increase firing"; at $\rho$ the arrow vanishes (equilibrium).

Feeling which real features these settled bulbs represent is the job of attribution methods and concept activation vectors.

---

The one-picture summary

The whole page in one frame: input flows through the encoder into a row of bulbs where only a few glow (sparsity), then the linear decoder rebuilds the input; three tugs (rebuild-error, KL-sparsity dialed by $\lambda$, weight-leash dialed by $\alpha$) balance into the single number ${\mathcal{L}}_{\text{SAE}}$.

Recall Feynman retelling — say it in plain words

We built a machine that squeezes an input into a row of little lightbulbs and then rebuilds the input from them. The rebuild step is linear — just adding up the patterns of the bulbs that are on — so that each lit bulb cleanly contributes one learned feature. Rule one: the rebuild must match the original — we measure the total squared length of the error arrows and try to make it small. Rule two: only a few bulbs may glow at once. To enforce that we watch each bulb's habit — the fraction of inputs it fires for — and compare that habit to a small target like 5%. The comparison tool is KL divergence, which is zero when the habit hits the target and grows (to infinity!) if a bulb is dead or always-on — which is exactly why we squash firings into 0-to-1 with a sigmoid so they never touch the infinite edges. A simpler alternative, L1, just adds up firing sizes; its sharp corner at zero snaps tiny firings to exactly off. We add a gentle weight leash on both weight matrices so no bulb grows monstrous. A single dial $\lambda$ sets how much we care about sparsity versus accuracy, and because the terms are scaled differently (averaged reconstruction, summed sparsity), the right $\lambda$ depends on batch size and bulb count — so we keep each term "per-unit" to make it transferable. Stack the three rules into one loss. Finally, we derived the slope of the sparsity rule by walking two chain-rule links — KL through ${\hat{\rho }}_j$, then ${\hat{\rho }}_j$ through the firing — and found a beautifully simple sign: too chatty → turn down, too quiet → turn up, just right → leave alone. Minimise the whole thing and each surviving bulb becomes a specialist detector — a readable feature.

Recall Quick self-check

Why must the decoder $g$ be linear (identity) in an SAE? ::: So the rebuild is a clean sum of decoder columns — one per lit bulb — letting each feature be read as one column of $W_d$; a squashing $g$ would bend and cap that sum and blur the feature-per-column interpretation. Why can't strong L2 weight decay create sparsity? ::: L2 shrinks all weights uniformly toward small-but-nonzero; sparsity is about the count of active bulbs, needing an activation penalty (KL or L1) that produces true zeros. What does $\lambda$ control, and why does its best value depend on $m$? ::: $\lambda$ trades sparsity against reconstruction; because the sparsity term is a sum over the $m$ bulbs, adding bulbs grows that sum, so $\lambda$ must be re-tuned (or the sum averaged over $m$). What does KL$(\rho \Vert {\hat{\rho }}_j)$ equal when ${\hat{\rho }}_j=\rho$, and why? ::: Zero — both log-ratios become $\log 1=0$, so a bulb at target rate is not penalised. Why must the sigmoid never output exactly 0 or 1? ::: Because KL divergence blows up to infinity at those edges; staying inside $(0,1)$ keeps the penalty and its gradient finite. Reading the sign of the sparsity gradient, what happens to a bulb with ${\hat{\rho }}_j>\rho$? ::: The bracket is positive, so gradient descent lowers its firing — it's too chatty.