Sparse autoencoders for features: Question bank

This page hunts the conceptual landmines around Sparse autoencoders for features. Every item is a one-line reveal: read the prompt, answer out loud, then check. The answer side always gives the reason, never a bare "yes/no" — because in interpretability the reason is the whole point.

Before we start, three words must be pinned down so the questions below make sense:

Keep those pictures in mind: a long shelf of specialist detectors, almost all of them silent at any moment, a handful lit up bright.

True or false — justify

TF1. "A sparse autoencoder is just a normal autoencoder that compresses data more."

False — a normal autoencoder narrows the hidden layer to force compression; an SAE usually widens it (overcomplete) and forces silence instead. The bottleneck here is the sparsity penalty, not the layer size.

TF2. "If the hidden layer is overcomplete (

m > n

), the network can just copy the input and reconstruction is trivially perfect."

True without a sparsity penalty — that trivial copy is exactly the failure mode SAEs prevent. The sparsity term makes "just copy everything" expensive, forcing selective features. See 6.3.11-Superposition-hypothesis for why we want the extra room.

TF3. "Strong L2 (weight decay) on the weights produces sparse activations."

False — L2 shrinks every weight uniformly, giving many weak non-zero activations. Sparsity is about how many neurons are non-zero, not how small they are. See 4.1.07-L1-L2-regularization.

TF4. "L1 penalty on activations pushes small activations exactly to zero."

True — the

L_{1}

penalty

λ \sum_{j} ∣ h_{j} ∣

has a non-smooth corner at zero, so gradient descent drives sub-threshold values all the way to

0

rather than merely shrinking them.

TF5. "Increasing the sparsity weight

λ

always makes features cleaner."

False — past a point the representation starves: too few neurons survive to reconstruct the input, so features become blurry or dead. There is a sweet spot, not a monotone "more is better."

TF6. "The KL sparsity term penalizes over-active and under-active neurons symmetrically."

False —

KL (ρ ∥ \overset{ρ}{^}_{j})

is asymmetric: it punishes a neuron that fires too often far more harshly than one that is slightly too quiet, because over-active neurons dominate the code.

TF7. "

L_{0}

(counting non-zeros) is what we truly want, so we should optimize it directly."

False in practice —

L_{0}

is exactly the goal but it is non-differentiable (piecewise constant, zero gradient almost everywhere). We use

L_{1}

or KL as differentiable stand-ins.

TF8. "A dead neuron (never activates on any input) is a good outcome — it proves sparsity."

False — a permanently dead neuron carries zero information and wastes dictionary capacity. Good sparsity means each neuron is rarely active, not never active.

TF9. "SAE features are guaranteed to be human-interpretable."

False — sparsity encourages one-pattern-per-neuron but guarantees nothing. Interpretation still requires inspecting what activates each neuron; some remain uninterpretable.

Spot the error

SE1. "I set target sparsity

ρ = 0.05

, so exactly 5% of neurons fire on every single input."

Error —

ρ

is the average firing rate over the batch (

\overset{ρ}{^}_{j} = \frac{1}{N} \sum_{i} h_{j}^{(i)}

), not a per-input count. Individual inputs can fire more or fewer neurons; only the average is pinned near

0.05

SE2. "I'll use a sigmoid decoder to reconstruct the pixels, matching the sigmoid encoder."

Error — the decoder is typically linear (no activation): with a sparse

h

, reconstruction becomes a sparse linear combination of learned feature vectors (dictionary atoms). A non-linearity there breaks that clean "sum of features" reading.

SE3. "L1 penalty and L2 penalty both create true zeros; L1 is just faster."

Error — only

L_{1}

creates exact zeros.

L_{2}

shrinks values toward zero but essentially never reaches it, so it produces no true sparsity.

SE4. "The k-sparse constraint is differentiable, so we can backprop through the top-k selection normally."

Error — top-

k

selection is a hard, non-differentiable choice. It needs a straight-through estimator: hard threshold on the forward pass, identity gradient on the backward pass.

SE5. "I dropped the weight-decay term because sparsity already stops the network from cheating."

Error — without weight decay, surviving neurons can grow arbitrarily large to compensate for silenced ones, a trivial rescaling that keeps loss low while defeating the intent. The

α ∥ W ∥_{F}^{2}

term blocks this.

SE6. "Since ReLU already outputs zeros, adding an L1 penalty is redundant."

Error — ReLU zeros only the negative pre-activations; a positive-but-tiny activation stays non-zero. The L1 penalty is what pushes those small positives down to true zero and controls how many survive.

SE7. "SAE features are the same thing as principal components, just computed differently."

Error — PCA components are orthogonal and dense (every input uses all of them). SAE features are non-orthogonal, overcomplete, and sparse. See 3.4.05-PCA-ICA.

Why questions

WHY1. Why do we widen the hidden layer (overcomplete) instead of narrowing it?

A real signal may contain more distinct concepts than its raw dimension; an overcomplete dictionary gives one neuron per concept so features can disentangle instead of being crammed together (the 6.3.11-Superposition-hypothesis motivation).

WHY2. Why use KL divergence for sparsity rather than just averaging activations?

KL measures deviation from a target rate

ρ

: it is zero when

\overset{ρ}{^}_{j} = ρ

and grows asymmetrically otherwise, giving a principled, target-anchored penalty rather than a raw "smaller is better" push.

WHY3. Why is

L_{1}

called the "tightest convex relaxation of

L_{0}

Among convex penalties,

L_{1}

is the closest surrogate to the true (non-convex) count of non-zeros, and its corner at zero recovers the exact-zero behaviour that

L_{0}

wants, while staying optimizable.

WHY4. Why average

\overset{ρ}{^}_{j}

over a batch instead of using a single example?

One example can be an outlier that fires an unusual neuron; the batch average is a stable estimate of a neuron's true firing rate, so the penalty aims at a genuine tendency, not noise.

WHY5. Why does the KL gradient blow up as

\overset{ρ}{^}_{j} \to 0

\to 1

The

- ρ / \overset{ρ}{^}_{j}

and

(1 - ρ) / (1 - \overset{ρ}{^}_{j})

terms have denominators heading to zero, so the pull-back grows without bound near the extremes — this keeps neurons from getting stuck permanently silent or permanently on.

WHY6. Why prefer L1 over KL for high-dimensional dense embeddings (word2vec case)?

L1 penalizes each activation directly and needs no per-neuron rate estimate, making it simpler and cheaper when the input is already a dense high-dimensional vector.

WHY7. Why is a sparse code more interpretable than a dense one?

With few neurons active, each active neuron carries a strong, isolable signal — you can point at "neuron 42 = royalty." In a dense code every meaning is smeared across all neurons, so no single one has a clean label. Compare 6.3.05-Concept-activation-vectors.

WHY8. Why does an SAE differ from a variational autoencoder even though both regularize the latent?

A VAE regularizes the latent distribution toward a prior (usually a smooth Gaussian) for generation; an SAE regularizes the activation pattern toward sparsity for interpretability. Different targets, different geometry. See 5.2.03-Variational-autoencoders.

Edge cases

EC1. What happens to the KL penalty when a neuron's rate

\overset{ρ}{^}_{j}

hits exactly

0

The term

ρ lo g (ρ / \overset{ρ}{^}_{j})

diverges to

+ \infty

— mathematically the penalty explodes, which is why implementations clamp

\overset{ρ}{^}_{j}

away from

0

and

1

to stay numerically stable.

EC2. If

λ = 0

, what does the SAE reduce to?

A plain (over-parameterized) autoencoder with no sparsity pressure, free to learn the trivial dense copy — reconstruction is easy but features are not interpretable.

EC3. If

λ \to \infty

, what representation do you get?

The sparsity term dominates and drives essentially all neurons to zero; reconstruction collapses and the model learns almost nothing useful — the degenerate over-starved regime.

EC4. What if you set the input dimension equal to the hidden dimension (

m = n

) with a sparsity penalty?

Not overcomplete, so there is no room for a rich dictionary; sparsity now fights reconstruction directly and you lose the "many specialists, few active" benefit — features tend to entangle.

EC5. A neuron activates on every input at rate

\overset{ρ}{^}_{j} = 1

. What does the code want to do to it?

The KL gradient is strongly positive there, pushing its activation down hard; such a neuron is uninformative (it never distinguishes inputs) and must be suppressed toward the target rate

ρ

EC6. Two neurons always co-activate together and encode the same feature. Is that a sparsity violation?

Not a violation of the count per se, but it is wasteful redundancy — the overcomplete-plus-sparse pressure normally discourages it, and it signals the dictionary is duplicating a concept instead of covering a new one.

EC7. On an all-zero input, how many neurons should fire?

Ideally none (or only a learned bias effect) — an empty input has no features to detect, so a well-trained SAE should produce a near-empty code, the natural floor of sparsity.

Recall Quick self-test

Sparsity counts what — magnitudes or non-zeros? ::: Non-zeros (how many neurons fire), not how large they are. Which penalty produces exact zeros, L1 or L2? ::: L1, because of its corner at zero. Is $ρ$ a per-input count or a batch-average rate? ::: A batch-average firing rate per neuron. Why keep the decoder linear? ::: So reconstruction reads as a sparse linear combination of learned feature vectors.