Sparse autoencoders for features

6.3.8 · AI-ML › Interpretability & Explainability

Overview

Sparse autoencoders (SAEs) ek specialized architecture hain jo interpretable feature representations seekhne ke liye design ki gayi hain, aur ye hidden layer activations mein sparsity enforce karke kaam karti hain. Standard autoencoders ke ulte jo data ko efficiently compress karte hain, SAEs ek overcomplete basis seekhne ko priority dete hain jahan kisi bhi input ke liye neurons ka sirf ek chhota subset activate hota hai, jo data ke fundamental components reveal karta hai.

Sparsity constraint network ki natural tendency se ladhti hai jo saare neurons ko weakly use karna chahti hai. Iske badle, hume strong, selective responses milte hain—jaise aisa experts hona jahan har ek sirf apne domain ke baare mein bolta hai.

Mathematical Foundation

Typically $m>n$ (overcomplete) jahan $n=\dim (\mathbf{x})$, jo model ko features ki ek rich dictionary seekhne deta hai.

Objective Function Derivation

Step 1: Reconstruction loss se shuru karo

Base objective mean squared error hai: ${\mathcal{L}}_{\text{recon}}=\frac{1}{N}{\sum }_{i=1}^N\Vert {\mathbf{x}}^{(i)}-{\hat{\mathbf{x}}}^{(i)}{\Vert }^2$

Ye step kyun? Hume autoencoder ko information preserve karna chahiye—perfect reconstruction ka matlab hai koi information loss nahi.

Step 2: Sparsity constraint add karo

$L_0$ norm (non-zero elements ki count) ideal hai lekin non-differentiable hai. Hum iske saath approximate karte hain:

${\mathcal{L}}_{\text{sparse}}=\lambda {\sum }_{j=1}^m\text{KL}(\rho \Vert {\hat{\rho }}_j)$

jahan:

• $\rho$ target sparsity hai (e.g., 0.05 matlab 5% average activation)
• ${\hat{\rho }}_j=\frac{1}{N}{\sum }_{i=1}^N{h}_j^{(i)}$ neuron $j$ ke liye empirical activation rate hai
• KL divergence: $\text{KL}(\rho \Vert {\hat{\rho }}_j)=\rho \log \frac{\rho }{{\hat{\rho }}_j}+(1-\rho )\log \frac{1-\rho }{1-{\hat{\rho }}_j}$

KL divergence kyun? Ye target sparsity se deviation penalize karne ka ek principled tarika hai. Jab ${\hat{\rho }}_j=\rho$, KL = 0 (koi penalty nahi). Jab ${\hat{\rho }}_j$ alag hota hai, to penalty asymmetrically badhti hai—un neurons ko heavily punish karti hai jo zyada baar activate hote hain.

Step 3: Complete objective

Weight decay $\alpha \Vert \mathbf{W}{\Vert }_F^2$ trivial solutions ko rokta hai jahan kuch neurons arbitrarily bade ho jaate hain doosron ki khamoshi compensate karne ke liye.

Alternative Sparsity Penalties

L1 Regularization

KL divergence ki jagah, activation magnitudes ko directly penalize karo: ${\mathcal{L}}_{\text{L1}}=\lambda {\sum }_{j=1}^m|h_j|$

Derivation insight: $L_1$ norm, $L_0$ ka tightest convex relaxation hai. Optimum par, ye exactly-zero activations encourage karta hai (unlike $L_2$ jo unhe sirf shrink karta hai).

Ye kyun kaam karta hai? $L_1$ penalty ka zero par ek "corner" hota hai. Gradient descent ke dauran, chhote activations poori tarah zero tak push ho jaate hain, true sparsity create karte hue.

k-Sparse Constraint

Hard constraint: Sirf top-$k$ activations rakhو, baaki zero karo. $h_j^{\text{sparse}}=\{\begin{array}{ll}{h}_j& \text{if }{h}_j\text{ in top-}k\\ 0& \text{otherwise}\end{array}$

Ye step kyun? Sparsity level par exact control deta hai, lekin non-differentiable hai. Straight-through estimator chahiye: forward pass hard threshold use karta hai, backward pass identity gradient use karta hai.

Worked Examples

Step-by-step:

1. Input: Patch $\mathbf{x}\in {\mathbb{R}}^{64}$ ko vectorize karo, [0,1] par normalize karo

   • Kyun? Raw pixels ka arbitrary scale hota hai; normalization training stabilize karta hai

2. Encoder: $\mathbf{h}=\sigma (W_e\mathbf{x}+{\mathbf{b}}_e)$ jahan $W_e\in {\mathbb{R}}^{256\times 64}$

   • Sigmoid kyun? Output naturally bounded [0,1] hai, "activation probability" ki interpretation se match karta hai

3. Sparsity apply karo: Batch par ${\hat{\rho }}_j=\frac{1}{N}{\sum }_i{h}_j^{(i)}$ compute karo

   • Average kyun? Single example outlier ho sakta hai; batch average stable estimate hai

4. KL penalty: $\text{KL}(\rho =0.05\Vert {\hat{\rho }}_j)$ heavily penalizes ${\hat{\rho }}_j>0.05$

   • Asymmetric kyun? Over-active neurons representation dominate karte hain; unhe suppress karna zaroori hai

5. Decoder: $\hat{\mathbf{x}}=W_d\mathbf{h}+{\mathbf{b}}_d$ (linear, koi activation nahi)

   • Linear kyun? Sparse $\mathbf{h}$ ke saath, decoder learned features ka linear combination ban jaata hai

Result: 256 neurons mein se har ek edge detector (horizontal/vertical/diagonal) ya specific location/orientation par Gabor-like filter seekhta hai. Har patch par sirf ~13 neurons activate hote hain, har ek present features detect karta hai.

Step-by-step:

1. Input: Word "king" ke liye pre-trained embedding $\mathbf{x}\in {\mathbb{R}}^{300}$

2. Encoder: $\mathbf{h}=\text{ReLU}(W_e\mathbf{x}+{\mathbf{b}}_e)$, $W_e\in {\mathbb{R}}^{1024\times 300}$

   • ReLU kyun? Natural sparsity (exactly 0 output karta hai), "feature presence" ki tarah interpretable

3. L1 penalty: $\lambda {\sum }_j{h}_j$ directly activation sum minimize karta hai

   • L1 yahan kyun? High-dimensional dense embeddings ke liye KL se simpler

4. Result: $\rho =0.03$ ke saath, ~31 neurons activate hote hain. Inspection dikhata hai:

   • Neuron 42: "royalty" (king, queen, prince → high)
   • Neuron 158: "male gender" (king, man, he → high)
   • Neuron 789: "power/authority" (king, emperor, boss → high)

Interpretable kyun? Overcomplete + sparse disentangled features force karta hai. Har neuron multiple meanings "hide" nahi kar sakta—sirf ek strong pattern sparsity pressure mein survive karta hai.

Training Dynamics

Neuron $j$ ke liye, sparsity gradient hai: $\frac{\partial {\mathcal{L}}_{\text{sparse}}}{\partial h_j^{(i)}}=\frac{\lambda }{N}\left(-\frac{\rho }{{\hat{\rho }}_j}+\frac{1-\rho }{1-{\hat{\rho }}_j}\right)$

Derivation:

1. $\text{KL}(\rho \Vert {\hat{\rho }}_j)=\rho \log \rho -\rho \log {\hat{\rho }}_j+(1-\rho )\log (1-\rho )-(1-\rho )\log (1-{\hat{\rho }}_j)$
2. $\frac{\partial \text{KL}}{\partial {\hat{\rho }}_j}=-\frac{\rho }{{\hat{\rho }}_j}+\frac{1-\rho }{1-{\hat{\rho }}_j}$
3. ${\hat{\rho }}_j=\frac{1}{N}{\sum }_i{h}_j^{(i)}$, isliye $\frac{\partial {\hat{\rho }}_j}{\partial h_j^{(i)}}=\frac{1}{N}$
4. Chain rule: $\frac{\partial \mathcal{L}}{\partial h_j^{(i)}}=\lambda \cdot \frac{\partial \text{KL}}{\partial {\hat{\rho }}_j}\cdot \frac{\partial {\hat{\rho }}_j}{\partial h_j^{(i)}}$

Interpretation: Agar ${\hat{\rho }}_j>\rho$, gradient positive hai → activation reduce karo. Agar ${\hat{\rho }}_j<\rho$, gradient negative hai → activation badhao. Magnitude ${\hat{\rho }}_j\to 0$ ya $1$ hone par badhti hai, saturation rok ke.

Common Mistakes

Ye sahi kyun lagta hai: Small weights → small activations → lagta hai kam neurons matter karte hain.

Ye galat kyun hai: L2 saare weights ko uniformly shrink karta hai. Tum har jagah weak activations paate ho, selective zero/non-zero pattern nahi. 0.01 ka weight phir bhi non-zero hai.

Fix: Sparsity ke liye activations par explicit $L_1$ ya $L_0$-like penalties chahiye, weights par nahi. Ye thresholding ya selection ke zariye true zeros create karte hain.

Steel-man: Intuition small values ko "irrelevance" se connect karta hai, jo kuch contexts mein sach hai. Lekin sparsity counts ke baare mein hai (kitne non-zero), magnitudes ke baare mein nahi (kitne bade).

Ye sahi kyun lagta hai: Zyada constraint → zyada forced specialization → clearer features.

Ye galat kyun hai: Bahut zyada sparsity representation ko starve kar deti hai. Extreme λ ke saath, zyaadatar neurons mar jaate hain (kabhi activate nahi hote), aur jo bachte hain wo input reconstruct nahi kar sakte—model trivial solution par converge karta hai.

Fix: λ ko validation reconstruction error ke zariye tune karo. Sweet spot: maximum sparsity while maintaining reasonable reconstruction (e.g., normalized images ke liye MSE < 0.01).

Steel-man: Zyada constraint interpretability mein zaroor help karta hai, lekin sirf information bottleneck tak. Usse aage tum structure nahi seekh rahe—tum sirf data model karne mein fail ho rahe ho.

Ye sahi kyun lagta hai: Standard dimensionality reduction (PCA, VAE) kam dimensions mein compress karta hai. Zyada inefficient lagta hai.

Ye galat kyun hai: Overcompleteness hi point hai. $m\gg n$ ke saath, har neuron ek narrow pattern mein specialize kar sakta hai. Sparsity ke saath combine karke (sirf kuch active), tum ek badi dictionary paate ho jahan har entry interpretable hai. Undercomplete autoencoders entangled, distributed codes seekhte hain.

Fix: Overcomplete hidden layers embrace karo (typically 2-4× input dimension). Sparsity redundancy rokti hai—zyaadatar features kisi bhi given input ke liye silent rehte hain.

Applications in AI Interpretability

1. Neural Network Dissection

CNN ke intermediate activations par SAE train karo (e.g., ResNet ki layer conv4). SAE features reveal karte hain:

• Object parts (wheels, windows, faces)
• Textures (wood grain, fabric, fur)
• Abstract concepts (symmetry, curvature)

Ye kyun kaam karta hai: CNN activations dense aur entangled hote hain. SAE unhe human-recognizable components mein disentangle karta hai.

2. Language Model Probing

Transformer hidden states par SAE apply karo. Seekhe gaye features correspond karte hain:

• Syntactic roles (subject, object, verb)
• Semantic categories (living things, tools, emotions)
• Discourse features (question markers, negation)

Use case: Ye samajhna ki har layer par kaunsi information encode hai, model editing ya debugging guide karna.

3. Mechanistic Interpretability

Recent kaam GPT-2 residual stream activations par SAEs train karta hai. Discovered features mein shaamil hain:

• "French language detector" (French text par activate hota hai)
• "Base64 encoder" (base64 reasoning ke dauran activate hota hai)
• "Python indentation" (correctly-indented code par activate hota hai)

Breakthrough: Dikhata hai ki complex language models bhi internally modular, interpretable features use karte hain.

Connections

• 6.3.01-Feature-visualization: SAE features ko activation maximization ke zariye visualize kiya ja sakta hai
• 6.3.02-Attribution-methods: SAE ground truth provide karta hai ki kaunse features ne prediction cause ki
• 6.3.05-Concept-activation-vectors: SAE features learned CAVs hain bina manual annotation ke
• 5.2.03-Variational-autoencoders: VAEs generation ke liye latent sparsity use karte hain; SAEs interpretability ke liye
• 4.1.07-L1-L2-regularization: Sparsity penalties ki mathematical foundation
• 3.4.05-PCA-ICA: ICA bhi independent components dhundhta hai, lekin linearly; SAE nonlinear hai
• 6.3.11-Superposition-hypothesis: Explain karta hai ki networks ko feature interference avoid karne ke liye sparsity kyun chahiye

Ek sparse autoencoder usi tarah seekhta hai. Apne saare "brain neurons" ko weakly use karne ki jagah, ye specialist neurons rakhna seekhta hai—ek "wheel detector" ke liye, ek "window detector" ke liye—aur sirf unhe on karta hai jo har picture ke liye zaroori hain.

"Sparse" part ka matlab hai "ek waqt mein sirf kuch use karo." "Autoencoder" part ka matlab hai ye apna input copy karne ki koshish karke seekhta hai (jaise ek LEGO car dekhna, phir use memory se rebuild karna). Sirf kuch specialists ke saath sahi se rebuild karne ke liye, har specialist ko apni ek cheez pehchaanne mein SACH MEIN achha banna padta hai.

Ye useful kyun hai? Kyunki phir humans specialists ko dekh ke samajh sakte hain ki AI kya dekh raha hai! Ye aise hai jaise AI ka dimaag kholna aur ek neat, labeled toolbox paana instead of ek messy pile ke.

Visual: Ek SPOTLIGHT (sparsity) imagine karo ek andhere theater (overcomplete space) mein, jo sirf un actors (features) ko roshan karta hai jo current scene (input) ke liye relevant hain.

#flashcards/ai-ml

Ek sparse autoencoder ka primary goal ek standard autoencoder se kya alag hai? :: Hidden layer mein sparsity enforce karke interpretable, disentangled features seekhna, pure compression efficiency ki jagah explainability ko priority dena.

Ek single neuron j ke liye KL divergence sparsity penalty formula likho :: $\text{KL}(\rho \Vert {\hat{\rho }}_j)=\rho \log \frac{\rho }{{\hat{\rho }}_j}+(1-\rho )\log \frac{1-\rho }{1-{\hat{\rho }}_j}$ jahan $\rho$ target sparsity hai aur ${\hat{\rho }}_j$ empirical average activation hai.

Do real-world applications batao jahan SAEs AI interpretability improve karte hain :: 1) Neural network dissection: CNN layers mein object parts aur textures reveal karna. 2) Language model probing: transformer hidden states mein syntactic/semantic features discover karna.

Concept Map