Circuits and superposition — Hinglish notes

6.3.7 · AI-ML › Interpretability & Explainability

Circuits Kya Hain?

Circuits Kyun Bante Hain: First Principles Se

Ek neural network par gradient descent se shuru karo. Loss function $L$ saare parameters $θ$ par depend karta hai, isliye:

$\frac{\partial L}{\partial θ _{i}} = \sum_{paths} \frac{\partial L}{\partial output} \cdot \frac{\partial output}{\partial θ _{i}}$

Key insight: Saare paths equally contribute nahi karte. Gradient routing ke through, sirf kuch specific weight combinations kisi task ke liye reinforce hote hain. Training ke dauran:

Specialization: Weights task-relevant paths ke saath strong connections banate hain
Pruning: Irrelevant paths ke gradients near-zero hote hain, unke weights chote rehte hain
Emergence: High-weight paths milke interpretable circuits banate hain

Worked example Example: Transformer mein Induction Circuit

Task: "When Mary and John went to the store, Mary gave a drink to..." → "John" complete karo Circuit components (layers ke across do-head composition):

Previous-token head (ek early layer, e.g. ek chhote GPT-2 mein layer 2): Current position se thik pehle wale token par attend karta hai → residual stream mein "mere pehle X token tha" likhta hai
Induction head (ek baad ki layer — empirically GPT-2 induction heads middle/deep layers mein hote hain jaise layers 5, 7, 9): Current token ki pichli occurrences ke baad wale tokens par attend karta hai → doosri "Mary" process karte waqt, ye previous-token head ka output use karke dekhta hai ki pehli "Mary" ke baad kya aaya
Copy mechanism: Induction head ka OV circuit attended token ki information ko output logits ki taraf move karta hai

Ye kyun kaam karta hai:

Step 1 (early layer): Previous-token head har position ko "mere pehle kya tha" se tag karta hai
Step 2 (baad ki layer): Doosre "Mary" par, induction head history mein pattern [X] → Mary dhundhta hai aur pehle "Mary" ke thik baad wali position par land karta hai (yaani "and John")
Step 3: "John" par attend karta hai, use prediction ki taraf copy karta hai Do heads alag-alag layers mein kyun hote hain: Induction head wo output padhta hai jo previous-token head likhta hai — ye cross-layer composition tabhi kaam karti hai jab writer pehle chale, isliye induction heads unpar depend karne wale previous-token heads se deeper hone chahiye. Verification: Identified induction heads ko ablate (zero out) karo → induction accuracy sharply gir jaati hai. Circuit causal hai, sirf correlated nahi.

Superposition Kya Hai?

Geometry Se Superposition Capacity Derive Karna

Setup: Model ke paas $n$ dimensions hain, $m$ features represent karne hain ( $m > n$ ). Har feature $i$ probability $p$ se appear hoti hai (sparsity).

Claim: Agar features sparse hain, toh superposition $m$ ko $n$ se bahut zyada exceed karne deta hai.

Derivation:

Orthogonal baseline: Har feature ke liye ek dimension dedicate karo. Capacity: exactly $n$ features. Agar features sparse hain, toh zyaatar dimensions zyaatar time zero hote hain — wasteful.
Superposition alternative: Feature $i$ ko unit vector $v_{i}$ ke roop mein represent karo, chhota overlap $∣ ⟨ v_{i}, v_{j} ⟩ ∣ \leq s$ for $i \neq = j$ tolerate karte hue (ek "almost-orthogonal" set).
Kitne almost-orthogonal vectors fit hote hain? Ye spherical-code / Johnson–Lindenstrauss question hai. Key, correct fact ye hai: $m ≲ exp (c s^{2} n)$ ek fixed maximum overlap $s$ ke liye. Yaani, jinhe aap pack kar sakte ho while overlaps $s$ se neeche rakhte ho unki sankhya $n$ mein exponentially badhti hai (fixed $s$ ke liye), linearly nahi. Isliye sirf $n$ dimensions wala ek model $n$ se bahut zyada features encode kar sakta hai.

Exponential kyun, $n / s^{2}$ nahi? $R^{n}$ mein random unit vectors ka pairwise overlap $0$ ke aas-paas concentrated hota hai standard deviation $\approx 1/ n$ ke saath. Probability ki ek random pair overlap $s$ exceed kare, $exp (- c s^{2} n)$ ki tarah decay hoti hai. $(2 m)$ pairs par union bound tab tak chhota rehta hai jab tak $m ≲ exp (c s^{2} n)$ — isliye exponential capacity.
Sparsity hi use karne layak banati hai: Kai vectors pack karna geometrically possible hai, lekin interference ke saath unhe decode karne ke liye active features kam hone chahiye. Agar ek waqt mein sirf $k \approx m p$ features active hain, toh kisi bhi feature par total interference $\sim s k$ hai (random-sign accumulation). Ise signal ( $\approx 1$ ) se neeche rakhne ke liye $p$ chhota hona chahiye — isliye sparse features → zyada usable superposition.

Formula Jab Superposition Decodable Ho (Toy Model)

Features tab tak separable rehte hain jab tak accumulated interference signal se neeche ho: $s m p ≲ SNR_{m i n}$ Tolerable feature count ke liye rearrange karo: $m ≲ \frac{SNR _{m i n}^{2}}{p s ^{2}}$

Dhyan se padho:

Interference $s k$ ke roop mein scale hoti hai $k = m p$ active features ke saath (random-walk accumulation), isliye SNR squared appear hota hai aur overlap $s$ squared appear hota hai.
Dense features ( $p \to 1$ ): interference dominate karta hai, effective capacity $n$ ki taraf collapse hoti hai (koi useful superposition nahi).
Sparse features ( $p \to 0$ ): denominator shrink hota hai, isliye decodable feature count bada ho jaata hai — heavy, usable superposition.
Geometric ceiling ( $m ≲ e^{c s^{2} n}$ ) alag hai aur bahut bada hai; practice mein sparsity, geometry nahi, binding constraint hai.

Neurons Polysemantic Kyun Bante Hain

Ek polysemantic neuron multiple unrelated features ke liye activate hota hai. Ye superposition ka ek consequence hai.

Mechanism:

Model $n$ -dimensional space mein feature vectors ${v_{1}, \dots, v_{m}}$ seekhta hai
Neuron $j$ ka activation $a_{j} = \sum_{i} f_{i} \cdot v_{ij}$ hota hai jahan $f_{i}$ feature presence hai
Agar features 3, 17, aur 42 sab neuron $j$ ke axis ke saath component rakhte hain, toh wo neuron teeno ke liye respond karta hai

Example: Ek LM neuron activate ho sakta hai:

Word "apple" ke liye (fruit feature)
Corporate tech mentions ke liye (Apple Inc. feature)
Color "red" ke liye (apple-color feature)

Teeno overlapping subspaces use karte hain → ek neuron teeno ko encode karne mein participate karta hai.

Worked example Worked Example: Toy 2D Superposition

Setup: 2 dimensions (2D space), 4 binary features store karne hain.

Feature vectors: $0°, 60°, 120°, 180°$ angles par rakho: $v_{1} = [10], v_{2} = [0.5 0.866], v_{3} = [- 0.5 0.866], v_{4} = [- 1 0]$

Sirf feature 2 ke liye activation: $a = 1 \cdot v_{2} = [0.5 0.866]$ Neuron 1, 0.5 par activate hota hai, neuron 2, 0.866 par.

Interference test: Agar features 2 aur 3 dono active hain: $a = v_{2} + v_{3} = [0 1.732]$

Kya hum decode kar sakte hain? $v_{2}$ par project karo: $a \cdot v_{2} = 0 \cdot 0.5 + 1.732 \cdot 0.866 = 1.5$ (expect 1, mila 1.5 → 50% interference).

Step-by-step kyun:

4 features store karne ke liye 2 dimensions use kiye (2× compression)
Overlap $s = cos (60°) = 0.5$ errors cause karta hai
Lekin agar features sparse hain (rarely co-occur karte hain), toh scheme kaam karta hai!

Circuits Aur Superposition Kaise Interact Karte Hain

Key relationship:

Circuits algorithm hain (computational graph)
Superposition storage format hai (features kaise pack hote hain)

Ek circuit "edges detect karo phir cat classify karo" implement kar sakta hai. "Edge" features 50 neurons mein superpose ho sakti hain, aur "cat" feature "dog" aur "fox" ke saath superpose ho sakti hai. Circuit specific linear combinations use karta hai un neurons ki taaki sahi features extract ho sakein.

Experimental Evidence

Induction heads (Anthropic, 2022): Repeating-token circuits identify kiye gaye jo previous-token heads (earlier) aur induction heads (deeper) ke cross-layer composition se bante hain. Ablation in-context copying ko strongly degrade karta hai.
Toy models mein superposition (Elhage et al., 2022): Sparse features par ReLU networks train kiye. Paya ki networks reliably dimensions se bahut zyada features superposition ke through seekhte hain jab features sufficiently sparse hoon.
Vision models mein polysemanticity (Olah et al., 2020): InceptionV1 mein neurons multiple unrelated concepts ke liye respond karte hain (e.g., "car" + "dog face"). Sparse autoencoders inhe monosemantic features mein disentangle karte hain.

Recall Ek 12-Saal Ke Bacche Ko Samjhao

Socho tumhara brain 1,000 alag types ke Pokemon yaad karna chahta hai, lekin tumhare paas sirf 100 storage boxes hain. Ek Pokemon per box nahi aa sakta!

Solution: Tum notice karte ho ki tum saare Pokemon ek saath kabhi nahi dekhte — shayad kisi bhi battle mein sirf 10 appear hote hain. Toh tum ek smart labeling system banate ho: "Box 47 thoda Pikachu, thoda Charizard, aur thoda Mewtwo store karta hai." Jab Pikachu appear hota hai, tum Box 47 (aur Boxes 12, 89) mein dekhte ho aur clues combine karke Pikachu pehchante ho. Ye hai superposition: overlap use karke apni jagah se zyada cheezein store karna (kyunki cheezein ek saath nahi aati).

Circuits teri battle strategies ki tarah hain: "Agar Pikachu dikhe, Water-type use karo." Ye wo rules hain jo tum follow karte ho. Storage trick (superposition) sirf zyada Pokemon yaad rakhne mein help karti hai taaki tumhari strategies smarter ho sakein!

Connections

6.3.01-Feature-visualization: Circuits features process karte hain; superposition unhe encode karta hai
6.3.05-Attention-head-interpretability: Attention heads Transformers mein circuit components hain
6.3.08-Concept-activation-vectors: CAVs directions probe karte hain; superposition ka matlab hai ek direction mein multiple concepts
4.2.03-Sparse-autoencoders: Superposed features ko disentangle karne ke liye use hote hain
2.1.02-ReLU-and-activation-functions: Nonlinearity interference ke bawajood feature recovery mein help karta hai (lekin superposition khud linear compression hai)

Flashcards

#flashcards/ai-ml

Neural network circuit kya hota hai?

Network ka ek subgraph (neurons + weighted connections) jo ek coherent, interpretable computational algorithm implement karta hai. Ise ablate karne se corresponding behavior causally toot jaata hai.

Neural networks mein superposition kya hai?

Jab ek model dimensions se zyada features represent karta hai features ko nearly-orthogonal (lekin perfectly orthogonal nahi) directions ke roop mein store karke. Ye fundamentally linear compression hai:

n

dimensions mein

m > n

features.

Sparsity usable superposition kyun enable karti hai?

Kyunki ek waqt mein sirf kuch features active hote hain, accumulated interference

\sim s m p

signal se neeche rehta hai, isliye features decodable rehte hain. Sparse features → zyada usable superposition.

R^{n}

mein fixed overlap

s

ke liye roughly kitne almost-orthogonal vectors fit hote hain?

Exponentially many:

m ≲ exp (c s^{2} n)

. Geometric ceiling

n

mein exponentially badhta hai, isliye ye rarely binding constraint hota hai — sparsity hoti hai.

Polysemantic neuron kya hota hai?

Ek neuron jo multiple unrelated features ke liye activate hota hai. Ye superposition ka consequence hai: multiple feature vectors ke components us neuron ke axis ke saath hote hain.

Circuit kaise identify karte hain?

(1) Behavior observe karo, (2) Hypothesize karo kaunse neurons/heads involved hain, (3) Un components ko ablate (zero out) karo, (4) Check karo ki behavior toot jaata hai ya nahi. Circuits causal necessity se define hote hain, correlation se nahi.

Transformer mein induction heads kahan hote hain, aur kyun?

Middle/deep layers mein (e.g. GPT-2 mein layers 5, 7, 9), un previous-token heads se deeper jo unhe read karte hain — cross-layer composition ke liye writer (earlier) ko reader (later) se pehle run karna hota hai.

Kya superposition ke liye nonlinear activation chahiye?

Nahi. Superposition linear compression hai (

m > n

ek linear map ke through, jaise random projections). Nonlinearity features recover/denoise karne mein help karta hai, lekin unhe superposition mein store karne ke liye zaruri nahi.

Circuits aur superposition kaise related hain?

Circuits algorithm hain (computational graph); superposition storage format hai (features kaise compress hote hain). Circuits superposed features ke linear combinations par operate karte hain.

Sirf activation magnitude se circuits kyun identify nahi kar sakte?

Kyunki (1) circuits causal paths ke baare mein hain, magnitude ke nahi, aur (2) superposition ka matlab hai same neuron multiple circuits mein participate karta hai. Causal necessity test karne ke liye ablation use karna padta hai.