Adapter layers and prefix tuning — Hinglish notes

4.3.9 · AI-ML › Pretraining & Fine-Tuning LLMs

YEH EXIST KYUN KARTA HAI?

HUM KYA TRADE KARTE HAIN? Hum har weight ko freely reshape karne ki ability chhodte hain, aur bet lagate hain ki ek low-capacity add-on kaafi hai ek aise model ko specialise karne ke liye jo pehle se general features seekh chuka hai. Empirically, yeh bet zyaadatar jeet jaati hai.

Adapter Layers

YEH DERIVE KAISE HOTA HAI (first principles se)

Hum chahte hain ek aisa module jo (1) pretrained forward pass ko tod ke bina insert ho sake, (2) ke paas kam parameters hon, (3) ek useful transformation express kar sake.

Sasti capacity → factorize karo. Ek full learned map $R^{d} \to R^{d}$ mein $d^{2}$ params lagte hain. Ise ek bottleneck $r$ se force karo: down-project $d \to r$ , phir up-project $r \to d$ . Params $2 d r$ ho jaate hain. Jab $r ≪ d$ , yeh bahut chhota hai.
- Yeh step kyun? $2 d r ≪ d^{2}$ jab $r ≪ d$ . $d = 1024, r = 16$ ke liye: $2 d r = 32, 768$ vs $d^{2} \approx 1 0^{6}$ .
Nonlinearity add karo. Do stacked linears ek linear map mein collapse ho jaate hain. $σ$ insert karo taaki adapter ek nonlinear correction seekh sake.
- Kyun? $σ$ ke bina, $W_{up} W_{down}$ sirf ek rank- $r$ linear map hai — kam expressive.
Residual mein wrap karo. $W_{up} \approx 0$ initialize karo taaki shuruat mein $Adapter (h) \approx h$ ho.
- Kyun? Agar adapter near-identity se shuru hota hai, toh frozen model ka behaviour step 0 par preserve rehta hai, isliye training ek known-good point se shuru hoti hai aur pretrained knowledge destroy nahi hogi.

Prefix Tuning

YEH KAISE / KYUN KAAM KARTA HAI

Real token queries $Q$ ab extra learned key/value slots par attend karte hain. Kyunki softmax saare keys par mix karta hai, prefix har position mein task-specific information inject kar sakta hai.
Sirf $P_{k}, P_{v}$ (size $2 p d$ per layer) train hote hain; $W_{Q}, W_{K}, W_{V}$ frozen rehte hain.
$K, V$ parametrize kyun karo aur sirf embeddings prepend kyun nahi? Raw embeddings prepend karna ("prompt tuning") sirf layer 0 ko affect karta hai; har layer par directly keys/values likhna zyaada control deta hai aur zyaada stably train hota hai. (Practice mein ek chhota MLP prefix ko training ke dauran stability ke liye reparametrize karta hai.)

Adapters vs Prefix Tuning (80/20 core)

	Adapters	Prefix tuning
Params kahaan jaate hain	FFN blocks ke andar	attention K/V mein
Trainable form	$W_{down}, W_{up}$	$P_{k}, P_{v}$
Inference cost	har layer mein extra tiny FFN	lamba effective sequence
Init trick	$W_{up} \to 0$ (identity)	stability ke liye reparam MLP
Params	$\approx 2 L k d r$	$\approx 2 L p d$

Woh 20% jo aapko pata hona chahiye: dono backbone freeze karte hain; adapters ek bottleneck FFN add karte hain residual init-to-identity ke saath; prefix tuning learned key/value vectors prepend karta hai; dono ~0.1–3% weights train karte hain aur ek shared model + tiny per-task deltas dete hain.

Common mistake Galat ideas ko steel-man karna

"Prefix tuning bas real prompt words add karta hai." Kyun sahi lagta hai: words se prompting sach mein help karta hai. Fix: prefix tuning continuous vectors in K/V space at every layer seekhta hai, discrete tokens nahi — kaafi zyaada expressive aur vocabulary tak limited nahi.

"Adapters FFN sublayer ko replace karte hain." Kyun sahi lagta hai: dono FFN-shaped hain. Fix: adapters ek sublayer ke baad add hote hain residual ke saath; original sublayer abhi bhi run karta hai. Kuch bhi replace nahi hota.

"Zyaada trainable params hamesha kam se behtar hota hai, isliye full fine-tuning safer hai." Kyun sahi lagta hai: zyaada capacity = lower training loss. Fix: chhote datasets par full fine-tuning overfit karta hai aur bhool jaata hai; PEFT ki low capacity ek regularizer hai aur aksar full FT ke barabar ya behtar perform karta hai jabki store karna 100× sasta hota hai.

"Adapter $W_{up}$ ko kisi bhi layer ki tarah randomly initialize karo." Kyun: hum nets ko aisi hi init karte hain. Fix: random init step 0 par adapter ko noise injection bana deta hai, pretrained forward pass distub karta hai. Near-zero init karo taaki yeh identity se shuru ho.

Recall Feynman: ek 12-saal ke bachche ko samjhao

Socho ek bahut smart robot hai jo pehle se har kitaab padh chuka hai. Tum nahi chahte ki sirf apna homework karwaane ke liye usse sab kuch dobara sikhao. Toh tum uspe ek chhota helper gadget clip karte ho (adapter) ya usse ek secret cheat-note dete ho jo woh har jawab se pehle padhta hai (prefix). Robot ka bada brain bilkul waise ka waisa rehta hai — tum sirf chhota gadget ya chhoti note train karte ho. Yeh sasta hai, fast hai, aur tum 100 alag gadgets 100 alag kamon ke liye rakh sakte ho, sab usi ek robot pe clip karke.

Flashcards

PEFT (adapters / prefix tuning) kaunsi problem solve karta hai?

Full fine-tuning har task ke liye ek full model copy chahta hai (O(W) params); PEFT backbone freeze karta hai aur sirf ~0.1–3% nayi params train karta hai, ek shared model + tiny deltas.

Adapter forward equation likho.

Adapter (h) = h + W_{up} σ (W_{down} h)

jahaan

W_{down} \in R^{r \times d}

W_{up} \in R^{d \times r}

Adapter bottleneck dimension

r ≪ d

kyun use karta hai?

Params ko

d^{2}

(full map) se

2 d r

tak cut karne ke liye; low capacity sasta hai aur regularizer ki tarah kaam karta hai.

Adapter ko nonlinearity

σ

kyun chahiye?

Do stacked linear maps ek mein collapse ho jaate hain;

σ

adapter ko nonlinear correction seekhne deta hai (warna sirf ek rank-

r

linear map hoga).

Adapter mein

W_{up} \approx 0

initialize kyun karte hain?

Taaki step 0 par

Adapter (h) \approx h

ho (near-identity), pretrained behaviour preserve ho aur training ek known-good point se shuru ho.

L layers mein adapters ke approximate trainable params?

\approx 2 L k d r

(k adapters/layer), vocab aur sequence length se independent.

Prefix tuning mein exactly kya seekha jaata hai?

Har layer ke liye key/value prefix matrices

P_{k}, P_{v} \in R^{p \times d}

; queries real rehte hain, K aur V ko learned prefix prepend hota hai.

Prefix-tuning attention equation?

K^{'} = [P_{k}; K], V^{'} = [P_{v}; V]

, phir

softmax (Q K^{'⊤} / d) V^{'}

Prefix tuning vs prompt tuning?

Prompt tuning sirf layer-0 input embeddings edit karta hai; prefix tuning har layer par learned K/V inject karta hai — zyaada expressive/stable.

Prefix param count formula?

\approx 2 L p d

(L layers mein keys+values, prefix length p).

Steel-man: full fine-tuning PEFT se hamesha behtar kyun NAHI hota?

Chhote data par yeh overfit karta hai aur bhool jaata hai; PEFT ki low capacity regularize karta hai aur aksar FT se match karta hai ~100× lower storage par.

Length-p prefix ka length-n sequence par compute cost kya hai?

Attention

O ((n + p)^{2})

ban jaata hai — har token ke liye lamba effective sequence.

Connections

LoRA and low-rank adaptation — ek aur PEFT: inserted modules ki jagah low-rank weight deltas train karta hai.
Prompt tuning and soft prompts — prefix tuning ka layer-0-only special case.
Transformer attention mechanism — prefix tuning iske K/V edit karta hai.
Residual connections — adapter init-to-identity ke peeche ka trick.
Catastrophic forgetting — backbone freeze karna kyun help karta hai.
Full fine-tuning vs PEFT — storage/compute trade-off.