Parameter-efficient fine-tuning (PEFT) — Hinglish notes

4.3.7 · AI-ML › Pretraining & Fine-Tuning LLMs

PEFT exist kyun karta hai?

Ek 7B-parameter model ki full fine-tuning ka matlab hai:

Saare 7B params ke liye gradients store karna.
Optimizer state store karna (Adam 2 moments rakhta hai) → ~2× zyada.
Har task ke liye ek alag 14 GB checkpoint save karna.

PEFT ki main families KAUNSI hain?

Hum LoRA (workhorse) par focus karenge aur use poori tarah derive karenge.

LoRA — Scratch se Derivation

Setup. Ek linear layer $h = W x$ compute karta hai, jahan $W \in R^{d \times k}$ frozen hai.

Step 1 — Fine-tuning kya chahta hai. Fine-tuning $W$ ko $W + Δ W$ mein badal deti hai, toh $h = (W + Δ W) x = W x + Δ W x .$ Yeh step kyun? Koi bhi adaptation frozen weights mein ek additive correction $Δ W$ hi hai.

Step 2 — $Δ W$ ko low rank tak constrain karo. Hum assume karte hain ki $Δ W$ ka rank $\leq r$ hai jahan $r ≪ min (d, k)$ . Koi bhi rank- $r$ matrix factorize hoti hai as $Δ W = B A, B \in R^{d \times r}, A \in R^{r \times k} .$ Yeh step kyun? Intrinsic-dimension observation: useful update low rank hai, toh usse do thin matrices ke roop mein compressed store karo.

Step 3 — Parameters count karo.

Full $Δ W$ : $d \cdot k$ params.
LoRA $B, A$ : $r (d + k)$ params.

$d = k = 4096$ , $r = 8$ ke liye: full $= 16, 777, 216$ vs LoRA $= 65, 536$ → 256× kam.

Step 4 — Initialization (kyun zaroori hai). $A \sim N (0, σ^{2})$ , $B = 0$ . Toh step 0 par, $B A = 0$ , isliye $h = W x$ — model bilkul pretrained wale jaisa hi start karta hai. Training sirf corrections add karti hai, init par pehle se maujood knowledge kabhi destroy nahi karti.

Step 5 — Inference cost ZERO hai. Training ke baad tum merge kar sakte ho: $W^{'} = W + \frac{α}{r} B A$ . Ab inference ek hi matrix use karta hai — zero extra latency, adapters ke unlike jo sequential layers add karte hain.

Figure — Parameter-efficient fine-tuning (PEFT)

QLoRA (memory killer)

Yeh kyun kaam karta hai: frozen weights ko sirf forward pass ke liye read kiya jaata hai — wahan precision loss tolerable hai — jabki learning tiny adapters ke andar full precision mein hoti hai.

Worked examples

Common mistakes

Active recall

Recall Feynman: 12 saal ke bachche ko explain karo

Socho ek bahut badi textbook (pretrained model) jo already sab kuch jaanti hai. Apne history test ke liye, tum poori book dobara nahi likhte — tum bas margins mein kuch sticky notes chipkaate ho jinmein sirf woh extra bits hain jo tumhe chahiye. Book same rehti hai; sticky notes tiny hote hain aur tum unhe doosre test ke liye swap kar sakte ho. Yahi PEFT hai: giant ko frozen rakho, tiny add-ons train karo.

PEFT kaunsi problem solve karta hai?

Bade LLMs ko tasks ke liye adapt karna, bina saare parameters update karne ki memory/storage/compute cost ke.

LoRA forward equation batao.

h = W x + \frac{α}{r} B A x

, jahan

W

frozen hai aur

A, B

trainable hain.

Δ W

low rank kyun assume kiya jaata hai?

Task adaptation ki low intrinsic dimension hoti hai, isliye update ko do thin matrices mein compress kiya ja sakta hai.

LoRA full

Δ W

ke comparison mein kitne params train karta hai?

r (d + k)

d k

— bahut badi saving jab

r ≪ d, k

LoRA initialize kaise hota hai aur kyun?

B = 0

A \sim N (0, σ^{2})

taaki

B A = 0

start mein → model pretrained wale jaisa ho, koi knowledge destroy nahi.

α / r

kya karta hai?

Update ko scale karta hai taaki rank

r

change karne par uski magnitude stable rahe.

LoRA zero inference latency kyun add karta hai?

Tum

W^{'} = W + \frac{α}{r} B A

ko ek single weight matrix mein merge kar lete ho.

QLoRA kya hai?

Frozen base ko 4-bit mein load karo, uske upar higher-precision LoRA adapters train karo → ek GPU par huge models fine-tune karo.

BitFit kya hai?

Ek selective PEFT method jo sirf bias terms train karta hai.

Adapters aur LoRA mein inference par kya fark hai?

Adapters sequential layers add karte hain (latency); LoRA weights mein merge ho jaata hai (koi latency nahi).

Connections

Full Fine-Tuning — PEFT ka mehanga alternative jise yeh approximate karta hai.
Low-Rank Matrix Factorization — LoRA ka mathematical basis ( $Δ W = B A$ ).
Quantization — woh 4-bit trick jis par QLoRA rely karta hai.
Transformer Attention — LoRA typically yahan $q, v$ projections par apply hota hai.
Prompt Tuning — PEFT ki prompt-based branch.
Catastrophic Forgetting — base freeze karna + zero-init ise mitigate karta hai.