Parameter-efficient fine-tuning (PEFT)

WHY does PEFT exist?

Full fine-tuning of a 7B-parameter model means:

Storing gradients for all 7B params.
Storing optimizer state (Adam keeps 2 moments) → ~2× more.
Saving a separate 14 GB checkpoint per task.

WHAT are the main PEFT families?

We focus on LoRA (the workhorse) and derive it fully.

LoRA — Derivation from scratch

Setup. A linear layer computes $h = Wx$ , where $W \in \mathbb{R}^{d\times k}$ is frozen.

Step 1 — What fine-tuning wants. Fine-tuning changes $W$ into $W + \Delta W$ , so $h = (W + \Delta W)x = Wx + \Delta W\, x.$ Why this step? Any adaptation is just an additive correction $\Delta W$ to the frozen weights.

Step 2 — Constrain $\Delta W$ to be low rank. We assume $\Delta W$ has rank $\le r$ with $r \ll \min(d,k)$ . Any rank- $r$ matrix factorizes as $\Delta W = B A,\qquad B\in\mathbb{R}^{d\times r},\; A\in\mathbb{R}^{r\times k}.$ Why this step? Intrinsic-dimension observation: the useful update is low rank, so store it compressed as two thin matrices.

Step 3 — Count the parameters.

Full $\Delta W$ : $d\cdot k$ params.
LoRA $B,A$ : $r(d+k)$ params.

For $d=k=4096$ , $r=8$ : full $=16{,}777{,}216$ vs LoRA $=65{,}536$ → 256× fewer.

Step 4 — Initialization (why it matters). $A \sim \mathcal{N}(0,\sigma^2)$ , $B = 0$ . Then at step 0, $BA = 0$ , so $h = Wx$ — the model starts identical to the pretrained one. Training only adds corrections, never destroys prior knowledge at init.

Step 5 — Inference cost is FREE. After training you can merge: $W' = W + \frac{\alpha}{r}BA$ . Now inference uses one matrix — zero extra latency, unlike adapters which add sequential layers.

Figure — Parameter-efficient fine-tuning (PEFT)

QLoRA (the memory killer)

Why it works: the frozen weights only need to be read for the forward pass — precision loss there is tolerable — while the learning happens in full precision inside the tiny adapters.

Worked examples

Common mistakes

Active recall

Recall Feynman: explain to a 12-year-old

Imagine a huge textbook (the pretrained model) that already knows almost everything. To use it for your history test, you don't rewrite the whole book — you stick a few sticky notes in the margins with just the extra bits you need. The book stays the same; the sticky notes are tiny and you can swap them for a different test. That's PEFT: keep the giant frozen, train tiny add-ons.

What problem does PEFT solve?

Adapting large LLMs to tasks without the memory/storage/compute cost of updating all parameters.

State the LoRA forward equation.

h = Wx + \frac{\alpha}{r}BA\,x

, with

W

frozen and

A,B

trainable.

Why is

\Delta W

assumed low rank?

Task adaptation has low intrinsic dimension, so the update can be compressed into two thin matrices.

How many params does LoRA train vs full

\Delta W

r(d+k)

d\,k

— a huge saving when

r \ll d,k

How is LoRA initialized and why?

B=0

A\sim\mathcal N(0,\sigma^2)

BA=0

at start → model equals the pretrained one, no knowledge destroyed.

What does

\alpha/r

do?

Scales the update so its magnitude stays stable as you change the rank

r

Why does LoRA add zero inference latency?

You merge

W'=W+\frac{\alpha}{r}BA

into a single weight matrix.

What is QLoRA?

Load frozen base in 4-bit, train higher-precision LoRA adapters on top → fine-tune huge models on one GPU.

What is BitFit?

A selective PEFT method that trains only the bias terms.

Difference between adapters and LoRA at inference?

Adapters add sequential layers (latency); LoRA merges into weights (no latency).

Connections

Full Fine-Tuning — PEFT's expensive alternative it approximates.
Low-Rank Matrix Factorization — mathematical basis of LoRA ( $\Delta W = BA$ ).
Quantization — the 4-bit trick QLoRA relies on.
Transformer Attention — LoRA is typically applied to $q,v$ projections here.
Prompt Tuning — the prompt-based PEFT branch.
Catastrophic Forgetting — freezing the base + zero-init mitigates it.

Concept Map

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Dekho, ek 7B parameter wala LLM already language ko achhe se jaanta hai. Jab hum use kisi specific task ke liye fine-tune karte hain, toh saari 7 billion weights ko badalne ki zaroorat nahi hoti — bas thoda sa nudge chahiye. Yahi PEFT ka core idea hai: base model ko freeze kar do, aur ek chhota sa naya trainable part add karke sirf usko train karo. Result: full fine-tuning jaisa accuracy, lekin sirf 1% se bhi kam parameters update karke.

Sabse popular technique LoRA hai. Idea simple hai — fine-tuning ka jo weight change $\Delta W$ chahiye, wo low rank hota hai, yaani usko do patli matrices $B$ aur $A$ se represent kiya ja sakta hai: $\Delta W = BA$ . Toh forward pass banta hai $h = Wx + \frac{\alpha}{r}BAx$ . Yahan $W$ frozen hai, sirf $A$ aur $B$ train hote hain. Initialization mein $B=0$ rakhte hain taaki shuruaat mein model bilkul pretrained wala hi rahe — knowledge kharaab na ho. Aur inference ke time $W$ mein $BA$ merge kar do, toh koi extra latency nahi.

QLoRA ek aur game-changer hai: base model ko 4-bit mein load karo (memory bachao), aur LoRA adapters higher precision mein train karo. Isse tum ek single GPU par 65B model bhi fine-tune kar sakte ho. Practical fayda: ek base model rakho, aur har task ke liye chhote-chhote adapters (kuch MB ke) plug-in ki tarah swap karo — storage aur cost dono kam.

Ek common galti: log sochte hain "zyada rank $r$ hamesha better". Nahi bhai — task update sach mein low-rank hota hai, isliye $r=8$ ya $16$ mostly kaafi hai. Zyada rank sirf memory waste aur overfitting deta hai. Yaad rakho: PEFT = giant ko freeze karo, tiny sticky-notes train karo.

WHY does PEFT exist?

WHAT are the main PEFT families?

LoRA — Derivation from scratch

QLoRA (the memory killer)

Worked examples

Common mistakes

Active recall

Connections

Concept Map

Hinglish (regional understanding)

Go deeper — visual, from zero

Test yourself — Pretraining & Fine-Tuning LLMs

Connections