Noesis
4.3.6Pretraining & Fine-Tuning LLMs

Full fine-tuning vs feature extraction

2,171 words10 min readdifficulty · medium4 backlinks
🇮🇳 Yeh note Hinglish mein padho →

WHY does this choice even exist?

WHAT are the two strategies?

HOW: derive the parameter & gradient picture from first principles

Let the model be a composition: y^=hϕ(fθ(x))\hat{y} = h_\phi\big(f_\theta(x)\big) where fθf_\theta is the pretrained backbone (params θ\theta) and hϕh_\phi is the head (params ϕ\phi). Loss L(y^,y)\mathcal{L}(\hat{y}, y).

Gradient update for any parameter set ww: w←w−η ∂L∂ww \leftarrow w - \eta \, \frac{\partial \mathcal{L}}{\partial w}

Why this step? Gradient descent only moves parameters that receive a nonzero, applied gradient.

Feature extraction sets θ\theta as constants (requires_grad=False): ∂L∂ϕ used;∂L∂θ discarded (or never computed)\frac{\partial \mathcal{L}}{\partial \phi} \ \text{used};\qquad \frac{\partial \mathcal{L}}{\partial \theta} \ \text{discarded (or never computed)} So only ϕ\phi updates. The features fθ(x)f_\theta(x) are fixed vectors — you can even precompute them once and cache them.

Full fine-tuning applies both: θ←θ−η ∂L∂θ,ϕ←ϕ−η ∂L∂ϕ\theta \leftarrow \theta - \eta\,\frac{\partial \mathcal{L}}{\partial \theta}, \qquad \phi \leftarrow \phi - \eta\,\frac{\partial \mathcal{L}}{\partial \phi} By the chain rule the backbone gradient is ∂L∂θ=∂L∂y^ ∂hϕ∂fθ ∂fθ∂θ\frac{\partial \mathcal{L}}{\partial \theta}=\frac{\partial \mathcal{L}}{\partial \hat y}\,\frac{\partial h_\phi}{\partial f_\theta}\,\frac{\partial f_\theta}{\partial \theta} Why this step? This chain must be evaluated in full fine-tuning, which is why it needs a full backward pass through the (huge) backbone — the source of its cost.

Figure — Full fine-tuning vs feature extraction

When to pick which? (the 80/20 rule)

Situation Prefer
Tiny labeled data, task close to pretraining domain Feature extraction
Lots of data, big domain shift Full fine-tuning
Limited GPU / many tasks to serve Feature extraction or PEFT
Need max accuracy, resources available Full fine-tuning

Worked examples

Common mistakes (steel-manned)

Active recall

Recall Answer before revealing
  • What is frozen in feature extraction? ⟶ the entire backbone; only the head trains.
  • Why can you cache embeddings in feature extraction but not in full FT? ⟶ frozen backbone gives constant fθ(x)f_\theta(x); full FT keeps changing θ\theta.
  • Why use a small LR in full FT? ⟶ avoid catastrophic forgetting of pretrained features.
  • Which strategy suits tiny datasets and why? ⟶ feature extraction; few trainable params → less overfitting.
Recall Feynman: explain to a 12-year-old

Imagine you hired a chef who already knows how to cook thousands of dishes (the pretrained model). You want them to make your family's special dish.

  • Feature extraction: you don't retrain the chef at all. You just tell them, "use everything you already know, and I'll add one final instruction at the end." Cheap, safe, but limited.
  • Full fine-tuning: you let the chef relearn their whole style for your recipe — powerful, but if you only give them 2 practice tries, they might forget their old skills and mess it up. So you teach them slowly (small learning rate) and only when you have lots of practice dishes (data).

Connections

  • Transfer Learning — the umbrella idea both strategies belong to.
  • Catastrophic Forgetting — the risk full fine-tuning must manage.
  • PEFT and LoRA — modern middle-ground: freeze backbone, add tiny trainable adapters.
  • Learning Rate Schedules — warmup & discriminative LRs make full FT stable.
  • Bias-Variance Tradeoff — explains why dataset size drives the choice.
  • Layer-wise Representations in Deep Nets — why deep layers are more task-specific.
Feature extraction: what is trainable vs frozen?
Only the new head is trainable; the entire pretrained backbone is frozen.
Full fine-tuning: what is trainable?
All parameters — backbone plus head — are updated.
Why can feature extraction precompute/cache embeddings?
The frozen backbone produces constant outputs fθ(x)f_\theta(x), so they only need to be computed once.
Why is full fine-tuning more memory-hungry?
It trains all params, so it stores gradients + Adam moment states (m,vm,v) for the whole backbone, plus full backward passes.
Which strategy for a tiny dataset, and why?
Feature extraction — few trainable params act as strong regularization and prevent overfitting.
Which strategy for lots of data with domain shift, and why?
Full fine-tuning — frozen features are biased for the new domain, so adapting them reduces bias.
Why use a small learning rate in full fine-tuning?
To adapt gently and avoid catastrophic forgetting of pretrained knowledge.
What is partial fine-tuning?
Unfreezing only the top kk (deeper, more task-specific) layers while keeping shallow layers frozen.
Chain rule term unique to full FT?
∂L/∂θ=∂L/∂y^⋅∂hϕ/∂fθ⋅∂fθ/∂θ\partial\mathcal L/\partial\theta = \partial\mathcal L/\partial\hat y \cdot \partial h_\phi/\partial f_\theta \cdot \partial f_\theta/\partial\theta, requiring a full backbone backward pass.
Gotcha with frozen backbone in feature extraction?
Set it to eval() so BatchNorm stats/dropout don't shift during training.

Concept Map

provides

creates

freeze backbone

unfreeze all

trains only

risk

updates

risk

anchors

middle ground

middle ground

uses

Pretrained model

Learned features

Downstream task, small data

Flexibility vs cost dilemma

Freeze or unfreeze?

Feature extraction

Full fine-tuning

New task head

Keeps knowledge but rigid

All params via chain rule

Overfitting and forgetting

Partial FT / PEFT / LoRA

Small learning rate

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Dekho, ek pretrained model already bahut kuch seekh chuka hota hai — uske andar achhe features (representations) bane hote hain. Ab jab tumhe apne chhote task pe kaam karna ho, tumhare paas do main options hain. Pehla: feature extraction — matlab backbone ko freeze kar do (uske weights fix), aur sirf ek chhota naya head train karo. Yahan model ke internal features change nahi hote, tum unko as-is use karte ho. Isliye ye sasta hai, fast hai, aur chhote data pe overfit nahi hota.

Doosra option: full fine-tuning — poore model ke saare weights ko train hone do, lekin chhoti learning rate ke saath. Isse model apne features ko tumhare domain ke hisaab se adjust kar leta hai. Ye powerful hai, par mehnga hai (Adam ke moment states, gradients sab store karne padte hain) aur agar LR bada rakh diya to model apni purani knowledge bhool sakta hai — isko catastrophic forgetting kehte hain.

Toh choose kaise karein? Simple bias-variance soch: agar data kam hai ya task pretraining domain ke close hai → feature extraction, kyunki kam trainable params matlab kam overfitting. Agar data zyada hai aur domain kaafi alag (jaise legal ya medical text) → full fine-tuning, kyunki frozen features tumhare domain ke liye galat honge, unhe adapt karna zaroori hai. Beech ka rasta bhi hai: sirf upar ke kuch layers unfreeze karo (deeper layers zyada task-specific hote hain), ya LoRA/PEFT use karo.

Yaad rakhne ka trick: "Freeze the Features, Fine-tune the Full." Feature extraction = frozen backbone, Full FT = sab free. Aur haan, feature extraction karte time frozen backbone ko eval() mode me rakhna, warna BatchNorm/dropout ke stats hil jayenge.

Go deeper — visual, from zero

D1FoundationsEvery symbol from zero · 2,737 wordsD2Visual walkthroughThe derivation in pictures · 2,087 wordsD3Worked examplesEvery scenario covered · 3,222 wordsD4ExercisesGraded, full solutions · 2,185 wordsD5Question bankConcept traps, quick reveals · 1,571 words

Test yourself — Pretraining & Fine-Tuning LLMs

L1 RecognitionL2 RecallL3 ProductionL4 ApplicationL5 Mastery

Connections

Transfer learning and fine-tuningAI-ML · 3.4.11Catastrophic forgettingAI-ML · 4.3.12Learning rate schedulingAI-ML · 3.2.6Bias-variance tradeoffAI-ML · 2.6.1Parameter-efficient fine-tuning (PEFT)AI-ML · 4.3.7LoRA and QLoRAAI-ML · 4.3.8Adapter layers and prefix tuningAI-ML · 4.3.9Probing classifiersAI-ML · 6.3.5