LoRA and QLoRA

4.3.8 · AI-ML › Pretraining & Fine-Tuning LLMs

Parameter-efficient fine-tuning: ek bahut bade frozen model ko adapt karo sirf thode extra parameters train karke.

Core problem kya hai

KYUN chahiye yeh: cost, storage (ek adapter per task = few MB vs full model copies), aur speed. KYA karta hai LoRA: original weights ko freeze karta hai, ek trainable low-rank correction inject karta hai. KAISE QLoRA extend karta hai: woh frozen base ko bhi quantize karta hai 4-bit mein taaki chhote GPU pe fit ho sake.

LoRA ko scratch se derive karna

Ek linear layer compute karta hai $h = W_{0} x$ jahan $W_{0} \in R^{d \times k}$ .

Fine-tuning mein ek naya weight chahiye $W = W_{0} + Δ W$ . Poora update $Δ W$ mein $d \times k$ parameters hain — utne hi jitne $W_{0}$ mein hain.

Yeh factorization kyun? Rank $r$ ki koi bhi matrix $r$ outer products ke sum ke roop mein likhi ja sakti hai. $B A$ exactly wohi hai: $B$ ke $r$ columns times $A$ ke $r$ rows. Toh $B A$ kisi bhi rank- $\leq r$ matrix ko represent kar sakta hai, aur usse zyada kuch nahi — yahi poora point hai (hum deliberately khud ko restrict kar rahe hain).

Parameter count: $d \cdot k$ ki jagah hum sirf $r (d + k)$ train karte hain. $d = k = 4096$ , $r = 8$ ke liye: full $= 16.7$ M vs LoRA $= 65 k$ — ek 256× reduction.

$B = 0$ init pe kyun? Agar $A, B$ dono random hote, toh step 0 pe model output corrupt ho jata. $Δ W = 0$ se shuru karna matlab pehla gradient step ek pure improvement direction mein hota hai.

$r$ se divide kyun karte hain? Jaise $r$ badhate ho, $B A x$ ka sum zyada terms accumulate karta hai aur barhta hai. $r$ se divide karna normalize karta hai taaki har baar rank change karne pe learning rate re-tune na karna pade.

Inference pe magic

Training ke baad merge kar sakte ho: $W \leftarrow W_{0} + \frac{α}{r} B A$ . Ab yeh ek normal weight matrix hai — zero extra latency. Yeh LoRA ki killer feature hai adapter layers ke comparison mein (jo compute add karte hain).

QLoRA = Quantization + LoRA

QLoRA ka goal: ek 65B model ko single 48GB GPU pe fine-tune karna. Teen ingredients hain:

Key trick yeh hai: base weights 4-bit (frozen) mein store hote hain. Forward pass ke dauran unhe on the fly bf16 mein dequantize kiya jata hai sirf matmul ke liye. Sirf LoRA matrices $A, B$ full precision mein rakhe jaate hain aur gradients receive karte hain.

Frozen base ko quantize karna safe kyun hai: hum usmein backpropagate nahi kar rahe. Frozen weights pe quantization noise ek fixed, chhoti perturbation hai jiske liye adapters compensate karna seekh lete hain. Tum kabhi bhi un weights ko quantize nahi karoge jinhe precisely train kar rahe ho.

Worked examples

Common mistakes (steel-manned)

Flashcards

Weight update ke liye LoRA kaunsi low-rank factorization use karta hai?

Δ W \approx B A

jahan

B \in R^{d \times r}

A \in R^{r \times k}

r ≪ min (d, k)

Rank-

r

LoRA ek

d \times k

matrix mein kitne parameters train karta hai?

r (d + k)

instead of

d \cdot k

B

ko zero se initialize kyun karte hain?

Taaki start mein

Δ W = B A = 0

rahe; training exactly pretrained model se shuru ho aur output corrupt na ho.

\frac{α}{r} B A

mein

r

se divide kyun karte hain?

Update magnitude ko rank se roughly independent rakhta hai, taaki har rank pe learning rate re-tune na karna pade.

Merging ke baad LoRA ka inference-latency cost kya hai?

Zero —

W_{0} + \frac{α}{r} B A

ko ek matrix mein merge karo; yeh ek normal linear layer hai.

QLoRA ke teen innovations kya hain?

4-bit NormalFloat (NF4), Double Quantization, Paged Optimizers.

QLoRA mein kaunse parts quantized hain aur kaunse gradients receive karte hain?

Frozen base weights 4-bit hain; sirf full-precision LoRA adapters

A, B

gradients receive karte hain.

Frozen base ko quantize karna safe kyun hai?

Usme gradients flow nahi karte; adapters fixed quantization noise ko compensate karna seekh lete hain.

NF4 kis cheez ke liye optimal hai aur kyun?

Normally-distributed data ke liye; iske bins normal distribution ke quantiles hain, jo weight distribution se match karte hain.

α

aur

r

ko link karne ka rule of thumb kya hai?

α \approx 2 r

set karo taaki effective update strength consistent rahe.

Recall Feynman: explain to a 12-year-old

Socho ek bada finished LEGO castle hai (pretrained model). Tum use birthday party ke liye change karna chahte ho, lekin poora castle rebuild karna bahut expensive hai. Uski jagah tum ek chhoti si removable decorations set add karte ho (chhoti $A, B$ matrices) — saste banane mein, store karna easy, aur alag-alag parties ke liye swap kar sakte ho. LoRA = chhoti decorations. QLoRA = tum original castle bricks ko bhi shrink karte ho (4-bit) taaki poori cheez tumhare chhote table pe fit ho sake, phir bhi high-quality add-ons se decorate karte ho.