Quantization (INT8, INT4, GPTQ)

WHY do we quantize at all?

WHAT problem is being solved: floats are wasteful. Weight distributions in trained nets are roughly bell-shaped and clustered near 0; we don't need 16 bits to say "roughly 0.03."

The core operation: mapping floats → integers

HOW do we choose $s$ and $z$ ? We want the range $[x_{\min}, x_{\max}]$ to map exactly onto the integer range $[q_{\min}, q_{\max}]$ (e.g. $[-128,127]$ for signed INT8).

Formula Deriving the scale and zero-point (from scratch)

We demand two endpoints line up: $x_{\min} = s\,(q_{\min}-z), \qquad x_{\max}=s\,(q_{\max}-z)$ Subtract the two equations to eliminate $z$ : $x_{\max}-x_{\min} = s\,(q_{\max}-q_{\min}) \;\Rightarrow\; \boxed{s=\dfrac{x_{\max}-x_{\min}}{q_{\max}-q_{\min}}}$ Why this step? $z$ appears identically in both, so subtraction cancels it, leaving only $s$ . Now plug $s$ back into the first equation and solve for $z$ : $z=q_{\min}-\dfrac{x_{\min}}{s}=\text{round}\!\left(q_{\min}-\frac{x_{\min}}{s}\right)$ Why round $z$ ? The zero-point must itself be a valid integer so that the real value $0$ maps exactly (important for padding/ReLU).

Granularity: one scale for how many numbers?

Figure — Quantization (INT8, INT4, GPTQ)

INT8 vs INT4

PTQ vs QAT

For LLMs, retraining is expensive, so PTQ dominates — hence GPTQ.

GPTQ — the clever part

Formula Where the Hessian comes from (derivation)

Let $E(\hat W)=\lVert (\hat W - W)X\rVert_2^2$ . Expand for a row vector error $\delta = \hat w - w$ : $E=\delta X X^\top \delta^\top.$ The second derivative (Hessian) of a quadratic $\delta A \delta^\top$ w.r.t. $\delta$ is $2A$ , so $H = 2XX^\top.$ Why this matters: $H^{-1}$ tells us how to redistribute a rounding error across the other weights to keep output unchanged. GPTQ's per-weight update is: $\delta_{\text{rest}} = -\frac{w_q - \hat w_q}{[H^{-1}]_{qq}}\;H^{-1}_{:,q}$ i.e. after fixing weight $q$ to its quantized value, push the correction $\propto H^{-1}$ into the remaining weights.

Worked examples

Worked example 1 — INT8 symmetric quantize a small weight vector

Weights $w=[-0.9,\,0.1,\,0.6]$ , signed INT8 so $q_{\max}=127$ .

$\max|w|=0.9 \Rightarrow s = 0.9/127 = 0.00709$ . Why? symmetric scale uses the largest magnitude.
$q = \text{round}(w/s) = [\text{round}(-127),\ \text{round}(14.1),\ \text{round}(84.6)] = [-127,\,14,\,85]$ .
Dequantize: $\hat w = s\cdot q = [-0.900,\,0.0993,\,0.6026]$ .
Error: $\approx[0,\,0.0007,\,0.0026]$ — tiny. Why small? 256 levels over range $[-0.9,0.9]$ gives step $\approx 0.007$ , far below the values.

Common mistakes

Flashcards

What are the scale

s

and zero-point

z

in affine quantization?

q=\text{round}(x/s)+z

;

s

sets the step size (real units per integer step),

z

is the integer that real-value 0 maps to.

Derive the quantization scale

s

Force

x_{\min}\!\to\!q_{\min}

and

x_{\max}\!\to\!q_{\max}

; subtract to cancel

z

s=(x_{\max}-x_{\min})/(q_{\max}-q_{\min})

Symmetric quantization scale for weights?

s=\max|x|/q_{\max}

, with

z=0

Why is INT4 much harder than INT8?

Only 16 levels vs 256, so the step

s

is ~16× larger → big rounding error; needs compensation (GPTQ/AWQ).

What does GPTQ minimize?

Layer output error

\lVert WX-\hat W X\rVert_2^2

, not raw weight error.

What is the Hessian in GPTQ and why?

H=2XX^\top

; its inverse tells how to redistribute each rounding error into remaining weights to keep output unchanged.

PTQ vs QAT?

PTQ quantizes a trained model without training (GPTQ, AWQ); QAT simulates quantization during training for robustness.

Per-tensor vs per-group granularity?

Per-tensor: one scale for whole matrix (cheap, worse). Per-group: one scale per block (e.g. 128 weights) — standard for INT4, handles outliers.

Memory of a 7B model in FP16 / INT8 / INT4?

14 GB / 7 GB / ~3.5–4 GB.

Why must you clamp during quantization?

Outliers push

q

past the integer range; clamping to

[q_{\min},q_{\max}]

prevents overflow.

Recall Explain to a 12-year-old (Feynman)

Imagine describing everyone's height using a ruler. FP16 is a ruler with a thousand tiny marks — super precise but bulky to carry. Quantization swaps it for a ruler with only 16 marks (INT4): lighter, but if you just snap each height to the nearest mark, some people get badly rounded. GPTQ is the clever teacher who, after rounding one kid's height up a bit, whispers to the next kids "stand a hair shorter" so that when you add the group up, the total is still right. The class fits in a small notebook, but the important totals are preserved.

Connections

Mixed-Precision Training (FP16, BF16) — the training-time cousin of quantization.
LoRA and QLoRA — QLoRA fine-tunes on top of a 4-bit (NF4) quantized base model.
Inference Optimization & KV Cache — quantization is a key memory-bandwidth win.
Hessian and Second-Order Methods — GPTQ's $H=2XX^\top$ is Optimal Brain Surgery reused.
Weight Distributions in Neural Nets — why near-zero, bell-shaped weights quantize well.
AWQ (Activation-aware Weight Quantization) — protects salient channels; sibling of GPTQ.

Concept Map

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Dekho, ek LLM ke weights normally FP16 (16-bit float) me store hote hain, jo bahut precise hai par memory bahut khata hai — 7B model ka matlab 14 GB sirf weights ke liye. Quantization ka idea simple hai: in floats ko chhote integers me convert kar do — INT8 (256 levels) ya INT4 (sirf 16 levels). Formula ka core hai $q=\text{round}(x/s)$ , jahan $s$ "scale" hai yaani ek integer step kitne real units ka hai. Scale nikalne ke liye hum bas range $[x_{\min},x_{\max}]$ ko integer range pe map karte hain, aur do endpoint equations subtract karke $s=(x_{\max}-x_{\min})/(q_{\max}-q_{\min})$ nikal aata hai.

INT8 me itne levels hain ki error almost zero rehta hai — bas simple round kar do, kaam ho gaya. Par INT4 me sirf 16 levels hain, to har weight ka rounding error bada ho jata hai (roughly 16 guna). Isliye INT4 me naive rounding se accuracy giR jati hai, aur yahin GPTQ kaam aata hai.

GPTQ ki trick genius hai: weights ko ek-ek karke quantize karo, aur jab ek weight ko grid pe snap karo to jo error aaya usko baaki bache huye weights me compensate kar do. Ye compensation Hessian $H=2XX^\top$ ke inverse ke through hota hai — matlab input correlations ke hisaab se error ko smartly distribute karta hai. Isse layer ka output almost same rehta hai, chahe har weight ab 4-bit integer ban gaya ho. Aur haan — GPTQ retraining nahi hai, ye sirf ek chhote calibration set (~128 samples) se closed-form solve karta hai, isliye fast hai.

Kyu important hai? Kyunki quantization ki wajah se hi ek bada model tumhare laptop/consumer GPU pe chal pata hai, aur inference bhi tez hota hai (kyunki inference memory-bandwidth bound hai, kam bytes = kam wait). QLoRA jaise techniques bhi isi 4-bit base pe fine-tuning karte hain. Short me: quantization = badे model ko chhota + tez banao, bina brain khoye.

WHY do we quantize at all?

The core operation: mapping floats → integers

Granularity: one scale for how many numbers?

INT8 vs INT4

PTQ vs QAT

GPTQ — the clever part

Worked examples

Common mistakes

Flashcards

Connections

Concept Map

Hinglish (regional understanding)

Go deeper — visual, from zero

Test yourself — Pretraining & Fine-Tuning LLMs

Connections