Question bank — Quantization (INT8, INT4, GPTQ)

True. Rounding a continuous float onto a finite integer grid is a many-to-one map, so distinct floats collapse to the same integer — information is destroyed. The art is making the lost part the least important.

True. With 256 levels the step size is far smaller than typical weight magnitudes, so rounding noise is negligible (<1% quality loss) and plain PTQ works out of the box.

False. Each dropped bit doubles the step $s$, but the effect on model output is nonlinear and depends on which weights are hit — that is exactly why INT4 needs GPTQ compensation while INT8 does not.

True. Symmetric mode assumes the range is centred at 0 ($x_{\max }=-x_{\min }$), so real-value 0 already lands on integer 0 with no offset needed — that is what "$z=0$" means.

False. GPTQ does no backprop over the net; it solves a closed-form layerwise least-squares using ~128 calibration samples. It is PTQ, not QAT.

Mostly false in practice — the extra scales are tiny and the arithmetic is still integer; the real cost is a slightly larger file and a per-group scale lookup, which memory-bandwidth-bound LLMs barely notice.

False. Weights are tame and near-symmetric (per-channel/per-group symmetric is fine); activations have huge per-channel outliers and need per-token schemes or outlier-preserving tricks.

False. Speedup comes because LLM inference is memory-bandwidth-bound, so moving 4× fewer bytes helps a lot — but compute, kernel overhead, and dequant steps mean the real speedup is under 4×.

True. If $z$ were fractional, the integer that real-value 0 maps to would not exist, and padding/ReLU zeros would dequantize to a nonzero value — so $z$ is rounded to a valid integer.

You forgot to clamp to $[q_{\min },q_{\max }]$. An outlier $x$ produces a $q$ outside the integer range, causing overflow/garbage. Clamping is what makes the range finite.

That is the symmetric formula. Asymmetric uses the full spread $s=(x_{\max }-x_{\min })/(q_{\max }-q_{\min })$ and a nonzero $z$; using $\max |x|$ wastes levels when the distribution is off-centre.

A single outlier inflates $\max |x|$, stretching the scale $s$ so all other weights get a coarser grid. It wastes the integer levels on empty range — the motivation for per-group scales.

It is $H=2X{X}^{\top }$; the factor of 2 comes from differentiating the quadratic $\delta X{X}^{\top }{\delta }^{\top }$ twice. The scaling matters for the correct per-weight update magnitude.

No — it minimizes the output error $\Vert WX-\hat{W}X{\Vert }^2$. Preserving what the layer does matters more than matching individual weights, which is the whole reason it beats naive rounding.

You skipped the correction step. Each snap produces an error that must be pushed into the remaining weights weighted by $H^{-1}$; omitting this makes GPTQ collapse to naive rounding.

Trained weights cluster symmetrically around 0, so a symmetric range wastes nothing and skips the zero-point add; activations are skewed with wild outliers, so they need asymmetric/per-token handling.

With only 16 levels the step is ~16× coarser than INT8, so independent rounding noise becomes large enough to shift layer outputs; GPTQ/AWQ compensate that noise instead of just tolerating it.

The layerwise objective is a quadratic in the weight error, whose curvature is set entirely by the input correlations $X{X}^{\top }$; this gives a closed-form $H^{-1}$ redistribution rule with no gradient training.

LLM decoding is dominated by streaming weights from memory; fewer bytes per weight means less traffic across the memory bus, which is the actual bottleneck — see Inference Optimization & KV Cache.

$z$ indexes an integer position on the grid, so it must be integer for real-0 to map exactly; the scale $s$ stays a float because it is a multiplier, applied at dequant time, not stored as a grid index.

Each group of ~128 weights carries its own float scale (and maybe zero-point); amortized over the group this is a small fraction of a bit per weight, which is why a real 7B INT4 file sits slightly above the naive 3.5 GB.

$\max |x|=0$ gives $s=0$, which is undefined division at dequant. Implementations set a tiny floor $s=\epsilon$ (or skip the group), so all-zero blocks stay zero.

$q=\text{round}(0/s)=0$ and $\hat{x}=s\cdot 0=0$ — zero is represented exactly, which is why symmetric mode is safe for padding and post-ReLU zeros.

Not perfectly: $-8$ has no positive twin ($+8$ is out of range). Symmetric-weight schemes typically restrict to $[-7,7]$ so $q_{\max }=7$ and the grid stays balanced around 0.

That column of $X{X}^{\top }$ is zero, making $H$ singular and $H^{-1}$ undefined; GPTQ adds a small dampening term (a scalar on the diagonal) so the inverse exists — see Hessian and Second-Order Methods.

A per-tensor activation scale stretches to cover that outlier, quantizing every normal channel to almost one level. The fix is to keep outlier channels in higher precision or scale them away — the AWQ (Activation-aware Weight Quantization) idea.

It keeps the base model frozen in 4-bit and trains only small high-precision LoRA adapters on top, so the coarse grid is compensated by learnable corrections — see LoRA and QLoRA.