Exercises — Min-max scaling and z-score normalization

WHAT: Plug into the min-max formula. WHY: $35$ is asked as a fraction of the way from the smallest to the largest value. $x_{\text{scaled}}=\frac{35-10}{60-10}=\frac{25}{50}=0.5$ WHAT IT LOOKS LIKE: $35$ is exactly halfway between $10$ and $60$, so on the squeezed $[0,1]$ ruler it lands dead center at $0.5$. ✅

$z=0$: the value equals the mean exactly ($x=\mu$, so numerator is $0$). $z=-2$: the value sits two standard deviations below the mean. The minus sign means "below"; the "2" means "two spreads away."

Step 1 (WHAT): find the endpoints. $x_{\text{min}}=150$, $x_{\text{max}}=190$, range $=40$. Step 2 (WHY): each value's distance above $150$, divided by $40$, gives its position on the ruler.

• $150:\frac{0}{40}=0.00$
• $160:\frac{10}{40}=0.25$
• $170:\frac{20}{40}=0.50$
• $180:\frac{30}{40}=0.75$
• $190:\frac{40}{40}=1.00$

Result: $[0,0.25,0.5,0.75,1.0]$. Evenly spaced input stays evenly spaced — min-max is linear, it never bends the shape.

WHY this formula: we want the generalized map $x_{\text{scaled}}=a+\frac{(x-x_{\text{min}})}{x_{\text{max}}-x_{\text{min}}}(b-a)$, where $a$ and $b$ are the endpoints of the target interval. Here the target is $[-1,1]$, so $a=-1$ (low end) and $b=1$ (high end); the data endpoints are $x_{\text{min}}=4$, $x_{\text{max}}=12$. Read it as: map to $[0,1]$ first via $\frac{x-4}{12-4}$, then stretch by $(b-a)=2$ and shift by $a=-1$. $x_{\text{scaled}}=-1+\frac{(x-4)}{12-4}(1-(-1))=-1+\frac{2(x-4)}{8}$

• $x=4:-1+\frac{2(0)}{8}=-1.0$
• $x=8:-1+\frac{2(4)}{8}=0.0$ (the middle value lands at the middle of the target range)
• $x=12:-1+\frac{2(8)}{8}=1.0$

Result: $[-1,0,1]$.

Step 1: mean $\mu =\frac{50+60+70}{3}=60$. Step 2: variance ${\sigma }^2=\frac{(50-60{)}^2+(60-60{)}^2+(70-60{)}^2}{3}=\frac{100+0+100}{3}=\frac{200}{3}\approx 66.67$. Step 3: $\sigma =\sqrt{66.67}\approx 8.165$. Step 4: apply $z=\frac{x-\mu }{\sigma }$:

• $50:\frac{-10}{8.165}\approx -1.225$
• $60:\frac{0}{8.165}=0.000$
• $70:\frac{10}{8.165}\approx 1.225$

Result: $[-1.225,0,1.225]$. Symmetric data gives symmetric z-scores.

Use the training min and max (never refit on test data): $x_{\text{scaled}}=\frac{120-0}{100-0}=1.2$ The output is $1.2$, outside $[0,1]$. This is expected and correct: the test value is larger than anything seen in training, so it lands past the right end of the ruler. Refitting the scaler on the test set (to force it back into $[0,1]$) would be the actual bug — it changes what a given raw value means and misaligns the model's learned boundaries.

Min-max: $x_{\text{min}}=30$, $x_{\text{max}}=200$, range $=170$. $45\to \frac{45-30}{170}=\frac{15}{170}\approx 0.088$ The four normal salaries ($30,35,40,45$) all crush into $[0,0.088]$ — the outlier $200$ stole the whole ruler.

Z-score: mean $\mu =\frac{350}{5}=70$. Variance ${\sigma }^2=\frac{(30-70{)}^2+(35-70{)}^2+(40-70{)}^2+(45-70{)}^2+(200-70{)}^2}{5}$ $=\frac{1600+1225+900+625+16900}{5}=\frac{21250}{5}=4250$. $\sigma =\sqrt{4250}\approx 65.19$. $45\to z=\frac{45-70}{65.19}\approx -0.383$

Which spreads normals more? Z-score. Under min-max the normal points span a width of only $0.088$; under z-score they span roughly $[-0.61,-0.38]$ — a width near $0.23$, still small but relatively larger, and crucially the outlier is only $z\approx 1.99$ (not the maximum of the ruler). Min-max is defined by the extremes, so a single outlier controls it; z-score is defined by mean and spread, which one point moves less violently.

We assume $x_{\text{max}}>x_{\text{min}}$ (a strictly positive range; the constant-feature case with zero range was handled separately in the opening callout, where we define the output as $0$). Start from $x_{\text{min}}\le x\le x_{\text{max}}$. Subtract $x_{\text{min}}$ from all three parts: $0\le x-x_{\text{min}}\le x_{\text{max}}-x_{\text{min}}$ The right end is the range, which is $>0$ by assumption. Divide all three parts by the positive range (dividing an inequality by a positive number preserves its direction): $0\le \frac{x-x_{\text{min}}}{x_{\text{max}}-x_{\text{min}}}\le 1$ The middle is exactly $x_{\text{scaled}}$, so $0\le x_{\text{scaled}}\le 1$. ∎ The guarantee holds only for values inside the training range — Problem 3.1 showed why test data can escape it.

Raw z-score: $\mu =\frac{2+4+9}{3}=5$. ${\sigma }^2=\frac{(2-5{)}^2+(4-5{)}^2+(9-5{)}^2}{3}=\frac{9+1+16}{3}=\frac{26}{3}$, so $\sigma =\sqrt{26/3}\approx 2.944$. z-scores: $2\to \frac{-3}{2.944}\approx -1.019$, $4\to \frac{-1}{2.944}\approx -0.340$, $9\to \frac{4}{2.944}\approx 1.359$.

Min-max first: $x_{\text{min}}=2,x_{\text{max}}=9$, range $=7$. Scaled $=[0,\frac{2}{7},1]=[0,0.2857,1]$. Now z-score the scaled data: mean $=\frac{0+0.2857+1}{3}=0.4286$. Deviations: $-0.4286,-0.1429,0.5714$; squares sum $÷3$ = $\frac{0.1837+0.0204+0.3265}{3}=0.1769$, $\sigma =0.4205$. z-scores: $\frac{-0.4286}{0.4205}\approx -1.019$, $\frac{-0.1429}{0.4205}\approx -0.340$, $\frac{0.5714}{0.4205}\approx 1.359$. Identical. ✅

WHY: min-max is a linear map $x_{\text{scaled}}=ax+b$ with $a>0$. Z-scoring subtracts the mean and divides by the standard deviation, and both mean and standard deviation scale by exactly the same factor $a$ under a linear map — so the constant $a$ (and shift $b$) cancel completely. Composing an affine map before standardization changes nothing.

The requirement: k-NN measures Euclidean distance, so both features must contribute comparably; a feature with a huge numeric span dominates the distance and silently decides every neighbor.

age: roughly symmetric, bounded, no extreme outliers → z-score (or min-max) is fine. Z-score is a clean default: centers at $0$, spread $1$.

income: heavily right-skewed with extreme outliers (billionaires). Naive min-max would set $x_{\text{max}}=500\text{k}$; every ordinary income crushes into a tiny sliver near $0$ (exactly the Problem 3.2 failure). Naive z-score is less bad but the outliers still inflate $\sigma$, compressing the normal range. Best: apply a log transform first (to tame the skew), then standardize — or use robust scaling which uses the median and interquartile range and ignores the tails.

What goes wrong with the naive choice (min-max both features): income's scaled distances become nearly $0$ for all normal people, so k-NN effectively classifies on age alone. The model looks trained but ignores half its inputs.

Bonus consistency note: whatever you pick, fit on train, transform test (Problem 3.1), and remember scaling is irrelevant for tree models — only distance/gradient methods like k-NN and gradient descent need it.

Formula (robust, clipped both ends): first form the raw ratio, then clamp it into $[0,1]$ from both sides using a nested $\min /\max$: $x_{\text{scaled}}=\min \left(1,\max \left(0,\frac{x-x_{\text{min}}}{p_{95}-x_{\text{min}}}\right)\right)$ WHY the outer $\min$: anything above $p_{95}$ gives a ratio $>1$; capping at $1$ stops the lone upper outlier from stretching the whole ruler. WHY the inner $\max$: anything below $x_{\text{min}}$ (a new, unusually low value) gives a negative ratio; the inner $\max (0,\cdot )$ pins it to $0$ so the output can never go negative — the lower edge case is now explicitly handled.

Compute $x=60$: $\frac{60-10}{110-10}=\frac{50}{100}=0.5$; both clamps are inactive, so $0.5$. Normal values keep full resolution.

Compute $x=200$: raw ratio $=\frac{200-10}{100}=1.9$, but $\min (1,\max (0,1.9))=1.0$. The upper outlier is capped at $1$.

Compute $x=5$: raw ratio $=\frac{5-10}{100}=-0.05$, but $\max (0,-0.05)=0$, then $\min (1,0)=0$. The below-range value is pinned to $0$ instead of going negative. This two-sided clamping is the core idea behind robust scaling — full treatment in the robust-scaling note.

Min-max maps $x_{\text{min}}\to$ 0 and $x_{\text{max}}\to$ 1. Z-score gives mean 0 and standard deviation 1. The method most damaged by a single extreme outlier is min-max. Always fit on train, transform test to keep the scale consistent. Composing a linear pre-scale before z-scoring changes the result by nothing (they cancel). When the range is zero (constant feature), define the output as 0 (and usually drop the feature).