Min-max scaling and z-score normalization: Foundations

Before you can read the parent note 2.1.5, you need to own every symbol it throws at you. This page builds each one from nothing, in the order they depend on each other. Nothing here assumes you have seen statistics before.

1. A "feature" and its values — the raw material

$x$ (no subscript) = "some value from this column", a stand-in for any one number.
$x_{3}$ = "the value in row 3".
$n$ = the count. If ages = [20, 25, 30, 35, 40], then $n = 5$ and $x_{2} = 25$ .

Figure 1 shows this: every value is a dot dropped onto a horizontal number line. A feature is a cloud of dots on a line — read the dot labels $x_{1} \dots x_{5}$ along the axis.

Figure — Min-max scaling and z-score normalization

Why the topic needs it: scaling changes where these dots sit on the line. You cannot describe "moving the dots" until you can name the dots. That is what $x_{i}$ and $n$ are for.

2. Minimum and maximum — the two edges of the cloud

For ages [20, 25, 30, 35, 40]: $x_{min} = 20$ , $x_{max} = 40$ .

Picture two vertical fence-posts, one at each end of the dot cloud. Everything else lives between them.

Why the topic needs it: min-max scaling works by pinning the left fence-post to $0$ and the right fence-post to $1$ . To do that you must first find those posts.

3. Range and the difference $x_{max} - x_{min}$ — the width

Ages: range $= 40 - 20 = 20$ .

Picture the length of the rubber band stretched between the two fence-posts.

Why the topic needs it: to squash a cloud of width $20$ down into a cloud of width $1$ , you divide by its current width. The range is exactly the number you divide by in min-max scaling. Dividing by a width is how you rescale a length — that is the whole trick.

4. Fractions and "fraction of the way" — what division means here

$x = 30$ in ages: top $= 30 - 20 = 10$ , bottom $= 20$ , ratio $= 10/20 = 0.5$ → halfway.

Figure 2 makes this literal: stand at the left post (value $= 0.0$ ), walk right until you reach the right post (value $= 1.0$ ); each dot's label shows what fraction of the walk it sits at, with $30$ landing exactly at $0.50$ .

Why the topic needs it: min-max scaling literally replaces each value by its "fraction of the way" number. Understanding division as "portion of a whole" is the seed of the entire formula.

5. A linear transformation $a x + b$ — the reshaping machine

If $a = 2, b = 3$ and $x = 5$ : result $= 2 \cdot 5 + 3 = 13$ .

Picture $a$ zooming the ruler in or out; $b$ picks the whole ruler up and moves it left or right. No bending — straight lines stay straight, order is preserved (as long as $a > 0$ ).

These two views are the same formula — here's the proof. We demand that the machine send the min to $0$ and the max to $1$ : $a x_{min} + b = 0 and a x_{max} + b = 1.$ The first equation gives $b = - a x_{min}$ . Substitute that into the second: $a x_{max} - a x_{min} = 1 ⟹ a (x_{max} - x_{min}) = 1 ⟹ a = \frac{1}{x _{max} - x _{min}} .$ Put $a$ back into $b = - a x_{min}$ : $b = - \frac{x _{min}}{x _{max} - x _{min}} .$ Now assemble $a x + b$ : $x_{scaled} = \frac{x}{x _{max} - x _{min}} - \frac{x _{min}}{x _{max} - x _{min}} = \frac{x - x _{min}}{x _{max} - x _{min}} .$ So the "fraction of the way" ratio from Section 4 and the " $a x + b$ machine" are algebraically identical — two names for one transformation. (Notice $a$ is exactly one over the range, so it too blows up when the range is $0$ — the same degenerate case as before.)

Why the topic needs it: both scaling methods are secretly just "find the right $a$ and $b$ ". Min-max solves for the $a, b$ that put min→0, max→1. Z-score solves for the $a, b$ that put the mean→0 and spread→1. Once you see them as the same $a x + b$ machine with different targets, the two formulas stop looking like magic.

6. Expectation $E [\cdot]$ and the mean $μ$ — the balance point

Scores [60,70,80,90,100]: $μ = \frac{400}{5} = 80$ .
$E [\cdot]$ is handy because you can write $E [a x + b]$ = "the average after transforming", which is how the parent note reasons about scaling.

Picture $μ$ as the balance point of the dot cloud — the spot where, if the number line were a see-saw with equal weights at each dot, it would tip level.

Why the topic needs it: z-score normalization re-centres the cloud so its balance point sits exactly at $0$ . To move a balance point to $0$ you subtract $μ$ from every value. Whenever the parent writes $E [x]$ , mentally read it as " $μ$ , the mean".

7. Variance $σ^{2}$ and standard deviation $σ$ — the spread

Why square the distances? Because raw distances $x_{i} - μ$ have plus and minus signs that cancel out (dots left of the mean vs right of it), summing to zero and telling you nothing. Squaring throws away the sign — now every dot contributes a positive "how far out am I" amount. We take the square root at the end to get back to the original units (dollars, not dollars-squared).

Figure 3 contrasts a tight cloud (small $σ$ ) with a wide one (large $σ$ ); the arrows mark one $σ$ measured from the mean line so you can see the "typical step size".

Why the topic needs it: z-score divides by $σ$ so that "one unit" of the new scale equals "one typical step" of the old cloud. That is what turns an absolute value into "how many steps from average".

8. The z-score $z$ — the finished new coordinate

$z = 0$ : at the mean. $z = + 1$ : one step above. $z = - 2$ : two steps below.

Picture re-labelling the number line so the mean is the new zero and one $σ$ is the new "1". The dots don't move relative to each other — only the ruler underneath them changes.

Why the topic needs it: $z$ is the output of standardization, just as $x_{scaled}$ is the output of min-max. Everything above was built so this one line makes complete sense.

Prerequisite map

Both roads (min-max and z-score) start from the same raw dots and pass through the same $a x + b$ machine — they just aim at different targets.

Where these feed next

The centring-and-scaling idea reappears inside deep nets as 5.3.02-Batch-normalization.
Scaling is what makes 3.1.02-Gradient-descent-optimization take even steps in all directions.
Distance and variance thinking powers 4.1.05-Principal-component-analysis and 2.2.01-Feature-selection-methods.
Outlier-robust cousins of these formulas live in 2.1.06-Robust-scaling-and-power-transforms.
Before scaling you must have clean, encoded numbers — see 2.1.01-Data-cleaning-techniques and 2.1.04-One-hot-and-label-encoding.

Equipment checklist

Read the question, answer in your head, then reveal.

What do

x_{i}

and

n

stand for?

x_{i}

is the value in row

i

of a column;

n

is how many values the column has.

What is the range of a feature and why do we divide by it?

Range

= x_{max} - x_{min}

, the cloud's width; dividing by it rescales any length into a 0-to-1 fraction.

What goes wrong if

x_{max} = x_{min}

(or

σ = 0

The formula divides by zero — a constant feature has no spread to scale; detect it and drop the column or output all-zeros.

Solve

a x_{min} + b = 0

and

a x_{max} + b = 1

for

a

and

b

a = 1/ (x_{max} - x_{min})

and

b = - x_{min} / (x_{max} - x_{min})

— giving exactly the min-max ratio.

What does

E [x]

mean, and how does it relate to

μ

E [\cdot]

is the "average of" operator;

E [x] = μ

, the mean.

Why do we square the distances when computing variance, and which divisor is used here?

Squaring removes signs so spread doesn't cancel to zero; we divide by

n

(population variance), not

n - 1

What is the difference between

σ^{2}

and

σ

σ^{2}

is variance (squared units);

σ

is its square root, back in original units — you divide by

σ

, never

σ^{2}

What does a z-score of

- 2

mean in plain words?

The value sits two standard deviations below the mean.