Noesis
2.1.5 · D5Data Preprocessing & Feature Engineering

Question bank — Min-max scaling and z-score normalization

1,553 words7 min read↑ Back to topic

Before we start, three plain-word reminders so every term below is earned:

  • Scaling = changing the numbers of a feature so they sit on a common ruler, without changing which value is bigger than which.
  • Min-max squeezes values into a fixed box like using the smallest and largest values.
  • Z-score re-centres values so the average becomes and one "typical spread" (one standard deviation, written ) becomes .

True or false — justify

True or false: Min-max scaling changes the shape of a feature's distribution.
False — it is a linear stretch-and-shift, so histogram bars keep their relative spacing; only the axis labels change. A skewed feature stays exactly as skewed after min-max.
True or false: After z-score normalization the data becomes a normal (bell) curve.
False — z-score only forces mean and ; a skewed or bimodal shape stays skewed or bimodal. Making data Gaussian needs a power transform, see 2.1.06-Robust-scaling-and-power-transforms.
True or false: Min-max scaled values are always inside .
False for new data — a test value larger than the training maps above , and one below maps below . It is only guaranteed for the data used to fit the scaler.
True or false: Z-score normalized values are always inside .
False — a value two standard deviations above the mean gives . Z-scores are unbounded; only their mean and spread are fixed.
True or false: Scaling a feature makes it more "important" to the model.
False — scaling removes an accidental importance caused by units; it puts features on equal footing so the model can judge real importance. It never adds genuine predictive power.
True or false: You should scale the target variable the same way you scale the inputs.
Usually false for classification (the target is a label), and optional for regression — scaling is a separate choice that only affects loss magnitude, not the features. Don't blindly apply the input scaler to the target.
True or false: Tree-based models (random forests, XGBoost) need feature scaling to perform well.
False — trees split on thresholds like "is ?", which is unaffected by any monotonic rescaling. Scaling wastes effort here and changes nothing.
True or false: If a feature is already between and , it needs no scaling when other features range in the thousands.
False — the mismatch between the two features is exactly the problem; the small feature will be swamped in distance and gradient calculations. Scale all features to the same ruler, not each to a "nice-looking" one.
True or false: Min-max and z-score always rank the values in the same order.
True — both are increasing linear maps (), so if then their scaled versions keep that order. Neither ever flips which value is larger.

Spot the error

A student scales train and test separately, each to , saying "both are now in the same range, so it's consistent." Where's the error?
The range matches but the mapping doesn't: the same raw value maps to different scaled numbers because train and test have different min/max. Fit the scaler on train only, then transform test with the training min/max — see the parent's mistake callout.
Someone computes the mean and std on the whole dataset (train + test together) before splitting. What leaks?
Test information leaks into the scaler's and , giving optimistically biased evaluation. Statistics must come from training data only, then be applied to the held-out set.
A pipeline computes but for a constant feature and returns NaN. What's the real issue?
A constant feature carries zero information, and dividing by is undefined. Drop the feature (or leave it as a fixed constant) rather than trying to scale it — this is a data-cleaning step, see 2.1.01-Data-cleaning-techniques.
A model uses min-max scaling and one training outlier of crushes every normal value into . Why did this happen and what fixes it?
is dragged to the outlier, so the useful values share a tiny slice of and lose resolution. Use z-score (spreads by , not by the extreme) or robust scaling in 2.1.06-Robust-scaling-and-power-transforms.
Code applies fit_transform to the test set. Why is that a bug even though it "runs fine"?
fit_transform re-learns min/max (or ) from the test set, so the model sees a different transformation than it trained on. Test data must only be transformed with parameters learned from train.
Someone one-hot encodes a category into columns, then z-score normalizes those columns too. What's off?
Standardizing binary indicators turns clean meaning into odd fractional z-scores and can distort sparse structure. One-hot columns from 2.1.04-One-hot-and-label-encoding are usually left as-is or min-max'd, not centered.

Why questions

Why divide by the range in min-max rather than by alone?
Because the raw range is the true "width" of the data after we shift the start to ; dividing by it maps the top of the shifted data exactly to . Dividing by would only work if were already .
Why does z-score divide by instead of the range like min-max does?
The range depends on just two extreme points, but summarizes how all points spread around the mean. This makes z-score far less sensitive to a single outlier warping the scale.
Why do gradient-descent optimizers converge faster on scaled data?
With mismatched feature scales the loss surface becomes a stretched valley, so the step size that suits one feature overshoots or crawls for another. Scaling makes the valley round, letting one learning rate work for all directions — see 3.1.02-Gradient-descent-optimization.
Why does PCA require scaling before it's applied?
PCA finds directions of largest variance, and an unscaled large-unit feature dominates variance purely by its units. Scaling first ensures the components reflect real structure, not accidental magnitude — see 4.1.05-Principal-component-analysis.
Why can L1/L2 regularization treat features unfairly if they aren't scaled?
Regularization penalizes the size of each weight, and a large-unit feature earns a small weight while a small-unit feature earns a large one for the same effect. The penalty then hits the small-unit feature harder purely because of units, distorting feature selection shrinkage.
Why does a z-score of mean something interpretable while a min-max value of means something different?
marks the mean (the balance point of the distribution), whereas min-max marks the minimum (the smallest observed value). One is a distribution center, the other a data boundary.
Why is batch normalization inside a network related to, but not a replacement for, input z-scoring?
Batch norm standardizes activations inside the network per mini-batch to stabilize training, while input scaling standardizes the raw features once before training. Both center-and-scale, but at different stages — see 5.3.02-Batch-normalization.

Edge cases

What happens to min-max scaling when all values are identical (a constant feature)?
gives division by zero. The feature is information-free; drop it or map it to a fixed constant like instead of scaling.
What does a single value dataset (one row) do to z-score?
With one point equals that point and , so is undefined. Scaling statistics need at least two distinct values to be meaningful.
If a test value equals the exact training minimum, what is its min-max output — and is that a problem?
It maps to exactly , which is fine and expected. Trouble only appears when test values fall outside the learned , producing values beyond .
For two features with identical spread but different means, what does z-score do that min-max also does?
Both align the features onto a comparable ruler, but z-score guarantees the same mean and spread , while min-max guarantees the same box regardless of internal spread. Choose based on whether outliers or fixed bounds matter more.
If new production data drifts and its true range doubles over time, which method degrades more gracefully?
Min-max breaks hard, pushing many new values outside ; z-score just yields larger magnitude z-scores but stays meaningful as "standard deviations from the (old) mean." Persistent drift, though, argues for re-fitting the scaler on fresh data.
Recall One-line self-check

Name the single failure mode min-max has that z-score largely avoids. ::: A single outlier squashes all normal values, because min-max scales by the range (two extreme points) while z-score scales by (all points).