Min-max scaling and z-score normalization

2.1.5 · AI-ML › Data Preprocessing & Feature Engineering

Overview

Feature scaling numerical features ko ek common scale pe transform karta hai, bina ranges ke differences ko distort kiye. Do fundamental methods hain: min-max scaling (range-based) aur z-score normalization (distribution-based). Dono ek hi problem solve karte hain: bade magnitudes wale features ko distance-based algorithms aur gradient descent mein dominate karne se rokna.

Why Feature Scaling Matters

Jab tumhe ZAROOR scale karna chahiye:

• Gradient descent optimization (neural networks, linear/logistic regression)
• Distance-based algorithms (k-NN, k-means, SVM with RBF kernel)
• PCA aur dimensionality reduction
• Regularization (L1/L2) — unscaled features ko unfairly penalize kiya jaata hai

Jab tumhe scale NAHI karna chahiye:

• Tree-based models (decision trees, random forests, XGBoost) — ye relative thresholds pe split karte hain
• Naive Bayes — ye probabilities ke saath kaam karta hai, distances ke saath nahi

Min-max Scaling (Normalization)

$x_{\text{scaled}}=\frac{x-x_{\text{min}}}{x_{\text{max}}-x_{\text{min}}}$

First principles se derivation:

Hum chahte hain ek linear transformation $x_{\text{scaled}}=ax+b$ jo map kare:

• $x_{\text{min}}\to 0$
• $x_{\text{max}}\to 1$

Pehli condition se: $0=a\cdot x_{\text{min}}+b⟹b=-a\cdot x_{\text{min}}$

Doosri condition se: $1=a\cdot x_{\text{max}}+b=a\cdot x_{\text{max}}-a\cdot x_{\text{min}}$

$a$ ke liye solve karte hain: $1=a(x_{\text{max}}-x_{\text{min}})⟹a=\frac{1}{x_{\text{max}}-x_{\text{min}}}$

Back substitute karte hain: $b=-\frac{x_{\text{min}}}{x_{\text{max}}-x_{\text{min}}}$

Therefore: $x_{\text{scaled}}=\frac{1}{x_{\text{max}}-x_{\text{min}}}\cdot x-\frac{x_{\text{min}}}{x_{\text{max}}-x_{\text{min}}}=\frac{x-x_{\text{min}}}{x_{\text{max}}-x_{\text{min}}}$

YE FORMULA KYU? Numerator $(x-x_{\text{min}})$ range ko 0 se start karne ke liye shift karta hai. Denominator $(x_{\text{max}}-x_{\text{min}})$ original range hai, toh isse divide karne par sab kuch [0, 1] mein compress ho jaata hai.

Derivation: Pehle [0, 1] pe map karo, phir $(b-a)$ se stretch karo aur $a$ se shift karo.

Step 1: Range find karo

• $x_{\text{min}}=20$, $x_{\text{max}}=40$
• Range $40-20=20$

Step 2: Har value pe formula apply karo

• $x=20$: $\frac{20-20}{20}=0.00$
• $x=25$: $\frac{25-20}{20}=0.25$ — Ye step kyun? 25, 20 aur 40 ke beech 1/4 distance pe hai
• $x=30$: $\frac{30-20}{20}=0.50$ — Kyun? Exactly midpoint hai
• $x=35$: $\frac{35-20}{20}=0.75$
• $x=40$: $\frac{40-20}{20}=1.00$

Result: [0, 0.25, 0.5, 0.75, 1.0]

Step 1: Generalized formula apply karo $a=-1$, $b=1$ ke saath $x_{\text{scaled}}=-1+\frac{(x-100)(1-(-1))}{300-100}=-1+\frac{2(x-100)}{200}$

Step 2: Calculate karo

• $x=100$: $-1+\frac{2(0)}{200}=-1.0$
• $x=200$: $-1+\frac{2(100)}{200}=0.0$ — Kyun? Middle value, range ke middle pe map hoti hai
• $x=300$: $-1+\frac{2(200)}{200}=1.0$

Result: [-1.0, 0.0, 1.0]

Properties of Min-max Scaling

Advantages:

• Original distribution shape preserve karta hai (sirf rescale hoti hai)
• Bounded output — un algorithms ke liye useful hai jo specific ranges expect karte hain (jaise neural networks with sigmoid)
• Saare features distance calculations mein equally contribute karte hain
• Interpretable: 0.5 ka matlab hai "min aur max ke beech adha"

Disadvantages:

• Outliers ke liye sensitive — ek single extreme value $x_{\text{max}}$ ya $x_{\text{min}}$ ko drastically change kar deti hai
• Training range ke bahar ka naya test data [0, 1] ke bahar values produce karta hai
• Tab suitable nahi jab new data ka range alag ho sakta hai

Kyun sahi lagta hai: Har dataset [0, 1] pe scale ho jaata hai, consistent lagta hai.

Kyun galat hai: Wahi original value train vs test mein alag scaled values pe map hoti hai. 50 ki value training mein 0.5 ban sakti hai lekin testing mein 0.7, agar test set ki range chhoti hai. Model ne training scale ke basis pe decision boundaries seekhi hain, ab wo misaligned hain.

Fix: Scaler ko sirf training data pe fit karo, wahi transformation test pe apply karo:

scaler.fit(train)  # Training se min aur max seekho
train_scaled = scaler.transform(train)
test_scaled = scaler.transform(test)  # Training min/max use karo

Z-score Normalization (Standardization)

$z=\frac{x-\mu }{\sigma }$

First principles se derivation:

Hum chahte hain transformation $z=ax+b$ aisi ki:

• $E[z]=0$ (mean zero ho)
• $\text{Var}(z)=1$ (standard deviation 1 ho)

Step 1: Mean condition $E[z]=E[ax+b]=aE[x]+b=a\mu +b=0$ $⟹b=-a\mu$

Step 2: Variance condition $\text{Var}(z)=\text{Var}(ax+b)=a^2\text{Var}(x)=a^2{\sigma }^2=1$ $⟹a=\pm \frac{1}{\sigma }$

Hum $a=\frac{1}{\sigma }$ choose karte hain (order preserve karne ke liye positive).

Step 3: Combine karo $z=\frac{1}{\sigma }\cdot x-\frac{\mu }{\sigma }=\frac{x-\mu }{\sigma }$

YE FORMULA KYU? $(x-\mu )$ distribution ko 0 pe center karta hai. $\sigma$ se divide karne par spread exactly 1 standard deviation ho jaata hai.

Ye absolute values ko distribution mein relative positions mein convert karta hai.

Step 1: Statistics calculate karo

• Mean: $\mu =\frac{60+70+80+90+100}{5}=80$
• Variance: ${\sigma }^2=\frac{(60-80{)}^2+(70-80{)}^2+(80-80{)}^2+(90-80{)}^2+(100-80{)}^2}{5}$ $=\frac{400+100+0+100+400}{5}=200$
• Standard deviation: $\sigma =\sqrt{200}\approx 14.14$

Step 2: Formula apply karo

• $x=60$: $z=\frac{60-80}{14.14}=-1.41$ — Kyun? 60, average se 1.41 std devs neeche hai
• $x=70$: $z=\frac{70-80}{14.14}=-0.71$
• $x=80$: $z=\frac{80-80}{14.14}=0.00$ — Kyun? Mean pe hai
• $x=90$: $z=\frac{90-80}{14.14}=0.71$
• $x=100$: $z=\frac{100-80}{14.14}=1.41$

Result: [-1.41, -0.71, 0, 0.71, 1.41]

Verification: Z-scores ka mean = 0, std dev = 1 ✓

Min-max scaling:

• $x_{\text{min}}=30000$, $x_{\text{max}}=200000$
• Range $=200000-30000=170000$
• $x=30000$: $\frac{30000-30000}{170000}=0.000$
• $x=45000$: $\frac{45000-30000}{170000}=\frac{15000}{170000}=0.088$ — Kyun? Saari normal salaries 0 ke paas squish ho gayi hain
• $x=200000$: $\frac{200000-30000}{170000}=1.000$ (outlier)

Notice: Charon normal salaries [0, 0.088] mein aati hain — outlier ki wajah se completely compressed ho gayi hain.

Z-score normalization:

• Mean: $\mu =\frac{30000+35000+40000+45000+200000}{5}=\frac{350000}{5}=70000$
• Variance: ${\sigma }^2=\frac{(30000-70000{)}^2+(35000-70000{)}^2+(40000-70000{)}^2+(45000-70000{)}^2+(200000-70000{)}^2}{5}$ $=\frac{1.6\times 10^9+1.225\times 10^9+0.9\times 10^9+0.625\times 10^9+16.9\times 10^9}{5}=\frac{21.25\times 10^9}{5}=4.25\times 10^9$
• Standard deviation: $\sigma =\sqrt{4.25\times 10^9}\approx 65192$
• $z_{45000}=\frac{45000-70000}{65192}=\frac{-25000}{65192}\approx -0.38$
• $z_{200000}=\frac{200000-70000}{65192}=\frac{130000}{65192}\approx 1.99$

Z-score yahan better kyun hai? Outlier ko ek bada z-score milta hai (≈2.0) lekin normal salaries phir bhi roughly $[-0.61,-0.38]$ mein spread hoti hain, na ki 0 ke paas ek tiny sliver mein crush hoti hain. Lekin: Z-score bhi outliers se affect hota hai kyunki wo $\mu$ aur $\sigma$ dono ko inflate karte hain — extreme cases ke liye, robust scaling (median aur IQR) use karo.

Properties of Z-score Normalization

Advantages:

• Min-max se outliers ke liye kam sensitive (lekin immune nahi)
• Bounded range nahi — kisi bhi value ke saath naya data handle karta hai
• Alag-alag units wale features ke beech comparable scales banata hai
• Normally distributed data ke saath achha kaam karta hai
• Un algorithms ke liye required hai jo standardized inputs assume karte hain (PCA, LDA)

Disadvantages:

• Output bounded nahi hai ([0,1] ki tarah)
• Interpret karna mushkil hai (intuitively z=-0.73 ka kya matlab hai?)
• Roughly bell-shaped distribution assume karta hai
• Extreme outliers ke saath break down karta hai

Kyun sahi lagta hai: Z-score "standardization" hai, universally applicable lagta hai.

Kyun problematic hai: Z-score assume karta hai ki distribution shape roughly preserve hogi. Skew ke saath, zyaadatar z-scores ek taraf cluster karte hain, aur "1 standard deviation" ki interpretation misleading ho jaati hai. Mean outliers se pull hota hai.

Fix: Robust scaling use karo median aur IQR ke saath: $x_{\text{robust}}=\frac{x-\text{median}(x)}{\text{IQR}(x)}$ Ya pehle log transformation apply karo skew reduce karne ke liye, phir z-score.

When to Use Which Method

Implementation Considerations

Fitting and Transforming

HAMESHA training data pe hi fit karo:

from sklearn.preprocessing import MinMaxScaler, StandardScaler
 
# Min-max
scaler_minmax = MinMaxScaler()
scaler_minmax.fit(X_train)  # Min aur max seekho
X_train_scaled = scaler_minmax.transform(X_train)
X_test_scaled = scaler_minmax.transform(X_test)
 
# Z-score
scaler_std = StandardScaler()
scaler_std.fit(X_train)  # Mean aur std seekho
X_train_scaled = scaler_std.transform(X_train)
X_test_scaled = scaler_std.transform(X_test)

Sirf training pe fit kyun karo? Data leakage rokne ke liye. Test set "unseen future data" represent karta hai — uski statistics use karne se test distribution ki information model mein leak ho jaati hai.

Feature-wise vs Global Scaling

Feature-wise (standard): Har column independently scale hoti hai

• Tab use karo jab features ke alag units/meanings hon
• Features ke beech relationships unke apne scale mein preserve karta hai

Global (rare): Saari features ek hi min/max ya mean/std use karte hain

• Sirf tab jab features same quantity represent karte hain (jaise pixel intensities)

Inverse Transformations

Dono methods reversible hain:

Min-max inverse: $x=x_{\text{scaled}}\cdot (x_{\text{max}}-x_{\text{min}})+x_{\text{min}}$

Z-score inverse: $x=z\cdot \sigma +\mu$

Use case: Predictions ko interpretation ke liye original scale pe wapas convert karo.

Min-max scaling aisa hai jaise kehna "Har hafte tumne apne total ka kitna percentage bachaya?" Tum dono apne paise ko 0% se 100% ke percentages mein convert karte ho. Ab compare kar sakte ho!

Z-score aisa hai jaise poochna "Tumhara har hafte tumhare average se kitna upar ya neeche tha?" Agar tumhara average ₹100/hafte hai, aur ek hafte tumne ₹150 bachaye, tum average se +50 upar ho. Agar tumhare dost ka average $2/haftehaiaurunhoneekhafte$3 bachaye, wo apne average se +1 upar hain. Z-score batata hai "tum average se kitne 'typical amounts' upar ya neeche ho" taki alag currencies ke saath bhi compare kar sako.

Dono methods "apples aur oranges" ko same unit mein convert karte hain taki tum unhe fairly compare kar sako!

Connections

• 2.1.01-Data-cleaning-techniques — scaling, missing values aur outliers handle karne ke baad apply hoti hai
• 2.1.04-One-hot-and-label-encoding — categorical encoding, scaling se pehle hoti hai
• 2.2.01-Feature-selection-methods — correlation-based selection se pehle scaling zaroori hai
• 3.1.02-Gradient-descent-optimization — scaling convergence ko dramatically speed up karti hai
• 4.1.05-Principal-component-analysis — PCA ke liye centered/standardized data chahiye
• 5.3.02-Batch-normalization — training ke dauran neural network layer-wise standardization
• 2.1.06-Robust-scaling-and-power-transforms — outliers aur skewed distributions ke liye advanced scaling

#flashcards/ai-ml

Feature scaling kya hai aur ye kyun zaroori hai? :: Feature scaling numerical features ko ek common scale pe transform karta hai. Ye isliye zaroori hai kyunki bade magnitudes wale features distance calculations aur gradient descent mein dominate karte hain, jisse algorithms unhe unfairly overweight karte hain.

Concept Map