Regression metrics (MAE, MSE, RMSE, MAPE) — Hinglish notes

2.6.10 · AI-ML › Model Evaluation & Selection

Core Concepts

Mean Absolute Error (MAE)

Worked example MAE Calculation

Apartment rent predict karna (in $1000s/month):

Sample	Actual ( $y_{i}$ )	Predicted ( $\overset{y}{^}_{i}$ )	Error ( $e_{i}$ ) $∥ e_{i} ∥$
1	2.5	2.8	-0.3
2	1.8	1.6	0.2
3	3.2	3.9	-0.7
4	2.1	2.0	0.1

$MAE = \frac{0.3 + 0.2 + 0.7 + 0.1}{4} = \frac{1.3}{4} = 0.325 (i.e., $325)$

Yeh step kyun? Absolute errors ka sum total magnitude deta hai; $n$ se divide karne par per-sample average milta hai. Result: "Average mein, predictions $325 se off hain."

Mean Squared Error (MSE)

Worked example MSE Calculation (Same Data)

Sample	$∥ e_{i} ∥$	$e_{i}^{2}$
1	0.3	0.09
2	0.2	0.04
3	0.7	0.49
4	0.1	0.01

$MSE = \frac{0.09 + 0.04 + 0.49 + 0.01}{4} = \frac{0.63}{4} = 0.1575$

Yeh step kyun? Sample 3 ki error (0.7) ab dominate karti hai: yeh total 0.63 mein se 0.49 contribute karta hai (78% error!). Compare karo MAE se jahan yeh 1.3 mein se 0.7 tha (54%).

Interpretation: MSE outliers ke liye zyada sensitive hai. Agar aapke paas kuch catastrophically galat predictions hain, toh MSE unke baare mein zyada chillata hai.

Root Mean Squared Error (RMSE)

Mean Absolute Percentage Error (MAPE)

Worked example MAPE Calculation

| Sample | Actual | Predicted | $∥ e_{i} ∥$ | $\frac{∥ e _{i} ∥}{y _{i}}$ | Percentage | |--------|--------|----------------------|------------| | 1 | 2.5 | 2.8 | 0.3 | 0.3/2.5 = 0.12 | 12% | | 2 | 1.8 | 1.6 | 0.2/1.8 ≈ 0.111 | 11.1% | | 3 | 3.2 | 3.9 | 0.7 | 0.7/3.2 ≈ 0.219 | 21.9% | | 4 | 2.1 | 2.0 | 0.1/2.1 ≈ 0.048 | 4.8% |

$MAPE = \frac{12 + 11.1 + 21.9 + 4.8}{4} = \frac{49.8}{4} = 12.45%$

Yeh step kyun? Ab sample 3 sabse zyada contribute karta hai (21.9%) isliye nahi ki absolute error sabse badi thi, balki isliye ki yeh true value ka 21.9% hai. Sample 4 muskil se register hota hai (4.8%) chahe absolute error (0.1) chhoti ho.

Interpretation: "Average mein, predictions true value ke 12.45% se off hain."

Kab Kaun Sa Metric Use Karein

Intuition Decision Framework

MAE use karo jab:

Aap original units mein interpretability chahte hain.
Saari errors equally weighted honi chahiyen ( $10 k i er r or$ 5 ki error se 2× buri hai, bas).
Aapke data mein outliers hain aur aap nahi chahte ki woh dominate karein (MAE robust hai).
Example: Delivery times predict karna (30min late hai toh 30min late hai, chahe package 1 ghante mein aana tha ya 5 ghante mein).

MSE use karo jab:

Aap training ke dauran optimize kar rahe hain (gradient descent smooth derivatives pasand karta hai).
Badi errors aapke application ke liye disproportionately buri hain.
Aapko kabhi non-technical stakeholders ko report nahi karni (squared units confusing hain). RMSE use karo jab:
Aap MSE ki outliers ke liye sensitivity chahte hain LEKIN interpretable units bhi.
Kai regression benchmarks ke liye standard choice hai (papers/models compare karna aasan hota hai).
Example: House prices predict karna jahan $1 M k e g ha r p a r$ 200K ki miss, do $100K ki misses se kaafi zyada hurt karni chahiye. MAPE use karo jab:
Aapka target variable multiple orders of magnitude span karta hai (stock prices: $5 se$ 5000 tak).
Stakeholders percentages mein sochte hain ("hum 8% off hain" zyada clear hai "$230 off" se).
WARNING: Undefined hota hai jab $y_{i} = 0$ . Asymmetric hai (overestimation ko underestimation se zyada penalize karta hai).
Example: Products par sales forecasting ( $10 i t e m s v s .$ 1000 items par error fairly compare karna).

Common mistake Common Pitfalls

Galti 1: "MAE aur MSE hamesha models ko same tarike se rank karte hain." Kyun sahi lagta hai: Dono error measure karte hain, toh sabse kam error sabse kam hoga, hai na?

Fix: Do models ko test set par consider karo:

Model A: Errors = [1, 1, 1] → MAE = 1, MSE = 1
Model B: Errors = [0, 0, 0, 4] → MAE = 1, MSE = 4 MAE kehta hai dono tie hain. MSE kehta hai B 4× bura hai. Metric choose karo apni large errors ki tolerance ke hisaab se.

Galti 2: "MAPE hamesha behtar hai kyunki yeh scale-independent hai." Kyun sahi lagta hai: Alag-alag scale ke datasets par errors compare karna ideal lagta hai.

Fix: MAPE mein fatal flaws hain:

Undefined hai jab $y_{i} = 0$ (division by zero).
Asymmetric: 100 true hone par 90 predict karna (10% error) vs. 100 true hone par 110 predict karna (10% error) → lekin pehla hai 10/100, doosra hai 10/110 ≈ 9.1%. Same absolute error ke liye alag penalties!
Underestimation ko overestimation se zyada heavily penalize karta hai.

Behtar alternative: sMAPE (symmetric MAPE) use karo agar aapko percentage-based metric chahiye: $sMAPE = \frac{100%}{n} \sum_{i = 1}^{n} \frac{∣ y _{i} - y ^ _{i} ∣}{( ∣ y _{i} ∣ + ∣ y ^ _{i} ∣ ) /2}$

Galti 3: "Main saare metrics use karunga aur sabse accha result pick karunga."

Kyun sahi lagta hai: Zyada information behtar hai, hai na?

Fix: Yeh metric shopping hai (overfitting ka ek roop). Aapko evaluation metric(s) training se pehle business requirements ke basis par choose karna chahiye. Warna aap results cherry-pick kar rahe hain.

Recall 12 Saal Ke Bachche Ko Explain Karo

Imagine karo tum dartboard par darts phenk rahe ho, aur hum measure karna chahte hain ki tum kitne acche ho.

MAE aisa hai: Har phenk ke baad, measure karo ki dart bullseye se kitna door hai (centimeters mein), sab add karo, aur throws ki sankhya se divide karo. Agar 3 cm off, phir 5 cm off, phir 2 cm off, toh average hai (3+5+2)/3 = 3.3 cm. Aasaan!

MSE alag hai: Sirf distance measure karne ki jagah, hum use square karte hain. Toh 3 cm ho jaata hai 9, 5 cm ho jaata hai 25, aur 2 cm ho jaata hai 4. Average hai (9+25+4)/3 = 12.7. Dhyan do ki 5 cm wala throw (jo sirf thoda bura tha) ab kaafi zyada count karta hai (25 vs. 9). Hum badi galtiyon ko extra hard punish kar rahe hain!

RMSE bas us 12.7 ka square root le raha hai, 3.6 cm wapas milta hai. Hum ne badi galtiyon ko punish karne ke liye square kiya, phir normal units wapas pane ke liye un-square kiya.

MAPE tab ke liye hai jab target khud size change karta hai. Teen dart boards imagine karo: ek tiny (10 cm across), ek medium (100 cm), ek huge (1000 cm). Tiny board par 5 cm ki miss "board ka 50%!" hai, lekin huge board par yeh sirf "0.5% off" hai. MAPE aapki error ko board size ke percentage ke roop mein measure karta hai, toh yeh alag-alag boards par fair hai.

Connections

Loss functions in neural networks - MSE standard regression loss hai
Gradient descent - MSE ki differentiability optimization ko smooth banati hai
Cross-validation - Yeh metrics model selection ke liye held-out folds par compute hote hain
Outlier detection - MAE vs. RMSE ka difference outliers flag karta hai
Model comparison - Architectures compare karne se pehle metric fix karna zaroori hai
Bias-variance tradeoff - High MSE high variance indicate kar sakta hai (noise par overfitting)
Feature scaling - MAPE scale ke liye invariant hai, MAE/MSE/RMSE target range se affected hain

#flashcards/ai-ml

Chaar basic regression metrics aur unke acronyms kya hain? :: MAE (Mean Absolute Error), MSE (Mean Squared Error), RMSE (Root Mean Squared Error), MAPE (Mean Absolute Percentage Error)

MAE ka formula likhein :: $MAE = \frac{1}{n} \sum_{i = 1}^{n} ∣ y_{i} - \overset{y}{^}_{i} ∣$ jahan $y_{i}$ actual hai, $\overset{y}{^}_{i}$ predicted hai

MAE ke units kya hain?

Same units as the target variable (interpretable: dollars, meters, etc.)

MSE ka formula likhein

MSE = \frac{1}{n} \sum_{i = 1}^{n} (y_{i} - \overset{y}{^}_{i})^{2}

MSE ke units kya hain aur yeh problem kyun hai?

Target ke squared units (dollars², meters²); directly interpretable nahi, lekin optimization ke liye mathematically convenient hai

MSE chhoti errors se badi errors ko zyada kyun penalize karta hai?

Kyunki errors square hoti hain; 10-unit error 100 ban jaati hai (10 nahi), jabki 2-unit error 4 ban jaati hai; ratio 5:1 ki jagah 25:1 ho jaata hai

RMSE ka formula likhein

RMSE = MSE = \frac{1}{n} \sum_{i = 1}^{n} (y_{i} - \overset{y}{^}_{i})^{2}

RMSE aur MAE ke beech kya relationship hai?

RMSE ≥ MAE hamesha; jab RMSE >> MAE, tab badi outlier errors hain; jab RMSE ≈ MAE, tab errors uniformly distributed hain

MAPE ka formula likhein

MAPE = \frac{100%}{n} \sum_{i = 1}^{n} \frac{∣ y _{i} - y ^ _{i} ∣}{∣ y _{i} ∣}

MAPE ke units kya hain? :: Percentage (dimensionless); alag-alag scales par comparison allow karta hai

MAPE ki critical limitation kya hai?

Undefined hai jab actual value

y_{i} = 0

(division by zero); aur asymmetric bhi hai (overestimation ko underestimation se alag penalize karta hai)

MSE par MAE kab use karna chahiye?

Jab aap saari errors ki equal weighting, interpretable units, aur outliers ke liye robustness chahte hain; jab errors magnitude ke saath linearly scale honi chahiyen

MAE par RMSE kab use karna chahiye?

Jab badi errors disproportionately costly hain aur unhe zyada penalize karna hai, lekin original scale mein interpretable units bhi chahiyen

MAPE kab use karna chahiye?

Jab target variable multiple orders of magnitude span kare aur relative error absolute error se zyada matter kare; jab stakeholders percentages mein sochte hain

Training ke dauran RMSE zyada interpretable hone ke bawajood MSE kyun prefer kiya jaata hai?

MSE har jagah differentiable hai (smooth gradient), jo gradient descent optimization ko zyada stable banata hai; RMSE ka square root zero ke paas numerical instability introduce karta hai

RMSE, MAE se kaafi bada ho toh iska matlab kya hai?

Badi outlier predictions hain; model ki kuch predictions catastrophically galat hain jo squared error ko drive kar rahi hain

Agar Model A ki errors [2,2,2,2] hain aur Model B ki errors [0,0,0,8] hain, toh dono ke liye MAE aur MSE calculate karo. Kaun sa metric kaun se model ko prefer karta hai?

Model A: MAE=2, MSE=4; Model B: MAE=2, MSE=16; MAE kehta hai tie hai, MSE strongly A ko prefer karta hai (4 vs 16); demonstrate karta hai ki metric choice matter karti hai

Metric shopping kya hai aur yeh bura kyun hai?

Apna evaluation metric results dekhne ke baad choose karna taaki best numbers milein; yeh overfitting/cherry-picking ka ek roop hai; metric training se pehle requirements ke basis par choose karna chahiye

MAPE ka ek behtar alternative kya hai jo uski asymmetry fix karta hai?

sMAPE (symmetric MAPE):

sMAPE = \frac{100%}{n} \sum_{i = 1}^{n} \frac{∣ y _{i} - y ^ _{i} ∣}{( ∣ y _{i} ∣ + ∣ y ^ _{i} ∣ ) /2}

Alag-alag scales wale datasets par MAE values directly compare kyun nahi kar sakte?

MAE original units mein hota hai; 100 ki MAE ka matlab house prices (

100 K sc a l e) a u r a p a r t m e n t r e n t s (

1K scale) ke liye alag hota hai; MAPE fair comparison allow karta hai