Simple linear regression model — Hinglish notes

2.2.1 · AI-ML › Linear & Logistic Regression

Overview

Simple linear regression ek single independent variable $x$ aur dependent variable $y$ ke beech ke relationship ko ek straight line se model karta hai. Yeh predictive modeling ki neenv hai aur sabhi supervised learning ko samajhne ka darwaza hai.

[!intuition] Core Idea

Socho tum ghar ke daam predict kar rahe ho. Tumne notice kiya: bade ghar zyada mehange hote hain. Simple linear regression tumhare (size, price) data points ke through ek "best-fit" straight line kheenchti hai. Jab yeh line mil jaaye, tum nayi prices line se padh ke predict karte ho.

Straight line kyun? Kyunki hum ek proportional relationship model kar rahe hain: har extra square foot price mein lagbhag utni hi rakam badhata hai. Duniya hamesha linear nahi hoti, lekin linear models:

Interpret karne mein aasaan hote hain (slope = "price per sq ft")
Compute karne mein fast hote hain
Bahut se real phenomena ke liye surprizingly effective hote hain

[!definition] Mathematical Model

Simple linear regression model yeh hai:

$y = β_{0} + β_{1} x + ϵ$

Jahan:

$y$ dependent variable (target, jo hum predict karte hain)
$x$ = independent variable (feature, predictor)
$β_{0}$ = intercept ( $y$ ki value jab $x = 0$ )
$β_{1}$ = slope ( $x$ mein ek unit change par $y$ mein change)
$ϵ$ = error term (random noise, jo line chhodd deti hai use capture karta hai)

Key assumptions:

Linearity: Saccha relationship roughly linear hai
Independence: Observations ek doosre ko affect nahi karte
Homoscedasticity: Saare $x$ ke liye error variance constant hai
Normality: Errors $ϵ$ , $N (0, σ^{2})$ follow karte hain

[!formula] Best-Fit Line Derive Karna (Ordinary Least Squares)

Goal: $β_{0}, β_{1}$ dhundho jo prediction errors minimize karein.

Step 1: Loss Function Define Karo

$n$ training points $(x_{i}, y_{i})$ ke liye, hamara prediction hai $\overset{y}{^}_{i} = β_{0} + β_{1} x_{i}$ .

Point $i$ ke liye residual (error) hai: $e_{i} = y_{i} - \overset{y}{^}_{i}$

Hum chhote errors chahte hain, isliye sum of squared residuals (RSS) minimize karo:

$RSS (β_{0}, β_{1}) = \sum_{i = 1}^{n} (y_{i} - β_{0} - β_{1} x_{i})^{2}$

Errors ko square kyun karein?

Squaring se saare errors positive ho jaate hain ( $- 5$ error utna hi bura hai jitna $+ 5$ )
Squaring se bade errors ko zyada penalty milti hai (outliers hurt karte hain)
Squaring se math differentiable ho jaata hai (hum calculus use kar sakte hain)

Step 2: Partial Derivatives Lo

Minimize karne ke liye, $\frac{\partial RSS}{\partial β _{0}} = 0$ aur $\frac{\partial RSS}{\partial β _{1}} = 0$ set karo.

$β_{0}$ ke saath respect karke derivative: $\frac{\partial RSS}{\partial β _{0}} = \frac{\partial}{\partial β _{0}} \sum_{i = 1}^{n} (y_{i} - β_{0} - β_{1} x_{i})^{2}$

Chain rule use karke: $\frac{\partial}{\partial β _{0}} (y_{i} - β_{0} - β_{1} x_{i})^{2} = 2 (y_{i} - β_{0} - β_{1} x_{i}) \cdot (- 1)$

$= - 2 \sum_{i = 1}^{n} (y_{i} - β_{0} - β_{1} x_{i}) = 0$

$- 2$ se divide karo: $\sum_{i = 1}^{n} (y_{i} - β_{0} - β_{1} x_{i}) = 0$

Expand karo: $\sum_{i = 1}^{n} y_{i} - n β_{0} - β_{1} \sum_{i = 1}^{n} x_{i} = 0$

Yeh step kyun? Saare residuals sum karke zero set karne ka matlab hai ki line data ke "center" se guzarti hai.

$β_{0}$ ke liye solve karo: $β_{0} = \frac{1}{n} \sum_{i = 1}^{n} y_{i} - β_{1} \frac{1}{n} \sum_{i = 1}^{n} x_{i} = \overset{y}{ˉ} - β_{1} \overset{x}{ˉ}$

jahan $\overset{x}{ˉ} = \frac{1}{n} \sum x_{i}$ aur $\overset{y}{ˉ} = \frac{1}{n} \sum y_{i}$ means hain.

$β_{1}$ ke saath respect karke derivative: $\frac{\partial RSS}{\partial β _{1}} = - 2 \sum_{i = 1}^{n} (y_{i} - β_{0} - β_{1} x_{i}) \cdot x_{i} = 0$

$\sum_{i = 1}^{n} x_{i} (y_{i} - β_{0} - β_{1} x_{i}) = 0$

$β_{0} = \overset{y}{ˉ} - β_{1} \overset{x}{ˉ}$ substitute karo: $\sum_{i = 1}^{n} x_{i} [y_{i} - (\overset{y}{ˉ} - β_{1} \overset{x}{ˉ}) - β_{1} x_{i}] = 0$

$\sum_{i = 1}^{n} x_{i} (y_{i} - \overset{y}{ˉ}) - β_{1} \sum_{i = 1}^{n} x_{i} (x_{i} - \overset{x}{ˉ}) = 0$

Yeh step kyun? Hum data ko uske mean ke around center kar rahe hain, jo covariance aur variance tak simplify ho jaata hai.

Note: $\sum x_{i} \overset{x}{ˉ} = \sum^{ˉ} x_{i} = n \overset{x}{ˉ}^{2}$ , aur $\sum x_{i} y_{i} - \sum x_{i} \overset{y}{ˉ} = \sum x_{i} (y_{i} - \overset{y}{ˉ})$

$β_{1}$ ke liye solve karo: $β_{1} = \frac{\sum _{i = 1}^{n} ( x _{i} - x ˉ ) ( y _{i} - y ˉ )}{\sum _{i = 1}^{n} ( x _{i} - x ˉ ) ^{2}}$

Statistics ke terms mein: $β_{1} = \frac{Cov ( x , y )}{Var ( x )} = \frac{s _{x y}}{s _{x}^{2}}$

Final OLS formulas: $β_{1} = \frac{\sum _{i = 1}^{n} ( x _{i} - x ˉ ) ( y _{i} - y ˉ )}{\sum _{i = 1}^{n} ( x _{i} - x ˉ ) ^{2}}$

$β_{0} = \overset{y}{ˉ} - β_{1} \overset{x}{ˉ}$

[!example] Worked Example 1: House Prices

Data: 5 ghar size (100s of sq ft) aur price (1000s of $) ke saath

$x$ (size)	$y$ (price)
10	250
15	350
20	450
25	550
30	650

Step 1: Means calculate karo $\overset{x}{ˉ} = \frac{10 + 15 + 20 + 25 + 30}{5} = 20$ $\overset{y}{ˉ} = \frac{250 + 350 + 450 + 550 + 650}{5} = 450$

Step 2: Slope $β_{1}$ calculate karo

$x_{i}$	$y_{i}$	$x_{i} - \overset{x}{ˉ}$	$y_{i} - \overset{y}{ˉ}$	$(x_{i} - \overset{x}{ˉ}) (y_{i} - \overset{y}{ˉ})$	$(x_{i} - \overset{x}{ˉ})^{2}$
10	250	-10	-200	2000	100
15	350	-5	-100	500	25
20	450	0	0	0
25	550	5	100	500	25
30	650	10	2000	100
Sum				5000	250

$β_{1} = \frac{5000}{250} = 20$

Yeh value kyun? Har 100 sq ft increase price ko $20k badhata hai. Intuitive sense banata hai!

Step 3: Intercept $β_{0}$ calculate karo $β_{0} = 450 - 20 \times 20 = 450 - 400 = 50$

Final model: $\overset{y}{^} = 50 + 20 x$

Interpretation: 0 sq ft ke ghar ka (jo exist nahi karta!) cost $50 k h o g a (s ha y a d z am ee nk i v a l u e ?) . H a r 100 s q f t$ 20k badhata hai.

Prediction: 200 sq ft ke ghar ke liye ( $x = 22$ ): $\overset{y}{^} = 50 + 20 (22) = 490 thousand dollars$

[!example] Worked Example 2: Study Hours vs Exam Score

Data: 4 students

Study Hours ( $x$ )	Score ( $y$ )
2	50
4	65
6	80
8	95

Step 1: $\overset{x}{ˉ} = 5$ , $\overset{y}{ˉ} = 72.5$

Step 2:

$\sum (x_{i} - \overset{x}{ˉ}) (y_{i} - \overset{y}{ˉ}) = (- 3) (- 22.5) + (- 1) (- 7.5) + (1) (7.5) + (3) (22.5)$ $= 67.5 + 7.5 + 67.5 = 150$

$\sum (x_{i} - \overset{x}{ˉ})^{2} = 9 + 1 + 1 + 9 = 20$

$β_{1} = \frac{150}{20} = 7.5$

Step 3: $β_{0} = 72.5 - 7.5 (5) = 35$

Model: $\overset{y}{^} = 35 + 7.5 x$

Matlab: Har extra study hour 7.5 points badhata hai. 0 hours ke saath, base score 35 hai (purani knowledge?).

[!formula] Model Evaluation Metrics

1. R-squared (Coefficient of Determination)

$R^{2} = 1 - \frac{RSS}{TSS} = 1 - \frac{\sum ( y _{i} - y ^ _{i} ) ^{2}}{\sum ( y _{i} - y ˉ ) ^{2}}$

jahan TSS (Total Sum of Squares) = $\sum (y_{i} - \overset{y}{ˉ})^{2}$ jo $y$ mein total variance measure karta hai.

Matlab ki derivation:

RSS = unexplained variance (jo model chhodd gaya)
TSS = data mein total variance
$\frac{RSS}{TSS}$ = unexplained fraction
$R^{2}$ = explained fraction (0 aur 1 ke beech)

Yeh metric kyun? $R^{2} = 0.85$ ka matlab hai " $y$ mein 85% variance $x$ se explain hoti hai". Zyada better hota hai.

2. Root Mean Squared Error (RMSE)

$RMSE = \frac{1}{n} \sum_{i = 1}^{n} (y_{i} - \overset{y}{^}_{i})^{2}$

Yeh metric kyun? RMSE $y$ ke same units mein hota hai. Agar price dollars mein predict kar rahe ho, toh RMSE bhi dollars mein hoga. "Average par, predictions $\pm RMSE$ se off hoti hain."

[!mistake] Common Pitfalls

Mistake 1: Data Range se Bahar Extrapolation

Galat soch: "Mera model 1000-3000 sq ft ke ghar ke liye kaam karta hai, toh main 10,000 sq ft ke mansion ke liye bhi predict karunga."

Kyun sahi lagta hai: Formula $\overset{y}{^} = β_{0} + β_{1} x$ kisi bhi $x$ ke liye kaam karta hai!

Fix: Linearity tumhari data range ke bahar nahi chalti. Bahut bade ghar ki alag price dynamics hoti hai (luxury market). Sirf observed $x$ range ke andar interpolate karo.

Mistake 2: Correlation ≠ Causation

Galat soch: "Ice cream sales aur drowning deaths correlated hain aur linear model mein fit hote hain, isliye ice cream drowning ka cause hai!"

Kyun sahi lagta hai: Math valid hai; regression line exist karti hai.

Fix: Regression association dhundta hai, causation nahi. Dono ek confounding variable (garmi) ki wajah se hote hain. Causation claim karne ke liye controlled experiments ya causal inference methods chahiye.

Mistake 3: Outliers ko Ignore Karna

Galat soch: "Ek data point bahut alag hai, lekin main ise waise hi rakhunga."

Kyun sahi lagta hai: Zyada data better hota hai, na?

Fix: Outliers line ko distort karte hain (squared errors unka influence amplify karte hain). Outliers investigate karo: data entry errors hain? Ya legitimate extreme cases hain? Robust regression methods consider karo ya justification ke baad remove karo.

Mistake 4: Check Kiye Bina Linearity Assume Karna

Galat soch: "Maine ek line fit ki, toh relationship linear hai."

Kyun sahi lagta hai: Chahe kuch bhi ho line milti hi hai; OLS hamesha $β_{0}, β_{1}$ return karta hai.

Fix: Residuals vs fitted values plot karo. Random scatter = achha. Patterns (curve, funnel shape) = violated assumptions. Data transform karo (log, sqrt) ya polynomial/nonlinear regression use karo.

[!recall]- 12 Saal Ki Umar Mein Samjhao

Socho tum apne doston ki height guess karne ki koshish kar rahe ho unka shoe size dekhkar.

Tumne notice kiya: bade shoes wale log usually lambe hote hain. Toh tum graph paper par ek straight line kheenchte ho jahan tumne sabka (shoe size, height) plot kiya hai.

Ab jab size 7 shoes wala naya bachha aata hai, tum apni line par 7 dhundh ke unki predicted height padh lete ho!

Simple linear regression bas tumhare dots ke through BEST line dhundhna hai. "Best" ka matlab hai wo line jo saare dots ke sabse kareeb ho (average par sabse chhoti mistakes).

Slope tumhe batata hai "har shoe size par kitne inches zyada lambe", aur intercept wahan hai jahan line height axis ko cross karti hai (halanki size 0 shoes wala baby actually utna lamba nahi hoga—line sirf ek model hai!).

[!mnemonic] SLOPE

Sum of (x - mean)(y - mean)
Less
Over
Product of (x - mean)²
Equals beta₁

Intercept: "y-bar minus beta times x-bar" (sounds like "why bother? Beat times X!")

Connections

2.1-Introduction-to-Regression: Broad context
2.2.02-Multiple-Linear-Regression: Multiple features tak extension
2.3-Gradient-Descent-for-Linear-Regression: Iterative optimization approach
2.5-Assumptions-and-Diagnostics: Model validity check karna
3.1-Logistic-Regression-Model: Classification tak extension
4.2-Overfitting-vs-Underfitting: Linear models mein bias-variance
1.3-Loss-Functions: RSS ek specific loss function hai

#flashcards/ai-ml

Simple linear regression kya model karta hai? :: Ek method jo single independent variable x aur dependent variable y ke beech ke relationship ko straight line equation y = β₀ + β₁x + ε se model karta hai

y = β₀ + β₁x + ε mein parameters kya hain?

β₀ intercept hai (x=0 par y-value), β₁ slope hai (x ki har unit par y mein change), ε random error term hai

Ordinary Least Squares (OLS) kya minimize karta hai?

Sum of squared residuals RSS = Σ(yᵢ - ŷᵢ)², jahan residuals actual aur predicted values ke beech ke differences hain

β₁ ke liye OLS formula first principles se derive karo :: ∂RSS/∂β₁ = 0 lo, β₀ = ȳ - β₁x̄ substitute karo, solve karo to β₁ = Σ(xᵢ-x̄)(yᵢ-ȳ) / Σ(xᵢ-x̄)² = Cov(x,y)/Var(x) milta hai

Intercept β₀ ke liye OLS formula kya hai?

β₀ = ȳ - β₁x̄, jo ensure karta hai ki regression line point (x̄, ȳ) se guzre

OLS mein residuals ko square kyun karte hain?

(1) Saare errors positive ho jaate hain, (2) Bade errors ko zyada heavily penalize karta hai, (3) Loss function ko calculus-based optimization ke liye differentiable banata hai

R² kya measure karta hai?

Coefficient of determination: R² = 1 - RSS/TSS, jo x se y mein explain hone wali variance ka proportion represent karta hai (0 se 1 ke beech)

RMSE kya measure karta hai aur ise kyun use karte hain?

Root Mean Squared Error = √(Σ(yᵢ-ŷᵢ)²/n), y ke same units mein average prediction error deta hai, jo ise interpretable banata hai

Simple linear regression ki chaar key assumptions kya hain?

(1) Linearity: saccha relationship linear hai, (2) Independence: observations ek doosre ko affect nahi karte, (3) Homoscedasticity: constant error variance, (4) Normality: errors N(0,σ²) follow karte hain

Data range se bahar extrapolation dangerous kyun hai?

Observed x values ke bahar linear relationship hold nahi kar sakta; alag dynamics apply ho sakti hain (jaise bade ghar ke liye luxury market)

Confounding variable kya hota hai aur yeh kyun matter karta hai?

Ek aisa variable jo x aur y dono ko influence karta hai, spurious correlation create karta hai (jaise garmi ice cream sales aur drowning dono cause karti hai); regression association dikhata hai, causation nahi

Outliers linear regression ko kaise affect karte hain?

Squared errors outlier ka influence amplify karte hain, fitted line ko distort karte hain; remove karne ka decide karne se pehle investigate karo ki woh errors hain ya legitimate extreme cases

Linearity assumption hold hoti hai ya nahi yeh kaise check karein?

Residuals vs fitted values plot karo; random scatter achha fit indicate karta hai, patterns (curves, funnels) violated assumptions indicate karte hain jiske liye transformation ya nonlinear models chahiye

House price model mein slope β₁ = 20 ka kya matlab hai?

x mein har ek unit increase (jaise 100 sq ft) predicted y (price) ko 20 units (jaise $20k) badhata hai

Study hours vs score ke model ŷ = 35 + 7.5x mein intercept ka kya matlab hai?

0 study hours ke saath, baseline predicted score 35 hai (possibly prior knowledge ya guessing ability represent karta hai)