Foundations — Linear regression — least squares, inference on coefficients

4.9.22 · D1 · Maths › Probability Theory & Statistics › Linear regression — least squares, inference on coefficients

Parent note padhne se pehle, tumhe har ek squiggle ko apna banana hoga jo woh likhti hai. Neeche, har symbol ko teen cheezein milti hain: seedhe alfaaz, ek picture, aur kyun is topic ko iska zaroorat hai. Yeh is order mein hain ki har ek sirf upar waalon par rely karta hai.

1. Kacche ingredients: paper par dots

Picture: har pair graph paper par pina hua ek single dot hai. $x_i$ neeche se padhein, $y_i$ side se.

Topic ko kyun chahiye: poora kaam "dots ke cloud mein se line dhoondho" hai. Dots nahi, toh fit karne ko kuch nahi.

Picture: socho jaise ek-ek karke dots ke saath chal rahe ho, aur har dot ki value ek running total mein daal rahe ho.

Kyun: topic ka har formula sabhi dots par liya gaya ek total hai — yeh sign sirf hamein "$+\cdots +$" likhne se bachata hai.

2. Cloud kahaan baitha hai: means

Picture: $\bar{x}$ dots ka horizontal balance-point hai (heights ignore karo); $\bar{y}$ vertical balance-point hai. Pair $(\bar{x},\bar{y})$ centre of mass mark karta hai — woh jagah jahan cloud ek pin par balance hoga.

Kyun: best-fit line ko $(\bar{x},\bar{y})$ se guzarna zaroor hai. Pehle centre jaanna baad ke har formula ko simpler banata hai, kyunki hum sab kuch is centre ke relative mein measure karte hain.

Picture: apna graph slide karo taaki centre of mass origin par aa jaaye. Ab har dot ke coordinates uske centred values hain — upper-right mein ek dot ke dono positive hain, lower-left mein dono negative.

Kyun: in centred values ka sign slope ka engine hai (agla section). Jo dots right-and-high ya left-and-low hain woh line ko upar kheenchenge.

3. Spread aur co-movement: $S_{xx}$ aur $S_{xy}$

Picture: har dot ke liye centre line se dot ki $x$-position tak ek horizontal stick kheencho; $S_{xx}$ un squared stick lengths ka total hai. Door left/right ke dots ise bada karte hain; beech mein ikatte dots ise chhota karte hain.

Kyun: wide horizontal spread ek lamba lever hai — yeh tumhe line ki tilt ko precisely aim karne deta hai. Yahi quantity baad mein slope ki precision ke neeche baithti hai.

Picture (zaroori): har product ka sign ek kahani batata hai.

Kyun: yeh tally slope ka numerator hai. Yeh Covariance (jo $S_{xy}/n$ hai) aur Correlation coefficient (jo ise $[-1,1]$ mein rescale karta hai) ka close cousin hai.

4. Line aur uske parts: ${\beta }_0,{\beta }_1$, hats, aur residuals

Picture: $x=0$ par khade ho; line ${\beta }_0$ height par hai. Ek step daayein chalo; line ${\beta }_1$ se upar jaati hai.

Picture: true line invisible hai; hatted line woh hai jo hum actually cloud mein se kheenchte hain.

Picture: $e_i$ dot se neeche (ya upar) fitted line tak vertical segment ki length hai — "miss." ${\epsilon }_i$ same idea hai par invisible true line ke against measure ki gayi.

5. Woh quantity jo hum minimise karte hain, aur us se nikalna slope formula

Picture: figure s04 ke har residual stick ko ek square mein badlo (side = miss); $S$ un saare squares ka total area hai. Behtar aimed line squares ko chhota karti hai, toh $S$ girta hai.

Kyun: least squares define hota hai "woh ${\hat{\beta }}_0,{\hat{\beta }}_1$ choose karo jo $S$ ko jitna possible ho utna chhota kare." Toh $S$ target hai — woh cheez hi jisko har derivation neeche dhakelta rehta hai. "Minimise the total squared residuals" aur symbol $S$ bilkul ek hi cheez matlab rakhte hain.

Picture: $S$ $({\beta }_0,{\beta }_1)$ plane ke upar ek bowl-shaped surface hai; minimum bowl ka sabse neecha point hai, jahan har direction mein zameen flat ho.

Dono partials ko zero set karke solve karna estimates ko un pieces ke terms mein pin down karta hai jo hum pehle se bana chuke hain:

Yeh abhi kyun important hai: parent topic in boxed formulas ko calculus se derive karta hai, par tumhe derivation se pehle unme har symbol pehchaan lena chahiye — aur woh ek input configuration ($S_{xx}=0$) jaanni chahiye jo unhe tod deti hai.

6. Trust language: variance, ${\sigma }^2$, standard error, aur $t$

Picture: true line ke around scatter ka mota vertical band bada $\sigma$ matlab hai; patla band chhota $\sigma$ matlab hai.

Picture: kai repeats se jo bahut saari lines milenge unhe overlay karo; woh ek fan banati hain. Narrow fan = chhota SE = trustworthy slope; wide fan = bada SE.

Kyun: summed squared residuals ($S$ phir se!) ko $n-2$ se divide karna (na ki $n$ se) ${\sigma }^2$ ka unbiased estimate deta hai.

Kyun: kyunki humein $\sigma$ usi chhote sample se estimate karna pada, bell curve ke tails mote ho jaate hain — woh moti curve $t$-distribution hai.

Yeh claim ki yeh fit best possible unbiased linear hai Gauss–Markov Theorem se aata hai; ek input $x$ se kai tak extend karna Multiple Regression ka kaam hai.

Prerequisite map

Baayein sab kuch line banata hai; daayein sab kuch us line par trust banata hai. Saath mein yeh bilkul wahi do sawaal hain jo parent topic ka jawaab deta hai.

Equipment checklist

Daayein side cover karo aur khud ko test karo.