Question bank — Linear regression — least squares, inference on coefficients

False. Yeh vertical miss $y_i-{\hat{y}}_i$ squared minimize karta hai. Perpendicular distance $x$ aur $y$ ko symmetrically treat karta hai (woh total least squares / errors-in-variables hai), lekin ordinary regression assume karta hai ki $x$ fixed hai aur sirf $y$ mein noise hai.

True. Pehla normal equation $n$ se divide karne par $\bar{y}={\hat{\beta }}_0+{\hat{\beta }}_1\bar{x}$ milta hai, isliye cloud ka center of mass hamesha line par hota hai — line usi ke around pivot karti hai.

False. $x$ ko $y$ par regress karna horizontal misses minimize karta hai aur slope $S_{xy}/S_{yy}$ deta hai, na ki $1/(S_{xy}/S_{xx})$. Dono lines alag hoti hain jab tak fit perfect na ho; dono $(\bar{x},\bar{y})$ se guzarti hain lekin alag tilt hoti hain.

False. Zero least-squares slope matlab zero linear co-movement ($S_{xy}=0$). Ek perfect U-shaped (quadratic) relation mein $S_{xy}=0$ ho sakta hai phir bhi $x$ puri tarah $y$ determine karta hai — strong dependence, koi straight-line signal nahi.

Ordinary least squares with an intercept ke liye False. Kyunki $R^2=\text{SSR}/\text{SST}$ aur fitted line flat line $\hat{y}=\bar{y}$ se badtar kabhi perform nahi kar sakti, hamesha $0\le R^2\le 1$ hota hai. (Ek forced no-intercept ya externally-imposed model mein negative ja sakta hai, lekin woh standard setup se bahar hai.)

True. $x$ ko rescale karne par $S_{xx}$ ko $4$ se multiply karta hai aur $S_{xy}$ ko $2$ se, isliye ${\hat{\beta }}_1$ half ho jaata hai lekin fitted values ${\hat{y}}_i$ identical rehti hain — SSR, SSE, SST sab unchanged, hence $R^2$ unchanged.

False. Unbiased matlab $E[{\hat{\beta }}_1]={\beta }_1$ — estimate bahut saare samples ke average par sahi hota hai. Koi bhi single estimate almost surely miss karta hai; SE humein roughly batata hai kitna.

False. Gauss–Markov ko sirf zero-mean, constant-variance, uncorrelated errors chahiye taaki least squares BLUE bane. Gaussianity sirf test statistic ki t-distribution aur exact confidence intervals ke liye chahiye.

True. Ek predictor ke saath, SSR/SST algebraically $r^2$ mein reduce ho jaata hai jahan $r=S_{xy}/\sqrt{S_{xx}{S}_{yy}}$ — dekho Correlation coefficient. Yeh ek se zyada predictor hone par fail ho jaata hai.

False. Width uncertainty measure karta hai, magnitude nahi. Ek wide CI (chhota $S_{xx}$, bada noise, chhota $n$) matlab hum simply slope pin nahi kar sakte — woh chhota, bada, ya zero kuch bhi ho sakta hai.

Ek high $R^2$ kehta hai ki line zyaadaatar variance explain karti hai, shape sahi hai nahi. Curved data mein bhi high $R^2$ aa sakta hai; sirf residual plot woh leftover pattern reveal karta hai jo line miss kar gayi.

Wrong divisor. Residuals do constraints follow karte hain ($\sum e_i=0$, $\sum x_i{e}_i=0$), sirf $n-2$ free pieces of information bachti hain, isliye $n$ se divide karna ${\sigma }^2$ underestimate karta hai. $\text{SSE}/(n-2)$ use karo.

Significance (chhota SE relative to ${\hat{\beta }}_1$) matlab effect precisely measured to be non-zero hai, large nahi. Huge $n$ ke saath ek trivially chhota slope significant ho sakta hai lekin practically meaningless.

Slope genuinely ek signed quantity hai: negative matlab $y$ tend karta hai girne ko jab $x$ badhta hai. Ise flip karna model ka meaning badal deta hai; yeh apna sign seedha $S_{xy}$ se inherit karta hai.

Yeh ${\beta }_1$ ke liye parameter interval hai, data interval nahi. Individual future observations ke liye ranges (bahut zyada wide) prediction intervals hoti hain jo estimation uncertainty ke upar noise ka ${\sigma }^2$ add karti hain.

Yeh generally correlated hote hain: inki covariance $-{\sigma }^2\bar{x}/S_{xx}$ hai, zero sirf tab jab $\bar{x}=0$. Uncorrelated errors ka matlab uncorrelated coefficient estimates nahi hota.

${\hat{\beta }}_0=0$ force karne par line $(\bar{x},\bar{y})$ se guzarna guarantee nahi hota, SST = SSR + SSE split toot jaata hai, aur slope badly bias ho sakta hai agar true intercept non-zero ho.

Squaring smooth aur differentiable hai (isliye calculus ek clean closed-form solution deta hai), large misses zyada punish karta hai, aur Gaussian noise ke under maximum likelihood ke saath coincide karta hai. Absolute error bhi "misses shrink" karta hai lekin koi tidy formula nahi hai aur fit unique nahi hoti.

$\text{Var}({\hat{\beta }}_1)={\sigma }^2/S_{xx}$, isliye bada $S_{xx}$ variance shrink karta hai. Geometrically, widely-spaced $x$'s line ko ek lamba lever arm dete hain — same vertical noise ek lambi line ko ek chhoti se bahut kam tilt karta hai.

Sample variance ek quantity (mean) estimate karta hai, ek degree of freedom loose karta hai. Yahan humne do parameters, ${\hat{\beta }}_0$ aur ${\hat{\beta }}_1$, estimate kiye, isliye do constraints consume ho gaye aur $n-2$ pieces of independent residual information bachti hain.

Kyunki residual vector $n$-dimensional space mein fitted-values vector ke orthogonal hai (normal equations ka consequence). Yeh ek Pythagoras hai: cross term $\sum ({\hat{y}}_i-\bar{y})(y_i-{\hat{y}}_i)$ exactly vanish ho jaata hai.

Kyunki ${\sigma }^2$ unknown hai. Ise estimate $s^2$ se replace karne par extra randomness inject hoti hai, yahi reason hai ki standardised statistic normal ki jagah $n-2$ df ke saath t-distribution follow karta hai.

Kyunki ${\hat{\beta }}_1={\sum }_i{c}_i{y}_i$ noisy $y_i$'s ka weighted sum hai. Experiment dobara run karne par errors ${\epsilon }_i$ redraw hote hain, alag $y_i$ milte hain aur thus alag slope — yahi iska sampling variability ka source hai.

$S({\beta }_0,{\beta }_1)$ do parameters mein ek convex (upward-opening) paraboloid hai, isliye iska gradient exactly ek point par vanish hota hai — provided $x_i$ sab identical nahi hain (yani $S_{xx}>0$).

$S_{xx}=0$, isliye ${\hat{\beta }}_1=S_{xy}/S_{xx}$ undefined hai (divide by zero). Geometrically sab points ek vertical strip par hain — koi horizontal lever exist nahi karta tilt determine karne ke liye.

Line dono points se exactly guzarti hai (SSE$=0$, $R^2=1$ trivially), lekin $s^2=\text{SSE}/(n-2)=0/0$ undefined hai — zero degrees of freedom bachne par noise estimate nahi kar sakte.

$\text{SSE}=0$ se $R^2=1$ aur $s^2=0$, isliye $\text{SE}({\hat{\beta }}_1)=0$ — slope zero uncertainty ke saath estimate hota hai in-sample. Real data practically kabhi aisa nahi karti; yeh overfit ya deterministic relation ki nishani hai.

Nahi — noise phir bhi kisi finite sample mein ek chhota non-zero ${\hat{\beta }}_1$ produce karta hai. Hypothesis test poochta hai ki woh wobble SE se chance alone se zyada bada hai ya nahi.

Yeh zero ki taraf shrink hota hai, kyunki $S_{xx}$ unbounded grow karta hai jabki ${\sigma }^2$ fixed rehta hai. Slope estimate arbitrarily precise ho jaata hai — least-squares estimator ki consistency.

Kyunki squaring large vertical misses ko huge weight deta hai, ek distant point poori line ko apni taraf pivot kar sakta hai, SSE inflate karta hai aur ${\hat{\beta }}_1$ aur $R^2$ dono distort karta hai. Least squares outliers ke liye robust nahi hai.

Inki covariance $-{\sigma }^2\bar{x}/S_{xx}$ zero ho jaati hai, isliye intercept aur slope estimates uncorrelated hote hain — predictor ko centre karna inhe decouple karne ka ek common trick hai.