Question bank — Multiple regression

False. Ye $x_j$ ka effect hai baaki predictors ko fixed rakhte hue — uska partial (unique) contribution, isolated effect nahi.

True — lekin dhyan raho condition sample orthogonality hai (${\mathbf{x}}_1^{\top }{\mathbf{x}}_2=0$ actual design matrix mein), sirf population level par "uncorrelated" hona kaafi nahi. Tabhi slopes decouple hote hain aur $x_1$ ka estimate $x_2$ ke saath ya uske bina same rehta hai; finite-sample mein columns ke beech correlation generally ye todta hai.

False. Coefficients apne predictor ki units carry karte hain; hours ko minutes mein badlo toh coefficient $60\times$ shrink ho jaayega bina kisi real importance ke change ke. Standardised coefficients ya t-statistics compare karo.

$R_{\text{in}}^2$ (in-sample, intercept ke saath) ke liye, nahi — ye Pythagorean split ki wajah se $[0,1]$ mein trapped hai. Lekin $R_{\text{oos}}^2$ held-out data par (ya no-intercept model mein) $SSE>SST$ ho sakta hai, jo negative value deta hai: model flat mean line se bhi bura predict karta hai.

$R_{\text{in}}^2$ (in-sample, intercept ke saath) ke liye False: jab tum column add karte ho toh ye kabhi decrease nahi karta, kyunki fit sirf same ya better ho sakti hai. (Held-out data par, ya no-intercept models mein, out-of-sample version gir sakta hai.) Isliye hi humein ==adjusted $R^2$== chahiye, jo in-sample gir sakta hai.

True — ye normal equations ${\mathbf{X}}^{\top }(\mathbf{y}-\mathbf{X}\hat{\mathbf{\beta }})=0$ ka geometric meaning hai: least squares column space par ek perpendicular drop karta hai. Dekho Orthogonal Projection.

False. Estimator $\hat{\mathbf{\beta }}=({\mathbf{X}}^{\top }\mathbf{X}{)}^{-1}{\mathbf{X}}^{\top }\mathbf{y}$ ko koi normality ki zaroorat nahi. Normality sirf baad mein exact t-tests aur confidence intervals ke liye assume ki jaati hai. Gauss–Markov optimality ko sirf zero-mean, constant-variance, uncorrelated errors chahiye.

True. Invertibility matlab predictors perfectly collinear nahi hain, columns independent hain, projection ka ek hi representation hai, aur $\hat{\mathbf{\beta }}$ unique hai.

True. Kisi bhi vector $\mathbf{v}$ ke liye, ${\mathbf{v}}^{\top }{\mathbf{X}}^{\top }\mathbf{Xv}=\Vert \mathbf{Xv}{\Vert }^2\ge 0$, jo PSD ki definition hai. Ye strictly positive definite (hence invertible) hota hai exactly jab $\mathbf{X}$ full column rank par ho.

False. Gauss–Markov ke under OLS estimates unbiased rehte hain; multicollinearity unki variance inflate karta hai, unhe unstable aur trust karne ke liye mushkil banata hai — average par systematically galat nahi.

Galat grouping: hona chahiye $({\mathbf{X}}^{\top }\mathbf{X}{)}^{-1}$. $\mathbf{X}{\mathbf{X}}^{\top }$ $n\times n$ hai aur generally singular hota hai; ${\mathbf{X}}^{\top }\mathbf{X}$ wo chota $(p+1)\times (p+1)$ matrix hai jo actually invertible hota hai. Dekho Matrix Inverse.

Isse hyperplane origin se hokar jaati hai (${\beta }_0=0$), yaani ye predict karna padega $y=0$ jab sab $x=0$ hoon — almost hamesha galat aur ye baaki sab slopes ko corrupt karta hai.

Perfect correlation kisi column ko doosron ka linear combination banata hai, isliye ${\mathbf{X}}^{\top }\mathbf{X}$ singular hota hai (determinant zero) aur uska koi inverse nahi hota; koi unique solution tab tak nahi milti jab tak ek predictor drop ya combine na karo.

Zyada $R^2$ association/fit measure karta hai, causation nahi. Causal claims ke liye experimental design ya explicit causal assumptions chahiye, sirf acha fit nahi.

Residuals ka plain sum construction se zero hota hai (jab intercept ho) aur positives aur negatives cancel ho jaate hain. Hum squared residuals ka sum minimize karte hain, jo cancel nahi ho sakta.

In-sample $R_{\text{in}}^2$ hamesha badhta hai (ya same rehta hai) jab predictors add karo, isliye ye comparison khaali hai. Adjusted $R^2$ compare karo, ya better, cross-validation se out-of-sample error dekho.

Negative partial slope matlab: baaki predictors ko account karne ke baad, is sample mein zyada neend lower $y$ se associated hai. Sign flip multicollinearity ka classic symptom hai, corrupt data ka nahi — VIF check karo.

Squares smooth hote hain (har jagah differentiable), bade misses ko zyada penalise karte hain, aur normal equations ke zariye ek unique closed-form minimum dete hain. Absolute values ek non-smooth problem dete hain jisme koi simple formula nahi hota.

Kyunki $S(\mathbf{\beta })$ ek quadratic hai jiska Hessian $2{\mathbf{X}}^{\top }\mathbf{X}$ hai, jo PSD hai, isliye koi bhi stationary point global minimum hai. Jab $\mathbf{X}$ full column rank par ho toh Hessian positive definite hota hai (strictly convex), jo us minimiser ko unique banata hai; agar rank deficient hai toh minima ek flat set banate hain.

Ek simple regression slope $x_j$ ke direct effect aur uski correlation dono ko absorb karta hai; multiple-regression slope wo part remove karta hai jo doosre predictors se explain hota hai, $x_j$ ka unique contribution isolate karta hai.

Har row mein wo 1, ${\beta }_0$ ko multiply karta hai, isliye ${\beta }_0$ har prediction mein add hota hai — ye hyperplane ka vertical offset hai, $x$ values chahe jo bhi hon.

Adjusted $R^2$ har sum of squares ko uske degrees of freedom se divide karta hai; ek useless predictor ek degree of freedom khata hai, $SSE/(n-p-1)$ inflate karta hai aur adjusted value girata hai — chahe raw $R^2$ thoda sa badhta rahe.

Near-collinearity ${\mathbf{X}}^{\top }\mathbf{X}$ ko nearly singular banati hai, isliye $({\mathbf{X}}^{\top }\mathbf{X}{)}^{-1}$ ke huge entries ho jaate hain; kyunki $\text{Var}(\hat{\mathbf{\beta }})={\sigma }^2({\mathbf{X}}^{\top }\mathbf{X}{)}^{-1}$, wo bade entries directly coefficient variances inflate karte hain.

$\hat{\mathbf{y}}=\mathbf{X}\hat{\mathbf{\beta }}$, $\mathbf{X}$ ke column space ke andar $\mathbf{y}$ ke sabse kareeb point hai; Euclidean distance mein "sabse kareeb" matlab perpendicular drop karna, jo precisely orthogonal projection hai.

Pythagorean argument ko zaroorat hai ki mean vector $\bar{y}1$ column space mein ho; 1's column hi ye guarantee karta hai. Isse drop karo aur residual $\bar{y}1$ ke orthogonal nahi rehta, isliye cross-term vanish nahi hoti aur teen sums add hone band ho jaate hain.

Full rank ke saath hyperplane har point se guzarta hai, $SSE=0$ aur $R^2=1$ — lekin ye pure interpolation hai jisme error ke liye zero degrees of freedom hain; ye generalisation ke baare mein kuch nahi batata.

Tab $R^2=1-SSE/SST$ zero se divide karta hai — $R^2$ undefined (degenerate) hai. Geometrically koi variation explain nahi karni hai: best fit flat line $\hat{y}=\bar{y}$ hai jisme $SSE=0$ bhi hai, toh ratio $0/0$ hai. Ek meaningless $R^2$ ke bajaaye report karo ki response mein koi variance nahi hai.

Uska column 1's column ke proportional hoga, perfect collinearity create karta hai; ${\mathbf{X}}^{\top }\mathbf{X}$ singular ho jaata hai aur intercept/slope split undefined ho jaati hai jab tak constant column remove na karo.

Fitted values har point ke liye mean $\bar{y}$ ke barabar hain — best hyperplane flat hai, toh koi bhi predictor kisi variation explain nahi karta aur $SSR=0$ hai.

Woh same hote hain: orthogonal columns ke saath har slope independently compute hota hai, isliye multiple regression exactly wahi coefficients deta hai jo alag-alag har simple regression mein milte.

$\mathbf{X}$ full column rank nahi rakh sakta, ${\mathbf{X}}^{\top }\mathbf{X}$ singular hota hai, aur infinitely many $\hat{\mathbf{\beta }}$ $SSE=0$ achieve karte hain; OLS ka koi unique solution nahi hota aur regularisation ya kam predictors chahiye.

Haan — exactly jab predictors collinear hoin, toh column space ek se zyada tareekon se span hota hai; projection $\hat{\mathbf{y}}$ unique hoti hai lekin uske coordinates $\hat{\mathbf{\beta }}$ nahi hote.

Multiple regression simple linear regression mein reduce ho jaata hai: hyperplane ek straight line mein collapse ho jaata hai aur $({\mathbf{X}}^{\top }\mathbf{X}{)}^{-1}{\mathbf{X}}^{\top }\mathbf{y}$ jaana-pehchana slope-aur-intercept pair deta hai.