Question bank — Linear regression — least squares, inference on coefficients

False. It minimises the vertical miss $y_i-{\hat{y}}_i$ squared. Perpendicular distance treats $x$ and $y$ symmetrically (that's total least squares / errors-in-variables), but ordinary regression assumes $x$ is fixed and only $y$ carries noise.

True. The first normal equation divided by $n$ gives $\bar{y}={\hat{\beta }}_0+{\hat{\beta }}_1\bar{x}$, so the centre of mass of the cloud is always on the line — the line pivots about it.

False. Regressing $x$ on $y$ minimises horizontal misses and gives slope $S_{xy}/S_{yy}$, not $1/(S_{xy}/S_{xx})$. The two lines differ unless the fit is perfect; both pass through $(\bar{x},\bar{y})$ but tilt differently.

False. A zero least-squares slope means zero linear co-movement ($S_{xy}=0$). A perfect U-shaped (quadratic) relation can have $S_{xy}=0$ yet $x$ fully determines $y$ — strong dependence, no straight-line signal.

False for ordinary least squares with an intercept. Because $R^2=\text{SSR}/\text{SST}$ and the fitted line can never do worse than the flat line $\hat{y}=\bar{y}$, we always have $0\le R^2\le 1$. (A forced no-intercept or externally-imposed model can go negative, but that's outside the standard setup.)

True. Rescaling $x$ multiplies $S_{xx}$ by $4$ and $S_{xy}$ by $2$, so ${\hat{\beta }}_1$ halves but the fitted values ${\hat{y}}_i$ are identical — SSR, SSE, SST all unchanged, hence $R^2$ unchanged.

False. Unbiased means $E[{\hat{\beta }}_1]={\beta }_1$ — the estimate is right on average across many samples. Any single estimate almost surely misses; the SE tells us by roughly how much.

False. Gauss–Markov needs only zero-mean, constant-variance, uncorrelated errors to make least squares BLUE. Gaussianity is needed only for the t-distribution of the test statistic and exact confidence intervals.

True. With one predictor, SSR/SST reduces algebraically to $r^2$ where $r=S_{xy}/\sqrt{S_{xx}{S}_{yy}}$ — see Correlation coefficient. This fails once there is more than one predictor.

False. Width measures uncertainty, not magnitude. A wide CI (small $S_{xx}$, big noise, small $n$) means we simply can't pin the slope down — it could be small, large, or zero.

A high $R^2$ says a line explains most of the variance, not that the shape is right. Curved data can still yield high $R^2$; only a residual plot reveals the leftover pattern the line missed.

Wrong divisor. The residuals obey two constraints ($\sum e_i=0$, $\sum x_i{e}_i=0$), leaving only $n-2$ free pieces of information, so dividing by $n$ underestimates ${\sigma }^2$. Use $\text{SSE}/(n-2)$.

Significance (small SE relative to ${\hat{\beta }}_1$) means the effect is precisely measured to be non-zero, not that it is large. A trivially small slope with huge $n$ can be significant yet practically meaningless.

The slope is a genuine signed quantity: negative means $y$ tends to fall as $x$ rises. Flipping it changes the model's meaning; it inherits its sign straight from $S_{xy}$.

That's a parameter interval for ${\beta }_1$, not a data interval. Ranges for individual future observations are (much wider) prediction intervals that add the ${\sigma }^2$ of the noise on top of estimation uncertainty.

They are generally correlated: their covariance is $-{\sigma }^2\bar{x}/S_{xx}$, zero only when $\bar{x}=0$. Uncorrelated errors do not imply uncorrelated coefficient estimates.

Forcing ${\hat{\beta }}_0=0$ no longer guarantees the line passes through $(\bar{x},\bar{y})$, breaks the SST = SSR + SSE split, and can badly bias the slope if the true intercept is non-zero.

Squaring is smooth and differentiable (so calculus gives a clean closed-form solution), punishes large misses more, and coincides with maximum likelihood under Gaussian noise. Absolute error also "shrinks misses" but has no tidy formula and a non-unique fit.

$\text{Var}({\hat{\beta }}_1)={\sigma }^2/S_{xx}$, so large $S_{xx}$ shrinks the variance. Geometrically, widely-spaced $x$'s give the line a long lever arm — the same vertical noise tilts a long line far less than a short one.

A sample variance estimates one quantity (the mean), losing one degree of freedom. Here we estimated two parameters, ${\hat{\beta }}_0$ and ${\hat{\beta }}_1$, so two constraints are consumed and $n-2$ pieces of independent residual information remain.

Because the residual vector is orthogonal to the fitted-values vector in $n$-dimensional space (a consequence of the normal equations). It's a Pythagoras: the cross term $\sum ({\hat{y}}_i-\bar{y})(y_i-{\hat{y}}_i)$ vanishes exactly.

Because ${\sigma }^2$ is unknown. Replacing it with the estimate $s^2$ injects extra randomness, which is precisely why the standardised statistic follows a t-distribution with $n-2$ df rather than a normal.

Because ${\hat{\beta }}_1={\sum }_i{c}_i{y}_i$ is a weighted sum of the noisy $y_i$'s. Re-running the experiment redraws the errors ${\epsilon }_i$, giving different $y_i$ and thus a different slope — that's the source of its sampling variability.

$S({\beta }_0,{\beta }_1)$ is a convex (upward-opening) paraboloid in the two parameters, so its gradient vanishes at exactly one point — provided the $x_i$ are not all identical (i.e. $S_{xx}>0$).

$S_{xx}=0$, so ${\hat{\beta }}_1=S_{xy}/S_{xx}$ is undefined (divide by zero). Geometrically all points sit on one vertical strip — no horizontal lever exists to determine a tilt.

The line passes exactly through both points (SSE$=0$, $R^2=1$ trivially), but $s^2=\text{SSE}/(n-2)=0/0$ is undefined — you cannot estimate noise with zero degrees of freedom left over.

$\text{SSE}=0$ gives $R^2=1$ and $s^2=0$, so $\text{SE}({\hat{\beta }}_1)=0$ — the slope is estimated with zero uncertainty in-sample. Real data essentially never do this; it's a sign of overfit or a deterministic relation.

No — noise still produces a small non-zero ${\hat{\beta }}_1$ in any finite sample. The hypothesis test asks whether that wobble is larger than the SE would explain by chance alone.

It shrinks toward zero, because $S_{xx}$ grows without bound while ${\sigma }^2$ stays fixed. The slope estimate becomes arbitrarily precise — consistency of the least-squares estimator.

Because squaring gives huge weight to large vertical misses, one distant point can pivot the whole line toward itself, inflating SSE and distorting both ${\hat{\beta }}_1$ and $R^2$. Least squares is not robust to outliers.

Their covariance $-{\sigma }^2\bar{x}/S_{xx}$ becomes zero, so the intercept and slope estimates are uncorrelated — centring the predictor is a common trick to decouple them.