Simple linear regression model: Exercises — AI-ML deep dive

This page is a self-testing ladder. Every problem stands alone: read it, cover the solution, try it, then reveal. The rungs climb from "can you spot the pieces?" (L1) to "can you invent and defend a model choice?" (L5).

Everything here uses only tools built in the parent note: the model $y = β_{0} + β_{1} x + ϵ$ , the OLS formulas, and the evaluation metrics $R^{2}$ and RMSE. Nothing new is assumed — where a symbol reappears, we restate what it means so you never hit a wall.

Here $\overset{x}{ˉ}$ (say "x-bar") is just the average of the $x$ values, and $\overset{y}{ˉ}$ the average of the $y$ values. A residual $e_{i} = y_{i} - \overset{y}{^}_{i}$ is the vertical gap between a real point and the line's prediction — picture a little spring pulling the line toward the point.

Level 1 — Recognition

Goal: identify the parts of the model and read a fitted line. No heavy arithmetic.

Exercise L1.1

A fitted model is $\overset{y}{^} = 12 + 3 x$ . (a) What is the slope? (b) What is the intercept? (c) If $x$ increases by $1$ , how much does $\overset{y}{^}$ change?

Recall Solution

Compare to $\overset{y}{^} = β_{0} + β_{1} x$ . (a) Slope $β_{1} = 3$ (the number multiplying $x$ ). (b) Intercept $β_{0} = 12$ (the standalone number, the value of $\overset{y}{^}$ when $x = 0$ ). (c) A one-unit rise in $x$ multiplies through the slope: $\overset{y}{^}$ increases by $β_{1} = 3$ .

Exercise L1.2

For the same model $\overset{y}{^} = 12 + 3 x$ , predict $\overset{y}{^}$ when $x = 5$ .

Recall Solution

Substitute $x = 5$ : $\overset{y}{^} = 12 + 3 (5) = 12 + 15 = 27$ .

Exercise L1.3

Which of these is the residual for a point $(x_{i}, y_{i})$ with prediction $\overset{y}{^}_{i}$ ? (i) $\overset{y}{^}_{i} - \overset{y}{ˉ}$ (ii) $y_{i} - \overset{y}{^}_{i}$ (iii) $y_{i} - \overset{y}{ˉ}$

Recall Solution

(ii) $y_{i} - \overset{y}{^}_{i}$ . The residual is the vertical miss: the actual value minus what the line predicted. Option (i) is the part the model explains; option (iii) is the total deviation from the mean. (These three are the pieces of the $R^{2}$ story.)

Level 2 — Application

Goal: run the OLS machine end-to-end on small datasets.

Exercise L2.1

Fit a line to these 3 points by OLS: $(1, 2), (2, 2), (3, 5)$ . Find $β_{1}$ and $β_{0}$ .

Recall Solution

Means: $\overset{x}{ˉ} = \frac{1 + 2 + 3}{3} = 2$ , $\overset{y}{ˉ} = \frac{2 + 2 + 5}{3} = 3$ . Build the deviation columns:

$x_{i}$	$y_{i}$	$x_{i} - \overset{x}{ˉ}$	$y_{i} - \overset{y}{ˉ}$	product	$(x_{i} - \overset{x}{ˉ})^{2}$
1	2	$- 1$	$- 1$	$1$	$1$
2	2	$0$	$- 1$	$0$	$0$
3	5	$1$	$2$	$2$	$1$
Σ				$3$	$2$

Slope: $β_{1} = \frac{3}{2} = 1.5$ . Intercept: $β_{0} = \overset{y}{ˉ} - β_{1} \overset{x}{ˉ} = 3 - 1.5 (2) = 0$ . Model: $\overset{y}{^} = 0 + 1.5 x = 1.5 x$ .

Exercise L2.2

Using the model $\overset{y}{^} = 1.5 x$ from L2.1, compute each residual and confirm they sum to (approximately) zero.

Recall Solution

Predictions: $\overset{y}{^}_{1} = 1.5, \overset{y}{^}_{2} = 3, \overset{y}{^}_{3} = 4.5$ . Residuals $e_{i} = y_{i} - \overset{y}{^}_{i}$ : $e_{1} = 2 - 1.5 = 0.5$ $e_{2} = 2 - 3 = - 1$ $e_{3} = 5 - 4.5 = 0.5$ Sum: $0.5 - 1 + 0.5 = 0$ . ✅ Why zero? The OLS $β_{0}$ equation forces $\sum e_{i} = 0$ — that is exactly the condition "the line passes through the center of the data".

Exercise L2.3

Fit an OLS line to $(0, 1), (1, 3), (2, 4), (3, 6)$ .

Recall Solution

$\overset{x}{ˉ} = \frac{0 + 1 + 2 + 3}{4} = 1.5$ , $\overset{y}{ˉ} = \frac{1 + 3 + 4 + 6}{4} = 3.5$ .

$x_{i} - \overset{x}{ˉ}$	$y_{i} - \overset{y}{ˉ}$	product	square
$- 1.5$	$- 2.5$	$3.75$	$2.25$
$- 0.5$	$- 0.5$	$0.25$	$0.25$
$0.5$	$0.5$	$0.25$	$0.25$
$1.5$	$2.5$	$3.75$	$2.25$
Σ		$8$	$5$

$β_{1} = 8/5 = 1.6$ , $β_{0} = 3.5 - 1.6 (1.5) = 3.5 - 2.4 = 1.1$ . Model: $\overset{y}{^} = 1.1 + 1.6 x$ .

Level 3 — Analysis

Goal: measure fit, interpret coefficients, and probe sensitivity.

Exercise L3.1

For the L2.1 model $\overset{y}{^} = 1.5 x$ on $(1, 2), (2, 2), (3, 5)$ , compute $R^{2}$ .

Recall Solution

RSS (from the residuals in L2.2): $0. 5^{2} + (- 1)^{2} + 0. 5^{2} = 0.25 + 1 + 0.25 = 1.5$ . TSS = $\sum (y_{i} - \overset{y}{ˉ})^{2}$ with $\overset{y}{ˉ} = 3$ : $(2 - 3)^{2} + (2 - 3)^{2} + (5 - 3)^{2} = 1 + 1 + 4 = 6$ . $R^{2} = 1 - \frac{RSS}{TSS} = 1 - \frac{1.5}{6} = 1 - 0.25 = 0.75.$ Reading it: the line explains $75%$ of the variance in $y$ ; the remaining $25%$ is scatter the straight line cannot capture.

Exercise L3.2

A model of exam score on study hours gives $\overset{y}{^} = 35 + 7.5 x$ (score in points, $x$ in hours). (a) Interpret the slope in words. (b) Interpret the intercept — and say why it may be dubious.

Recall Solution

(a) Each extra hour of study is associated with a $7.5$ -point higher predicted score. (b) $β_{0} = 35$ is the predicted score at $x = 0$ hours (base/prior knowledge). It is dubious if $x = 0$ lies outside the observed range of study hours — predicting there is extrapolation (see 2.5-Assumptions-and-Diagnostics).

Exercise L3.3

Two datasets are fit. Dataset A has RMSE $= 2$ ; dataset B has RMSE $= 8$ , but B's $y$ values are measured in the thousands while A's are single digits. Can you conclude A's model is better? RMSE is defined $RMSE = \frac{1}{n} \sum (y_{i} - \overset{y}{^}_{i})^{2}$ .

Recall Solution

No. RMSE carries the units of $y$ . An error of $8$ on values in the thousands may be relatively tiny, while an error of $2$ on single-digit values may be huge. To compare across scales, use a unitless metric like $R^{2}$ , or normalise RMSE by the range/mean of $y$ .

Exercise L3.4

In the L2.3 fit $\overset{y}{^} = 1.1 + 1.6 x$ , the point $(3, 6)$ is nudged to $(3, 10)$ . Without a full recompute, will the slope $β_{1}$ increase or decrease? Then verify by recomputing.

Recall Solution

Prediction: $(3, 10)$ sits far above the old line at the largest $x$ . High- $x$ points act as levers (they have large $∣ x_{i} - \overset{x}{ˉ} ∣$ ), so pulling this one up should increase the slope. Verify: new $\overset{y}{ˉ} = \frac{1 + 3 + 4 + 10}{4} = 4.5$ , $\overset{x}{ˉ} = 1.5$ still.

$x_{i} - \overset{x}{ˉ}$	$y_{i} - \overset{y}{ˉ}$	product	square
$- 1.5$	$- 3.5$	$5.25$	$2.25$
$- 0.5$	$- 1.5$	$0.75$	$0.25$
$0.5$	$- 0.5$	$- 0.25$	$0.25$
$1.5$	$5.5$	$8.25$	$2.25$
Σ		$14$	$5$

$β_{1} = 14/5 = 2.8$ — up from $1.6$ , confirming the prediction.

Level 4 — Synthesis

Goal: combine derivation, algebra, and interpretation into one argument.

Exercise L4.1

Prove algebraically that the OLS line always passes through the mean point $(\overset{x}{ˉ}, \overset{y}{ˉ})$ .

Recall Solution

Evaluate the fitted line at $x = \overset{x}{ˉ}$ : $\overset{y}{^} (\overset{x}{ˉ}) = β_{0} + β_{1} \overset{x}{ˉ} .$ Substitute the intercept formula $β_{0} = \overset{y}{ˉ} - β_{1} \overset{x}{ˉ}$ : $\overset{y}{^} (\overset{x}{ˉ}) = (\overset{y}{ˉ} - β_{1} \overset{x}{ˉ}) + β_{1} \overset{x}{ˉ} = \overset{y}{ˉ} .$ So the point $(\overset{x}{ˉ}, \overset{y}{ˉ})$ lies exactly on the line — for any dataset. This is the algebraic face of "the residuals sum to zero" (see figure s02).

Exercise L4.2

Show that if you rescale every $x_{i}$ to $x_{i}^{'} = c x_{i}$ (with constant $c \neq = 0$ ), the new slope is $β_{1}^{'} = β_{1} / c$ , while the intercept is unchanged.

Recall Solution

The mean rescales too: $\overset{x}{ˉ}^{'} = c \overset{x}{ˉ}$ , so $x_{i}^{'} - \overset{x}{ˉ}^{'} = c (x_{i} - \overset{x}{ˉ})$ . The $y$ 's are untouched. Numerator of the slope: $\sum (x_{i}^{'} - \overset{x}{ˉ}^{'}) (y_{i} - \overset{y}{ˉ}) = c \sum (x_{i} - \overset{x}{ˉ}) (y_{i} - \overset{y}{ˉ})$ . Denominator: $\sum (x_{i}^{'} - \overset{x}{ˉ}^{'})^{2} = c^{2} \sum (x_{i} - \overset{x}{ˉ})^{2}$ . $β_{1}^{'} = \frac{c \cdot N}{c ^{2} \cdot D} = \frac{1}{c} \cdot \frac{N}{D} = \frac{β _{1}}{c} .$ Intercept: $β_{0}^{'} = \overset{y}{ˉ} - β_{1}^{'} \overset{x}{ˉ}^{'} = \overset{y}{ˉ} - \frac{β _{1}}{c} (c \overset{x}{ˉ}) = \overset{y}{ˉ} - β_{1} \overset{x}{ˉ} = β_{0} .$ Unchanged. ✅ Meaning: converting $x$ from, say, metres to centimetres ( $c = 100$ ) shrinks the slope by $100$ but leaves predictions and the intercept identical — the model is the same line, just relabelled.

Exercise L4.3

A dataset gives $β_{1} = 20$ , $β_{0} = 50$ , and $R^{2} = 0.98$ for house price vs size. A colleague predicts the price of a mansion whose size is four times the largest house in the data. Give a two-part critique: (i) is the arithmetic valid? (ii) is the conclusion trustworthy?

Recall Solution

(i) Arithmetic: yes — $\overset{y}{^} = 50 + 20 x$ returns a finite number for any $x$ ; the formula never fails. (ii) Conclusion: not trustworthy. This is extrapolation far outside the observed $x$ range. The linearity assumption was only ever supported inside the data; luxury pricing dynamics may differ entirely. A high $R^{2}$ on the fitted range gives zero guarantee out there. The honest move: report the prediction with a loud caveat, or gather data covering that range.

Level 5 — Mastery

Goal: derive, decide, and defend under an open-ended prompt.

Exercise L5.1

Derive the OLS intercept and slope for the special case of a single relationship type: suppose all data lie perfectly on a line, i.e. $y_{i} = a + b x_{i}$ exactly for every $i$ . Show OLS recovers $β_{1} = b$ and $β_{0} = a$ , and that $R^{2} = 1$ .

Recall Solution

With $y_{i} = a + b x_{i}$ , the mean is $\overset{y}{ˉ} = a + b \overset{x}{ˉ}$ , so $y_{i} - \overset{y}{ˉ} = b (x_{i} - \overset{x}{ˉ})$ . Slope: $β_{1} = \frac{\sum ( x _{i} - x ˉ ) \cdot b ( x _{i} - x ˉ )}{\sum ( x _{i} - x ˉ ) ^{2}} = \frac{b \sum ( x _{i} - x ˉ ) ^{2}}{\sum ( x _{i} - x ˉ ) ^{2}} = b .$ Intercept: $β_{0} = \overset{y}{ˉ} - b \overset{x}{ˉ} = (a + b \overset{x}{ˉ}) - b \overset{x}{ˉ} = a$ . $R^{2}$ : every prediction $\overset{y}{^}_{i} = a + b x_{i} = y_{i}$ , so every residual is $0$ , giving $RSS = 0$ and $R^{2} = 1 - 0/ TSS = 1$ . OLS reproduces the exact line — as it must when there is no noise.

Exercise L5.2 (Degenerate case)

What does OLS do when every $x_{i}$ is identical (all data at $x = c$ )? Explain both the algebra and the geometry.

Recall Solution

If all $x_{i} = c$ then $\overset{x}{ˉ} = c$ and every deviation $x_{i} - \overset{x}{ˉ} = 0$ . Denominator: $\sum (x_{i} - \overset{x}{ˉ})^{2} = 0$ . The slope formula becomes $β_{1} = \frac{0}{0}$ — undefined. Geometry: all points sit on a single vertical strip. Infinitely many lines pass through that strip's mean point with equally "best" vertical fit — there is no unique slope. A vertical spread of $x$ is required to pin a slope down. Takeaway: the model needs $Var (x) > 0$ . This is the simplest form of a collinearity failure — the multi-feature version haunts 2.2.02-Multiple-Linear-Regression.

Exercise L5.3 (Design & defend)

You have 200 patients' (age, cholesterol) data. A straight-line fit gives $R^{2} = 0.31$ , and the residual plot shows a clear U-shape (residuals high for young and old, low in the middle). Argue whether simple linear regression is appropriate here, and propose one concrete next step using only ideas linked from this vault.

Recall Solution

Diagnosis: the U-shaped residuals violate the linearity assumption — a straight line systematically under/over-predicts at the extremes. The low $R^{2} = 0.31$ agrees: most variance is unexplained. Simple linear regression is not appropriate as-is. Why it's a real signal, not noise: random noise gives residuals with no pattern; a shape means the model is missing structure (here, curvature). Concrete next steps (all in-vault):

Confirm the violation with formal residual diagnostics — 2.5-Assumptions-and-Diagnostics.
Add an $age^{2}$ term (still a linear model in its coefficients) — a case of 2.2.02-Multiple-Linear-Regression — to capture the curve, while watching for 4.2-Overfitting-vs-Underfitting.
Consider gradient-based fitting if the design grows — 2.3-Gradient-Descent-for-Linear-Regression.

Recall Self-test checklist

Which formula do you compute first — slope or intercept, and why? ::: Slope first: $β_{0} = \overset{y}{ˉ} - β_{1} \overset{x}{ˉ}$ needs $β_{1}$ . What does $R^{2} = 0.75$ mean in words? ::: The line explains $75%$ of the variance in $y$ ; $25%$ is unexplained scatter. Why can OLS fail entirely? ::: If $Var (x) = 0$ (all $x$ equal), the slope denominator is $0$ — undefined. What does a U-shaped residual plot tell you? ::: The linearity assumption is violated; the true relationship is curved.

Parent: Simple linear regression model · Hinglish: 2.2.01 Simple linear regression model (Hinglish)