Assumptions of linear regression: Worked examples

This page is a workout gym for the five assumptions. Instead of re-defining them, we throw every kind of situation at you and work each one to the end. Before we start, one word we will lean on constantly:

Definition The OLS line and the averaging bar

$\overset{ˉ}{\cdot}$ When we write $\overset{x}{ˉ}$ (say "x-bar") it means the average of all the $x$ values: $\overset{x}{ˉ} = \frac{1}{n} \sum_{i} x_{i}$ . Likewise $\overset{y}{ˉ}$ is the average of the $y$ values. Picture the whole cloud of dots and mark their balance point $(\overset{x}{ˉ}, \overset{y}{ˉ})$ — that is the pivot the OLS line always passes through. Ordinary Least Squares picks the intercept $β_{0}$ and slope $β_{1}$ that make the residual sticks as short as possible in total squared length, $\sum_{i} e_{i}^{2}$ . Solving that minimisation gives the two standard formulas we will reuse: $\hat{β}_{1} = \frac{\sum _{i} ( x _{i} - x ˉ ) ( y _{i} - y ˉ )}{\sum _{i} ( x _{i} - x ˉ ) ^{2}}, \hat{β}_{0} = \overset{y}{ˉ} - \hat{β}_{1} \overset{x}{ˉ} .$ When the data is already centered ( $\overset{x}{ˉ} = 0$ and $\overset{y}{ˉ} = 0$ ) these collapse to $\hat{β}_{1} = \frac{\sum _{i} x _{i} y _{i}}{\sum _{i} x _{i}^{2}}$ and $\hat{β}_{0} = 0$ — the simpler shape we use in Ex 2. See Ordinary Least Squares (OLS) for the full derivation.

Figure s01 (below), what its two panels show. Left panel: a fitted line (teal) with data dots (plum) and their orange residual sticks — this is what a residual is, and here the sticks are tiny so the band is flat (linearity holds, our Cell A). Right panel: the residuals left over when a straight line is forced onto cubic data — they snake up and down in a wave, the linearity-violation fingerprint of Cell B.

Figure — Assumptions of linear regression

The scenario matrix

Every worked example below fills one row. The columns are the kinds of trouble a regression can hand you.

Cell	Case class	What is being tested	Example
A	Linearity — clean pass	flat residual band	Ex 1
B	Linearity — U-shape fail	curved residuals, cubic truth	Ex 2
C	Homoscedasticity — funnel	growing spread, log fix	Ex 3
D	Independence — time correlation	Durbin–Watson number	Ex 4
E	Normality — heavy tail vs CLT	Q–Q read + does $n$ save you	Ex 5
F	Multicollinearity — near-perfect	VIF blows up, sign flips	Ex 6
G	Degenerate / zero input	one predictor, VIF undefined, $det = 0$	Ex 7
H	Real-world word problem	pick the right violation & fix	Ex 8
I	Exam twist	high $R^{2}$ but assumptions broken	Ex 9

Cells A–I are all covered. Signs (positive/negative residuals), the zero/degenerate input (Ex 7), and the limiting behaviour (VIF $\to \infty$ , DW $\to 0$ , CLT as $n \to \infty$ ) all appear explicitly.

Cell A — Linearity passes cleanly

Worked example Ex 1 · A flat residual band

Fit $\overset{y}{^} = 3 + 2 x$ to five points: $(1, 5), (2, 7), (3, 9), (4, 11), (5, 13)$ . Are the residuals patternless?

Forecast: guess — will the residuals be exactly zero, or wiggle?

Predict each point. $\overset{y}{^}_{i} = 3 + 2 x_{i}$ gives $5, 7, 9, 11, 13$ . Why this step? The residual needs $\overset{y}{^}_{i}$ first — you can't measure a leftover without the prediction.
Subtract. $e_{i} = y_{i} - \overset{y}{^}_{i} = 0, 0, 0, 0, 0$ . Why this step? This is the definition of residual applied to every row.
Read the plot. All sticks have length zero → a perfectly flat band on the zero line. Why this step? A flat band = no leftover structure = linearity holds (the ideal in figure s01, left panel).

Verify: the data literally sits on the line $y = 3 + 2 x$ (each $y$ jumps by 2 when $x$ jumps by 1), so residuals must vanish. Units: dollars minus dollars = dollars, consistent. ✅

Cell B — Linearity fails (U-shape)

Worked example Ex 2 · Fitting a line to a cubic

True world: $y = x^{3}$ . We collect $x = - 2, - 1, 0, 1, 2$ (no noise) and stubbornly fit a straight line by OLS. Show the residuals form a curve.

Forecast: guess the shape of the residual pattern — line? bowl? wave?

Get the data. $y = x^{3} = - 8, - 1, 0, 1, 8$ .
Fit OLS line. The $x$ values are symmetric about zero, so their average is $\overset{x}{ˉ} = 0$ ; the $y$ values are also symmetric ( $- 8, - 1, 0, 1, 8$ ), so $\overset{y}{ˉ} = 0$ . With centered data the OLS formulas from the definition above collapse to $\hat{β}_{1} = \frac{\sum _{i} x _{i} y _{i}}{\sum _{i} x _{i}^{2}}$ and $\hat{β}_{0} = 0$ . Plug in: $\hat{β}_{1} = \frac{( - 2 ) ( - 8 ) + ( - 1 ) ( - 1 ) + 0 + 1 + 2 \cdot 8}{4 + 1 + 0 + 1 + 4} = \frac{34}{10} = 3.4$ . So $\overset{y}{^} = 3.4 x$ . Why this step? We must fit the wrong model to see how it fails; the centered OLS slope formula was defined up top, so no symbol appears unexplained.
Residuals. $e_{i} = x^{3} - 3.4 x$ : $x = - 2 : - 8 + 6.8 = - 1.2; x = - 1 : - 1 + 3.4 = 2.4; x = 0 : 0; x = 1 : - 2.4; x = 2 : 1.2$ . Why this step? The sign pattern $(-, +, 0, -, +)$ is the fingerprint — it snakes, it does not stay flat.
Read the plot. Residuals go down–up–down–up: a wave / U-ish curve, not a flat band (figure s01, right panel). Why this step? A systematic curve is exactly the linearity-violation signature named in the parent note.

Verify: $\sum e_{i} = - 1.2 + 2.4 + 0 - 2.4 + 1.2 = 0$ (OLS residuals always sum to zero when an intercept is fit — a built-in sanity check). ✅ Fix: add an $x^{3}$ feature (polynomial regression) and residuals collapse to zero.

Cell C — Heteroscedasticity (funnel) + log fix

Worked example Ex 3 · The growing spread

House prices ( $y$ , $1000s) vs area $x$ . Fit gives $\overset{y}{^} = 50 + 0.2 x$ . Residual ranges: small houses $\overset{y}{^} \approx 100$ → $e \in [- 5, 5]$ ; big houses $\overset{y}{^} \approx 500$ → $e \in [- 50, 50]$ . Is variance constant? Does a log transform stabilize it?

Forecast: guess — is the spread growing additively or proportionally with price?

Compare raw spreads. The band at $\overset{y}{^} = 100$ has half-width $5$ ; at $\overset{y}{^} = 500$ it has half-width $50$ . The absolute spread grew tenfold. Why this step? Homoscedasticity means the band width is the same everywhere; here it clearly grows, so constant variance is already suspect.
Spread as a fraction of the fitted value. At $\overset{y}{^} = 100$ : $\frac{5}{100} = 0.05$ . At $\overset{y}{^} = 500$ : $\frac{50}{500} = 0.10$ . The relative spread doubles (from $5%$ to $10%$ ), and the absolute spread scales even faster with price — both signatures point to variance rising with $\overset{y}{^}$ . Why this step? Whether we measure spread absolutely or as a percentage, it is not constant — that is the funnel.
Diagnose. Residual-vs-fitted plot = a funnel opening to the right → heteroscedasticity, not curvature. Why this step? Distinguishes it from Ex 2 (curve = linearity; funnel = variance).
Apply $lo g$ . If $y = (fit) \cdot ϵ$ with $ϵ$ multiplicative, then $lo g y = lo g (fit) + lo g ϵ$ . A spread that scaled with size becomes an additive constant on the log scale. Why this step? Log turns "spread proportional to size" into "spread the same size" — variance stabilized.

Verify: absolute half-width goes $5 \to 50$ (a factor of $10$ ) exactly as $\overset{y}{^}$ goes $100 \to 500$ (a factor of $5$ ) — spread grows at least as fast as price, confirming non-constant variance. So the WLS weight would be $w_{i} \propto 1/ σ_{i}^{2}$ (down-weighting the noisy big houses), and log is the cheap equivalent. ✅

Cell D — Independence (time correlation)

Worked example Ex 4 · Durbin–Watson by hand

Residuals from a 6-day time series (in order): $e = 1, 2, 3, - 1, - 2, - 3$ . Compute the Durbin–Watson statistic and decide if errors are independent.

Forecast: the residuals drift from $+$ to $-$ smoothly — will DW land near $2$ (good) or near $0$ (trouble)?

Formula. $d = \frac{\sum _{i = 2}^{n} ( e _{i} - e _{i - 1} ) ^{2}}{\sum _{i = 1}^{n} e _{i}^{2}}$ . Why this step? DW compares neighbouring residuals. If today's leftover looks like yesterday's, consecutive differences are small → small $d$ .
Numerator (differences). $(2 - 1)^{2} + (3 - 2)^{2} + (- 1 - 3)^{2} + (- 2 + 1)^{2} + (- 3 + 2)^{2} = 1 + 1 + 16 + 1 + 1 = 20$ .
Denominator. $1 + 4 + 9 + 1 + 4 + 9 = 28$ . Why this step? Denominator scales by overall residual size so $d$ is unit-free.
Divide. $d = 20/28 \approx 0.714$ . Why this step? $d < 1$ is the parent note's red flag → positive autocorrelation, independence violated.

Verify: DW ranges $[0, 4]$ with $2$ = no correlation. Our smooth run-up-then-down gives $0.71$ , far below $1$ — matches the "trouble" band. Limiting check: perfectly identical consecutive residuals would give numerator $0 \Rightarrow d = 0$ . ✅ Fix: GLS / ARIMA (see Ordinary Least Squares (OLS) vs GLS discussion).

Cell E — Normality: heavy tail vs CLT rescue

Worked example Ex 5 · Reading a Q–Q plot and deciding

A Q–Q plot of residuals: the middle points hug the 45° reference line, but the top-right points bend above the line. Two scenarios: (a) $n = 15$ , (b) $n = 200$ .

Forecast: guess which tail is heavy and whether the small- $n$ case can still trust a $t$ -test.

Decode the deviation. A Q–Q plot puts the sample quantiles (vertical axis) against the normal quantiles (horizontal axis). At the top-right, the sample points sit above the reference line, meaning the largest residuals are bigger than a normal distribution would produce → a heavy right tail, i.e. right-skewed errors. Why this step? Deviations at the ends of a Q–Q plot are the tails; reading which side pulls away tells you which tail is heavy. (A left tail lighter than normal would instead sit above the line at the bottom-left; here the story is only the right tail.)
Effect on inference. A heavy right tail means a few large residuals dominate $\hat{β}$ ; $t$ -based intervals can be too narrow and $p$ -values off. Why this step? $t$ -tests assume normal errors; a broken assumption makes $p$ -values only approximate.
Apply the CLT rule. The sampling distribution of $\hat{β} \to$ normal as $n \to \infty$ .
- (a) $n = 15$ : too small — CLT has not "kicked in", use bootstrap $p$ -values.
- (b) $n = 200 > 30 - 50$ : CLT saves you, $t$ -test valid. Why this step? Normality of errors is only needed as a shortcut to normality of $\hat{β}$ ; enough data delivers that directly.

Verify: the threshold $n > 30$ – $50$ is the parent note's rule; $200$ clears it, $15$ does not. No numeric coefficient here, only a decision. ✅

Cell F — Multicollinearity (near-perfect)

Worked example Ex 6 · VIF explosion and a sign flip

Two predictors: area in sq ft $X_{1}$ and area in sq m $X_{2} = 0.0929 X_{1}$ plus tiny rounding noise giving $R_{2}^{2} = 0.999$ when $X_{2}$ is regressed on $X_{1}$ . Compute VIF.

Forecast: guess the VIF — single digits, or off the chart?

Formula. $VIF_{j} = \frac{1}{1 - R _{j}^{2}}$ , with $R_{j}^{2}$ from regressing $X_{j}$ on the other predictors. Why this step? $R_{j}^{2}$ near 1 means $X_{j}$ is almost predictable from the rest — the redundancy we fear.
Plug in. $VIF_{2} = \frac{1}{1 - 0.999} = \frac{1}{0.001} = 1000$ . Why this step? $1000 ≫ 10$ , the parent note's "serious multicollinearity" line.
Interpret. Standard error of $\hat{β}_{2}$ scales as $VIF \approx 31.6 \times$ larger than with no collinearity → huge, unstable coefficients that can flip sign on tiny data changes. Why this step? This is the $1/ λ_{m i n}$ blow-up from the parent derivation made concrete.

Verify: as $R_{j}^{2} \to 1$ , $VIF \to \infty$ (limiting behaviour). Check the standard-error inflation: $1000 \approx 31.62$ . ✅ Fix: drop one area column, or ridge to stabilize.

Cell G — Degenerate / zero input

Worked example Ex 7 · A constant column kills the fit

You accidentally add a column that is all the same value: $X_{2} = (7, 7, 7, 7)$ , alongside intercept and $X_{1}$ . What happens to $det (X^{T} X)$ , and to VIF?

Forecast: guess — does the model just ignore it, or break?

Spot the duplication. A constant column equals $7 \times$ the intercept column of 1s. So $X_{2} = 7 \cdot 1$ — a perfect linear combination. Why this step? Multicollinearity's extreme case is exact dependence.
Determinant. Two proportional columns make $X^{T} X$ singular: $det (X^{T} X) = 0$ . Why this step? $(X^{T} X)^{- 1}$ needs a nonzero determinant; zero means no unique OLS solution.
VIF. Regressing $X_{2}$ on the intercept gives $R_{2}^{2} = 1 \Rightarrow VIF = \frac{1}{1 - 1} = \frac{1}{0}$ = undefined / infinite. Why this step? This is the limiting boundary of Ex 6 — collinearity taken to its breaking point.

Verify: a $2 \times 2$ example — columns $[11]$ and $[77]$ give matrix $[2141498]$ , $det = 2 \cdot 98 - 1 4^{2} = 196 - 196 = 0$ . ✅ Fix: drop the constant/degenerate column before fitting.

Cell H — Real-world word problem

Worked example Ex 8 · Which assumption is broken?

A hospital predicts recovery time from patient age. Data: each patient measured weekly for 8 weeks (repeated rows per patient), and the residual spread is wider for older patients. Name every violated assumption and its fix.

Forecast: guess how many of the five assumptions are in trouble.

Repeated measures on the same person → errors within a patient are correlated → Independence violated. Why this step? $Cov (ϵ_{i}, ϵ_{j}) \neq = 0$ for rows of the same patient.
Spread wider for older patients → Homoscedasticity violated (a funnel keyed to age). Why this step? $Var (ϵ_{i} ∣ X_{i})$ changes with $X_{i}$ .
Fixes. Independence → mixed-effects / GLS; heteroscedasticity → WLS with $w_{i} = 1/ σ_{i}^{2}$ or log-transform recovery time. Why this step? Each fix targets the specific broken clause, per the summary table.

Verify: count = two violations (Independence + Homoscedasticity); Linearity, Normality, Multicollinearity are not implicated by the story. Units: recovery time (days) stays days after log only if you exponentiate back. ✅

Cell I — Exam twist: high $R^{2}$ , broken assumptions

Worked example Ex 9 · The seductive

$R^{2}$ A student fits $y = x^{3}$ data (from Ex 2) with the line $\overset{y}{^} = 3.4 x$ and reports a high $R^{2}$ , concluding "assumptions hold." Compute the true $R^{2}$ and expose the fallacy.

Forecast: can a curved-residual model still score close to $0.9$ ?

$R^{2}$ formula. $R^{2} = 1 - \frac{\sum e _{i}^{2}}{\sum ( y _{i} - y ˉ ) ^{2}}$ . Why this step? $R^{2}$ measures variance explained, nothing about residual shape.
Numerator (SSE). From Ex 2, $e = - 1.2, 2.4, 0, - 2.4, 1.2 \Rightarrow \sum e_{i}^{2} = 1.44 + 5.76 + 0 + 5.76 + 1.44 = 14.4$ .
Denominator (SST). $\overset{y}{ˉ} = 0$ , $y = - 8, - 1, 0, 1, 8 \Rightarrow \sum y_{i}^{2} = 64 + 1 + 0 + 1 + 64 = 130$ .
Divide. $R^{2} = 1 - 14.4/130 = 1 - 0.1108 \approx 0.889$ . Why this step? A high $0.889$ despite the wave-shaped residuals of Ex 2 — the fit "looks good" yet linearity is clearly broken.

Verify: $R^{2} \approx 0.889 > 0.8$ while residuals form a curve → confirms the parent-note mistake callout: big $R^{2}$ never certifies assumptions. Always read the residual plot. ✅

Recall Quick self-test

Durbin–Watson for residuals $1, 2, 3, - 1, - 2, - 3$ ? ::: $d = 20/28 \approx 0.714$ → independence violated. VIF when $R_{j}^{2} = 0.999$ ? ::: $1/ (1 - 0.999) = 1000$ → serious multicollinearity. $R^{2}$ of the line $3.4 x$ fit to $x^{3}$ data? ::: $\approx 0.889$ — high yet linearity is broken. Funnel vs curve in a residual plot? ::: funnel = heteroscedasticity; curve = nonlinearity.

See also: Residual Analysis · Hypothesis Testing in Regression · Generalized Linear Models (GLMs).