Worked examples — Interpreting model coefficients

Before we work anything, let us earn every symbol from zero.

---

The scenario matrix

Every cell below is a class of situation this topic can hand you. The worked examples that follow each name the cell(s) they cover, so nothing is left unshown.

#	Case class	The trap / question it tests	Covered by
A	Linear, positive $\beta$, continuous feature	plain "add per unit"	Ex 1
B	Linear, negative $\beta$	sign direction	Ex 2
C	Linear, many units at once ($\Delta x\ne 1$)	scale the change	Ex 2
D	Linear, intercept / all-features-zero	is $x=0$ meaningful?	Ex 1, Ex 3
E	Standardized coefficients, comparing importance	different scales	Ex 3
F	Logistic, positive $\beta$, odds ratio $>1$	multiply the odds	Ex 4
G	Logistic, negative $\beta$, odds ratio $<1$	odds shrink	Ex 5
H	Logistic, $\beta =0$ (degenerate)	$e^0=1$, no effect	Ex 5
I	Logistic, small $\beta$ approximation $e^{\beta }\approx 1+\beta$	when it's valid	Ex 6
J	Logistic, log-odds → actual probability	the non-linear jump	Ex 7
K	Dummy (0/1) feature interpretation	"switching on" a category	Ex 4, Ex 9
L	Correlated features / sign flip (exam twist)	ceteris paribus breaks	Ex 9
M	Logistic, $\Delta x\ne 1$ continuous	odds $\times e^{\beta \Delta x}$	Ex 6b
N	Logistic, intercept (all features 0)	baseline odds & probability	Ex 8

---

Worked examples

Example 1 — Linear, positive $\beta$, and the intercept (cells A, D)

Step 1 — Plug in the base case. $\widehat{\text{score}}=20+4(5)+2(8)=20+20+16=56.$ Why this step? We need a concrete anchor so "one more hour" has something to compare against.

Step 2 — Add one unit of hours, keep sleep fixed. The rule says the change is exactly ${\beta }_{\text{hours}}=4$. $\widehat{\text{score}}(6,8)=20+4(6)+2(8)=20+24+16=60.$ Why this step? This demonstrates $\frac{\partial \hat{y}}{\partial x_{\text{hours}}}={\beta }_{\text{hours}}$ directly: the difference $60-56=4={\beta }_{\text{hours}}$.

Step 3 — Read the intercept (cell D). Set every feature to $0$: $\widehat{\text{score}}=20$. That is the predicted score for a student who studies 0 hours and sleeps 0 hours. Why this step? To flag that $x=0$ may be nonsense (nobody sleeps 0 hours) — the intercept is real for the math but not always a meaningful person.

Verify: $60-56=4$ ✓ matches ${\beta }_{\text{hours}}$. Units: score is unitless points; ${\beta }_{\text{hours}}$ has units "points per hour," $4\times 1\text{hour}=4$ points ✓.

---

Example 2 — Linear, negative $\beta$, several units at once (cells B, C)

Step 1 — Read the sign. ${\beta }_{\text{miles}}=-0.15<0$, so more miles → lower price. Direction settled. Why this step? The sign is the first thing to interpret; getting it wrong reverses the story.

Step 2 — Scale by the actual change (cell C). The per-unit change is ${\beta }_{\text{miles}}$, so for $\Delta =20000$ units: $\Delta \hat{y}={\beta }_{\text{miles}}\cdot \Delta x=(-0.15)(20000)=-3000.$ Why this step? Because linear means the effect scales linearly: 20,000 units cost $20000\times \beta$, not just $\beta$.

Verify: predict at miles $=0$: $30,000. At miles $=20000$: $30000-0.15(20000)=30000-3000=27000$. Difference $=-3000$ ✓. Units: \text{(\/mile)} \times \text{miles} = \text{$}$ ✓.

---

Example 3 — Standardized coefficients: which feature matters more? (cells D, E)

Step 1 — Recognise the trap. Raw magnitudes aren't comparable because income is measured in tens of thousands while age is in single digits. See Feature Scaling & Standardization. Why this step? The whole point of standardizing: a big-range feature is forced to have a small coefficient, and vice versa. Comparing raw $\beta$ is meaningless.

Step 2 — Compute standardized coefficients ${\beta }_j^{\text{std}}={\beta }_j{\sigma }_j$. This is the effect per one standard deviation of the feature. ${\beta }_{\text{income}}^{\text{std}}=0.02\times 40000=800,{\beta }_{\text{age}}^{\text{std}}=300\times 12=3600.$ Why this step? One SD of income is a "typical spread" of income, one SD of age a typical spread of age — now both are on the same "one typical wiggle" footing.

Step 3 — Compare. $3600>800$: age still wins, but by a factor of $4.5$, not $15000$. Why this step? Only now that both numbers are in the same unit ("$y$-change per one typical wiggle of the feature") is the comparison $3600>800$ a fair statement about importance — comparing the raw $0.02$ vs $300$ would have compared apples ($-per-$) to oranges ($-per-year). The figure makes the reversal-of-scale visible.

Verify: $0.02\times 40000=800$ ✓; $300\times 12=3600$ ✓; ratio $3600/800=4.5$ ✓.

---

Example 4 — Logistic, positive $\beta$, dummy feature, odds ratio (cells F, K)

Step 1 — Identify the tool: exponentiate. The coefficient sits on log-odds, and $e^{\text{log-odds}}=\text{odds}$. So $e^{\beta }$ is the factor by which odds multiply. Why this step? Adding $\beta$ to a log means multiplying the original quantity by $e^{\beta }$ — that's the defining property of logs/exponentials, and it's exactly why we reach for $e$ here rather than any other operation.

Step 2 — Compute the odds ratio OR (cell F). $\text{OR}=e^{1.1}\approx \mathrm{3.004.}$ Why this step? Exponentiation is what converts the additive log-odds change ($+1.1$) into the multiplicative odds change: because $\text{new odds}=e^{\text{old log-odds}+1.1}=e^{\text{old log-odds}}\cdot e^{1.1}=(\text{old odds})\cdot e^{1.1}$, the factor $e^{1.1}$ is literally the odds ratio. Any other operation would not preserve this "add-in-logs = multiply-in-odds" bridge.

Step 3 — Interpret the dummy (cell K). has_degree goes from $0$ to $1$ — a "1-unit increase" is literally switching the category on. So having a degree multiplies the odds of approval by $\approx 3.0$. Why this step? For 0/1 features there is no "half a unit"; the coefficient is the whole effect of belonging vs. not belonging to that group.

Verify: odds at no-degree $=e^{-1}\approx 0.3679$; odds at degree $=e^{-1+1.1}=e^{0.1}\approx 1.1052$. Ratio $1.1052/0.3679=3.004=e^{1.1}$ ✓.

---

Example 5 — Logistic, negative $\beta$ and the $\beta =0$ degenerate case (cells G, H)

Step 1 — Negative $\beta$ → odds ratio below 1 (cell G). ${\text{OR}}_{\text{loyalty}}=e^{-0.7}\approx \mathrm{0.4966.}$ So each extra loyalty year multiplies churn odds by $\approx 0.50$ — it roughly halves the odds. Why this step? $\beta <0\Rightarrow e^{\beta }<1\Rightarrow$ multiplying by a number below 1 shrinks the odds. The sign of $\beta$ becomes "above or below 1" after exponentiation.

Step 2 — The degenerate $\beta =0$ (cell H). ${\text{OR}}_{\text{newsletter}}=e^0=1.$ Multiplying odds by $1$ changes nothing: the newsletter feature has no modelled effect on churn odds. Why this step? Zero is the pivot. Left of it (negative $\beta$) odds shrink; right of it (positive $\beta$) odds grow; exactly at it, no effect.

Verify: $e^{-0.7}=0.4966$ ✓ (below 1); $e^0=1$ ✓.

---

Example 6 — When is $e^{\beta }\approx 1+\beta$ safe? (cell I)

Step 1 — Why the shortcut exists. For small $x$, $e^x\approx 1+x$ (the first two terms of its expansion). So a small coefficient means "odds go up by about $\beta \times 100\%$." Why this step? It saves mental exponentiation for tiny coefficients, which are extremely common in standardized or fine-grained features.

Step 2 — Test the small one, ${\beta }_a=0.03$.

• True: $e^{0.03}\approx 1.03045$ → $+3.045\%$.
• Shortcut: $1+0.03=1.03$ → $+3\%$.
• Gap: about $0.045\%$ — negligible. ✓ Safe.

Why this step? We test near $x=0$ first because that is exactly where the tangent line $1+x$ hugs the curve $e^x$ tightest — verifying the gap is tiny confirms the shortcut is licensed here, not just asserting it.

Step 3 — Test the big one, ${\beta }_b=0.9$.

• True: $e^{0.9}\approx 2.4596$ → $+145.96\%$.
• Shortcut: $1+0.9=1.9$ → $+90\%$.
• Gap: about $56$ percentage points — wildly wrong. ✗ Do not use.

Why this step? We test a far-from-zero value to expose the failure: the approximation is a straight line tangent to a curve at $0$, so far from $0$ the curve races ahead and the line lags. Seeing the 56-point gap gives a concrete "how far is too far" and yields the rule of thumb: fine only for $|\beta |\lesssim 0.1$.

Verify: $e^{0.03}=1.03045$, shortcut $1.03$, gap $\approx 0.00045$ ✓; $e^{0.9}=2.4596$, shortcut $1.9$, gap $\approx 0.56$ ✓.

---

Example 6b — Logistic, continuous feature, $\Delta x\ne 1$ (cell M)

Step 1 — Recall: log-odds is linear, so a $\Delta x$ change adds $\beta \Delta x$. $\Delta (\text{log-odds})=\beta \cdot \Delta x=0.4\times 3=\mathrm{1.2.}$ Why this step? Just like the linear car example scaled the effect by $\Delta x$, here the log-odds (which is linear) scales the same way before we exponentiate.

Step 2 — Exponentiate the whole change (cell M). Adding $1.2$ to log-odds multiplies the odds by $e^{1.2}$: $\text{OR}=e^{\beta \Delta x}=e^{0.4\times 3}=e^{1.2}\approx \mathrm{3.320.}$ Why this step? You must exponentiate the total log-odds change, not raise $e^{\beta }$ and eyeball it — though note $e^{1.2}=(e^{0.4}{)}^3$, so "$e^{\beta }$ applied three times" gives the same thing.

Verify: $e^{0.4}\approx 1.4918$; $(1.4918{)}^3\approx 3.320=e^{1.2}$ ✓. So a naive "$3\times e^{0.4}\approx 4.5$" would be wrong — odds multiply, they don't add.

---

Example 7 — Log-odds → actual probability (cell J)

Step 1 — Compute each person's log-odds $\ell$.

• No degree: $\ell =-1+1.1(0)=-1$.
• Degree: $\ell =-1+1.1(1)=0.1$. Why this step? Probability lives at the end of the pipeline; we first get the linear log-odds, then bend it through the sigmoid.

Step 2 — Apply the sigmoid $p=\frac{1}{1+e^{-\ell }}$. $p_{\text{no}}=\frac{1}{1+e^1}\approx 0.2689,p_{\text{deg}}=\frac{1}{1+e^{-0.1}}\approx \mathrm{0.5250.}$ Why this step? The sigmoid is the only function that turns any real log-odds into a valid probability in $(0,1)$ — that's why logistic regression uses it and why the effect on probability is not linear.

Step 3 — See why "add 1.1 to probability" is nonsense (cell J). The probability rose by $0.5250-0.2689=0.2561$, not $1.1$. And $0.2689+1.1=1.3689>1$ — impossible for a probability. Why this step? To hammer home that the same additive log-odds bump produces a different probability jump depending on where you start on the S-curve — and can never push probability past 1, unlike naive addition.

Verify: $\sigma (-1)=0.26894$ ✓; $\sigma (0.1)=0.52498$ ✓; difference $\approx 0.2560\ne 1.1$ ✓; and $p_{\text{deg}}<1$ ✓.

---

Example 8 — The logistic intercept: baseline odds and probability (cell N)

Step 1 — Identify the baseline person (cell N). Set every feature to $0$: here has_degree $=0$, i.e. a person without a degree. Then log-odds $={\beta }_0=-1$. Why this step? The intercept is the logistic twin of the linear intercept: it is the prediction (in log-odds) for the all-zeros individual. For a dummy feature, "0" is a real group (no degree), so unlike the sleep-0 case it is meaningful here.

Step 2 — Turn baseline log-odds into baseline odds. Exponentiate: ${\text{odds}}_0=e^{{\beta }_0}=e^{-1}\approx \mathrm{0.3679.}$ Below 1 → fewer approvals than rejections for the baseline group. Why this step? $e^{\text{log-odds}}=\text{odds}$; this is the cleanest reading of the intercept — it is the baseline odds.

Step 3 — Turn baseline log-odds into baseline probability. Apply the sigmoid: $p_0=\sigma (-1)=\frac{1}{1+e^1}\approx \mathrm{0.2689.}$ So a no-degree applicant has about a $27\%$ chance of approval before any positive feature helps. Why this step? To show the intercept can be read three equivalent ways — log-odds $-1$, odds $0.37$, probability $0.27$ — and probability is the one humans understand.

Verify: $e^{-1}=0.36788$ ✓; $\sigma (-1)=0.26894$ ✓; and $\frac{0.26894}{1-0.26894}=0.36788=e^{-1}$ ✓ (probability and odds agree).

---

Example 9 — Exam twist: correlated features and a sign flip (cells K, L)

Step 1 — Recall what ${\beta }_{\text{rooms}}$ literally claims. It is the effect of one more room holding sqft fixed — a room added without any extra floor area. In reality sqft and rooms move together, so this "ceteris paribus" move barely exists in the data. Why this step? The interpretation only makes sense if you can actually vary one feature alone. Correlation destroys that. See Multicollinearity & VIF.

Step 2 — Why the signs flip. When two features carry almost the same information, the model can split the shared effect between them in infinitely many near-equivalent ways. A nudge to the data reshuffles the split, so individual $\beta$'s swing violently and can flip sign — even though the combined prediction is stable. Why this step? This is the concrete mechanism behind "coefficients become unreliable under multicollinearity."

Step 3 — Sanity check the combined effect. Take a realistic joint move: +1 room typically comes with +200 sqft.

• Run 1: $\Delta \hat{y}=5(200)+(-100)(1)=1000-100=900$.
• Run 2: $\Delta \hat{y}=-3(200)+900(1)=-600+900=300$. Both are positive — rent still rises. The individual coefficients lied; the realistic bundle did not (mostly). Why this step? It shows the prediction can be trustworthy while the per-feature story is not.

Step 4 — The fix (resolving cell L). Concretely: (1) measure the redundancy with a correlation matrix or VIF — a VIF above ~5–10 flags trouble; (2) remove or merge one of the twins (keep sqft, drop rooms, or build a "rooms-per-sqft" feature); (3) shrink the coefficients with Regularization (Ridge & Lasso) — Ridge in particular spreads the shared effect smoothly and stops the wild sign-swings; (4) report bundles, not singletons — quote the effect of a realistic joint move (as in Step 3) instead of a lone coefficient. After any of these, the surviving coefficient becomes stable and interpretable again. Why this step? Without a fix, scenario L stays unresolved: you would keep reading a meaningless lonely coefficient. The fixes restore the ceteris paribus interpretation by removing the thing that broke it.

Verify: Run 1 bundle $=5\cdot 200-100=900$ ✓; Run 2 bundle $=-3\cdot 200+900=300$ ✓; both $>0$ ✓.

---

Active recall

Recall Cover the answers

Linear: 20,000 more miles at $\beta =-0.15$ changes price by? ::: -\3000$(down).Standardizedcoefficientformula?:::$\beta^{\text{std}}_j = \beta_j,\sigma_j$=effectper1SD.Logistic$\beta = 1.1$giveswhatoddsratio?:::$\text{OR}=e^{1.1}\approx 3.0$;oddstriple.Logistic$\beta = -0.7$doeswhattoodds?:::multipliesby$e^{-0.7}\approx 0.50$;roughlyhalvesthem.Logistic$\beta = 0$?:::$e^0 = 1$;noeffectonodds.Logistic,$\beta=0.4$,featurerisesby3—oddschangefactor?:::$e^{0.4\times 3}=e^{1.2}\approx 3.32$(multiply,do{n}^{\prime }tadd).Whenis$e^\beta\approx 1+\beta$safe?:::onlyforsmall$|\beta|\lesssim 0.1$.Turnlog-odds$\ell=0.1$intoprobability?:::$\sigma(0.1)=\frac{1}{1+e^{-0.1}}\approx 0.525$.Logisticintercept$\beta_0=-1$means?:::baseline(allfeatures0)log-odds$-1$\to odds$e^{-1}\approx 0.37$\to probability$\approx 0.27$. A single coefficient flips sign between runs — likely cause and fix? ::: multicollinearity; fix by VIF check, dropping/merging a twin, or regularization.

---

Connections

• Parent: Interpreting model coefficients
• Linear Regression — additive coefficients (Ex 1–3)
• Logistic Regression — sigmoid & log-odds (Ex 4–8)
• Odds and Log-Odds — the $e^{\beta }$ transform
• Feature Scaling & Standardization — Ex 3
• Multicollinearity & VIF — Ex 9
• Regularization (Ridge & Lasso) — stabilizing flipped coefficients
• Correlation vs Causation — the causal caveat