Visual walkthrough — Interpreting model coefficients

This walkthrough builds on the parent topic and touches Linear Regression, Logistic Regression, and Odds and Log-Odds.

---

Step 1 — What is a coefficient, really? A line on a grid

WHAT. Start with the simplest possible model: one feature $x$, one prediction $\hat{y}$. We claim they are joined by a straight line: $\hat{y}=\underset{\text{where the line meets the }\hat{y}\text{-axis}}{\underbrace{{\beta }_0}}+\underset{\text{steepness of the line}}{\underbrace{{\beta }_1}}\cdot \underset{\text{the feature we feed in}}{\underbrace{x}}.$

Read each symbol on the picture: $x$ is a position along the horizontal floor, $\hat{y}$ is the height the line reaches above that position, ${\beta }_0$ is the height where $x=0$ (the "start"), and ${\beta }_1$ is how much the line climbs for each step to the right.

WHY a straight line. A straight line is the only shape whose steepness never changes. That constant steepness is exactly the number we want to interpret: "one nudge in $x$ always does the same thing to $\hat{y}$." If the shape curved, "the effect of $x$" would be different at every point and could not be a single number.

PICTURE. The blue line rises over the grid; the yellow bracket at $x=0$ marks ${\beta }_0$.

---

Step 2 — Nudge $x$ by one unit and watch $\hat{y}$

WHAT. Pick any point on the line at feature value $x$. Now step one unit to the right, to $x+1$. Measure how much the height changed: $\Delta \hat{y}=\hat{y}(x+1)-\hat{y}(x).$ Here $\Delta$ (a triangle, Greek "delta") is shorthand for "the change in." $\hat{y}(x+1)$ is the height at the new spot; $\hat{y}(x)$ is the height where we started.

Plug the line into each: $\Delta \hat{y}=[{\beta }_0+{\beta }_1(x+1)]-[{\beta }_0+{\beta }_{1}x]={\beta }_1\underset{=1}{\underbrace{(x+1-x)}}={\beta }_1.$

Watch the cancellation: ${\beta }_0$ appears in both brackets and vanishes; ${\beta }_{1}x$ appears in both and vanishes; the only survivor is one clean ${\beta }_1$.

WHY. This is the definition of the coefficient made concrete. The "+1" is a controlled experiment: change exactly one thing by exactly one unit, read the result. The answer ${\beta }_1$ doesn't depend on where you started — that's the flatness of a straight line paying off.

PICTURE. Red horizontal arrow = the $+1$ step in $x$; green vertical arrow = the resulting rise, labelled ${\beta }_1$.

---

Step 3 — The hidden fine print: "holding others fixed"

WHAT. Real models have many features: $\hat{y}={\beta }_0+{\beta }_1{x}_1+{\beta }_2{x}_2$. When we nudged $x_1$ by one unit, we quietly froze $x_2$. Redo the subtraction keeping $x_2$ still: $\Delta \hat{y}=[{\beta }_0+{\beta }_1(x_1+1)+{\beta }_2{x}_2]-[{\beta }_0+{\beta }_1{x}_1+{\beta }_2{x}_2]={\beta }_1.$ The ${\beta }_2{x}_2$ term is identical in both brackets, so it cancels — but only because we did not move $x_2$.

WHY. With two features, $\hat{y}$ is a tilted plane, not a line. ${\beta }_1$ is the tilt in the $x_1$ direction alone. If you walk diagonally (changing both features at once), the height change is not ${\beta }_1$ — it mixes both tilts. "Ceteris paribus" means "walk parallel to one axis."

PICTURE. A plane over an $(x_1,x_2)$ floor. The red path moves along $x_1$ only (rise $={\beta }_1$); the grey diagonal path moves both and rises by a different amount.

---

Step 4 — Switching worlds: probability lives in a box, not on a line

WHAT. Now the target is yes/no (disease: 1 or 0). We want $p=P(y=1)$. But $p$ is trapped between $0$ and $1$, while a line ${\beta }_0+{\beta }_{1}x$ runs freely from $-\infty$ to $+\infty$. A line cannot equal $p$ — it would predict probabilities above 1 or below 0.

WHY we need a transform. We need a dial that stretches the imprisoned interval $[0,1]$ out to the whole number line, so a straight line can live there honestly. That dial is built in two moves in the next step. For now: see the mismatch.

PICTURE. Left: a blue line escaping past $y=1$ and below $y=0$ (nonsense as probability). Right: the same $x$-range, but $p$ correctly squashed inside the $[0,1]$ band.

---

Step 5 — Building the dial: odds, then log-odds

WHAT. Two moves turn the caged $p$ into a free number.

Move 1 — odds. Define $\text{odds}=\frac{p}{1-p}=\frac{\text{chance it happens}}{\text{chance it doesn’t}}.$ As $p$ climbs from $0$ toward $1$, the numerator grows and the denominator shrinks, so odds sweep from $0$ up to $+\infty$. That frees the top end but the bottom still bunches at $0$.

Move 2 — logarithm. Take $\log$ of the odds: $\text{log-odds}=\log \left(\frac{p}{1-p}\right).$ The log sends the range $(0,\infty )$ onto the full $(-\infty ,+\infty )$: numbers below $1$ become negative, $1$ becomes $0$, numbers above $1$ become positive. Now — and only now — a straight line can safely equal it: $\log \left(\frac{p}{1-p}\right)={\beta }_0+{\beta }_1{x}_1+\cdots +{\beta }_p{x}_p.$

WHY the log specifically. Odds multiply when you stack effects (twice as likely, then thrice as likely = $6\times$). The log is the one tool that turns multiplying into adding — so multiplicative odds become an additive line, and Step 2's clean cancellation works again. That's the whole reason logs enter here.

PICTURE. A ladder of three panels: $p$ in $[0,1]$ → odds in $(0,\infty )$ → log-odds in $(-\infty ,\infty )$, with matching sample points carried across.

See Odds and Log-Odds for the full anatomy of this ladder.

---

Step 6 — Nudge again, then undo the log

WHAT. Because log-odds is linear, repeat the Step 2 trick. Nudge $x_j$ by one unit (others frozen): $\text{log-odds}(x_j+1)-\text{log-odds}(x_j)={\beta }_j.$ So one unit of $x_j$ adds ${\beta }_j$ to the log-odds. But we want to speak about odds, not their log. Undo the log by raising $e$ to each side. Using $\log a-\log b=\log \frac{a}{b}$: $\log \left(\frac{\text{odds}(x_j+1)}{\text{odds}(x_j)}\right)={\beta }_j⟹\frac{\text{odds}(x_j+1)}{\text{odds}(x_j)}=e^{{\beta }_j}.$

Term by term: the left side is a ratio of the new odds to the old odds; the right side $e^{{\beta }_j}$ is a plain multiplier. So a sum on the log-odds scale becomes a product on the odds scale.

WHY $e$ and not $10$? $e^x$ is the exact inverse of the natural log we used in Step 5; any other base would leave a leftover conversion factor. Same tool in reverse.

PICTURE. Top track (log-odds): a step of $+{\beta }_j$ added. Bottom track (odds): the same step is a $\times e^{{\beta }_j}$ stretch.

---

Step 7 — All the cases: sign of $\beta$ and the tiny-$\beta$ shortcut

WHAT. The multiplier $e^{{\beta }_j}$ behaves in three regimes:

• ${\beta }_j>0\Rightarrow e^{{\beta }_j}>1$ → odds grow (e.g. $\beta =0.8\Rightarrow e^{0.8}\approx 2.23$, odds up $\sim 123\%$).
• ${\beta }_j=0\Rightarrow e^0=1$ → odds unchanged, the feature is inert.
• ${\beta }_j<0\Rightarrow 0<e^{{\beta }_j}<1$ → odds shrink (e.g. $\beta =-0.8\Rightarrow e^{-0.8}\approx 0.449$).

And the shortcut for small values: near $\beta =0$ the exponential hugs the line $1+\beta$, so $e^{\beta }\approx 1+\beta (\text{good for }|\beta |\lesssim 0.1).$ Example: $\beta =0.05\Rightarrow e^{0.05}\approx 1.051\approx 1+0.05$, i.e. "$+5\%$ odds."

WHY show every case. A reader who only saw $\beta >0$ would misread a negative coefficient. The multiplier is never negative (an $e$-power can't be), so odds can approach $0$ but never go below — a degenerate limit worth seeing.

PICTURE. The curve $e^{\beta }$ against $\beta$: crossing $1$ at $\beta =0$, the tangent line $1+\beta$ kissing it there, and shaded regions marking $>1$ and $<1$.

---

The one-picture summary

Both worlds are the same three moves — freeze the others, nudge by one, read the result — differing only in what scale the reading lives on: linear reads on $y$ directly (an addition), logistic reads on log-odds and then un-logs to the odds scale (a multiplication).

Recall Feynman: tell the whole walkthrough back

A coefficient is the steepness of a straight line — how much the output climbs when you take one step right. To measure it, I plant myself anywhere on the line, step one unit sideways, and read the rise; the start-height and everything else cancel, leaving just $\beta$. With many features I must step straight along one axis and keep the rest frozen, otherwise I'm walking diagonally on a tilted plane and reading the wrong slope.

For yes/no problems the output is a probability locked between 0 and 1, and no straight line can live in a cage. So I stretch it out: first to "odds" (chance-yes over chance-no, which runs from 0 to infinity), then take a log (which spreads that onto the whole number line). Now a line fits, and my one-step trick works again — but one step now adds $\beta$ to the log-odds. To talk about odds instead of their log, I raise $e$ to both sides, and adding $\beta$ becomes multiplying by $e^{\beta }$. Positive $\beta$ makes odds grow, zero leaves them alone, negative shrinks them toward (but never below) zero — and for tiny $\beta$, $e^{\beta }\approx 1+\beta$ reads straight off as a percent.

---

Connections

• Interpreting model coefficients — the parent this walkthrough unpacks
• Linear Regression — the additive line of Steps 1–2
• Logistic Regression — the sigmoid behind Steps 4–6
• Odds and Log-Odds — the ladder built in Step 5
• Feature Scaling & Standardization — comparing the slopes fairly
• Multicollinearity & VIF — why "hold others fixed" can break (Step 3)
• Regularization (Ridge & Lasso) — taming shaky coefficients
• Correlation vs Causation — a slope is association, not proof