Foundations — Interpreting model coefficients

This page assumes nothing. If you have never seen a subscript or the letter $\beta$, start here and read top to bottom. Every symbol is earned before it is used again.

0. The picture we keep returning to

Before any symbol, hold this image: a machine with dials. You feed in numbers (the features), each dial has a strength setting (the coefficient), and out comes a prediction. Interpreting coefficients = reading the dial strengths.

Keep this machine in mind — every symbol below is a labelled part of it.

1. A feature: the input number $x$

We write a feature as the letter $x$. When there are several features, we number them with a small subscript — a tiny number written below and to the right:

$x_1,x_2,x_3,\dots$

The letter $j$ is a stand-in — a placeholder that means "pick any one of them." When we say "feature $x_j$," read it as "whichever feature we happen to be talking about right now."

If there are $m$ features total, they run $x_1$ up to $x_m$. Here $m$ is just "how many features there are" — a plain count, nothing more. (We deliberately use $m$, not $p$, because we will soon need $P$ for probability and don't want two look-alike symbols fighting for the same letter.)

2. The prediction: the output $\hat{y}$

The plain letter $y$ means the true answer — the real house price, the real yes/no of disease.

But the machine doesn't know the truth; it produces a guess. We mark a guess with a hat:

$\hat{y}\text{(say: "y-hat")}$

3. The coefficient: dial strength ${\beta }_j$

$\beta$ is the Greek letter beta (say "BAY-tuh"). It is just a name — we use a Greek letter so it never gets confused with a feature ($x$) or an answer ($y$).

There is one special coefficient with subscript $0$:

4. Putting the machine together: the linear equation

Now we can assemble the parent note's central line, symbol by symbol:

$\hat{y}={\beta }_0+{\beta }_1{x}_1+{\beta }_2{x}_2+\cdots +{\beta }_m{x}_m$

Read it as a sentence:

• Start at the baseline ${\beta }_0$.
• For each feature, multiply its slider position $x_j$ by its dial strength ${\beta }_j$.
• Add all those pushes together.

The $+$ signs matter: this machine adds contributions. That is exactly why the parent's mnemonic says "Linear adds."

5. "One unit" and "holding others fixed"

The parent note leans hard on the phrase "increase $x_j$ by one unit, holding the others fixed." Two ideas hide in there.

6. The slope idea: change over change

The parent calls ${\beta }_j$ a slope and even a partial derivative. You do not need calculus to get the picture.

The fancy symbol $\frac{\partial \hat{y}}{\partial x_j}$ ("partial derivative") is just formal notation for "the slope in the $x_j$ direction while the other sliders are frozen." The curly $\partial$ (say "partial dee") is a "d" that means "we're varying only this one variable." So:

$\frac{\partial \hat{y}}{\partial x_j}={\beta }_j$

is nothing more than "the slope along feature $j$ equals its dial strength." Same idea, formal clothing.

7. The second world: probability, odds, and the log

Linear regression outputs a number that can be anything. But some questions are yes/no (rain / no rain, disease / no disease). For those we need a few more symbols. See Logistic Regression and Odds and Log-Odds for the full story; here we just define the pieces.

Two symbols in that line need unpacking:

• $\pi$ (Greek "pie") is our name for the predicted probability. We use $\pi$ — not the letter $p$ — so it never collides with a feature count. It has nothing to do with the circle number $3.14\dots$; it's just a label.
• $P(y=1\mid \mathbf{x})$ reads "the Probability that the true answer $y$ equals $1$, given ($\mid$) the whole feature vector $\mathbf{x}$." The bar $\mid$ means "given that we know," and $\mathbf{x}$ is the full slider setting from §1.

Now we can write the logistic machine itself — the exact bridge the parent note builds on. The left side is the stretched output; the right side is the same additive machine from §4:

$\log \left(\frac{\pi }{1-\pi }\right)={\beta }_0+{\beta }_1{x}_1+{\beta }_2{x}_2+\cdots +{\beta }_m{x}_m$

The partner symbol is $e$, the special number $e\approx 2.718$. It undoes the logarithm: if $\log (a)=b$ then $e^b=a$. So exponentiating ($e^{{\beta }_j}$) turns an added effect on log-odds back into a multiplied effect on odds — the parent's "Logistic multiplies."

$\log (\text{odds})\underset{\text{apply }e}{\to }\text{odds}$

8. Three Greek helpers for scaling: $\mu$, $\sigma$, and $z_j$

To compare dials fairly across different-sized sliders, the parent standardizes features. That brings a few more symbols from Feature Scaling & Standardization.

The clump $\frac{x_j-{\mu }_j}{{\sigma }_j}$ reads: "how many typical spreads is this value away from its average?" Subtract the mean ($x_j-{\mu }_j$) to centre it, divide by the spread (${\sigma }_j$) to rescale it. That is all standardization does.

Prerequisite map

The chain below shows, in plain arrows, which idea you need before the next one makes sense — read it left as "comes first," right as "builds on it."

If any box on the left is fuzzy, re-read its section above before moving right.

Equipment checklist

Cover the right side; say the answer aloud before revealing.

Connections

• Parent · Interpreting model coefficients — where these symbols are used in full
• Linear Regression — the additive machine of §4
• Logistic Regression — the yes/no machine of §7
• Odds and Log-Odds — the odds/log-odds transform
• Feature Scaling & Standardization — $\mu$, $\sigma$, and $z_j$
• Multicollinearity & VIF — when "holding others fixed" breaks
• Regularization (Ridge & Lasso) — taming unstable dials
• Correlation vs Causation — reading dials honestly