Exercises — Interpreting model coefficients

• ${\beta }_{\text{kWh}}=0.15$: each one extra kilowatt-hour used adds $0.15 to the predicted bill, holding rooms fixed.
• ${\beta }_{\text{rooms}}=5$: each one extra room adds $5, holding kWh fixed.
• The $20$ is the intercept: predicted bill when $\text{kWh}=0$ and $\text{rooms}=0$.

The odds get multiplied by $e^0=1$ — i.e. nothing changes. A zero coefficient means "no effect on the odds."

(a) is linear — coefficients add. (b) is logistic — coefficients multiply the odds after exponentiating. Mnemonic: Linear adds, Logistic multiplies.

First prediction: $50,000+300(1000)+10,000(2)=50,000+300,000+20,000=370,000$. After +150 sqft: $50,000+300(1150)+20,000=50,000+345,000+20,000=415,000$. Rise $=415,000-370,000=45,000=300\times 150$. The change is just ${\beta }_{\text{sqft}}\times (\text{change in sqft})$.

Odds ratio $=e^{0.8}\approx 2.2255$. A smoker's odds of disease are about 2.23 times a non-smoker's — a $\approx 123\%$ increase in odds (not probability).

Non-smoker: log-odds $=-3$, so $p=\frac{1}{1+e^3}=\frac{1}{1+20.0855}\approx 0.0474$. Smoker: log-odds $=-3+0.8=-2.2$, so $p=\frac{1}{1+e^{2.2}}=\frac{1}{1+9.0250}\approx 0.0998$. Notice: probability roughly doubled ($0.047\to 0.100$) even though the odds multiplied by $2.23$. Odds and probability are different animals when $p$ is small they happen to be close; they diverge near $p=0.5$.

Standardized coefficient $={\beta }_j\cdot {\sigma }_j$ (effect per one standard deviation of the feature).

• Income: $0.002\times 25,000=50$.
• Age: $400\times 12=4,800$. Age still leads, but by a factor of $4800/50=96$, not 200,000. The raw comparison ($400/0.002=200,000$) was inflated purely by units (dollars vs. years). See Feature Scaling & Standardization.

No. This is multicollinearity: hours_studied and hours_of_tutoring move together, so the fit cannot uniquely credit the effect to one of them. The two coefficients trade off, and either can flip sign with high variance. The joint effect can still be positive; the individual numbers just became unstable. Fix: check the correlation / VIF, or shrink with Regularization (Ridge & Lasso). The sign flip is a red flag, not a finding.

Small-$\beta$ rule: $e^{\beta }\approx 1+\beta$, so $e^{0.03}\approx 1.03$ → about +3% odds. Exact: $e^{0.03}=1.03045\dots$ → +3.045%. The approximation is off by only $\approx 0.045$ percentage points — excellent for small $\beta$. (It gets worse as $|\beta |$ grows; at $\beta =0.8$ the rule gives +80% but the truth is +123%.)

(a) log-odds $=-2.0+0.5(1)-0.9(0)=-1.5$. (b) odds $=e^{-1.5}\approx 0.2231$ (about 1 default per 4.48 non-defaults). (c) $p=\frac{1}{1+e^{1.5}}=\frac{1}{1+4.4817}\approx 0.1824$. Income has $\beta =-0.9$, so each +1 SD of income multiplies default-odds by $e^{-0.9}\approx 0.4066$ — cutting odds by about $59\%$. Debt raises odds ($e^{0.5}\approx 1.649$).

Difference in log-odds $=[0.5(2)-0.9(1)]-[0.5(0)-0.9(0)]=1.0-0.9=0.1$. Odds ratio A vs B $=e^{0.1}\approx 1.1052$. Check the product form: debt contributes $e^{0.5\cdot 2}=e^{1.0}$, income contributes $e^{-0.9\cdot 1}=e^{-0.9}$; product $=e^{1.0-0.9}=e^{0.1}$. Yes, it factors, because log-odds is additive in the features, and $e^{a+b}=e^a{e}^b$. Additivity in log-odds becomes multiplicativity in odds — exactly the "Logistic multiplies" mnemonic.

Log-odds is linear, so moving $x_j$ by $\Delta$ changes log-odds by ${\beta }_j\Delta$: $\text{log-odds}(x_j+\Delta )-\text{log-odds}(x_j)={\beta }_j\Delta .$ Exponentiate: $\frac{\text{odds}(x_j+\Delta )}{\text{odds}(x_j)}=e^{{\beta }_j\Delta }=(e^{{\beta }_j}{)}^{\Delta }.$ Apply: $e^{0.04\times 10}=e^{0.4}\approx 1.4918$. Ten extra years multiplies the odds by about 1.49 ($\approx +49\%$). Note this is $(e^{0.04}{)}^{10}=(1.0408{)}^{10}$, not $10\times$ the per-year effect — odds compound multiplicatively.

With $\hat{y}={\beta }_0+{\sum }_k{\beta }_k{x}_k$, differentiate w.r.t. $x_j$: every term without $x_j$ is a constant (derivative 0), and $\frac{\partial }{\partial x_j}({\beta }_j{x}_j)={\beta }_j$. So $\frac{\partial \hat{y}}{\partial x_j}={\beta }_j$. Why a derivative: a derivative is precisely the definition of "instantaneous change in output per unit change in one input, holding others fixed" — the exact question a coefficient answers. For a straight line the derivative is constant, so the per-unit change is the same everywhere, and "per 1 unit" equals the derivative exactly.

(a) Each extra cup of coffee is associated with the odds of heart disease multiplying by $e^{1.2}\approx 3.32$, holding the genetic marker fixed — an association within this fitted model, nothing more. (b) Reasons the causal claim fails:

1. Confounding / correlation ≠ causation — heavy coffee drinkers might also smoke, sleep less, or be more stressed; those unmodeled confounders could drive both. Regression measures association, not causation. See Correlation vs Causation.
2. Model-dependence & collinearity — if coffee correlates with the genetic marker or omitted variables, the coefficient is unstable and its "holding others fixed" clause is physically unrealisable. A different feature set could shrink or flip $1.2$. Correct wording: "In this sample, coffee is associated with higher modeled odds; a causal claim needs a controlled/causal design."