Worked examples — Feature creation and interaction terms

This page is the "no scenario left behind" drill for feature creation. The parent note showed you the ideas; here we hit every case class that feature engineering can throw at you — with numbers you can check by hand.

Before we start, one plain-words reminder so nothing is assumed:

The scenario matrix

Every worked example below is tagged with the cell of this matrix it covers. The goal: by the last example you have seen at least one number from each row.

Cell	Case class	What makes it tricky
A	Polynomial, positive inputs	ordinary expansion, count features
B	Interaction with a 0/1 (binary) flag	term switches ON/OFF
C	Interaction with a negative input	sign flips the joint effect
D	Ratio feature, safe denominator	scale-invariance
E	Ratio feature, denominator → 0	degenerate / divide-by-zero
F	Both inputs zero (degenerate)	interaction collapses, main effects vanish
G	Limiting value (feature → very large)	which term dominates, overfitting smell
H	Real-world word problem	translate English → features
I	Exam-style twist	"drop main effects" trap, interpretation

Let's fill every cell.

---

Cell A — Polynomial features, all positive

Step 1 — List the degree-2 features. $[T,R,T^2,TR,R^2]=[25,10,625,250,100].$ Why this step? A degree-2 expansion adds every product you can build using at most two of the originals, counting repeats: $T\cdot T$, $T\cdot R$, $R\cdot R$, plus the two degree-1 terms $T$ and $R$.

Step 2 — Multiply each by its weight and sum. $2(25)+3(10)-0.05(625)+0.8(250)-0.1(100)+5.$ Why this step? This is literally what the linear model computes: dot the weights into the features, add bias.

Step 3 — Arithmetic. $50+30-31.25+200-10+5=\mathrm{243.75.}$ Why this step? Keep signs honest: the $T^2$ and $R^2$ terms are negative (too hot / too much rain hurts), the interaction $TR$ is the big positive contributor.

Verify: Recompute grouped by sign: positives $50+30+200+5=285$, negatives $31.25+10=41.25$, difference $=243.75$. ✓ Answer sits near 250, as the forecast anchor suggested.

---

Cell B — Interaction with a binary flag (switch ON/OFF)

Step 1 — Set $w=0$ (weekday). $\text{sales}=50a+0+0+1000=50a+1000.$ Why this step? Multiplying by $w=0$ switches the interaction and the weekend main effect off. Slope of $a$ is $50$.

Step 2 — Set $w=1$ (weekend). $\text{sales}=50a+200+80a+1000=130a+1200.$ Why this step? Now the interaction term $80a$ is alive, so it adds to the slope, not just the intercept. Slope of $a$ is $130$.

Step 3 — Ratio of slopes. $130/50=\mathrm{2.6.}$ Why this step? "How much more effective" is a comparison of slopes, not intercepts.

Step 4 — Predict at $a=20$. Weekday: $50(20)+1000=2000$. Weekend: $130(20)+1200=3800$.

Verify: The gap $3800-2000=1800$ should equal the weekend-only terms: $200+80(20)=200+1600=1800$. ✓

Look at the figure: two straight lines share nothing but the same $x$-axis. The interaction tilts the weekend line upward — its steeper slope is the whole point.

---

Cell C — Interaction with a negative input (sign flip)

Step 1 — Plug the numbers. $4(-6)+2(8)+1.5(-6)(8).$ Why this step? $d$ is negative because it's colder than pleasant. The interaction inherits that sign.

Step 2 — Evaluate term by term. $-24+16+1.5(-48)=-24+16-72=-80.$ Why this step? The interaction $(dh)=-48$ is negative, so a positive weight $1.5$ produces a negative contribution. High humidity on a cold day reduces the (heat-based) discomfort score in this model — the sign of $d$ flipped the joint effect's direction.

Verify: Flip to a hot day $d=+6$, same $h=8$: $24+16+1.5(48)=24+16+72=112$. Same magnitudes, opposite interaction sign. ✓ This is exactly why we never assume interaction terms are positive.

---

Cell D — Ratio feature, safe denominator

Step 1 — Build the ratio for X. $\frac{50000}{80000}=\mathrm{0.625.}$

Step 2 — Build the ratio for Y. $\frac{50000}{30000}=\frac{5}{3}\approx \mathrm{1.667.}$ Why this step? Division normalizes by scale. The raw debt is identical ($50000$), so a model on raw features sees them as equal — the ratio is what reveals Y carries a heavier relative burden.

Verify: Units cancel: dollars ÷ dollars = dimensionless. A ratio $>1$ means debt exceeds income; $\frac{5}{3}>0.625$, so Y is flagged riskier. ✓ Scale-invariance confirmed — see Feature Scaling and Normalization for why unit-free features behave nicely.

---

Cell E — Ratio feature, denominator → 0 (degenerate)

Step 1 — Try the naive formula. $\text{ctr}=\frac{0}{0}=\text{undefined (NaN)}.$ Why this step? You must see the failure. A NaN silently poisons gradients — the whole model can produce NaN loss.

Step 2 — Add a smoothing prior (pseudocounts $\alpha ,\beta$). $\text{ctr}=\frac{\text{clicks}+\alpha }{\text{impressions}+\beta },\alpha =1,\beta =100.$ Here: $\frac{0+1}{0+100}=0.01$. Why this step? Adding $\beta$ to the denominator guarantees it is never zero, and the prior $\alpha /\beta =0.01$ is a sensible "unknown ad" baseline click rate. Note this is not classical Laplace (add-one) smoothing, which would use the same constant top and bottom ($\alpha =\beta =1$); here we deliberately pick $\alpha \ne \beta$ so the prior encodes a realistic 1% click rate rather than 50%.

Step 3 — Check it converges to the raw ratio for big data. With clicks=500, impressions=10000: $\frac{501}{10100}\approx 0.04960$, vs raw $\frac{500}{10000}=0.05$. Why this step? The smoothing must vanish as data grows — otherwise you'd bias real measurements.

Verify: Smoothed value $\approx 0.04960$ differs from raw $0.05$ by about $0.0004$ — negligible at scale, life-saving at zero. ✓

---

Cell F — Both inputs zero (interaction collapses)

Step 1 — Substitute zeros. $50(0)+200(0)+80(0)(0)+1000=1000.$ Why this step? Every feature term, including the interaction, is multiplied by a zero factor, so all vanish. Only the bias survives.

Verify: This is the definition of the bias $b$: the prediction with no signal. $\text{sales}=1000$. ✓ Whenever you doubt a model, plug all-zeros — you should recover exactly $b$.

---

Cell G — Limiting value (which term dominates)

Step 1 — Evaluate at three scales. $x=10:30-5=25,x=50:150-125=25,x=100:300-500=-200.$ Why this step? The quadratic term $-0.05x^2$ grows faster than the linear term $3x$. Doubling $x$ quadruples the $x^2$ term but only doubles the $x$ term.

Step 2 — Find the turning point with a derivative. The peak of a smooth curve is where its slope is zero. The slope of $y=3x-0.05x^2$ is its derivative $\frac{dy}{dx}=3-0.1x.$ Why a derivative? The derivative $\frac{dy}{dx}$ measures the instantaneous slope of the curve; at a maximum the curve momentarily stops rising, so its slope is exactly $0$. Setting $3-0.1x=0$ gives $x=30$, and $y(30)=90-45=45$. Why this matters? Beyond $x=30$ the slope goes negative and $y$ falls — a warning that high-degree features extrapolate wildly and invite overfitting. Tame them with Regularization (L1, L2).

Verify: $y(30)=45$ is larger than $y(10)=25$ and $y(50)=25$ (which are symmetric about the peak). ✓ Symmetry of a parabola confirms $x=30$ is the max.

What to see in the figure. The magenta curve rises steeply near $x=0$, flattens to its top at the orange dot ($x=30,y=45$ — the dashed violet line marks it), then bends back down and crosses zero near $x=60$. Trace the curve left to right and notice the slope shrinking to zero at the peak and then turning negative: that visual "bend" is exactly what the derivative $3-0.1x$ computes. The plunge on the right is the quadratic term overpowering the linear one — the geometric picture of why unchecked polynomial features extrapolate to nonsense.

---

Cell H — Real-world word problem

Step 1 — Convert degree gaps to radians. $\Delta \varphi =(28.70-28.61)\frac{\pi }{180}=0.09\times 0.017453=0.0015708,$ $\Delta \lambda =(77.10-77.20)\frac{\pi }{180}=-\mathrm{0.001745.}$ Why this step? Trig functions ($\sin ,\cos$) take radians, not degrees. This is the #1 silent bug in geo features.

Step 2 — Haversine core. $a={\sin }^2\left(\frac{\Delta \varphi }{2}\right)+\cos {\varphi }_1\cos {\varphi }_2{\sin }^2\left(\frac{\Delta \lambda }{2}\right).$ Why this step? Cities lie on a sphere; straight-line coordinate differences would ignore curvature. Haversine gives true great-circle distance.

Step 3 — Distance. $d=2r\arcsin (\sqrt{a})\approx 13.3\text{km}.$ Why this step? The raw four coordinates never encode "distance" — this single engineered feature is the primary driver of fare (parent note, geospatial example).

Verify (back-of-envelope): Break the trip into a north–south leg and an east–west leg.

• North–south: $1°$ of latitude is always $\approx r\cdot \frac{\pi }{180}$ km, so $0.09°$ gives $r\Delta \varphi =6371\times 0.0015708\approx 10.0$ km.
• East–west: at latitude $\varphi \approx 28.6°$, lines of longitude are squeezed together by a factor $\cos \varphi$ (they meet at the poles), so $1°$ of longitude spans only $r\cos \varphi \cdot \frac{\pi }{180}$ km. With $\cos (28.6°)\approx 0.878$, the $0.10°$ gap ($|\Delta \lambda |=0.001745$ rad) gives $6371\times 0.878\times 0.001745\approx 9.76$ km.

Combine the two perpendicular legs with Pythagoras: $\sqrt{10.0^2+9.76^2}\approx 14.0$ km. ✓ Lands near the "15 km" forecast anchor and within about $1$ km of the exact Haversine value $13.3$ km (the small gap comes from our flat-plane approximation of a curved surface). The $\cos \varphi$ factor is essential — without it you'd overcount the east–west distance.

---

Cell I — Exam-style twist (the "drop the main effects" trap)

Step 1 — Compute the true value. $y=10(3)+5(4)+2(3)(4)=30+20+24=74.$

Step 2 — Show the interaction alone cannot separate two points. The interaction feature value is $x_1{x}_2=12$. The same $12$ arises from $(3,4)$, $(6,2)$, $(12,1)$ — all give the same feature but different true $y$. True $y$ at $(6,2)$ is $60+10+24=94$. Why this step? Both points force the model to output the same $\hat{y}=12\beta$, but the truths differ.

Step 3 — Quantify the unavoidable error. Whatever $\beta$ the student chooses, both points get the identical prediction $\hat{y}=12\beta$. The kindest choice is the midpoint of the two truths, $\hat{y}=\frac{74+94}{2}=84.$ With that best-possible $\hat{y}=84$, the absolute error at each point is $|84-74|=10,|84-94|=10.$ Why this step? Because a single number ($12\beta$) must serve two different targets, the best it can do is sit halfway, leaving an error of $10$ at each point. The irreducible spread the interaction-only model can never resolve is $94-74=20$ units — that is precisely the information the dropped main effects $10x_1+5x_2$ carried. See Linear Regression hypothesis space and the parent note's Mistake 2.

Verify: $74\ne 94$ while both have $x_1{x}_2=12$; the irreducible spread is $94-74=20$, and the best-case error at the split point is $20/2=10$. ✓ No single $\beta$ fits both.

---

Recall Self-test: which cell?

A model outputs exactly its bias term. Which matrix cell? ::: Cell F (all inputs zero) You feed impressions = 0 into clicks/impressions. Which cell, and the fix? ::: Cell E; add a smoothing prior $\alpha ,\beta$ so the denominator is never zero Two people have identical raw debt but the model ranks them differently. What feature made that possible? ::: Cell D — a ratio (debt-to-income), which is scale-invariant Why does a degree-2 poly on 2 features give 5 new columns, not 6? ::: $\left(\genfrac{}{}{0ex}{}{4}{2}\right)-1=5$; we subtract the constant term because it duplicates the bias An interaction term went negative even with a positive weight. What caused it? ::: Cell C — one input was negative, flipping the product's sign

See also: Feature Selection Methods and Cross-Validation to decide which of these engineered features actually earn their place; Decision Trees and Random Forests can learn some interactions automatically; start every project with Exploratory Data Analysis to know which cells you'll even face.