Feature creation and interaction terms: Question bank

Before starting, a shared vocabulary so nothing below uses an unearned word:

True or false — justify

A linear model with enough data can eventually learn

y = x_{1} \cdot x_{2}

from just

x_{1}

and

x_{2}

False. The product isn't in its hypothesis space — a weighted sum can never become a product, so no amount of data helps until you create

x_{1} x_{2}

explicitly.

Adding a degree-2 polynomial expansion always improves test accuracy.

False. It raises expressive power but also variance; on noisy or small data it usually causes overfitting, hurting test accuracy even as train accuracy rises.

A ratio feature like debt/income throws away information compared to keeping both raw numbers.

Partly false as stated. The ratio adds a scale-invariant view, but you should usually keep the raw features too, since the ratio alone hides absolute magnitude (0.5 on $1M behaves differently from 0.5 on $10k).

Tree-based models need manually created interaction terms just as much as linear models do.

False. Trees learn interactions automatically by splitting on one feature then another down a branch; explicit interaction terms help linear models far more than trees.

Standardizing features to mean 0, unit variance removes the need to worry about interaction scaling.

False. After you multiply two standardized features the product has its own (non-unit) scale and skew, so you often re-scale interaction terms; see Feature Scaling and Normalization.

If two features are perfectly correlated, their interaction term adds genuinely new information.

False. If

x_{2}

is a linear function of

x_{1}

, then

x_{1} x_{2}

is essentially

x_{1}^{2}

— it collapses into the polynomial terms and worsens multicollinearity rather than adding signal.

Creating price_per_sqft from price and sqft is safe to use as a predictor of price.

False — that's target leakage. The ratio contains price itself, so at prediction time (when price is unknown) the feature cannot be computed honestly.

Polynomial degree 3 is strictly more powerful than degree 2, so it's the safer default.

False. "More powerful" means "more ways to overfit." Degree 3 explodes feature count and correlated terms; degree 2 is the sane default, escalated only with evidence from Cross-Validation.

Spot the error

"I dropped

x_{1}

and

x_{2}

but kept

x_{1} \cdot x_{2}

, since the interaction already captures their relationship."

Error: without main effects the model is forced through a constrained form and mis-attributes each variable's own linear effect to the product, biasing every coefficient. Keep the main effects.

"My model has

x_{1}

x_{2}

, and

x_{1} x_{2}

; the coefficient on

x_{1}

is the total effect of

x_{1}

Error. With an interaction present,

x_{1}

's effective slope is

β_{1} + β_{3} x_{2}

— the lone

β_{1}

is only the effect when $x_{2} = 0$ , not the total effect.

"I built degree-4 features on all 10 columns to be thorough, then trained a plain linear model."

Error: that's ~1000 features, a combinatorial explosion inviting overfitting and multicollinearity. Use Feature Selection Methods or L1/L2 and start at degree 2.

"I computed rolling_mean_7 centered on each day, including future days, for my stock model."

Error: a centered rolling window peeks into the future → look-ahead leakage. Use only trailing windows (past days) for any forecast feature.

"I created interactions using EDA on the full dataset, then split into train/test."

Error: choosing features by looking at all data leaks test information into feature design. Do Exploratory Data Analysis and feature construction on the training fold only, validated by Cross-Validation.

"Since

T^{2}

and

R^{2}

are both squares, they're basically the same feature."

Error. They square different variables, capturing independent curvatures (optimal temperature vs. flood-diminishing rain). They're only redundant if

T

and

R

themselves are collinear.

"The interaction ad_spend × is_weekend doubles my ad effect everywhere."

Error. Because is_weekend is 0 on weekdays, the product is 0 then — the extra slope activates only on weekends, not everywhere.

Why questions

Why multiplication for interactions, and not addition?

Addition is already available (a linear model sums features), so it captures nothing new. Multiplication makes one feature's slope depend on the other's value — that dependence is the whole point of "interaction."

Why does creating features help linear models more than flexible ones?

Linear models are locked to weighted sums; feature creation is the only way to smuggle nonlinearity and interactions into their fixed hypothesis space. Flexible models can find some of that on their own.

Why subtract 1 in

(d n + d) - 1

for polynomial feature counts?

The expansion includes the constant term (the "degree-0" feature

1

), which is redundant with the model's bias/intercept, so we don't count it as a new feature.

Why can a ratio be dangerous even when it's not leaking the target?

Division blows up when the denominator nears zero, producing huge unstable values, and it discards magnitude. A near-zero income makes debt/income explode, distorting the model and its scaling.

Why pair interaction engineering with L1 regularization specifically?

Interactions create many candidate features, most useless; L1 drives unhelpful coefficients to exactly zero, performing automatic selection and taming multicollinearity-driven overfitting.

Why does an inverted-U outcome (like crop yield vs. temperature) require a squared term rather than the raw term alone?

A single linear term is monotonic — always up or always down. Only a curved term (

T^{2}

with a negative weight) can rise then fall, encoding an interior optimum.

Why check interaction usefulness with cross-validation instead of just training error?

Training error always drops when you add features (more flexibility), so it can't distinguish real signal from memorized noise. CV estimates generalization, revealing whether the interaction actually helps.

Edge cases

What happens to

x_{1} \cdot x_{2}

when one feature is a 0/1 indicator that is 0?

The interaction vanishes to 0, so that feature's extra slope contributes nothing — the model falls back to the main-effect-only form for those rows. This is exactly the intended "activate only when the flag is on" behavior.

What if you form the interaction of a feature with itself,

x_{1} \cdot x_{1}

That's just the polynomial term

x_{1}^{2}

, capturing curvature rather than a two-feature synergy — a legitimate feature but conceptually a self-interaction, not an interaction between distinct variables.

What if both interacting features are centered so their means are 0?

Centering keeps the product meaningful but shifts interpretation: main-effect coefficients now describe the effect at the average of the other feature, which is often the desired, more stable interpretation.

What does a percent-change feature

\frac{x _{i} - x _{i - 1}}{x _{i - 1}}

do when the previous value is 0?

It divides by zero — undefined/infinite. You must guard with a small epsilon, an imputed value, or a rule flagging the row, otherwise the feature is invalid.

What if a created feature is a perfect linear combination of existing ones?

It adds zero new information and makes the design matrix rank-deficient, so the linear model's weights become non-unique/unstable — extreme multicollinearity. Drop it or rely on regularization.

What happens to Haversine distance when pickup and dropoff coordinates are identical?

The distance is exactly 0, a valid degenerate case (same point). It's well-behaved, but a fare model should still handle 0-distance rows sensibly (e.g. base fare only).

Recall One-line survival kit

Keep main effects with interactions? ::: Yes — dropping them biases every coefficient. Default polynomial degree? ::: 2, escalate only with CV evidence. Biggest silent killer of created features? ::: Leakage — a feature built from the target or the future. Best tools to tame feature explosion? ::: L1/L2 plus selection.