Intuition The One Core IdeaA learning model can only see relationships you have already written into the columns of your data — if the pattern isn't in a feature, no training will find it. Feature creation is the craft of building brand-new columns (products, ratios, powers) so that even a simple straight-line model can trace a curved or "it depends" relationship.
This page assumes nothing . Every letter, subscript, and squiggle used in the parent note (the topic note) is unpacked here, one at a time, each building on the last. If a symbol confused you upstairs, it is defined below with a picture.
Definition Feature, sample, and the data tableA feature is one measured quantity — one column in a spreadsheet. A sample (or row) is one thing you measured — one house, one day, one customer. The whole dataset is a grid: rows are samples, columns are features.
Look at the figure. Each column is a feature (say temperature ). Each row is a sample (one day's record). "Creating a feature" literally means drawing a new column whose values are computed from the existing ones. That is the entire game.
x x i x subscript . So x 1 x 2 x i i i whichever one we're talking about .
Picture a row of labelled boxes: box x 1 x 2 address of a box.
x x = [ x 1 , x 2 , … , x n ] x x entire row of numbers . The three dots … n count of features — the last box's address.
x i x 2 x i address — which feature. x 2 power — the feature multiplied by itself. Same letter, totally different jobs. Low = label, high = maths.
⋅ × multiply . x 1 ⋅ x 2 x 1 × x 2 ⋅ ×
Why does the topic care so much about a humble multiply sign? Because multiplication is how you encode "it depends" — see §6.
w b y y weighting each feature and adding them up:
y = w 1 x 1 + w 2 x 2 + b
w 1 , w 2 weights — dials the model turns to say how much each feature matters . Big weight = strong influence.b bias — a constant baseline added no matter what the features are (the value of y y prediction — the number that comes out.
Intuition Why "linear" means "straight"In the figure, plot one feature x 1 y y = w 1 x 1 + b straight line : b x 1 = 0 w 1 steepness (rise over run). Turning the weight tilts the line; turning the bias slides it up or down. A straight line is all this model can draw — and that is exactly the limitation feature creation exists to fix.
Real relationships bend. Crop yield rises with temperature, peaks, then falls when it's too hot. That shape is a hill , not a line — a straight line can never trace it.
x 2 x 2 x × x x … , 4 , 1 , 0 , 1 , 4 , … U-shaped curve called a parabola.
Look at the two coloured curves. A positive weight on x 2 negative weight flips it into a hill (∩) — exactly the "peaks in the middle" shape crop yield needs. This is why the topic adds x 2 polynomial feature .
Definition Polynomial and degreeA polynomial is a sum of a variable raised to whole-number powers, e.g. w 1 x + w 2 x 2 degree is the highest power present. Degree 1 = straight line; degree 2 = one bend; degree 3 = two bends; and so on.
The parent counts how many new features appear. That count uses a special symbol.
( d n ) ( d n ) "n choose d" ) counts how many different groups of size d you can pick from n ( d n ) = d ! ( n − d )! n ! n ! factorial ) means "multiply every whole number from 1 up to n 4 ! = 4 ⋅ 3 ⋅ 2 ⋅ 1 = 24
Worked example Counting degree-2 features forn = 2 2 2 ( d n + d ) − 1 = ( 2 4 ) − 1 ( 2 4 ) = 2 ! 2 ! 4 ! = 2 ⋅ 2 24 = 4 24 = 6 , 6 − 1 = 5. x 1 , x 2 , x 1 2 , x 1 x 2 , x 2 2 1 b
Why does the topic need this symbol? To warn you : the feature count explodes as degree grows, which drives the "combinatorial explosion" mistake in the parent note.
This is the heart of the topic, so we build it fully.
Definition Interaction termAn interaction term is a new feature made by multiplying two existing features : x 1 ⋅ x 2 y = β 1 x 1 + β 2 x 2 + β 3 ( x 1 ⋅ x 2 ) + b . β β w
Why multiplication and not, say, addition? Watch what algebra reveals. Group every term that contains x 1 y = x 1 ( β 1 + β 3 x 2 ) + β 2 x 2 + b .
Intuition The "effective slope" pictureThe quantity in the bracket, β 1 + β 3 x 2 effective weight on x 1 — and it changes as x 2 . When x 2 β 1 x 2 β 3 x 2 x 1 tilts differently at different x 2 . That is the mathematical face of "it depends" — the effect of ad spend depends on whether it's the weekend.
In the figure, the two lines are the same feature x 1 x 2 without the interaction is forced to use one slope for both — it literally cannot draw two.
x j x i x i − x j divide : x j x i x i x j relative size (debt relative to income). The minus x i − x j change or gap (today's price minus yesterday's).
Intuition Why dividing "normalizes"$50k debt is scary on $30k income (ratio 1.67 0.25 both numbers into one scale-free value, so the model compares burdens, not dollars. (More on making features comparable in scale: Feature Scaling and Normalization .)
The parent's distance formula uses two Greek letters you can now decode:
ϕ phi ) = latitude (how far north/south), λ lambda ) = longitude (how far east/west), r sin cos arcsin angle differences on a sphere into a straight-line-on-the-globe distance. You don't need to derive them here — just know each symbol names a real, pictured quantity (an angle or a radius), never a free-floating letter.
Data table rows and columns
Feature x and subscript i
Linear model weights w and bias b
Polynomial features and degree
Choose symbol counts features
Interaction term x1 times x2
Effective slope depends on other feature
Ratio and difference features
Cover each answer and test yourself; reveal only after you've tried.
What does a subscript x i Which feature — it is an address/label, not a maths operation.
How is x i x 2 Low i 2
What do the bold x x Bold x x
In y = w 1 x 1 + b w 1 b w 1 b x 1 = 0
Why can't a linear model draw a hill shape? A linear (straight-line) model has no term that bends — it needs a created feature like x 2
What shape does x 2 A parabola (U-shape); a negative weight flips it into a ∩ (hill).
What does ( d n ) 4 ! The number of ways to choose d n 4 ! = 4 ⋅ 3 ⋅ 2 ⋅ 1 = 24
Compute ( 2 4 ) − 1 6 − 1 = 5 x 1 , x 2 , x 1 2 , x 1 x 2 , x 2 2
Why is an interaction made by multiplying, not adding? Multiplying makes the effective slope of one feature ( β 1 + β 3 x 2 ) depend on the other — encoding "it depends."
What does a ratio x i / x j A scale-free, relative value (e.g. burden relative to income) instead of absolute magnitude.
What do ϕ λ ϕ λ
