You have met the four data types in the parent note . Now we do the thing that actually builds skill: we walk through every kind of situation the topic can hand you, one worked example per situation, until no surprise is left.
Before any symbol appears, here is the vocabulary we will lean on, in plain words:
Definition The words we will use
Mean (μ Standard deviation (σ σ σ ∑ ∑ i = 1 n x i x 1 , x 2 , … x n Vector : just an ordered list of numbers, like [ 1 , 0 , 0 ] Cardinality : how many distinct categories a column has.
Everything below is built only from these.
Every problem this topic throws at you is one of the cells below. Our job is to hit all of them.
#
Case class
What makes it tricky
Example that covers it
C1
Numerical, normal spread
baseline scaling
Ex 1
C2
Numerical, negative & positive values
sign of ( x − μ )
Ex 2
C3
Numerical, degenerate: all values equal
σ = 0 Ex 3
C4
Numerical, outlier present
mean & σ
Ex 4
C5
Categorical, nominal (no order)
one-hot, equal distances
Ex 5
C6
Categorical, high cardinality
one-hot explodes
Ex 6
C7
Ordinal, ordered categories
integer codes must respect order
Ex 7
C8
Ordinal used in a distance model — the trap
uneven gaps assumed uniform
Ex 8
C9
Real-world word problem mixing typespick the right tool per column
Ex 9
C10
Exam twist : discrete number → categorical?judgement call
Ex 10
The picture below is the mental model for every numerical example: a number line with the mean nailed at the middle, and scaling stretches or squeezes the ruler so one step = one σ
Worked example Scale the ages[20, 24, 22, 26, 28]
Forecast: guess — will the value 22 end up positive or negative after scaling?
Step 1. Mean: μ = 5 20 + 24 + 22 + 26 + 28 = 5 120 = 24 Why this step? We need the centre before we can measure distance from it.
Step 2. Deviations from mean: − 4 , 0 , − 2 , 2 , 4 Why? ( x − μ )
Step 3. σ = 5 16 + 0 + 4 + 4 + 16 = 5 40 = 8 ≈ 2.83 Why? σ
Step 4. Scale 22: 2.83 22 − 24 = 2.83 − 2 ≈ − 0.71 Why? 22 sits below the mean, so it must come out negative — confirming the forecast.
Verify: the whole scaled list is [ − 1.41 , 0 , − 0.71 , 0.71 , 1.41 ] 0 ( − 1.41 + 0 − 0.71 + 0.71 + 1.41 ) = 0
Worked example Scale temperatures[-10, 0, 5, -5, 10] (°C)
Forecast: the raw values already have both signs. Does that break anything?
Step 1. μ = 5 − 10 + 0 + 5 − 5 + 10 = 5 0 = 0 Why? The data already balances at zero — a lucky mean.
Step 2. Deviations = − 10 , 0 , 5 , − 5 , 10 μ = 0 Why? ( x − μ ) = ( x − 0 ) = x
Step 3. σ = 5 100 + 0 + 25 + 25 + 100 = 5 250 = 50 ≈ 7.07
Step 4. Scale − 10 7.07 − 10 − 0 ≈ − 1.41 10 + 1.41 Why? Sign of the raw number is preserved — scaling never invents or hides signs, it only re-rulers them.
Verify: scaled = [ − 1.41 , 0 , 0.71 , − 0.71 , 1.41 ] = 0 negative inputs are not special — ( x − μ )
[7, 7, 7, 7]
Forecast: what is σ
Step 1. μ = 4 7 + 7 + 7 + 7 = 7
Step 2. Every deviation = 7 − 7 = 0 Why? No number differs from the mean.
Step 3. σ = 4 0 + 0 + 0 + 0 = 0 = 0
Step 4. Scaling would be 0 7 − 7 = 0 0 undefined .
Why does this matter? A column with zero variance carries no information — it is the same for every row.
Verify: σ = 0 The fix: detect σ = 0 StandardScaler) leave such columns at 0. See Normalization and Scaling for the safe-guard.
Common mistake Divide-by-zero on constant columnsFeeding a constant feature into a naive scaler produces NaN (not-a-number), which then poisons every gradient and silently kills training — see Feature Scaling Impact on Gradient Descent . Always check variance first.
Worked example Scale incomes (in lakhs)[3, 4, 5, 4, 100]
Forecast: the value 100 is an outlier. Will the ordinary values (3,4,5) land near 0 or get squashed to near 0?
Step 1. μ = 5 3 + 4 + 5 + 4 + 100 = 5 116 = 23.2 Why? One huge value yanks the mean far above the typical value — the mean is not robust.
Step 2. σ = 5 ( 3 − 23.2 ) 2 + ( 4 − 23.2 ) 2 + ( 5 − 23.2 ) 2 + ( 4 − 23.2 ) 2 + ( 100 − 23.2 ) 2 408.04 + 368.64 + 331.24 + 368.64 + 5898.24 = 7374.8 σ = 1474.96 ≈ 38.41
Step 3. Scale 3: 38.41 3 − 23.2 ≈ − 0.526 38.41 100 − 23.2 ≈ 2.0 Why? The three normal values crowd into a narrow band near − 0.5
Verify: scaled list ≈ [ − 0.526 , − 0.500 , − 0.474 , − 0.500 , 1.999 ] ≈ 0 Lesson (C4): when outliers exist, prefer robust scaling (median/IQR) — see Normalization and Scaling . Standard scaling works but compresses the majority.
Picture each category as its own axis. One-hot encoding places every category on a different axis, so no two categories are "closer" than any others — the geometry below.
Worked example One-hot encodecolor ∈ {red, green, blue} for rows [red, blue, green]
Forecast: how many new columns, and what is the distance between any two colours?
Step 1. Distinct categories = { } 3 Why? One binary column per category so none implies order.
Step 2. Encode: red = [ 1 , 0 , 0 ] = [ 0 , 1 , 0 ] = [ 0 , 0 , 1 ]
Step 3. Rows become [ 1 , 0 , 0 ] , [ 0 , 0 , 1 ] , [ 0 , 1 , 0 ]
Step 4. Distance red↔blue = ( 1 − 0 ) 2 + ( 0 − 0 ) 2 + ( 0 − 1 ) 2 = 2 Why? Every pair of distinct one-hot vectors differs in exactly two slots → distance always 2 One-Hot Encoding vs Label Encoding .
Verify: 2 ≈ 1.414
Worked example One-hot encode apincode column with 19{,}000 distinct pincodes
Forecast: how many columns does one-hot create, and why is that a problem?
Step 1. One-hot columns = = 19 , 000 Why? One column per distinct value — the rule from Ex 5 scales linearly with cardinality.
Step 2. Each row has 19 , 000 0 1 sparse vector).
Why worry? Memory blows up and models drown in dimensions — the Curse of Dimensionality .
Step 3. Better tools: target/frequency encoding, hashing, or grouping rare values — see Handling High-Cardinality Categorical .
Verify: columns = 19000 Lesson (C6): one-hot is correct but impractical past ~50 categories.
size ∈ {S, M, L, XL} for rows [M, XL, S, L]
Forecast: does the order of the codes matter here (unlike colours)?
Step 1. Fix the order: S < M < L < X L Why? Ordinal data has real order; we must state it before coding.
Step 2. Map S → 0 , M → 1 , L → 2 , X L → 3 Why? Integers preserve < ≥ 2
Step 3. Rows → [ 1 , 3 , 0 , 2 ]
Verify: codes are strictly increasing along the size order: 0 < 1 < 2 < 3 >
Worked example Satisfaction{Poor=0, Fair=1, Good=2, Excellent=3} fed to k-NN
Forecast: the model computes distances. Is Poor→Fair the "same size" gap as Good→Excellent?
Step 1. In the codes, gap Poor→Fair = 1 − 0 = 1 = 3 − 2 = 1 Why? Integer codes force uniform spacing — every step is exactly 1.
Step 2. But in reality jumping from Poor to Fair may feel far smaller than Good to Excellent ; the true gaps are uneven.
Why it matters: k-NN treats "distance 1" as one fixed amount, so it silently assumes gaps are equal.
Step 3. Distance Poor↔Excellent = ∣3 − 0∣ = 3
Verify: both adjacent gaps = 1 Lesson (C8): ordinal integer codes are safe for tree splits but risky for distance/linear models unless the gaps really are near-uniform.
Worked example A loan dataset has columns:age (years), city (43 values), rating (Bronze<Silver<Gold), salary (₹). Choose a treatment for each, for a logistic regression model.
Forecast: which columns get scaled, which get one-hot, which get ordinal codes?
Step 1. age, salary → numerical → standard scaling (C1 logic), because a linear model's weights are scale-sensitive (see Feature Scaling Impact on Gradient Descent ).
Step 2. city → nominal, cardinality 43 → not plain one-hot (43 columns, C6 problem) → use frequency/target encoding via Handling High-Cardinality Categorical .
Step 3. rating → ordinal → integer codes Bronze= 0 = 1 = 2 acceptable here because a logistic model can learn a monotonic weight, and only 3 near-even levels.
Verify (sanity): salary and age now share the same scale (mean 0, sd 1); city no longer costs 43 columns; rating preserves order. Every column is now numeric — the model can consume it. Count of engineered numeric inputs = 2 ( scaled ) + 1 ( city enc ) + 1 ( rating ) = 4
num_bedrooms ∈ {1,2,3,4,5} for a tree model. Treat as numerical or one-hot categorical?
Forecast: both are defensible — what tips the decision?
Step 1. It is genuinely numerical and ordered (5 bedrooms > 3 bedrooms is meaningful).
Why? So keeping it as an integer preserves true information.
Step 2. Cardinality is tiny (5). One-hot would add only 5 columns — cheap, and lets the tree pick out a specific count (e.g. exactly 3) without an ordering assumption.
Why? Sometimes "exactly 3 bedrooms" is special (a market sweet-spot), which a ≥ / ≤
Step 3. Decision: keep numeric for trees (they split on thresholds fine, cost 1 column), consider one-hot only if a non-monotonic pattern is suspected.
Verify: one-hot column count would be 5 1 1 < 5
Recall Quick self-testScaled value of 22 in [20,24,22,26,28] ::: ≈ − 0.71 σ [7,7,7,7] ::: 0 2
N umbers → N ormalize · C ategory → C olumns (one-hot) · O rder → O rdinal ints. Watch for the two ambushes: σ = 0
Back to the parent topic .