Level 4 — ApplicationData Preprocessing & Feature Engineering

Data Preprocessing & Feature Engineering

⏱ 60 minutes50 marksprintable — key stays hidden on paper

Level 4 — Application (novel problems, no hints) Time limit: 60 minutes Total marks: 50

Answer all questions. Show all working. Round final numeric answers to 3 decimal places unless told otherwise.

Question 1 — Scaling & transformation pipeline (10 marks)

A feature income (in thousands) has the following training-set values:

$\{20,\ 25,\ 30,\ 40,\ 900\}$

(a) Compute the min-max scaled value of the point $income = 30$ using training-set statistics. (2)

(b) Compute the z-score standardized value of $income = 30$ using the population standard deviation. (3)

(c) A validation-set point has $income = 1000$ . Give its min-max scaled value and explain in one sentence what problem this reveals about min-max scaling with this feature. (2)

(d) You apply a $\log_{10}(x)$ transform first, then min-max scale. Compute the min-max scaled value of the original point $income = 30$ under this new pipeline. Explain why the log step changes the result for the point 30. (3)

Question 2 — Missing values & encoding on a novel table (12 marks)

A dataset of loan applicants:

id	region	grade (A>B>C)	balance
1	North	A	500
2	South	C	?
3	North	B	300
4	East	A	700
5	South	?	300

(a) The balance for id 2 is missing. Impute it using the median of observed balances. State the value. (2)

(b) grade for id 5 is missing. Impute it using the mode. State the value and one risk of mode imputation here. (2)

(c) After imputation, apply ordinal encoding to grade with $A=3, B=2, C=1$ , and one-hot encoding to region, dropping the first category alphabetically to avoid the dummy trap. Write the full encoded feature row for id 3. (4)

(d) Explain why one-hot encoding is preferred over ordinal encoding for region but ordinal encoding is appropriate for grade. (2)

(e) You consider target encoding region using default probability. Describe one concrete way this can cause data leakage and one technique to prevent it. (2)

Question 3 — Outliers & IQR (8 marks)

Consider the sorted feature values:

$\{4,\ 7,\ 8,\ 9,\ 11,\ 12,\ 14,\ 15,\ 18,\ 41\}$

(a) Compute $Q_1$ , $Q_3$ and the IQR (use linear interpolation, $Q_1$ at position $0.25(n+1)$ , $Q_3$ at $0.75(n+1)$ ). (4)

(b) Using the $1.5 \times IQR$ rule, determine the lower and upper fences and list any outliers. (3)

Question 4 — Splitting, leakage & imbalance (12 marks)

A fraud dataset has 100,000 transactions, of which 500 are fraud.

(a) State the fraud class proportion, and explain why plain random splitting into train/test may be unreliable for evaluation here. Name the split strategy that fixes it. (3)

(b) A colleague standardizes the entire dataset using the global mean and std, then splits into train/test. Identify the data-leakage flaw and state the correct order of operations. (3)

(c) SMOTE is applied to balance the classes. Explain where in the pipeline SMOTE must be applied relative to the train/test split, and why applying it before the split inflates measured performance. (3)

(d) After undersampling the majority to a 1:1 ratio, the model reports 92% accuracy on the balanced set but performs poorly in production. Explain the mismatch and name a more appropriate evaluation metric. (3)

Question 5 — Correlation & multicollinearity (8 marks)

Two features and a target from 5 samples:

$x_1$	$x_2$	$y$
1	2	3
2	4	5
3	6	7
4	8	9
5	10	11

(a) Compute the Pearson correlation between $x_1$ and $x_2$ . Show enough working to justify the value. (3)

(b) State what this implies about multicollinearity and one problem it causes for a linear model's coefficients. (2)

Answer keyMark scheme & solutions

Question 1 (10)

Training stats: $\min=20$ , $\max=900$ , range $=880$ . Mean $=\frac{20+25+30+40+900}{5}=\frac{1015}{5}=203$ .

(a) Min-max: $\frac{30-20}{900-20}=\frac{10}{880}=0.01136$ . (2) (1 formula, 1 value)

(b) Population variance: Deviations from 203: $-183,-178,-173,-163,697$ . Squares: $33489,31684,29929,26569,485809 = 607480$ . $\sigma^2=607480/5=121496$ , $\sigma=348.563$ . z $=\frac{30-203}{348.563}=\frac{-173}{348.563}=-0.496$ . (3) (1 mean, 1 std, 1 z)

(c) $\frac{1000-20}{880}=1.114$ (>1). Reveals min-max does not bound out-of-range future data and is highly sensitive to the extreme training max (900), which compresses normal values. (2)

(d) After $\log_{10}$ : values $=\{1.301,1.398,1.477,1.602,2.954\}$ . $\min=1.301$ , $\max=2.954$ , range $=1.653$ . Point 30 → $\log_{10}30=1.477$ . Scaled $=\frac{1.477-1.301}{2.954-1.301}=\frac{0.176}{1.653}=0.107$ . (3) (1 log values, 1 computation, 1 explanation) Explanation: log compresses the large 900 value, reducing the range distortion so 30 maps to a much higher scaled position (0.107 vs 0.011).

Question 2 (12)

(a) Observed balances: 500, 300, 700, 300. Sorted: 300,300,500,700. Median $=\frac{300+500}{2}=400$ . (2)

(b) Grades observed: A, C, B, A → mode = A (appears twice). Risk: mode imputation biases the distribution toward the majority class and may be wrong for a South/300-balance applicant. (2)

(c) id 3: region=North, grade=B, balance=300. grade → 2. Regions {East, North, South}; drop first alphabetically (East). Dummies: region_North=1, region_South=0. Row: [grade=2, region_North=1, region_South=0, balance=300]. (4) (1 grade, 2 one-hot, 1 balance)

(d) region is nominal (no inherent order); ordinal encoding would impose a false ordering/magnitude, misleading the model. grade is ordinal (A>B>C), so integer ranks preserve genuine order. (2)

(e) Leakage: encoding a row using target info that includes its own target (or the full dataset before splitting) leaks the label. Prevention: compute target statistics using cross-fold / out-of-fold encoding or fit encoder only on the training fold, with smoothing. (2)

Question 3 (8)

$n=10$ . (a) $Q_1$ position $=0.25(11)=2.75$ → between 2nd (7) and 3rd (8): $7+0.75(8-7)=7.75$ . $Q_3$ position $=0.75(11)=8.25$ → between 8th (15) and 9th (18): $15+0.25(18-15)=15.75$ . IQR $=15.75-7.75=8.0$ . (4)

(b) Lower fence $=7.75-1.5(8)=7.75-12=-4.25$ . Upper fence $=15.75+1.5(8)=15.75+12=27.75$ . Outliers: values outside $[-4.25, 27.75]$ → 41. (3)

(c) 41 may be a genuine/valid extreme observation (not an error); deleting it discards real signal and biases the model. (Or: small sample, deletion loses information.) (1)

Question 4 (12)

(a) Proportion $=500/100000=0.005=0.5\%$ . Random split may put too few/zero fraud cases in test, giving unstable/misleading evaluation. Fix: stratified splitting (preserve class ratio in each split). (3)

(b) Standardizing on the whole dataset lets test-set statistics leak into training (mean/std computed with test data). Correct order: split first, fit scaler on train only, then transform train and test with train parameters. (3)

(c) SMOTE must be applied only to the training set, after the split (inside CV folds). Applying before splitting creates synthetic points from samples that later appear in the test set, so test points resemble training data → optimistically inflated performance. (3)

(d) Undersampling makes the evaluation set 1:1, so 92% accuracy is measured on an artificial distribution; in production the class ratio is 0.5%, where accuracy is dominated by the majority class and is misleading. Use F1 / precision-recall AUC / recall on the minority class. (3)

Question 5 (8)

(a) $x_2=2x_1$ exactly (perfect linear relation). Means: $\bar{x_1}=3,\ \bar{x_2}=6$ . Covariance numerator $\sum(x_1-3)(x_2-6)$ with deviations $x_1:(-2,-1,0,1,2)$ , $x_2:(-4,-2,0,2,4)$ : products $8+2+0+2+8=20$ . $\sum(x_1-3)^2=10$ , $\sum(x_2-6)^2=40$ . $r=\frac{20}{\sqrt{10\cdot40}}=\frac{20}{20}=1.0$ . (3)

(b) Perfect multicollinearity ( $r=1$ ). The linear model's coefficients become unidentifiable/unstable (matrix singular; coefficients not uniquely determined, high variance). (2)

(c) Any two: (1) Drop one redundant feature — removes exact duplication of information. (2) PCA / dimensionality reduction — combines correlated features into orthogonal components. (3) Regularization (Ridge) — stabilizes coefficient estimates under collinearity. (Justification required for each.) (3)

[
  {"claim":"Q1a min-max of 30 = 10/880","code":"result = abs(Rational(10,880) - Rational(1,88)) < Rational(1,10**9)"},
  {"claim":"Q1b z-score of 30 approx -0.496","code":"sig=sqrt(Rational(607480,5)); z=(30-203)/sig; result = abs(float(z)+0.496) < 0.01"},
  {"claim":"Q3 IQR=8, upper fence=27.75, outlier 41","code":"Q1=Rational(775,100); Q3=Rational(1575,100); IQR=Q3-Q1; up=Q3+Rational(3,2)*IQR; result = (IQR==8) and (up==Rational(2775,100)) and (41>up)"},
  {"claim":"Q5a Pearson r(x1,x2)=1","code":"d1=[-2,-1,0,1,2]; d2=[-4,-2,0,2,4]; num=sum(a*b for a,b in zip(d1,d2)); den=sqrt(sum(a*a for a in d1)*sum(b*b for b in d2)); result = Rational(num)/den == 1"}
]