Data Preprocessing & Feature Engineering

Q1 (12)

(a) Mean $\mu = \frac{2+4+4+4+5+5+7+9}{8} = \frac{40}{8} = 5$. (1) Deviations squared: $(2-5)^2+3(4-5)^2+2(5-5)^2+(7-5)^2+(9-5)^2 = 9+3+0+4+16 = 32$. (1) Population variance $\sigma^2 = 32/8 = 4$; $\sigma = 2$. (2) $z = \frac{9-5}{2} = 2$. (1)

(b) $x_{\min}=2$, $x_{\max}=9$, range $=7$. For $x=4$: $\frac{4-2}{9-2} = \frac{2}{7} \approx 0.2857$. (2) General range $[a,b]$: $x' = a + (b-a)\cdot\frac{x-x_{\min}}{x_{\max}-x_{\min}}$. (2)

(c) Fit on train only, transform val/test with those stored params. (1) If you fit using the whole dataset, statistics of val/test leak into the transform → data leakage, giving optimistically biased performance estimates. (2)

Q2 (10)

(a) Sorted (n=9): $[7,8,9,10,10,11,12,13,40]$. Using linear interpolation (numpy default): $Q_1$ at position $0.25(9-1)=2$ → index 2 → $Q_1 = 9$. (2) $Q_3$ at position $0.75\cdot8 = 6$ → index 6 → $Q_3 = 12$. IQR $= 12-9 = 3$. (2)

(b) Lower fence $= 9 - 1.5(3) = 4.5$; Upper fence $= 12 + 1.5(3) = 16.5$. (2) Points outside $[4.5, 16.5]$: only 40 is an outlier. (2)

(c) When the outlier is a genuine/valid extreme value carrying real signal (e.g., fraud, rare disease, sensor spike of interest) — deleting it loses information. (2)

Q3 (12)

(a) One-hot (columns blue, green, red order or any consistent order):

3 columns. (3) Drop-first is used to avoid the dummy variable trap / perfect multicollinearity (one column is redundant given the intercept). (1)

(b) A: mean$= (1+0+1+1)/4 = 0.75$. (1.5) B: $(0+0+1)/3 = 0.3333$. (1.5)

(c) Naive encoding uses each row's own target to build the feature → the target directly informs the feature, leaking label information into $X$. (1.5) Prevention (any one): out-of-fold/K-fold target encoding, leave-one-out encoding, or smoothing with the global mean + adding noise. (1.5)

(d) When cardinality is very high (one-hot explodes dimensionality) or the categories have a genuine order (ordinal), or for tree-based models that handle integer splits well. (2)

Q4 (8)

(a) $\log_{10}[1,10,100,1000] = [0, 1, 2, 3]$. (3)

(b) Log compresses large values more than small ones (multiplicative → additive spacing), pulling in the long right tail so the distribution becomes more symmetric and variance more constant across the range. (3)

(c) Use $\log(x+1)$ (i.e., log1p) to handle zeros: $x' = \log(1+x)$. (2)

Q5 (10)

(a) Imbalance ratio $950:50 = 19:1$ (5% positives). (1) A model predicting all-negative gets 95% accuracy while catching zero positives → accuracy hides failure on the minority class. (1)

(b) SMOTE steps: (4, 1 each)

1. For each minority sample, find its $k$ nearest minority neighbours.
2. Randomly pick one neighbour.
3. Draw random $\lambda \in [0,1]$.
4. Synthetic point $= x + \lambda\,(x_{\text{neighbour}} - x)$ (interpolate along the line segment). Repeat to reach target count.

(c) Apply SMOTE after the split, on the training set only. (2) If applied before splitting, synthetic points generated from test/val neighbours (or duplicates thereof) leak into training → inflated, invalid evaluation. Val/test must reflect the true class distribution. (2)

Q6 (8)

(a) $\bar x = 2.5$, $\bar y = 5$. Deviations $x$: $[-1.5,-0.5,0.5,1.5]$, $y$: $[-3,-1,1,3]$. Cov numerator $= 4.5+0.5+0.5+4.5 = 10$; $\text{cov} = 10/4 = 2.5$ (or /3 for sample). (2) $\sigma_x = \sqrt{(2.25+0.25+0.25+2.25)/4} = \sqrt{1.25}$, $\sigma_y = \sqrt{20/4}=\sqrt5$. $r = \frac{2.5}{\sqrt{1.25}\cdot\sqrt5} = \frac{2.5}{2.5} = 1$. (2) (Perfect positive since $y=2x$.)

(b) $\text{VIF}_i = \frac{1}{1-R_i^2}$. (1) For $R_i^2=0.9$: $\text{VIF}=1/0.1 = 10$. (1)

(c) Consequence: unstable/inflated coefficient variances → unreliable coefficient estimates & signs. (1) Remedy: drop one collinear feature, use PCA, or apply regularization (ridge). (1)

[
  {"claim":"Z-score of x=9 with pop std is 2","code":"x=[2,4,4,4,5,5,7,9]; mu=sum(x)/len(x); var=sum((v-mu)**2 for v in x)/len(x); z=(9-mu)/sqrt(var); result=(z==2)"},
  {"claim":"Min-max of x=4 in [0,1] is 2/7","code":"val=(4-2)/(9-2); result=simplify(val-Rational(2,7))==0"},
  {"claim":"IQR Tukey upper fence flags only 40","code":"import numpy as np; d=np.array([7,8,9,10,10,11,12,13,40]); q1=np.percentile(d,25); q3=np.percentile(d,75); iqr=q3-q1; up=q3+1.5*iqr; lo=q1-1.5*iqr; out=[v for v in d if v>up or v<lo]; result=(out==[40])"},
  {"claim":"Pearson r of x and y=2x is 1","code":"import numpy as np; x=np.array([1,2,3,4.]); y=np.array([2,4,6,8.]); r=np.corrcoef(x,y)[0,1]; result=bool(abs(r-1)<1e-9)"},
  {"claim":"VIF at R2=0.9 equals 10","code":"vif=1/(1-Rational(9,10)); result=(vif==10)"}
]