Level 1 — RecognitionLinear & Logistic Regression

Linear & Logistic Regression

⏱ 20 minutes30 marksprintable — key stays hidden on paper

Difficulty Level: 1 (Recognition — MCQ, Matching, True/False with justification) Time Limit: 20 minutes Total Marks: 30

Section A — Multiple Choice (1 mark each) [10 marks]

Q1. The simple linear regression model is written as $y = \beta_0 + \beta_1 x + \varepsilon$ . The term $\beta_0$ represents the:

(a) slope
(b) intercept
(c) error term
(d) residual variance

Q2. The sigmoid function used in logistic regression is:

(a) $\sigma(z) = \max(0, z)$
(b) $\sigma(z) = \dfrac{1}{1 + e^{-z}}$
(c) $\sigma(z) = z^2$
(d) $\sigma(z) = \tanh(z) + 1$

Q3. The normal equation closed-form solution for linear regression coefficients is:

(a) $\hat{\beta} = (X^TX)^{-1}X^Ty$
(b) $\hat{\beta} = X^Ty(X^TX)^{-1}$
(c) $\hat{\beta} = (XX^T)^{-1}Xy$
(d) $\hat{\beta} = X^{-1}y$

Q4. L1 (Lasso) regularization is preferred over L2 (Ridge) primarily because it:

(a) always gives lower training error
(b) can drive some coefficients exactly to zero (feature selection)
(c) is differentiable everywhere
(d) never overfits

Q5. The cost function most commonly minimized in linear regression is:

(a) binary cross-entropy
(b) mean squared error (MSE)
(c) hinge loss
(d) absolute intercept

Q6. For a classification task with more than two classes, the appropriate generalization of logistic regression is:

(a) polynomial regression
(b) softmax (multinomial) regression
(c) ridge regression
(d) simple linear regression

Q7. Which quantity always lies between 0 and 1 for an ordinary least squares fit (with intercept) and measures the proportion of variance explained?

(a) MSE
(b) $R^2$
(c) $\beta_1$
(d) log-loss

Q8. Elastic Net regularization combines:

(a) MSE and cross-entropy
(b) L1 and L2 penalties
(c) softmax and sigmoid
(d) gradient descent and normal equation

Q9. In gradient descent, the learning rate $\alpha$ controls the:

(a) number of features
(b) step size of each parameter update
(c) intercept value
(d) number of classes

Q10. Polynomial regression fits a nonlinear curve while still being a linear model because it is linear in:

(a) the input feature $x$
(b) the parameters (coefficients)
(c) the output $y$ only
(d) the learning rate

Section B — Matching (1 mark each) [8 marks]

Q11–Q18. Match each term in Column X with its correct description in Column Y. Write the letter.

#	Column X		Column Y
Q11	Log-loss	A	Penalty term $\lambda\sum\beta_j^2$
Q12	L2 penalty	B	Threshold in feature space separating predicted classes
Q13	Decision boundary	C	Loss for binary classification: $-[y\log p + (1-y)\log(1-p)]$
Q14	Adjusted $R^2$	D	Adds features via powers of $x$
Q15	Multiple linear regression	E	Adjusts $R^2$ for number of predictors
Q16	Polynomial regression	F	More than one input feature: $y=\beta_0+\beta_1x_1+\dots+\beta_px_p$
Q17	Interpreting coefficient $\beta_j$	G	Expected change in $y$ per unit change in $x_j$ , others fixed
Q18	Sigmoid output	H	A probability between 0 and 1

Section C — True/False with Justification (2 marks each: 1 verdict + 1 justification) [12 marks]

Q19. True or False: One assumption of linear regression is that residuals are homoscedastic (constant variance). Justify.

Q20. True or False: Adding more predictors to a multiple regression model can never decrease the $R^2$ value. Justify.

Q21. True or False: The normal equation is always preferable to gradient descent regardless of the number of features. Justify.

Q22. True or False: In logistic regression, the decision boundary produced by a linear model is always a straight line/hyperplane in the original feature space. Justify.

Q23. True or False: Increasing the regularization strength $\lambda$ in Ridge regression tends to shrink coefficient magnitudes toward zero. Justify.

Q24. True or False: Minimizing MSE for classification with a sigmoid output is preferred over log-loss because it is convex in the parameters. Justify.

Answer keyMark scheme & solutions

Section A — MCQ (1 mark each)

Q	Ans	Why
Q1	(b)	$\beta_0$ is the value of $y$ when $x=0$ — the intercept; $\beta_1$ is the slope.
Q2	(b)	The logistic/sigmoid maps $\mathbb{R}\to(0,1)$ as $1/(1+e^{-z})$ .
Q3	(a)	OLS minimizer: setting $\nabla_\beta\\|y-X\beta\\|^2=0$ gives $X^TX\hat\beta=X^Ty\Rightarrow\hat\beta=(X^TX)^{-1}X^Ty$ .
Q4	(b)	The L1 penalty's non-smooth corners at zero produce sparse solutions (exact zeros).
Q5	(b)	Regression minimizes mean squared error.
Q6	(b)	Softmax/multinomial regression handles $K>2$ classes.
Q7	(b)	$R^2\in[0,1]$ (with intercept) = fraction of variance explained.
Q8	(b)	Elastic Net = $\lambda_1\\|\beta\\|_1 + \lambda_2\\|\beta\\|_2^2$ .
Q9	(b)	$\alpha$ scales the gradient step: $\beta\leftarrow\beta-\alpha\nabla J$ .
Q10	(b)	Model $y=\sum\beta_j x^j$ is linear in the $\beta_j$ , so OLS applies.

Section B — Matching (1 mark each)

Q	Ans
Q11	C (log-loss / binary cross-entropy)
Q12	A (L2 penalty = sum of squared coefficients)
Q13	B (decision boundary)
Q14	E (adjusted $R^2$ )
Q15	F (multiple linear regression)
Q16	D (polynomial regression)
Q17	G (coefficient interpretation, ceteris paribus)
Q18	H (sigmoid → probability)

Section C — True/False with Justification (1 + 1 marks)

Q19. TRUE. (1) Homoscedasticity — constant error variance across all fitted values — is a standard OLS assumption. (1) It ensures OLS estimators are efficient (BLUE) and that standard errors/inference are valid.

Q20. TRUE. (1) Adding predictors cannot decrease $R^2$ . (1) OLS can set new coefficients to zero, so the residual sum of squares can only stay equal or fall; hence $R^2$ is non-decreasing. (This is exactly why adjusted $R^2$ exists — to penalize added predictors.)

Q21. FALSE. (1) The normal equation is not always preferable. (1) Computing $(X^TX)^{-1}$ costs ~ $O(p^3)$ and is unstable/infeasible for very large numbers of features; gradient descent scales better in high dimensions.

Q22. TRUE (in the given features). (1) True. (1) The boundary is where $\beta^Tx=0$ (i.e. $p=0.5$ ), which is a hyperplane linear in the features supplied. (Curved boundaries arise only if nonlinear/polynomial features are added — then it's still linear in the expanded feature space.)

Q23. TRUE. (1) Larger $\lambda$ shrinks coefficients toward zero. (1) The penalty $\lambda\sum\beta_j^2$ makes large coefficients costly; increasing $\lambda$ raises that cost, forcing smaller magnitudes (though not exactly zero, unlike L1).

Q24. FALSE. (1) Log-loss is preferred, not MSE. (1) MSE with a sigmoid is non-convex in the parameters and can stall in flat regions; log-loss (cross-entropy) is convex for logistic regression, giving reliable optimization.

[
  {"claim":"Sigmoid(0)=0.5","code":"z=symbols('z'); sig=1/(1+exp(-z)); result = (sig.subs(z,0)==Rational(1,2))"},
  {"claim":"OLS normal equation solves X^T X beta = X^T y (2x2 identity design gives beta=X^T y)","code":"X=Matrix([[1,0],[0,1]]); y=Matrix([3,5]); beta=(X.T*X).inv()*X.T*y; result = (beta==Matrix([3,5]))"},
  {"claim":"Ridge shrinks: for X=[[1]],y=[2],lambda=1 solution 2/(1+1)=1 < OLS 2","code":"lam=1; Xr=Matrix([[1]]); yr=Matrix([2]); b=(Xr.T*Xr+lam*eye(1)).inv()*Xr.T*yr; result = (b[0]==Rational(1,1)) and (Rational(1,1) < 2)"},
  {"claim":"Elastic net penalty at beta=(1,-1) with l1=l2=1 equals 4","code":"b1,b2=1,-1; pen=(abs(b1)+abs(b2))+(b1**2+b2**2); result = (pen==4)"}
]