Level 1 — RecognitionMachine Learning (Aerospace Applications)

Machine Learning (Aerospace Applications)

⏱ 20 minutes40 marksprintable — key stays hidden on paper

Time limit: 20 minutes
Total marks: 40
Instructions: Answer all questions. For True/False items, a correct justification is required for full marks. Use $...$ notation where needed.

Section A — Multiple Choice (1 mark each, 12 marks)

Q1. The normal equation for linear regression is:

(a) $\theta = (X^TX)^{-1}X^Ty$
(b) $\theta = (X^TX)X^Ty$
(c) $\theta = (XX^T)^{-1}Xy$
(d) $\theta = X^{-1}y$

Q2. The sigmoid activation function $\sigma(z)$ is defined as:

(a) $\max(0,z)$
(b) $\dfrac{1}{1+e^{-z}}$
(c) $\dfrac{e^z-e^{-z}}{e^z+e^{-z}}$
(d) $\dfrac{e^z}{\sum e^{z_j}}$

Q3. L1 regularization (lasso) adds which penalty term to the loss?

(a) $\lambda \sum \theta_j^2$
(b) $\lambda \sum |\theta_j|$
(c) $\lambda \sum \theta_j$
(d) $\lambda \max_j |\theta_j|$

Q4. A model with high bias and low variance most likely suffers from:

(a) Overfitting
(b) Underfitting
(c) Perfect fit
(d) Data leakage

Q5. In $k$ -fold cross-validation with $k=5$ , the model is trained:

(a) once
(b) 5 times, each on 4 folds
(c) 5 times, each on 1 fold
(d) 25 times

Q6. The derivative of the sigmoid $\sigma(z)$ equals:

(a) $\sigma(z)$
(b) $1-\sigma(z)$
(c) $\sigma(z)(1-\sigma(z))$
(d) $\sigma(z)^2$

Q7. The Adam optimizer combines:

(a) L1 and L2 regularization
(b) momentum (first moment) and RMSProp-style (second moment) estimates
(c) dropout and batch normalization
(d) forward and backward passes

Q8. In a CNN, a max-pooling layer primarily:

(a) increases spatial resolution
(b) downsamples by taking the maximum in each window
(c) adds learnable weights per pixel
(d) applies the cross-entropy loss

Q9. In an LSTM, the gate that decides what information to discard from the cell state is the:

(a) input gate
(b) output gate
(c) forget gate
(d) reset gate

Q10. The scaled dot-product attention weights are computed as:

(a) $\text{softmax}(QK^T)V$
(b) $\text{softmax}\!\left(\dfrac{QK^T}{\sqrt{d_k}}\right)V$
(c) $QK^TV$
(d) $\text{softmax}(Q+K)V$

Q11. The Bellman optimality equation for state value is:

(a) $V^*(s)=\max_a \sum_{s'}P(s'|s,a)[r+\gamma V^*(s')]$
(b) $V^*(s)=\sum_a \pi(a|s)r$
(c) $V^*(s)=r+\gamma V^*(s)$
(d) $V^*(s)=\min_a Q(s,a)$

Q12. Mini-batch gradient descent, compared to full-batch, generally offers:

(a) exact gradients every step
(b) a trade-off between gradient noise and computational efficiency
(c) no stochasticity at all
(d) guaranteed convergence to the global minimum

Section B — Matching (8 marks: 1 each)

Q13. Match each concept (i–viii) to its correct description (A–H).

#	Concept
i	ReLU
ii	Cross-entropy loss
iii	Backpropagation
iv	REINFORCE
v	System identification
vi	Dropout
vii	BPTT
viii	Q-learning

Letter	Description
A	Randomly deactivates neurons during training to reduce overfitting
B	$f(z)=\max(0,z)$
C	Policy-gradient method scaling log-prob by return
D	Off-policy value-based RL update using $\max_a Q$
E	Applying the chain rule to compute loss gradients w.r.t. weights
F	Learning system dynamics (a model) from measured input/output data
G	Backpropagation unrolled through time for RNNs
H	$-\sum y\log\hat{y}$ , used for classification

Section C — True/False with Justification (4 marks each, 20 marks)

Q14. The tanh activation function outputs values in the range $[0,1]$ . — True or False? Justify.

Q15. L2 (ridge) regularization tends to drive some weights exactly to zero, producing sparse models. — True or False? Justify.

Q16. Increasing model complexity always decreases the test error. — True or False? Justify.

Q17. In fault detection for aerospace systems, an ML classifier can be trained on sensor features to flag anomalous engine behaviour before failure. — True or False? Justify.

Q18. In stochastic gradient descent (SGD), the gradient is computed using the entire training set at every update. — True or False? Justify.

Answer keyMark scheme & solutions

Section A (1 mark each)

Q1 — (a) $\theta=(X^TX)^{-1}X^Ty$ . This is the closed-form minimizer of $\|X\theta-y\|^2$ , obtained by setting $\nabla_\theta J = 2X^T(X\theta-y)=0$ . (1)

Q2 — (b) $\sigma(z)=\frac{1}{1+e^{-z}}$ ; (a) is ReLU, (c) is tanh, (d) is softmax. (1)

Q3 — (b) L1 uses the sum of absolute values; (a) is L2. (1)

Q4 — (b) High bias = underfitting (model too simple to capture structure). (1)

Q5 — (b) Data split into 5 folds; each iteration trains on 4 and validates on 1, so 5 trainings. (1)

Q6 — (c) $\sigma'(z)=\sigma(z)(1-\sigma(z))$ . (1)

Q7 — (b) Adam = adaptive moment estimation: first moment (momentum) + second moment (RMSProp). (1)

Q8 — (b) Max-pooling downsamples by taking the max over each window; it has no learnable weights. (1)

Q9 — (c) The forget gate $f_t=\sigma(W_f[h_{t-1},x_t]+b_f)$ controls what is removed from the cell state. (1)

Q10 — (b) Scaled dot-product attention divides by $\sqrt{d_k}$ to stabilize gradients. (1)

Q11 — (a) Bellman optimality: value = best action's expected reward + discounted next value. (1)

Q12 — (b) Mini-batches trade off gradient noise vs. compute cost. (1)

Section B — Matching (1 each)

Q13: i→B, ii→H, iii→E, iv→C, v→F, vi→A, vii→G, viii→D. (8)

Section C — True/False (4 marks each)

Q14 — False. (1 for F) tanh outputs lie in $(-1,1)$ , not $[0,1]$ ; it is $\frac{e^z-e^{-z}}{e^z+e^{-z}}$ , zero-centred. (3 for justification: range, formula, sigmoid gives $[0,1]$ )

Q15 — False. (1) L2 shrinks weights smoothly toward (but not exactly to) zero; it is L1 (lasso) that induces exact-zero sparsity because its penalty has a non-differentiable corner at the origin. (3)

Q16 — False. (1) Increasing complexity lowers bias but raises variance; beyond an optimum the test error rises (overfitting) — the bias–variance trade-off gives a U-shaped test-error curve. (3)

Q17 — True. (1) A supervised classifier trained on labelled sensor/telemetry features (vibration, temperature, pressure) can learn to distinguish nominal from anomalous signatures and issue early warnings; this is a standard fault-detection / predictive-maintenance application. (3)

Q18 — False. (1) SGD uses a single training example (or small mini-batch) per update; using the entire set is batch gradient descent. The stochastic estimate is noisier but cheaper. (3)

[
  {"claim":"Normal equation theta=(X^T X)^{-1} X^T y solves least squares",
   "code":"X=Matrix([[1,1],[1,2],[1,3]]); y=Matrix([1,2,2]); th=(X.T*X).inv()*X.T*y; import sympy; g=2*X.T*(X*th-y); result=(g==zeros(2,1))"},
  {"claim":"Sigmoid derivative equals sigma(1-sigma)",
   "code":"z=symbols('z'); s=1/(1+exp(-z)); d=diff(s,z); result=simplify(d-s*(1-s))==0"},
  {"claim":"tanh range is (-1,1): tanh at large positive ->1, large negative ->-1",
   "code":"result=(limit(tanh(z),z,oo)==1) and (limit(tanh(z),z,-oo)==-1)"},
  {"claim":"tanh(0)=0 (zero-centred), sigmoid(0)=1/2 (not zero-centred)",
   "code":"result=(tanh(0)==0) and ((1/(1+exp(-Integer(0))))==Rational(1,2))"}
]