Noesis
Level 3 — ProductionMachine Learning (Aerospace Applications)

Machine Learning (Aerospace Applications)

⏱ 45 minutes60 marksprintable — key stays hidden on paper

Chapter: 5.6 Machine Learning for Aerospace Level: 3 — Production (from-scratch derivations, code-from-memory, explain-out-loud) Time limit: 45 minutes Total marks: 60

Instructions: Show every derivation step. Where code is requested, write it from memory (NumPy-style pseudocode acceptable if syntactically consistent). Use ....../...... for math.

Question 1 — Linear Regression from Scratch (10 marks)

(a) Starting from the mean-squared-error cost J(w)=12m∥Xw−y∥2,J(\mathbf{w}) = \tfrac{1}{2m}\lVert X\mathbf{w} - \mathbf{y}\rVert^2, derive the normal equation by setting ∇wJ=0\nabla_\mathbf{w} J = 0. Show the gradient explicitly. (4)

(b) Add L2 (ridge) regularization λ2m∥w∥2\tfrac{\lambda}{2m}\lVert\mathbf{w}\rVert^2 and derive the modified closed-form solution. State one numerical benefit of the regularizer. (3)

(c) Write the batch gradient-descent update rule for the unregularized cost and give the vectorized gradient expression. (3)

Question 2 — Logistic Regression & Cross-Entropy (10 marks)

(a) Define the sigmoid σ(z)\sigma(z) and prove that σ′(z)=σ(z) (1−σ(z))\sigma'(z) = \sigma(z)\,(1-\sigma(z)). (3)

(b) For a single training example with label y∈{0,1}y\in\{0,1\}, prediction y^=σ(w⊤x)\hat{y}=\sigma(\mathbf{w}^\top\mathbf{x}), write the binary cross-entropy loss LL. Derive ∂L∂w\dfrac{\partial L}{\partial \mathbf{w}} and show it simplifies to (y^−y) x(\hat{y}-y)\,\mathbf{x}. (5)

(c) Explain in one sentence why cross-entropy is preferred over MSE for logistic regression training. (2)

Question 3 — Backpropagation by Hand (12 marks)

Consider a 2-layer network: input xx (scalar), hidden unit h=tanh⁡(w1x+b1)h=\tanh(w_1 x + b_1), output y^=w2h+b2\hat{y}=w_2 h + b_2, loss L=12(y^−y)2L=\tfrac{1}{2}(\hat{y}-y)^2.

(a) Do the forward pass symbolically. (2)

(b) Using the chain rule, derive ∂L∂w2\dfrac{\partial L}{\partial w_2}, ∂L∂w1\dfrac{\partial L}{\partial w_1}, and ∂L∂b1\dfrac{\partial L}{\partial b_1}. Recall tanh⁡′(z)=1−tanh⁡2(z)\tanh'(z)=1-\tanh^2(z). (6)

(c) Numerically evaluate all three gradients for x=1, y=0, w1=0.5, b1=0, w2=1, b2=0x=1,\ y=0,\ w_1=0.5,\ b_1=0,\ w_2=1,\ b_2=0. Show intermediate values. (4)

Question 4 — Optimizers from Memory (10 marks)

(a) Write the update equations for SGD with momentum (velocity form) and explain the role of the momentum coefficient β\beta. (3)

(b) Write the full Adam update equations including bias correction. Define every symbol. (5)

(c) In one sentence each, contrast batch, mini-batch, and stochastic gradient descent by variance and compute cost. (2)

Question 5 — Attention / Self-Attention (8 marks)

(a) Write the scaled dot-product attention formula and explain the purpose of the dk\sqrt{d_k} scaling factor. (3)

(b) Given query/key/value matrices Q,K,VQ, K, V, explain out loud (in words) the shape flow and what each matrix multiplication computes in self-attention. (3)

(c) State why self-attention is more parallelizable than an RNN for sequence modeling. (2)

Question 6 — Reinforcement Learning for GNC (10 marks)

(a) Write the Bellman optimality equation for the action-value function Q∗(s,a)Q^*(s,a). (3)

(b) Write the tabular Q-learning update rule and identify the learning rate and discount factor. (3)

(c) A guidance agent in state ss takes action aa, receives reward r=2r=2, lands in s′s' where max⁡a′Q(s′,a′)=10\max_{a'}Q(s',a')=10. With current Q(s,a)=5Q(s,a)=5, learning rate α=0.1\alpha=0.1, discount γ=0.9\gamma=0.9, compute the updated Q(s,a)Q(s,a). (4)

Answer keyMark scheme & solutions

Question 1 (10)

(a) Expand J=12m(Xw−y)⊤(Xw−y)J=\tfrac{1}{2m}(X\mathbf{w}-\mathbf{y})^\top(X\mathbf{w}-\mathbf{y}). Gradient: ∇wJ=1mX⊤(Xw−y)\nabla_\mathbf{w}J = \tfrac{1}{m}X^\top(X\mathbf{w}-\mathbf{y}) (2) — from matrix calculus ∂∥Xw−y∥2/∂w=2X⊤(Xw−y)\partial \lVert Xw-y\rVert^2/\partial w = 2X^\top(Xw-y). Set to zero: X⊤Xw=X⊤yX^\top X\mathbf{w}=X^\top\mathbf{y} → w=(X⊤X)−1X⊤y\boxed{\mathbf{w}=(X^\top X)^{-1}X^\top\mathbf{y}} (2).

(b) Cost gradient 1mX⊤(Xw−y)+λmw=0\tfrac{1}{m}X^\top(X\mathbf{w}-\mathbf{y})+\tfrac{\lambda}{m}\mathbf{w}=0 → (X⊤X+λI)w=X⊤y(X^\top X+\lambda I)\mathbf{w}=X^\top\mathbf{y} w=(X⊤X+λI)−1X⊤y\boxed{\mathbf{w}=(X^\top X+\lambda I)^{-1}X^\top\mathbf{y}} (2). Benefit: X⊤X+λIX^\top X+\lambda I is always invertible / better conditioned (reduces variance) (1).

(c) Update: w←w−α 1mX⊤(Xw−y)\mathbf{w}\leftarrow \mathbf{w}-\alpha\,\tfrac{1}{m}X^\top(X\mathbf{w}-\mathbf{y}) (3) (α\alpha = learning rate).

Question 2 (10)

(a) σ(z)=11+e−z\sigma(z)=\dfrac{1}{1+e^{-z}} (1). σ′(z)=e−z(1+e−z)2=σ(z)⋅e−z1+e−z=σ(z)(1−σ(z))\sigma'(z)=\dfrac{e^{-z}}{(1+e^{-z})^2}=\sigma(z)\cdot\dfrac{e^{-z}}{1+e^{-z}}=\sigma(z)(1-\sigma(z)) (2).

(b) L=−[yln⁡y^+(1−y)ln⁡(1−y^)]L=-[y\ln\hat{y}+(1-y)\ln(1-\hat{y})] (1). ∂L∂y^=−yy^+1−y1−y^=y^−yy^(1−y^)\dfrac{\partial L}{\partial \hat{y}}=-\dfrac{y}{\hat{y}}+\dfrac{1-y}{1-\hat{y}}=\dfrac{\hat{y}-y}{\hat{y}(1-\hat{y})} (2). With z=w⊤xz=\mathbf{w}^\top\mathbf{x}, ∂y^∂z=y^(1−y^)\dfrac{\partial\hat{y}}{\partial z}=\hat{y}(1-\hat{y}), ∂z∂w=x\dfrac{\partial z}{\partial\mathbf{w}}=\mathbf{x}. Chain: ∂L∂w=y^−yy^(1−y^)⋅y^(1−y^)⋅x=(y^−y)x\dfrac{\partial L}{\partial\mathbf{w}}=\dfrac{\hat{y}-y}{\hat{y}(1-\hat{y})}\cdot\hat{y}(1-\hat{y})\cdot\mathbf{x}=(\hat{y}-y)\mathbf{x} (2).

(c) MSE with sigmoid is non-convex and its gradient vanishes when saturated; cross-entropy gives convex loss and non-vanishing gradient (y^−y)(\hat y-y) (2).

Question 3 (12)

(a) h=tanh⁡(w1x+b1)h=\tanh(w_1x+b_1), y^=w2h+b2\hat{y}=w_2h+b_2, L=12(y^−y)2L=\tfrac12(\hat y-y)^2 (2).

(b) Let δ=y^−y\delta=\hat y-y. ∂L∂w2=δ h\dfrac{\partial L}{\partial w_2}=\delta\,h (2). ∂L∂w1=δ w2 (1−h2) x\dfrac{\partial L}{\partial w_1}=\delta\,w_2\,(1-h^2)\,x (2). ∂L∂b1=δ w2 (1−h2)\dfrac{\partial L}{\partial b_1}=\delta\,w_2\,(1-h^2) (2).

(c) z1=0.5z_1=0.5, h=tanh⁡(0.5)=0.46212h=\tanh(0.5)=0.46212, y^=1⋅0.46212=0.46212\hat y=1\cdot0.46212=0.46212, δ=0.46212−0=0.46212\delta=0.46212-0=0.46212. 1−h2=1−0.21356=0.786441-h^2=1-0.21356=0.78644.

  • ∂L/∂w2=0.46212⋅0.46212=0.21356\partial L/\partial w_2=0.46212\cdot0.46212=0.21356 (1.5)
  • ∂L/∂w1=0.46212⋅1⋅0.78644⋅1=0.36343\partial L/\partial w_1=0.46212\cdot1\cdot0.78644\cdot1=0.36343 (1.5)
  • ∂L/∂b1=0.36343\partial L/\partial b_1=0.36343 (same, x=1x=1) (1)

Question 4 (10)

(a) v←βv+(1−β)∇θJv\leftarrow \beta v+(1-\beta)\nabla_\theta J (or v←βv+∇Jv\leftarrow\beta v+\nabla J); θ←θ−αv\theta\leftarrow\theta-\alpha v (2). β\beta accumulates an exponentially-weighted average of past gradients, damping oscillations and accelerating along consistent directions (1).

(b) (5) mt=β1mt−1+(1−β1)gtm_t=\beta_1 m_{t-1}+(1-\beta_1)g_t (1st moment) vt=β2vt−1+(1−β2)gt2v_t=\beta_2 v_{t-1}+(1-\beta_2)g_t^2 (2nd moment) m^t=mt/(1−β1t)\hat m_t=m_t/(1-\beta_1^t), v^t=vt/(1−β2t)\hat v_t=v_t/(1-\beta_2^t) (bias correction) θt=θt−1−α m^t/(v^t+ϵ)\theta_t=\theta_{t-1}-\alpha\,\hat m_t/(\sqrt{\hat v_t}+\epsilon). Symbols: gtg_t gradient, α\alpha step size, β1,β2\beta_1,\beta_2 decay rates, ϵ\epsilon numerical stabilizer.

(c) Batch: uses all data → lowest gradient variance, highest per-step cost. Mini-batch: subset → moderate variance and cost (standard). Stochastic: one sample → highest variance, cheapest per step (2).

Question 5 (8)

(a) Attn(Q,K,V)=softmax ⁣(QK⊤dk)V\text{Attn}(Q,K,V)=\text{softmax}\!\left(\dfrac{QK^\top}{\sqrt{d_k}}\right)V (2). Scaling by dk\sqrt{d_k} keeps dot-product magnitudes bounded so softmax gradients don't vanish (avoids saturation) (1).

(b) QK⊤QK^\top → (n×n)(n\times n) score matrix comparing each query to each key; softmax normalizes rows into attention weights; multiply by VV (n×dvn\times d_v) to get weighted sums → output (n×dv)(n\times d_v) (3).

(c) All positions attend simultaneously via matrix multiply (no sequential recurrence), so computation parallelizes across the sequence (2).

Question 6 (10)

(a) Q∗(s,a)=E[r+γmax⁡a′Q∗(s′,a′) ∣ s,a]Q^*(s,a)=\mathbb{E}\big[r+\gamma\max_{a'}Q^*(s',a')\,\big|\,s,a\big] (3).

(b) Q(s,a)←Q(s,a)+α[r+γmax⁡a′Q(s′,a′)−Q(s,a)]Q(s,a)\leftarrow Q(s,a)+\alpha\big[r+\gamma\max_{a'}Q(s',a')-Q(s,a)\big] (2); α\alpha = learning rate, γ\gamma = discount (1).

(c) TD target =2+0.9⋅10=11=2+0.9\cdot10=11. TD error =11−5=6=11-5=6. Update =5+0.1⋅6=5.6=5+0.1\cdot6=\boxed{5.6} (4).

[
  {"claim":"Q3 gradient dL/dw2 = 0.21356 for given inputs","code":"import math; h=math.tanh(0.5); yhat=1*h; delta=yhat-0; dw2=delta*h; result = abs(dw2-0.21356)<1e-4"},
  {"claim":"Q3 dL/dw1 = 0.36343","code":"import math; h=math.tanh(0.5); yhat=h; delta=yhat; dw1=delta*1*(1-h**2)*1; result = abs(dw1-0.36343)<1e-4"},
  {"claim":"Q6 Q-learning update yields 5.6","code":"Q=5; r=2; g=0.9; a=0.1; maxQ=10; newQ=Q+a*(r+g*maxQ-Q); result = abs(newQ-5.6)<1e-9"},
  {"claim":"sigmoid derivative identity at z=0 equals 0.25","code":"import math; s=1/(1+math.exp(0)); dsig=s*(1-s); result = abs(dsig-0.25)<1e-9"}
]