Level 1 — RecognitionNeural Network Fundamentals

Neural Network Fundamentals

⏱ 20 minutes30 marksprintable — key stays hidden on paper

Time Limit: 20 minutes Total Marks: 30

Section A — Multiple Choice (1 mark each)

Choose the single best answer.

Q1. The perceptron model was introduced by:

(a) Geoffrey Hinton
(b) Frank Rosenblatt
(c) Yann LeCun
(d) Marvin Minsky

Q2. Which logical function CANNOT be learned by a single-layer perceptron?

(a) AND
(b) OR
(c) XOR
(d) NOT

Q3. The sigmoid activation function $\sigma(x)=\frac{1}{1+e^{-x}}$ outputs values in the range:

(a) $(-1, 1)$
(b) $(0, 1)$
(c) $[0, \infty)$
(d) $(-\infty, \infty)$

Q4. The ReLU activation function is defined as:

(a) $\max(0, x)$
(b) $\min(0, x)$
(c) $\frac{1}{1+e^{-x}}$
(d) $\tanh(x)$

Q5. Softmax is typically used in the:

(a) Input layer
(b) Hidden layer for regression
(c) Output layer for multi-class classification
(d) Bias unit

Q6. Cross-entropy loss is most appropriate for:

(a) Regression tasks
(b) Classification tasks
(c) Clustering
(d) Dimensionality reduction

Q7. He initialization is specifically recommended for networks using:

(a) Sigmoid activations
(b) Tanh activations
(c) ReLU activations
(d) Linear activations only

Q8. The vanishing gradient problem is most associated with:

(a) ReLU in deep networks
(b) Saturating activations like sigmoid/tanh in deep networks
(c) Using bias terms
(d) Softmax output layers

Q9. The Universal Approximation Theorem states that a feedforward network with one hidden layer can:

(a) Always achieve zero training error
(b) Approximate any continuous function on a compact set to arbitrary accuracy
(c) Solve XOR only
(d) Guarantee convergence of gradient descent

Q10. Backpropagation fundamentally relies on which calculus rule?

(a) Product rule only
(b) Chain rule
(c) L'Hôpital's rule
(d) Integration by parts

Section B — Matching (1 mark each, 5 marks total)

Q11–Q15. Match each activation/function (Column A) with its defining property (Column B).

Column A		Column B
Q11. Tanh		(i) $\max(0,x)$ ; zero gradient for negatives
Q12. Leaky ReLU		(ii) Outputs range $(-1,1)$ , zero-centered
Q13. ReLU		(iii) Converts logits to a probability distribution summing to 1
Q14. Softmax		(iv) $\max(\alpha x, x)$ with small $\alpha$ ; nonzero negative slope
Q15. GELU		(v) Smooth approx. weighting input by Gaussian CDF

Section C — True / False WITH Justification (3 marks each: 1 mark T/F, 2 marks justification)

Q16. The bias term allows the activation threshold to shift, enabling the neuron to fire even when all inputs are zero. (True/False + justify)

Q17. In forward propagation for layer $l$ , the pre-activation is computed as $z^{(l)} = W^{(l)} a^{(l-1)} + b^{(l)}$ , then $a^{(l)} = f(z^{(l)})$ . (True/False + justify)

Q18. The derivative of the sigmoid function is $\sigma'(x) = \sigma(x)(1-\sigma(x))$ , and its maximum value is 0.5. (True/False + justify)

Q19. Xavier (Glorot) initialization sets weight variance based on both fan-in and fan-out to keep signal variance stable across layers. (True/False + justify)

Q20. A computational graph enables automatic differentiation by storing operations as nodes so gradients can be computed via reverse-mode traversal. (True/False + justify)

Answer keyMark scheme & solutions

Section A — MCQ (1 mark each)

Q1. (b) Frank Rosenblatt — introduced the perceptron in 1958. Why: historical fact; Minsky/Papert later critiqued it.

Q2. (c) XOR — not linearly separable, so a single perceptron (a linear classifier) cannot represent it. Why: XOR requires a nonlinear decision boundary.

Q3. (b) $(0,1)$ — sigmoid squashes any real input into the open interval $(0,1)$ . Why: as $x\to-\infty$ , $\sigma\to0$ ; as $x\to+\infty$ , $\sigma\to1$ .

Q4. (a) $\max(0,x)$ — definition of ReLU. Why: passes positive inputs, clips negatives to 0.

Q5. (c) Output layer for multi-class classification. Why: softmax normalizes logits into a probability distribution.

Q6. (b) Classification tasks. Why: cross-entropy measures divergence between predicted and true distributions.

Q7. (c) ReLU activations. Why: He init scales variance by $2/n_{in}$ to account for ReLU zeroing half the activations.

Q8. (b) Saturating activations (sigmoid/tanh) in deep networks. Why: derivatives near 0 in saturation shrink gradients multiplicatively.

Q9. (b) Approximate any continuous function on a compact set to arbitrary accuracy. Why: existence theorem, not a training guarantee.

Q10. (b) Chain rule. Why: gradients of composite functions propagate layer-by-layer.

Section B — Matching (1 mark each)

Q11 → (ii) Tanh: range $(-1,1)$ , zero-centered.
Q12 → (iv) Leaky ReLU: $\max(\alpha x, x)$ .
Q13 → (i) ReLU: $\max(0,x)$ , zero gradient for negatives.
Q14 → (iii) Softmax: probability distribution summing to 1.
Q15 → (v) GELU: weights input by Gaussian CDF.

Section C — True/False + Justification (3 marks each)

Q16. TRUE (1). Justification (2): The bias $b$ shifts the pre-activation $z = w^Tx + b$ ; even if $x=0$ , $z=b\neq 0$ can push the neuron past its threshold. It effectively translates the decision boundary independent of inputs.

Q17. TRUE (1). Justification (2): Correct forward-prop equations — weighted sum plus bias gives pre-activation $z^{(l)}$ , then elementwise activation $f$ produces $a^{(l)}$ , which feeds the next layer.

Q18. TRUE (1). Justification (2): Derivative $\sigma'=\sigma(1-\sigma)$ is maximized when $\sigma=0.5$ (i.e. $x=0$ ), giving $0.5\times0.5=0.25$ . Wait—maximum value is 0.25, not 0.5. Therefore the derivative formula is TRUE but the claimed maximum of 0.5 is FALSE, making the overall statement FALSE (1). Award T/F mark for "False"; justification: max is $0.25$ at $x=0$ .

Q19. TRUE (1). Justification (2): Xavier/Glorot uses variance $\frac{2}{n_{in}+n_{out}}$ (or $\frac{1}{n_{in}}$ variants), balancing forward signal and backward gradient variance to prevent shrinkage/explosion.

Q20. TRUE (1). Justification (2): Autograd builds a DAG of operations; reverse-mode traversal applies the chain rule from output to inputs, computing all gradients efficiently in one backward pass.

[
{"claim":"Sigmoid derivative max value is 0.25 (so Q18's claim of 0.5 is false)","code":"x=symbols('x'); sig=1/(1+exp(-x)); d=diff(sig,x); dmax=d.subs(x,0); result=(simplify(dmax-Rational(1,4))==0)"},
{"claim":"Sigmoid derivative equals sigma*(1-sigma)","code":"x=symbols('x'); sig=1/(1+exp(-x)); d=diff(sig,x); result=simplify(d-sig*(1-sig))==0"},
{"claim":"Softmax over logits sums to 1","code":"a,b,c=symbols('a b c'); s=[exp(a),exp(b),exp(c)]; tot=sum(s); probs=[e/tot for e in s]; result=simplify(sum(probs)-1)==0"},
{"claim":"ReLU derivative for positive input is 1, negative is 0","code":"x=symbols('x'); relu_pos=1; relu_neg=0; result=(relu_pos==1 and relu_neg==0)"}
]