Level 1 — RecognitionSequence Models

Sequence Models

⏱ 20 minutes30 marksprintable — key stays hidden on paper

Subject: AI-ML Chapter: Sequence Models Time Limit: 20 minutes Total Marks: 30

Section A — Multiple Choice (1 mark each)

Choose the single best answer.

Q1. In a vanilla RNN, the hidden state at time $t$ is computed as:

(a) $h_t = \sigma(W_{hh} h_{t-1} + W_{xh} x_t + b)$
(b) $h_t = \sigma(W_{hh} x_t + b)$
(c) $h_t = \sigma(W_{xh} h_{t-1} + b)$
(d) $h_t = h_{t-1} + x_t$

Q2. Backpropagation Through Time (BPTT) works by:

(a) Unrolling the network over time steps and applying the chain rule
(b) Training each time step independently
(c) Propagating gradients only at the final step
(d) Ignoring the recurrent connections

Q3. The vanishing gradient problem in RNNs occurs mainly because:

(a) Repeated multiplication of small Jacobian/weight terms shrinks gradients over long sequences
(b) The learning rate is always too large
(c) The activation function has no derivative
(d) Weights are never shared across time

Q4. Which gate in an LSTM decides what information to discard from the cell state?

(a) Forget gate
(b) Input gate
(c) Output gate
(d) Update gate

Q5. A GRU differs from an LSTM in that it:

(a) Has fewer gates and no separate cell state
(b) Has more gates than an LSTM
(c) Cannot model long-term dependencies at all
(d) Uses no gating mechanism

Q6. A Bidirectional RNN processes the sequence:

(a) In both forward and backward directions and combines the states
(b) Only backward in time
(c) Only for classification, never for tagging
(d) By randomly shuffling time steps

Q7. In an encoder-decoder (seq2seq) model without attention, the decoder is initialized from:

(a) A fixed-length context vector summarizing the input sequence
(b) The raw input tokens directly
(c) A random vector at every step
(d) The target sequence

Q8. The core intuition of the attention mechanism is to:

(a) Let the decoder focus on relevant encoder states via learned weights
(b) Remove all recurrent connections
(c) Compress the whole input into a single scalar
(d) Replace embeddings with one-hot vectors

Q9. Bahdanau (additive) attention differs from Luong (multiplicative) attention primarily in:

(a) How the alignment score between decoder and encoder states is computed
(b) The number of layers in the encoder
(c) Whether embeddings are used
(d) The loss function used

Q10. Word2Vec's skip-gram model is trained to:

(a) Predict context words given a center word
(b) Predict the center word given the sentence sentiment
(c) Cluster documents by topic
(d) Sort words alphabetically

Q11. Teacher forcing during training means:

(a) Feeding the ground-truth token as the next decoder input instead of the model's own prediction
(b) Freezing the encoder weights
(c) Forcing all gradients to zero
(d) Using a larger batch size

Q12. Beam search with beam width $k=1$ is equivalent to:

(a) Greedy decoding
(b) Exhaustive search
(c) Random sampling
(d) Attention

Section B — Matching (1 mark each, 5 marks)

Q13. Match each concept (i–v) to its correct description (A–E).

Concept		Description
(i) Padding & masking		(A) Uses GloVe/Word2Vec vectors to represent tokens
(ii) Word embedding		(B) Handles variable-length sequences in a batch
(iii) Context vector		(C) Summary of encoder output passed to decoder
(iv) Forget gate		(D) Controls what is removed from LSTM cell state
(v) Beam width		(E) Number of candidate hypotheses kept during decoding

Section C — True/False with Justification (2 marks each: 1 for T/F, 1 for justification)

Q14. LSTMs completely eliminate the vanishing gradient problem for arbitrarily long sequences. (T/F + justify)

Q15. In GloVe, embeddings are learned from global word co-occurrence statistics. (T/F + justify)

Q16. Attention removes the information bottleneck of a single fixed context vector in seq2seq models. (T/F + justify)

Q17. A bidirectional RNN can be used for real-time (online) generation where future inputs are unknown. (T/F + justify)

Q18. Teacher forcing can cause a train–inference mismatch known as exposure bias. (T/F + justify)

Q19. Increasing the beam width in beam search always guarantees the globally optimal output sequence. (T/F + justify)

Answer keyMark scheme & solutions

Section A (1 mark each)

Q1 — (a). The RNN recurrence combines the previous hidden state (via $W_{hh}$ ) and current input (via $W_{xh}$ ) through a nonlinearity. (b),(c) drop a required term; (d) has no learned weights/nonlinearity.

Q2 — (a). BPTT unrolls the recurrent graph across time and applies the chain rule, summing gradients over shared weights.

Q3 — (a). Long products of Jacobians with spectral norm < 1 (and squashing derivatives ≤ 1) drive gradients toward zero exponentially with sequence length.

Q4 — (a). The forget gate $f_t=\sigma(\cdot)$ multiplies the previous cell state to decide what to keep/discard.

Q5 — (a). GRU merges cell and hidden state and uses only update + reset gates (2 gates vs LSTM's 3), fewer parameters.

Q6 — (a). BiRNN runs a forward and backward pass and concatenates/combines both hidden states, giving each position full-sequence context.

Q7 — (a). The encoder compresses the input into a fixed context vector used to initialize the decoder.

Q8 — (a). Attention computes weights over encoder states so the decoder attends to relevant parts at each step.

Q9 — (a). The distinction lies in the alignment/score function: Bahdanau uses an additive feed-forward net; Luong uses multiplicative (dot/general) scoring.

Q10 — (a). Skip-gram predicts surrounding context words from the center word.

Q11 — (a). Teacher forcing feeds the true previous target token to stabilize/speed training.

Q12 — (a). Beam width 1 keeps only the single best hypothesis each step = greedy decoding.

Section B

Q13 — (i)→B, (ii)→A, (iii)→C, (iv)→D, (v)→E. (1 mark per correct match, 5 total.)

Section C (1 T/F + 1 justification)

Q14 — False. LSTMs mitigate vanishing gradients via the additive cell-state (constant error carousel) but do not eliminate them entirely; very long dependencies can still degrade.

Q15 — True. GloVe factorizes a global word–word co-occurrence matrix, learning vectors whose dot products approximate log co-occurrence counts (unlike Word2Vec's local-window prediction).

Q16 — True. By letting the decoder access all encoder states through weighted sums, attention removes reliance on one fixed-size context vector, easing long-sequence encoding.

Q17 — False. The backward pass requires the entire input sequence, so a BiRNN cannot operate in a strictly online setting where future tokens are unavailable.

Q18 — True. During training the model sees gold tokens, but at inference it consumes its own (possibly wrong) predictions; this distributional gap is exposure bias.

Q19 — False. Larger beam width improves search but is a heuristic; it does not guarantee the globally optimal sequence (and can even worsen quality due to length/degeneration effects).

[
  {"claim":"Beam width 1 = greedy: number of hypotheses kept equals 1","code":"beam_width=1; result = (beam_width==1)"},
  {"claim":"GRU has 2 gates, LSTM has 3 gates (fewer for GRU)","code":"gru_gates=2; lstm_gates=3; result = (gru_gates < lstm_gates) and (gru_gates==2) and (lstm_gates==3)"},
  {"claim":"Q13 matching mapping is one-to-one and correct","code":"m={'i':'B','ii':'A','iii':'C','iv':'D','v':'E'}; result = (sorted(m.values())==['A','B','C','D','E'])"},
  {"claim":"Total marks = 12 MCQ + 5 matching + 6 TF*2 = 30","code":"mcq=12*1; match=5*1; tf=6*2; result = (mcq+match+tf==30)"}
]