Question bank — Encoder-decoder architecture

False — a fixed-size vector is an information bottleneck; long inputs (50+ tokens) lose detail, especially from early positions. This is exactly what attention fixes.

False — the whole point is variable-length mapping; encoding to one vector $\mathbf{c}$ decouples input length $T_x$ from output length $T_y$.

False — you can pair, say, an LSTM encoder with a GRU decoder; they only need to agree on the shape/interface of $\mathbf{c}$.

False — at inference there are no ground-truth tokens, so the decoder must feed its own predictions back (free-running). Teacher forcing only exists during training.

False — $\log$ is monotonically increasing, so the argmax over sequences is unchanged; log is used only for numerical stability and to turn products into sums.

False — greedy is locally optimal per step; a lower first-step token can lead to much higher-probability continuations, which beam search can recover.

False — that's the common default, but $\mathbf{c}=q(\{h_1^{enc},\dots ,h_{T_x}^{enc}\})$ can be any function of the states (mean-pool, learned combination, etc.).

False in general — an autoencoder reconstructs its input; seq2seq maps an input to a different target sequence (translation, summary), and lengths/vocabularies can differ.

False — the encoder recursion $h_t^{enc}=f_{enc}(x_t,h_{t-1}^{enc})$ is order-dependent, so "I love AI" and "AI love I" produce different states and outputs.

The decoder never re-reads the input, but $\mathbf{c}$ flows forward through every decoder state via the recurrence, so its influence persists across all steps — it isn't "used up".

Impossible — $\exp (z_v)>0$ and dividing by a positive sum keeps every value in $(0,1)$; they also sum to exactly $1$.

Backwards — teacher forcing hides the model's mistakes by always feeding the correct previous token, causing exposure bias; the model never trains on its own error states.

They are identical — keeping only one hypothesis per step is greedy decoding; beam search only differs when the width exceeds 1.

The stated objective is a likelihood to maximise, but frameworks minimise the negative log-likelihood; you either negate it or ascend, not descend on the positive form.

No — 512 float32 numbers give only $\approx 16,384$ bits, while a length-50 sequence over a $10^4$ vocabulary needs ${\log }_2(V^{T_x})\approx 664,000$ bits; the compression is enormous.

Only in teacher-forced training; at inference the true token is unknown, so the model feeds its own $y_{t-1}$ (the sampled/argmax prediction).

Multiplying many small probabilities underflows to zero in floating point; $\log$ converts the product into a numerically stable sum and preserves the argmax.

Each token depends on what was already generated ($y_{<t}$); feeding $y_{t-1}$ back lets the hidden state carry that history so outputs stay coherent.

The single fixed-size $\mathbf{c}$ becomes a bottleneck; long-range and early-sequence details get overwritten in the encoder's recurrence — motivating attention.

Averaging discards order and long-range structure; the recurrence $h_t=f(x_t,h_{t-1})$ accumulates context sequentially so position and dependencies are encoded.

Raw scores can be negative, and softmax's $\exp$ keeps everything positive while sharpening the distribution toward the largest logit — a valid, peaked probability.

Feeding ground-truth prefixes stops early errors from compounding, so gradients are computed from correct contexts and the model converges faster (at the cost of exposure bias).

The encoder's states are chained across time steps, so the gradient of the loss must be unrolled backward through every step to update the shared recurrent weights.

There are no tokens to process, so $\mathbf{c}$ falls back to the initial state $h_0^{enc}$ (usually zeros) — the decoder then generates purely from its prior/START token.

Generation halts when the model emits the special $⟨\text{END}⟩$ token (or hits a max-length cap); without this stopping rule autoregression would never terminate.

A special $⟨\text{START}⟩$ token is fed as $y_0$ so the recurrence has an input to condition the first prediction on.

Both are kept up to the beam width; ties are broken arbitrarily (or by length/order), and pruning only occurs once more candidates than the width exist.

It's mapped to an $⟨\text{UNK}⟩$ (unknown) token; softmax can only assign probability to symbols in $V$, so out-of-vocabulary words are unrepresentable directly.

No — there is no previous generated token to feed back, so teacher-forced and free-running training coincide for that single step.