Question bank — Context window and sequence length

False. Even inside the window there is a lost-in-the-middle effect — tokens near the start and end are attended to far better than those in the middle. Fitting is not the same as attending well.

False. Sequence length $L$ is the token count of the current input (your per-request choice); the context window $n_{ctx}$ is the model's fixed trained ceiling. The rule is $L\le n_{ctx}$.

False. Attention compute is $\mathcal{O}(L^{2}d)$, so doubling $L$ quadruples the pairwise-comparison cost. Linear intuition fails here.

False. The stored attention matrix holds one scalar weight per (query, key) pair, so it is $\mathcal{O}(L^2)$. The extra $d$ only appears in the compute of each dot product, plus a separate $\mathcal{O}(Ld)$ term for storing Q/K/V.

False. It is measured in tokens (subword units). "internationalization" is one word but can be 5+ tokens, so a word count badly underestimates $L$. See Tokenization.

False. Prompt and generated output share the same window. If the prompt already fills $n_{ctx}$, there is zero budget left for output and the reply is cut off immediately.

False. Beyond guaranteeing more text fits, extra length can add noise and dilute attention (lost-in-the-middle). Quality depends on what you put in the window, not just how much fits.

False. Raw self-attention is a permutation-invariant weighted sum, so "dog bites man" and "man bites dog" would look identical without positional encoding added in. See Positional Encoding.

False. These tricks extend usable length beyond the training window, but quality is only guaranteed up to the length the model was actually trained on; far past it, behaviour degrades.

Wrong. Halving $L$ cuts attention cost by ~4× (quadratic), so twice the batch at half length is actually cheaper per token, not cost-neutral. The $L^2$ law breaks the linear bookkeeping.

Wrong. The KV Cache stores one key and value vector per token, so it grows linearly, $\mathcal{O}(Ld)$. The quadratic $L^2$ term is the attention-weight matrix, which is recomputed, not the cache.

Wrong. Shrinking $d$ scales cost linearly but the quadratic-in-$L$ explosion remains. The context ceiling is driven by the $L^2$ factor, which $d$ cannot cancel.

Wrong. English averages ~1.3 tokens/word, so 6000 words ≈ 7800 tokens. It barely fits and leaves almost nothing for the output budget. Always estimate in tokens, not words.

Wrong. Truncation and Chunking discards the tokens that fall outside the window — that content is gone unless you chunk or summarize. Truncation trades completeness for fitting.

Wrong. Padding still occupies positions and, unless masked, still enters the $L^2$ attention computation — wasting compute and potentially leaking attention onto meaningless slots.

There are $L^2$ query–key pairs; each pair's dot product costs $d$ multiply-adds (the compute), but the result stored is a single scalar weight (the memory). The $d$ lives in the calculation, not in the stored number.

Attention compares every token with every token, giving $L\times L=L^2$ pairs. Doubling $L$ makes $(2L{)}^2=4L^2$ pairs, so the workload is squared, not merely doubled.

Self-attention computes an order-independent weighted sum, so it sees a set of tokens, not an ordered list. Without an explicit position signal, all permutations produce the same representation.

Classic learned/sinusoidal position tables only have entries up to $n_{ctx}$. Position $n_{ctx}+1$ has no encoding at all, so the model is fed a signal it has never seen — a cliff, not a slope.

Each generated token is appended to the running sequence and must be attended to on the next step, so it occupies a real position inside the shared window: $\text{input}+\text{output}\le n_{ctx}$.

The cache stores past keys/values so each new token only attends against them instead of recomputing every earlier token's K and V. It trades $\mathcal{O}(Ld)$ memory to avoid redundant work per step, even though total attention across the sequence stays quadratic.

$L=1$: the token attends only to itself, a trivial $1\times 1$ attention matrix. Attention still "runs" but does nothing informative — there are no other tokens to compare against.

The window is completely full, so there is zero budget for generation. The model can produce nothing new (or must evict the oldest tokens), which is why you must reserve space for output in advance.

No hard failure — it simply fits with spare budget. You pay attention cost proportional to the actual $L^2$, not the full $n_{ctx}^2$, so short inputs are cheaper. Unused window is free.

It becomes literally invisible — the model has no encoding or attention path to it, so any information it carried is forgotten as if it were never present.

Yes. Tokenization is tokenizer-specific: the same string may split into more or fewer subword pieces, so $L$ (and whether it fits $n_{ctx}$) depends on which Tokenization scheme you use.

Not without failure — you must truncate, chunk, or summarize first, because there is no valid position encoding or allocated slot beyond $n_{ctx}$. The requirement $L\le n_{ctx}$ is a hard precondition.