Context window and sequence length: Worked examples

Before we start, two quantities everything below uses:

The scenario matrix

Every question this topic can ask lives in one of these cells. The Example column links straight to the worked example that fills that cell.

Cell	Case class	The question it tests	Example
A	$L < n_{c t x}$ (fits, room to spare)	Does it fit? How much slack?	Ex 1
B	$L = n_{c t x}$ (fits exactly, degenerate boundary)	What happens at the exact edge?	Ex 2
C	$L > n_{c t x}$ (overflow)	How many chunks? How much to cut?	Ex 3
D	$L = 0$ (empty / zero input)	Limiting case: no tokens at all	Ex 4
E	Prompt + output share the window	Budget split; generation overflow	Ex 5
F	Cost scaling (the $L^{2}$ law)	Non-doubling ratios, fractional scale	Ex 6
G	Real-world word problem	Convert words/chars/code → tokens	Ex 7
H	Exam twist (combine cells)	Multi-turn chat eating the window	Ex 8

We'll use English ≈ 1.3 tokens/word (the parent's rule) unless told otherwise.

Figure — Context window and sequence length

Read the figure as our example map. The white box is the same fixed window ( $n_{c t x}$ ) in every row — the dashed white line on the right is the ceiling. The coloured fill is how many tokens $L$ you actually feed:

Blue row (Cell A / Ex 1): the memo fills only part of the box — white space remains, matching the ~38% used in Ex 1.
Yellow row (Cell B / Ex 2): the fill reaches the dashed line exactly — $L = n_{c t x}$ , zero slack, exactly Ex 2's boundary.
Red row (Cell C / Ex 3): the fill spills past the dashed line (hatched red) — the overflow that forces the 4 chunks of Ex 3.
Green row (Cell D / Ex 4): the box is empty — $L = 0$ , the limit case of Ex 4.

Every other example (5–8) is a combination of these four fill states, so keep this picture in mind: you are always asking "where does the fill land relative to the dashed ceiling?"

Cell A — fits with room to spare

Worked example Ex 1 (Cell A): A short memo

A model has $n_{c t x} = 4096$ . Your memo is 1200 words of plain English. Does it fit, and how much room is left?

Forecast: guess now — does it fit? Roughly what fraction of the window does it use?

Step 1 — convert words to tokens. $L \approx 1200 \times 1.3 = 1560 tokens$ Why this step? The window counts tokens, not words. A subword tokenizer splits rare/long words into pieces, so tokens outnumber words — that's where the $1.3$ comes from.

Step 2 — compare against the ceiling. $1560 \leq 4096 \Rightarrow it fits.$ Why this step? The single law is $L \leq n_{c t x}$ . Checking it is the whole game.

Step 3 — measure the slack. $4096 - 1560 = 2536 tokens free$ $\frac{1560}{4096} \approx 0.381 = 38.1% used$ Why this step? The free tokens are what's left for the model's reply (see Cell E). Knowing you use only ~38% tells you there's plenty of room to generate. This is the blue row of the figure — fill well short of the ceiling.

Verify: $1560 + 2536 = 4096$ ✓. And $1560/4096 = 0.3808 \dots$ , under 1, consistent with "fits." Units: tokens throughout. ✓

Cell B — the exact boundary (degenerate)

Worked example Ex 2 (Cell B): Filling it to the brim

$n_{c t x} = 2048$ . After tokenizing, your input is exactly 2048 tokens. Is this legal? How many tokens can you generate?

Forecast: legal or not? And can you still get any output?

Step 1 — check the law at the edge. $L = 2048, n_{c t x} = 2048 \Rightarrow L \leq n_{c t x} holds (equality allowed).$ Why this step? The law uses $\leq$ , not $<$ . Equality is the boundary case — legal, but with zero margin. This is the yellow row of the figure: fill flush against the dashed ceiling.

Step 2 — compute output budget. $output room = n_{c t x} - L = 2048 - 2048 = 0 tokens.$ Why this step? The window holds input + output together (parent's Example 3). If input alone fills it, there is nothing left.

Step 3 — interpret the degenerate result. The input is valid, but the model can emit 0 new tokens — effectively it cannot reply. In practice the API errors ("max tokens must be ≥ 1") or immediately hits the end. Why this step? A boundary that's technically legal can still be useless. Always check what the edge case means, not just whether it's allowed.

Verify: $2048 - 2048 = 0$ ✓. This is the exact transition point: one token more input → Cell C (overflow). ✓

Cell C — overflow, needs chunking

Worked example Ex 3 (Cell C): A long report

$n_{c t x} = 8192$ . A report is 20{,}000 words. How many tokens? How many chunks do you need (using chunking)?

Forecast: guess the token count and how many pieces this splits into.

Step 1 — words to tokens. $L \approx 20000 \times 1.3 = 26000 tokens$ Why this step? Same reason as Ex 1 — the ceiling is in tokens.

Step 2 — confirm overflow. $26000 > 8192 \Rightarrow does NOT fit; must split.$ Why this step? Break the law and the model silently drops the start of the document ("forgets" the beginning). This is the red row of the figure: fill spills past the dashed ceiling.

Step 3 — count chunks. $chunks = ⌈ \frac{26000}{8192} ⌉ = ⌈ 3.174 \dots ⌉ = 4$ Why this step? Each chunk must satisfy $L \leq n_{c t x}$ , so we divide and round up — the $⌈ \cdot ⌉$ ("ceiling") because a leftover 0.17 of a window still needs a whole 4th pass. Why ceiling and not floor? Rounding down would throw away the tail tokens. You never drop text you were asked to process.

Verify: $3 \times 8192 = 24576 < 26000$ , so 3 chunks is not enough; $4 \times 8192 = 32768 \geq 26000$ ✓. Four chunks it is.

Cell D — the zero / empty limit

Worked example Ex 4 (Cell D): Nothing in the window

$n_{c t x} = 4096$ . You send an empty string ( $L = 0$ tokens). What is the attention compute? How much output room?

Forecast: with zero input tokens, is the attention cost zero? Is the whole window available for output?

Step 1 — attention cost of $L = 0$ . The parent's law is $Compute = O (L^{2} d)$ , where $d$ is the model width defined above. Plug $L = 0$ : $0^{2} \cdot d = 0.$ Why this step? With no tokens there are no query–key pairs to compare — the $L \times L$ attention matrix is $0 \times 0$ , empty. The limiting cost is genuinely zero. This is the green (empty) row of the figure.

Step 2 — output room. $n_{c t x} - L = 4096 - 0 = 4096 tokens available.$ Why this step? Empty input means the entire window is free for generation. (In practice a model still needs some prompt to know what to say — but purely by the token budget, all 4096 are open.)

Step 3 — sanity of the degenerate case. Zero is the smallest legal $L$ : $0 \leq n_{c t x}$ always holds. This is the opposite extreme from Cell B.

Verify: $0^{2} \cdot d = 0$ ✓ for any $d$ ; $4096 - 0 = 4096$ ✓. The two limits Cell B ( $L = n_{c t x}$ ) and Cell D ( $L = 0$ ) bracket every legal input.

Worked example Ex 5 (Cell E): The reply that gets cut off

$n_{c t x} = 8192$ . Your prompt is 5000 tokens. You request a 4000-token reply. What happens?

Forecast: does the full 4000-token reply come back, or does something break?

Step 1 — total tokens requested. $5000 (prompt) + 4000 (output) = 9000 tokens.$ Why this step? Prompt and generated tokens live in the same window — you must add them. (Picture the yellow fill from Ex 2, but the fill starts at 5000 and tries to grow past the ceiling.)

Step 2 — compare to ceiling. $9000 > 8192 \Rightarrow overflow by 9000 - 8192 = 808 tokens.$ Why this step? The law $L_{total} \leq n_{c t x}$ applies to the sum.

Step 3 — find the real output you get. $actual output = n_{c t x} - prompt = 8192 - 5000 = 3192 tokens.$ The reply is truncated at 3192 tokens — cut off mid-sentence, losing the last 808 you wanted. Why this step? Generation stops the instant the window is full, regardless of whether the thought is finished.

Verify: $5000 + 3192 = 8192 = n_{c t x}$ ✓ (exactly full). Requested $4000 - 3192 = 808$ tokens lost, matching the overflow in Step 2 ✓.

Cell F — the $L^{2}$ cost law, non-obvious ratios

Worked example Ex 6 (Cell F): Growing the window 1.5×

Model A: $n_{c t x} = 4000$ . Model B: $n_{c t x} = 6000$ (a $1.5 \times$ longer window). Both have the same model width $d$ (recall: $d$ = the length of each token's vector). How much more attention compute? How much more memory?

Forecast: $1.5 \times$ the window — is compute $1.5 \times$ ? More? Guess a number.

Step 1 — recall the two laws. $Compute \propto L^{2} d, Memory (attn weights) \propto L^{2} .$ Why this step? Both scale with $L^{2}$ because every token attends to every token — $L$ queries $\times$ $L$ keys. The extra $d$ in compute is the cost of one $d$ -dimensional dot product per pair (that's why $d$ appears in compute but not in the scalar-weight memory).

Step 2 — compute ratio (with $d$ fixed and equal in both models). $\frac{600 0 ^{2} d}{400 0 ^{2} d} = (\frac{6000}{4000})^{2} = 1. 5^{2} = 2.25 \times$ Why this step? The $d$ cancels since both models have the same width. Only the $L^{2}$ factor survives — so we square the length ratio.

Step 3 — memory ratio. $\frac{600 0 ^{2}}{400 0 ^{2}} = 1. 5^{2} = 2.25 \times$ Why this step? Attention-weight memory is $O (L^{2})$ too, so the ratio is the same $2.25 \times$ . (The separate Q/K/V storage grows only $1.5 \times$ since it's $O (L d)$ — but the $L^{2}$ term dominates.)

Verify: $1. 5^{2} = 2.25$ ✓. Note: a $1.5 \times$ window is more than $1.5 \times$ the cost — squaring always inflates ratios above 1. This is why the parent's mnemonic says "Double the view → Quadruple the queue."

Cell G — real-world: mixed content

Worked example Ex 7 (Cell G): Code + prose, different token rates

$n_{c t x} = 16384$ . You paste 800 words of English docs (1.3 tok/word) plus 300 lines of code (code averages ~8 tokens/line). Does it fit?

Forecast: two different token rates — will the total squeak under 16384?

Step 1 — tokenize the prose. $800 \times 1.3 = 1040 tokens$ Why this step? English uses the standard 1.3 rate.

Step 2 — tokenize the code. $300 \times 8 = 2400 tokens$ Why this step? Code has many symbols ((){};=), indentation, and rare identifiers — each often its own token. Its per-unit rate is higher, so we must estimate it separately, not lump it with prose.

Step 3 — total and compare. $L \approx 1040 + 2400 = 3440 tokens, 3440 \leq 16384 \Rightarrow fits.$ Why this step? Different content types get different rates, but they all land in the same token count and the same window. Add them, then apply the one law. (Blue-row situation from the figure — fill well short of the ceiling.)

Verify: $1040 + 2400 = 3440$ ✓; $3440/16384 \approx 0.21 = 21%$ used — comfortably fits, with ~12944 tokens free. ✓

Cell H — exam twist: multi-turn chat

Worked example Ex 8 (Cell H): The conversation that fills up

$n_{c t x} = 4096$ . A chat resends the whole history each turn. The system prompt is 500 tokens. Each user turn averages 200 tokens and each model reply 300 tokens (so 500 tokens per round-trip added to history). After how many full rounds does the history overflow the window?

Forecast: guess the number of rounds before it breaks.

Step 1 — model the growth. After $r$ complete rounds the history holds: $L (r) = 500 + 500 r tokens$ Why this step? The system prompt (500) is fixed; each round permanently adds its 500 tokens to the resent history. This is a straight line in $r$ — linear growth, because history accumulates. (Think of the red row from the figure filling up a little more each round until it spills.)

Step 2 — set $L (r) \leq n_{c t x}$ and solve. $500 + 500 r \leq 4096 \Rightarrow 500 r \leq 3596 \Rightarrow r \leq 7.192$ Why this step? The law $L \leq n_{c t x}$ still rules. Solving the inequality finds the last safe round.

Step 3 — take the floor. $r_{m a x} = ⌊ 7.192 ⌋ = 7 full rounds.$ Why floor here (but ceiling in Ex 3)? Different questions! Ex 3 asked "how many chunks to cover the text" → round up so nothing is left out. Here we ask "how many rounds fit" → round down, because a partial 8th round overflows. Match the rounding to the question.

Verify: after 7 rounds $L = 500 + 500 \times 7 = 4000 \leq 4096$ ✓ (fits, 96 to spare). After 8 rounds $L = 500 + 500 \times 8 = 4500 > 4096$ ✗ (overflows). So 7 is the exact last safe round. ✓ This is where history truncation kicks in — old turns get dropped to make room.

Recap: which rounding, which law?

Recall One line per cell — forecast before revealing

Cell A/B/C decision rule ::: check $L \leq n_{c t x}$ ; slack $= n_{c t x} - L$ .
Cell C chunk count ::: $⌈ L / n_{c t x} ⌉$ (round UP to cover everything).
Cell D limit ::: $L = 0 \Rightarrow$ compute $= 0$ , full window free.
Cell E output room ::: $n_{c t x} - prompt$ ; reply truncates there.
Cell F cost ratio ::: square the length ratio: $(L_{2} / L_{1})^{2}$ .
Cell G mixed content ::: tokenize each type at its own rate, then add.
Cell H rounds that fit ::: $⌊(n_{c t x} - fixed) / per-round ⌋$ (round DOWN to fit).

Flashcards

To chunk a text of

L

tokens into a window of

n_{c t x}

, how many chunks?

⌈ L / n_{c t x} ⌉

— ceiling, so no text is left out.

A model has prompt = 5000 tokens,

n_{c t x} = 8192

; you ask for a 4000-token reply. How many tokens actually come back?

8192 - 5000 = 3192

; the reply is truncated (808 tokens lost).

If the context window grows

1.5 \times

, how much does attention compute grow?

1. 5^{2} = 2.25 \times

— square the length ratio.

What is the attention compute when

L = 0

Zero — the

L \times L

attention matrix is empty (

0^{2} d = 0

What does

d

mean in the cost law

O (L^{2} d)

The model width / embedding dimension — the length of each token's vector; it's the cost of one query–key dot product.

Chat resends history: system 500 tok + 500 tok/round,

n_{c t x} = 4096

. How many full rounds fit?

⌊(4096 - 500) /500 ⌋ = ⌊ 7.19 ⌋ = 7

rounds.

Ceiling vs floor — when each?

Ceiling to cover a document in chunks; floor to count rounds/replies that fit inside the window.