Guardrails and constrained generation: Exercises

Before we start, one reminder of the only formula we lean on repeatedly. When a language model picks its next word (a token), it first computes a logit for each candidate — a raw, unbounded score where bigger means "the model likes this more". These scores are turned into probabilities by softmax:

P (x_{t} = i) = \frac{e ^{logit_{i}}}{\sum _{j} e ^{logit_{j}}}

The bottom (the sum over all tokens $j$ ) just makes the numbers add to $1$ . To forbid a token we do NOT delete it — we add a huge negative number (a mask bias of $- \infty$ , in practice $- 1 0^{9}$ ) to its logit, so $e^{- \infty} = 0$ and it contributes nothing.

Picture it once. The figure below shows the whole idea as a set of bars. Softmax turns raw logits (tall/short poles) into a set of coloured bars whose heights add to $1$ — think of the total probability as one litre of water poured across the bars. When we mask a token, we knock its bar to the floor, and the water it held flows into the survivors — that flowing-water step is renormalization.

Figure — Guardrails and constrained generation

Refer back to this picture in every Level-2 solution: masking = "remove a bar, the water redistributes".

Level 1 — Recognition

L1.1

Classify each as a guardrail or constrained generation: (a) A regex run on the finished text to redact credit-card numbers. (b) A JSON-schema state machine that zeroes invalid tokens while decoding. (c) A toxicity classifier that rejects a reply scoring $0.9$ . (d) Compiling $\d{3}$ \d{3}-\d{4} into a state machine that masks logits each step.

Recall Solution

The dividing question is timing: does it act after text exists (guardrail) or during sampling (constrained generation)?

(a) Guardrail — runs on finished text.
(b) Constrained generation — masks tokens during decoding.
(c) Guardrail — reactive check after generation.
(d) Constrained generation — masks logits step by step.

L1.2

Fill the blanks: Guardrails are reactive (generate → check → maybe reject). Constrained generation is proactive (only valid tokens are ever produced).

Recall Solution

Reactive vs proactive is the core contrast. A guardrail can waste compute generating something it later throws away; constrained generation never produces an invalid token in the first place.

Level 2 — Application

L2.1

Three tokens have logits $logit = [2, 1, 0]$ for tokens A, B, C. Compute the softmax probability of each (round to 3 decimals).

Recall Solution

WHAT: exponentiate each logit, then divide by the total. $e^{2} = 7.389, e^{1} = 2.718, e^{0} = 1$ . Sum $= 11.107$ . $P (A) = \frac{7.389}{11.107} = 0.665, P (B) = \frac{2.718}{11.107} = 0.245, P (C) = \frac{1}{11.107} = 0.090.$ WHY it sums to 1: the denominator is the sum, so the fractions must total $1$ ( $0.665 + 0.245 + 0.090 = 1.000$ ). WHAT IT LOOKS LIKE: the three left-hand ("no mask") bars in the top figure — one litre of water split $0.665/0.245/0.090$ .

L2.2

Now forbid token C by adding a mask bias of $- 1 0^{9}$ to its logit only. Recompute $P (A), P (B), P (C)$ .

Recall Solution

WHAT: C's logit becomes $0 - 1 0^{9} \approx - 1 0^{9}$ , so $e^{- 1 0^{9}} \approx 0$ . C drops out of the sum entirely. New denominator $= e^{2} + e^{1} + 0 = 7.389 + 2.718 = 10.107$ . $P (A) = \frac{7.389}{10.107} = 0.731, P (B) = \frac{2.718}{10.107} = 0.269, P (C) = 0.000.$ WHY the others rose: the probability mass C used to hold got renormalized across the survivors. WHAT IT LOOKS LIKE: the right-hand ("C masked") bars in the top figure — C's bar is on the floor, and its $0.090$ of water flowed into A and B (which is why $0.665 \to 0.731$ and $0.245 \to 0.269$ ). This is exactly the mask-then-softmax step from 6.2.10 Guardrails and constrained generation (Hinglish).

L2.3

A JSON generator has just emitted the open brace {. The schema says the next token must be a quote " (start of a key) or the closing }. The raw logits are ": $3$ , }: $1$ , 5 (a digit): $4$ , x (a letter): $2$ . After applying the schema mask (only " and } legal), what is $P (")$ ?

Recall Solution

WHAT: legal tokens keep bias $0$ ; illegal (5, x) get $- 1 0^{9}$ , so $e^{- 1 0^{9}} \approx 0$ . Surviving terms: $e^{3} = 20.086$ ("), $e^{1} = 2.718$ (}). Sum $= 22.804$ . $P (") = \frac{20.086}{22.804} = 0.881.$ WHY it lands at $0.881$ and not the raw softmax value: originally four tokens shared the water, and the digit 5 (logit $4$ , the tallest pole) would have grabbed the largest slice. The schema knocks 5 and x to the floor, so their water vanishes from the denominator and renormalizes across only " and }. Removing the biggest competitor is precisely why " jumps up to $0.881$ — the mask didn't change "'s own logit, it deleted the rivals that were dividing the litre with it. WHAT IT LOOKS LIKE: the FSM figure below — from state "after {" only two arrows leave, so only two bars survive.

Level 3 — Analysis

L3.1

A team masks invalid tokens by multiplying their logit by $0$ . Token bad has logit $- 4$ . Show its probability among bad, and two valid tokens good1, good2 with logits $1$ and $1$ — first without any mask, then with the buggy multiplicative mask. Does bad become more or less likely?

Recall Solution

No mask: $e^{- 4} = 0.0183, e^{1} = 2.718$ twice. Sum $= 0.0183 + 2.718 + 2.718 = 5.455$ . $P (bad)_{no mask} = \frac{0.0183}{5.455} = 0.00336.$ Buggy multiply (bad's logit $\to 0$ ): $e^{0} = 1$ . Sum $= 1 + 2.718 + 2.718 = 6.436$ . $P (bad)_{buggy} = \frac{1}{6.436} = 0.1554.$ Analysis: the "mask" made the forbidden token ~46× more likely ( $0.1554/0.00336 \approx 46$ ). Multiplicative masking is not just wrong, it is actively dangerous for negative logits. WHAT IT LOOKS LIKE: the third figure — the coral "buggy" bar for bad towers over the mint "correct" bar.

L3.2

You must produce valid JSON and block toxic content. Which job goes to constrained generation, which to a guardrail, and why can't one tool do both?

Recall Solution

Format (valid JSON) → constrained generation. Validity is a formal grammar: a finite-state machine can decide legality token by token.
Toxicity → guardrail (classifier or LM-as-judge). Toxicity is a fuzzy semantic property; no finite-state machine over tokens decides "is this hateful?" — it depends on meaning, not syntax. Why not one tool: constrained generation only prunes tokens a grammar can rule out. Meaning-level judgements need a learned model. So you combine: constrain for shape, guard for content (see 6.2.8-Function-calling-and-tool-use for why tool arguments need the JSON constraint especially).

L3.3

With constrained JSON decoding, is rejection sampling (generate, check validity, retry) ever needed for format? What about for content?

Recall Solution

Format: No. A correct schema FSM guarantees validity on the first pass — every step masks illegal tokens, so the denominator only sums valid ones. Zero wasted generations.
Content: Possibly yes. If a guardrail rejects toxic output you may regenerate (rejection sampling). Content cannot be guaranteed at decode time, so retries remain a fallback.

Level 4 — Synthesis

L4.1

Design a pipeline that outputs a product review as {"rating": int 1-5, "comment": str} and is safe. List the ordered stages and say which formula/tool each uses.

Recall Solution

Constrained JSON decode — build an FSM from the schema; at each step $P = softmax (logits + maskbias)$ with $- 1 0^{9}$ on illegal tokens. After "rating": only digits 1–5 are legal. Guarantees valid, in-range JSON.
Parse — safe, because step 1 guaranteed validity (no try/except needed for format).
Content guardrail — run toxicity(comment); if score $> 0.8$ , replace with "[Redacted]".
Log & return. Constraint handles shape, guardrail handles meaning — the L3.2 split, now sequenced.

L4.2

A phone-number generator uses regex $\d{3}$ \d{3}-\d{4}. The FSM is at the state right after ( and three digits, expecting ). Raw logits: ): $2$ , -: $5$ , 4: $3$ . Only ) is legal here. Give $P (")")$ and explain why the model's favourite (-, logit $5$ ) is silenced.

Recall Solution

Only ) legal ⇒ its bias $0$ , both others $- 1 0^{9}$ . Numerator $= e^{2} = 7.389$ ; denominator $= e^{2} + 0 + 0 = 7.389$ . $P (")") = \frac{7.389}{7.389} = 1.000.$ The FSM permits exactly one token, so regardless of logits that token gets probability $1$ . The model preferred - (logit $5$ ), but the regex state forbids it — the constraint fully overrides preference at forced positions.

L4.3

Combine constrained generation with a guardrail where the guardrail redacts a field. Given {"rating": 4, "comment": "this is garbage and hateful"} with toxicity $0.92 > 0.8$ , what does the final object look like, and why doesn't redaction break the JSON validity?

Recall Solution

Final: {"rating": 4, "comment": "[Redacted]"}. Why still valid JSON: redaction replaces the string value with another string of the same JSON type. Type-preserving edits keep the schema satisfied, so the constraint's guarantee survives post-processing. (If you replaced comment with a number, you'd break it — always redact type-for-type.) This is the exact "[Redacted]" replacement used in the L4.1 pipeline.

Level 5 — Mastery

L5.1 (Degenerate case)

The FSM reaches a state where every vocabulary token is illegal (a "dead end" — the grammar allows no continuation). What happens to the softmax denominator, and what should a well-built library do?

Recall Solution

Every logit gets $- 1 0^{9}$ , so every $e^{logit + bias} \approx 0$ . The denominator $\to 0$ and $P = 0/0 =$ NaN — the softmax is undefined. A correct library must never reach this: the FSM should always keep at least the end-of-sequence token or a valid continuation reachable. If a dead end is truly unavoidable (bad schema), the library must raise an error, not silently emit NaN. This is why literal $- \infty$ is replaced by $- 1 0^{9}$ and dead-ends are checked separately.

L5.2 (Interaction with sampling temperature)

Recall from 4.3.4-Sampling-strategies that temperature $T$ scales logits: $softmax (logits / T)$ . If we mask with $- 1 0^{9}$ and use $T = 0.5$ , does the invalid token stay dead? Show it.

Recall Solution

Masked logit is $- 1 0^{9}$ . Dividing by $T = 0.5$ gives $- 2 \times 1 0^{9}$ , still enormously negative, so $e^{- 2 \times 1 0^{9}} \approx 0$ . The token stays dead. Subtlety: apply the mask before the temperature divide is fine here, but the safe rule is: mask should dominate whatever scaling follows. With $- 1 0^{9}$ and any sane $T > 0$ , invalidity survives. (With a literal $- \infty$ , $- \infty/ T = - \infty$ too — also safe.)

L5.3 (Adversarial)

An attacker crafts a prompt so the model wants to emit a forbidden token with logit $30$ (very high). Under mask bias $- 1 0^{9}$ , what is that token's probability? Does the attack succeed against constrained generation? What about against a guardrail?

Recall Solution

Masked logit $= 30 - 1 0^{9} = - 999, 999, 970 \approx - 1 0^{9}$ . $e^{- 1 0^{9}} \approx 0$ , so $P \approx 0$ — the attack fails against constrained generation, because the mask is additive and $- 1 0^{9}$ dwarfs any realistic logit. Against a guardrail the token does get generated, then a classifier must catch it — and classifiers can be fooled (see 7.3.2-Adversarial-attacks-on-LLMs). Lesson: for formal forbidden sets, constrained generation is a hard wall; for semantic ones, guardrails are only as strong as the classifier. This is why 6.2.11-Multi-agent-systems often stack both, and why 5.1.5-RLHF-and-preference-learning tunes the base model to want safe outputs in the first place.

L5.4 (Quantitative mastery)

Vocabulary of 4 tokens, logits $[5, 5, 5, 5]$ (a perfectly uniform model). Constraint allows only tokens 1 and 2. Compute each probability, and confirm the two forbidden tokens are exactly $0$ while the allowed two are equal.

Recall Solution

WHAT: allowed tokens (1,2) keep bias $0$ , so their logit stays $5$ ; forbidden tokens (3,4) get bias $- 1 0^{9}$ . Allowed: $e^{5} = 148.413$ each. Forbidden: $e^{5 - 1 0^{9}} = e^{- 1 0^{9}} \approx 0$ each. Denominator $= 148.413 + 148.413 + 0 + 0 = 296.826$ . $P_{1} = P_{2} = \frac{148.413}{296.826} = 0.5, P_{3} = P_{4} = \frac{0}{296.826} = 0.$ Confirmation: the two forbidden tokens are exactly $0$ ( $P_{3} = P_{4} = 0$ ); the two allowed tokens each get $0.5$ and together carry all the mass ( $0.5 + 0.5 = 1$ ). WHY they land at exactly $0.5$ : before masking, all four tokens are identical (logit $5$ ), so the litre splits evenly into quarters ( $0.25$ each). Masking removes tokens 3 and 4, and their half-litre renormalizes across the two survivors — which are still equal to each other — so each rises from $0.25$ to $0.5$ . By symmetry the survivors split evenly: the mask changed the support (which tokens exist), not the relative shape among survivors.

Recall Self-test recap

The one mask formula ::: $P = softmax (logits + maskbias)$ with $0$ for legal, $- 1 0^{9}$ for illegal tokens. Why add $- \infty$ not multiply by 0 ::: multiply leaves logit-0 tokens at $e^{0} = 1 > 0$ and boosts negative logits; adding $- \infty$ gives $e^{- \infty} = 0$ exactly. Constrained generation guarantees ::: format validity (syntax), never content safety. Guardrails guarantee ::: fuzzy/semantic checks, but only as strong as the classifier, and only after text exists. A single-legal-token FSM state gives that token probability ::: exactly $1$ , no matter the raw logits.