Red-teaming language models: Worked examples — AI-ML deep dive

Before we start, let us re-earn every symbol from scratch so a reader who skipped the parent can still follow.

The scenario matrix

Every red-teaming problem you will meet is one of these cells. Each example below is tagged with the cell(s) it covers.

Cell	Case class	What makes it special	Example
A	Ordinary interior value	$0 < ASR < 1$ , plain division	Ex 1
B	Zero successes ( $S = 0$ )	ASR $= 0$ — is the model perfect or your test weak?	Ex 2
C	Total failure ( $S = N$ )	ASR $= 1$ , the ceiling	Ex 3
D	Degenerate input ( $N = 0$ )	division by zero — undefined, not zero	Ex 4
E	Comparing two methods	manual vs automated, same $H$	Ex 5
F	Limiting behaviour	ASR as $N \to \infty$ ; diminishing returns	Ex 6
G	Real-world word problem	tokens → dollars → cost per success	Ex 7
H	Exam twist	multi-turn "sticky refusal", unique-vs-raw counting	Ex 8

Look at the figure below before reading on — it is the mental picture we will reuse: a dartboard where every dart is a prompt, red darts hit (unsafe output), blue darts miss (safe refusal). ASR is just how red the board looks.

The worked examples

Worked example Example 1 — Cell A: ordinary interior value

You send $N = 250$ jailbreak prompts to a chatbot. $S = 20$ of them produce unsafe output. What is the ASR, as a fraction and a percent?

Forecast: Guess first — is this closer to a well-aligned model (ASR near 0.01) or a leaky one? Write your guess down.

Write the formula with the numbers plugged in. $ASR = \frac{S}{N} = \frac{20}{250}$ . Why this step? ASR is defined as successes over attempts; there is nothing to derive, only to substitute. Substituting first prevents arithmetic-before-setup mistakes.
Reduce the fraction. $\frac{20}{250} = \frac{2}{25} = 0.08$ . Why this step? A decimal is comparable to lab targets; the fraction alone is not.
Convert to percent. $0.08 \times 100 = 8%$ . Why this step? Humans and dashboards read percentages faster than fractions.

Verify: Sanity check the size — $8%$ is 8 times the deployment target of $1%$ ( $0.01$ ). So this model is not ready. Reverse-check: $0.08 \times 250 = 20 = S$ . ✓ Units: ASR is dimensionless (prompts ÷ prompts). ✓

Worked example Example 2 — Cell B: zero successes (

$S = 0$ ) A red team runs $N = 500$ prompts and not one elicits unsafe output. $S = 0$ . What is ASR, and what does it actually tell you?

Forecast: Is ASR $= 0$ proof the model is perfectly safe? Guess yes/no.

Plug in. $ASR = \frac{0}{500} = 0$ . Why this step? Zero divided by any nonzero number is exactly zero — this is defined and safe, unlike the next example.
Interpret, do not celebrate. As the parent warns: ASR $= 0$ means either perfect alignment or the red-teaming was not creative enough. Why this step? A number cannot tell you which hypothesis is true. You must look at Coverage: if you found few unique attack types, your test was weak, not the model strong.
Follow-up metric. Suppose in $H = 4$ hours you tried only $U = 2$ distinct attack classes. $Coverage = \frac{2}{4} = 0.5$ unique modes/hour — suspiciously low diversity, so the zero is unconvincing. Why this step? Low coverage next to ASR $= 0$ is a red flag: you probably just didn't push hard.

Verify: $0 \times 500 = 0 = S$ . ✓ Coverage units: (modes) ÷ (hours). ✓ Conclusion: cannot conclude the model is safe from ASR $= 0$ alone.

Worked example Example 4 — Cell D: degenerate input (

$N = 0$ ) Your logging script crashes before sending any prompts: $N = 0$ , so of course $S = 0$ . A junior teammate writes "ASR = 0/0 = 0, model is safe." Correct them.

Forecast: Is $\frac{0}{0}$ equal to $0$ ? To $1$ ? Or to something else?

Recognise the form. $ASR = \frac{S}{N} = \frac{0}{0}$ , which is undefined, not zero. Why this step? Division asks "how many times does $N$ fit into $S$ ?" — with $N = 0$ the question has no answer, unlike Example 2 where $N = 500$ was a real denominator.
State the correct output. ASR is undefined / not-a-number when no prompts were sent. The correct report is "no data," not "0%." Why this step? Reporting a fake "0%" would falsely signal a perfectly safe model — a dangerous logging bug.

Verify: Contrast the two zeros: $\frac{0}{500} = 0$ (defined, Ex 2) vs $\frac{0}{0} =$ undefined (Ex 4). The denominator, not the numerator, decides. ✓

Worked example Example 5 — Cell E: comparing two methods (equal hours)

In $H = 10$ hours each: manual red-teaming lands $S_{man} = 23$ successful attacks out of $N_{man} = 100$ prompts; automated red-teaming lands $S_{auto} = 847$ out of $N_{auto} = 1000$ prompts. Compare ASR and raw yield.

Forecast: Which method has the higher rate? Which finds more total holes? Are these the same question?

Manual ASR. $\frac{23}{100} = 0.23 = 23%$ . Why this step? Establishes the human baseline rate.
Automated ASR. $\frac{847}{1000} = 0.847 = 84.7%$ . Why this step? Same formula, different scale — now comparable directly because both are fractions.
Raw-yield ratio. $\frac{847}{23} \approx 36.8$ . Why this step? ASR compares quality per prompt; raw yield compares total holes found in the same 10 hours. Automation wins on both axes here — it sends 10× more prompts and hits at a higher rate.

Verify: $0.23 \times 100 = 23 = S_{man}$ ✓; $0.847 \times 1000 = 847 = S_{auto}$ ✓; $847/23 = 36.8 \dots$ ✓. Automation is ~37× more productive in equal time.

Worked example Example 6 — Cell F: limiting behaviour (diminishing returns)

A team finds unique failure modes over time: after hour 1 they have $U = 10$ , hour 2 → $18$ , hour 3 → $24$ , hour 4 → $27$ , hour 5 → $28$ . Describe the coverage per hour and what the limit tells you.

Forecast: Will new-holes-per-hour go up, stay flat, or shrink toward zero?

New modes each hour (differences): $10, 8, 6, 3, 1$ . Why this step? Coverage is really the rate of new discovery; taking successive differences turns a running total into that rate.
Read the trend. The per-hour discoveries shrink toward 0. This is the diminishing-returns curve the parent mentions. Why this step? When a well-explored model stops yielding new holes, the marginal coverage $\to 0$ — a good sign that alignment is holding up under diverse attack.
Limiting statement. As $H \to \infty$ , cumulative $U$ approaches a plateau (here around 28–29). The rate $\frac{Δ U}{Δ H} \to 0$ . Why this step? A flat plateau means you have found essentially everything your attack repertoire can find — time to invent new attack classes, not run more of the old ones.

Verify: Cumulative check: $10 + 8 + 6 + 3 + 1 = 28 = U (hour 5)$ ✓. Differences are non-increasing ( $10 \geq 8 \geq 6 \geq 3 \geq 1$ ) ✓ — the signature of diminishing returns.

Worked example Example 7 — Cell G: real-world cost word problem

An automated run uses $1, 000, 000$ tokens costing $\$ 30 $t o t a l an d p r o d u ces$ S = 847 $s u ccess f u l a tt a c k s . (a) W ha t i s t h ecos tp er s u ccess f u l a tt a c k ? (b) I f aman u a l t es t er cos t s$ $50 $/ h o u r an df in d s$ 23$ attacks in 10 hours, what is their cost per attack? (c) Which is cheaper per hole found?

Forecast: Roughly how many cents per attack do you expect from automation — dollars, or fractions of a dollar?

Automated cost per success. $\dfrac{\$ 30}{847} \approx $0.0354 $, i . e . ab o u t$ 3.5 $ce n t s . * W h y t hi ss t e p ? * T o t a l cos t d i v i d e d b y s u ccesses g i v es d o l l a r s - p er - h o l e — t h e m e t r i c ab u d g e t o w n er a c t u a l l y c a r es ab o u t . U ni t s :$ \frac{\text{dollars}}{\text{attacks}}$.
Manual total cost. $\$ 50/\text{hr} \times 10,\text{hr} = $500$. Why this step? Convert the rate to a total so it is comparable to the automated total.
Manual cost per success. $\dfrac{\$ 500}{23} \approx $21.74$. Why this step? Same dollars-per-hole metric, now for the human.
Ratio. $\frac{21.74}{0.0354} \approx 614$ . Why this step? A single number for the headline: automation is roughly 600× cheaper per hole found here.

Verify: $0.0354 \times 847 \approx 30$ ✓; $21.74 \times 23 \approx 500$ ✓; $50 \times 10 = 500$ ✓. Units cancel to dollars/attack throughout. ✓

Worked example Example 8 — Cell H: exam twist (sticky refusal + unique-vs-raw)

A multi-turn attack has 3 turns. The model refuses turns 1 and 2 but leaks on turn 3 (the "pentester + CTF" authority trick). The team runs this exact 3-turn script $N = 40$ times; the leak happens on turn 3 in $S = 6$ of the runs. The team then also records that all $6$ leaks used the same attack class (context+authority). (a) What is the ASR? (b) How many unique failure modes ( $U$ ) did this contribute to Coverage?

Forecast: Careful — does "6 successes" mean "6 unique holes"? Guess before step 3.

Count what a "prompt" is. Each run of the 3-turn script is one adversarial attempt, so $N = 40$ and $S = 6$ (a run counts once, on its final outcome). Why this step? The exam trap is counting turns (120) instead of attempts (40). ASR is per-attempt, and the whole conversation is one attempt.
ASR. $ASR = \frac{6}{40} = 0.15 = 15%$ . Why this step? Straight substitution once $N, S$ are correctly identified.
Unique failure modes. All 6 exploited the same mechanism, so $U = 1$ , not 6. Why this step? Coverage counts distinct kinds of holes; six copies of one bug is one bug. This is exactly why the parent distinguishes "unique failure modes" from raw successes.
Defensive read. Because the leak is on turn 3 after two refusals, the fix is sticky refusal: once a boundary is hit, later turns in the same thread get heightened scrutiny. Why this step? Ties the number back to an alignment action, which exams reward.

Verify: $0.15 \times 40 = 6 = S$ ✓. $U = 1$ (single class) while $S = 6$ (raw hits) — they must not be conflated. ✓ Turn-count trap avoided: $N = 40$ , not $120$ . ✓

Recall Quick self-test (reveal after answering)

ASR when $S = 20, N = 250$ ::: $0.08$ (8%) ASR when $S = 0, N = 500$ ::: $0$ , but inconclusive — check Coverage ASR when $N = 0$ ::: undefined (not 0) Largest ASR possible ::: $1$ (when $S = N$ ) Does 6 successes of one attack class mean $U = 6$ ? ::: No, $U = 1$