Adversarial examples and robustness: Worked examples

This page is the "muscle memory" companion to Adversarial examples and robustness. There we derived the Fast Gradient Sign Method and PGD. Here we run them by hand on tiny numbers so you can watch every gear turn — including the weird corners: zero gradients, gradients that vanish, the exact moment clipping bites, and a full targeted attack.

Before we do anything, the symbols we will lean on constantly:

Definition The workhorses

$\nabla_{x} L$ (read "grad-x of L") is the gradient of the loss with respect to the input: a vector with one entry per input coordinate. Entry $i$ is $\partial L / \partial x_{i}$ — "how fast the loss rises as I raise pixel $i$ ." A positive entry means raising that pixel raises loss (good for an attacker who wants a wrong answer); a negative entry means raising that pixel lowers loss.
$L (θ, x, y)$ is the loss written out in full: $θ$ is the model's weights (the trained numbers inside the network — fixed during an attack; we never change them here), $x$ is the input we do change, and $y$ is the label. When we attack we differentiate only with respect to $x$ , holding $θ$ frozen — that is exactly what the little " $x$ " under $\nabla_{x}$ means.
$sign (v)$ takes a number and returns $+ 1$ if it is positive, $- 1$ if negative, and $0$ if it is exactly zero. Applied to a vector, it acts entry-by-entry. This is the whole trick of FGSM: throw away the magnitude of each gradient entry, keep only its direction, then step by the fixed budget $ϵ$ in that direction.

Recall Where is

$L_{\infty}$ -optimality proven? The claim "under $∥ δ ∥_{\infty} \leq ϵ$ the best move is $ϵ sign (\nabla_{x} L)$ " is derived in the parent note (FGSM derivation, Steps 1–3). Quick reminder of why: we want to maximize the dot product $δ^{⊤} \nabla_{x} L = \sum_{i} δ_{i} g_{i}$ . Each term $δ_{i} g_{i}$ is biggest when $δ_{i}$ has the same sign as $g_{i}$ and is as large as allowed, namely $\pm ϵ$ . Hence $δ_{i} = ϵ sign (g_{i})$ maximizes every term at once.

The scenario matrix

Every worked example below is tagged with the cell(s) it covers. If a cell has no example, you have found a gap — there are none.

Cell	What makes it tricky	Covered by
A. All-positive gradient	plain FGSM, every sign $= + 1$	Ex 1
B. Mixed signs	some pixels pushed up, some down	Ex 2
C. Zero gradient entry	$sign (0) = 0$ : that pixel does not move	Ex 3
D. Zero-vector gradient (degenerate)	attack does nothing — a local flat/optimum	Ex 3
E. Targeted attack (minus sign)	drive toward a chosen class $t$	Ex 4
F. PGD with no clipping	steps stay inside the ball, projection is idle	Ex 5
G. PGD where projection bites	a step leaves the $ϵ$ -ball, clip pulls it back	Ex 6
H. Pixel-range clipping (valid image)	$0 \leq x_{i} \leq 1$ also clips	Ex 6
I. High-dimensional accumulation	why $ϵ d$ blows up	Ex 7
J. Real-world word problem	$ϵ = 8/255$ in RGB units	Ex 8
K. Exam twist	$L_{2}$ optimal step $\neq =$ sign step	Ex 9

The FGSM recipe on a 2-pixel image

Worked example Ex 1 — Cell A (all-positive gradient)

A model has a 2-pixel input $x = (0.40, 0.60)$ . Its loss gradient at $x$ is $\nabla_{x} L = (0.9, 0.2)$ . Budget $ϵ = 0.1$ , $L_{\infty}$ . Compute the FGSM adversarial input.

Forecast: both gradient entries are positive, so both pixels get pushed up by the full $ϵ$ . Guess the answer before reading on.

Take the sign of the gradient. $sign (0.9, 0.2) = (+ 1, + 1)$ . Why this step? FGSM keeps only direction, not size. A gradient of $0.9$ and one of $0.2$ both say "raise this pixel," so both become $+ 1$ . The magnitudes $0.9$ vs $0.2$ are deliberately thrown away — that is what makes the step fast and keeps every pixel exactly at the budget edge.
Scale by $ϵ$ to get the perturbation. $δ = ϵ \cdot (+ 1, + 1) = (0.1, 0.1)$ . Why this step? We want the biggest loss increase allowed. Under $L_{\infty}$ (max any pixel may move is $ϵ$ ), stepping every pixel the full $ϵ$ in its gradient's direction is optimal — see the reminder box above.
Add to the input. $x_{adv} = x + δ = (0.40 + 0.1, 0.60 + 0.1) = (0.50, 0.70)$ . Why this step? An adversarial example is by definition the clean input plus the perturbation ( $x^{'} = x + δ$ ); adding $δ$ is how the chosen move actually lands on the new input.

Verify: $∥ δ ∥_{\infty} = max (0.1, 0.1) = 0.1 = ϵ$ ✓ (budget hit exactly, not exceeded). Linearized loss increase $= δ^{⊤} \nabla_{x} L = 0.1 (0.9) + 0.1 (0.2) = 0.11 > 0$ ✓ (loss went up, attack succeeds in the linear model).

Figure s01 (sketch it yourself): on axes "pixel 1" and "pixel 2", a teal square box of half-width $0.1$ is centred on the clean point $x = (0.40, 0.60)$ . A plum arrow from $x$ points up-and-right along the gradient $(0.9, 0.2)$ . FGSM lands on the orange top-right corner $(0.50, 0.70)$ — the corner of the box most aligned with that arrow.

Figure — Adversarial examples and robustness

Worked example Ex 2 — Cell B (mixed signs)

Same input $x = (0.40, 0.60)$ , $ϵ = 0.1$ , but now the gradient is $\nabla_{x} L = (0.9, - 0.3)$ .

Forecast: pixel 1's gradient is positive (push up), pixel 2's is negative (push down). The corner chosen will be a different corner than Ex 1.

Sign. $sign (0.9, - 0.3) = (+ 1, - 1)$ . Why this step? A negative gradient entry means raising that pixel lowers the loss — the attacker wants loss higher, so it moves that pixel the other way: down.
Perturbation. $δ = 0.1 \cdot (+ 1, - 1) = (0.1, - 0.1)$ . Why this step? Same rule as Ex 1: scale the sign vector by the budget $ϵ$ so each pixel moves the maximum allowed amount in its chosen direction — up on pixel 1, down on pixel 2.
Add. $x_{adv} = x + δ = (0.40 + 0.1, 0.60 - 0.1) = (0.50, 0.50)$ . Why this step? The adversarial input is clean-input-plus-perturbation; here the negative second component pulls pixel 2 down to $0.50$ .

Verify: $∥ δ ∥_{\infty} = 0.1 = ϵ$ ✓. Linear loss change $= 0.1 (0.9) + (- 0.1) (- 0.3) = 0.09 + 0.03 = 0.12 > 0$ ✓ — notice both terms are positive: choosing the sign correctly makes even the down-move help the attacker.

Figure s02 (sketch it yourself): the same teal box on $x = (0.40, 0.60)$ , but now the plum gradient arrow points down-and-right (sign $(+, -)$ ), so FGSM snaps to the orange bottom-right corner $(0.50, 0.50)$ — a different corner than Ex 1.

The degenerate cases: when the sign is zero

Worked example Ex 3 — Cells C & D (a zero entry, and the all-zero gradient)

Part (i), Cell C. $x = (0.5, 0.5, 0.5)$ , $ϵ = 0.1$ , gradient $\nabla_{x} L = (0.7, 0, - 0.4)$ . Part (ii), Cell D. Same $x$ , but gradient $= (0, 0, 0)$ .

Forecast: in (i), pixel 2 has gradient exactly $0$ — $sign (0) = 0$ , so it does not move. In (ii), nothing moves; the attack is powerless.

Part (i):

Sign. $sign (0.7, 0, - 0.4) = (+ 1, 0, - 1)$ . Why this step? $sign (0) = 0$ by definition. Geometrically, a zero gradient entry means the loss is momentarily flat along that pixel — nudging it up or down changes nothing to first order, so the optimal $L_{\infty}$ move leaves it put.
Perturbation. $δ = 0.1 \cdot (+ 1, 0, - 1) = (0.1, 0, - 0.1)$ . Why this step? Scaling by $ϵ$ moves the two live pixels to the budget edge; the middle entry is $0.1 \cdot 0 = 0$ , so that pixel stays frozen — the zero sign carries straight through.
Add. $x_{adv} = x + δ = (0.5 + 0.1, 0.5 + 0, 0.5 - 0.1) = (0.6, 0.5, 0.4)$ . Why this step? Clean-input-plus-perturbation again; the untouched middle pixel confirms a zero-gradient coordinate is simply left alone.

Verify (i): $∥ δ ∥_{\infty} = 0.1$ ✓. Linear change $= 0.1 (0.7) + 0 + (- 0.1) (- 0.4) = 0.07 + 0.04 = 0.11 > 0$ ✓. Middle pixel unchanged ✓.

Part (ii):

$sign (0, 0, 0) = (0, 0, 0)$ , so $δ = 0$ and $x_{adv} = x$ . Why? An all-zero input gradient is a critical point of the loss with respect to $x$ — a flat spot. FGSM has no downhill (or uphill) to grab. In practice this is rare and often means you are at a genuine minimum of the input loss, or the gradient underflowed; either way, single-step FGSM fails here and you would need a random restart (a trick PGD uses).

Verify (ii): $∥ δ ∥_{\infty} = 0 = (no change)$ , loss change $= 0$ ✓ — the honest, disappointing answer.

Targeted attacks: minus instead of plus

Worked example Ex 4 — Cell E (targeted, subtract the sign)

Untargeted FGSM asks "make any wrong answer." Targeted FGSM asks "make it say class $t$ ." We minimize the loss toward $t$ , so we step in the negative sign direction: $x_{adv} = x - ϵ sign (\nabla_{x} L (θ, x, t))$ . (Recall from the workhorses box: $θ$ is the frozen model weights, and $\nabla_{x} L (θ, x, t)$ is just "the input-gradient of the loss, but scored against the target label $t$ instead of the true label.")

$x = (0.30, 0.80)$ , $ϵ = 0.05$ . The gradient of the loss for the target class $t$ is $\nabla_{x} L (θ, x, t) = (0.6, - 0.5)$ .

Forecast: because we subtract, positive gradient entries get pushed down and negative ones up — the mirror image of untargeted.

Sign of the target-loss gradient. $sign (0.6, - 0.5) = (+ 1, - 1)$ . Why this step? Same "keep direction, drop magnitude" rule; it tells us which way each pixel would raise the target loss.
Subtract $ϵ \cdot$ sign. $δ = - 0.05 \cdot (+ 1, - 1) = (- 0.05, + 0.05)$ . Why this step? To make the model predict $t$ , we want its loss for $t$ to be small — so we descend, i.e. move opposite the gradient. The minus sign encodes "descend on target loss" instead of "ascend on true loss."
Add. $x_{adv} = x + δ = (0.30 - 0.05, 0.80 + 0.05) = (0.25, 0.85)$ . Why this step? Clean-input-plus-perturbation; the flipped signs (down on pixel 1, up on pixel 2) are exactly the mirror of what an untargeted attack on the same gradient would do.

Verify: $∥ δ ∥_{\infty} = 0.05 = ϵ$ ✓. Change in target loss (linearized) $= δ^{⊤} \nabla_{x} L (θ, x, t) = (- 0.05) (0.6) + (0.05) (- 0.5) = - 0.03 - 0.025 = - 0.055 < 0$ ✓ — target loss dropped, so class $t$ becomes more likely. Exactly what we wanted.

PGD by hand: watching the projection

FGSM takes one big leap of size $ϵ$ . Projected Gradient Descent (PGD) instead takes several small steps, re-reading the gradient after each one, and "projects" (clips) back into the budget whenever a step wanders out. This finds stronger attacks because the loss surface bends, and one linear leap can overshoot or miss.

Worked example Ex 5 — Cell F (PGD, projection never triggers)

One pixel for clarity. $x = 0.50$ , $ϵ = 0.10$ , $α = 0.03$ , run $T = 3$ steps ( $t = 0, 1, 2$ ), gradient sign is $+ 1$ every step. The legal window is $[x - ϵ, x + ϵ] = [0.40, 0.60]$ .

Forecast: each step adds $α = 0.03$ . After 3 steps that is $+ 0.09$ , still under the $ϵ = 0.10$ cap — so clipping never does anything and we end at $0.59$ .

Step $t = 0 \to 1$ . $x^{(1)} = clip (0.50 + 0.03, 0.40, 0.60) = clip (0.53, 0.40, 0.60) = 0.53$ . Why this step? $0.53$ is between the allowed bounds $[0.40, 0.60]$ , so $clip$ returns it unchanged (recall: $clip$ only acts when a value pokes outside the interval).
Step $t = 1 \to 2$ . $x^{(2)} = clip (0.53 + 0.03, 0.40, 0.60) = clip (0.56, 0.40, 0.60) = 0.56$ . Why this step? We re-take a small step of size $α = 0.03$ from the current point — PGD always steps from where it currently is, not from the original $x$ . $0.56$ is still inside the window, so no clipping.
Step $t = 2 \to 3$ . $x^{(3)} = clip (0.56 + 0.03, 0.40, 0.60) = clip (0.59, 0.40, 0.60) = 0.59$ . Why this step? Third small step for the same reason; $0.59 < 0.60$ , so still legal. Three small steps beat one big FGSM leap because each recomputes the gradient (here fixed for simplicity).

Verify: $∣ x^{(3)} - x ∣ = ∣0.59 - 0.50∣ = 0.09 \leq ϵ = 0.10$ ✓. Projection idle in all 3 steps ✓ (each pre-clip value was already legal).

Worked example Ex 6 — Cells G & H (projection

and pixel-range clip both bite) Same setup but push harder: $x = 0.95$ , $ϵ = 0.10$ , $α = 0.06$ , gradient sign $+ 1$ , and the pixel must stay a valid intensity $0 \leq x \leq 1$ . Run $T = 3$ steps.

Two constraints apply each step: the $ϵ$ -ball $[x - ϵ, x + ϵ] = [0.85, 1.05]$ and the valid-pixel range $[0, 1]$ . Order does not matter — applying two clips in a row is the same as clipping once to the intersection of the intervals, because clipping to $[0, 1]$ then to $[0.85, 1.05]$ (or vice-versa) both land you in the overlap $[max (0, 0.85), min (1.05, 1.00)] = [0.85, 1.00]$ . So we simply clip to that combined window $[0.85, 1.00]$ each step.

Forecast: naive steps would climb $0.95 \to 1.01 \to 1.07 \dots$ , but $1.00$ is the ceiling — clipping pins us at $1.00$ from step 1 onward.

Step $t = 0 \to 1$ . raw $= 0.95 + 0.06 = 1.01$ . Clip to $[0.85, 1.00]$ : $min (1.01, 1.00) = 1.00$ . So $x^{(1)} = 1.00$ . Why this step? $1.01$ exceeds the pixel ceiling $1.00$ , so the image-validity clip (Cell H) fires. Note it is $1.00$ , not $1.05$ : the tighter of the two upper bounds ( $min (1.05, 1.00) = 1.00$ ) wins — this is exactly why the intersection is what matters.
Step $t = 1 \to 2$ . raw $= 1.00 + 0.06 = 1.06$ . Clip to $[0.85, 1.00]$ : $1.00$ . Unchanged. Why this step? PGD steps again from the current $1.00$ ; the raw $1.06$ is over both ceilings, so it is dragged back to $1.00$ .
Step $t = 2 \to 3$ . raw $= 1.00 + 0.06 = 1.06 \to$ clip $\to 1.00$ . Still $1.00$ . Why this step? Once pinned at a saturated boundary, further same-direction steps stay pinned — extra iterations buy nothing here.

Verify: final $x^{(3)} = 1.00$ . Both constraints satisfied: $∣1.00 - 0.95∣ = 0.05 \leq ϵ$ ✓ and $0 \leq 1.00 \leq 1$ ✓. Lesson: a pixel already near the boundary of valid values has less room than $ϵ$ suggests — real attacks account for this.

Figure s03 (sketch it yourself): PGD step number on the x-axis ( $0, 1, 2, 3$ ), pixel value on the y-axis. A plum dashed line traces the pre-clip "raw" values climbing past $1.00$ ( $0.95 \to 1.01 \to 1.06 \to 1.06$ ); a solid ink line shows the post-clip values flattening onto the orange ceiling at $1.00$ from step 1 on. A dotted teal line marks the looser $ϵ$ -ceiling $x + ϵ = 1.05$ that never gets to bind because the pixel ceiling $1.00$ is tighter.

Why high dimensions are the attacker's friend

Worked example Ex 7 — Cell I (the

$ϵ d$ blow-up) Suppose every one of $d$ pixels has the same gradient sign, and we move each by the full $ϵ$ (FGSM). Compare $d = 4$ and $d = 150, 528$ (a $224 \times 224 \times 3$ image). Take a per-pixel gradient magnitude of $g = 1$ so numbers stay clean, and $ϵ = 0.01$ .

Forecast: the loss increase adds up over all pixels, so it grows with $d$ even though each pixel barely moves.

Per-pixel contribution. Each pixel contributes $δ_{i} \cdot g_{i} = ϵ \cdot 1 = 0.01$ to the linearized loss change $δ^{⊤} \nabla_{x} L = \sum_{i} δ_{i} g_{i}$ . Why this step? With all signs aligned, no cancellation happens — every term is positive and equal.
Sum over $d$ pixels. Total loss change $= d \cdot ϵ = d \cdot 0.01$ . Why this step? $d$ equal terms of $0.01$ add straight up; the dot product is just a sum, so more dimensions means more terms piling on.
- $d = 4$ : change $= 0.04$ (small).
- $d = 150528$ : change $= 1505.28$ (enormous).
The "per-dimension-imperceptible" version. If instead we shrink each move to $ϵ / d$ (so the whole $L_{2}$ size stays fixed at $ϵ$ ), the total is $\sum_{i} \frac{ϵ}{d} \cdot 1 = d ϵ$ . For $ϵ = 0.01$ , $d = 150528$ : $150528 \approx 388.0$ , times $0.01 \approx 3.88$ . Why this step? This is the headline of the parent note: even with a fixed, tiny total budget, the achievable loss change scales as $d$ — the curse of dimensionality, turned into a weapon.

Verify: $d ϵ$ at $d = 4$ is $0.04$ ✓; at $d = 150528$ is $1505.28$ ✓. $150528 ϵ \approx 3.88$ ✓ (vs a mere $0.02$ for $d = 4$ ).

A real-world unit problem

Worked example Ex 8 — Cell J (the

$8/255$ budget in real RGB units) An RGB pixel channel is an integer $0$ – $255$ . A standard robustness benchmark uses $ϵ = 8/255$ on inputs normalized to $[0, 1]$ . A particular channel reads $200/255$ and its loss-gradient sign there is $+ 1$ . What is the attacked value, in both $[0, 1]$ and $0$ – $255$ units?

Forecast: $8/255 \approx 0.0314$ , so the value climbs by about $3%$ — visually invisible.

Convert the budget to $[0, 1]$ units. $ϵ = 8/255 \approx 0.03137$ . Why this step? The input lives on the normalized $[0, 1]$ scale, so the budget must be expressed on that same scale before we add anything.
Apply the FGSM step (one pixel). normalized value $= 200/255 + 0.03137 = 0.7843 + 0.0314 = 0.8157$ . Why this step? Sign $+ 1$ means push up by the full budget; add $ϵ$ to the current normalized value.
Convert back to $0$ – $255$ units. $0.8157 \times 255 = 208$ , i.e. $208/255$ . So the channel goes $200 \to 208$ — a change of exactly $8$ levels. Why this step? Multiplying a normalized value by $255$ returns the integer intensity; confirming the move is exactly $8$ levels shows the budget does what it advertises.

Verify: in $[0, 1]$ units the answer is $208/255 \approx 0.8157$ ; in $0$ – $255$ units it is $208$ . Check: $200/255 + 8/255 = 208/255$ ✓ exactly; $∥ δ ∥_{\infty} = 8/255 \approx 0.0314 = ϵ$ ✓. A shift of $8$ out of $255$ is below human perception, yet enough to flip a classifier — that is why this budget is the field's default.

The exam twist: $L_{2}$ ball changes the optimal step

Worked example Ex 9 — Cell K (

$L_{2}$ optimum is NOT the sign) Recall the == $L_{2}$ norm== is the ordinary Euclidean length $∥ δ ∥_{2} = \sum_{i} δ_{i}^{2}$ ; bounding it by $ϵ$ carves out a ball (circle in 2-D) of radius $ϵ$ , not a square box. Under this $L_{2}$ budget, what is the loss-maximizing $δ$ ? Use gradient $\nabla_{x} L = (3, 4)$ and $ϵ = 1$ .

Forecast: for $L_{\infty}$ we used the sign. For $L_{2}$ the optimal move points exactly along the gradient, scaled to length $ϵ$ — a different vector. Guess whether the two answers match.

State the optimum for $L_{2}$ . Maximizing $δ^{⊤} \nabla_{x} L$ with $∥ δ ∥_{2} \leq ϵ$ is solved (Cauchy–Schwarz) by aligning $δ$ with $\nabla_{x} L$ : $δ = ϵ \frac{\nabla _{x} L}{∥ \nabla _{x} L ∥ _{2}}$ . Why this step? Cauchy–Schwarz says $δ^{⊤} g \leq ∥ δ ∥_{2} ∥ g ∥_{2}$ , with equality only when $δ$ is parallel to $g$ . So keep the magnitude ratios of the gradient — the opposite of FGSM which discards them.
Compute the norm. $∥ (3, 4) ∥_{2} = 9 + 16 = 25 = 5$ . Why this step? We need the gradient's length to rescale it to exactly $ϵ$ ; dividing by its own length makes a unit vector.
Plug in. $δ = 1 \cdot \frac{( 3 , 4 )}{5} = (0.6, 0.8)$ . Why this step? Multiplying the unit gradient by $ϵ = 1$ gives the longest legal move along the gradient direction — the $L_{2}$ -optimal perturbation.
Compare to the $L_{\infty}$ /sign answer. $sign (3, 4) = (1, 1)$ ; scaling that to $L_{2}$ length $1$ gives $(0.707, 0.707)$ — a different direction. The two balls give genuinely different attacks. Why this step? It makes the exam point concrete: the shape of the budget (box vs ball) decides the optimal move, so you cannot blindly reuse the sign rule.

Verify: $∥ (0.6, 0.8) ∥_{2} = 0.36 + 0.64 = 1 = 1 = ϵ$ ✓. Loss change $= (0.6) (3) + (0.8) (4) = 1.8 + 3.2 = 5.0 = ∥ \nabla_{x} L ∥_{2} \cdot ϵ$ ✓ (equals the Cauchy–Schwarz bound, so it is truly optimal). Sign-based $(0.707, 0.707)$ would give $0.707 (3) + 0.707 (4) = 4.95 < 5.0$ — strictly worse under $L_{2}$ , confirming the norm choice changes the answer.

Figure s04 (sketch it yourself): axes "delta 1" and "delta 2". Draw a teal $L_{2}$ circle of radius $1$ and a plum dashed $L_{\infty}$ square of half-width $1$ . An ink arrow points along the gradient $(3, 4)$ . The orange $L_{2}$ -optimal dot sits on the circle at $(0.6, 0.8)$ (on the gradient line); the plum "sign, rescaled" dot sits at $(0.707, 0.707)$ — clearly not on the gradient line, showing the two attacks differ.

Recall Self-test

FGSM under $L_{\infty}$ uses which function of the gradient? ::: $sign$ — direction only, magnitude discarded. $sign (0)$ equals? ::: $0$ — that pixel does not move. What does $∥ δ ∥_{\infty}$ measure? ::: The largest single-pixel move, $max_{i} ∣ δ_{i} ∣$ . What does $clip (z, lo, hi)$ do? ::: Returns $max (lo, min (z, hi))$ — squashes $z$ into $[lo, hi]$ . Targeted FGSM uses plus or minus $ϵ sign$ ? ::: Minus — we descend the target-class loss. In PGD, what does the step size $α$ satisfy and why? ::: $α < ϵ$ , so no single step can jump the whole budget; the attack creeps and corrects course. In PGD, what does the clip/projection step guarantee? ::: The perturbation stays within the $ϵ$ -ball around the original $x$ (and often within valid pixel range). Under an $L_{2}$ budget, the optimal $δ$ points along what? ::: The gradient $\nabla_{x} L$ itself, scaled to length $ϵ$ (not its sign). Why does high $d$ help the attacker? ::: Aligned per-pixel nudges accumulate; loss change grows like $ϵ d$ under a fixed $L_{2}$ budget.

See also: Gradient descent optimization (the same $\nabla_{x} L$ machinery, but PGD ascends the input not the weights), Neural network training, Out-of-distribution detection, Interpretability and explainability, and the framing in AI Safety fundamentals and Reward hacking.