Question bank — Emergent abilities in large models

False. Loss follows a smooth power law $L(N,D)=E+A/N^{\alpha }+B/D^{\beta }$ with no discontinuity; the kink lives only in the downstream metric $\text{Acc}=p^L$, which is the smooth-loss-composed-with-exponentiation transform shown above.

False (usually). Under the $\text{Acc}=p^L$ picture accuracy is a continuous function of $p$ — it is sharp but smooth. A genuine discontinuity would require a jump in the underlying loss, which we do not observe.

False. The jump can be an artifact of a harsh, thresholded metric (exact-match). The same model measured by edit distance or per-token log-likelihood often improves smoothly — the "new skill" was rising continuously the whole time.

False. The model is identical; only the lens differs. The smooth aggregate metric is $\text{TokenScore}=Lp$ (expected number of correct tokens out of $L$, linear in $p$), while $\text{Acc}=p^L$ is sharp — same weights, two lenses. (If you want the smooth metric on a $0\text{–}1$ scale, use the per-token fraction $\text{TokenScore}/L=p$ itself.)

False. Emergence is task- and metric-specific: only high-$L$ all-or-nothing metrics manufacture the jump. A short-$L$ task, or the same task read by a per-token metric, improves smoothly with no leap.

False. On Inverse scaling tasks accuracy falls as $N$ grows: the model latches onto a superficially-correct pattern (e.g. copying a memorized surface form) that a bigger model reproduces more confidently, so the harmful behavior strengthens rather than washing out. "Bigger is better" is a trend, not a law.

False. Exact-match sits at chance because $p^L\approx 0$ when $p$ is modest, but the underlying $p$ (and its smooth surrogates: edit distance, per-token log-likelihood) is already moving. Fit the trend of the smooth metric on small models and you can forecast large-model skill — the key early-warning signal.

False. No contradiction: loss obeys the smooth law $L(N,D)=E+A/N^{\alpha }+B/D^{\beta }$, and $\text{Acc}=\exp (-L(L(N,D)-E))$ is that smooth loss pushed through exponentiation. The smoothness and the sharpness are two views of one continuous object.

False. Grokking (delayed generalization) is delayed generalization over training time at fixed scale; emergence is a jump across model scale $N$. Related in spirit (sharp transition) but on different axes.

False. Chain-of-thought prompting hurts small models — the extra intermediate steps add fragile factors, so $p^k$ shrinks — and only helps once per-step reliability $p$ is high enough that decomposition buys more than it costs.

The error is measuring the harsh metric to detect smooth progress — self-defeating. The actionable surrogates are computed before thresholding: (1) per-token log-likelihood $=\frac{1}{L}{\sum }_i\log p_i$ (average token log-prob, exactly what cross-entropy already tracks); (2) token edit distance between output and target; (3) expected correct-token fraction $p$. Log and plot these against $N$ — they move on small models where exact-match is still flat.

The error is assuming the $p^L$ model holds, which requires tokens to be independent and equally reliable. It breaks when errors are correlated (one confusion derails a whole span) or when $p$ varies token-to-token; then the effective exponent is not $L$ and the inversion is wrong. Detect the breakdown by checking whether measured accuracy at several lengths $L$ actually falls off geometrically — if it doesn't, independence has failed.

The error is collapsing three axes into one. $C\approx 6ND$ ties params and tokens together, but Compute-optimal training shows the split between $N$ and $D$ matters; the same $C$ with different $N:D$ ratios gives different loss and thus different capabilities.

The error is treating $p$ as scale- and length-invariant. In truth $p=p(N)$ rises with scale, and the effective $p$ can also drift with position or context length (later tokens condition on possibly-wrong earlier ones). The single-number $p^L$ story is a first-order idealization; watch for length-dependent $p$ when a task's accuracy-vs-$L$ curve bends away from a clean geometric decay.

The error is over-reading "mirage." Schaeffer et al. show the discontinuity may be a metric artifact — but the sharp, practically-relevant swing (near-0 to high accuracy over a narrow scale range) is real and still demands forecasting and safety attention.

The error is adding instead of chaining. Independent serial steps compound multiplicatively: success $\approx p^k$, not $kp$. That multiplication is exactly what makes composition fragile and emergence-prone.

Raising a fixed $p<1$ to a larger power drives $p^L$ toward $0$ faster for small $p$ while leaving it near $1$ for $p\to 1$, so the band of $p$ over which $p^L$ climbs from ~0 to ~1 narrows — pure geometry of exponentiation.

Log-likelihood averages a continuous per-token quantity (no all-or-nothing threshold), so it tracks the smooth loss directly; exact-match applies an $L$-fold AND that amplifies tiny residuals into a step.

Each step is a fragile factor; overall success $\approx p^k$. Until $p$ crosses a knee, adding steps lowers success, so the composite metric stays pinned near chance and then leaps once $p$ is high enough.

At low $p$ the extra intermediate tokens are error-prone, so decomposition multiplies mistakes ($p^k$ shrinks); once $p$ is high, decomposing a hard task into individually-reliable easy steps raises effective reliability and pays off.

Because exact-match sits at chance for small $N$ — a flat line carries no slope to extrapolate. The information lives in a smooth surrogate (log-likelihood / edit distance); extrapolate that, then push it through the $p^L$ transform.

Apt because both show sharp macroscopic change from smooth microscopic parameters (see Phase transitions in physics); imperfect because a true phase transition is genuinely singular in the thermodynamic limit, whereas $p^L$ emergence is continuous — sharp, not singular.

The model must infer a latent task and apply it consistently across many output tokens — a high-$L$, all-or-nothing metric that only clears once per-token reliability $p$ is high enough.

It collapses to $\text{Acc}=p$ — perfectly smooth. A single-token task shows no emergence at all; the sharpness is entirely a many-token effect.

$\text{Acc}=1^L=1$ for all $L$ — length no longer matters. The emergence "engine" only operates strictly between chance and perfection ($0<p<1$).

The $p^L$ form assumes independence; correlation can make the true curve sharper or softer, and the effective exponent is no longer $L$. Detect it by measuring accuracy at several lengths and checking whether it decays geometrically — deviation signals correlated errors.

The "pinned at chance" floor is then high, so there is little headroom for a dramatic jump — emergence looks muted. Chance level sets the floor the $p^L$ curve rises from.

Then the sharpness is not a metric artifact — it points toward a genuine capability transition (Explanation 1). The two explanations are distinguished precisely by re-measuring with a smooth metric.

Yes. The model's smooth internal change gets read differently by different metrics; a harsh metric may show a late jump while a pathology-sensitive metric shows worsening (Inverse scaling) over some range.

As $N\to \infty$, $A/N^{\alpha }\to 0$, so loss $\to E+B/D^{\beta }$; if $D$ is also large, $p\to 1$ and $\text{Acc}\to 1$. Compute alone can't reach $1$ if data is bottlenecked — a Compute-optimal training point.