Question bank — Probing classifiers

False — probing measures presence, not usage. The info can sit in the representation as a passenger the model never consults; you need Causal Intervention Methods to test usage.

False — the property is present but nonlinearly entangled. The MLP untangles it; the linear probe can't draw one flat plane through it.

False — if you unfreeze, the base model can learn $\tau$ during probing, so you'd measure its capacity to learn, not what it already knew.

False — accuracy can rise just because a stronger/deeper probe was used. Explicitness is about linear accessibility at fixed probe complexity, not raw accuracy.

False — that gap is memorization. Only held-out accuracy counts; this is why L2 regularization is applied to the probe.

Not necessarily — one might be linearly separable with a large margin $\gamma$ (robust) and the other barely separable with a thin margin (see figure s02). Same accuracy hides very different encodings.

False — word identity alone strongly correlates with POS (e.g. "the" is almost always a determiner), so layer 0 legitimately scores high without any context.

Not necessarily — with 45 tags, chance is far below 45%; you must compare to the majority-tag baseline, not to 50%.

Error — without a control (probing with shuffled/random labels or a random network), you don't know if 88% reflects real encoding or just the probe's own expressive power.

Error — the MLP succeeding proves the info is encoded (nonlinearly). The correct read is: present but entangled, not absent.

Error — fine-tuning breaks the frozen assumption; the boost may come from the model newly learning $\tau$, invalidating the diagnostic.

Error — an unregularized deep probe has enough free parameters to bend around individual training points (memorize them), so high train accuracy no longer tells you the info was easily accessible; you measure the probe, not the representation. See figure s02 (the right panel there shows exactly this wiggly boundary threading through noise).

Error — the small drop reflects the output layer specializing for the pre-training objective, not degradation. A 2-point dip is within normal layer-wise variation.

Error — joint training entangles the two objectives; the representation you're probing is no longer the pre-trained one you meant to study.

Error — mutual information / correlation ≠ causation. High MI only says $\mathbf{h}$ predicts the label; you'd need an intervention (edit the feature, watch the output change) to claim causal responsibility.

Because a powerful probe can extract info from any representation, so it can't distinguish "easily accessible" from "buried deep." Simplicity turns the probe into a fair yardstick.

Because where a property lives is itself the finding — early layers tend to hold surface/syntactic features, deeper layers hold semantics, revealing the model's processing pipeline.

Bigger $\lambda$ forces $\mathbf{w}$ smaller and simpler, so the probe can't memorize idiosyncrasies; it must rely on genuinely accessible, robust structure. Setting $\lambda =0$ removes the leash and invites overfitting.

A large $\gamma$ means a wide clean gap — the linear encoding is robust to noise; a tiny positive $\gamma$ means the classes almost touch, so the probe barely works and the info is nearly nonlinear or noisy.

It tells you what information is present and where, but not why the model behaves as it does — behavior requires causal tests, linking to Causal Intervention Methods.

Random high-dimensional projections can accidentally make simple properties separable; that's why a random-network control tells you how much accuracy is "free" before real learning.

A shrinking linear-vs-MLP gap across layers signals the network is untangling the property into an explicit, axis-aligned direction — the hallmark of disentanglement.

Attention lets a token gather information from far-away words (like the subject's number), so the local activation ends up carrying context that a linear probe can read off.

With enough hidden units it can draw a boundary that wraps each point individually (like memorizing a lookup table), so it "fits" even random labels — which is exactly why train accuracy alone is worthless as evidence.

Ambiguous by itself: either the info isn't there, OR it's so nonlinearly entangled that even the probe can't reach it. You must add a stronger-probe or control to disambiguate.

No — 95% is exactly the majority-class baseline (always guess the common class). Always compare against this prior; here the probe may have learned nothing.

Binary or multiclass: the majority-class baseline = fraction of examples in the single most common class (always predict it). For a regression probe, the baseline is predicting the mean value (its error is the label variance).

The probe may be exploiting the confound, not the property. High accuracy then says nothing about $\tau$ being encoded — control for the confound.

Yes, the probe can still read that direction from the frozen activations, even though the model ignores it — another reminder that presence ≠ usage.

Only the majority-class rate — no hyperplane can separate coincident points, so no probe, however complex, exceeds the prior.

Trends flatten toward the random-projection baseline; meaningful layer structure only emerges because pre-training shaped the representations.

There is no notion of "correct class" for a real number, so you score with mean-squared error or correlation instead; a perfect probe drives error to zero, and the baseline is the variance of the target.