Question bank — Concept-based explanations

False. TCAV only says nudging activations toward the stripe direction raises the zebra logit; the model's internal notion of "stripe" may be alien and entangled with other features.

Roughly true but state it carefully: 0.5 is the random baseline, so a score near it means the concept has no consistent directional effect — the model neither favours nor opposes it.

False. 0.0 means the concept consistently opposes the class — pushing toward it lowers the logit — which is strong (negative) usage, not absence.

False. The CAV is trained on activations at a specific layer $\ell$; different layers give different CAVs because concepts are encoded differently across depth.

False. The CAV is the normal of a learned hyperplane; different positive/negative sets shift the boundary, so CAVs (and scores) can differ — this is why the random-baseline test matters.

True in spirit: the expectation over a class set is exactly what dampens single-sample noise, which is why we test on many images rather than one.

True. $\lambda$ regularises the weights, shifting the separating hyperplane's normal, so the CAV — and therefore the score — can move.

Careful-false. TCAV measures a local directional sensitivity of the logit, which is stronger than correlation but is not a full interventional causal proof; the concept could co-vary with the true cause.

The error is skipping the random baseline. In high dimensions random CAVs routinely score 0.6–0.7 by chance; without a p-value from random trials you cannot claim significance.

The error is a non-atomic concept. Mixing semantics means the score can't tell you which sub-concept the model uses — test each atomic concept separately.

The error is a layer mismatch. The gradient and the CAV must live in the same activation space ${\mathbb{R}}^d$ at the same layer, or their dot product is meaningless.

The error is confusing is used with should be used. A high score can expose a spurious shortcut (background) — a bug worth fixing via Model Editing, not a feature to celebrate.

The error is too few trials. With $n=3$ the smallest achievable p-value from $\frac{1+\#\{\ge \}}{n+1}$ is $\frac{1}{4}=0.25$; you need many random sets for a trustworthy p-value.

The error is ignoring the degenerate gradient. If ${\nabla }_{{\mathbf{h}}_{\ell }}{S}_k\approx 0$ the dot product's sign is noise; such saturated examples contribute unreliable votes to the fraction.

The error's conclusion is wrong, not the setup: the indicator only checks the sign of the dot product, and scaling either vector by a positive number never changes that sign.

The dot product's sign answers exactly the question "does moving toward the concept increase the logit?"; magnitude is thrown away by the indicator because we only count direction of influence.

High activation shows the concept is present; the derivative shows the concept is used to push the decision — the whole point is distinguishing presence from causal influence.

Averaging raw values lets a few huge-magnitude examples dominate; counting the fraction that vote positive gives a stable, interpretable number in $[0,1]$ robust to scale.

Clustering only finds recurring activation patterns (candidate concepts); it says nothing about whether they influence a class — TCAV supplies the importance test.

TCAV assumes concepts are (approximately) linearly encoded as directions in activation space; a linear separator recovers that single direction, whereas a nonlinear one wouldn't give one usable vector.

Probing asks "is concept X encoded here?" while TCAV asks "is X used for decision Y?"; a concept can be perfectly encoded yet never steer the logit, so one says yes and the other no.

Saliency shows where the model looks (pixels), but not what semantic idea those pixels represent; concepts fill the semantic gap that raw attribution leaves open.

The positive and negative directional votes cancel, driving the score toward the 0.5 baseline — the model is indifferent to the concept for that class.

The linear classifier fits poorly, so the "direction" is unreliable; a low separation accuracy is a warning that the concept isn't linearly encoded there — try a different layer.

There is no separating direction (labels are arbitrary), the CAV is essentially random, and its expected TCAV score collapses to the ~0.5 baseline by construction.

Suspicious: a high score that random CAVs also reach means the direction isn't special — the concept is likely not genuinely used despite the flashy number.

That's expected — concepts emerge at different depths, so you should report the layer, and a null result at one layer never rules out usage elsewhere.

If the logit is insensitive to the concept direction, gradients dot the CAV to near-zero-sign-noise, pushing the score toward 0.5 — high accuracy tells you nothing about concept reliance.

A high TCAV for a "watermark" concept exposes the shortcut; the model's decision hinges on a spurious artefact, which is a diagnostic win, not a modelling success.