Foundations — Concept-based explanations

Everything below is a tool you need before that sentence makes full sense. We build each symbol from nothing, anchor it to a picture, and say why the topic can't live without it. We introduce them in strict order: no symbol appears before the section that defines it.

1. A vector: a list of numbers that is also an arrow

Why the word "vector" and not just "list"? Because a list of numbers has a picture: an arrow from the origin (the point $(0,0)$) to the point those numbers name.

Look at the burnt-orange arrow: it starts at the origin and its tip lands at $(3,1)$. The two numbers "3 right, 1 up" are the arrow. In this topic, when a picture (say a zebra) is fed to a model, the model turns it into a list of numbers — so the picture becomes an arrow in a space we can reason about geometrically.

2. Two vector operations: adding arrows, and scaling arrows

Before we compare arrows we need two ways to combine them. Both are defined slot by slot, so both work in any dimension.

Division by a number is just scaling by its reciprocal: $\mathbf{a}/s=\frac{1}{s}\mathbf{a}$, again slot by slot. We will need exactly this in $\S 6$ to shrink an arrow to length 1.

3. Dimension and ${\mathbb{R}}^d$: how many numbers in the list

• ${\mathbb{R}}^2$ = points on a flat page (2 numbers: across, up).
• ${\mathbb{R}}^3$ = points in a room (3 numbers).
• ${\mathbb{R}}^{2048}$ = a list of 2048 numbers. We can't draw it, but every rule from 2D still holds.

4. Activations ${\mathbf{h}}_{\ell }$: the model's thought at layer $\ell$

A neural network is a stack of layers. Each layer takes the numbers from the layer before, mixes them, and produces a new list of numbers. That output list at a chosen layer is what we call the activation.

Follow the teal path: an image enters on the left, flows through layers, and at the marked layer $\ell$ we tap off the numbers. Different images give different taps — each becomes its own dot in activation space.

5. Norm and length: how long an arrow is

We define length before the dot product, because the dot-product identity will refer to it.

The norm is $0$ for one and only one vector — the zero vector $(0,0,\dots ,0)$ — because that's the only arrow with no length. We will use this fact in $\S 6$.

6. Normalization: shrinking an arrow to length 1

Using scaling from $\S 2$ and length from $\S 5$, we can force any arrow to length exactly 1 while keeping its direction.

7. The dot product: how much two arrows agree

Here is the single most important tool of the whole topic. We only use symbols already defined above — vectors ($\S 1$), scaling ($\S 2$), and length ($\S 5$).

Why THIS tool and not another? We want a single number that answers "do these two arrows point the same way?" The dot product does exactly that. Here is why those slot-wise products secretly measure an angle.

First, what does "angle" even mean when there are more than 3 slots? We define $\theta$ to be the angle you would measure inside the ordinary flat triangle formed by the two arrows and the segment joining their tips — and any two arrows, no matter how many slots, always lie inside a single flat plane, so that triangle is a real 2D picture even in ${\mathbb{R}}^{2048}$.

Now the connection. Suppose both arrows have length 1. Slide arrow $\mathbf{b}$'s shadow onto arrow $\mathbf{a}$: the length of that shadow (the projection) is exactly $\cos \theta$ — that is the very definition of cosine on a right triangle ("adjacent over hypotenuse," and here the hypotenuse is 1). The slot-wise sum $\mathbf{a}\cdot \mathbf{b}$ turns out to compute precisely this shadow length. Scaling each arrow back up by its own length $\Vert \mathbf{a}\Vert ,\Vert \mathbf{b}\Vert$ (defined in $\S 5$) multiplies the shadow by both lengths, giving the identity:

$\mathbf{a}\cdot \mathbf{b}=\Vert \mathbf{a}\Vert \Vert \mathbf{b}\Vert \cos \theta$

So the sum-of-products is not a coincidence: it is "how much one arrow's shadow falls along the other," and shadow direction is exactly what "do they agree?" means.

Read the three cases in the figure:

• Same direction ($\theta$ small): $\cos \theta$ near $+1$, dot product positive — the arrows agree.
• Perpendicular ($\theta =90^{\circ }$): $\cos \theta =0$, dot product zero — no agreement either way.
• Opposite ($\theta$ near $180^{\circ }$): $\cos \theta$ near $-1$, dot product negative — the arrows disagree.

8. A hyperplane and its normal vector: the fence and the direction across it

A general hyperplane is described by a direction $\mathbf{w}$ and a number $b$ (the bias or intercept): the wall is the set of points $\mathbf{h}$ with $\mathbf{w}\cdot \mathbf{h}+b=0$ (using the dot product from $\S 7$).

Every hyperplane has one special direction: the normal vector $\mathbf{w}$, the arrow that points straight across the wall (perpendicular to it). The bias only moves the wall; it does not change this crossing direction.

In the figure the plum dots are "striped" activations and the grey dots are "random" activations. The teal line is the hyperplane that best separates them — notice it does not pass through the origin, which is exactly what the bias $b$ buys us. The burnt-orange arrow is its normal $\mathbf{w}$ — pointing from the random side toward the striped side.

9. A linear classifier and its loss: drawing that fence

To get the hyperplane, we train a linear classifier: it finds a direction $\mathbf{w}$ and a bias $b$ so that the score ${\mathbf{w}}^{\top }\mathbf{h}+b$ is positive for one group and negative for the other.

Now the pieces of the training objective:

• $\arg {\min }_{\mathbf{w},b}$ means "the values of $\mathbf{w}$ and $b$ that make the following quantity smallest." (arg = argument = the input that wins.)
• $\mathcal{L}$ (script L) is a loss — a number measuring how wrong the classifier is on one example; smaller is better. We must pick a concrete form.

• $\lambda \Vert \mathbf{w}{\Vert }^2$ is a regularizer: a penalty on large weights (using the norm from $\S 5$) so the fence stays simple. $\lambda$ (Greek lambda) sets how hard we push. By convention the bias $b$ is left out of this penalty.

10. The logit $S_k(\mathbf{x})$: the model's raw score for class $k$

We use the raw score, not the final probability, because it's easier to differentiate cleanly (see $\S 11$).

11. The gradient $\nabla$: the arrow of steepest increase

Why THIS tool? We want to know "which way should the thought ${\mathbf{h}}_{\ell }$ move to make the model more sure of class $k$?" The gradient answers exactly that question and hands back an arrow — which we can dot ($\S 7$) with the concept arrow ${\mathbf{v}}_c$ ($\S 9$).

12. Expectation $\mathbb{E}$, the example set $X_k$, and the indicator $1\{\cdot \}$

Putting the last three sections together, the complete score the parent note writes is:

${\text{TCAV}}_{\ell ,c,k}={\mathbb{E}}_{\mathbf{x}\in X_k}\left[1\left\{{\nabla }_{{\mathbf{h}}_{\ell }}{S}_k(\mathbf{x})\cdot {\mathbf{v}}_c>0\right\}\right]$

Equipment checklist

Read each question, answer in your head, then reveal.

Connections

• Same foundations in Hinglish →
• Probing Classifiers — also lives in activation space; here we built the vector/hyperplane tools both methods share.
• Feature Attribution Methods — uses the same gradient $\nabla S_k$, but attributes to inputs instead of concept directions.
• Layer-wise Relevance Propagation — another way to trace a score $S_k$ backward through layers.
• Model Editing — once the concept direction is a concrete arrow, you can push activations away from it.
• Adversarial Examples — small moves in activation space that flip $S_k$; same geometry, different intent.