Visual walkthrough — Inner product spaces — dot product generalization

Before line one, three plain-word promises about the symbols you'll meet:

---

Step 1 — Slide one arrow along another

WHAT. Take two vectors $\mathbf{u}$ and $\mathbf{v}$. Build a family of new vectors by subtracting a scaled copy of $\mathbf{v}$ from $\mathbf{u}$: $\mathbf{w}(t)=\mathbf{u}-t\mathbf{v}.$

• $\mathbf{u}$ — the fixed arrow we start from.
• $t\mathbf{v}$ — a stretched copy of $\mathbf{v}$; the dial $t$ decides how much of $\mathbf{v}$ we remove.
• $\mathbf{w}(t)$ — what's left over. As $t$ turns, the tip of $\mathbf{w}(t)$ slides along a straight line parallel to $\mathbf{v}$.

WHY. We want a quantity that is guaranteed non-negative no matter what. Length-squared is exactly such a quantity (axiom 3, positive-definiteness). By making a whole sliding family, we get one inequality for every value of $t$ at once — that's a lot of free information from one idea.

PICTURE. The blue arrow $\mathbf{u}$ stays put. The yellow arrow $t\mathbf{v}$ grows as we turn the dial. The red arrow $\mathbf{w}(t)$ is the gap. Watch it shrink to its shortest, then grow again.

---

Step 2 — Its length can never be negative

WHAT. Measure the squared length of that leftover arrow: $0\le ⟨\mathbf{u}-t\mathbf{v},\mathbf{u}-t\mathbf{v}⟩.$

• The left $0\le$ — the whole point: length-squared is never below zero.
• Inside the brackets — the same vector $\mathbf{w}(t)$ in both slots, because $\Vert \mathbf{w}{\Vert }^2=⟨\mathbf{w},\mathbf{w}⟩$.

WHY. This single line is our engine. Positive-definiteness says every real arrow has length $\ge 0$, so for every $t$ the right-hand side stays at or above zero. We now squeeze this promise for all it's worth.

PICTURE. The red gap arrow from Step 1, and its length plotted as a bar that never dips below the floor line at $0$.

---

Step 3 — Expand into a parabola in $t$

WHAT. Open the brackets using linearity and symmetry (axioms 1 and 2). Doing it slot by slot:

= \langle\mathbf u,\mathbf u\rangle \;-\; t\langle\mathbf u,\mathbf v\rangle \;-\; t\langle\mathbf v,\mathbf u\rangle \;+\; t^2\langle\mathbf v,\mathbf v\rangle.$$ The two middle terms $-t\langle\mathbf u,\mathbf v\rangle$ and $-t\langle\mathbf v,\mathbf u\rangle$ are **equal**, because symmetry (axiom 1) says $\langle\mathbf u,\mathbf v\rangle=\langle\mathbf v,\mathbf u\rangle$. So they collapse into a single $-2t\langle\mathbf u,\mathbf v\rangle$. Regrouping by powers of $t$: $$0 \;\le\; \underbrace{\langle\mathbf v,\mathbf v\rangle}_{a}\,t^2 \;\underbrace{-\,2\langle\mathbf u,\mathbf v\rangle}_{b}\,t \;+\; \underbrace{\langle\mathbf u,\mathbf u\rangle}_{c}.$$ Reading off the coefficients of $f(t)=a\,t^2 + b\,t + c$ (note the brace over $b$ **includes** the minus sign): - $a=\langle\mathbf v,\mathbf v\rangle=\|\mathbf v\|^2$ — coefficient of $t^2$, the "how fast the parabola opens" number. It is $\ge 0$. - $b=-2\langle\mathbf u,\mathbf v\rangle$ — the coefficient of $t$; the minus sign is **part of $b$**, so that $b\,t$ reproduces the term $-2\langle\mathbf u,\mathbf v\rangle\,t$ above. This is where our target quantity $\langle\mathbf u,\mathbf v\rangle$ is hiding. - $c=\langle\mathbf u,\mathbf u\rangle=\|\mathbf u\|^2$ — the constant, the height at $t=0$. **WHY.** With those three coefficients, the right side is exactly $f(t)=a\,t^2+b\,t+c$ — a **parabola** in the dial $t$. We chose the tool "quadratic" because a parabola's shape is completely pinned down by three numbers, and one clean fact — the discriminant — reads off whether it ever dips below zero. **PICTURE.** The three annotated pieces feeding into one upward-opening parabola $f(t)$. ![[deepdives/dd-maths-4.5.33-d2-s03.png]] --- ## Step 4 — "Never below zero" forces the parabola to sit on the axis **WHAT.** Because $f(t)\ge 0$ for **every** $t$, this upward parabola can never cross below the horizontal axis. It may *kiss* the axis (one touching point) or float entirely above it — but it can never dip through. **WHY.** If it dipped below, there would be some dial value $t_0$ where $f(t_0)<0$, i.e. a leftover arrow with negative length-squared. That contradicts positive-definiteness. So "dipping below" is forbidden — geometry itself blocks it. **PICTURE.** Three candidate parabolas: green floats above (OK), yellow just touches (OK, the boundary), red dips below (FORBIDDEN — crossed out). ![[deepdives/dd-maths-4.5.33-d2-s04.png]] --- ## Step 5 — Translate "never crosses" into the discriminant test **WHAT.** For a quadratic $at^2+bt+c$ with $a>0$, the number of times it crosses the axis is decided by the ==discriminant== $\;b^2-4ac$: $$b^2-4ac \;>\;0 \Rightarrow \text{two crossings (dips below)},\qquad b^2-4ac\;\le\;0 \Rightarrow \text{never dips below}.$$ Since Step 4 forbids dipping below: $$b^2 - 4ac \;\le\; 0.$$ **WHY.** The discriminant is the algebra of "where does the parabola meet the axis?". The roots are $t=\frac{-b\pm\sqrt{b^2-4ac}}{2a}$; a positive $b^2-4ac$ gives two real roots (the parabola pokes through between them), a zero gives one (a kiss), a negative gives none (floats). We need "no poking through", hence $\le 0$. **PICTURE.** The root formula with the $\sqrt{b^2-4ac}$ highlighted, and a number-line showing sign of the discriminant vs. crossing behaviour. ![[deepdives/dd-maths-4.5.33-d2-s05.png]] --- ## Step 6 — Substitute our actual $a,b,c$ and clean up **WHAT.** Put back $a=\|\mathbf v\|^2$, $b=-2\langle\mathbf u,\mathbf v\rangle$, $c=\|\mathbf u\|^2$: $$\underbrace{\big(-2\langle\mathbf u,\mathbf v\rangle\big)^2}_{b^2} \;-\; \underbrace{4\,\|\mathbf v\|^2\,\|\mathbf u\|^2}_{4ac} \;\le\; 0.$$ The left square is $4\langle\mathbf u,\mathbf v\rangle^2$. Divide the whole thing by $4$: $$\langle\mathbf u,\mathbf v\rangle^2 \;\le\; \|\mathbf u\|^2\,\|\mathbf v\|^2.$$ Take the (non-negative) square root of both sides: $$\boxed{\;|\langle\mathbf u,\mathbf v\rangle| \;\le\; \|\mathbf u\|\,\|\mathbf v\|\;}\qquad\blacksquare$$ - The $|\cdot|$ appears because $\sqrt{x^2}=|x|$ — the reading could be negative, its size cannot exceed the length product. **WHY.** This is exactly the statement that the top of $\cos\theta$ can never outgrow the bottom. So $\cos\theta\in[-1,1]$, and "angle" $\theta$ finally makes honest sense. Divide by $\|\mathbf u\|\|\mathbf v\|$ (both nonzero) to see it. **PICTURE.** The chain of three shrinking boxes: squared inequality → rooted inequality → the $\cos\theta$ meter pinned inside $[-1,1]$. ![[deepdives/dd-maths-4.5.33-d2-s06.png]] --- ## Step 7 — The degenerate cases (never skip these) **WHAT.** We assumed we could turn the dial and divide. What if a vector is zero? **Case A — $\mathbf v = \mathbf 0$.** Then $a=\|\mathbf v\|^2=0$, so $f(t)=-2\langle\mathbf u,\mathbf 0\rangle t + \|\mathbf u\|^2$ is not a parabola but a line. But $\langle\mathbf u,\mathbf 0\rangle=0$ (linearity: $\langle\mathbf u,0\cdot\mathbf 0\rangle=0$), so both sides of Cauchy–Schwarz read $0\le 0$. **True.** No division needed. **Case A′ — $\mathbf u = \mathbf 0$.** By symmetry (axiom 1), $\langle\mathbf 0,\mathbf v\rangle=\langle\mathbf v,\mathbf 0\rangle=0$ and $\|\mathbf 0\|=0$, so again both sides read $0\le 0$. **True.** More generally, because Cauchy–Schwarz is completely **symmetric in $\mathbf u$ and $\mathbf v$**, any argument that handles $\mathbf v=\mathbf 0$ handles $\mathbf u=\mathbf 0$ by just swapping the names — so no case is left uncovered. **Case B — equality, $|\langle\mathbf u,\mathbf v\rangle| = \|\mathbf u\|\|\mathbf v\|$.** This is the *kiss* case (discriminant $=0$): the parabola touches the axis at one dial value $t_0$, meaning $\|\mathbf w(t_0)\|^2=0$, so $\mathbf u - t_0\mathbf v = \mathbf 0$, i.e. $\mathbf u = t_0\mathbf v$. **Equality happens exactly when the two arrows are parallel** (one is a scalar multiple of the other). Geometrically: the red gap arrow can be shrunk all the way to nothing. **WHY.** A theorem you can't apply at $\mathbf 0$ or at the boundary is a theorem with holes. Covering these means every possible pair of vectors is handled — no reader hits an unshown scenario. **PICTURE.** Left panel: $\mathbf v=\mathbf 0$, the family collapses to a single arrow $\mathbf u$. Right panel: $\mathbf u$ parallel to $\mathbf v$, red gap vanishes, parabola kisses the axis. ![[deepdives/dd-maths-4.5.33-d2-s07.png]] --- ## The one-picture summary Everything above, compressed: turn the dial $t$ → leftover arrow $\mathbf w(t)$ → its length-squared is a non-negative parabola → non-negative forces discriminant $\le 0$ → that is Cauchy–Schwarz → $\cos\theta$ lives in $[-1,1]$, so the angle $\theta$ between $\mathbf u$ and $\mathbf v$ is honest. ![[deepdives/dd-maths-4.5.33-d2-s08.png]] ```mermaid graph TD A["Slide u minus t times v"] --> B["Length squared is at least 0"] B --> C["Expand into parabola f of t"] C --> D["f never dips below 0"] D --> E["Discriminant at most 0"] E --> F["Cauchy Schwarz inequality"] F --> G["cos theta between minus 1 and 1"] ``` > [!recall]- Feynman: the whole walkthrough in plain words > Take two arrows, $\mathbf u$ and $\mathbf v$. Now play a game: slowly remove copies of $\mathbf v$ from $\mathbf u$, controlled by a dial called $t$. What's left is a "gap" arrow. A gap arrow, like any arrow, cannot have negative length — that's the one rock-solid rule. If you write the gap's length-squared as the dial turns, you get a smiley-shaped curve (a parabola). Because length can never be negative, this smiley can never dunk below the floor. A smiley that never dunks below the floor obeys a tidy algebra test called "discriminant at most zero". Cash that test out with the actual numbers, and out pops: the magic-ruler reading of $\mathbf u$ and $\mathbf v$ can never be bigger than their two lengths multiplied. That single fact is what lets us divide and call the result a cosine — because now it's trapped between $-1$ and $+1$, exactly where a real angle $\theta$ lives. Two loose ends: if $\mathbf v$ (or $\mathbf u$) is the zero arrow the whole thing is just $0\le 0$, still true; and the smiley touches the floor (equality) precisely when the two arrows point the same way, so the gap can be squeezed to nothing. > [!recall]- Quick self-check > Why must the parabola's discriminant be $\le 0$ and not $\ge 0$? ::: Because $f(t)=\|\mathbf u-t\mathbf v\|^2\ge 0$ never dips below the axis; a positive discriminant would give two real roots and a dip between them, contradicting positive-definiteness. > When does Cauchy–Schwarz become an equality? ::: Exactly when $\mathbf u=t_0\mathbf v$ for some scalar — the vectors are parallel (the parabola kisses the axis). > Where in the derivation is the target quantity $\langle\mathbf u,\mathbf v\rangle$ hiding? ::: In the middle coefficient $b=-2\langle\mathbf u,\mathbf v\rangle$ of the parabola.