Inner product spaces — dot product generalization: Visual walkthrough

4.5.33 · D2 · Maths › Linear Algebra (Full) › Inner product spaces — dot product generalization

Intuition Yeh page kya prove karta hai, aur kyun aapko care karni chahiye

Parent note (Inner Product Spaces) ne bataya tha ki ek inner product $⟨ u, v ⟩$ aapko do vectors $u$ aur $v$ ke beech ek angle $θ$ define karne deta hai, is tarah: $cos θ = \frac{⟨ u , v ⟩}{∥ u ∥ ∥ v ∥} .$ Yahan $θ$ bas do arrows ke beech ka angle hai $u$ aur $v$ ke beech — wahi idea jaise kagaz pe do arrows ke beech ka angle, bas protractor ki jagah inner product se measure kiya jaata hai. Lekin ek cosine sirf $- 1$ aur $+ 1$ ke beech hi reh sakta hai. Toh yeh formula bakwaas hai jab tak yeh proven na ho ki upar wala kabhi neeche wale se bada nahi hota. Yeh prove karna ki woh kabhi bada nahi hota — yahi Cauchy–Schwarz inequality hai: $∣ ⟨ u, v ⟩ ∣ \leq ∥ u ∥ ∥ v ∥.$ Yeh page ise zero se derive karta hai — koi formula assume kiye bina — pictures mein. Abstract statement ke liye Cauchy-Schwarz Inequality dekhein.

Pehli line se pehle, un symbols ke baare mein teen simple-words ke promises jinse aap milenge:

Step 1 — Ek arrow ko doosre ke saath slide karein

KYA. Do vectors $u$ aur $v$ lo. $v$ ki ek scaled copy ko $u$ se subtract karke naye vectors ki ek family banao: $w (t) = u - t v .$

$u$ — woh fixed arrow jisse hum shuru karte hain.
$t v$ — $v$ ki ek stretched copy; dial $t$ decide karta hai ki hum $v$ ka kitna remove karte hain.
$w (t)$ — jo bacha rehta hai. Jaise $t$ ghoomta hai, $w (t)$ ki tip $v$ ke parallel ek seedhi line pe slide karti hai.

KYUN. Hum chahte hain ek aisi quantity jo guaranteed non-negative ho chahe kuch bhi ho. Length-squared bilkul aisi quantity hai (axiom 3, positive-definiteness). Ek poori sliding family banake, hum ek hi baar har value of $t$ ke liye ek inequality lete hain — ek idea se bahut saari free information milti hai.

PICTURE. Neela arrow $u$ wahi rehta hai. Peela arrow $t v$ dial ghoomate waqt barhta hai. Laal arrow $w (t)$ gap hai. Dekho yeh apne sabse chhote tak sihrta hai, phir dobara barhta hai.

Figure — Inner product spaces — dot product generalization

Step 2 — Iski length kabhi negative nahi ho sakti

KYA. Us bachi hui arrow ki squared length measure karo: $0 \leq ⟨ u - t v, u - t v ⟩ .$

Baayein taraf ka $0\le$ — poora point: length-squared kabhi zero se neeche nahi jaata.
Brackets ke andar — wahi same vector $w (t)$ dono slots mein, kyunki $∥ w ∥^{2} = ⟨ w, w ⟩$ .

KYUN. Yeh single line hamara engine hai. Positive-definiteness kehta hai ki har real arrow ki length $\geq 0$ hoti hai, toh har $t$ ke liye right-hand side zero pe ya usse upar rehti hai. Ab hum is promise ko jitna ho sake utna squeeze karte hain.

PICTURE. Step 1 ka laal gap arrow, aur uski length ek bar ke roop mein plot ki gayi jo $0$ par floor line se neeche kabhi nahi jaati.

Step 3 — $t$ mein ek parabola mein expand karo

KYA. Linearity aur symmetry (axioms 1 aur 2) use karke brackets kholein. Slot by slot karte hue:

= \langle\mathbf u,\mathbf u\rangle \;-\; t\langle\mathbf u,\mathbf v\rangle \;-\; t\langle\mathbf v,\mathbf u\rangle \;+\; t^2\langle\mathbf v,\mathbf v\rangle.$$ Do beech wale terms $-t\langle\mathbf u,\mathbf v\rangle$ aur $-t\langle\mathbf v,\mathbf u\rangle$ **equal** hain, kyunki symmetry (axiom 1) kehti hai $\langle\mathbf u,\mathbf v\rangle=\langle\mathbf v,\mathbf u\rangle$. Toh yeh ek single $-2t\langle\mathbf u,\mathbf v\rangle$ mein collapse ho jaate hain. $t$ ki powers ke hisaab se regroup karte hue: $$0 \;\le\; \underbrace{\langle\mathbf v,\mathbf v\rangle}_{a}\,t^2 \;\underbrace{-\,2\langle\mathbf u,\mathbf v\rangle}_{b}\,t \;+\; \underbrace{\langle\mathbf u,\mathbf u\rangle}_{c}.$$ $f(t)=a\,t^2 + b\,t + c$ ke coefficients padhte hue (dhyan dein ki $b$ ke upar brace mein **minus sign bhi shamil hai**): - $a=\langle\mathbf v,\mathbf v\rangle=\|\mathbf v\|^2$ — $t^2$ ka coefficient, "parabola kitni tezi se kholta hai" wala number. Yeh $\ge 0$ hai. - $b=-2\langle\mathbf u,\mathbf v\rangle$ — $t$ ka coefficient; minus sign **$b$ ka hissa hai**, taaki $b\,t$ upar wale term $-2\langle\mathbf u,\mathbf v\rangle\,t$ ko reproduce kare. Yahan hamaari target quantity $\langle\mathbf u,\mathbf v\rangle$ chupi hui hai. - $c=\langle\mathbf u,\mathbf u\rangle=\|\mathbf u\|^2$ — constant, $t=0$ par height. **KYUN.** Un teen coefficients ke saath, right side bilkul $f(t)=a\,t^2+b\,t+c$ hai — dial $t$ mein ek **parabola**. Humne "quadratic" tool choose kiya kyunki ek parabola ki shape teen numbers se poori tarah fix ho jaati hai, aur ek clean fact — the discriminant — padhta hai ki yeh kabhi zero se neeche jaata hai ya nahi. **PICTURE.** Teen annotated pieces ek upward-opening parabola $f(t)$ mein jaate hue. ![[deepdives/dd-maths-4.5.33-d2-s03.png]] --- ## Step 4 — "Kabhi zero se neeche nahi" parabola ko axis par baithne par majboor karta hai **KYA.** Kyunki **har** $t$ ke liye $f(t)\ge 0$ hai, yeh upward parabola kabhi horizontal axis ke neeche cross nahi kar sakta. Yeh axis ko *kiss* kar sakta hai (ek touching point) ya poori tarah uske upar float kar sakta hai — lekin yeh kabhi neeche dip nahi kar sakta. **KYUN.** Agar yeh neeche dip karta, toh koi dial value $t_0$ hoti jahan $f(t_0)<0$ hota, yaani ek leftover arrow jis ka length-squared negative ho. Yeh positive-definiteness ko contradict karta hai. Toh "neeche dipping" forbidden hai — geometry khud ise rokti hai. **PICTURE.** Teen candidate parabolas: green upar float karta hai (OK), yellow bas touch karta hai (OK, boundary), red neeche dip karta hai (FORBIDDEN — cross out). ![[deepdives/dd-maths-4.5.33-d2-s04.png]] --- ## Step 5 — "Kabhi cross nahi karta" ko discriminant test mein translate karo **KYA.** $a>0$ ke saath ek quadratic $at^2+bt+c$ ke liye, yeh kitni baar axis ko cross karta hai yeh ==discriminant== $\;b^2-4ac$ se decide hota hai: $$b^2-4ac \;>\;0 \Rightarrow \text{two crossings (dips below)},\qquad b^2-4ac\;\le\;0 \Rightarrow \text{never dips below}.$$ Kyunki Step 4 neeche dipping ko forbid karta hai: $$b^2 - 4ac \;\le\; 0.$$ **KYUN.** Discriminant "parabola axis se kahan milta hai?" ka algebra hai. Roots hain $t=\frac{-b\pm\sqrt{b^2-4ac}}{2a}$; positive $b^2-4ac$ do real roots deta hai (parabola unke beech se poke through karta hai), zero ek deta hai (ek kiss), negative koi nahi deta (float karta hai). Hume "koi poking through nahi" chahiye, isliye $\le 0$. **PICTURE.** Root formula jisme $\sqrt{b^2-4ac}$ highlight hai, aur ek number-line jo discriminant ka sign vs. crossing behaviour dikhata hai. ![[deepdives/dd-maths-4.5.33-d2-s05.png]] --- ## Step 6 — Apne actual $a,b,c$ substitute karo aur clean up karo **KYA.** $a=\|\mathbf v\|^2$, $b=-2\langle\mathbf u,\mathbf v\rangle$, $c=\|\mathbf u\|^2$ wapas daalo: $$\underbrace{\big(-2\langle\mathbf u,\mathbf v\rangle\big)^2}_{b^2} \;-\; \underbrace{4\,\|\mathbf v\|^2\,\|\mathbf u\|^2}_{4ac} \;\le\; 0.$$ Baayein taraf ka square $4\langle\mathbf u,\mathbf v\rangle^2$ hai. Puri cheez ko $4$ se divide karo: $$\langle\mathbf u,\mathbf v\rangle^2 \;\le\; \|\mathbf u\|^2\,\|\mathbf v\|^2.$$ Dono sides ka (non-negative) square root lo: $$\boxed{\;|\langle\mathbf u,\mathbf v\rangle| \;\le\; \|\mathbf u\|\,\|\mathbf v\|\;}\qquad\blacksquare$$ - $|\cdot|$ isliye aata hai kyunki $\sqrt{x^2}=|x|$ — reading negative ho sakti hai, uska size length product se bada nahi ho sakta. **KYUN.** Yahi woh statement hai ki $\cos\theta$ ka upar wala kabhi neeche wale se bada nahi ho sakta. Toh $\cos\theta\in[-1,1]$, aur "angle" $\theta$ finally honest sense deta hai. Isse dekhne ke liye $\|\mathbf u\|\|\mathbf v\|$ se divide karo (dono nonzero hain). **PICTURE.** Teen sihrte boxes ki chain: squared inequality → rooted inequality → $\cos\theta$ meter $[-1,1]$ ke andar pin hua. ![[deepdives/dd-maths-4.5.33-d2-s06.png]] --- ## Step 7 — Degenerate cases (inhe kabhi skip mat karo) **KYA.** Humne assume kiya tha ki hum dial ghuma sakte hain aur divide kar sakte hain. Kya ho agar koi vector zero ho? **Case A — $\mathbf v = \mathbf 0$.** Tab $a=\|\mathbf v\|^2=0$, toh $f(t)=-2\langle\mathbf u,\mathbf 0\rangle t + \|\mathbf u\|^2$ ek parabola nahi balki ek line hai. Lekin $\langle\mathbf u,\mathbf 0\rangle=0$ (linearity: $\langle\mathbf u,0\cdot\mathbf 0\rangle=0$), toh Cauchy–Schwarz ke dono sides $0\le 0$ padhte hain. **Sach.** Koi division needed nahi. **Case A′ — $\mathbf u = \mathbf 0$.** Symmetry se (axiom 1), $\langle\mathbf 0,\mathbf v\rangle=\langle\mathbf v,\mathbf 0\rangle=0$ aur $\|\mathbf 0\|=0$, toh phir se dono sides $0\le 0$ padhte hain. **Sach.** Aur generally, kyunki Cauchy–Schwarz $\mathbf u$ aur $\mathbf v$ mein poori tarah **symmetric** hai, koi bhi argument jo $\mathbf v=\mathbf 0$ handle karta hai woh $\mathbf u=\mathbf 0$ bhi handle karta hai bas names swap karke — toh koi bhi case uncovered nahi rehta. **Case B — equality, $|\langle\mathbf u,\mathbf v\rangle| = \|\mathbf u\|\|\mathbf v\|$.** Yeh *kiss* case hai (discriminant $=0$): parabola ek dial value $t_0$ par axis ko touch karta hai, matlab $\|\mathbf w(t_0)\|^2=0$, toh $\mathbf u - t_0\mathbf v = \mathbf 0$, yaani $\mathbf u = t_0\mathbf v$. **Equality exactly tab hoti hai jab do arrows parallel hoon** (ek doosre ka scalar multiple ho). Geometrically: laal gap arrow poori tarah kuch nahi tak squeeze ho sakta hai. **KYUN.** Ek theorem jise aap $\mathbf 0$ par ya boundary par apply nahi kar sakte woh holes wala theorem hai. Inhe cover karne ka matlab hai ki vectors ki har possible pair handle ki gayi hai — koi reader ek unshown scenario pe nahi phansega. **PICTURE.** Baaya panel: $\mathbf v=\mathbf 0$, family ek single arrow $\mathbf u$ mein collapse ho jaati hai. Daaya panel: $\mathbf u$ $\mathbf v$ ke parallel hai, laal gap gayab ho jaata hai, parabola axis ko kiss karta hai. ![[deepdives/dd-maths-4.5.33-d2-s07.png]] --- ## Ek-picture summary Upar sab kuch, compressed: dial $t$ ghoomao → leftover arrow $\mathbf w(t)$ → iski length-squared ek non-negative parabola hai → non-negative hona discriminant $\le 0$ force karta hai → wahi Cauchy–Schwarz hai → $\cos\theta$ $[-1,1]$ mein rehta hai, toh $\mathbf u$ aur $\mathbf v$ ke beech angle $\theta$ honest hai. ![[deepdives/dd-maths-4.5.33-d2-s08.png]] ```mermaid graph TD A["Slide u minus t times v"] --> B["Length squared is at least 0"] B --> C["Expand into parabola f of t"] C --> D["f never dips below 0"] D --> E["Discriminant at most 0"] E --> F["Cauchy Schwarz inequality"] F --> G["cos theta between minus 1 and 1"] ``` > [!recall]- Feynman: plain words mein poora walkthrough > Do arrows lo, $\mathbf u$ aur $\mathbf v$. Ab ek game khelo: $\mathbf u$ se $\mathbf v$ ki copies dheere dheere remove karo, $t$ naam ke ek dial se control karke. Jo bacha woh ek "gap" arrow hai. Ek gap arrow, kisi bhi arrow ki tarah, negative length nahi rakh sakta — yahi ek rock-solid rule hai. Agar aap gap ki length-squared likhein jab dial ghoomta hai, toh aapko ek smiley-shaped curve milti hai (ek parabola). Kyunki length kabhi negative nahi ho sakti, yeh smiley kabhi floor ke neeche dunk nahi kar sakti. Ek smiley jo kabhi floor ke neeche dunk nahi karti woh ek tidy algebra test follow karti hai jise "discriminant at most zero" kehte hain. Us test ko actual numbers ke saath cash out karo, aur nikalta hai: $\mathbf u$ aur $\mathbf v$ ki magic-ruler reading kabhi unki do lengths ke multiply se badi nahi ho sakti. Wahi single fact hai jo hume divide karne aur result ko cosine kehne deta hai — kyunki ab yeh $-1$ aur $+1$ ke beech trapped hai, bilkul wahan jahan ek real angle $\theta$ rehta hai. Do loose ends: agar $\mathbf v$ (ya $\mathbf u$) zero arrow hai toh poori cheez bas $0\le 0$ hai, phir bhi sach; aur smiley floor ko tab touch karti hai (equality) jab bilkul do arrows ek hi direction mein point karte hain, toh gap ko kuch nahi tak squeeze kiya ja sakta hai. > [!recall]- Quick self-check > Parabola ka discriminant $\le 0$ kyun hona chahiye naa ki $\ge 0$? ::: Kyunki $f(t)=\|\mathbf u-t\mathbf v\|^2\ge 0$ kabhi axis ke neeche dip nahi karta; ek positive discriminant do real roots deta aur unke beech ek dip deta, jo positive-definiteness ko contradict karta. > Cauchy–Schwarz equality kab banti hai? ::: Exactly tab jab kisi scalar ke liye $\mathbf u=t_0\mathbf v$ ho — vectors parallel hain (parabola axis ko kiss karta hai). > Derivation mein target quantity $\langle\mathbf u,\mathbf v\rangle$ kahan chupi hui hai? ::: Parabola ke middle coefficient $b=-2\langle\mathbf u,\mathbf v\rangle$ mein.