Vector norms (L1, L2, L-infinity, Lp)

1.1.4 · AI-ML › Linear Algebra Essentials

Norm kya hai? (first principles)

Yeh teen axioms kyun? Yeh woh minimum hain jo "size" ko sensibly behave karaane ke liye chahiye:

Axiom 1 kehta hai sirf zero vector ki zero length hoti hai.
Axiom 2 kehta hai ek vector ko double karne se uski length double hoti hai (scaling linear hai).
Axiom 3 kehta hai ek detour kabhi seedhe raaste se chota nahi hota.

Lp family

Ise kaise padhen: har coordinate ki magnitude ko power $p$ pe raise karo, sum karo, phir $p$ -th root lo. Root "undo" karta hai power ko taaki homogeneity hold kare.

L∞ limit ki derivation (scratch se). Maano $M = max_{i} ∣ x_{i} ∣$ . Ise factor out karo: $∥ x ∥_{p} = (\sum_{i} ∣ x_{i} ∣^{p})^{1/ p} = M (\sum_{i} (\frac{∣ x _{i} ∣}{M})^{p})^{1/ p} .$ Har ratio $\frac{∣ x _{i} ∣}{M} \leq 1$ hota hai. Maano $k$ coordinates max ke equal hain (ratio $= 1$ ), baaki $< 1$ hain. Jab $p \to \infty$ , ratio $< 1$ waale terms vanish ho jaate hain, toh sum $\to k$ ho jaata hai. Phir $∥ x ∥_{p} \to M \cdot k^{1/ p} \to M \cdot 1 = M,$ kyunki kisi bhi fixed $k$ ke liye $k^{1/ p} \to 1$ hota hai. Is tarah $∥ x ∥_{\infty} = max_{i} ∣ x_{i} ∣$ . ∎

Figure — Vector norms (L1, L2, L-infinity, Lp)

Worked examples

Poore examples mein $x = (3, - 4)$ hai.

Common mistakes (Steel-manned)

Feynman check

Recall Ek 12-saal ke bachche ko explain karo

Imagine karo tum ghar pe ho aur tumhara dost 3 blocks east aur 4 blocks north rehta hai.

Agar tum galiyoon pe chalte ho toh 3 + 4 = 7 blocks jaoge. Yeh L1 distance hai.
Agar tum seedhi line mein ud sako toh yeh 5 blocks hai (diagonal). Yeh L2 hai.
Agar tumhe sirf sabse lamba single stretch chahiye, toh woh 4 hai (north wala hissa). Yeh L∞ hai. Same do dost, teen jawaab — kyunki "kitni door" depend karta hai tum kaise move kar sakte ho.

Flashcards

Norm ko define karne waale teen axioms kya hain?

Non-negativity (

∥ x ∥ = 0 ⟺ x = 0

), absolute homogeneity (

∥ α x ∥ = ∣ α ∣∥ x ∥

), triangle inequality (

∥ x + y ∥ \leq ∥ x ∥ + ∥ y ∥

Lp norm ka general formula?

∥ x ∥_{p} = (\sum_{i} ∣ x_{i} ∣^{p})^{1/ p}

for

p \geq 1

(3, - 4)

ka L1 norm?

∣3∣ + ∣ - 4∣ = 7

(3, - 4)

ka L2 norm?

9 + 16 = 5

(3, - 4)

ka L∞ norm?

max (3, 4) = 4

L1 sparsity kyun induce karta hai lekin L2 nahi?

L1 ka unit ball ek diamond hai jiske corners axes pe hain, toh optimal solutions axes pe land karte hain → coordinates exactly 0 ho jaate hain; L2 ka ball smooth hai (ek circle) jiske koi corners nahi.

Same vector ke norms ke beech general inequality?

∥ x ∥_{\infty} \leq ∥ x ∥_{2} \leq ∥ x ∥_{1}

; bada

p

chota-ya-equal norm deta hai.

L0 true norm kyun nahi hai?

Yeh nonzeros count karta hai lekin homogeneity violate karta hai:

∥2 x ∥_{0} = ∥ x ∥_{0} \neq = 2∥ x ∥_{0}

∥ x ∥_{\infty} = max_{i} ∣ x_{i} ∣

derive karo.

M = max ∣ x_{i} ∣

factor karo:

∥ x ∥_{p} = M (\sum (∣ x_{i} ∣/ M)^{p})^{1/ p}

; ratios

< 1

vanish ho jaate hain jab

p \to \infty

M \cdot k^{1/ p} \to M

bachta hai.

Dot product se kaunsa norm relate karta hai?

L2, kyunki

∥ x ∥_{2} = x^{⊤} x

Lp kis

p

ke liye valid norm hai?

p \geq 1

(usse neeche triangle inequality fail ho jaati hai).

Connections

Dot product and inner products — L2 is $x^{⊤} x$ .
Regularization L1 vs L2 (Lasso vs Ridge) — norm geometry ka direct application.
Distance metrics and k-NN — Manhattan vs Euclidean choice.
Convexity and unit balls — kyun $p \geq 1$ ek convex ball guarantee karta hai.
Gradient clipping — L2 norm threshold se clip karna.
Matrix norms — vectors se matrices tak generalization.