2.3.16
Instance-based methods (jaise k-Nearest Neighbors ) mein koi "training" nahi hoti — ye bas data yaad rakhte hain aur prediction ke waqt poochte hain "kaun se stored points sabse paas hain?" Lekin "paas" ka koi matlab nahi jab tak hum distance DEFINE na karein. Ek distance metric woh ruler hai jisse hum measure karte hain ki do feature vectors kitne (un)similar hain.
YE KYUN ZAROORI HAI: ruler badlo → neighbors badlenge → prediction badlegi. Metric hi model ka inductive bias hai.HUM KYA COMPARE KARTE HAIN: do vectors a , b ∈ R n YE KAISE ALAG HAIN: Euclidean = seedhi line, Manhattan = grid-walk, Cosine = angle (magnitude ignore karta hai).
Ek function d ( a , b ) true metric hai agar ye satisfy karta hai:
Non-negativity: d ( a , b ) ≥ 0 Identity: d ( a , b ) = 0 ⟺ a = b Symmetry: d ( a , b ) = d ( b , a ) Triangle inequality: d ( a , c ) ≤ d ( a , b ) + d ( b , c )
Euclidean aur Manhattan chaaron satisfy karte hain. Cosine distance (1 − cos θ dissimilarity hai, strict metric nahi — lekin hum ise continuously use karte hain.
Intuition Ek parent, kai baccheTeen formulas yaad karne ki jagah, inhe ek hi Minkowski distance of order p d p ( a , b ) = ( ∑ i = 1 n ∣ a i − b i ∣ p ) 1/ p p badi coordinate differences ko kitni sakht saza di jaaye .
p = ∞ p → ∞ d ∞ ( a , b ) = max i ∣ a i − b i ∣ Chebyshev distance hai (chessboard par king ka move).
Kabhi kabhi magnitude noise hoti hai . Document A = "cat cat cat" aur document B = "cat cat cat cat cat cat" dono same direction mein point karte hain (dono "all cat" hain) lekin unki lengths bahut alag hain. Euclidean kehta hai ye door hain; cosine kehta hai ye identical hain. Toh cosine orientation measure karta hai, scale ko ignore karta hai.
Worked example Example 1 — teeno 2D points para = ( 1 , 2 ) b = ( 4 , 6 )
Manhattan: ∣1 − 4∣ + ∣2 − 6∣ = 3 + 4 = 7 Yeh step kyun? L1 sirf axis-wise moves sum karta hai — koi squaring nahi.
Euclidean: ( 1 − 4 ) 2 + ( 2 − 6 ) 2 = 9 + 16 = 25 = 5 Yeh step kyun? Squaring do legs ko Pythagorean hypotenuse mein badal deta hai; diagonal (5) grid walk (7) se chhota hota hai — hamesha.
Cosine: 1 + 4 16 + 36 1 ⋅ 4 + 2 ⋅ 6 = 5 52 4 + 12 = 260 16 ≈ 0.992 = 1 − 0.992 = 0.008 Yeh step kyun? Dono points origin se almost same ray par hain, toh unka angle tiny hai chahe Euclidean 5 bata raha ho. Alag sawaal, alag jawab!
Worked example Example 2 — scaling Euclidean/Manhattan ko kyun todti haiFeatures: income (0–100000) aur age (0–100). Points a = ( 50000 , 30 ) b = ( 50100 , 60 ) = 10 0 2 + 3 0 2 = 10000 + 900 = 10900 ≈ 104.4 Yeh kyun important hai: 100 dominate karta hai 30 Fix: features standardize karo (z = σ x − μ
Worked example Example 3 — cosine magnitude ignore karta haia = ( 1 , 1 ) b = ( 10 , 10 ) = 81 + 81 = 162 ≈ 12.7 = 2 200 1 ⋅ 10 + 1 ⋅ 10 = 400 20 = 20 20 = 1 0 Yeh step kyun? Same direction ( 4 5 ∘ )
Common mistake "Higher cosine similarity matlab smaller distance, toh main directly similarity se sort kar sakta hoon."Kyun sahi lagta hai: zyada similar = zyada paas, intuitively. Trap: kNN code mein jo distance expect karta hai (chhota = zyada paas), similarity pass karna tumhare neighbors ko flip kar deta hai aur worst matches chunata hai. Fix: hamesha distance = 1 − similarity
Common mistake "Euclidean hamesha best default hai."Kyun sahi lagta hai: yeh wahi "real" geometric distance hai jo hum school mein seekhe. Trap: high dimensions mein sab Euclidean distances concentrate ho jaate hain (sab equidistant lagta hai — curse of dimensionality ), aur unscaled features se yeh kharab ho jaata hai. Fix: features scale karo; high-d mein L1 consider karo (zyada robust) ya sparse/text data ke liye cosine.
Common mistake "Main square root lena bhool gaya, lekin ranking same hai, toh kya fark padta hai?"Kyun sahi lagta hai: x ↦ x order preserve hota hai. Steel-man ranking ke liye actually correct hai — isliye libraries speed ke liye squared distance use karti hain. Lekin: jis moment tum actual distance value report karo, distances average karo, ya ise RBF kernel e − γ d 2 e − γ d kab legal hai.
Common mistake "Cosine ek proper metric hai."Kyun sahi lagta hai: yeh non-negative aur symmetric hai. Trap: 1 − cos θ Fix: agar true metric chahiye toh angular distance π a r c c o s ( c o s θ )
What single parent formula generates Euclidean, Manhattan, and Chebyshev? Minkowski distance d p = ( ∑ i ∣ a i − b i ∣ p ) 1/ p p = 2 , 1 , ∞
Write the Euclidean distance formula. ∑ i ( a i − b i ) 2 — Pythagorean theorem se.
Write the Manhattan distance formula. ∑ i ∣ a i − b i ∣
Derive cosine similarity from the dot product. a ⋅ b = ∥ a ∥∥ b ∥ cos θ ⇒ cos θ = ∥ a ∥∥ b ∥ a ⋅ b
How do you turn cosine similarity into a distance? distance = 1 − cos θ 0 2
Which metric ignores vector magnitude and why is that useful? Cosine — sirf angle/orientation measure karta hai; text/TF-IDF ke liye ideal hai jahan doc length matter nahi karni chahiye.
Which metric axiom does cosine distance violate? Triangle inequality (isliye yeh dissimilarity hai, true metric nahi).
Why must you scale features before Euclidean/Manhattan kNN? Bade-unit features sum mein dominate karte hain, "closeness" distort karte hain; standardize karo z = ( x − μ ) / σ
For the same two points, which is larger: Euclidean or Manhattan? Manhattan ≥
What does Minkowski p → ∞ Chebyshev distance max i ∣ a i − b i ∣
Recall Feynman: 12-saal ke bacche ko samjhaoEk map par do ghar socho. Euclidean distance woh hai jitna ek chidiya seedha unke beech uda kar jaaye. Manhattan distance woh hai jitna ek taxi sadkon par chalti hai — woh ud nahi sakti, toh pehle seedha phir upar jaati hai, jo zyada hota hai. Cosine bilkul alag hai: "kitna door" bhool jao, poochho "kya woh sheher ke center se same direction mein hain?" Do dost dono east mein chal rahe hain toh "same taraf" ja rahe hain chahe ek 1 km door ho aur doosra 10 km. Computer decide karta hai ki kaun se ghar "neighbors" hain in rulers mein se ek use karke — aur galat ruler chuno toh galat dost milenge.
Mnemonic Teeno yaad rakho"Manhattan Managers grid par chalte hain, Euclid seedha kheenchta hai, Cosine angle ki parwah karta hai length ki nahi."
Minkowski p 1 = 1 street (L1) , 2 = 2-leg triangle (L2) , ∞ = sirf sabse bada gap matter karta hai .
Minkowski Distance order p
Euclidean L2 straight-line
Cosine Distance angle-based
