2.4.7
Intuition Ek sentence mein ideaNaive Bayes kehta hai: jab aap class label jaante ho, features ek-dusre se baat karna band kar dete hain. Matlab, class diye jaane par features conditionally independent ho jaate hain. Ye "naive" shortcut ek impossible-to-estimate joint distribution ko ek simple chhoti 1-D distributions ki product mein badal deta hai.
Hum classify karna chahte hain: ek feature vector x = ( x 1 , x 2 , … , x n ) y
Masla hai likelihood P ( x ∣ y ) = P ( x 1 , x 2 , … , x n ∣ y )
Intuition Raw joint kyun hopeless haiMaano n k P ( x 1 , … , x n ∣ y ) k n − 1 har class ke liye . n = 30 factorize karna hoga.
P ( A , B ) = P ( A ∣ B ) P ( B ) P ( x 1 , … , x n ∣ y ) = P ( x 1 ∣ y ) P ( x 2 ∣ x 1 , y ) P ( x 3 ∣ x 1 , x 2 , y ) ⋯
Ye exact hai lekin useless — baad ke terms abhi bhi sab pehle wali features par depend karte hain. Ab hum woh leap lete hain.
Definition Naive Bayes (conditional independence) assumptionClass y P ( x i ∣ x 1 , … , x i − 1 , y ) = P ( x i ∣ y ) product mein collapse ho jaata hai:
P ( x 1 , … , x n ∣ y ) = i = 1 ∏ n P ( x i ∣ y )
Har P ( x i ∣ y ) k n n ⋅ k Yahi fayda hai.
Bayes' rule ko factorization ke saath combine karo:
P ( y ∣ x ) = P ( x ) P ( y ) ∏ i = 1 n P ( x i ∣ y )
Vocabulary features. P ( spam ) = 0.4 P ( ham ) = 0.6
word
P ( w ∣ spam ) P ( w ∣ ham )
"free"
0.7
0.1
"win"
0.5
0.05
Message mein dono "free" aur "win" hain. Har class ko score karo:
Spam: 0.4 × 0.7 × 0.5 = 0.14 Ye step kyun? Prior × Ham: 0.6 × 0.1 × 0.05 = 0.003
0.14 > 0.003 ⇒ spam . Normalize karke: P ( spam ∣ x ) = 0.143 0.14 ≈ 0.979 Normalize kyun? Raw scores ko probability mein badalne ke liye unhe unke sum = P ( x )
Feature = height, do classes. Continuous x i P ( x i ∣ y ) = 2 π σ y 2 1 exp ( − 2 σ y 2 ( x i − μ y ) 2 )
Maano class A: μ = 170 , σ = 10 μ = 185 , σ = 8 x = 178
log P ( x ∣ A ) = − 2 1 log ( 2 π ⋅ 100 ) − 2 ⋅ 100 ( 178 − 170 ) 2 = − 2 1 log ( 628.3 ) − 0.32 log P ( x ∣ B ) = − 2 1 log ( 2 π ⋅ 64 ) − 2 ⋅ 64 ( 178 − 185 ) 2 = − 2 1 log ( 402.1 ) − 0.383
Logs kyun? Numerical stability milti hai aur sums compare kar sakte hain. Class B ki chhoti variance + apne mean ke zyada paas hona yahan jeetta hai → predict karo B . (Narrow Gaussian apne mean ke paas hone ko reward karta hai.)
Common mistake "Features sach mein INDEPENDENT hain, hai na?"Kyun sahi lagta hai: math ko independence chahiye, toh students maante hain ki data mein bhi hogi.
Sachchi baat: real data mein features usually correlated hote hain (jaise "free" aur "money" spam mein saath aate hain). Naive Bayes aksar assumption mein galat hota hai phir bhi achha classify karta hai , kyunki usse sirf argmax sahi chahiye, probabilities nahi. Fix: NB probabilities ko poorly-calibrated maano lekin ranking ko usable.
Common mistake "Main ek zero-probability feature multiply kar sakta hoon."Kyun sahi lagta hai: spam mein kabhi na dekha gaya word P ( w ∣ spam ) = 0 Sachchi baat: ek zero poori product ko zero kar deta hai → class impossible ho jaati hai chahe 50 doosre words spam chilla rahe hon. Fix: Laplace (add-one) smoothing : P ( w ∣ y ) = count ( y ) + ∣ V ∣ count ( w , y ) + 1
Common mistake "Conditional independence = plain independence."Kyun sahi lagta hai: dono milte-julte lagte hain.
Sachchi baat: P ( x i ∣ y ) P ( x j ∣ y ) = P ( x i , x j ∣ y ) sirf ek class ke andar hold karta hai; features overall (marginally) strongly dependent ho sakte hain phir bhi y Fix: pehle hamesha y
Recall Feynman: ek 12-saal ke bachche ko samjhaoSocho fruit ko "apple" ya "orange" mein sort karna. Jab main tumhe bata deta hoon ki ye orange hai, tum uska rang aur uski bumpy skin alag-alag guess kar sakte ho — dono ko ek-dusre se consult nahi karna — orange hona already explain kar deta hai ki dono saath kyun hain. Naive Bayes pretend karta hai ki har clue aisa hi hai: jab tum answer-box jaante ho, har clue apni baat khud karta hai. Phir tum bas multiply karte ho ki har clue har box mein kitna fit baithta hai aur sabse bada choose karte ho.
"C.I.P." — C lass pata ho → clues I ndependent ho jaate hain → unki P robabilities multiply karo. Aur: "Naive = features gossip karte hain, lekin class unhe chup kara deti hai."
Recall Active recall — khud se quiz karo
Exactly kya independent hota hai, aur kis condition mein?
Hum argmax se P ( x )
Hum logs kyun lete hain?
Laplace smoothing kaunsi ek problem solve karta hai?
Naive Bayes assumption kya hai? Class label diya ho, toh saari features conditionally independent hain: P ( x i ∣ x < i , y ) = P ( x i ∣ y )
Assumption likelihood ko kaise simplify karti hai? Ye joint P ( x 1 , … , x n ∣ y ) ∏ i P ( x i ∣ y )
Naive Bayes decision rule batao. y ^ = arg max y P ( y ) ∏ i P ( x i ∣ y )
P ( x ) Ye classes mein constant hai, isliye ye decide nahi karta ki kaun si class maximal hai.
Naive Bayes mein logarithms kyun lete hain? Kai chhoti probabilities multiply karne se numerical underflow avoid karne ke liye; log monotonic hai isliye argmax nahi badlta.
Conditional aur marginal independence mein kya fark hai? Conditional independence y
Agar kisi feature ki zero conditional probability ho toh kya hoga? Poori product zero ho jaati hai, us class ko khatam kar deti hai chahe doosre evidence kuch bhi kahein.
Zero-probability features ko kya fix karta hai? Laplace/add-one smoothing: total + ∣ V ∣ count + 1
Galat assumption ke bawajood Naive Bayes kaam kyun karta hai? Classification ko sirf sahi argmax (ranking) chahiye, calibrated probabilities nahi.
Continuous x i 2 π σ y 2 1 exp ( − ( x i − μ y ) 2 /2 σ y 2 ) .
conditional independence given y
take logs to avoid underflow
Likelihood P of x given y
Product of 1-D P xi given y
