Conditional probability — Hinglish notes

1.3.2 · AI-ML › Probability & Statistics

Overview

Conditional probability measure karta hai ki kisi event ki probability kaise badal jaati hai jab hume pata chalta hai ki doosra event ho chuka hai. Yeh Bayesian reasoning, naive Bayes classifiers, Hidden Markov Models, aur ML mein essentially saari probabilistic inference ki neenv hai.

[!intuition] Core Insight

Jab aapko information milti hai, tab aap possible outcomes ki universe ko shrink kar dete hain. Conditional probability poochhta hai: "Yeh jaante hue ki B hua, un B-outcomes mein se kitne fraction mein A bhi hai?"

Physical analogy: Aapke paas 100 emails hain. 60 spam hain, 40 legitimate hain. Spam mein se 50 mein word "free" hai. Aap ek email kholte hain aur "free" dekhte hain — aapka probability estimate ki yeh spam hai update hota hai kyunki aapne apni universe sirf "free"-wali emails tak restrict kar li hai.

Formula $P (A ∣ B)$ padhte hain "probability of A given B" — yeh wo probability hai ki A tab hota hai jab B already true ho.

[!definition] Formal Definition

Event $A$ ki conditional probability, given event $B$ (jahan $P (B) > 0$ ) hai:

$P (A ∣ B) = \frac{P ( A \cap B )}{P ( B )}$

Yeh formula kyun?

Numerator $P (A \cap B)$ : Woh probability ki A aur B dono hote hain (overlap).
Denominator $P (B)$ : B ko naye "total universe" ki tarah treat karke normalize karta hai (renormalization).
Result: B-outcomes ka woh fraction jo A ko bhi satisfy karta hai.

First principles se derivation (frequency interpretation):

$N$ trials run karo. Event $B$ $N_{B}$ trials mein hota hai.
Un $N_{B}$ trials mein event $A$ $N_{A B}$ trials mein hota hai.
Empirical conditional probability: $\frac{N _{A B}}{N _{B}}$ .
Probabilities mein express karo: $\frac{N _{A B} / N}{N _{B} / N} = \frac{P ( A \cap B )}{P ( B )}$ .

[!formula] Key Formulas & Derivations

1. Multiplication Rule (Chain Rule)

Definition ko rearrange karne par: $P (A \cap B) = P (A ∣ B) \cdot P (B)$

Kyun? Agar aap chahte hain ki A aur B dono ho:

Pehle, B hona chahiye: probability $P (B)$ .
Phir, B hone ke baad, A bhi hona chahiye: probability $P (A ∣ B)$ .
Multiply karo kyunki yeh sequential restrictions hain.

General chain rule (multiple events ke liye): $P (A_{1} \cap A_{2} \cap \dots \cap A_{n}) = P (A_{1}) \cdot P (A_{2} ∣ A_{1}) \cdot P (A_{3} ∣ A_{1} \cap A_{2}) \dots P (A_{n} ∣ A_{1} \cap \dots \cap A_{n - 1})$

Derivation: Multiplication rule ko recursively apply karo.

2. Law of Total Probability

Agar $B_{1}, B_{2}, \dots, B_{n}$ sample space ko partition karte hain (mutually exclusive, exhaustive): $P (A) = \sum_{i = 1}^{n} P (A ∣ B_{i}) \cdot P (B_{i})$

Kyun? Event A kisi bhi $B_{i}$ "pathway" se ho sakta hai. Har pathway $P (A ∣ B_{i}) \cdot P (B_{i})$ contribute karti hai (us pathway tak pahunchne ki probability times wahan A hone ki probability).

Derivation:

Kyunki $B_{i}$ space ko partition karte hain: $A = ⋃_{i = 1}^{n} (A \cap B_{i})$ (disjoint union).
Additivity se: $P (A) = \sum_{i = 1}^{n} P (A \cap B_{i})$ .
Multiplication rule se: $P (A \cap B_{i}) = P (A ∣ B_{i}) \cdot P (B_{i})$ .
Combine karo: $P (A) = \sum_{i = 1}^{n} P (A ∣ B_{i}) \cdot P (B_{i})$ .

3. Bayes' Theorem

$P (A ∣ B) = \frac{P ( B ∣ A ) \cdot P ( A )}{P ( B )}$

Multiplication rule se derivation:

Definition se: $P (A \cap B) = P (A ∣ B) \cdot P (B)$ .
Yeh bhi: $P (A \cap B) = P (B ∣ A) \cdot P (A)$ (symmetry).
Equate karo: $P (A ∣ B) \cdot P (B) = P (B ∣ A) \cdot P (A)$ .
$P (A ∣ B)$ ke liye solve karo: $P (A ∣ B) = \frac{P ( B ∣ A ) \cdot P ( A )}{P ( B )}$ .

Expanded form (denominator ke liye total probability use karte hue): $P (A ∣ B) = \frac{P ( B ∣ A ) \cdot P ( A )}{P ( B ∣ A ) \cdot P ( A ) + P ( B ∣\neg A ) \cdot P ( \neg A )}$

Yeh kyun matter karta hai: Aap aksar $P (B ∣ A)$ (likelihood) aur $P (A)$ (prior) jaante hain lekin $P (A ∣ B)$ (posterior) chahte hain. Bayes conditioning ko flip karta hai.

[!example] Worked Example 1: Medical Diagnosis

Setup: Ek disease 1% population ko affect karti hai. Ek test disease ko 95% time correctly identify karta hai (sensitivity) aur healthy patients ko 90% time correctly identify karta hai (specificity). Aap positive test karte hain. Kya probability hai ki aapko disease hai?

Events define karo:

$D$ : Disease hai. $P (D) = 0.01$ , $P (\neg D) = 0.99$ .
$T^{+}$ : Test positive aata hai.
$P (T^{+} ∣ D) = 0.95$ (sensitivity).
$P (T^{-} ∣\neg D) = 0.90$ → $P (T^{+} ∣\neg D) = 0.10$ (false positive rate).

Chahiye: $P (D ∣ T^{+})$ .

Step 1: Bayes' theorem apply karo. $P (D ∣ T^{+}) = \frac{P ( T ^{+} ∣ D ) \cdot P ( D )}{P ( T ^{+} )}$

Step 2: $P (T^{+})$ calculate karo total probability use karke ( $D$ aur $\neg D$ se partition karte hue). $P (T^{+}) = P (T^{+} ∣ D) \cdot P (D) + P (T^{+} ∣\neg D) \cdot P (\neg D)$ $= 0.95 \times 0.01 + 0.10 \times 0.99 = 0.0095 + 0.099 = 0.1085$

Yeh step kyun? Hume dono tareekon ko account karna hai jisse test positive ho sakta hai: sach mein bimaar aur falsely positive.

Step 3: Posterior compute karo. $P (D ∣ T^{+}) = \frac{0.95 \times 0.01}{0.1085} = \frac{0.0095}{0.1085} \approx 0.0875 \approx 8.75%$

Interpretation: Positive test ke baad bhi, disease hone ki chance sirf ~9% hai kyunki disease rare hai (low prior). Zyaadatar positives healthy majority ke false positives hain.

[!example] Worked Example 2: Spam Filter (Naive Bayes Foundation)

Setup:

60% emails spam hain, 40% ham hain.
Word "free" 80% spam mein aur 10% ham mein aata hai.
Ek email mein "free" hai. Kya yeh spam hai?

Events:

$S$ : Spam. $P (S) = 0.6$ , $P (H) = 0.4$ (ham).
$F$ : "free" contain karta hai.
$P (F ∣ S) = 0.8$ , $P (F ∣ H) = 0.1$ .

Chahiye: $P (S ∣ F)$ .

Step 1: Bayes apply karo. $P (S ∣ F) = \frac{P ( F ∣ S ) \cdot P ( S )}{P ( F )}$

Step 2: $P (F)$ calculate karo. $P (F) = P (F ∣ S) \cdot P (S) + P (F ∣ H) \cdot P (H)$ $= 0.8 \times 0.6 + 0.1 \times 0.4 = 0.48 + 0.04 = 0.52$

Yeh step kyun? "Free" spam ya ham dono mein aa sakta hai; inke prior probabilities se weight karo.

Step 3: Compute karo. $P (S ∣ F) = \frac{0.8 \times 0.6}{0.52} = \frac{0.48}{0.52} \approx 0.923 \approx 92.3%$

Interpretation: "Free" dekhna strongly spam suggest karta hai kyunki yeh spam mein bahut zyaada common hai (80% vs 10%) aur spam already majority class hai.

[!example] Worked Example 3: Independence Check

Setup: Do coin flips. $A$ = pehla heads hai, $B$ = doosra heads hai.

Independence check karo: Events independent hain agar $P (A ∣ B) = P (A)$ .

Calculate karo:

$P (A) = 0.5$ (fair coin).
$P (A \cap B) = 0.25$ (dono heads).
$P (B) = 0.5$ .
$P (A ∣ B) = \frac{P ( A \cap B )}{P ( B )} = \frac{0.25}{0.5} = 0.5 = P (A)$ .

Yeh step kyun? Agar B jaanna A ki probability nahi badlaata, toh woh independent hain. Doosra flip pehle ko affect nahi karta.

Alternative check: $P (A \cap B) = P (A) \cdot P (B)$ ? $0.25 = 0.5 \times 0.5 = 0.25 ✓$

Dono checks independence confirm karte hain.

[!mistake] Common Mistakes

Mistake 1: $P (A ∣ B)$ aur $P (B ∣ A)$ ko confuse karna

Galat soch: "Agar disease wale 90% log positive test karte hain, toh 90% positive tests matlab disease hai."

Kyun sahi lagta hai: Numbers similar lagte hain; hamaara dimaag dono directions ko confuse karta hai.

Fix: $P (A ∣ B) \neq = P (B ∣ A)$ generally. Yeh Bayes' theorem se related hain lekin priors se weighted hain. Medical testing mein, $P (test + ∣ disease)$ (sensitivity) $P (disease ∣ test +)$ (posterior) ke barabar NAHI hai.

Steel-man: Confusion isliye hota hai kyunki symmetric scenarios mein (jaise fair coins), $P (A ∣ B) = P (B ∣ A)$ . Lekin asymmetric priors ke saath (rare disease, common test), yeh wildly alag ho jaate hain.

Mistake 2: Base Rate ko ignore karna (Priors neglect karna)

Galat soch: "Test 95% accurate hai, toh positive result matlab 95% chance of disease."

Kyun sahi lagta hai: Hum test ki accuracy par focus karte hain aur population prevalence ignore karte hain.

Fix: Bayes' theorem mein hamesha prior $P (D)$ include karo. 1% prevalence disease par 95% sensitive test sirf ~9% posterior deta hai (Example 1).

Steel-man: Test accuracy sabse salient information hai jo present ki jaati hai, isliye hamaara attention wahan anchor ho jaata hai. Base rate abstract hai aur easily bhool jaata hai.

Mistake 3: Normalize karna bhool jaana

Galat soch: "Dono events ka B given 50% probability hai, toh equally likely hain."

Kyun sahi lagta hai: Hum likelihoods ko directly compare karte hain bina renormalize kiye.

Fix: $P (A ∣ B)$ mein $P (B)$ se divide karna zaroori hai, jo conditioning event ki total probability hai. Normalization ke bina, aap naye universe ka fraction nahi measure kar rahe.

Steel-man: Kai contexts mein (jaise hypotheses compare karna), unnormalized probabilities (likelihoods) ke ratios ranking ke liye kaafi hain, toh hum normalization skip karte hain. Lekin actual probabilities ke liye, normalize karna zaroori hai.

[!recall]- Ek 12-Saal Ke Bacche Ko Explain Karo

Socho tumhare paas ek bag hai jisme 10 marbles hain: 6 red, 4 blue. Normally, red pakarne ki chance 6/10 = 60% hai.

Ab tumhara dost jhank ke kehta hai, "Mujhe dikh raha hai yeh bag ke left half ka marble hai." Maano left half mein 4 red aur 1 blue hai. Ab tumhari universe sirf un 5 marbles tak shrink ho gayi. Red ki chance given left half se hai 4/5 = 80%. "Given left half" ne tumhari probability update kar di kyunki tumhare paas naya information hai jo possible outcomes ko change karta hai.

Conditional probability bas yahi hai: Jab kuch naya pata chale, toh us choti duniya mein chances recalculate karo jahan woh naya information sach hai.

[!mnemonic] Yaad Rakho

"B is the New Total": Jab aap $P (A ∣ B)$ dekhte hain, socho B ab 100% hai (poori universe). A us ka ek fraction hai. Overlap ko B ke total se divide karo.

"Bayes Flips the Pipe": $P (A ∣ B)$ vs $P (B ∣ A)$ — Bayes' theorem flipping tool hai. Prior → Likelihood → Posterior.

Properties of Conditional Probability

Range: $0 \leq P (A ∣ B) \leq 1$ (yeh abhi bhi ek probability hai).
Certainty: $P (B ∣ B) = 1$ (agar B hua, toh B definitely hua).
Additivity: $P (A \cup C ∣ B) = P (A ∣ B) + P (C ∣ B)$ agar $A \cap C = \emptyset$ .
Independence: Agar $A$ aur $B$ independent hain, toh $P (A ∣ B) = P (A)$ (B jaanna A ko nahi badlaata).

Connections

Bayes-Theorem — Conditioning ka direct application aur inversion.
Law-of-Total-Probability — Marginalization ke liye partitioning.
Independence — Special case jahan $P (A ∣ B) = P (A)$ .
Naive-Bayes-Classifier — $P (class ∣ features)$ use karta hai Bayes + independence assumptions ke through.
Hidden-Markov-Models — Transition probabilities conditional hain: $P (s_{t} ∣ s_{t - 1})$ .
Bayesian-Inference — Beliefs update karna: prior → likelihood → posterior.
Mutual-Information — Measure karta hai ki Y jaanna X ke baare mein uncertainty kitni kam karta hai.
Chain-Rule-of-Probability — Conditionals ke through joint distributions ko factorize karna.

Flashcards

Conditional probability

P (A ∣ B)

ka formula kya hai?

P (A ∣ B) = \frac{P ( A \cap B )}{P ( B )}

(dono ki probability divided by condition ki probability).

Conditional probability mein

P (B)

se kyun divide karte hain?

Renormalize karne ke liye — hum B ko naye total universe ki tarah treat kar rahe hain, isliye overlap ko B ki total probability se divide karte hain.

Conditional probability se derived multiplication rule bolo.

P (A \cap B) = P (A ∣ B) \cdot P (B)

ya equivalently

P (B ∣ A) \cdot P (A)

Law of total probability kya hai?

Agar

B_{1}, \dots, B_{n}

space ko partition karte hain:

P (A) = \sum_{i} P (A ∣ B_{i}) \cdot P (B_{i})

(saari pathways ka sum).

Bayes' theorem bolo.

P (A ∣ B) = \frac{P ( B ∣ A ) \cdot P ( A )}{P ( B )}

(prior aur likelihood use karke conditioning flip karta hai).

Bayes' theorem mein prior, likelihood, aur posterior kya hain?

Prior:

P (A)

, Likelihood:

P (B ∣ A)

, Posterior:

P (A ∣ B)

Agar

P (A ∣ B) = P (A)

ho, toh yeh kya imply karta hai?

A

aur

B

independent hain (B jaanna A ki probability nahi badlaata).

Ek disease ki 1% prevalence hai, test sensitivity 95%, specificity 90%. Aap positive test karte hain.

P (disease ∣ positive) = 95%

kyun nahi hai?

Kyunki aapko low base rate (1%) aur 99% healthy population ke false positives ko account karna hoga. Actual posterior ~9% hai.

P (A ∣ B)

aur

P (B ∣ A)

ko confuse karna" kya common mistake hai?

Yeh sochna ki A given B ki probability equals B given A ki probability, bina Bayes' theorem apply kiye aur priors consider kiye.

P (A ∣ B)

Bayes se compute karte time hamesha prior

P (A)

kyun include karna zaroori hai?

Prior yeh weight karta hai ki population mein A kitna common hai; ise ignore karne se base rate neglect hoti hai aur posteriors bilkul galat aa jaate hain.

Given

P (S) = 0.6

P (F ∣ S) = 0.8

P (F ∣ H) = 0.1

jahan

H

not-

S

hai,

P (F)

kya hai?

P (F) = 0.8 \times 0.6 + 0.1 \times 0.4 = 0.48 + 0.04 = 0.52

(law of total probability).

P (A \cap B) = P (A) \cdot P (B)

kya indicate karta hai?

A

aur

B

independent events hain.