Joint, marginal, and conditional distributions

1.3.13 · AI-ML › Probability & Statistics

WHY — humein teeno ki zaroorat kyun hai?

WHAT hamare paas hai: do RVs, maano $X$ = "kya email spam hai?" aur $Y$ = "kya usme word free hai?"
WHY hum care karte hain: ek joint $p (X, Y)$ complete description hai — aap isse $X$ aur $Y$ ke baare mein koi bhi sawaal ka jawab nikal sakte ho. Lekin aksar hum sirf story ka ek hissa chahte hain:
- "Email kitni baar spam hoti hai, words ko ignore karke?" → marginal
- "Agar word free aaya, toh spam kitna likely hai?" → conditional
HOW yeh connect hote hain: marginal = ek dimension collapse karo; conditional = ek slice lo aur rescale karo.

Definitions from first principles

Conditional derive karna (memorize mat karo — build karo)

Joint ko factor karne ke do tareekon se humein turant Bayes' theorem milta hai: $p_{X ∣ Y} (x ∣ y) = \frac{p _{Y ∣ X} ( y ∣ x ) p _{X} ( x )}{p _{Y} ( y )} .$

Figure — Joint, marginal, and conditional distributions

Worked Example 1 — ek 2×2 joint table (discrete)

Maano $X$ = spam? ( $x \in {0, 1}$ ), $Y$ = "free" word hai? ( $y \in {0, 1}$ ). Joint pmf:

$p (x, y)$	$Y = 0$	$Y = 1$	Row sum = $p_{X}$
$X = 0$ (ham)	0.60	0.10	0.70
$X = 1$ (spam)	0.05	0.25	0.30
Col sum = $p_{Y}$	0.65	0.35	1.00

Marginals (edge sums — isliye hi inhe margins mein likha jaata hai):

$p_{X} (1) = 0.05 + 0.25 = 0.30$ . Yeh step kyun? Dono $Y$ values par sum karne se word info erase ho jaati hai.
$p_{Y} (1) = 0.10 + 0.25 = 0.35$ .

Conditional $P (spam ∣ "free")$ : $p_{X ∣ Y} (1 ∣ 1) = \frac{p _{X, Y} ( 1 , 1 )}{p _{Y} ( 1 )} = \frac{0.25}{0.35} \approx 0.714.$ Yeh step kyun? Hum "free"-column (total 0.35) tak restrict karte hain aur poochte hain ki kitna fraction spam hai.

Independent? Check karo $p (1, 1) = ? p_{X} (1) p_{Y} (1) = 0.30 \times 0.35 = 0.105 \neq = 0.25$ . Equal nahi ⇒ dependent. Sahi — word free spam probability ko 0.30 se 0.714 tak badhata hai.

Worked Example 2 — continuous joint

Maano $f_{X, Y} (x, y) = 4 x y$ unit square $0 \leq x \leq 1, 0 \leq y \leq 1$ par.

Step 1 — valid hai? Kyun? Ek pdf ko 1 tak integrate karna chahiye. $\int_{0}^{1} \int_{0}^{1} 4 x y d x d y = 4 (\frac{1}{2}) (\frac{1}{2}) = 1. ✓$

Step 2 — marginal of $X$ : $y$ ko integrate out karo. $f_{X} (x) = \int_{0}^{1} 4 x y d y = 4 x \cdot \frac{1}{2} = 2 x, 0 \leq x \leq 1.$

Step 3 — conditional of $Y$ given $X = x$ : $f_{Y ∣ X} (y ∣ x) = \frac{4 x y}{2 x} = 2 y .$ Yeh interesting kyun hai? Yeh $x$ par depend nahi karta ⇒ $X$ aur $Y$ yahan independent hain. Aur waqai $f_{X, Y} = 2 x \cdot 2 y = f_{X} f_{Y}$ . ✔

Common mistakes (Steel-manned)

Recall Feynman: ek 12-saal ke bachche ko samjhao

Socho ek bada grid of boxes hai jo bachche count karta hai (favorite fruit) aur (favorite sport) ke hisaab se. Joint har chote box mein count hai. Agar aap har row ke saare boxes ek pile mein dhakel do, toh aapko pata chalta hai kitne logo ko kaun sa fruit pasand hai, sport se regardless — woh hai marginal. Agar aap sirf football khelne wale bachche dekhein aur poochein "inme se kitno ko apples pasand hain?", toh pehle woh column uthao phir usse fractions mein baanto jo ek tak jodein — woh hai conditional. Marginal = ek direction squeeze karo; conditional = ek strip chuno aur usse 100% mein baanto.

Active recall

Joint

p_{X, Y}

se marginal

p_{X} (x)

kaise nikalte hain?

Joint ko doosre variable ki saari values par sum (ya integrate) karo:

p_{X} (x) = \sum_{y} p_{X, Y} (x, y)

Conditional pmf

p_{Y ∣ X} (y ∣ x)

define karo.

p_{Y ∣ X} (y ∣ x) = \frac{p _{X, Y} ( x , y )}{p _{X} ( x )}

jab

p_{X} (x) > 0

— joint slice re-normalized.

Conditioning karte waqt marginal se divide kyun karna padta hai?

Kyunki

X = x

par joint slice

p_{X} (x)

tak sum hoti hai, 1 tak nahi; divide karne se woh valid distribution ban jaati hai.

Joint ke liye product/chain rule batao.

p_{X, Y} (x, y) = p_{Y ∣ X} (y ∣ x) p_{X} (x) = p_{X ∣ Y} (x ∣ y) p_{Y} (y)

X

aur

Y

independent kab hote hain (distributions ke terms mein)?

Jab

p_{X, Y} (x, y) = p_{X} (x) p_{Y} (y)

sabhi

x, y

ke liye, equivalently

p_{Y ∣ X} = p_{Y}

Joint ki do factorizations se Bayes' theorem derive karo.

p_{X ∣ Y} p_{Y} = p_{Y ∣ X} p_{X}

equate karo ⇒

p_{X ∣ Y} (x ∣ y) = \frac{p _{Y ∣ X} ( y ∣ x ) p _{X} ( x )}{p _{Y} ( y )}

Spam table di gayi hai,

P (spam ∣ "free")

compute karo.

0.25/0.35 \approx 0.714

True/False: uncorrelated hona independent hona imply karta hai.

False — independence uncorrelated imply karta hai, lekin ulta nahi.

Connections

Bayes' theorem — joint ki do factorizations ka direct consequence.
Independence and conditional independence — factorization condition.
Covariance and correlation — full distribution ki jagah joint ke weaker summaries.
Naive Bayes classifier — label given features ki conditional independence assume karta hai.
Marginalization and the law of total probability — $p_{Y} (y) = \sum_{x} p_{Y ∣ X} (y ∣ x) p_{X} (x)$ .
Probabilistic graphical models — encode karta hai ki kaun se conditionals joint ko factor karte hain.
Expectation and variance — inhi distributions ke against compute hote hain.