Probability mass and density functions — Hinglish notes

1.3.6 · AI-ML › Probability & Statistics

Intuition Outcomes aur Probabilities ke Beech ka Bridge

Socho tum ek AI system design kar rahe ho jo ghar ke prices predict karta hai. Kuch predictions exactly $300k, $301k, etc. honge (discrete), lekin reality mein prices continuous hoti hain—$300,245.73. To hum outcomes ko probabilities kaise assign karein?

PMF (Probability Mass Function) discrete random variables ke liye hai—yeh tumhe har outcome ki exact probability batata hai (jaise ek vending machine: button A tumhe chips deta hai probability 0.3 ke saath).

PDF (Probability Density Function) continuous random variables ke liye hai—yeh direct probabilities nahi deta (ek single point ki probability 0 hoti hai!), balki intervals ke across relative likelihood describe karta hai (jaise paani mein alag-alag depths par sugar ki concentration measure karna).

ML MEIN YEH KYU IMPORTANT HAI: Har prediction model ek distribution output karta hai. Classification → classes par PMF. Regression → possible values par PDF. Inhein samajhna hi woh raasta hai jisse "mera model numbers output karta hai" se "mera model uncertainty quantify karta hai" tak pahuncha ja sake.

Probability Mass Function (PMF)

First Principles se Derivation

SHURU: Random variable kya hai? Ek function jo outcomes ko numbers se map karta hai.

STEP 1: Sample space $Ω$ mein discrete outcomes $ω_{1}, ω_{2}, \dots$ ke liye, har ek ki probability $P (ω_{i})$ hoti hai.

STEP 2: Random variable $X$ , $ω_{i}$ ko $x_{i}$ se map karta hai. Multiple outcomes ek hi $x$ se map ho sakti hain.

STEP 3: Probability ki $X$ value $x$ le, woh sabhi outcomes ki probabilities ka sum hai jo $x$ se map hoti hain: $p_{X} (x) = P (X = x) = \sum_{ω : X (ω) = x} P (ω)$

YEH FORM KYU? Axiom of countable additivity ki wajah se: agar events disjoint hain (alag-alag outcomes), to unki probabilities add ho jaati hain.

Property 1 proof: Kyunki har outcome ke liye $0 \leq P (ω) \leq 1$ hai, aur hum non-negative terms add kar rahe hain, isliye $0 \leq p_{X} (x) \leq 1$ .

Property 2 proof: $\sum_{x} p_{X} (x) = \sum_{x} \sum_{ω : X (ω) = x} P (ω) = \sum_{ω \in Ω} P (ω) = 1$ Hum sirf unhi outcome probabilities ko regroup kar rahe hain.

Figure — Probability mass and density functions

Probability Density Function (PDF)

Points ki Probability Zero Kyu Hoti Hai

Intuition: Socho kisi ki height measure karna. Probability kya hai ki unki height EXACTLY 170.000... cm ho (infinite precision)? Zero. Lekin probability ki unki height 169.5 aur 170.5 cm ke beech ho? Non-zero.

Mathematical reason: $P (X = c) = \int_{c}^{c} f_{X} (x) d x = 0$ $c$ se $c$ tak ke integral ki width zero hoti hai.

Consequence: Continuous variables ke liye, $P (a \leq X \leq b) = P (a < X < b)$ —boundary points matter nahi karte.

Discrete se Continuous tak ki Derivation

SHURU: Ek discrete variable socho jiske outcomes $x_{i} = i Δ x$ par hain, chote $Δ x$ ke liye.

STEP 1: PMF deta hai $p_{X} (x i) = P (X = x_{i})$ .

STEP 2: Jaise $Δ x \to 0$ , hum kisi bhi interval $[a, b]$ mein zyada se zyada outcomes pack karte hain.

STEP 3: Interval $[x, x + Δ x]$ ki probability approach karti hai $p_{X} (x) \approx f_{X} (x) \cdot Δ x$ .

KYU? $f_{X} (x)$ ko "probability per unit length" samjho. Length $Δ x$ se multiply karo to probability milo.

STEP 4: Interval $[a, b]$ par sum karo: $P (a \leq X \leq b) = lim_{Δ x \to 0} \sum_{x_{i} \in [a, b]} f_{X} (x_{i}) Δ x = \int_{a}^{b} f_{X} (x) d x$

Yeh Riemann integral ki definition hai.

Worked example Example 4: Exponential Distribution (Waiting Times)

Setup: Time $X$ agla customer store par aane tak. Memoryless property: past future ko affect nahi karta. Heuristic derivation:

Chote time $Δ t$ mein, arrival ki probability $λ Δ t$ hai (constant rate $λ$ )
$[0, x]$ mein KISI arrival ki probability nahi hone mein exponential decay involved hai
Isse CDF milta hai: $F_{X} (x) = 1 - e^{- λ x}$ $x \geq 0$ ke liye Differentiation se PDF: $f_{X} (x) = \frac{d}{d x} (1 - e^{- λ x}) = λ e^{- λ x} for x \geq 0$ Yeh form kyu?
$λ$ rate control karta hai (zyada $λ$ = chota wait, tez decay)
Exponential decay: zyaatar waits chote hote hain, lambe waits rare hote hain Normalization verify karo:

\int_{0}^{\infty} λ e^{- λ x} d x = λ [- \frac{1}{λ} e^{- λ x}]_{0}^{\infty} = λ \cdot \frac{1}{λ} = 1

$f_{X} (x) = \frac{1}{σ 2 π} exp (- \frac{( x - μ ) ^{2}}{2 σ ^{2}})$

Parameters:

$μ$ = mean (bell curve ka center)
$σ^{2}$ = variance (spread; bada $σ$ = chaudi curve)

Yeh specific form kyu?

Symmetry: $(x - μ)^{2}$ dono directions mein deviations ko equally treat karta hai
Exponential decay: Extreme values exponentially unlikely ho jaati hain
Normalization constant $\frac{1}{σ 2 π}$ : ensure karta hai ki $\int_{- \infty}^{\infty} f_{X} (x) d x = 1$ Constant ki derivation (sketch):

Chahiye $\int_{- \infty}^{\infty} e^{- x^{2} / (2 σ^{2})} d x = σ 2 π$
Polar coordinates use karo, $e^{- r^{2}}$ ke 2D integral mein convert karo, $π$ factors milte hain ML uses: Neural network initialization, noise models, Gaussian processes, variational inference.

PMF vs PDF: Critical Differences

Property	PMF (Discrete)	PDF (Continuous)
Values	$p_{X} (x) = P (X = x)$	$f_{X} (x)$ probability NAHI hai
Range	$0 \leq p_{X} (x) \leq 1$	$f_{X} (x) \geq 0$ , 1 se zyada ho sakta hai!
Total	$\sum_{x} p_{X} (x) = 1$	$\int_{- \infty}^{\infty} f_{X} (x) d x = 1$
Point prob	$p_{X} (x)$ answer hai	$P (X = x) = 0$
Interval prob	$\sum_{x \in [a, b]} p_{X} (x)$	$\int_{a}^{b} f_{X} (x) d x$

Recall Feynman Explanation (12 saal ke bacche ko samjhao)

Socho tumhare paas ek bada jar hai marbles ka, har ek par ek number likha hai. Agar tum marbles GIN sako (1, 2, 3, ..), to yeh discrete hai. Main tumhe exactly bata sakta hoon: "Marble #7 pick karne ki probability 20 mein se 1 hai." Yeh PMF hai—yeh har marble ke liye exact chance deta hai.

Ab socho jar mein reth bhari hai, aur tum dekh rahe ho usme kitni gold dust mixed hai. Tum ek single grain nahi nikal sakte aur uski probability nahi pooch sakte—infinite grains hain! Lekin main bata sakta hoon: "Jar ke IS REGION mein bahut gold hai; US region mein, bahut kam." Yahi PDF karta hai—yeh batata hai "gold" (probability) kahaan concentrated hai.

Agar main high-density region se ek chammach nikaaloon, shayad mujhe gold mile. Low-density region se nikaaloon, shayad nahi mile. Lekin ek single grain? Zero gold (zero probability). Density tumhe batati hai: gold dhundhna ho to IS neighborhood mein dhundho.

Machine learning mein, jab bhi tumhara model kehta hai "mujhe lagta hai jawab yahan ke aas paas hai," woh tumhe ek density function de raha hai!

Connections

1.3.01-Random-variables-and-distributions — PMF/PDF ek random variable ki distribution ko formalize karte hain
1.3.07-Cumulative-distribution-functions — CDF discrete aur continuous cases ko unify karta hai; $F_{X}^{'} = f_{X}$
1.3.08-Expected-value-and-variance — Compute hota hai $E [X] = \sum x \cdot p_{X} (x)$ ya $\int x \cdot f_{X} (x) d x$ se
1.3.09-Common-probability-distributions — Specific PMFs (Binomial, Poisson) aur PDFs (Gaussian, Exponential)
1.4.02-Maximum-likelihood-estimation — Parameters dhundhna jo $\prod p_{X} (x_{i})$ ya $\prod f_{X} (x_{i})$ maximize kare
2.1.05-Softmax-activation — Softmax logits ko classes par ek PMF mein convert karta hai
3.2.04-Gaussian-processes — Functions ko ek distribution se draws ke roop mein model karta hai jisme function space par PDF hoti hai

#flashcards/ai-ml

PMF aur PDF ke beech ka key difference kya hai? :: PMF discrete outcomes ke liye exact probabilities deta hai: $p_{X} (x) = P (X = x)$ . PDF continuous variables ke liye probability density deta hai—probability value nahi balki integral (area) hai: $P (a \leq X \leq b) = \int_{a}^{b} f_{X} (x) d x$ .

Koi bhi PMF ki do properties kya honi chahiye?

(1)

0 \leq p_{X} (x) \leq 1

sabhi

x

ke liye, aur (2)

\sum_{all x} p_{X} (x) = 1

Koi bhi PDF ki do properties kya honi chahiye?

(1)

f_{X} (x) \geq 0

sabhi

x

ke liye, aur (2)

\int_{- \infty}^{\infty} f_{X} (x) d x = 1

Ek continuous random variable ki kisi specific value

c

ke liye

P (X = c) = 0

kyu hota hai?

Kyunki probability PDF curve ke neeche ka area hai, aur ek single point ki width zero hoti hai, isliye area bhi zero:

P (X = c) = \int_{c}^{c} f_{X} (x) d x = 0

PDF aur CDF mein kya relationship hai?

PDF, CDF ka derivative hai:

f_{X} (x) = \frac{d}{d x} F_{X} (x)

. CDF, PDF ka integral hai:

F_{X} (x) = \int_{- \infty}^{x} f_{X} (t) d t

Kya ek PDF value

f_{X} (x)

1 se zyada ho sakti hai?

Haan! PDF ek density hai (probability per unit), probability nahi. Example:

[0, 0.5]

par uniform ka

f_{X} (x) = 2

hota hai.

Success probability

p

wale Bernoulli random variable ka PMF kya hota hai?

p_{X} (1) = p

aur

p_{X} (0) = 1 - p

, ya compactly:

p_{X} (x) = p^{x} (1 - p)^{1 - x}

x \in {0, 1}

ke liye.

[a, b]

par uniform distribution ka PDF kya hai?

f_{X} (x) = \frac{1}{b - a}

x \in [a, b]

ke liye, aur

0

otherwise. Constant density ensure karti hai ki total probability 1 ho.

Rate

λ

wale exponential distribution ka PDF kya hai?

f_{X} (x) = λ e^{- λ x}

x \geq 0

ke liye, aur

0

otherwise. Yeh memoryless processes mein waiting times model karta hai.

Mean

μ

aur variance

σ^{2}

wale Gaussian distribution ka PDF kya hai?

f_{X} (x) = \frac{1}{σ 2 π} exp (- \frac{( x - μ ) ^{2}}{2 σ ^{2}})

sabhi

x \in R

ke liye.

Ek discrete random variable ke liye

P (a \leq X \leq b)

kaise compute karte hain?

P (a \leq X \leq b) = \sum_{x = a}^{b} p_{X} (x)

(interval par PMF sum karo).

Ek continuous random variable ke liye

P (a \leq X \leq b)

kaise compute karte hain?

P (a \leq X \leq b) = \int_{a}^{b} f_{X} (x) d x

(interval par PDF integrate karo).

ML mein classification ke liye PMF aur regression ke liye PDF kyu use karte hain?

Classification discrete class labels output karta hai (finite outcomes) → classes par PMF. Regression continuous values output karta hai (infinite possibilities) → real line par PDF.

Agar do dice kheele jaayein aur

X

unka sum ho, to

p_{X} (7)

kya hai?

Sum 7 dene wale 6 outcomes hain:

(1, 6), (2, 5), (3, 4), (4, 3), (5, 2), (6, 1)

36 total equally likely outcomes mein se. To

p_{X} (7) = 6/36 = 1/6

Exponential distribution ki "memoryless property" ka kya matlab hai?

P (X > s + t ∣ X > s) = P (X > t)

: additional

t

time wait karne ki probability is baat par depend nahi karti ki tum pehle se kitna wait kar chuke ho. Past matter nahi karta.