Cumulative distribution functions

---

What is a Cumulative Distribution Function?

Key properties that every CDF must satisfy:

1. Monotonically non-decreasing: If $x_1 < x_2$, then $F_X(x_1) \leq F_X(x_2)$
2. Limits: $\lim_{x \to -\infty} F_X(x) = 0$ and $\lim_{x \to \infty} F_X(x) = 1$
3. Right-continuous: $\lim_{h \to 0^+} F_X(x+h) = F_X(x)$

---

Deriving the CDF from First Principles

For Discrete Random Variables

Start with the PMF: Suppose we have a discrete random variable $X$ with probability mass function $p_X(x) = P(X = x)$.

Step 1 — What does $P(X \leq x)$ mean?
We want the total probability of all outcomes up to and including $x$. Since probabilities of disjoint events add:

$F_X(x) = P(X \leq x) = \sum_{x_i \leq x} p_X(x_i)$

Why this step? We're accumulating individual point probabilities. Each value $x_i$ that's $\leq x$ contributes its probability mass.

For Continuous Random Variables

Start with the PDF: For continuous $X$ with probability density function $f_X(x)$, probabilities are areas under the curve.

Step 1 — Probability as an integral:
Since $P(a < X \leq b) = \int_a^b f_X(x)\,dx$, the cumulative probability from $-\infty$ to $x$ is:

$F_X(x) = P(X \leq x) = \int_{-\infty}^{x} f_X(t)\,dt$

Why this step? We're integrating (accumulating) all the infinitesimal probability "slices" $f_X(t)\,dt$ from the far left up to point $x$.

Step 2 — Recovering the PDF from the CDF:
By the Fundamental Theorem of Calculus:

$f_X(x) = \frac{d}{dx} F_X(x)$

Why this relationship? The PDF is the rate of change of cumulative probability. The CDF is the antiderivative.

---

Computing Probabilities with the CDF

---

Connection to Quantiles and Percentiles

Why do we care in ML?

• Confidence intervals: "95% of predictions fall below this threshold"
• Random sampling: Generate samples via $X = F^{-1}(U)$ where $U \sim \text{Uniform}(0,1)$
• Outlier detection: Flag data points beyond the 99th percentile

---

Properties and Relationships

---

Applications in Machine Learning

1. Model Evaluation:

   • Computing prediction intervals: $P(y_{\text{pred}} \in [a, b])$
   • Calibration curves compare predicted CDF to empirical CDF

2. Data Preprocessing:

   • Quantile normalization: map data to uniform distribution via CDF
   • Outlier detection: flag points where $F_X(x) > 0.99$ or $< 0.01$

3. Generative Models:

   • Inverse transform sampling: Generate samples from any distribution using uniform random numbers
   • Copulas: Model joint distributions by combining marginal CDFs

4. Statistical Tests:

   • Kolmogorov-Smirnov test compares empirical CDF to theoretical CDF
   • p-values are tail probabilities: $p =1 - F_{\text{null}}(t_{\text{obs}})$

---

Recall Explain to a 12-Year-Old

Imagine you're collecting Pokémon cards, and you track how rare each card is.

The probability density is like a bar chart showing how common each card level is. A CDF is different — it answers: "If I pick a random card, what's the chance it's this rare or less?"

Picture walking through the cards from least rare to most rare, and keeping running count of "How many have I seen so far?" That running percentage is the CDF.

When you reach the end (the rarest card), your CDF hits 100% because you've counted all the cards.

Why is this useful? If someone asks "What's the chance I pull a card that's in the bottom 75%?", you just look at the CDF at the 75% mark. No complicated addition needed!

---

---

Connections

• Probability Distributions — CDF is the integral of the PDF
• Random Variables — CDF is defined for any random variable
• Expected Value and Variance — Can compute $E[X]$ using CDF
• Inverse Transform Sampling — Uses $F^{-1}$ to generate random samples
• Quantile Functions — The inverse of the CDF
• Empirical Distribution — Sample-based estimate of the CDF
• Kolmogorov-Smirnov Test — Compares CDFs for goodness-of-fit
• Copulas — Join marginal CDFs to model dependence
• Survival Analysis — Uses survival function $S(x) = 1 - F_X(x)$

---

#flashcards/ai-ml

What does the CDF $F_X(x)$ represent? :: The probability that random variable $X$ is less than or equal to $x$: $F_X(x) = P(X \leq x)$

Concept Map

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Dekho, CDF ka core idea bahut simple hai — ye ek "running total" hai probability ka. Socho tum shooting range pe ho aur poochte ho ki "kitne percent bullets 50 meters ke andar giri?" Toh yahan tum ek specific point tak ki saari probability jod rahe ho. Formally, $F_X(x) = P(X \leq x)$, matlab random variable $x$ ya usse chhoti value lene ka chance kitna hai. Left se right ko sweep karte jao aur probability accumulate hoti jati hai — isliye ye hamesha 0 se start hoke 1 par khatam hoti hai, aur beech mein kabhi ghatti nahi (monotonically non-decreasing).

Ab discrete aur continuous case ka fark samajh lo. Discrete case mein — jaise dice roll — tum bas alag-alag point probabilities ko jodte ho, isliye CDF ek "staircase" jaisi dikhti hai, har value pe ek step jump. Continuous case mein — jaise Uniform distribution — probability area hoti hai curve ke neeche, toh tum integrate karte ho PDF ko $-\infty$ se lekar $x$ tak. Aur ek khoobsurat baat: agar tum CDF ka derivative lo, toh wapas PDF mil jati hai (Fundamental Theorem of Calculus). Matlab PDF cumulative probability ki rate of change hai, aur CDF uska antiderivative — dono ek dusre se connected hain.

Ye cheez ML mein kyun important hai? Kyunki CDF se tum percentiles nikaal sakte ho — jaise "95% predictions is threshold ke neeche aati hain", ya confidence intervals banate ho. Random sampling generate karne mein bhi CDF ka inverse use hota hai (inverse transform sampling), aur probabilistic models samajhne ka backbone yahi hai. Toh jab bhi tumhe distribution ke baare mein "kitna tak" wale sawaal aaye, CDF hi tumhara tool hai. Isko achhe se pakad lo — aage har jagah kaam aayega.