Intuition The big pictureA discrete random variable lands on separate dots (X = 1 , 2 , 3 , … X=1,2,3,\dots X = 1 , 2 , 3 , … continuous random variable can take any value in a range — like a person's exact height, 172.4839 … 172.4839\ldots 172.4839 … infinitely many possible exact values, so the probability of hitting any single exact value is zero . We can only talk about probability over an interval . The tool that measures "how densely packed" the probability is around a point is the probability density function (PDF) .
Intuition Mass vs. density analogyThink of a 1-metre metal rod with total mass 1 kg. Ask: "what is the mass at the exact point x = 0.5 x=0.5 x = 0.5 zero — a single point has no width. But the rod has a mass density ρ ( x ) \rho(x) ρ ( x ) ∫ ρ d x \int \rho\,dx ∫ ρ d x f ( x ) f(x) f ( x ) f f f
So we never read probability off the height of f ( x ) f(x) f ( x ) density ; area is probability.
WHAT: A function f ( x ) f(x) f ( x )
WHY it must integrate to 1: some value must occur, so the total area = total probability = 1.
WHAT: The cumulative distribution function F ( x ) F(x) F ( x ) x x x
WHY define it: it converts "area" questions into simple subtraction and is defined for every RV (discrete or continuous).
Definition Cumulative Distribution FunctionF ( x ) = P ( X ≤ x ) = ∫ − ∞ x f ( t ) d t F(x) = P(X \le x) = \int_{-\infty}^{x} f(t)\,dt F ( x ) = P ( X ≤ x ) = ∫ − ∞ x f ( t ) d t
Derivation of its properties (from the PDF axioms):
F ( − ∞ ) = ∫ − ∞ − ∞ f = 0 F(-\infty)=\int_{-\infty}^{-\infty} f = 0 F ( − ∞ ) = ∫ − ∞ − ∞ f = 0 F ( ∞ ) = ∫ − ∞ ∞ f = 1 F(\infty)=\int_{-\infty}^{\infty} f = 1 F ( ∞ ) = ∫ − ∞ ∞ f = 1 F F F non-decreasing : adding more area (since f ≥ 0 f\ge 0 f ≥ 0 F F F continuous for a continuous RV (no jumps, because no point carries mass).
WHAT: The p p p x p x_p x p p p p
Definition Percentile / quantilex p x_p x p F ( x p ) = p , 0 < p < 1. F(x_p) = p, \qquad 0<p<1. F ( x p ) = p , 0 < p < 1. x p = F − 1 ( p ) x_p = F^{-1}(p) x p = F − 1 ( p ) inverse CDF , a.k.a. quantile function ).
Median = 50th percentile: F ( m ) = 0.5 F(m)=0.5 F ( m ) = 0.5 Quartiles : Q 1 = x 0.25 Q_1=x_{0.25} Q 1 = x 0.25 Q 2 = Q_2= Q 2 = Q 3 = x 0.75 Q_3=x_{0.75} Q 3 = x 0.75
WHY F F F for a continuous RV with f > 0 f>0 f > 0 F F F p p p x p x_p x p
Worked example Example 1 — A linear densityLet f ( x ) = 2 x f(x)=2x f ( x ) = 2 x [ 0 , 1 ] [0,1] [ 0 , 1 ] 0 0 0
(a) Check it's a valid PDF. ∫ 0 1 2 x d x = [ x 2 ] 0 1 = 1 \int_0^1 2x\,dx = [x^2]_0^1 = 1 ∫ 0 1 2 x d x = [ x 2 ] 0 1 = 1 2 x ≥ 0 2x\ge0 2 x ≥ 0 [ 0 , 1 ] [0,1] [ 0 , 1 ] Why this step? A function is only a PDF if it's non-negative and has total area 1.
(b) Find the CDF. For 0 ≤ x ≤ 1 0\le x\le1 0 ≤ x ≤ 1 F ( x ) = ∫ 0 x 2 t d t = x 2 . F(x)=\int_0^x 2t\,dt = x^2. F ( x ) = ∫ 0 x 2 t d t = x 2 . F ( x ) = 0 F(x)=0 F ( x ) = 0 x < 0 x<0 x < 0 = x 2 =x^2 = x 2 0 ≤ x ≤ 1 0\le x\le 1 0 ≤ x ≤ 1 = 1 =1 = 1 x > 1 x>1 x > 1 Why this step? Accumulate density from the left edge of the support.
(c) P ( 0.2 ≤ X ≤ 0.5 ) P(0.2\le X\le 0.5) P ( 0.2 ≤ X ≤ 0.5 ) = F ( 0.5 ) − F ( 0.2 ) = 0.25 − 0.04 = 0.21. =F(0.5)-F(0.2)=0.25-0.04 = 0.21. = F ( 0.5 ) − F ( 0.2 ) = 0.25 − 0.04 = 0.21. Why this step? Interval prob = difference of CDF — no need to re-integrate.
(d) Median. Solve F ( m ) = m 2 = 0.5 ⇒ m = 0.5 ≈ 0.707. F(m)=m^2=0.5 \Rightarrow m=\sqrt{0.5}\approx 0.707. F ( m ) = m 2 = 0.5 ⇒ m = 0.5 ≈ 0.707. Why this step? Median is the 50th percentile, F ( m ) = 0.5 F(m)=0.5 F ( m ) = 0.5
Worked example Example 2 — Exponential (memoryless waiting time)f ( x ) = λ e − λ x f(x)=\lambda e^{-\lambda x} f ( x ) = λ e − λ x x ≥ 0 x\ge 0 x ≥ 0 λ = 2 \lambda=2 λ = 2
CDF: F ( x ) = ∫ 0 x 2 e − 2 t d t = [ − e − 2 t ] 0 x = 1 − e − 2 x . F(x)=\int_0^x 2e^{-2t}\,dt = [-e^{-2t}]_0^x = 1-e^{-2x}. F ( x ) = ∫ 0 x 2 e − 2 t d t = [ − e − 2 t ] 0 x = 1 − e − 2 x . Why this step? ∫ λ e − λ t d t = − e − λ t \int \lambda e^{-\lambda t}dt = -e^{-\lambda t} ∫ λ e − λ t d t = − e − λ t
90th percentile: solve 1 − e − 2 x = 0.9 ⇒ e − 2 x = 0.1 ⇒ x = − ln 0.1 2 = ln 10 2 ≈ 1.151. 1-e^{-2x}=0.9 \Rightarrow e^{-2x}=0.1 \Rightarrow x=\frac{-\ln 0.1}{2}=\frac{\ln 10}{2}\approx 1.151. 1 − e − 2 x = 0.9 ⇒ e − 2 x = 0.1 ⇒ x = 2 − l n 0.1 = 2 l n 10 ≈ 1.151. Why this step? Set F ( x p ) = p F(x_p)=p F ( x p ) = p
Worked example Example 3 — From CDF backwardsGiven F ( x ) = 1 − 1 x 2 F(x)=1-\frac{1}{x^2} F ( x ) = 1 − x 2 1 x ≥ 1 x\ge 1 x ≥ 1 0 0 0 f f f f ( x ) = F ′ ( x ) = 2 x 3 , x ≥ 1. f(x)=F'(x)=\frac{2}{x^3},\quad x\ge1. f ( x ) = F ′ ( x ) = x 3 2 , x ≥ 1. Why this step? PDF is the derivative of the CDF (FTC). Check: ∫ 1 ∞ 2 x − 3 d x = [ − x − 2 ] 1 ∞ = 1. \int_1^\infty 2x^{-3}dx=[-x^{-2}]_1^\infty = 1. ∫ 1 ∞ 2 x − 3 d x = [ − x − 2 ] 1 ∞ = 1.
f ( x ) f(x) f ( x ) ≤ 1 \le 1 ≤ 1 Why it feels right: for discrete variables P ( X = x ) ≤ 1 P(X=x)\le1 P ( X = x ) ≤ 1 f f f The fix: f f f density , not a probability. It can exceed 1 (e.g. f ( x ) = 2 x f(x)=2x f ( x ) = 2 x f ( x ) = 5 f(x)=5 f ( x ) = 5 [ 0 , 0.2 ] [0,0.2] [ 0 , 0.2 ] = 1 =1 = 1 area must stay ≤ 1 \le 1 ≤ 1 = 1 =1 = 1
P ( X = c ) > 0 P(X=c)>0 P ( X = c ) > 0 Why it feels right: discrete intuition — every outcome has some chance.
The fix: a single point is an interval of width 0, so ∫ c c f = 0 \int_c^c f=0 ∫ c c f = 0
f f f F F F Why it feels right: the percentile is "about" the density curve.
The fix: percentile is an accumulated quantity — solve F ( x p ) = p F(x_p)=p F ( x p ) = p f ( x p ) = p f(x_p)=p f ( x p ) = p
Common mistake Forgetting to restrict to the support.Why it feels right: the formula f ( x ) = 2 x f(x)=2x f ( x ) = 2 x The fix: always state where f = 0 f=0 f = 0
What two conditions make f ( x ) f(x) f ( x ) f ( x ) ≥ 0 f(x)\ge0 f ( x ) ≥ 0 everywhere, and
∫ − ∞ ∞ f ( x ) d x = 1 \int_{-\infty}^\infty f(x)\,dx=1 ∫ − ∞ ∞ f ( x ) d x = 1 .
For a continuous RV, what is P ( X = c ) P(X=c) P ( X = c ) 0 0 0 , since
∫ c c f = 0 \int_c^c f=0 ∫ c c f = 0 .
How do you get the CDF from the PDF? F ( x ) = ∫ − ∞ x f ( t ) d t F(x)=\int_{-\infty}^x f(t)\,dt F ( x ) = ∫ − ∞ x f ( t ) d t (accumulate area from the left).
How do you get the PDF from the CDF? f ( x ) = F ′ ( x ) f(x)=F'(x) f ( x ) = F ′ ( x ) , by the Fundamental Theorem of Calculus.
Express P ( a ≤ X ≤ b ) P(a\le X\le b) P ( a ≤ X ≤ b ) F ( b ) − F ( a ) F(b)-F(a) F ( b ) − F ( a ) .
Define the p p p x p x_p x p The value with
F ( x p ) = p F(x_p)=p F ( x p ) = p ; i.e.
x p = F − 1 ( p ) x_p=F^{-1}(p) x p = F − 1 ( p ) .
Can a PDF value exceed 1? Yes — it's a density, only the total area must equal 1.
Why is the < < < ≤ \le ≤ Endpoints have zero probability, so including/excluding them changes nothing.
CDF of f ( x ) = 2 x f(x)=2x f ( x ) = 2 x [ 0 , 1 ] [0,1] [ 0 , 1 ] F ( x ) = x 2 F(x)=x^2 F ( x ) = x 2 on
[ 0 , 1 ] [0,1] [ 0 , 1 ] .
Median of exponential with λ \lambda λ Solve
1 − e − λ m = 0.5 ⇒ m = ln 2 / λ 1-e^{-\lambda m}=0.5 \Rightarrow m=\ln2/\lambda 1 − e − λm = 0.5 ⇒ m = ln 2/ λ .
Recall Feynman: explain to a 12-year-oldImagine spreading 1 kg of jam over a long piece of bread. You can't ask "how much jam is exactly at this one dot?" — a dot is too tiny, the answer is zero. But you can ask "how much jam between here and there?" — you scoop that strip and weigh it. The PDF is how thickly the jam is spread at each spot. The CDF is "how much jam have I scooped from the left edge up to here." The median is the spot where exactly half the jam is on each side.
Mnemonic Remember the trio"Density spreads, Cumulative collects, Percentile asks where."
PDF = f f f
CDF = F F F F = ∫ f F=\int f F = ∫ f
Percentile = invert F F F where a fraction sits).
And: "f to F you integrate, F to f you differentiate."
Discrete random variables — PMF (the discrete cousin: sums instead of integrals)Expectation and Variance of continuous RVs (E [ X ] = ∫ x f ( x ) d x E[X]=\int x f(x)\,dx E [ X ] = ∫ x f ( x ) d x Normal distribution (its CDF Φ \Phi Φ Exponential distribution (worked Example 2; memorylessness)Fundamental Theorem of Calculus (the f ↔ F f\leftrightarrow F f ↔ F Uniform distribution (constant density, linear CDF)Quantile function and inverse-transform sampling (using F − 1 F^{-1} F − 1
height is density, area is prob
Fundamental Theorem of Calculus
F non-decreasing and continuous
Intuition Hinglish mein samjhoDekho, continuous random variable wahi hota hai jo koi bhi value le sakta hai ek range me — jaise kisi banda ki exact height 172.48... 172.48... 172.48... ek exact value par probability hamesha zero hoti hai, kyunki ek point ki width zero hai. Isliye hum probability ko hamesha ek interval (chunk) par measure karte hain. Yeh measure karne ka tool hai PDF f ( x ) f(x) f ( x ) f ( x ) f(x) f ( x ) height probability nahi , density hai; probability to curve ke neeche ka area hai. Isliye f ( x ) f(x) f ( x )
CDF F ( x ) = P ( X ≤ x ) F(x)=P(X\le x) F ( x ) = P ( X ≤ x ) integrate karte hain (F = ∫ f F=\int f F = ∫ f differentiate (f = F ′ f=F' f = F ′ P ( a ≤ X ≤ b ) = F ( b ) − F ( a ) P(a\le X\le b)=F(b)-F(a) P ( a ≤ X ≤ b ) = F ( b ) − F ( a )
Percentile ka funda: p p p x p x_p x p p p p F ( x p ) = p F(x_p)=p F ( x p ) = p F ( m ) = 0.5 F(m)=0.5 F ( m ) = 0.5 F F F f f f f f f F F F
