Exercises — Uniform, Exponential, and Beta distributions

WHAT distribution? "Equally likely at any point of an interval" is the signature of the Uniform. Measuring minutes after 10:00, the interval is $[a,b]=[0,15]$, so $X\sim U(0,15)$.

WHY the CDF? $P(X\le 5)$ is exactly what the CDF $F(x)=\frac{x-a}{b-a}$ answers — "how much probability has piled up by the point $x$." $F(5)=\frac{5-0}{15-0}=\frac{5}{15}=\frac{1}{3}\approx \mathrm{0.3333.}$ So a $1/3$ chance.

Read the figure below: the teal rectangle is the whole uniform density — its full area is exactly $1$. The burnt-orange shaded slab covers only the first $5$ of the $15$ minutes, so it captures one third of the width and (because the height is constant) one third of the area. That shaded area is $P(X\le 5)=\frac{1}{3}$. The dashed callout points at the constant height $1/(b-a)=1/15$.

WHAT distribution? "Time until the next event in a Poisson process" is the Exponential, $T\sim \text{Exp}(3)$ (rate in emails per hour).

Mean: $\mu =\frac{1}{\lambda }=\frac{1}{3}$ hour $=20$ minutes. Variance: ${\sigma }^2=\frac{1}{{\lambda }^2}=\frac{1}{9}$, so $\sigma =\frac{1}{3}$ hour $=20$ minutes.

WHY mean = std here? For the exponential these are always equal ($\mu =\sigma =1/\lambda$) — a fingerprint of this distribution.

Put $\alpha =\beta =1$ into $f(x)=\frac{x^{\alpha -1}(1-x{)}^{\beta -1}}{B(\alpha ,\beta )}$. The exponents become $x^0(1-x{)}^0=1\cdot 1=1$. The normaliser is $B(1,1)=\frac{\Gamma (1)\Gamma (1)}{\Gamma (2)}=\frac{0!0!}{1!}=1$. So $f(x)=\frac{1}{1}=1$ for $0\le x\le 1$ — a flat line of height 1, exactly $U(0,1)$. WHY it matters: "no data yet" (0 pseudo-successes, 0 pseudo-failures) means total ignorance, and total ignorance is the flat uniform.

The density is $f(x)=\frac{1}{2-0}=\frac{1}{2}$ on $[0,2]$. $E[X^2]={\int }_0^2{x}^2\cdot \frac{1}{2}dx=\frac{1}{2}\cdot \frac{x^3}{3}{|}_0^2=\frac{1}{2}\cdot \frac{8}{3}=\frac{4}{3}.$ Check: $E[X]=\frac{0+2}{2}=1$, $\text{Var}=\frac{(2-0{)}^2}{12}=\frac{4}{12}=\frac{1}{3}$. Then $\text{Var}+(E[X]{)}^2=\frac{1}{3}+1=\frac{4}{3}$. ✓ They agree, as the identity $E[X^2]=\text{Var}+{\mu }^2$ guarantees.

Rate from the mean: $\mu =1/\lambda =500\Rightarrow \lambda =1/500=0.002$ per hour. Median $m$ solves $F(m)=0.5$: $1-e^{-\lambda m}=0.5\Rightarrow e^{-\lambda m}=0.5\Rightarrow -\lambda m=\ln 0.5\Rightarrow m=\frac{\ln 2}{\lambda }.$ $m=\frac{\ln 2}{0.002}=500\ln 2\approx 346.6\text{hours}.$ WHY median < mean? The exponential is right-skewed: a few very long lifetimes drag the mean ($500$) above the median ($\approx 347$). Half the parts die before $347$ h even though the average is $500$ h.

Mean: $\mu =\frac{\alpha }{\alpha +\beta }=\frac{3}{3+7}=\frac{3}{10}=\mathrm{0.3.}$ Mode (valid since $\alpha ,\beta >1$): $\frac{\alpha -1}{\alpha +\beta -2}=\frac{2}{8}=\mathrm{0.25.}$ Both are below $0.5$, so the bump leans toward $0$ — sensible, because $\beta =7$ pseudo-failures outnumber $\alpha =3$ pseudo-successes, so we believe the true rate is low.

Read the figure below: the plum curve is the $\text{Beta}(3,7)$ density. The teal dashed line marks the mode $0.25$ (the top of the hump — the single most likely rate), and the burnt-orange dashed line marks the mean $0.30$ (the balance point). Notice the mean sits to the right of the mode: the long tail stretching toward $1$ pulls the average up, which is the visual signature of a right-skewed Beta.

Conditional (waited 1s already): use the memoryless property $P(T>s+t\mid T>s)=P(T>t)$ with $s=1,t=0.5$: $P(T>1.5\mid T>1)=P(T>0.5)=e^{-\lambda \cdot 0.5}=e^{-2\cdot 0.5}=e^{-1}\approx \mathrm{0.3679.}$ Unconditional: $P(T>0.5)=e^{-\lambda \cdot 0.5}=e^{-1}\approx 0.3679$ — identical. WHY identical? The exponential "forgets" the elapsed second; the future wait resets as if the clock just started. This is the only continuous distribution with that property, which is why constant-hazard models (radioactive decay, bandit arrival gaps) use it.

Set $\alpha =\beta =k$. Then $\text{Var}=\frac{k\cdot k}{(2k{)}^2(2k+1)}=\frac{k^2}{4k^2(2k+1)}=\frac{1}{4(2k+1)}.$

• $k=1:\frac{1}{4\cdot 3}=\frac{1}{12}\approx 0.08333$ (this is exactly $U(0,1)$'s variance!).
• $k=5:\frac{1}{4\cdot 11}=\frac{1}{44}\approx \mathrm{0.02273.}$
• $k=50:\frac{1}{4\cdot 101}=\frac{1}{404}\approx \mathrm{0.002475.}$ As $k\to \infty$, $\text{Var}=\frac{1}{4(2k+1)}\to 0$: the belief piles up into a thin spike at $0.5$. Interpretation: more pseudo-data (bigger $\alpha ,\beta$) = more confidence = narrower distribution — the mathematical face of "the more evidence, the surer we are."

The conjugate rule: with a $\text{Beta}(\alpha ,\beta )$ prior and Binomial data of $h$ heads, $t$ tails, the posterior is simply $\text{Beta}(\alpha +h,\beta +t)$ — you add the counts. WHY it works: the Beta PDF $\propto p^{\alpha -1}(1-p{)}^{\beta -1}$ and the likelihood $\propto p^h(1-p{)}^t$ multiply into $p^{\alpha +h-1}(1-p{)}^{\beta +t-1}$ — the same Beta shape.

Here $\alpha =2,\beta =2,h=7,t=3$: $\text{posterior}=\text{Beta}(2+7,2+3)=\text{Beta}(9,5).$ Mean: $\frac{9}{9+5}=\frac{9}{14}\approx \mathrm{0.6429.}$ Mode: $\frac{9-1}{14-2}=\frac{8}{12}=\frac{2}{3}\approx \mathrm{0.6667.}$ The belief has moved from a symmetric hump at $0.5$ toward the observed heads-rate $0.7$, but not all the way — the prior "pseudo-data" pulls it back a little (this is regularisation in disguise).

Read the figure below: the teal curve is the flat-ish prior $\text{Beta}(2,2)$ centred at $0.5$; the plum curve is the posterior $\text{Beta}(9,5)$, taller, narrower, and shifted right. The dotted burnt-orange line marks the raw observed rate $0.7$. See how the posterior peak lands just short of $0.7$: the prior tugs the belief back toward $0.5$, so the update is a compromise between prior and data.

The idea (inverse-transform): if $U\sim U(0,1)$, then $T=F^{-1}(U)$ has CDF $F$. So set the exponential CDF equal to $U$ and solve for $t$: $U=1-e^{-\lambda t}\Rightarrow e^{-\lambda t}=1-U\Rightarrow t=-\frac{\ln (1-U)}{\lambda }.$ Since $1-U$ is also uniform on $(0,1)$, the common shortcut is $T=-\frac{\ln U}{\lambda }$.

Domain edge-cases — why we require $U\in (0,1)$ (open interval):

• As $U\to 1$, $1-U\to 0$ and $\ln (1-U)\to -\infty$, so $T\to +\infty$ — the rare, very long waits live here (that is correct: the exponential has an infinite tail).
• As $U\to 0$, $1-U\to 1$ and $\ln (1)=0$, so $T\to 0$ — the near-instant waits.
• The endpoints $U=0$ and $U=1$ would give $T=0$ exactly or $T=\infty$; a good RNG returns values strictly inside $(0,1)$, which is why we treat $U$ as living on the open interval and never actually hit those degenerate points.

Test ($\lambda =2,U=0.5$, using the derived form with $1-U$): $T=-\frac{\ln (1-0.5)}{2}=-\frac{\ln 0.5}{2}=\frac{\ln 2}{2}\approx \mathrm{0.3466.}$ Sanity check: $U=0.5$ is the median of the uniform, so it should map to the median of the exponential, which from L2.2 is $\frac{\ln 2}{\lambda }=\frac{\ln 2}{2}$. ✓

(a) Half-width. For $U(-r,r)$ the range is $b-a=2r$, so ${\sigma }^2=\frac{(2r{)}^2}{12}=\frac{4r^2}{12}=\frac{r^2}{3}.$ Set this to $\frac{1}{n}=\frac{1}{64}$: $\frac{r^2}{3}=\frac{1}{64}\Rightarrow r^2=\frac{3}{64}\Rightarrow r=\frac{\sqrt{3}}{8}\approx \mathrm{0.2165.}$ (Note: this equals the classic Xavier bound $r=\sqrt{3/n}$.)

(b) PDF height. The uniform density is $\frac{1}{b-a}=\frac{1}{2r}$. Substitute $r=\frac{\sqrt{3}}{8}$: $f(x)=\frac{1}{2r}=\frac{1}{2\cdot \frac{\sqrt{3}}{8}}=\frac{1}{\frac{\sqrt{3}}{4}}=\frac{4}{\sqrt{3}}\approx 2.309,$ for $x\in [-r,r]$ and $0$ elsewhere. So the exact height is $\frac{4}{\sqrt{3}}$.

(c) Matching variance with an exponential. Exponential variance is $1/{\lambda }^2$. Setting it equal to the weight variance $\frac{1}{64}$: $\frac{1}{{\lambda }^2}=\frac{1}{64}\Rightarrow {\lambda }^2=64\Rightarrow \lambda =8.$ WHY the mean is then forced: for the exponential $\mu =\frac{1}{\lambda }$ and ${\sigma }^2=\frac{1}{{\lambda }^2}$, so ${\sigma }^2={\mu }^2$ always — the variance and the squared mean are the same quantity. Fixing the variance to $\frac{1}{64}$ therefore fixes $\lambda =8$ and, with it, pins the mean to $\mu =\frac{1}{\lambda }=\frac{1}{8}=0.125$. You cannot choose spread and centre independently for an exponential (one parameter controls both), unlike the uniform where you could slide the interval $[-r,r]$ sideways to move the mean without touching the variance.

Uniform $U(a,b)$ variance ::: $\frac{(b-a{)}^2}{12}$ Exponential CDF ::: $F(t)=1-e^{-\lambda t}$ Exponential mean and variance relationship ::: ${\sigma }^2={\mu }^2$ (both $=1/\lambda ,1/{\lambda }^2$) Beta$(\alpha ,\beta )$ mean ::: $\frac{\alpha }{\alpha +\beta }$ Conjugate update after $h$ heads, $t$ tails ::: $\text{Beta}(\alpha +h,\beta +t)$ Uniform-to-exponential transform ::: $T=-\frac{\ln (1-U)}{\lambda }$