Probability Distributions

The unifying principle

A random variable $X$ takes values in a space $\Omega$. A probability distribution assigns non-negative weight normalized to one:

$\sum_i p_i = 1 \quad\text{(discrete)} \qquad \int_\Omega p(x)\,dx = 1 \quad\text{(continuous)}, \qquad p(x)\ge 0.$

Everything measurable is an expectation against this measure:

$\mathbb{E}[f(X)] = \int_\Omega f(x)\,p(x)\,dx.$

Two moments dominate practice:

$\mu = \mathbb{E}[X], \qquad \sigma^2 = \mathbb{E}[(X-\mu)^2] = \mathbb{E}[X^2]-\mu^2.$

A recurring theme is the maximum-entropy / exponential family form. Maximizing entropy $H = -\int p\ln p$ under constraints gives

$p(x) = \frac{1}{Z}\,e^{-\lambda \cdot T(x)},$

where $Z$ (the partition function / normalizer) ties everything together. This single template reappears in every field below — only the meaning of $T(x)$ and $\lambda$ changes.

How it shows up in each field

Maths — the abstract source

The distribution is a measure $\mu$ on a $\sigma$-algebra, characterized by its CDF $F(x)=P(X\le x)$ or density $p=dF/dx$. The Gaussian

$p(x)=\frac{1}{\sqrt{2\pi\sigma^2}}\exp\!\left(-\frac{(x-\mu)^2}{2\sigma^2}\right)$

is central because of the Central Limit Theorem: sums of many independent finite-variance contributions converge to it. Example: rolling $n$ dice, the sum's distribution flattens toward a bell curve as $n\to\infty$.

Physics — quantum amplitudes and thermal ensembles

Two guises, both distributions:

• Quantum: the wavefunction $\psi(x)$ gives a distribution by the Born rule $p(x) = |\psi(x)|^2,$ the probability density of finding a particle at $x$ upon measurement.
• Statistical mechanics: the Boltzmann distribution over energy states, $p(E_i) = \frac{1}{Z}e^{-E_i/k_BT}, \qquad Z=\sum_i e^{-E_i/k_BT},$ is exactly the max-entropy form with $T(x)=E_i$ and $\lambda = 1/k_BT$.

Example: particle in a harmonic trap — position measurements scatter as a Gaussian whose width encodes both quantum zero-point motion and thermal energy.

AI-ML — modeling belief and learning by likelihood

Models output distributions, not point guesses. A classifier's softmax is identical to Boltzmann:

$p(y=k\mid x) = \frac{e^{z_k}}{\sum_j e^{z_j}},$

with logits $z_k$ playing the role of $-E_k$. Training minimizes cross-entropy = maximizes likelihood, which is minimizing $\mathrm{KL}(p_{\text{data}}\,\|\,p_\theta)$. Uncertainty splits into aleatoric (irreducible, in the data) and epistemic (model's own ignorance, e.g. a posterior over weights).

Example: a Bayesian regressor predicts $p(y\mid x)=\mathcal{N}(\mu_\theta(x),\sigma_\theta^2(x))$ — wide $\sigma$ where data is sparse.

Stock-Market — asset returns and risk

Log-returns $r_t = \ln(S_t/S_{t-1})$ are modeled as random variables. The Black–Scholes world assumes them Gaussian, giving lognormal prices

$S_T = S_0\exp\!\left(\big(\mu-\tfrac{\sigma^2}{2}\big)T + \sigma\sqrt{T}\,Z\right),\quad Z\sim\mathcal N(0,1).$

But real returns are fat-tailed (leptokurtic): extreme moves far more likely than Gaussian predicts. Risk is a distributional statement — Value at Risk is just a quantile:

$\mathrm{VaR}_\alpha = -\,F_r^{-1}(\alpha).$

Example: the 2008 crash was a $\sim$10σ event under a Gaussian model — proof the tails are wrong, motivating Student-t or jump-diffusion distributions.

Why this bridge matters

• Max-entropy transfers everywhere: recognizing softmax = Boltzmann means physics intuition about temperature (sharpening/smoothing distributions) directly explains ML temperature scaling in sampling and calibration.
• The partition function $Z$ is the same headache in stat-mech, in Bayesian evidence, and in normalizing energy-based models — techniques to approximate it (MCMC, variational bounds) cross fields freely.
• CLT is a blessing and a trap: it justifies Gaussians in physics measurement noise and portfolio aggregation, but the failure of finite-variance assumptions in markets (fat tails) is the single most important cautionary tale ML risk modeling must inherit.
• Expectation is the universal action verb: quantum observables, thermal averages, model predictions, and expected returns are all $\mathbb{E}[f(X)]$ — so variance-reduction tricks from Monte Carlo finance help quantum simulation and ML gradient estimation alike.

Connections

• 01-Measure-Theory-and-Random-Variables
• 02-Central-Limit-Theorem
• 03-Maximum-Entropy-and-Exponential-Families
• 04-Born-Rule-and-Wavefunctions
• 05-Boltzmann-Distribution-and-Partition-Function
• 06-Softmax-and-Cross-Entropy
• 07-Bayesian-Inference-and-Uncertainty
• 08-Log-Returns-and-Fat-Tails
• 09-Value-at-Risk-and-Quantiles

#bridge