Bayesian Updating

The unifying principle

Start from the product rule of probability, $P(H,E) = P(E\mid H)\,P(H) = P(H\mid E)\,P(E)$. Rearranging gives Bayes' theorem:

$$\underbrace{P(H\mid E)}_{\text{posterior}} \;=\; \frac{\overbrace{P(E\mid H)}^{\text{likelihood}}\;\overbrace{P(H)}^{\text{prior}}}{\underbrace{P(E)}_{\text{evidence}}}, \qquad P(E)=\sum_i P(E\mid H_i)P(H_i).$$

Because $P(E)$ is just a normalizer, the working form is proportionality:

$$P(H\mid E) \;\propto\; P(E\mid H)\,P(H).$$

The key invariant across every field: replicators/beliefs that predicted the observed data better get up-weighted, exactly in proportion to their relative likelihood. That single sentence is the bridge.

How it shows up in each field

Maths — the theorem itself

The form: pure conditional probability. Notation is $P(H\mid E)$. It is the source; everything else is a specialization. The clean intuition is odds form, which strips away the normalizer:

$$\frac{P(H_1\mid E)}{P(H_2\mid E)} = \underbrace{\frac{P(H_1)}{P(H_2)}}_{\text{prior odds}}\times\underbrace{\frac{P(E\mid H_1)}{P(E\mid H_2)}}_{\text{Bayes factor}}.$$

Example. A disease has prevalence $P(D)=0.01$. A test with sensitivity $P(+\mid D)=0.99$ and false-positive rate $P(+\mid \neg D)=0.05$. A positive test gives $P(D\mid +) = \frac{0.99\cdot0.01}{0.99\cdot0.01 + 0.05\cdot0.99} \approx 0.167$. Base rate dominates — the update is real but modest.

AI-ML — learning as posterior inference

The form: parameters $\theta$ are the hypothesis, data $\mathcal{D}$ is the evidence.

$$p(\theta\mid \mathcal{D}) \propto p(\mathcal{D}\mid\theta)\,p(\theta).$$

Maximum-likelihood training maximizes $\log p(\mathcal{D}\mid\theta)$; L2 regularization is literally a Gaussian prior ($-\lambda\|\theta\|^2$ is $\log$ of $\mathcal N(0,\sigma^2 I)$). Naive Bayes classifiers, Kalman filters, and Bayesian neural nets are direct applications; even SGD is an online approximation to sequential updating. Example. A spam filter starts with prior $P(\text{spam})=0.5$. The word "lottery" has $P(\text{"lottery"}\mid\text{spam})=0.9$ vs $0.01$ for ham. Seeing it drives posterior spam-odds up by a Bayes factor of $90$.

Stock-Market — repricing on news

The form: a trader's belief about an asset's true value $V$ is a prior; earnings reports, macro data, and order flow are evidence. Prices move because the market posterior moves.

$$p(V\mid \text{news}) \propto p(\text{news}\mid V)\,p(V).$$

The efficient-market ideal is that price = current Bayesian posterior mean; surprise (news minus expectation) drives returns, mirroring $\log P(E\mid H)$. Overreaction/underreaction are miscalibrated likelihoods; anchoring is an over-weighted prior. Example. A stock is priced expecting $1.00 EPS. Actual EPS is $1.20. The 20¢ surprise — not the level — moves the price; if the prior expectation had already been $1.20, the same report barely budges it. Same "surprise-scaled" update as the disease test.

Biology — evolution as belief revision

The form: a population is a "belief distribution" over genotypes; the frequency of type $i$ is its prior probability, fitness $w_i$ is its likelihood of surviving/reproducing. Replicator dynamics is a Bayesian update:

$$p_i' = \frac{w_i\,p_i}{\sum_j w_j\,p_j} \;\;\Longleftrightarrow\;\; P(H_i\mid E)=\frac{P(E\mid H_i)P(H_i)}{\sum_j P(E\mid H_j)P(H_j)}.$$

Genotype frequency $p_i \leftrightarrow$ prior, fitness $w_i \leftrightarrow$ likelihood, next generation $\leftrightarrow$ posterior. Natural selection literally renormalizes by relative fitness each generation. Example. Antibiotic-resistant bacteria at frequency $0.001$ with relative fitness $2$ under the drug vs $1$ for susceptibles: after exposure their frequency updates to $\frac{2\cdot0.001}{2\cdot0.001+1\cdot0.999}\approx0.002$ — the environment is "evidence" up-weighting the fitter hypothesis.

Why this bridge matters

What transfers is a single mental checklist: prior, likelihood, surprise, renormalize.

• Biology → ML: the replicator equation is multiplicative-weights update, which is softmax/exponentiated-gradient learning. Evolution gives geometric intuition for why boosting and mixture-of-experts work: reweight by performance.
• Maths → Markets: odds form explains why only surprise moves prices — the prior is already baked in. It also warns of the base-rate fallacy: a "99% accurate" signal on a rare event is mostly noise.
• ML → Biology/Markets: overfitting = trusting a coincidental likelihood spike; the cure (a prior/regularizer) is the same cure for evolutionary stochastic drift and for traders over-updating on one earnings call.
• The deep unlock: iterated multiplication of likelihoods → addition of information (log-evidence). Learning, adaptation, and price discovery are all accumulating log-evidence over time, and all share the same failure mode — a zero prior can never be revived ($P(H)=0 \Rightarrow P(H\mid E)=0$). In evolution that's an extinct allele; in ML a dead neuron; in markets a bankrupted belief. Never assign a hard zero.

Connections

• 01-Probability-Theory
• 02-Conditional-Probability-and-Odds
• 07-Bayesian-Inference-and-Priors
• 11-Naive-Bayes-and-Kalman-Filters
• 14-Regularization-as-Priors
• 22-Efficient-Market-Hypothesis
• 23-Earnings-Surprise-and-Price-Reaction
• 31-Replicator-Dynamics
• 34-Natural-Selection-and-Fitness

#bridge