Foundations — Statistical estimation — MLE, method of moments

This page assumes you have never seen a single symbol on the parent note. We build them one at a time, each on top of the last, each anchored to a picture. By the end you will be able to read the parent note like a sentence.

1. Data: the raw numbers $x_1,x_2,\dots ,x_n$

Why does the topic need this? Every recipe on the parent page starts from "given data $x_1,\dots ,x_n$…". Without a name for the numbers, we cannot say anything about them.

2. Big vs small letters: $X$ (random) vs $x$ (observed)

Why the topic needs this: an estimator is built from the still-spinning $X$'s (so it is itself random and we can ask about its spread), while an estimate is the frozen number it spits out once the $x$'s land. The whole "estimator vs estimate" distinction lives entirely in this capital-vs-lowercase choice.

3. Summation $\sum$ and the average $\bar{x}$

Why the topic needs it: almost every answer on the parent page ($\hat{\lambda }=1/\bar{x}$, $\hat{p}=\bar{x}$, $\hat{\mu }=\bar{x}$) is "the average, possibly flipped or reshaped". If you own $\bar{x}$, you own half the results.

4. The parameter $\theta$ and its guess $\hat{\theta }$

Why the topic needs it: estimation is precisely "produce $\hat{\theta }$ from the $x_i$". Every boxed answer on the parent page is a formula for some hat.

5. Probability distribution, density $f$, and "$;\theta$"

Why the topic needs it: MLE literally reads these hill-heights at your observed points and multiplies them. MoM reads averages of these hills. Both start from $f(x;\theta )$.

6. Expectation $E[\cdot ]$ and moments ${\mu }_k$

Why the topic needs it: MoM = "set ${\mu }_k=m_k$ and solve". You cannot even read that sentence without both moment ideas.

7. Variance ${\sigma }^2$ and standard deviation $\sigma$

Why the topic needs it: the normal distribution $\mathcal{N}(\mu ,{\sigma }^2)$ has two dials, $\mu$ and ${\sigma }^2$. The parent page estimates both, and the famous "divide by $n$ vs $n-1$" bias story is entirely about ${\hat{\sigma }}^2$.

8. Product $\prod$ — why likelihoods multiply

Why the topic needs it: the likelihood is born as a product. Understanding "independent ⇒ multiply" is the whole reason $L$ has a $\prod$ in it.

9. The logarithm $\log$ / $\ln$ — turning $\prod$ into $\sum$

Why the topic needs it: every MLE derivation goes $L\to \ell =\log L$, so that $\prod$ becomes $\sum$ and calculus becomes bearable.

10. The derivative $\frac{d\ell }{d\theta }$ and setting it to zero

Prerequisite map

Read it top-down: raw data feeds two streams — the moment stream (average, expectation) that powers MoM, and the likelihood stream (density, product, log, derivative) that powers MLE. Both pour into the final guess $\hat{\theta }$.

Equipment checklist

Read each item as a question; the answer after ::: is the readiness you should already feel.