Maximum a posteriori estimation (MAP): Question bank

Quick vocabulary refresher so nothing below is a mystery:

Definition Words and symbols you must not confuse

$x$ = the whole collection of observations, i.e. $x \equiv (x_{1}, x_{2}, \dots, x_{n})$ . The bold letter is shorthand for the entire data list, not a single number.
Posterior = $P (θ ∣ x)$ , our belief about the parameter after seeing the data $x$ . See Posterior Distribution.
Likelihood = $P (x ∣ θ)$ , how probable the observed data $x$ is for a given value of $θ$ . This is the quantity MLE maximizes on its own.
Evidence = $P (x)$ , the total probability of the data averaged over all parameter values — the normalizing denominator in Bayes Theorem. It does not depend on $θ$ .
Precision = the inverse of variance, $1/ σ^{2}$ . Big precision = small variance = high confidence; small precision = big variance = vagueness.
Mode = the location of the peak (highest point) of a distribution — the single most probable value.
Mean = the balance point (average) of a distribution — where it would sit on a see-saw.
$n$ = the number of observations (sample size), e.g. how many coin flips or data points we collected.
$σ_{0}^{2}$ = the prior variance — how spread out our prior belief is before seeing data. Small $σ_{0}^{2}$ = confident prior; large $σ_{0}^{2}$ = vague prior. Its inverse $1/ σ_{0}^{2}$ is the prior precision. MAP reports the mode of the posterior, not its mean.

Formula The Gaussian-mean example — the full "why" behind the closed form

Observe $n$ i.i.d. points $x_{1}, \dots, x_{n}$ (so $x \equiv (x_{1}, \dots, x_{n})$ ) from $N (μ, 1)$ (known variance $1$ ), with a Gaussian prior $μ \sim N (0, σ_{0}^{2})$ . The log-posterior is $lo g P (μ ∣ x) = - \frac{1}{2} \sum_{i = 1}^{n} (x_{i} - μ)^{2} - \frac{μ ^{2}}{2 σ _{0}^{2}} + const .$ Step 1 — differentiate. The peak of a smooth curve has zero slope, so we ask where $\partial / \partial μ = 0$ . Differentiating the first term: each $- \frac{1}{2} (x_{i} - μ)^{2}$ has derivative $+ (x_{i} - μ)$ (chain rule, the inner derivative of $(x_{i} - μ)$ w.r.t. $μ$ is $- 1$ , cancelling the leading minus). Differentiating $- μ^{2} / (2 σ_{0}^{2})$ gives $- μ / σ_{0}^{2}$ . So $\frac{\partial}{\partial μ} lo g P (μ ∣ x) = \sum_{i = 1}^{n} (x_{i} - μ) - \frac{μ}{σ _{0}^{2}} = 0.$ Step 2 — expand the sum. Since $\sum_{i = 1}^{n} x_{i} = n \overset{x}{ˉ}$ (definition of the mean $\overset{x}{ˉ}$ ) and $\sum_{i = 1}^{n} μ = n μ$ : $n \overset{x}{ˉ} - n μ - \frac{μ}{σ _{0}^{2}} = 0.$ Step 3 — collect the $μ$ terms. Move the data part to one side and factor $μ$ out of the rest: $n \overset{x}{ˉ} = n μ + \frac{μ}{σ _{0}^{2}} = μ (n + \frac{1}{σ _{0}^{2}}) .$ Step 4 — divide. Isolate $μ$ : $\overset{μ}{^}_{MAP} = \frac{n x ˉ}{n + 1/ σ _{0}^{2}} .$ Read this as a weighted average: the data mean $\overset{x}{ˉ}$ carries weight $n$ (its precision from $n$ observations of unit variance), the prior mean $0$ carries weight $1/ σ_{0}^{2}$ (the prior precision), and we divide by the total weight. As $σ_{0}^{2} \to \infty$ the prior weight $1/ σ_{0}^{2} \to 0$ and $\overset{μ}{^}_{MAP} \to \overset{x}{ˉ}$ (the MLE).

True or false — justify

MAP always gives a different answer than MLE.

False. With a uniform (flat) prior the log-prior is constant, so it drops out of the argmax and MAP collapses exactly onto MLE.

As the sample size

n

(number of observations) grows, the MAP estimate drifts further from the MLE.

False — the opposite. The likelihood grows with every data point while the prior stays fixed, so the prior's relative weight shrinks and MAP

\to

MLE as

n \to \infty

MAP returns the mean of the posterior distribution.

False. MAP returns the mode (the peak). Mean and mode agree only for symmetric posteriors like the Gaussian; for skewed ones like Beta or Gamma they differ.

The MAP estimate is always a valid probability (between 0 and 1) when estimating a coin bias.

True for the parameter's own range, because the prior support forces

θ \in [0, 1]

; but that is guaranteed by the model's domain, not by MAP itself.

Multiplying by the prior can only shrink the estimate toward zero.

False. The prior pulls the estimate toward the prior's high-density region, which may be above or below the MLE. A Beta(3,2) prior (mean 0.6) can pull a low MLE up.

The evidence term

P (x)

must be computed to find the MAP estimate.

False. The evidence

P (x)

is constant in

θ

, so it never affects the location of the maximum and is dropped. You need it only for the normalized posterior, not for the argmax.

L2 regularization and MAP with a Gaussian prior are two names for the same optimization.

True. Minimizing

- lo g P (x ∣ θ) + λ ∥ θ ∥^{2}

is exactly maximizing the log-posterior with

θ \sim N (0, \frac{1}{2 λ})

. See Regularization in ML and Ridge Regression.

A stronger prior (smaller prior variance

σ_{0}^{2}

) means the data has more influence.

False. Small prior variance = high prior precision = high confidence, so the data has less pull and the estimate stays closer to the prior mean.

MAP requires the prior and likelihood to be conjugate.

False. Conjugate Priors make the posterior a clean closed form, but MAP only needs you to maximize log-likelihood + log-prior — that can be done numerically for any prior.

Spot the error

"Since the posterior is proportional to likelihood times prior, MAP = MLE times prior mode."

Wrong. You multiply the functions

P (x ∣ θ) P (θ)

and then find the argmax of the product; you cannot multiply the two separate estimates together.

"I'll drop

lo g P (θ)

because it's a constant, just like I dropped

P (x)

Error. The evidence

P (x)

is constant in

θ

, but

P (θ)

depends on $θ$ — dropping it deletes the prior and turns MAP back into MLE.

"Beta(1,1) is an informative prior favoring low values."

Wrong. Beta(1,1) is the uniform prior on

[0, 1]

— it is flat and uninformative, so MAP under it equals MLE.

"The mode of a Beta

(α, β)

α / (α + β)

That is the mean. The mode is

\frac{α - 1}{α + β - 2}

(for

α, β > 1

); confusing them is the classic mode/mean slip.

"Taking the log changes where the maximum is, so log-posterior MAP ≠ posterior MAP."

Wrong.

lo g

is monotonically increasing, so it preserves the location of the argmax. It only makes products into sums for easier differentiation.

"A larger

λ

in ridge regression corresponds to a wider (weaker) Gaussian prior."

Reversed. From the Gaussian prior

N (0, σ^{2})

the penalty is

- lo g P (θ) = \frac{∥ θ ∥ ^{2}}{2 σ ^{2}}

, so matching it to

λ ∥ θ ∥^{2}

forces

λ = \frac{1}{2 σ ^{2}}

; larger

λ

therefore means smaller

σ^{2}

— a narrower, stronger prior that shrinks weights harder.

Why questions

Why does MAP prevent Overfitting when data is scarce?

With few data points the likelihood is flat and noisy; the prior supplies extra structure that anchors the estimate to plausible values instead of chasing sampling noise.

Why do we add the log-prior instead of multiplying the prior in the objective?

Because we took the log of a product (likelihood × prior), and

lo g (ab) = lo g a + lo g b

turns the product into a sum that is far easier to differentiate.

Why is L1 / Lasso regularization the MAP estimate under a Laplace prior, not a Gaussian one?

The Laplace density has a

\propto e^{- ∣ θ ∣/ b}

shape (where

b > 0

is its scale parameter, controlling spread), so its negative log is

\propto ∣ θ ∣

— exactly the L1 penalty, whereas Gaussian gives the squared L2 penalty.

Why does the Gaussian-mean example give a weighted average of data mean and prior mean?

The log-posterior is a sum of two quadratics; setting its derivative to zero linearly balances each term by its precision (inverse variance), which literally is a weighted average of

\overset{x}{ˉ}

(weight = data precision

n

) and

0

(weight = prior precision

1/ σ_{0}^{2}

Why can't we always just report the posterior mean instead of the MAP mode?

The mean requires integrating (normalizing) the whole posterior, which is often intractable; the mode only needs an argmax, which optimization handles cheaply.

Why does a flat prior make MAP "objective-free"?

A flat prior treats every parameter value as equally plausible a priori, so it injects no preference — the estimate is driven purely by the data, i.e. MLE.

Edge cases

What is MAP when the prior puts zero probability on a region of

θ

MAP can never land there — the log-prior is

- \infty

, so that region is forbidden regardless of how strongly the data supports it.

What happens to the Beta-coin MAP if you observe zero flips (

n = 0

The likelihood is constant, so the posterior equals the prior and MAP returns the prior mode — pure belief, no evidence. Caveat: if the prior is the uniform Beta(1,1), that prior is flat, so its mode is not unique (every

θ

is a peak) and MAP is undefined here.

For a Beta

(α, β)

posterior with exactly one of

α < 1

β < 1

, where is the mode?

The density blows up at one endpoint (0 if

α < 1

, 1 if

β < 1

), so the mode sits at that boundary, and the smooth "set derivative to zero" formula no longer applies.

For a Beta

(α, β)

posterior with both

α < 1

and

β < 1

, where is the mode?

The density blows up at both endpoints, giving two equal boundary modes at 0 and 1 — a genuine tie, and MAP cannot pick between them.

What is the MAP estimate under an improper (unnormalizable) flat prior over the whole real line?

It still works as long as the posterior itself is proper — the constant prior drops out and MAP equals MLE, even though

P (θ)

isn't a real distribution.

If the posterior is bimodal (two equal peaks), what does MAP report?

MAP returns just one peak (or is ambiguous between them), which is misleading — a single point badly summarizes a two-humped belief.

As prior variance

σ_{0}^{2} \to \infty

in the Gaussian-mean example, what does

\overset{μ}{^}_{MAP}

approach?

It approaches

\overset{x}{ˉ}

, the MLE, because an infinitely wide prior is effectively flat and contributes nothing to the optimization.

What does MAP give if likelihood and prior peak at opposite ends of the range?

A compromise located between them, pulled toward whichever term is sharper (higher curvature / more confident), not simply the midpoint.

Recall One-line summary to lock in

MAP = argmax of (log-likelihood + log-prior) = the mode of the posterior; it becomes MLE with a flat prior and becomes L2/L1 regularization with a Gaussian/Laplace prior.