Visual walkthrough — Gaussian - Normal distribution properties

Before symbols, some plain words:

• A distribution is a way of sharing out "how likely" across all the values $x$ a number could take. Picture a fixed amount of sand (total = 1 bucket) sprinkled along a horizontal line. Tall piles = likely values.
• The height of the sand at position $x$ is written $f(x)$ and called the probability density. It is just "how deep is the sand here".
• $\mu$ (Greek letter mu, say "mew") will be the center — where the pile is balanced.
• $\sigma$ (Greek sigma) will be the spread — how wide the pile is.
• $\exp (u)$ just means $e^u$, the number $e\approx 2.718$ raised to the power $u$. We will meet it only when the maths demands it, and explain why then.

---

Step 1 — Sand on a line, with three rules

WHAT. We have a density $f(x)$: a curve whose height is the depth of sand at $x$. We impose exactly three requirements and nothing else:

$\underset{\text{all sand adds to one bucket}}{\underbrace{\int f(x)dx=1}}\underset{\text{balance point sits at }\mu }{\underbrace{\int xf(x)dx=\mu }}\underset{\text{average squared distance from }\mu \text{ is fixed}}{\underbrace{\int (x-\mu {)}^{2}f(x)dx={\sigma }^2}}$

The symbol $\int (\dots )dx$ ("integral") means add up over every position on the line — think of sliding along and summing every sliver of sand. Term by term:

• $\int f(x)dx=1$ — the total sand is one full bucket. Nothing created, nothing lost.
• $\int xf(x)dx=\mu$ — weigh each position $x$ by how much sand is there; the weighted center is $\mu$.
• $\int (x-\mu {)}^{2}f(x)dx={\sigma }^2$ — $(x-\mu {)}^2$ is the squared distance from the center; averaged over the sand, it equals ${\sigma }^2$. This pins how far the sand typically wanders.

WHY. These are the only two facts we claim to know: where the pile balances and how wide it is. We refuse to sneak in any other assumption.

PICTURE.

---

Step 2 — "Fairest" means maximum entropy

WHAT. Among the infinitely many sand-piles obeying those three rules, we pick the one that assumes the least. The measure of "assuming the least" is entropy:

$H=-\int f(x)\ln f(x)dx$

Reading it: $\ln f(x)$ is the natural logarithm of the height; the minus sign makes flat, spread-out piles score high and sharp, opinionated spikes score low.

WHY this tool and not another? We need a single number that says "how non-committal is this pile?". A spike claims "I know the answer is right here" — that is a strong claim, low entropy. A gentle spread claims little — high entropy. Maximising $H$ therefore = making the fewest extra assumptions beyond our three rules. Any other choice would secretly smuggle in information we do not have.

PICTURE.

---

Step 3 — Balancing slopes: Lagrange multipliers

WHAT. We want to maximise $H$ but we are chained by three constraints. The trick: build one combined quantity, the Lagrangian $\mathcal{L}$, that adds a "price tag" to each constraint:

$\mathcal{L}=-\int f\ln fdx+{\lambda }_0\left(\int fdx-1\right)+{\lambda }_1\left(\int xfdx-\mu \right)+{\lambda }_2\left(\int (x-\mu {)}^{2}fdx-{\sigma }^2\right)$

Each $\lambda$ (lambda) is an unknown price that we tune so its constraint is exactly obeyed.

WHY. At the best pile, you cannot increase entropy without breaking a rule — the "push" from entropy is perfectly cancelled by the "pull" of the constraints. Lagrange multipliers are the standard way to find that balance: raise entropy and pay the constraint prices simultaneously, then demand zero net movement.

PICTURE.

---

Step 4 — Set the change to zero, solve for $f$

WHAT. Nudge the height $f(x)$ at one point by a hair and ask "did $\mathcal{L}$ change?". At the maximum the answer must be no change for every point. That derivative (the functional derivative $\frac{\delta \mathcal{L}}{\delta f}$) is:

$\frac{\delta \mathcal{L}}{\delta f}=\underset{\text{from entropy }-f\ln f}{\underbrace{-\ln f-1}}+\underset{\text{price of total}}{\underbrace{{\lambda }_0}}+\underset{\text{price of mean}}{\underbrace{{\lambda }_{1}x}}+\underset{\text{price of spread}}{\underbrace{{\lambda }_2(x-\mu {)}^2}}=0$

Solve for $\ln f$, then exponentiate (undo the log) to free $f$:

$\ln f=({\lambda }_0-1)+{\lambda }_{1}x+{\lambda }_2(x-\mu {)}^2⟹f(x)=A\exp ({\lambda }_{1}x+{\lambda }_2(x-\mu {)}^2)$

Here $A=e^{{\lambda }_0-1}$ is just a positive constant, and $\exp$ appears because we undid a logarithm — that is the honest reason $e$ enters the story: solving $\ln f=(\text{stuff})$ forces $f=e^{\text{stuff}}$.

WHY. "Every tiny nudge leaves $\mathcal{L}$ unchanged" is exactly what maximum means. Writing that condition and unwrapping the log delivers the shape of the winner: an exponential of a quadratic.

PICTURE.

---

Step 5 — The mean constraint kills the linear term

WHAT. We still have three unknowns $A,{\lambda }_1,{\lambda }_2$. The mean rule ("balance at $\mu$") demands the pile be symmetric about $\mu$: as much sand a step to the right as a step to the left. But the term ${\lambda }_{1}x$ tilts the pile — it makes one side heavier. The only way to stay balanced is

${\lambda }_1=0.$

So the density collapses to $f(x)=A\exp ({\lambda }_2(x-\mu {)}^2)$, depending on $x$ only through the distance $(x-\mu )$.

WHY. A nonzero ${\lambda }_1$ would shift the balance point away from $\mu$, contradicting our second rule. Symmetry is not assumed — it is forced by fixing the center.

PICTURE.

---

Step 6 — The sign of ${\lambda }_2$: it must be negative

WHAT. Look at $f(x)=A\exp ({\lambda }_2(x-\mu {)}^2)$ and test the sign of ${\lambda }_2$:

• If ${\lambda }_2>0$: as $x$ runs far from $\mu$, $(x-\mu {)}^2\to \infty$ and $f\to \infty$. The sand piles up at the edges of the universe — total sand is infinite. Impossible (breaks rule 1).
• If ${\lambda }_2=0$: $f=A$, a flat line stretching forever — again infinite total sand. Impossible.
• If ${\lambda }_2<0$: far from $\mu$ the exponent is a big negative number, so $f\to 0$. The sand fades to nothing at the edges and the total can be finite. This is the only survivor.

Write the surviving constant as ${\lambda }_2=-\frac{1}{2{\sigma }^2}$ (negative, since ${\sigma }^2>0$; the $2$ is chosen to make later algebra clean). Then

$f(x)=A\exp \left(-\frac{(x-\mu {)}^2}{2{\sigma }^2}\right).$

WHY. We covered all three sign cases and only one keeps the sand finite. The bell shape — high in the middle, decaying to zero on both sides — is the unavoidable outcome.

PICTURE.

---

Step 7 — Fix $A$ so the sand totals exactly one bucket

WHAT. The last unknown $A$ is nailed by normalization $\int fdx=1$. This uses one famous fact, the Gaussian integral:

${\int }_{-\infty }^{\infty }\exp \left(-\frac{(x-\mu {)}^2}{2{\sigma }^2}\right)dx=\sigma \sqrt{2\pi }.$

So $\int fdx=A\cdot \sigma \sqrt{2\pi }=1$, which forces

$\boxed{A=\frac{1}{\sigma \sqrt{2\pi }}}.$

The $\sqrt{2\pi }$ is not decoration — it is precisely the amount of area under $e^{-(x-\mu {)}^2/2{\sigma }^2}$, so dividing by it makes the total exactly one full bucket.

Assembling Steps 5–7:

$f(x)=\frac{1}{\sigma \sqrt{2\pi }}\exp \left(-\frac{(x-\mu {)}^2}{2{\sigma }^2}\right).$

The variance rule from Step 1 then confirms this same $\sigma$ really is the spread — everything is consistent.

WHY. A density that does not total 1 is not a probability. $A$ is the single scaling knob that turns our shape into a legal distribution.

PICTURE.

---

Step 8 — Degenerate edge: what happens as $\sigma \to 0$?

WHAT. Push the spread to zero. The width $\sigma \sqrt{2\pi }\to 0$ so the peak height $\frac{1}{\sigma \sqrt{2\pi }}\to \infty$, while the curve away from $\mu$ is crushed to zero. The bell becomes an infinitely tall, infinitely thin spike at $\mu$ — a "certain" value. In the other extreme $\sigma \to \infty$, the bell flattens toward a barely-there horizontal smear: total ignorance across the whole line.

WHY. These limits are the two boundaries of belief the Gaussian can express: $\sigma \to 0$ = "I know it exactly", $\sigma \to \infty$ = "I know nothing". Every real Gaussian lives between them.

PICTURE.

---

The one-picture summary

The whole journey in one frame: three constraints (Step 1) → be maximally humble (Step 2) → balance humility against constraints with Lagrange (Step 3) → set the nudge to zero and undo the log, so $\exp$ appears (Step 4) → mean forces the tilt away (${\lambda }_1=0$, Step 5) → finiteness forces a negative quadratic (Step 6) → total-one fixes the front constant (Step 7). Out drops the bell.

Recall Feynman retelling — say it back in plain words

Imagine one bucket of sand you must spread along a line. You only agree to two things: where it balances, and how far it typically strays from that balance point. Beyond that you want to be as un-opinionated as possible — you refuse to claim anything extra. "Un-opinionated" gets a score called entropy, and you make it as big as possible while still honouring your two agreements. Doing the balancing algebra (Lagrange multipliers, then setting every tiny nudge to zero) spits out a formula for the sand's height that involves $e$ raised to something. That "something" turns out to be a downward-bending parabola in $(x-\mu )$: it must bend down or the sand would run off to infinity, and it must be centred (no sideways tilt) or the balance point would drift. Finally you scale the whole thing so the sand really does add to one bucket — that scaling is the $\frac{1}{\sigma \sqrt{2\pi }}$ out front. What falls out is the bell curve. Nothing was assumed about "bells"; the bell is simply the fairest pile with a fixed center and spread.

This is why the Gaussian shows up everywhere (see Central Limit Theorem), why standardizing works (see Maximum Likelihood Estimation), and why it is the workhorse noise model in Linear Regression, Kalman Filters and Bayesian Inference.

Recall

Which sign of ${\lambda }_2$ survives, and why? ::: Negative — only ${\lambda }_2<0$ makes $f\to 0$ at the edges so the total area stays finite. Why does the linear term ${\lambda }_{1}x$ vanish? ::: Fixing the mean at $\mu$ forces symmetry; any tilt would move the balance point, so ${\lambda }_1=0$. Where does the $\exp$ (the $e$) come from? ::: From undoing the logarithm: solving $\ln f=(\text{stuff})$ gives $f=e^{\text{stuff}}$. What does $\frac{1}{\sigma \sqrt{2\pi }}$ do? ::: It rescales the shape so the total area (total probability) equals exactly 1. Why maximise entropy? ::: It is the least-assuming distribution consistent with the known mean and variance.