Exercises — Gaussian - Normal distribution properties

The notation is $\mathcal{N}(\mu ,{\sigma }^2)$: first slot is the mean, second slot is the variance (not the standard deviation).

• Mean $\mu =0$.
• Variance ${\sigma }^2=4$.
• Standard deviation $\sigma =\sqrt{4}=2$.

Width is governed by the standard deviation $\sigma$, not the variance directly.

• ${\sigma }_A=\sqrt{9}=3$, ${\sigma }_B=\sqrt{1}=1$.
• $A$ is wider; its spread is $3/1=3$ times that of $B$.

The ==$z$-score== answers "how many standard deviations above the mean?" — it is exactly the recipe $Z=\frac{X-\mu }{\sigma }$ that turns $H$ into the standard normal $Z$ from the definition above. $z=\frac{x-\mu }{\sigma }=\frac{190-170}{\sqrt{100}}=\frac{20}{10}=2.$ So $190$ cm sits exactly $2\sigma$ above the mean.

From L2.1, $190\text{ cm}\Rightarrow z=2$. Because $Z\sim \mathcal{N}(0,1)$, we read the tail straight off the $\Phi$ picture: $P(H>190)=P(Z>2)=1-\Phi (2)\approx 1-0.9772=\mathrm{0.0228.}$ About 2.28% — roughly 1 in 44. Why $1-\Phi$? $\Phi (2)$ is the green area to the left of $z=2$ in the figure above. We want the right-hand piece, and the whole area is $1$, so we subtract.

Use affine closure: $Y=aX+b\Rightarrow Y\sim \mathcal{N}(a\mu +b,a^2{\sigma }^2)$ with $a=2,b=-1$.

• New mean: $a\mu +b=2(3)-1=5$.
• New variance: $a^2{\sigma }^2=2^2\cdot 4=16$. $Y\sim \mathcal{N}(5,16).$ Sanity check: the spread scales by $|a|=2$, so $\sigma$ goes $2\to 4$, i.e. variance $4\to 16$. ✓

$\sigma =\sqrt{25}=5$. The 68–95–99.7 rule says $\approx 95.4\%$ of the mass lies within $\mu \pm 2\sigma$. $[\mu -2\sigma ,\mu +2\sigma ]=[70-10,70+10]=[60,80].$ Why $2\sigma$ and not $2$ units? The rule is stated in units of $\sigma$, so we scale by the actual $\sigma =5$ before adding to the mean.

Step 1 — sum (additive closure). Means add, and for independent variables variances add: $S={\sum }_{i=1}^4{X}_i\sim \mathcal{N}(4\cdot 20,4\cdot 0.8)=\mathcal{N}(80,3.2).$ Step 2 — divide (affine closure with $a=\frac{1}{4}$). $\bar{X}=\frac{1}{4}S\sim \mathcal{N}\left(\frac{80}{4},{\left(\frac{1}{4}\right)}^2\cdot 3.2\right)=\mathcal{N}\left(20,\frac{3.2}{16}\right)=\mathcal{N}(20,0.2).$ The mean stays $20$; the variance shrinks to $0.2$ — averaging reduces spread. This is exactly the seed of the Central Limit Theorem.

$P(-1\le X\le 2)=\Phi (2)-\Phi (-1).$ By symmetry $\Phi (-1)=1-\Phi (1)=1-0.8413=0.1587$. $=0.9772-0.1587=\mathrm{0.8185.}$ Why $\Phi (2)-\Phi (-1)$? A CDF gives area to the left; subtracting the left endpoint's area leaves exactly the strip between them.

Step 1 — write $D$ as a linear combination. $D=A+(-1)B$. Step 2 — mean. ${\mu }_D=50-48=2$. Step 3 — variance. Independence means variances add, and the $-1$ contributes $(-1{)}^2=1$, so: ${\sigma }_D^2=1^2\cdot 4+(-1{)}^2\cdot 5=4+5=9.$ Note the variances add even though we subtract the variables — subtraction cannot make spread smaller. So $D\sim \mathcal{N}(2,9)$, ${\sigma }_D=3$. Step 4 — standardize and read the tail. Since $D\sim \mathcal{N}(2,9)$, standardizing gives a $Z\sim \mathcal{N}(0,1)$: $P(D<0)=P\left(Z<\frac{0-2}{3}\right)=\Phi (-0.667)=1-\Phi (0.667)\approx 1-0.7475=\mathrm{0.2525.}$ About 25.3% chance $A$ reads below $B$.

Step 1 — translate the tail into a CDF value. "Top $2.28\%$" is a right-tail: $P(X>w)=0.0228$. But our table gives $\Phi$, which is left-area. Since left-area and right-area together make the whole ($1$), the left-area up to $w$ must be $\Phi \left(\frac{w-\mu }{\sigma }\right)=1-0.0228=0.9772$. Why flip to the complement? Tables list $\Phi$ (area to the left), so we always convert a "fraction above" into "fraction below" before looking up $z$ — that is what closes the loop between the tail we care about and the CDF we can read. Step 2 — look up $z$. From the table, $\Phi (2)=0.9772\Rightarrow z=2$. Step 3 — un-standardize. Invert $z=\frac{w-\mu }{\sigma }$ with $\sigma =\sqrt{100}=10$: $\frac{w-500}{10}=2\Rightarrow w=500+2\cdot 10=520\text{ g}.$ Flag anything above $520$ g. This "find $z$ from a probability, then un-standardize" pattern powers threshold-setting in Naive Bayes Classifier and detection systems.

Where the formulas come from (intuition + calculus sketch). The likelihood is the product of the density at every data point; taking a $\ln$ turns that product into a sum, giving the log-likelihood $\ell (\mu ,{\sigma }^2)=-\frac{n}{2}\ln (2\pi )-\frac{n}{2}\ln {\sigma }^2-\frac{1}{2{\sigma }^2}{\sum }_{i=1}^n(x_i-\mu {)}^2.$ To maximize, set each partial derivative to zero.

• For $\mu$: only the last term depends on $\mu$, and $\frac{\partial }{\partial \mu }\sum (x_i-\mu {)}^2=-2\sum (x_i-\mu )$. Setting $\frac{\partial \ell }{\partial \mu }=0$ gives $\sum (x_i-\mu )=0$, i.e. $\hat{\mu }=\frac{1}{n}\sum x_i$. Intuition: the value that makes the total signed deviation cancel is precisely the balancing point — the sample mean.
• For ${\sigma }^2$: $\frac{\partial \ell }{\partial {\sigma }^2}=-\frac{n}{2{\sigma }^2}+\frac{1}{2{\sigma }^4}\sum (x_i-\hat{\mu }{)}^2=0$ solves to ${\hat{\sigma }}^2=\frac{1}{n}\sum (x_i-\hat{\mu }{)}^2$. Intuition: ${\sigma }^2$ wants to equal the average squared distance of the data from the fitted centre. (Full derivation in Maximum Likelihood Estimation.) Step 1 — $\hat{\mu }$. Sum $=2+4+4+4+5+5+7+9=40$. So $\hat{\mu }=40/8=5$. Step 2 — squared deviations from $5$: $9,1,1,1,0,0,4,16$. Sum $=32$. Step 3 — biased variance. ${\hat{\sigma }}^2=32/8=4$. For comparison, the unbiased estimate (Bessel's correction, divide by $n-1=7$) is $32/7\approx 4.571$.

Key fact. For independent variables, the MGF of the sum is the product of MGFs (the expectation of a product factorises). $M_{X+Y}(t)=M_X(t)M_Y(t)=\exp ({\mu }_{1}t+\frac{{\sigma }_1^2{t}^2}{2})\exp ({\mu }_{2}t+\frac{{\sigma }_2^2{t}^2}{2}).$ Multiply exponentials by adding exponents: $=\exp (({\mu }_1+{\mu }_2)t+\frac{({\sigma }_1^2+{\sigma }_2^2)t^2}{2}).$ This is exactly the MGF of $\mathcal{N}({\mu }_1+{\mu }_2,{\sigma }_1^2+{\sigma }_2^2)$. Since the MGF uniquely determines the distribution, the claim holds. $■$

The rule says $P(-3\le Z\le 3)=0.997$, so the total outside is $1-0.997=0.003$. By symmetry this splits equally between the two tails: $P(Z>3)=\frac{0.003}{2}=\mathrm{0.0015.}$ About $0.15\%$ — roughly 1 false alarm per 667 readings. Why halve? The Gaussian is symmetric about $0$, so the right tail and left tail carry identical mass; splitting the leftover $0.003$ evenly is exact for a symmetric distribution.

L2 transform ::: Affine closure — $aX+b\sim \mathcal{N}(a\mu +b,a^2{\sigma }^2)$ L3.2 average ::: Additive closure (sum) then affine scaling by $1/n$ L4.1 difference ::: Additive closure with coefficient $-1$; variances add L5.1 estimates ::: Maximum Likelihood: $\hat{\mu }=\bar{x}$, ${\hat{\sigma }}^2=\frac{1}{n}\sum (x_i-\hat{\mu }{)}^2$ L5.3 tail ::: Symmetry + empirical rule, halving the two-tail mass