Worked examples — Covariance and correlation

4.9.12 · D3 · Maths › Probability Theory & Statistics › Covariance and correlation

---

The scenario matrix

Neeche har problem is table ka ek cell hai. Agar ek cell filled hai, toh aapne dekh liya hai ki use kaise handle karna hai.

Cell	Relationship type	Kya special / degenerate hai	Example
A	Positive, non-extreme	ordinary discrete joint pmf	Ex 1
B	Negative	$\rho <0$, deviations disagree karte hain	Ex 2
C	Exactly zero, linear boundary	$\rho =0$ genuine symmetry se, no dependence issue	Ex 3
D	Zero cov but dependent	famous trap $Y=X^2$	Ex 4
E	Perfect $\rho =+1$	Cauchy–Schwarz ka equality case	Ex 5
F	Perfect $\rho =-1$	negative-slope exact line	Ex 5
G	Degenerate input	ek variable constant hai ($\sigma =0$) — $\rho$ undefined!	Ex 6
H	Real-world, with units	covariance kg·cm carry karta hai; $\rho$ units strip karta hai	Ex 7
I	Exam twist (toolbox, not sum)	bilinearity / $\mathrm{Var}(X+Y)$ use karo raw $E[XY]$ ki jagah	Ex 8

Kuch bhi compute karne se pehle, ek picture jo covariance ke sign ka geometrically matlab fix kare — yeh poore page ke liye hamara compass hoga.

---

Example 1 — Cell A: ordinary positive relationship

1. Marginals. Joint ko doosre variable ke upar sum karo. $P(X=1)=0.2+0.4=0.6$, toh $E[X]=0.6$. $P(Y=1)=0.1+0.4=0.5$, toh $E[Y]=0.5$. Yeh step kyun? 0/1 variable ka Expectation sirf $P(\text{variable}=1)$ hota hai; koi bhi deviation ki baat karne se pehle humein dono means chahiye.

2. $E[XY]$. Product $XY=1$ sirf tab hota hai jab dono 1 hon. $E[XY]=1\cdot P(1,1)=0.4$. Yeh step kyun? $E[XY]$ woh piece hai jo hamare working formula ko chahiye; har doosra $(x,y)$ pair product ko zero kar deta hai.

3. Covariance. $\mathrm{Cov}=E[XY]-E[X]E[Y]=0.4-0.6\cdot 0.5=0.4-0.30=0.10$. Yeh step kyun? Yeh parent se shortcut formula $E[XY]-E[X]E[Y]$ hai — haath se deviations build karne ki zaroorat nahi.

4. Standard deviations. $X$ hai Bernoulli$(0.6)$: $\mathrm{Var}(X)=0.6\cdot 0.4=0.24$, ${\sigma }_X=\sqrt{0.24}$. $Y$ hai Bernoulli$(0.5)$: ${\sigma }_Y=0.5$. Yeh step kyun? $\sigma$ covariance ko $\rho$ mein rescale karta hai.

5. Correlation. $\rho =\frac{0.10}{\sqrt{0.24}\cdot 0.5}=\frac{0.10}{0.2449}\approx 0.408$.

Verify: $\rho \approx 0.41\in [-1,1]$ ✓, positive as forecast ✓. Sanity: agar hum saara mass $(0,0)$ aur $(1,1)$ par load karte toh $\rho \to 1$ expect karte; yahan "disagree" corners abhi bhi $0.3$ mass carry karte hain, toh ek moderate $0.41$ reasonable hai.

---

Example 2 — Cell B: negative relationship

1. Marginals. $P(X=1)=0.4+0.1=0.5\Rightarrow E[X]=0.5$. Numbers ki symmetry se $E[Y]=0.5$. Yeh step kyun? Pehle jaisa hi — pehle means nikalo.

2. $E[XY]$. Sirf $(1,1)$ contribute karta hai: $E[XY]=1\cdot 0.1=0.1$.

3. Covariance. $\mathrm{Cov}=0.1-0.5\cdot 0.5=0.1-0.25=-0.15$. Yeh step kyun? Negative product-average compass confirm karta hai: disagree quadrants jeete.

4. Correlation. Dono Bernoulli$(0.5)$ hain, toh ${\sigma }_X={\sigma }_Y=0.5$. $\rho =\frac{-0.15}{0.5\cdot 0.5}=-0.6$.

Verify: $\rho =-0.6\in [-1,1]$ ✓, negative ✓. Note karo yeh parent note ke Example 1 ka mirror image hai (jiska $\rho =+0.6$ tha): agree-corners se disagree-corners par weight swap karne se poore relationship ka sign flip ho jaata hai.

---

Example 3 — Cell C: exactly zero from genuine symmetry

1. Means. $E[X]=\frac{1}{2}(-1)+\frac{1}{2}(+1)=0$, isi tarah $E[Y]=0$. Yeh step kyun? 0 ke aas paas symmetric values mean ko 0 banate hain — yeh sab kuch simplify karta hai.

2. $E[XY]$. Chaar products $(-1)(-1),(-1)(1),(1)(-1),(1)(1)=+1,-1,-1,+1$, har ek weight $1/4$: $E[XY]=\frac{1}{4}(1-1-1+1)=0$.

3. Covariance. $\mathrm{Cov}=0-0\cdot 0=0$.

4. Correlation. ${\sigma }_X,{\sigma }_Y>0$ (har ek ka variance $1$ hai), toh $\rho =0/(1\cdot 1)=0$.

Verify: $\rho =0$ ✓. Cross-check independence: $P(X=1,Y=1)=1/4$ equals $P(X=1)P(Y=1)=\frac{1}{2}\cdot \frac{1}{2}=1/4$, aur similarly saare cells ke liye — toh yahan independence genuinely hold karta hai. Is baat ko agale example se contrast karo.

---

Example 4 — Cell D: zero covariance, lekin fully dependent (the trap)

1. Mean of $X$. $E[X]=\frac{1}{4}(-2-1+1+2)=0$. Yeh step kyun? Symmetric spread $E[X]=0$ banata hai, jo correction term ko kill karega.

2. $E[XY]=E[X\cdot X^2]=E[X^3]$. Cubes: $(-8),(-1),(1),(8)$. $E[X^3]=\frac{1}{4}(-8-1+1+8)=0$. Yeh step kyun? $Y=X^2$ substitute karne se $E[XY]$ sirf $X$ ka moment ban jaata hai.

3. Covariance. $\mathrm{Cov}(X,Y)=E[XY]-E[X]E[Y]=0-0\cdot E[Y]=0$.

4. Independence check — yeh FAIL karta hai. $X=2$ lo: toh $Y=4$ certainty ke saath, toh $P(Y=4\mid X=2)=1\ne P(Y=4)=\frac{1}{2}$. $X$ jaanna $Y$ ki distribution change karta hai ⇒ not independent.

Verify: $\mathrm{Cov}=0$ ✓ phir bhi dependence total hai. Yeh parent ke [!mistake] callout se one-way street hai: independence $\Rightarrow \mathrm{Cov}=0$, lekin $\mathrm{Cov}=0\nRightarrow$ independence. Covariance relationship ke symmetric (even) part ke liye andha hai.

---

Example 5 — Cells E & F: perfect-line boundary $\rho =\pm 1$

1. Up-slope means. $\bar{x}=\frac{1+2+3+4}{4}=2.5$; $\bar{y}=\frac{5+7+9+11}{4}=8$. Yeh step kyun? Sample correlation ko deviations form karne ke liye sample means chahiye.

2. Deviation sums. $x$ ke deviations: $-1.5,-0.5,0.5,1.5$; $y$ ke: $-3,-1,1,3$ (exactly $2\times$ the $x$-deviations, kyunki slope $2$ hai). $S_{xy}=\sum (x_i-\bar{x})(y_i-\bar{y})=(-1.5)(-3)+(-0.5)(-1)+(0.5)(1)+(1.5)(3)=4.5+0.5+0.5+4.5=10$. $S_{xx}=2.25+0.25+0.25+2.25=5$; $S_{yy}=9+1+1+9=20$. Yeh step kyun? $r=S_{xy}/\sqrt{S_{xx}{S}_{yy}}$ — $\mathrm{Cov}/({\sigma }_X{\sigma }_Y)$ ka sample version.

3. Up-slope $r$. $r=\frac{10}{\sqrt{5\cdot 20}}=\frac{10}{\sqrt{100}}=\frac{10}{10}=1$.

4. Down-slope. Har $y$-deviation ka sign flip hota hai, toh $S_{xy}\to -10$ jabki $S_{xx},S_{yy}$ unchanged rehte hain (squares). $r=\frac{-10}{10}=-1$. Yeh step kyun? Toolbox ki property 5 hai — $Y$ ko negative scale se multiply karna $\mathrm{sign}(\rho )$ flip karta hai, magnitude preserved.

Verify: $r=+1$ aur $r=-1$ ✓, dono $[-1,1]$ ki boundary par. Yahi woh jagah hai jahan Cauchy–Schwarz proof mein $U-tV=0$ hota hai — deviations perfectly proportional hain. Dekho Linear Regression: ek perfect line par fitted slope data ko zero error ke saath reproduce karta hai, toh $r^2=1$.

---

Example 6 — Cell G: degenerate input, $\rho$ undefined

1. $Y$ ka deviation. $E[Y]=7$, toh $Y-{\mu }_Y=7-7=0$ har outcome ke liye. Yeh step kyun? Ek constant ka zero variance hota hai — yeh cell ki degenerate feature hai.

2. Covariance. $\mathrm{Cov}(X,Y)=E[(X-{\mu }_X)\cdot 0]=0$. Yeh step kyun? Expectation ke andar $0$ ka koi bhi factor poori cheez zero kar deta hai.

3. Correlation attempt. ${\sigma }_Y=\sqrt{\mathrm{Var}(Y)}=\sqrt{0}=0$, toh $\rho =\frac{0}{{\sigma }_X\cdot 0}=\frac{0}{0}$ — undefined (0/0, "0" nahi).

Verify: $\mathrm{Cov}=0$ ✓. Lekin mat report karo $\rho =0$: ${\sigma }_Y=0$ ke saath ratio $0/0$ hai, aur correlation simply defined nahi hota jab koi bhi variable constant ho. Yeh boundary case hai jo log silently mishandle karte hain — data mein ek constant column ki koi correlation nahi hoti, "zero correlation" nahi.

---

Example 7 — Cell H: real-world word problem with units

1. Means. $\bar{h}=\frac{150+160+170+180}{4}=165$ cm; $\bar{w}=\frac{50+60+65+75}{4}=62.5$ kg. Yeh step kyun? Deviations ko sample means chahiye.

2. Deviation products. $h$-devs: $-15,-5,5,15$; $w$-devs: $-12.5,-2.5,2.5,12.5$. $S_{hw}=(-15)(-12.5)+(-5)(-2.5)+(5)(2.5)+(15)(12.5)=187.5+12.5+12.5+187.5=400$. Sample covariance $=\frac{S_{hw}}{n-1}=\frac{400}{3}\approx 133.3$ cm·kg. Yeh step kyun? Sample covariance denominator mein $n-1$ use karta hai; product mixed units carry karta hai.

3. $r$. $S_{hh}=225+25+25+225=500$; $S_{ww}=156.25+6.25+6.25+156.25=325$. $r=\frac{400}{\sqrt{500\cdot 325}}=\frac{400}{\sqrt{162500}}\approx \frac{400}{403.11}\approx 0.9923$.

4. Metres mein switch. Heights $100$ se divide ho jaati hain: bilinearity se $\mathrm{Cov}\to \frac{1}{100}\mathrm{Cov}$, toh sample covariance $\approx 1.333$ m·kg ho jaata hai. Lekin ${\sigma }_h$ bhi $100$ se divide hota hai, toh $r$ mein factors cancel ho jaate hain: $r$ $\approx 0.9923$ hi rehta hai. Yeh step kyun? Property 5 — $\rho$ scale-invariant hai; covariance nahi.

Verify: covariance $\approx 133.3$ cm·kg (unit-bearing) ✓; $r\approx 0.99\in [-1,1]$ ✓. Yeh exactly parent ka doosra [!mistake] hai: covariance ka number ek harmless unit change mein $100\times$ badal gaya, jabki strength $r$ nahi hili. Strength ko hamesha $r$ se judge karo.

---

Example 8 — Cell I: exam twist (toolbox use karo, raw sum nahi)

1. $\mathrm{Cov}(U,V)$ bilinearity se. $\mathrm{Cov}(2X-1,-Y+5)=(2)(-1)\mathrm{Cov}(X,Y)=-2\cdot 3=-6$. Yeh step kyun? Toolbox property 2: shifts $b,d$ vanish ho jaate hain, scales $a,c$ multiply out karte hain. Kisi bhi $E[XY]$ computation se zyada faster aur cleaner.

2. Pehle ${\rho }_{XY}$. ${\rho }_{XY}=\frac{\mathrm{Cov}(X,Y)}{{\sigma }_X{\sigma }_Y}=\frac{3}{\sqrt{4}\sqrt{9}}=\frac{3}{2\cdot 3}=0.5$. Yeh step kyun? Sign rule apply karne ke liye humein yeh chahiye.

3. ${\rho }_{UV}$ scale-invariance se. $\mathrm{sign}(ac)=\mathrm{sign}(2\cdot (-1))=-1$, toh ${\rho }_{UV}=-{\rho }_{XY}=-0.5$. Yeh step kyun? Property 5 — ek negative scale sign flip karta hai, magnitude preserved.

4. $\mathrm{Var}(U+V)$. Pehle $\mathrm{Var}(U)=2^2\mathrm{Var}(X)=4\cdot 4=16$; $\mathrm{Var}(V)=(-1{)}^2\mathrm{Var}(Y)=9$. $\mathrm{Var}(U+V)=\mathrm{Var}(U)+\mathrm{Var}(V)+2\mathrm{Cov}(U,V)=16+9+2(-6)=13$. Yeh step kyun? Property 3, variance-of-a-sum identity; negative covariance sum ki spread reduce karta hai.

Verify: $\mathrm{Cov}(U,V)=-6$; ${\rho }_{UV}=-0.5\in [-1,1]$ ✓; $\mathrm{Var}(U+V)=13>0$ ✓ (variance non-negative hona chahiye). ${\rho }_{UV}$ ka direct check: $\frac{-6}{\sqrt{16}\sqrt{9}}=\frac{-6}{4\cdot 3}=-0.5$ ✓ — step 3 se match karta hai.

---

Recall Kaun sa cell kaun sa tha?

Positive ordinary ::: Ex 1 (cell A), $\rho \approx 0.41$. Negative ::: Ex 2 (cell B), $\rho =-0.6$. Honest zero (independent) ::: Ex 3 (cell C), $\rho =0$ AUR independent. Zero-cov trap (dependent) ::: Ex 4 (cell D), $\mathrm{Cov}=0$ lekin $Y=X^2$. Perfect line ::: Ex 5 (cells E,F), $r=+1$ aur $r=-1$. Constant input ::: Ex 6 (cell G), $\rho$ UNDEFINED (0/0). Units word problem ::: Ex 7 (cell H), cov units ke saath badlta hai, $r$ nahi. Toolbox twist ::: Ex 8 (cell I), bilinearity + variance-of-sum.

---

Connections

• Covariance and Correlation — yeh page uska worked-example companion hai.
• Expectation of Random Variables — yahan har mean aur $E[XY]$.
• Variance and Standard Deviation — woh $\sigma$'s jo rescale karte hain, aur degenerate $\sigma =0$ case.
• Independence of Random Variables — Ex 3 (genuine) vs Ex 4 (fails).
• Cauchy–Schwarz Inequality — Ex 5 mein $\rho =\pm 1$ boundary.
• Linear Regression — perfect line ⇒ $r^2=1$.
• Covariance Matrix — Ex 8 ka toolbox poore random vectors tak scale karta hai.