Covariance and correlation: Exercises — Hinglish deep dive

4.9.12 · D4 · Maths › Probability Theory & Statistics › Covariance and correlation

Shuru karne se pehle, ek shared reminder un do engines ka jo hum baar baar use karenge:

Level 1 — Recognition

Goal: pehchano ki question kis quantity ke baare mein pooch raha hai, aur signs padho.

L1.1 — Ek picture se covariance ka sign

Neeche figure mein points ka scatter neeche ki taraf trend karta hai (jaise $x$ badhta hai, $y$ girta hai). Bina kisi arithmetic ke, $Cov (X, Y)$ aur $ρ$ ka sign batao.

Recall Solution

Picture kya dikhata hai: jab $x$ apne mean se upar hai (vertical dashed line ke daayein) toh points apne mean se neeche baithe hain (horizontal dashed line ke neeche), aur vice-versa. Toh ek variable ka high hona doosre ke low hone ke saath pairi banta hai. Deviation product: $(x - μ_{X})$ aur $(y - μ_{Y})$ ke signs zyaadatar points ke liye opposite hain, isliye unka product $(x - μ_{X}) (y - μ_{Y})$ negative hai. Negatives ka average negative aata hai. Answer: $Cov (X, Y) < 0$ , aur kyunki $ρ$ $Cov$ ko do positive spreads $σ_{X} σ_{Y}$ se divide karta hai, woh sign waise ka waisa rakhta hai: $ρ < 0$ .

L1.2 — Kaun sa formula?

Aapko bataya gaya hai $E [X] = 2$ , $E [Y] = 3$ , $E [X Y] = 7$ . Kaunsa listed formula $Cov (X, Y)$ deta hai, aur uski value kya hai?

Recall Solution

Pehchano: hamare paas means aur product ka mean hai — yeh exactly working formula $Cov = E [X Y] - E [X] E [Y]$ hai. Compute: $7 - (2) (3) = 7 - 6 = 1$ . Answer: $Cov (X, Y) = 1$ .

L1.3 — Ek variable ka apne aap se covariance

$Cov (X, X)$ ko ek word mein simplify karo.

Recall Solution

Definition mein $Y = X$ set karo: $Cov (X, X) = E [(X - μ_{X}) (X - μ_{X})] = E [(X - μ_{X})^{2}] = Var (X)$ . Answer: yeh $X$ ka variance hai. (Covariance, variance ko generalise karta hai — dekho Variance and Standard Deviation.)

Level 2 — Application

Goal: real numbers par poori arithmetic pipeline chalao.

L2.1 — Discrete joint pmf

Ek fair setup yeh joint pmf deta hai $P (0, 0) = 0.3, P (0, 1) = 0.2, P (1, 0) = 0.2, P (1, 1) = 0.3.$ $Cov (X, Y)$ aur $ρ_{X Y}$ nikalo.

Recall Solution

Marginals (doosre variable par sum karo): $P (X = 1) = P (1, 0) + P (1, 1) = 0.2 + 0.3 = 0.5$ , toh $E [X] = 0.5$ . Symmetry se $E [Y] = 0.5$ . $E [X Y]$ : sirf corner $(1, 1)$ mein $X Y = 1$ hai; baaki sabhi corners mein $X Y = 0$ hai. Toh $E [X Y] = 1 \cdot 0.3 = 0.3$ . Covariance: $0.3 - (0.5) (0.5) = 0.3 - 0.25 = 0.05$ . Positive ⇒ halka agreement. Variances: $X$ Bernoulli $(0.5)$ hai, toh $Var (X) = 0.5 \cdot 0.5 = 0.25$ , giving $σ_{X} = 0.5$ ; likewise $σ_{Y} = 0.5$ . Correlation: $ρ = \frac{0.05}{0.5 \cdot 0.5} = \frac{0.05}{0.25} = 0.2$ . Answer: $Cov = 0.05$ , $ρ = 0.2$ .

L2.2 — Sample data

Data $(x, y)$ : $(2, 1), (4, 2), (6, 5), (8, 4), (10, 8)$ . Sample correlation $r$ compute karo.

Recall Solution

Means: $\overset{x}{ˉ} = \frac{2 + 4 + 6 + 8 + 10}{5} = 6$ , $\overset{y}{ˉ} = \frac{1 + 2 + 5 + 4 + 8}{5} = 4$ . Deviation products $\sum (x_{i} - \overset{x}{ˉ}) (y_{i} - \overset{y}{ˉ})$ : $(- 4) (- 3) + (- 2) (- 2) + (0) (1) + (2) (0) + (4) (4) = 12 + 4 + 0 + 0 + 16 = 32$ . $S_{xx} = \sum (x_{i} - \overset{x}{ˉ})^{2} = 16 + 4 + 0 + 4 + 16 = 40$ . $S_{y y} = \sum (y_{i} - \overset{y}{ˉ})^{2} = 9 + 4 + 1 + 0 + 16 = 30$ . Yeh sums kyun? Sample $r = \frac{\sum ( x - x ˉ ) ( y - y ˉ )}{S _{xx} S _{y y}}$ — wahi $E [X Y] - E [X] E [Y]$ wala idea, lekin data par (upar aur neeche ke $\frac{1}{n}$ factors cancel ho jaate hain). Answer: $r = \frac{32}{40 \cdot 30} = \frac{32}{1200} \approx 0.924$ . Strong positive.

L2.3 — Sum ka variance

$Var (X) = 4$ , $Var (Y) = 9$ , $Cov (X, Y) = - 2$ diya hua hai, $Var (X + Y)$ aur $Var (X - Y)$ nikalo.

Recall Solution

Tool (property 3): $Var (X \pm Y) = Var (X) + Var (Y) \pm 2 Cov (X, Y)$ . Sum: $4 + 9 + 2 (- 2) = 13 - 4 = 9$ . Difference: $4 + 9 - 2 (- 2) = 13 + 4 = 17$ . Answer: $Var (X + Y) = 9$ , $Var (X - Y) = 17$ .

Level 3 — Analysis

Goal: behaviour ke baare mein reason karo, sirf numbers mat ghisao.

L3.1 — Rescaling attack

$Cov (X, Y) = 3$ , $σ_{X} = 2$ , $σ_{Y} = 5$ . Ab define karo $X^{'} = 1000 X$ (units badlo, jaise kg→g). Nayi covariance $Cov (X^{'}, Y)$ aur nayi correlation $ρ_{X^{'} Y}$ nikalo. Comparison kya sikhata hai?

Recall Solution

Bilinearity (property 2): shifts vanish ho jaate hain, scales bahar aa jaate hain: $Cov (1000 X, Y) = 1000 \cdot Cov (X, Y) = 1000 \cdot 3 = 3000.$ $X^{'}$ ka naaya spread: $σ_{X^{'}} = 1000 σ_{X} = 2000$ . Nayi correlation: $ρ_{X^{'} Y} = \frac{3000}{2000 \cdot 5} = \frac{3000}{10000} = 0.3$ . Original correlation: $ρ_{X Y} = \frac{3}{2 \cdot 5} = 0.3$ . Insight: covariance $1000 \times$ explode hua ( $3$ se $3000$ tak) lekin correlation same raha ( $0.3$ ). Rescaling sirf units badalta hai, asli strength kabhi nahin — exactly isliye hum $ρ$ prefer karte hain. Answer: $Cov (X^{'}, Y) = 3000$ , $ρ_{X^{'} Y} = 0.3 = ρ_{X Y}$ .

L3.2 — Zero covariance, poori dependence

Maano $X$ values $- 2, - 1, 1, 2$ lete hai har ek probability $1/4$ ke saath, aur set karo $Y = ∣ X ∣$ . Dikhao ki $Cov (X, Y) = 0$ phir bhi $X, Y$ independent nahin hain.

Recall Solution

Means: $E [X] = \frac{1}{4} (- 2 - 1 + 1 + 2) = 0$ ( $0$ ke baare mein symmetric). $E [X Y]$ : $X Y = X ∣ X ∣$ , values $(- 2) (2), (- 1) (1), (1) (1), (2) (2) = - 4, - 1, 1, 4$ . Average $= \frac{1}{4} (- 4 - 1 + 1 + 4) = 0$ . Covariance: $E [X Y] - E [X] E [Y] = 0 - 0 \cdot E [Y] = 0$ . Lekin dependent hai: $X = 2$ jaanna $Y = 2$ force karta hai; $Y = 1$ jaanna batata hai $X \in {- 1, 1}$ . Toh $P (Y = y ∣ X = x) \neq = P (Y = y)$ — yeh information share karte hain. Kyun covariance miss kar jaata hai: covariance sirf relationship ka linear part detect karta hai (dekho Independence of Random Variables). Yahan link $Y = ∣ X ∣$ symmetric hai, toh uske "up-slope" aur "down-slope" halves cancel ho jaate hain. Answer: $Cov = 0$ , phir bhi independent nahin. ∎

L3.3 — Equality case padhna

Maano $Y = 3 X - 7$ exactly (ek deterministic line). Compute karne se pehle $ρ_{X Y}$ predict karo, phir tools se verify karo.

Recall Solution

Prediction: $Y$ ek perfect increasing linear function hai $X$ ki, toh Cauchy–Schwarz equality case deta hai $ρ = + 1$ . Verify: $Cov (X, 3 X - 7) = 3 Cov (X, X) = 3 Var (X)$ (shift $- 7$ bahar ho jaata hai). Spread: $σ_{Y} = Var (3 X - 7) = 9 Var (X) = 3 σ_{X}$ . Phir $ρ = \frac{3 Var ( X )}{σ _{X} \cdot 3 σ _{X}} = \frac{3 σ _{X}^{2}}{3 σ _{X}^{2}} = 1$ . (Agar slope negative hota, jaise $Y = - 3 X + 7$ , toh same steps $ρ = - 1$ dete.) Answer: $ρ_{X Y} = + 1$ .

Level 4 — Synthesis

Goal: ek argument mein kai properties chain karo.

L4.1 — Ek part ke saath sum ki correlation

$X$ aur $Z$ independent hain aur $Var (X) = Var (Z) = 1$ . Define karo $Y = X + Z$ . $ρ_{X Y}$ nikalo.

Recall Solution

Covariance: $Cov (X, Y) = Cov (X, X + Z) = Cov (X, X) + Cov (X, Z)$ bilinearity se. Independence ⇒ $Cov (X, Z) = 0$ , aur $Cov (X, X) = Var (X) = 1$ . Toh $Cov (X, Y) = 1$ . $Y$ ka spread: $Var (Y) = Var (X) + Var (Z) + 2 Cov (X, Z) = 1 + 1 + 0 = 2$ , toh $σ_{Y} = 2$ . Correlation: $ρ_{X Y} = \frac{1}{σ _{X} σ _{Y}} = \frac{1}{1 \cdot 2} = \frac{1}{2} \approx 0.707$ . Answer: $ρ_{X Y} = \frac{1}{2} \approx 0.707$ . (Ek part apne noisy total se correlated hota hai, lekin kabhi perfectly nahin, kyunki $Z$ use dilute karta hai.)

L4.2 — Regression slope link

L2.2 ke data ke liye, $y$ ko $x$ se predict karne wali least-squares line ka slope $b = Cov / Var (X)$ hai sample form mein, yaani $b = S_{x y} / S_{xx}$ . Slope $b$ , intercept $a = \overset{y}{ˉ} - b \overset{x}{ˉ}$ compute karo, aur confirm karo ki $r^{2}$ us fraction ke barabar hai jitna $y$ -variance line explain karti hai.

Recall Solution

L2.2 se: $S_{x y} = 32$ , $S_{xx} = 40$ , $S_{y y} = 30$ , $\overset{x}{ˉ} = 6$ , $\overset{y}{ˉ} = 4$ . Slope: $b = \frac{32}{40} = 0.8$ . Intercept: $a = \overset{y}{ˉ} - b \overset{x}{ˉ} = 4 - 0.8 \cdot 6 = 4 - 4.8 = - 0.8$ . Goodness of fit: $r^{2} = (\frac{S _{x y}}{S _{xx} S _{y y}})^{2} = \frac{S _{x y}^{2}}{S _{xx} S _{y y}} = \frac{3 2 ^{2}}{40 \cdot 30} = \frac{1024}{1200} \approx 0.853$ . Matlab: $y$ ke upar-neeche spread ka lagbhag $85.3%$ $x$ par straight-line dependence se explain ho jaata hai (dekho Linear Regression). Answer: $b = 0.8$ , $a = - 0.8$ , $r^{2} \approx 0.853$ .

L4.3 — Covariance matrix banana

Random vector $(X, Y)$ ke liye jo aapne L2.1 mein nikala ( $Var (X) = Var (Y) = 0.25$ , $Cov (X, Y) = 0.05$ ), $2 \times 2$ Covariance Matrix $Σ$ likho aur check karo ki woh symmetric hai aur diagonal par variances hain.

Recall Solution

Definition: $Σ = (Var (X) Cov (Y, X) Cov (X, Y) Var (Y))$ . Fill in: $Σ = (0.25 0.05 0.05 0.25) .$ Checks: diagonal variances $0.25, 0.25$ hold karta hai; off-diagonal entries equal hain ( $0.05 = 0.05$ ) kyunki $Cov (X, Y) = Cov (Y, X)$ (symmetry). ✓ Answer: upar wala matrix.

Level 5 — Mastery

Goal: ek general fact prove karo, koyi numbers nahi lean karne ke liye.

L5.1 — Prove karo ki $ρ$ scale-invariant hai ek sign flip ke saath

Dikhao ki constants $a \neq = 0, c \neq = 0$ aur kisi bhi shifts $b, d$ ke liye: $ρ_{a X + b, c Y + d} = sign (a c) ρ_{X Y} .$

Recall Solution

Numerator (bilinearity, property 2): shifts vanish ho jaate hain, scales bahar aa jaate hain: $Cov (a X + b, c Y + d) = a c Cov (X, Y) .$ Denominator (spreads): $σ_{a X + b} = Var (a X + b) = a^{2} Var (X) = ∣ a ∣ σ_{X}$ , aur likewise $σ_{c Y + d} = ∣ c ∣ σ_{Y}$ . Assemble: $ρ^{'} = \frac{a c Cov ( X , Y )}{∣ a ∣ ∣ c ∣ σ _{X} σ _{Y}} = \frac{a c}{∣ a c ∣} \cdot \frac{Cov ( X , Y )}{σ _{X} σ _{Y}} = sign (a c) ρ_{X Y} .$ Padhna: koi bhi axis stretch karna strength $∣ ρ ∣$ fix rakhta hai; ek axis flip karna ( $a < 0$ ya $c < 0$ ) sign flip karta hai. Isliye correlation ek honest, unit-free strength score hai. ∎

L5.2 — Scratch se Cauchy–Schwarz bound $∣ ρ ∣ \leq 1$ prove karo

Sirf $E [(U - t V)^{2}] \geq 0$ sabhi real $t$ ke liye use karke, prove karo $Cov (X, Y)^{2} \leq σ_{X}^{2} σ_{Y}^{2}$ , aur precisely batao ki equality kab hold karta hai.

Recall Solution

Set-up: let $U = X - μ_{X}$ , $V = Y - μ_{Y}$ , toh $E [U^{2}] = σ_{X}^{2}$ , $E [V^{2}] = σ_{Y}^{2}$ , $E [U V] = Cov (X, Y)$ . Ek square kabhi negative nahin hota: har real $t$ ke liye, $0 \leq E [(U - t V)^{2}] = E [U^{2}] - 2 t E [U V] + t^{2} E [V^{2}] .$ Ise $t$ mein ek quadratic ki tarah padho: yeh $q (t) = E [V^{2}] t^{2} - 2 E [U V] t + E [U^{2}]$ hai, upar ki taraf open hota hai aur kabhi $0$ se neeche nahin jaata. Ek quadratic jo $\geq 0$ rehta hai uska discriminant $\leq 0$ hota hai: $(2 E [U V])^{2} - 4 E [V^{2}] E [U^{2}] \leq 0 \Rightarrow E [U V]^{2} \leq E [U^{2}] E [V^{2}] .$ Wapis translate karo: $Cov (X, Y)^{2} \leq σ_{X}^{2} σ_{Y}^{2}$ , hence $\frac{Cov}{σ _{X} σ _{Y}} = ∣ ρ ∣ \leq 1$ . Equality: discriminant $0$ exactly tab hota hai jab $q (t)$ ka ek real double root $t^{⋆}$ ho, yaani $E [(U - t^{⋆} V)^{2}] = 0$ , matlab $U = t^{⋆} V$ probability $1$ ke saath — yaani $X - μ_{X}$ exactly $Y - μ_{Y}$ ka ek multiple hai. Geometrically $Y$ ek perfect linear function hai $X$ ki, jo $ρ = \pm 1$ case hai. ∎ (Dekho Cauchy–Schwarz Inequality.)

Connections

Covariance and Correlation — parent note jise yeh exercises drill karte hain.
Expectation of Random Variables — har step $E [\cdot]$ se bana hua hai.
Variance and Standard Deviation — L1.3, L2.3, L4.1 mein use hua.
Independence of Random Variables — $ρ = 0 \neq \Rightarrow$ independent theme (L3.2, L4.1).
Cauchy–Schwarz Inequality — L5.2 mein prove kiya gaya.
Linear Regression — slope aur $r^{2}$ L4.2 mein.
Covariance Matrix — L4.3 mein assemble kiya gaya.