Covariance and correlation

4.9.12 · Maths › Probability Theory & Statistics

HUM MEASURE KYA KAR RAHE HAIN?

YEH EXACT FORMULA KYO? (Scratch se Derivation)

Step 1 — "Saath chalna" encode karo. Deviation $X - μ_{X}$ lo. Yeh positive hota hai jab $X$ high ho, negative jab low ho. Same $Y$ ke liye.

Step 2 — Deviations ko multiply karo. Product $(X - μ_{X}) (Y - μ_{Y})$ hai:

positive jab dono high ya dono low hon (agree karte hain),
negative jab ek high ho aur doosra low (disagree karte hain).

Yeh step kyun? Multiplication sabse sasta operation hai jo agreement ke liye $+$ aur disagreement ke liye $-$ deta hai.

Step 3 — Distribution par average lo. Expectation lo taaki agreements aur disagreements net out ho jaayein: $Cov (X, Y) = E [(X - μ_{X}) (Y - μ_{Y})] .$ Net positive ⇒ woh generally saath move karte hain.

Step 4 — Ek computational shortcut. Expand karo: $E [(X - μ_{X}) (Y - μ_{Y})] = E [X Y - X μ_{Y} - μ_{X} Y + μ_{X} μ_{Y}]$ $= E [X Y] - μ_{Y} E [X] - μ_{X} E [Y] + μ_{X} μ_{Y} = E [X Y] - μ_{X} μ_{Y} .$

CORRELATION $[- 1, 1]$ MEIN KYO REHTA HAI? (Cauchy–Schwarz)

Maano $U = X - μ_{X}$ , $V = Y - μ_{Y}$ . Kisi bhi real $t$ ke liye consider karo: $E [(U - t V)^{2}] \geq 0 (ek square kabhi negative nahi hota).$ Expand karo: $E [U^{2}] - 2 t E [U V] + t^{2} E [V^{2}] \geq 0.$ Yeh $t$ mein ek quadratic hai jo hamesha $\geq 0$ hai, isliye iska discriminant $\leq 0$ hona chahiye: $(2 E [U V])^{2} - 4 E [U^{2}] E [V^{2}] \leq 0$ $\Rightarrow (E [U V])^{2} \leq E [U^{2}] E [V^{2}] .$ Lekin $E [U V] = Cov (X, Y)$ , $E [U^{2}] = σ_{X}^{2}$ , $E [V^{2}] = σ_{Y}^{2}$ . Isliye $Cov (X, Y)^{2} \leq σ_{X}^{2} σ_{Y}^{2} \Rightarrow \frac{Cov}{σ _{X} σ _{Y}} \leq 1.$ Yeh step kyun? Equality tabhi hoti hai jab $U - t V = 0$ ho, yaani $Y$ , $X$ ka exact linear function ho — isliye $ρ = \pm 1$ ka matlab perfectly linear hota hai.

Key properties (sabhi definition se prove ho sakti hain)

Property 3 derive karo: $Var (X + Y) = Cov (X + Y, X + Y)$ ; bilinearity se expand karo $Cov (X, X) + Cov (X, Y) + Cov (Y, X) + Cov (Y, Y)$ mein. ∎

Worked examples

Common mistakes (steel-manned)

Recall Feynman: ek 12-saal ke bacche ko samjhao

Do doston ko jhoolon par imagine karo. Covariance poochhti hai: jab ek aage jhuulta hai, kya doosra bhi aage jhuulta hai? Agar haan → positive; agar ek aage jaaye aur doosra peeche → negative; agar koi pattern nahi → almost zero. Lekin "woh kitna jhuulte hain" jhoolon ki size par depend karta hai, jo compare karna unfair hai. Toh correlation ek fixed report card par unki teamwork measure karna hai $- 1$ (perfectly opposite) se $+ 1$ (perfectly saath) tak, jahaan $0$ ka matlab "koi teamwork nahi." Yeh jhuulon ki size ignore karta hai aur sirf judge karta hai ki woh kitna match karte hain.

Active recall

Covariance ko words mein define karo.

Har variable ke apne mean se deviation ke product ka average:

E [(X - μ_{X}) (Y - μ_{Y})]

Covariance ka computational formula batao.

Cov (X, Y) = E [X Y] - E [X] E [Y]

Cov (X, X)

kya hota hai?

Var (X)

— covariance, variance ko generalise karta hai.

Correlation coefficient define karo.

ρ = Cov (X, Y) / (σ_{X} σ_{Y})

ρ

hamesha

[- 1, 1]

mein kyun rehta hai?

Cauchy–Schwarz:

Cov^{2} \leq σ_{X}^{2} σ_{Y}^{2}

, kyunki

E [(U - t V)^{2}] \geq 0

ka discriminant

\leq 0

hota hai.

ρ = \pm 1

ka geometrically kya matlab hai?

Y

X

ka exact linear function hai (Cauchy–Schwarz ka equality case).

Kya

ρ = 0

independence imply karta hai?

Nahi. Yeh sirf LINEAR dependence rule out karta hai; jaise symmetric

X

ke saath

Y = X^{2}

mein

ρ = 0

hota hai phir bhi full dependence hai.

Kya independence

Cov = 0

imply karta hai?

Haan, kyunki

E [X Y] = E [X] E [Y]

a X + b

ke under covariance kaise behave karta hai?

Cov (a X + b, c Y + d) = a c Cov (X, Y)

; shifts matter nahi karte, scales multiply ho jaate hain.

Var (X + Y)

ka formula?

Var (X) + Var (Y) + 2 Cov (X, Y)

Relationships compare karne ke liye covariance ke upar correlation kyun prefer karte hain?

Correlation unitless aur

[- 1, 1]

mein bounded hai, isliye yeh scale/units se independent hokar strength measure karta hai.

Connections

Variance and Standard Deviation — covariance do-variable wala generalisation hai.
Expectation of Random Variables — yahan har formula $E [\cdot]$ se bana hai.
Independence of Random Variables — independence ⇒ zero covariance (ulta nahi).
Cauchy–Schwarz Inequality — guarantee karta hai $∣ ρ ∣ \leq 1$ .
Linear Regression — slope $= Cov (X, Y) / Var (X)$ ; $r^{2}$ goodness of fit hai.
Covariance Matrix — ek random vector ke liye saare pairwise covariances collect karta hai.