Understand correlation between instruments

What Is Correlation?

WHY this range? Correlation normalizes covariance by the volatility of each asset, creating a dimensionless measure. The math forces it between -1 and +1.

Derivation from First Principles

Let's build the correlation formula step by step.

Step 1: Define returns For two instruments A and B over n periods:

• Returns: $R_A = \{r_{A,1}, r_{A,2}, .., r_{A,n}\}$
• Returns: $R_B = \{r_{B,1}, r_{B,2}, ..., r_{B,n}\}$

Step 2: Measure how they vary together Covariance captures joint movement: $\text{Cov}(A, B) = \frac{1}{n-1} \sum_{i=1}^{n} (r_{A,i} - \bar{r}_A)(r_{B,i} - \bar{r}_B)$

WHY n-1? Bessel's correction for sample variance (unbiased estimator). We're using sample data, not the entire population.

WHAT does this mean? Each term $(r_{A,i} - \bar{r}_A)(r_{B,i} - \bar{r}_B)$ is:

• Positive when both returns are above or below their means (move together)
• Negative when one is above and one is below (move opposite)
• Zero when at mean

Step 3: Normalize by volatility Covariance depends on the assets' scales. A stock at ₹5000 has larger absolute moves than one at ₹50. We need a scale-free measure.

Standard deviation of returns: $\sigma_A = \sqrt{\frac{1}{n-1} \sum_{i=1}^{n} (r_{A,i} - \bar{r}_A)^2}$

HOW to normalize? Divide covariance by both standard deviations:

WHY does this work? Dividing by $\sigma_A \sigma_B$ scales the covariance to [-1, +1]. Think of it as measuring "covariance per unit of volatility." The Cauchy–Schwarz inequality guarantees $|\text{Cov}(A,B)| \le \sigma_A \sigma_B$, so $|\rho| \le 1$ always.

Expanded form: $\rho_{A,B} = \frac{\sum_{i=1}^{n} (r_{A,i} - \bar{r}_A)(r_{B,i} - \bar{r}_B)}{\sqrt{\sum_{i=1}^{n} (r_{A,i} - \bar{r}_A)^2} \cdot \sqrt{\sum_{i=1}^{n} (r_{B,i} - \bar{r}_B)^2}}$

Real-World Examples

Step 1: Calculate mean returns

• $\bar{r}_{\text{Infosys}} = 0.4\%$
• $\bar{r}_{\text{TCS}} = 0.5\%$

Step 2: Calculate deviations and products

• Day 1: $(2.1 - 0.4)(1.8 - 0.5) = 1.7 \times 1.3 = 2.21$
• Day 2: $(-1.5 - 0.4)(-1.2 - 0.5) = -1.9 \times -1.7 = 3.23$

WHY these multiplications? Positive products mean they moved in the same direction (both up or both down).

Step 3: Sum and divide by n-1

• $\text{Cov} = \frac{\sum (\ldots)}{19} = 2.35$ (numerator sum ≈ 44.7 over the 20 days)

Step 4: Calculate standard deviations

• $\sigma_{\text{Infosys}} = 1.8\%$
• $\sigma_{\text{TCS}} = 1.5\%$

Step 5: Compute correlation $\rho = \frac{\text{Cov}}{\sigma_A \times \sigma_B} = \frac{2.35}{1.8 \times 1.5} = \frac{2.35}{2.7} = 0.87$

WHY this step? We must keep $\rho \le 1$. Since the maximum possible covariance is $\sigma_A \sigma_B = 2.7$, the covariance cannot exceed 2.7. A correlation of 0.87 corresponds to $\text{Cov} = 0.87 \times 2.7 = 2.35$ — consistent and valid. (An earlier draft mistakenly used Cov = 3.09, which would give an impossible ρ = 1.14 > 1; that's a red flag your covariance is wrong.)

Interpretation: Strong positive correlation (0.87). When Infosys rises, TCS usually rises too. They're in the same sector, face similar macro factors.

WHAT does -0.3 mean?

• Negative: They tend to move in opposite directions
• Weak: The relationship isn't strong; gold can rise while Nifty rises too

WHY negative? Gold is a "safe haven" asset. When stock markets crash (fear), investors flee to gold. When markets boom (greed), they sell gold for equities.

Trading implication: Holding both provides diversification. If Nifty drops 10%, gold might rise 3%, cushioning your portfolio.

WHY negative? Oil is airlines' biggest cost. When crude spikes, airline profits shrink and stocks fall.

HOW to trade this?

• Hedge: Long airline stocks? Buy crude oil futures as insurance
• Pairs trade: If correlation breaks (oil up, airlines up), bet on mean reversion

Step-by-step pairs trade:

1. Identify the divergence: Oil up 20%, IndiGo unchanged (expected IndiGo down 12% based on historical ρ = -0.6)
2. Short IndiGo (expecting catch-up decline)
3. Long oil futures (protection if oil continues rising)
4. Exit when correlation normalizes

Common Mistakes

Why it feels right: Negative correlation means when one goes up, the other tends to go down. That sounds like cause and effect.

The fix: Correlation measures association, not causation. Both might be driven by a third factor (risk sentiment, inflation expectations). The correlation could even be coincidental.

Steel-man: You observe that every time gold rises, Nifty falls within days. That temporal pattern suggests causation, but correlation alone doesn't prove it. You need a mechanism: "Risk-off sentiment → sell stocks, buy gold."

Why it feels right: You calculate correlation on historical data, and it's high. Extrapolating seems reasonable.

The fix: Correlation is dynamic. It changes with:

• Market regimes: During crashes, all correlations spike toward +1 (everything falls together)
• Company events: Infosys scandal → TCS decouples
• Macro shifts: New regulations affect one sector differently

HOW to handle it: Use rolling correlation (e.g., 60-day window) to track changes. Recalculate monthly.

Why it feels right: Pearson correlation measures linear relationships. Low ρ means no straight-line pattern.

The fix: Assets can have strong non-linear relationships even with ρ near zero. Example: VIX (volatility index) and Nifty. When Nifty drops sharply, VIX spikes (non-linear, convex). Zero linear correlation does not imply statistical independence — only under joint normality does it. Two variables can be perfectly dependent yet have ρ = 0.

WHAT to do: Supplement with rank correlation (Spearman's ρ) or visual scatter plots.

Why it feels right: Diversification reduces risk. More uncorrelated assets = less risk.

The fix: Two issues:

1. Diminishing returns: Beyond 15-20 stocks, additional diversification adds minimal benefit
2. Systemic risk remains: In a market crash, correlations converge to +1. Your "diversified" portfolio crashes together.

Steel-man: You're right that low correlation helps in normal times. But you're underestimating tail risk. Better: Mix asset classes (stocks, bonds, commodities) not just stocks.

How to Use Correlation in Trading

1. Portfolio Construction

• Goal: Maximize return for given risk
• Method: Combine assets with ρ < 0.5. If A and B have ρ = 0, portfolio variance = $w_A^2\sigma_A^2 + w_B^2\sigma_B^2$ (cross-term vanishes). If ρ = 1, variance = $(w_A\sigma_A + w_B\sigma_B)^2$ (much higher).

Formula for two-asset portfolio volatility: $\sigma_P = \sqrt{w_A^2 \sigma_A^2 + w_B^2 \sigma_B^2 + 2w_A w_B \rho_{A,B} \sigma_A \sigma_B}$

WHY the cross-term? The $2w_A w_B \rho_{A,B} \sigma_A \sigma_B$ captures joint movement. When ρ = 0, this vanishes (linear risks don't reinforce).

2. Risk Management

• Concentrated risk: If all your stocks have ρ > 0.8, you're exposed to sector risk
• Hedge: Add negatively correlated assets (gold, inverse ETFs)

3. Pairs Trading

• Find highly correlated pairs (ρ > 0.9)
• When correlation temporarily breaks, bet on mean reversion
• Example: HDFC Bank and ICICI Bank usually move together. If HDFC drops 5% on earnings miss but ICICI unchanged, short ICICI (expecting catch-up decline)

4. Regime Detection

• Monitor average correlation across portfolio
• Rising correlations: Market stress (risk-off), reduce exposure
• Falling correlations: Normal market, safe to add risk

Calculation Tools

Quick correlation in Excel:

=CORREL(A2:A21, B2:B21)

Rolling correlation: Use 60-day windows, shift by 1 day, plot the time series.

Python (for algorithmic traders):

import pandas as pd
df['rolling_corr'] = df['stock_a'].rolling(60).corr(df['stock_b'])

Or: "Correlation is between -1 and 1, like my mood swings between 'ugh' and 'yay'."

If every time you trip, your friend trips too, that's perfect positive correlation (+1). You're walking in sync.

If whenever you trip, your friend stays steady (and vice versa), that's perfect negative correlation (-1). You're opposites.

If your tripping has nothing to do with your friend's tripping in a straight-line way, that's zero correlation (0). But careful — maybe you both trip only near the same weird crack in the road, just in a twisty pattern the simple "same time" rule can't catch. So zero correlation doesn't always mean truly unrelated!

In stocks, correlation tells you: "If Stock A falls, will Stock B fall too?" If they're correlated, they're like friends who trip together. If you own both, you're in double trouble. But if they're negatively correlated, one falls while the other rises—you're safer!

Connections

• Diversification-and-portfolio-theory – Correlation is the foundation of diversification; Markowitz portfolio theory uses correlation matrices
• Beta-and-systematic-risk – Beta measures correlation between a stock and the market
• Hedging-strategies – Negative correlation enables hedging (e.g., long stocks, short index futures)
• Pairs-trading – Exploits temporary correlation breakdowns in historically correlated instruments
• Volatility-and-standard-deviation – Correlation combines with volatilities to compute portfolio risk
• Covariance-matrix – Multi-asset correlation requires covariance matrices
• Asset-allocation – Strategic allocation depends on cross-asset correlations (stocks-bonds-gold)

#flashcards/stock-market

Concept Map

Hinglish (regional understanding)

Hinglish (regional understanding)