6.2.10Backtesting Frameworks

Learn Monte Carlo simulation of returns

2,495 words11 min readdifficulty · medium

Core Concept

Why this approach?

  1. Reality check: Historical backtest = sample size of 1. Monte Carlo = sample size of 10,000.
  2. Tail risk: Reveals the worst5% of scenarios that might not appear in your data window.
  3. Robustness: If your strategy fails in 40% of simulations, it's fragile—even if the historical backtest looked good.

Mathematical Foundation

Geometric Brownian Motion (GBM)

Most Monte Carlo return simulations use Geometric Brownian Motion, the standard model for stock prices:

dSt=μStdt+σStdWtdS_t = \mu S_t \, dt + \sigma S_t \, dW_t

Decode each term:

  • StS_t = stock price at time tt
  • μ\mu = drift (expected annualized return, e.g., 0.08 for 8%)
  • σ\sigma = volatility (annualized standard deviation of returns, e.g., 0.20 for 20%)
  • dWtdW_t = Wiener process increment (random shock from normal distribution)
  • dtdt = tiny time step

Why this form? The StS_t multiplier makes returns proportional: a 10% move on a 100stockis100 stock is 10, but on a 200stockits200 stock it's 20. This matches reality—stocks move in percentages, not fixed dollars.

Discrete-Time Solution

We can't simulate continuous dtdt, so we discretize. The analytical solution over one time step Δt\Delta t is:

St+Δt=Stexp[(μσ22)Δt+σΔtZ]S_{t+\Delta t} = S_t \exp\left[\left(\mu - \frac{\sigma^2}{2}\right)\Delta t + \sigma \sqrt{\Delta t} \, Z\right]

where ZN(0,1)Z \sim \mathcal{N}(0, 1) is a standard normal random variable.

Derivation from scratch:

  1. Apply Itô's lema to lnSt\ln S_t:

    d(lnSt)=1StdSt12St2(dSt)2d(\ln S_t) = \frac{1}{S_t} dS_t - \frac{1}{2S_t^2} (dS_t)^2
  2. Substitute the GBM equation:

    d(lnSt)=μdt+σdWtσ22dt=(μσ22)dt+σdWtd(\ln S_t) = \mu \, dt + \sigma \, dW_t - \frac{\sigma^2}{2} dt = \left(\mu - \frac{\sigma^2}{2}\right) dt + \sigma \, dW_t

    Why the σ2/2-\sigma^2/2 term? This is the Itô correction. Variance itself contributes to drift because (σStdWt)2=σ2St2dt(\sigma S_t dW_t)^2 = \sigma^2 S_t^2 dt (quadratic variation). Without it, we'd overestimate growth.

  3. Integrate from tt to t+Δtt + \Delta t:

    lnSt+ΔtlnSt=(μσ22)Δt+σ(Wt+ΔtWt)\ln S_{t+\Delta t} - \ln S_t = \left(\mu - \frac{\sigma^2}{2}\right)\Delta t + \sigma (W_{t+\Delta t} - W_t)
  4. Random walk property: Wt+ΔtWtN(0,Δt)W_{t+\Delta t} - W_t \sim \mathcal{N}(0, \Delta t), so:

    Wt+ΔtWt=ΔtZ,ZN(0,1)W_{t+\Delta t} - W_t = \sqrt{\Delta t} \, Z, \quad Z \sim \mathcal{N}(0, 1)
  5. Exponentiate both sides:

    St+Δt=Stexp[(μσ22)Δt+σΔtZ]S_{t+\Delta t} = S_t \exp\left[\left(\mu - \frac{\sigma^2}{2}\right)\Delta t + \sigma \sqrt{\Delta t} \, Z\right]

Units check:

  • μ\mu and σ\sigma are annualized, Δt\Delta t is in years (e.g., 1/252 for daily).
  • σΔt\sigma \sqrt{\Delta t} has units of yearyear1/2=1\sqrt{\text{year}} \cdot \text{year}^{1/2} = 1 (dimensionless).

Estimating Parameters from Data

Given historical daily returns r1,r2,,rnr_1, r_2, \ldots, r_n:

  1. Sample mean (annualized):

    μ^=2521ni=1nri\hat{\mu} = 252 \cdot \frac{1}{n}\sum_{i=1}^n r_i
  2. Sample volatility (annualized):

    σ^=2521n1i=1n(rirˉ)2\hat{\sigma} = \sqrt{252} \cdot \sqrt{\frac{1}{n-1}\sum_{i=1}^n (r_i - \bar{r})^2}

Why multiply by 252? Returns scale linearly with time (adding independent increments), but volatility scales with time\sqrt{\text{time}} (variance adds, SD is square root of variance).

Why n1n-1 for volatility? Bessel's correction for unbiased sample variance.

Simulation Algorithm

Setup:

  • Historical data: SPY daily returns, 2010–2023
  • Estimated: μ^=0.10\hat{\mu} = 0.10, σ^=0.18\hat{\sigma} = 0.18
  • Simulation horizon: 252 days (1 year)
  • Number of paths: 10,000

Algorithm:

For each simulation path i = 1 to 10,000:
    1. Set S[0] = current_price (e.g., $450)
    2. For each day t = 0 to 251:
        a. Draw Z ~ N(0, 1)
        b. S[t+1] = S[t] * exp((μ - σ²/2)/252 + σ/√252 * Z)
    3. Record final value S[252] and path statistics
    
Calculate percentiles of final values:5th, 50th, 95th

Why this step? Each path is an independent "what-if" scenario. The collection shows the range of outcomes your strategy might face.

Given:

  • S0=450S_0 = 450, μ=0.10\mu = 0.10, σ=0.18\sigma = 0.18
  • Random draws: Z1=0.523Z_1 = -0.523, Z2=1.246Z_2 = 1.246

Day 1:

S1=450exp[(0.100.1822)1252+0.181252(0.523)]=450exp[0.003330.005926]=450exp(0.005593)=4500.944=447.5\begin{align} S_1 &= 450 \exp\left[\left(0.10 - \frac{0.18^2}{2}\right)\frac{1}{252} + 0.18\sqrt{\frac{1}{252}}(-0.523)\right] \\ &= 450 \exp\left[0.00333 - 0.005926\right] \\ &= 450 \exp(-0.005593) \\ &= 450 \cdot 0.944 = 447.5 \end{align}

Why the negative exponent? The negative Z1Z_1 means below-average return this day (bad luck).

Day 2:

S2=447.5exp[0.0003333+0.181252(1.246)]=447.5exp(0.01409)=447.51.0142=453.8\begin{align} S_2 &= 447.5 \exp\left[0.0003333 + 0.18\sqrt{\frac{1}{252}}(1.246)\right] \\ &= 447.5 \exp(0.01409) \\ &= 447.5 \cdot 1.0142 = 453.8 \end{align}

Repeat for 252 days × 10,000 paths.

Interpreting Results

  1. Expected terminal value: Mean of all final prices

    E[ST]=S0eμT\mathbb{E}[S_T] = S_0 e^{\mu T}
  2. Value at Risk (95% VaR): The 5th percentile loss

    • If5th percentile is 380,VaR=450380=380, VaR = 450 - 380 = 70
  3. Probability of profit: Fraction of paths where ST>S0S_T > S_0

  4. Maximum drawdown distribution: Percentiles of worst peak-to-trough decline across paths

Why these matter?

  • Mean tells you average outcome (but averages lie—you experience ONE path, not the average).
  • VaR quantifies downside: "In 19out of 20 scenarios, I won't lose more than this."
  • Profit probability is intuitive for decision-makers.

Scenario: You have a momentum strategy: "Buy when 50-day MA > 200-day MA."

Historical backtest (2015–2023): 12% annual return, max drawdown 18%.

Monte Carlo validation:

  1. Estimate μ\mu, σ\sigma from historical returns.
  2. Simulate 10,000 price paths.
  3. Apply your strategy rules to each path.
  4. Record return and max drawdown for each simulation.

Results:

  • Median return: 9% (worse than historical12%—overfitting alert!)
  • 5th percentile return: -15% (your strategy can blow up badly)
  • In 2,300 paths (23%), max drawdown exceeded 30%

Why this step matters: The historical backtest was lucky. Monte Carlo reveals your strategy is riskier than it appeared. Now you can adjust position sizing or add stop-losses.

Common Mistakes

Why it feels right: Simple multiplication, matches intuition about "average."

Why it's wrong: Returns compound multiplicatively, not additively. The geometric mean accounts for volatility drag. If you gain 50% then lose 50%, you're down 25%, not flat.

The fix:

μcorrect=ln(i=1n(1+ri))252nrˉ252+σ22\mu_{\text{correct}} = \ln\left(\prod_{i=1}^n (1 + r_i)\right) \cdot \frac{252}{n} \approx \bar{r} \cdot 252 + \frac{\sigma^2}{2}

or use the log-returns formula from GBM directly.

Why it feels right: "Just add drift plus random shock."

Why it's wrong: This systematically overestimates price growth. The missing σ2/2-\sigma^2/2 term accounts for the fact that variance itself acts as a drag (Jensen's inequality for exponentials).

Result: Your simulated returns will be ~σ2/2\sigma^2/2 too high annually. For σ=0.20\sigma = 0.20, that's 2% annual overestimation—huge over time!

Why standard GBM fails: The2008 crash was a "20-sigma event" under normal assumptions—should happen once per universe lifetime. But it happened.

The fix: Use Student's t-distribution for ZZ (heavier tails), or jump-diffusion models that add sudden crash components:

dSt=μStdt+σStdWt+StdJtdS_t = \mu S_t dt + \sigma S_t dW_t + S_t dJ_t

where JtJ_t is a Poisson jump process (rare, large shocks).

Recall Feynman Explain-to-a-12-Year-Old

Imagine you're playing a video game where your character's health bar goes up and down randomly. Monte Carlo simulation is like playing that game 10,000 times to see: "How often do I survive to the end? What's my health usually at? What's the worst that happened?"

For stocks, we don't know the EXACT future, but we know the "game rules" from history: stocks tend to go up about 8% per year on average, but they wigle up and down about 20% along the way (that's called volatility). So we use a computer to play10,000 "pretend futures" that follow those same rules—same average, same wigliness—but each time the random wiggles are different.

Why? Because if you only look at what ACTUALLY happened (one game), you might've gotten lucky. Maybe your stock went up 20% but in8 out of 10 alternate universes it would've crashed. Monte Carlo shows you all those alternate universes so you know if your strategy is REALLY good or just got lucky.

Active Recall Flashcards

#flashcards/stock-market

What is the purpose of Monte Carlo simulation in backtesting? :: To generate thousands of plausible alternate price paths with the same statistical properties (mean, volatility) as historical data, revealing the distribution of outcomes instead of relying on a single historical backtest.

What is Geometric Brownian Motion (GBM)?
The stochastic differential equation dSt=μStdt+σStdWtdS_t = \mu S_t dt + \sigma S_t dW_t used to model stock prices, where returns are proportional to price and follow a log-normal distribution.
Write the discrete-time GBM formula for simulating the next price.
St+Δt=Stexp[(μσ2/2)Δt+σΔtZ]S_{t+\Delta t} = S_t \exp\left[(\mu - \sigma^2/2)\Delta t + \sigma\sqrt{\Delta t} Z\right] where ZN(0,1)Z \sim \mathcal{N}(0,1).
Why is the σ2/2-\sigma^2/2 term necessary in the GBM drift?
It's the Itô correction. Variance itself contributes to drift due to quadratic variation of the Wiener process. Without it, simulated prices grow too fast (violates martingale property under risk-neutral measure).
How do you annualize daily volatility?
Multiply by 252\sqrt{252}: σannual=σdaily×252\sigma_{\text{annual}} = \sigma_{\text{daily}} \times \sqrt{252}. Volatility scales with the square root of time because variances add for independent increments.
What is Value at Risk (VaR) at 95% confidence?
The loss threshold at the 5th percentile of the simulated outcome distribution. Example: if 5th percentile terminal value is $380 starting from $450, VaR = $70.
Why might a strategy that backtests well fail in Monte Carlo validation?
The historical backtest is ONE lucky path. Monte Carlo reveals the strategy may fail in 30-40% of equally plausible alternate scenarios, indicating overfitting or fragility.
What is the main limitation of GBM for stock simulation?
It assumes normally distributed returns, missing fat tails (extreme events). Real markets have more frequent large crashes than GBM predicts (Black Swans).
How do you fix the fat-tails problem in Monte Carlo?
Use Student's t-distribution for random shocks instead of normal, or add a jump-diffusion component (Poisson jumps) to model sudden crashes.
What is the geometric mean return, and why does it differ from arithmetic mean?
Geometric mean accounts for compounding: μgeom=ln((1+ri)/n\mu_{\text{geom}} = \ln(\prod(1+r_i)/n. It's lower than arithmetic mean due to volatility drag (a 50% gain + 50% loss = -25% total, not 0%).

Connections

  • 6.2.1-Walk-forward-analysis: Use Monte Carlo to simulate future out-of-sample periods for walk-forward testing.
  • 6.2.8-Sharpe-ratio-calculation: Calculate Sharpe ratio distribution across Monte Carlo paths to assess consistency.
  • 6.1.4-Maximum-drawdown-analysis: Monte Carlo generates the full drawdown distribution, not just historical max.
  • 5.3.5-Value-at-Risk-VaR: VaR is directly computed from Monte Carlo percentiles.
  • 4.2.3-Volatility-clusteringGARCH: Advanced models use GARCH for time-varying σt\sigma_t in simulations.
  • 7.1.2-Position-sizing-Kelly-criterion: Monte Carlo tests position sizes across many paths to optimize Kelly fraction.
  • 2.1.6-Log-returns-vs-simple-returns: GBM naturally works with log-returns; ensure data preprocessing matches.

"In God we trust. All others must bring Monte Carlo simulations." — Adapted from W. Edwards Deming

Concept Map

single lucky path

mean drift and volatility

models prices via

discretized into

needs sigma^2/2

generates

aggregate into

reveals

assesses

fragile if fails often

Monte Carlo Simulation

Historical Backtest n=1

Statistical Properties

Geometric Brownian Motion

Discrete Solution

Ito Lemma Correction

Thousands of Price Paths

Distribution of Outcomes

Tail Risk 5% Scenarios

Strategy Robustness

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Monte Carlo simulationek powerful technique hai jo tumhe yeh samajhne mein mad karta hai kiagar history thodi alag hoti, toh tumhara trading strategy kaisa perform karta. Socho tumne 2015 se 2023 tak apni strategy ka backtest kiya aur usme acha return mila—lekin kya yeh luck tha ya skill? Monte Carlo yeh answer deta hai.

Iska kaam simple hai: hum historical data se do chezein nikalte hain—average return (mu) aur volatility (sigma). Phir computer se 10,000 fake price paths generate karte hain jo same statistical properties follow karte hain. Matlab har path realistic hai, bas random wiggles alag hain. Har path par tumhari strategy chalao aur dekho kitne paths mein profit hua, kitne mein loss. Agar 30% paths mein tumhara strategy fail ho raha hai toh samajh jao ki backtest lucky tha, strategy weak hai.

GBM (Geometric Brownian Motion) formula use hota hai: price exponential grow karti hai with drift aur random shocks. Ek important term hai minus sigma-squared by 2—yeh volatility drag ko account karta hai.Agar yeh miss karo toh tumhare simulated returns artificially high ho jayenge aur tum overconfident feel karoge. Monte Carlo ka sabse bada fayda yeh hai ki tum worst-case scenarios dekh sakte ho (5th percentile), not just average. Risk management ke liye yeh gold hai—tumhe pata chal jayega ki maximum kitna loss ho sakta hai 95% confidence ke sath.

Real markets mein fat tails hote hain (extreme events zyada frequent hain normal distribution ke comparison mein), toh advanced simulations mein Student's t-distribution ya jump-diffusion models use karte hain. But basic GBM bhi kafi useful hai most strategies ke liye. Yad rakho: ek backtest = ek dice roll, Monte Carlo =10,000 dice rolls. Probability tumhare sath hai toh hi trade karo!

Test yourself — Backtesting Frameworks