Backtesting Frameworks

Question 1 (14 marks)

(a) CAGR — (4) Formula: $\text{CAGR} = (V_f/V_i)^{1/n} - 1$, with $n=3$ years. $\text{CAGR} = (180000/100000)^{1/3} - 1 = 1.8^{0.3333} - 1$ $1.8^{1/3} = 1.21644$, so CAGR $= 0.21644 = \mathbf{21.64\%}$.

• 1 mark formula, 1 mark $n=3$, 1 mark computation, 1 mark % answer.

(b) Max Drawdown — (4) Track running peak, find largest peak→trough decline.

• Peak = 130,000 (2021-07); subsequent trough = 90,000 (2022-01). $\text{MDD} = (90000 - 130000)/130000 = -40000/130000 = -0.30769 = \mathbf{-30.77\%}$ (Later drop 150k→120k = −20% is smaller.) Peak = $130,000, Trough = $90,000.
• 2 marks correct peak/trough, 2 marks computed −30.77%.

(c) Sharpe — (3) $\text{Sharpe} = \frac{R_p - R_f}{\sigma} = \frac{0.2164 - 0.03}{0.30} = \frac{0.1864}{0.30} = \mathbf{0.621}$

• 1 mark formula, 1 mark numerator, 1 mark answer ≈ 0.62.

(d) — (3) CAGR ignores risk/path. Two identical CAGRs can have very different volatility and drawdown; a strategy with 30% CAGR and −60% MDD is far worse risk-adjusted than one with 20% CAGR and −10% MDD. Should also report Sharpe (risk-adjusted return) and/or MDD to capture volatility and downside path. (3 marks: mention risk-adjustment + name Sharpe/MDD + coherent reasoning.)

Question 2 (13 marks)

(a) Commission per round-trip — (3) Shares per trade = $50,000 / $100 = 500 shares. Commission per side = 500 × $0.005 = $2.50. Round-trip (buy + sell) = $5.00.

• 1 mark shares, 1 mark per side, 1 mark round-trip.

(b) Slippage per round-trip — (3) Half-spread = ½ × 4 bp = 2 bp = 0.0002 of price. Cost per side = 0.0002 × $50,000 = $10. Round-trip (both sides) = $20.00.

• 1 mark half-spread, 1 mark per side, 1 mark round-trip = $20.

(c) Total annual cost — (4) Fees per side = 0.001% × $50,000 = 0.00001 × 50000 = $0.50; both sides = $1.00/round-trip. Per round-trip total = commission $5 + fees $1 + slippage $20 = $26. Annual (250 round-trips) = 250 × $26 = $6,500.

• 1 mark fees, 1 mark per-round-trip sum, 2 marks × 250 = $6,500.

(d) Net return — (3) Net profit = $60,000 − $6,500 = $53,500. Net return = 53,500 / 50,000 = 1.07 = 107%. Comment: Costs consume $6,500 (~10.8% of gross profit); strategy remains strongly profitable and survives cost modelling. (1 mark net profit, 1 mark %, 1 mark comment.)

Question 3 (12 marks)

(a) — (4) (i) Survivorship bias — using today's constituents omits companies that were delisted/removed/bankrupt during the test window, inflating returns. (2) (ii) Correction: use a point-in-time / historical constituent database that includes delisted and removed tickers as they existed on each historical date. (2)

(b) — (4) (i) Look-ahead bias — executing at today's close using a signal that itself uses today's close assumes information available only at end-of-day is tradeable at that same instant (and you can't act on the close you're computing from). (2) (ii) Correction: execute at the next bar's open (or next-day price), lagging the signal by one period so only past-available data drives trades. (2)

(c) — (4) (i) Overfitting / in-sample (data-snooping) bias — reporting the optimised in-sample Sharpe overstates expected live performance. (2) (ii) Correction: reserve out-of-sample data (or walk-forward/cross-validation) and report performance on data never used for optimisation. (2)

Question 4 (11 marks)

(a) Walk-forward scheme — (5) 3-yr train + 1-yr test, rolling by 1 year:

7 out-of-sample folds.

• 3 marks correct rolling structure, 2 marks count = 7 with ranges.

(b) Degradation ratio — (3) $\text{ratio} = 0.4 / 2.1 = 0.190$ (≈ 19%). Interpretation: OOS retains only ~19% of IS performance — severe degradation indicating overfitting; the strategy is unlikely to generalise. (1 mark ratio, 2 marks interpretation.)

(c) — (3) A single split trains on one regime and tests on one regime; results depend heavily on where the split falls and can be luck. Walk-forward tests across multiple, sequential out-of-sample periods spanning different market regimes (bull, bear, high/low vol), giving a more robust, regime-diverse estimate of live robustness. (3 marks.)

Question 5 (10 marks)

(a) Annualised return — (3) $(1.0008)^{252} - 1 = 1.22326 - 1 = 0.22326 = \mathbf{22.33\%}$

• 1 mark formula, 1 mark exponent, 1 mark % (≈ 22.33%).

(b) Annualised volatility — (2) $0.012 \times \sqrt{252} = 0.012 \times 15.8745 = 0.19049 = \mathbf{19.05\%}$

• 1 mark $\sqrt{252}$, 1 mark answer.

(c) Monte Carlo procedure — (5) Procedure (3 marks):

1. Simulate $N$ (e.g. 10,000) independent 1-year paths: for each path draw 252 daily returns $r_t \sim \mathcal{N}(0.0008, 0.012^2)$.
2. Compute terminal wealth $W = 100000 \times \prod_{t=1}^{252}(1+r_t)$ for each path.
3. Sort the $N$ terminal wealths and take the 5th percentile as the worst-case (VaR-style) 1-year outcome.

Advantage (2 marks): a single backtest is one realisation of a random process; Monte Carlo produces a distribution of outcomes, quantifying the range/tail risk (e.g. 5th-percentile drawdown) rather than relying on one lucky/unlucky historical path — better for risk assessment and confidence intervals.

[
  {"claim":"CAGR of equity curve = 21.64%","code":"cagr=(180000/100000)**(1/3)-1; result=abs(cagr-0.2164)<0.001"},
  {"claim":"Max drawdown = -30.77%","code":"mdd=(90000-130000)/130000; result=abs(mdd+0.30769)<0.001"},
  {"claim":"Sharpe ratio = 0.621","code":"s=(0.2164-0.03)/0.30; result=abs(s-0.6213)<0.001"},
  {"claim":"Annual transaction cost = $6500","code":"per_rt=5+1+20; total=250*per_rt; result=total==6500"},
  {"claim":"Net return = 107%","code":"net=(60000-6500)/50000; result=abs(net-1.07)<1e-9"},
  {"claim":"MC annualised return = 22.33%","code":"r=(1.0008)**252-1; result=abs(r-0.22326)<0.001"},
  {"claim":"Annualised vol = 19.05%","code":"import sympy; v=0.012*sympy.sqrt(252); result=abs(float(v)-0.19049)<0.001"},
  {"claim":"Degradation ratio = 0.19","code":"result=abs(0.4/2.1-0.1905)<0.001"}
]