6.2.11Backtesting Frameworks

Understand paper trading before going live

1,853 words8 min readdifficulty · medium

WHY does paper trading exist?

The deepest reason: a backtest can suffer from look-ahead bias and survivorship bias, while paper trading, by construction, cannot look ahead because the future literally hasn't happened yet.


WHAT exactly is paper trading?


HOW do we measure whether paper trading "passes"?

We need to compare backtest performance to paper performance on the same period going forward. Let me derive the tracking metric from scratch.

Deriving the tracking error

Let rtbtr^{bt}_t be the backtest return on day tt and rtptr^{pt}_t the paper-traded return on the same day. Define the daily difference:

dt=rtptrtbtd_t = r^{pt}_t - r^{bt}_t

WHY subtract? Because a systematically negative dtd_t means paper trading is bleeding value the backtest never charged you — usually slippage and fees.

The average leakage per day:

dˉ=1Nt=1Ndt\bar d = \frac{1}{N}\sum_{t=1}^{N} d_t

The tracking error is the standard deviation of that difference — how unpredictable the gap is:

Deriving realistic slippage cost

Suppose you want to buy QQ shares but each unit of demand pushes price up by a factor kk (the impact coefficient). Price paid on the marginal share at quantity qq is P0(1+kq)P_0(1+kq). Total cost:

C=0QP0(1+kq)dq=P0(Q+12kQ2)C = \int_0^{Q} P_0(1+kq)\,dq = P_0\left(Q + \tfrac{1}{2}kQ^2\right)

The average price per share is C/QC/Q:

Pˉ=P0(1+12kQ)\bar P = P_0\left(1 + \tfrac{1}{2}kQ\right)

Figure — Understand paper trading before going live

Worked examples


Common mistakes (steel-manned)


Recall Feynman: explain to a 12-year-old

Imagine you invented a video-game move that always wins... when you replay old recorded games. Cool — but replays can't surprise you, you already know what the enemy does. Paper trading is playing live against the game right now, but with a save-file so you don't lose real coins. If your move still wins live, it's real. If it only won in replays, you just avoided losing your real coins.


Active-recall flashcards

#flashcards/stock-market

Paper trading uses which kind of data and which kind of money?
Live/streaming market data with simulated (fake) capital.
Why can a backtest lie but paper trading structurally cannot lie about timing?
A backtest can accidentally use future data (look-ahead bias); paper trading decides tick-by-tick with the future genuinely unknown.
Define tracking error between backtest and paper returns.
The standard deviation of the daily return differences dt=rtptrtbtd_t=r^{pt}_t-r^{bt}_t, i.e. 1N1(dtdˉ)2\sqrt{\frac{1}{N-1}\sum(d_t-\bar d)^2}.
What does a consistently negative mean difference dˉ\bar d indicate?
Systematic leakage — usually slippage and fees the backtest ignored.
Derived slippage-per-share formula with linear price impact kk?
12kP0Q\tfrac{1}{2}kP_0Q — linear in order size QQ.
Why does a strategy that works on 1 lot fail at large size?
Slippage grows linearly with quantity (Q\propto Q), so market impact eats the edge at scale.
Order of the validation funnel?
Backtest → Paper trade → Live (small size).
Why is a perfect paper-vs-backtest match a warning sign?
It suggests the paper engine also assumes costless instant fills — it inherited the backtest's fantasy instead of adding realism.
Roughly how many trades before paper results are statistically meaningful?
At least ~30, since standard error of the mean shrinks as 1/n1/\sqrt{n}.
Two things paper trading tests that a backtest never can?
Real data-feed/latency behaviour and the trader's own psychological discipline.

Connections

Concept Map

test 1

test 2

test 3

suffers from

cannot look ahead

uses

reveals

measured by

averaged as

std dev gives

near 0 and small TE

Bessel correction

Trading idea

Backtest on history

Paper trade live data

Live small size

Look-ahead and survivorship bias

Future unknown tick by tick

Live streaming data + fake capital

Slippage, fees, latency, discipline

Daily diff d_t = paper minus backtest

Mean leakage d-bar

Tracking error

Backtest is honest, go live

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Dekho, backtesting matlab tum apni strategy ko purana data pe chala ke dekhte ho — bahut fast aur sasta hai, par isme ek dhoka chhupa hota hai. History pehle se pata hoti hai, isliye tumhara code galti se "future ki jhaank" (look-ahead bias) le sakta hai, aur data bhi ekdam saaf hota hai jaisa real market me kabhi nahi hota. Yahi gap paper trading bharta hai.

Paper trading ka matlab: strategy ko live, real-time market data pe chalao, lekin paisa nakli — koi real order exchange ko nahi jaata. Kyunki future abhi hua hi nahi, isliye tumhara code cheat nahi kar sakta. Ye tumhare pure system ko test karta hai — data feed, latency, aur sabse important, tumhari khud ki discipline (dar aur laalach jo backtest me kabhi mehsoos nahi hote).

Sabse bada point: backtest me fills muft aur turant maan liye jaate hain, par real me bade order price hila dete hain — isko slippage kehte hain, aur ye 12kP0Q\tfrac12 kP_0Q ke hisaab se order size ke saath badhta hai. Isiliye 1 lot pe chamakti strategy 5000 shares pe mar jaati hai. Tracking error (dt=rptrbtd_t = r^{pt}-r^{bt} ka standard deviation) dekh ke pata chalta hai backtest sach bol raha tha ya sapna dikha raha tha.

Toh rule simple hai — BPL funnel: pehle Backtest, phir Paper trade (kam se kam ~30 trades tak), phir chhoti size me Live. Jab tak paper me green nahi aata realistic costs ke saath, apna asli paisa daalna sabse mehnga sabak hota hai.

Test yourself — Backtesting Frameworks

Connections