6.2.8Backtesting Frameworks
Learn Python tools (pandas, backtrader, zipline)
3,276 words15 min readdifficulty · medium
The Foundation: pandas for Market Data
Why pandasFirst? Because all strategies are just transformations of OHLCV data. You need to:
- Load CSVs/databases into memory
- Calculate indicators (SMA = rolling mean of Close)
- Generate signals (buy when price > SMA)
- Align multiple stocks on the same timeline
Core pandas Operations for Trading
Operation 1: Loading and Indexing
import pandas as pd
# Load stock data
df = pd.read_csv('AAPL.csv', parse_dates=['Date'], index_col='Date')
# Why parse_dates? Makes 'Date' a datetime object, unlocking time operations
# Why index_col? Puts Date as the row index, so df.loc['2024-01-15'] works
print(df.head())
# Open High Low Close Volume
# Date
# 2024-01-02 185.0 186.5 184.0 185.5 5000000Why this matters: DatetimeIndex lets you slice by date ranges (df['2024-01':'2024-03']), resample to different frequencies, and do time-based joins.
Operation 2: Calculating Indicators
# Simple Moving Average (20-day)
df['SMA_20'] = df['Close'].rolling(window=20).mean()
# Why rolling()? It creates a moving window that slides over your data.
# .mean() calculates the average of those20 values at each step.
# Exponential Moving Average (gives more weight to recent prices)
df['EMA_12'] = df['Close'].ewm(span=12, adjust=False).mean()
# Why adjust=False? Matches industry-standard EMA calculation.Derivation of Rolling Mean: where = price at time , = window size.
For 20-day SMA on day100:
The Framework: backtrader for Strategy Logic
Why not just pandas? You could compute returns in pandas, but you'd have to manually:
- Track cash and positions across time
- Handle partial fills, slippage, commissions
- Avoid lookahead by careful indexing
- Compute portfolio metrics
backtrader does this for you, letting you focus on strategy logic.
Anatomy of a backtrader Strategy
import backtrader as bt
class SmaCrossover(bt.Strategy):
params = (('sma_short', 20), ('sma_long', 50)) # Configurable parameters
def __init__(self):
# Indicators are calculated ONCE on all data, but accessed bar-by-bar
self.sma_short = bt.indicators.SMA(self.data.close, period=self.params.sma_short)
self.sma_long = bt.indicators.SMA(self.data.close, period=self.params.sma_long)
self.crossover = bt.indicators.CrossOver(self.sma_short, self.sma_long)
def next(self):
# Called for each bar (after SMA warmup period)
if not self.position: # No position: look for entry
if self.crossover > 0: # Short MA crossed above Long MA
self.buy(size=100) # Buy 100 shares
else: # Have position: look for exit
if self.crossover < 0: # Short MA crossed below Long MA
self.close() # Close positionWhy separate __init__ and next?
__init__: Define indicators. backtrader computes them on ALL data upfront (efficient).next: Trading logic. Called sequentially, can only see current + past bars (prevents lookahead).
How CrossOver Works:
+1 & \text{if } \text{SMA}_{\text{short},t} > \text{SMA}_{\text{long},t} \text{ and } \text{SMA}_{\text{short},t-1} \leq \text{SMA}_{\text{long},t-1} \\ -1 & \text{if } \text{SMA}_{\text{short},t} < \text{SMA}_{\text{long},t} \text{ and } \text{SMA}_{\text{short},t-1} \geq \text{SMA}_{\text{long},t-1} \\ 0 & \text{otherwise} \end{cases}$$ > [!example] Running a Backtest > ```python > cerebro = bt.Cerebro() # The engine > cerebro.addstrategy(SmaCrossover) > # Load data > data = bt.feeds.PandasData(dataname=df) # df is your pandas DataFrame > cerebro.adddata(data) > > # Set initial capital and commission > cerebro.broker.setcash(100000.0) # $100k starting capital > cerebro.broker.setcommission(commission=0.001) # 0.1% per trade > > print(f'Starting Portfolio Value: {cerebro.broker.getvalue():.2f}') > cerebro.run() > print(f'Final Portfolio Value: {cerebro.broker.getvalue():.2f}') > cerebro.plot() # Visualize equity curve and trades > ``` > > **Why `Cerebro`?** Spanish for "brain"—it coordinates everything: data feeds, strategy, broker simulation, analyzers. > > **Output:** > ``` > Starting Portfolio Value: 100000.00 > Final Portfolio Value: 125430.50 > ``` > The plot shows price with SMA overlays, buy/sell markers, and portfolio value over time. > [!formula] Portfolio Value Update > At each bar, backtrader updates: > $$V_t = \text{Cash}_t + \sum_{i} Q_{i,t} \cdot P_{i,t}$$ > where $V_t$ = portfolio value, $Q_{i,t}$ = quantity of asset $i$ held, $P_{i,t}$ = price of asset $i$. > > When you `buy(size=100)` at price $P$: > - $\text{Cash}_t \leftarrow \text{Cash}_{t-1} - 100 \cdot P \cdot (1 + c)$ where $c$ = commission rate > - $Q_t \leftarrow Q_{t-1} + 100$ ### backtrader Analyzers ```python cerebro.addanalyzer(bt.analyzers.SharpeRatio, _name='sharpe') cerebro.addanalyzer(bt.analyzers.DrawDown, _name='drawdown') results = cerebro.run() strat = results[0] print(f"Sharpe Ratio: {strat.analyzers.sharpe.get_analysis()['sharperatio']:.2f}") print(f"Max Drawdown: {strat.analyzers.drawdown.get_analysis()['max']['drawdown']:.2f}%") ``` **Sharpe Ratio Derivation:** $$\text{Sharpe} = \frac{\bar{R} - R_f}{\sigma_R}$$ where $\bar{R}$ = mean return period, $R_f$ = risk-free rate (often 0), $\sigma_R$ = std dev of returns. If daily returns are $[0.01, -0.005, 0.02, \ldots]$: 1. $\bar{R} = \text{mean}([0.01, -0.005, 0.02, \ldots])$ 2. $\sigma_R = \text{std}([0.01, -0.005, 0.02, \ldots])$ 3. Annualize: Multiply by $\sqrt{252}$ (trading days/year) **Drawdown:** Maximum peak-to-trough decline. $$\text{DD}_t = \frac{V_t - \max_{s \leq t} V_s}{\max_{s \leq t} V_s}$$ --- ## The Industrial Upgrade: zipline > [!definition] zipline's Purpose > ==zipline== is Quantopian's open-source engine, designed for **realistic backtesting** at scale. Key differences from backtrader: > - **Pipeline API:** Efficiently computes factors (indicators) across1000s of stocks simultaneously. > - **Built-in data bundles:** Quandl, Yahoo Finance (though Quantopian's data is defunct, you can ingest custom). > - **Minute-level granularity:** Handles intraday bars. > - **Realistic constraints:** Slippage models, market impact, shorting costs. **When to use zipline?** When you're moving from "backtest one stock" to "scan the entire market daily and rank500 stocks by momentum". ### zipline Strategy Structure ```python from zipline.api import order_target_percent, record, symbol def initialize(context): # Called once at start context.stock = symbol('AAPL') context.sma_window = 20 def handle_data(context, data): # Called every bar (daily or minute) hist = data.history(context.stock, 'close', context.sma_window + 1, '1d') current_price = data.current(context.stock, 'price') sma = hist.mean() if current_price > sma: order_target_percent(context.stock, 1.0) # Allocate 100% to AAPL elif current_price < sma: order_target_percent(context.stock, 0.0) # Exit position record(price=current_price, sma=sma) # Log for plotting ``` **Why `order_target_percent`?** Instead of "buy 100 shares", you say "I want 50% of my portfolio in this stock". zipline calculates the order needed to reach that allocation, accounting for current holdings and cash. > [!example] zipline Pipeline for Multi-Stock Ranking > **Task:** Every day, rank all S&P 500 stocks by 20-day momentum, go long the top 10. > > ```python > from zipline.pipeline import Pipeline > from zipline.pipeline.factors import SimpleMovingAverage, Returns > from zipline.pipeline.data import USEquityPricing > def make_pipeline(): > momentum = Returns(window_length=20) # 20-day return > top_10 = momentum.top(10) # Boolean: True for top 10 stocks > > return Pipeline( > columns={'momentum': momentum}, > screen=top_10 # Only pass top 10 to handle_data > ) > > def initialize(context): > attach_pipeline(make_pipeline(), 'ranking') > > def before_trading_start(context, data): > context.output = pipeline_output('ranking') > context.longs = context.output.index # List of top 10 symbols > > def handle_data(context, data): > # Allocate 10% to each of the 10 stocks > for stock in context.longs: > order_target_percent(stock, 0.10) > > # Close any positions not in top 10 > for stock in context.portfolio.positions: > if stock not in context.longs: > order_target_percent(stock, 0.0) > ``` > > **Why Pipeline?** Computing momentum for 500 stocks individually in `handle_data` would be slow. Pipeline does it in one vectorized pass across all stocks, returning only the filtered results (top 10). **How Returns Factor Works:** $$R_{t,n} = \frac{P_t}{P_{t-n}} - 1$$ For 20-day return: $R_{t,20} = \frac{P_t}{P_{t-20}} - 1$ Pipeline computes this for ALL stocks in parallel using NumPy, then ranks and filters. > [!mistake] Mixing Time Frequencies > **Wrong Code:** > ```python > def handle_data(context, data): > hist = data.history(context.stock, 'close', 50, '1d') # Daily bars > if len(hist) < 50: > return # Not enough data > current_minuteprice = data.current(context.stock, 'price') # Minute bar! > ``` > **Why it feels right:** "I need daily SMA but want to trade on minute bars for precision." > > **The Problem:** `data.history(..., '1d')` in a minute-frequency backtest returns the last 50 **days** worth of minute bars (not 50 daily closes). Your SMA is computed on minute data, not daily closes—completely different signal. > > **Steel-man:** "But I specified '1d' frequency!" In zipline, the frequency parameter in `history` means "aggregate to this frequency". If your backtest runs minute-by-minute, you're still calling `handle_data` every minute. You need to explicitly resample or use `before_trading_start` (called once per day) for daily logic. > > **Fix:** > ```python > def before_trading_start(context, data): > # Runs once per day, gets clean daily bars > hist = data.history(context.stock, 'close', 50, '1d') > context.sma = hist.mean() > def handle_data(context, data): > # Runs every minute, uses pre-calculated daily SMA > if data.current(context.stock, 'price') > context.sma: > order_target_percent(context.stock, 1.0) > ``` --- ## Comparison: When to Use Each Tool | Tool | Best For | Pros | Cons | |------|----------|------| | **pandas** | Data cleaning, indicator calculation, quick analysis | Fast, flexible full control | No built-in backtesting, easy to introduce lookahead | | **backtrader** | Single-stock strategies, custom order types, detailed inspection | Easy to learn, great plotting, active community | Slower on large universes (100+ stocks) | | **zipline** | Multi-stock factor models, institutional realism, production pipelines | Pipeline API is fast, realistic execution model | Steper learning curve, data ingestion setup required | **Workflow:** Prototype in pandas → Validate in backtrader → Scale zipline. --- > [!recall]- Explain to a 12-Year-Old > Imagine you're running a lemonade stand, and you want to figure out the best times to buy lemons (when they're cheap) and sell lemonade (when it's hot and people pay more). > > **pandas** is your notebook where you write down lemon prices every day and calculate stuff like "What was the average price over the last 2 weeks?" It's for organizing your data. > > **backtrader** is like a practice game where you pretend to run your lemonade stand with fake money. You set rules: "If lemons cost less than $2/bag, buy 10 bags. If it's over 90°F, sell at $3/cup instead of $2." The game runs through a year of weather and lemon prices, showing you how much money you'd make. It keeps score automatically. > > **zipline** is the same game but for 100 lemonade stands across the whole city. Instead of just your stand, you rank all neighborhoods by "Which has the most parks?" (more customers) and "Which has the cheapest lemons?" Then you put your 10 stands in the top neighborhoods. It handles the complicated math of managing all those stands at once. > > The key: **pandas** = your calculator, **backtrader** = practice game for one stand, **zipline** = practice game for a whole lemonade business. --- > [!mnemonic] PBZ Stack > **P**andas: **P**repare data (clean, calculate) > **B**acktrader: **B**uild strategy (test logic, one stock) > **Z**ipline: **Z**oom out (scale to many stocks, realistic) > > Or think "Pizza BeforeZz": You prepare the dough (pandas), build the pizza (backtrader), then deliver 100 pizzas (zipline) before you sleep. --- ## Connections - [[6.2.07-Data-sourcesand-cleaning]] – pandas loads the CSVs you cleaned here - [[6.2.09-Walk-forward-analysis]] – backtrader's `cerebro.optstrategy()` tests parameters; zipline does rolling windows - [[6.3.01-Risk-adjusted-returns]] – Both engines calculate Sharpe via analyzers - [[Technical-Indicators]] – All the SMAs, RSI, Bollinger Bands you compute with pandas/backtrader built-ins - [[Order-Types]] – backtrader's `buy()`, `sell()`, `order_target_percent()` in zipline - [[Slippage-and-Transaction-Costs]] – zipline models this realistically; backtrader uses `broker.setcommission()` --- ## #flashcards/stock-market What is the purpose of pandas in backtesting? :: pandas handles tabular data manipulation—loading OHLCV data, calculating indicators (rolling means, EMA), and generating signals. It's the data preparation layer before feeding into a backtesting engine. Why use `.rolling(window=20).mean()` instead of manually suming? ::: `.rolling()` creates a sliding window that automatically handles edge cases (not enough data at the start), shifts the window correctly (no lookahead), and is vectorized (fast on large datasets). Manual suming risks off-by-one errors and lookahead bias. What is lookahead bias and how do you prevent it in pandas? ::: Lookahead bias is using future data to make past decisions (e.g., using tomorrow's return to decide today's trade via `.shift(-1)`). Prevent it by only using `.shift(1)` or positive shifts (looking backwards) and verifying that signal generation only accesses current/past bars. What does backtrader's `next()` method do? ::: `next()` is called once per data bar (after indicators warm up). It contains your trading logic—checking conditions and placing orders. It can only access the current bar and earlier, preventing lookahead bias. How does backtrader's `CrossOver` indicator work? ::: `CrossOver(seriesA, seriesB)` returns +1 when seriesA crosses above seriesB, -1 when it crosses below, and 0 otherwise. It compares current values and previous values to detect the moment of crossing. What is the formula for portfolio value in backtrader? ::: $V_t = \text{Cash}_t + \sum_{i} Q_{i,t} \cdot P_{i,t}$ where $Q_{i,t}$ is quantity of asset $i$ held and $P_{i,t}$ is its price. When you buy, cash decreases by price × quantity × (1 + commission); quantity held increases. What does `order_target_percent(stock, 0.5)` do in zipline? ::: It places orders to make the given stock 50% of your portfolio value. If you currently hold 20%, it buys more. If you hold 70%, it sells some. It's a rebalancing function, not a fixed-size order. Why is zipline's Pipeline API faster than loping over stocks? ::: Pipeline uses vectorized NumPy operations to compute factors (indicators) across all stocks simultaneously in one pass. Looping calls `data.history()` thousands of times, each with overhead. Vectorization is 10-100x faster. What is the Sharpe Ratio formula? ::: $\text{Sharpe} = \frac{\bar{R} - R_f}{\sigma_R}$ where $\bar{R}$ is mean return, $R_f$ is risk-free rate (often 0), and $\sigma_R$ is standard deviation of returns. It measures return per unit of risk. How do you calculate 20-day momentum? ::: $R_{t,20} = \frac{P_t}{P_{t-20}} - 1$ where $P_t$ is today's price and $P_{t-20}$ is the price 20 days ago. It's the percentage return over that period. What is drawdown? ::: Drawdown is the decline from a portfolio's peak value to a subsequent trough, expressed as a percentage: $\text{DD}_t = \frac{V_t - \max_{s \leq t} V_s}{\max_{s \leq t} V_s}$. Max drawdown is the worst such decline over the backtest period. When should you use backtrader vs zipline? ::: Use backtrader for single-stock or small-universe strategies where you want detailed inspection and easy plotting. Use zipline for multi-stock factor models, ranking100+ stocks, or when you need realistic execution modeling (slippage, market impact) at scale. What is the danger of mixing time frequencies in zipline? ::: If your backtest runs minute-by-minute but you call `data.history(..., '1d')` inside `handle_data`, you get 1-day worth of minute bars (not daily closes), corrupting your indicator. Use `before_trading_start` for daily logic or explicitly resample. Why declare indicators `__init__` instead of `next()` in backtrader? ::: Indicators are calculated once on all historical data in `__init__` (efficient). `next()` is called per bar and only accesses the indicator's current value (via indexing like `self.sma[0]`). Recalculating in `next()` would be slow and error-prone. What does `cerebro.broker.setcommission(0.001)` do? ::: Sets a0.1% commission on each trade. When you buy 100 shares at $50, you pay $5000 +0.001×$5000 = $5005. This models real trading costs, reducing backtest profitability to realistic levels. ## 🖼️ Concept Map ```mermaid flowchart TD Q[Does strategy work?] -->|solved by| STACK[Trading Stack] STACK -->|foundation| PD[pandas] STACK -->|framework| BT[backtrader] STACK -->|institutional upgrade| ZP[zipline] PD -->|stores| OHLCV[OHLCV tabular data] OHLCV -->|indexed by| DTI[DatetimeIndex] DTI -->|enables| SLICE[Date slicing and resampling] PD -->|rolling mean| SMA[SMA indicator] PD -->|ewm span| EMA[EMA indicator] SMA -->|price crosses above| SIG[Buy/Sell Signal] SIG -->|diff detects| TRADE[Position Change trade] TRADE -->|executed by| BT ZP -->|built by| QUANT[Quantopian] ``` ## 🔊 Hinglish (regional understanding) > [!intuition]- Hinglish mein samjho > Dekho, backtesting system banane ko ghar banane jaise samajh lo. Iska core intuition yeh hai ki teen tools milke ek pura "stack" banate hain, aur har tool ek alag layer solve karta hai. Sabse pehle **pandas** aata hai — yeh tumhari foundation hai jo saara raw data handle karta hai. Har stock ki history basically ek table hoti hai: rows mein timestamps aur columns mein OHLCV (Open, High, Low, Close, Volume). pandas isko SQL jaise operations aur time-series superpowers deta hai — jaise rolling windows se SMA (moving average) calculate karna, ya signals generate karna jab price SMA se upar cross kare. Yaad rakho, har strategy end mein OHLCV data ka bas ek transformation hi hoti hai, isliye pandas seekhna sabse zaroori first step hai. > > Ab jab tumhara data ready ho jata hai, tab **backtrader** framework kaam mein aata hai — yeh buy/sell ki logic manage karta hai aur tumhara portfolio track karta hai. Aur **zipline** ek industrial-grade upgrade hai jo Quantopian ne banaya, realistic constraints ke saath institution-level backtesting ke liye. Toh teeno alag-alag layer pe kaam karte hue ek hi sawaal ka jawaab dhoondte hain: "Kya meri trading strategy sach mein kaam karti hai?" > > Par sabse important cheez jo tumhe dhyaan mein rakhni hai woh hai **lookahead bias** ka trap. Yeh tab hota hai jab tum galti se future ka data use karke aaj ka trade decide karte ho — jaise `.shift(-1)` se tomorrow ka return laake aaj ka signal banana. Yeh galti backtest mein toh shaandaar results dikhati hai, lekin real trading mein tum kabhi future nahi dekh sakte, isliye live mein strategy fail ho jaati hai. Isliye hamesha check karo ki tumhara signal sirf past aur present data pe based ho. Yeh chhoti si mistake beginners ko sabse zyada dhoka deti hai, aur yahi backtesting seekhne ka asli reason hai — taaki tum apne aap ko fool na karo. ![[audio/6.2.08-Learn-Python-tools-(pandas,-backtrader,-zipline).mp3]]