Understand machine learning in trading (caution) — Stock-Market notes

The Promise vs. The Reality

What ML Can Actually Do

Machine learning in trading uses algorithms that learn patterns from historical data to make predictions about future price movements, trade timing, or risk.

Legitimate applications:

Feature engineering - Transform raw price data into predictive signals
Regime detection - Identify when market conditions change (volatility shifts, trend vs. mean-reversion)
Execution optimization - Minimize market impact when placing large orders
Risk modeling - Predict volatility, correlations, tail risks
Alternative data processing - Extract signals from satellite images, credit card data, social sentiment

WHY these work: They solve well-defined problems with clear evaluation criteria and don't require predicting exact future prices.

The Fatal Traps

Common mistake The Seven Deadly Sins of ML Trading

1. Overfitting (The Biggest Killer)

Why it feels right: Your model achieves 95% accuracy on historical data! Every parameter is tuned perfectly!
Why it's wrong: You've memorized noise. The model learned "on October 15, 2018, tech stocks dropped" instead of generalizable patterns. Overfitting means the model captures random fluctuations specific to the training data rather than true underlying relationships.
The fix: Use walk-forward analysis, k-fold cross-validation on time series, regularization (L1/L2), and simple models (fewer parameters). If your model has 50 parameters trained on 200 data points, you've fit noise.

2. Look-Ahead Bias

Why it feels right: Your backtest uses "closing price" to generate signals. That's historical data, right?
Why it's wrong: Look-ahead bias occurs when your model uses information not available at the time of the decision. Examples:
- Using the day's closing price to make a trade "at the open"
- Training on the full dataset including future data
- Using revised/restated earnings before they were announced
The fix: Point-in-time datasets. Only use data available before each trade decision. Walk-forward testing.

3. Survivorship Bias

Why it feels right: Test your strategy on the S&P 500 constituents.
Why it's wrong: Survivorship bias means testing only on winners. The S&P 500 today excludes companies that failed and were delisted. Your strategy might have bought Enron, Lehman Brothers, or hundreds of bankrupt firms.
The fix: Use survivorship-bias-free datasets that include delisted stocks.

4. Data Snooping

Why it feels right: After testing100 features, you found one that works!
Why it's wrong: Data snooping (p-hacking) means trying many hypotheses until one appears significant by chance. With 100 tests at 95% confidence, 5 will appear significant purely by luck.
The fix: Bonferroni correction, hold-out validation sets never touched during development, pre-register hypotheses.

5. Non-Stationarity

Why it feels right: Trained on 10 years of data, the patterns are stable.
Why it's wrong: Markets are non-stationary - statistical properties change over time. Volatility regimes shift, correlations break down, the Federal Reserve changes policy. Your 2010-2020 model breaks in 2021.
The fix: Shorter training windows, adaptive models that retrain, regime-switching approaches, regular out-of-sample testing. 6. Transaction Costs Ignored
Why it feels right: Strategy returns15% annually in backtest.
Why it's wrong: After commissions, slippage, spread, market impact, and taxes, it returns -5%. Transaction costs scale with trading frequency and position size.
The fix: Model realistic costs. Include bid-ask spread, slippage (2-5 bps for liquid stocks), market impact for large orders.

7. Regime Change

Why it feels right: Model trained on 2015-2020 performs great.
Why it's wrong: COVID crash, inflation surge, rate hikes - the market regime changed completely. Your model learned "low rates, low volatility" patterns that no longer apply.
The fix: Test across different regimes (2008 crisis, dot-com bubble, rising rate environments). Use multiple validation periods.

Derivation: Why Overfitting is Mathematically Guaranteed

Let's prove why complex models always overfit small datasets.

Setup: You have $N$ data points (e.g., 200 days of returns) and a model with $P$ parameters.

The Degrees of Freedom Problem

Derivation from first principles:

Sample size: $N$ observations (e.g., 200 trading days)
Model parameters: $P$ weights/features (e.g., 50 technical indicators)
Effective degrees of freedom: $\text{DOF} = N - P$

WHY does this matter?

For the model to learn generalizable patterns rather than noise:

$\text{Required: } \frac{N}{P} \geq 10 \text{ to } 20$

Proof by example:

If $P = N$ : Your model has as many knobs as data points. It can fit any random sequence perfectly by assigning one parameter per point.
If $P > N$ : You have more parameters than data - infinitely many solutions exist, all fitting training data perfectly, most worthless.

The Bias-Variance Tradeoff

Formula Expected Prediction Error

$\text{Error} = \text{Bias}^2 + \text{Variance} + \text{Irreducible Error}$

Derivation:

Let $Y$ be true return, $\hat{f}(x)$ be model prediction, $f(x)$ be true function.

$\mathbb{E}[(Y - \hat{f}(x))^2] = \mathbb{E}[(f(x) + \epsilon - \hat{f}(x))^2]$

Expand: $= \mathbb{E}[(f(x) - \hat{f}(x))^2] + \mathbb{E}[\epsilon^2] + 2\mathbb{E}[(f(x) - \hat{f}(x)\epsilon]$

Since $\epsilon$ is noise independent of model: $\mathbb{E}[(f(x) - \hat{f}(x)\epsilon] = 0$

$= \underbrace{\mathbb{E}[(f(x) - \mathbb{E}[\hat{f}(x)])^2]}_{\text{Bias}^2} + \underbrace{\mathbb{E}[(\hat{f}(x) - \mathbb{E}[\hat{f}(x)])^2]}_{\text{Variance}} + \underbrace{\mathbb{E}[\epsilon^2]}_{\text{Irreducible}}$

WHAT this means:

Bias = how far your model's average prediction is from truth (underfitting)
Variance = how much your model's predictions vary with different training sets (overfitting)
Irreducible error = randomness you can't predict

WHY complex models overfit:

As model complexity ↑:

Bias ↓ (fits training data better)
Variance ↑ (becomes unstable, changes wildly with small data changes)
At high complexity: Variance dominates, model fits noise

How to Use ML Responsibly in Trading

1. Walk-Forward Analysis

HOW to implement:

Training Window: [Year 1, Year 2, Year 3] → Test on [Year 4 Q1]
Roll forward:    [Year 2, Year 3, Year 4 Q1] → Test on [Year 4 Q2]
Continue rolling...

WHY this works: Simulates reality - you only know the past when making decisions.

2. Cross-Validation for Time Series

3. Sharpe Ratio as Evaluation Metric

4. Regularization Techniques

L1 Regularization (Lasso):

$\text{Loss} = \text{MSE} + \lambda \sum_{i=1}^{P} |w_i|$

WHY: Forces many weights to exactly zero, performs automatic feature selection. Prevents using100 features when 5 drive all signal.

L2 Regularization (Ridge):

$\text{Loss} = \text{MSE} + \lambda \sum_{i=1}^{P} w_i^2$

WHY: Shrinks all weights toward zero, prevents any single feature from dominating. Reduces variance.

HOW to choose $\lambda$ : Use cross-validation to find the penalty strength that minimizes out-of-sample error.

Worked Examples

Worked example Example 1: Detecting Overfitting with Learning Curves

Setup: You train a neural network to predict next-day S&P 500 returns using 20 technical indicators.

Training data: 500 days, Test data: 100 days

Results:

Training R² = 0.92 (explains 92% of variance)
Test R² = 0.08 (explains 8% of variance) Analysis: Step 1: Calculate the gap $\text{Overfitting Gap} = 0.92 - 0.08 = 0.84$ WHY this step? A large gap indicates the model memorized training data rather than learning generalizable patterns.

Step 2: Check parameter-to-data ratio

Neural network has ~1000 trainable parameters
Training samples: 500
Ratio: $\frac{N}{P} = \frac{500}{1000} = 0.5$

WHY this step? Rule of thumb: need $\frac{N}{P} \geq 10$ . With 0.5, you have more parameters than data - severe overfitting is guaranteed.

Step 3: Solution - Apply regularization

Add L2 penalty: $\lambda = 0.01$
Add dropout: 50% during training
Reduce to 5most important features

New results:

Training R² = 0.35
Test R² = 0.28
Gap = 0.07 ✓

WHY this works: Model is now simpler, focuses on robust patterns. Lower training performance but better generalization.

Worked example Example 2: Walk-Forward Backtest of Mean Reversion Strategy

Strategy: Use ML to predict when stocks will mean-revert after extreme moves.

HOW to test properly:

Step 1: Define rolling windows

Training: 252 days (1 year)
Test: 63 days (1 quarter)
Roll forward: 21 days (1 month)

WHY these windows? Training window captures various regimes, test period is long enough to evaluate, frequent retraining adapts to changes.

Step 2: Walk-forward loop

Period 1: Train [Day 1-252], Test [Day 253-315]
Period 2: Train [Day 22-273], Test [Day 274-336]
...continue...

Step 3: Calculate performance metrics for EACH test period | Period | Return | Sharpe | Max DD | |--------|--------|------| | Q1 2023 | 8.2% | 1.4 | -3.1% | | Q2 2023 | -.1% | 0.3 | -5.8% | | Q3 2023 | 5.5% | 1.1 | -4.2% | | Q4 2023 | 12.1% | 1.9 | -2.5% |

WHY individual periods? Reveals consistency. If Q2 failed during volatility spike, strategy may not handle regime changes.

Step 4: Aggregate statistics

Average Sharpe: 1.175
Win rate: 75% of quarters positive
Worst drawdown: -5.8%

Decision: Proceed with live testing at small scale if average Sharpe > 1.0AND worst quarter isn't catastrophic.

Worked example Example 3: Transaction Cost Reality Check

Backtest result: Strategy generates 25% annual return with 200 trades/year.

Let's calculate real returns: Step 1: Define costs per trade

Commission: $1 per side =$ 2 round-trip
Spread: 0.05% of position size (bid-ask)
Slippage: 0.03% (market impact)
Total: $2 + 0.08% of position

WHY include all three? Backtests often ignore spread/slippage. Real execution always faces these.

Step 2: Calculate annual cost Assume average position = $10,000 Per trade cost: $$C_{\text{trade}} = \$ 2 + (0.008 \times $10,000) = $2 + $8 = $10$$ WHY this calculation? Fixed commission is small, but percentage costs scale with position size.

Annual trades: 200 $C_{\text{annual}} = 200 \times \$10 = \$2,000$

Step 3: Convert to return impact Portfolio value: $100,000 $$\text{Cost as \% return} = \frac{\$ 2,000}{$100,000} = 2%$$

Step 4: Adjusted return $\text{Net return} = 25\% - 2\% = 23\%$ Conclusion: Strategy still profitable, but 2% drag is significant. If returns were 10%, costs would eat 20% of profits.

Additional consideration: If strategy trades2000 times/year (high frequency), cost =20%, wiping out all gains.

The Right Way to Build ML Trading Systems

Phase 1: Hypothesis-Driven Development

Start with a theory, not data mining:

Economic intuition: "Stocks with rising revenue growth and falling inventory growth tend to outperform" ← testable hypothesis
NOT data mining: "Let me try500 features and see what correlates" ← guaranteed overfitting

WHY this matters: Hypothesis-driven research reduces multiple testing problems and builds on market logic.

Phase 2: Feature Engineering

Good features:

Based on economic theory
Stationary (statistical properties don't drift over time)
Available in real-time (no look-ahead bias)

Example features:

$\text{Momentum}_t = \frac{P_t - P_{t-20}}{P_{t-20}}$ (20-day return)
$\text{Volatility}_t = \text{std}(R_{t-20:t})$ (20-day return std)
$\text{Relative Strength}_t = \frac{\text{Momentum}_{\text{stock}}}{\text{Momentum}_{\text{sector}}}$

Phase 3: Simple Models First

Model complexity hierarchy:

Linear regression - Start here. If it doesn't work, complex models won't either.
Regularized linear (Lasso, Ridge, Elastic Net)
Tree-based (Random Forest, Gradient Boosting)
Neural networks - Last resort, only if you have massive data

WHY simple first: Occam's Razor. Simpler models generalize better, are easier to interpret, and fail less catastrophically.

Phase 4: Validation Hell (This is Where You Live)

Validation protocol:

Out-of-sample test: 20% of data never seen during development
Walk-forward: Roll through time, retrain, test next period
Cross-regime: Test during 2008 crisis, 2020 COVID, 2022 rate hikes
Stress test: What happens if volatility doubles? Correlations break?
Paper trade: 3-6 months simulated real-time trading

Only deploy if successful at ALL stages.

When ML Actually Works in Trading

Success stories (institutional level):

Renaissance Technologies (Medallion Fund):
- Uses massive alternative datasets
- Extremely short holding periods (minutes to hours)
- Statistical arbitrage on thousands of instruments
- 66% annual returns for 30 years

WHY they succeed:

Huge data advantage (proprietary datasets)
PhD-level quant teams
Infrastructure to test billions of hypotheses rigorously
Capital to exploit tiny inefficiencies at scale

Two Sigma, Citadel, DE Shaw:
- Similar approaches: massive data, rigorous validation
- Focus on execution optimization, risk management

What retail/small institutional can do:

ML for risk management (easier than returns prediction)
- Volatility forecasting
- Correlation regime detection
- Tail risk estimation
Alternative data processing
- Sentiment analysis from earnings calls
- Satellite imagery of retail parking lots
- Credit card transaction data
Execution optimization
- Optimal trade timing within the day
- Minimizing market impact

WHY these work: They don't require predicting future prices exactly, just improving process efficiency or risk assessment.

Recall Explain to a 12-Year-Old

Imagine you're trying to predict tomorrow's weather by looking at the last 10 days. You notice that every time it was sunny for 3 days, it rained on the 4th day. So you create a rule: "3 sunny days → rain next."

But here's the problem: you only looked at 10 days. Maybe by pure luck, those 3 sunny stretches were followed by rain. If you look at 1000 days, the pattern might disappear. You memorized noise instead of learning real weather patterns.

Machine learning in trading makes this mistake constantly. Your computer finds patterns in stock prices (like "when Apple goes up 3 days, it drops on day 4"), but most of those patterns are just coincidences in the small amount of data you tested. When you try to use them with real money, they stop working.

The other big problem: imagine practicing basketball by watching videos of NBA games, but you skip all the videos where players missed shots. You'd think "everyone makes every shot!" Your practice is based on fake data.

That's survivorship bias - when you test trading strategies on companies that succeeded (like current S&P 500 members) but ignore all the companies that failed and disappeared. Your strategy might have lost all your money on those failed companies, but you never tested them!

ML can work in trading, but you have to be extremely careful to test it honestly, use simple strategies, and never trust patterns until you've seen them work in brand new data you never looked at before.

Key Formulas Summary

| Concept | Formula | When to Use | |---------|-------------| | Overfitting Check | $\frac{N}{P} \geq 10$ | Before training any model | | Sharpe Ratio | $\frac{\mathbb{E}[R_p - R_f]}{\sigma_p}$ | Evaluating risk-adjusted returns | | Bias-Variance | $\text{Error} = \text{Bias}^2 + \text{Variance} + \text{Irreducible}$ | Understanding model tradeoffs | | L1 Regularization | $\text{Loss} + \lambda \sum \|w_i\|$ | Feature selection | | L2 Regularization | $\text{Loss} + \lambda \sum w_i^2$ | Reducing overfitting |

#flashcards/stock-market

What is overfitting in ML trading? :: When a model memorizes random patterns in training data (noise) instead of learning true underlying relationships, causing it to fail on new data.

Why is overfitting guaranteed when P > N?

A model with more parameters (P) than data points (N) can fit any random sequence perfectly, but has learned nothing generalizable. Need/P ≥ 10.

What is look-ahead bias?

Using information your model that would not have been available at the time of the trading decision, like using closing prices to make opening trades.

What is survivorship bias in backtesting?

Testing strategies only on companies that currently exist, excluding all the companies that failed/delisted, creating artificially good results.

What is the bias-variance tradeoff?

Total error = Bias² + Variance + Irreducible. As model complexity increases, bias decreases (fits training better) but variance increases (overfits), eventually making performance worse.

What is walk-forward analysis?

Training on a window of past data, testing on the next unseen period, then rolling the window forward. Simulates real-time trading where you only know the past.

Why is Sharpe ratio better than raw returns?

Sharpe = (Return - Risk-free) / Volatility. It penalizes strategies that achieve returns through excessive risk. A volatile20% return may be worse than a stable 10% return.

What is L1 regularization?

Adding λΣ|wᵢ| to the loss function forces many weights to exactly zero, performing automatic feature selection and preventing the model from using too many features.

What is data snooping (p-hacking)?

Testing many features/strategies until one appears significant by random chance. With 100 tests at 95% confidence, 5 will appear significant purely by luck.

What makes markets non-stationary?

Statistical properties change over time - volatility regimes shift, correlations break, Fed policy changes, new regulations appear. Past patterns may not repeat.

What transaction costs must you model?

Commission (fixed per trade), bid-ask spread (0.05-0.1%), slippage (market impact, 0.03-0.05%), and taxes. High-frequency strategies often lose all returns to costs.

Name three things ML can legitimately do in trading :: 1) Risk management (volatility/correlation forecasting), 2) Execution optimization (timing trades to minimize impact), 3) Alternative data processing (satellite imagery, sentiment).

What is the minimum out-of-sample Sharpe for deployment?

Generally > 1.0 for consideration, > 1.5 for live deployment, after accounting for all transaction costs and tested across multiple market regimes.

Why start with linear models instead of neural networks?

If simple linear models don't work, complex models won't either - they'll just overfit. Simple models generalize better, are interpretable, and fail less catastrophically.

What is feature engineering?

Transforming raw data (prices, volume) into predictive signals based on economic theory (momentum, volatility, relative strength) that are stationary and available real-time.

Connections

Technical Analysis Indicators - ML often uses these as features, but must validate they add predictive power
Backtesting Strategies - Proper validation protocol is critical before deploying ML
Risk Management Principles - ML predictions must feed into position sizing and stop-loss rules
High-Frequency Trading - Where ML transaction cost problems are most severe
Quantitative Factor Investing - Alternative to ML: simple, interpretable factors with economic rationale
Market Efficiency Hypothesis - ML strategies must identify genuine inefficiencies, not overfit noise
Alternative Data Sources - Where ML can add value: processing unstructured data
Portfolio Optimization - ML can improve mean-variance optimization inputs (return/risk estimates)

Remember: In trading, the graveyard is full of strategies that worked perfectly in backtests. ML is a powerful tool, but markets are adversarial environments designed to punish overfitted models. Validation, simplicity, and humility are your only defenses.

Concept Map

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Dekho, machine learning trading mein bahut powerful lagti hai kyunki idea simple hai - data se pattern seekho, price predict karo, aur profit kamao. Lekin yahan asli baat samajhne wali yeh hai ki market ek adversarial environment hai, matlab yahan aapke against dusre smart log actively khel rahe hain. Jaise hi aapko koi edge milta hai, dusre bhi wahi discover kar lete hain aur woh edge gayab ho jata hai. Upar se markets non-stationary hote hain - aaj jo pattern kaam kar raha hai, kal Fed policy change hone ya volatility shift hone par woh toot sakta hai. Isliye ML kaam toh karti hai, par sirf tab jab aap extreme discipline maintain karo.

Ab sabse bada dushman hai overfitting. Yeh tab hota hai jab aapka model historical data pe 95% accuracy deta hai, aur aapko lagta hai wah! perfect hai. Lekin actually model ne noise ko yaad kar liya hai, real pattern nahi seekha. Jaise "15 October 2018 ko tech stocks gire the" - yeh koi generalizable rule nahi, sirf random ek din ki baat hai. Agar aapke paas 200 data points hain aur 50 parameters, toh aap guaranteed noise fit kar rahe ho. Iske saath aur bhi traps hain - look-ahead bias (future ki info galti se use karna), survivorship bias (sirf jeetne walon pe test karna, jaise aaj ka S&P 500 jismein failed companies nahi hain), aur data snooping (100 features try karke jo chance se kaam kare use pakad lena).

Yeh sab why-matters isliye hai ki agar aap in traps ko nahi samjhoge, toh aapka backtest zabardast dikhega par real trading mein paisa dubega - kyunki backtest jhooth bol raha tha. Solution simple hai: walk-forward analysis karo, simple models use karo (kam parameters), transaction costs kabhi ignore mat karo, aur hold-out validation set rakho jise development ke dauraan chhuo hi mat. Yaad rakhna, ML mein asli skill prediction banane mein nahi, balki apne aap ko fool hone se bachane mein hai.

Test yourself — Algorithmic & Quant Trading

Connections