Backtesting Best Practices: How to Test Without Fooling Yourself

A backtest is a simulation of how a trading strategy would have performed on historical data. It's the primary tool algo traders use to evaluate strategies before risking real capital. But here's the catch: most backtests are misleading. They look profitable because of subtle errors that create an illusion of edge where none exists.

This article covers the critical best practices that separate rigorous, trustworthy backtests from dangerous, misleading ones.

Why Backtesting Goes Wrong

The fundamental problem with backtesting is that you're using the same data to both develop and evaluate your strategy. This creates multiple opportunities for data snooping — finding patterns that exist in your specific dataset but don't generalize to future data.

Research by Marcos Lopez de Prado (author of "Advances in Financial Machine Learning") estimates that most published trading strategies fail in live trading because of backtesting errors. The most common culprits:

• Look-ahead bias: Using information that wouldn't have been available at the time of the trade
• Survivorship bias: Only testing on stocks that still exist today (ignoring delisted companies)
• Over-optimization: Tweaking parameters until the backtest looks perfect
• Unrealistic assumptions: Zero slippage, instant fills, unlimited liquidity

The Proper Backtesting Framework

Step 1: Data Quality and Preparation

Your backtest is only as good as your data. Common data issues in Indian markets:

• Corporate actions: Stock splits, bonuses, and dividends must be adjusted. If Reliance did a 1:1 bonus split, the pre-split prices need to be halved in your data.
• Survivorship bias: Ensure your dataset includes stocks that were delisted or merged. For example, TATAMOTORS was demerged in 2026 — a backtest that excludes such events is unreliable.
• Data gaps: NSE data sometimes has gaps on special trading sessions. Handle missing data explicitly (forward-fill or exclude).

Step 2: Train-Test Split

Never evaluate a strategy on the same data you used to develop it. The standard approach:

• In-sample (training): Use 60-70% of your data to develop and optimize the strategy
• Out-of-sample (testing): Use the remaining 30-40% to evaluate performance — touch this data ONLY ONCE
• Golden rule: If you modify your strategy after looking at out-of-sample results, the test is contaminated and you need fresh data

Step 3: Walk-Forward Optimization

Walk-forward testing is the gold standard for backtesting. Instead of a single train-test split, you:

1. Train on months 1-12, test on months 13-15
2. Train on months 4-15, test on months 16-18
3. Train on months 7-18, test on months 19-21
4. Continue rolling forward...

This simulates how you'd actually use the strategy in real life — periodically re-optimizing on recent data and trading on unseen future data.

Step 4: Realistic Cost Modeling

Include these costs in every backtest:

• Brokerage: ₹20 per order (Zerodha) or percentage-based (0.01-0.05%)
• STT: 0.025% on sell-side for delivery, 0.0125% for intraday
• Exchange charges: ~0.00325% (NSE)
• GST: 18% on brokerage + exchange charges
• Slippage: Assume 0.05-0.1% per trade for liquid Nifty 50 stocks, 0.2-0.5% for smaller stocks
• Impact cost: For large orders, your own buying/selling pressure moves the price against you

Step 5: Statistical Validation

Don't just look at total return. Evaluate these metrics:

• Sharpe Ratio: Risk-adjusted return (above 1.0 is good, above 2.0 is excellent)
• Maximum Drawdown: The worst peak-to-trough decline (would you survive this psychologically and financially?)
• Win Rate + Average Win/Loss: Together these determine if the strategy has positive expected value
• Number of Trades: Fewer than 30 trades is statistically meaningless. Aim for 100+ trades in your test period.
• Profit Factor: Gross profits / gross losses (above 1.5 is good)

Red Flags in Backtests

Be immediately suspicious if you see:

• Sharpe ratio above 3.0: Almost certainly overfitted. Real-world Sharpe ratios for good strategies are 1.0-2.5.
• Win rate above 80%: Unless it's a very specific pattern with tight filters, this usually indicates curve-fitting.
• Tested only on bull markets: A strategy that only works when everything goes up is not a strategy — it's just being long.
• Too many parameters: If your strategy has more than 5-7 tunable parameters, you're probably overfitting.
• No losing months: Every real strategy has losing periods. Zero losses in a backtest means something is wrong.

Key Takeaways

• Most backtests are misleading — assume yours is too until rigorously validated
• Always use out-of-sample data for evaluation and walk-forward optimization for robustness
• Include realistic transaction costs: brokerage, STT, slippage, and impact cost for Indian markets
• Validate with multiple metrics: Sharpe ratio, max drawdown, trade count, profit factor
• Red flags include Sharpe > 3, win rate > 80%, too many parameters, and no losing periods

Conclusion

Backtesting is simultaneously the most valuable and most dangerous tool in an algo trader's arsenal. Done right, it gives you confidence to deploy a strategy with real capital. Done wrong, it gives you false confidence that leads to real losses. Follow the framework in this article, be ruthlessly honest with yourself about your results, and always paper trade for at least 2-3 months before going live.

This content is for educational purposes only and does not constitute investment advice.