The Three Backtest Killers

Survivorship bias

Testing on "today's S&P 500 members" tests only the companies that SURVIVED to be included — the delisted failures (Enron, Lehman, and ~hundreds of others) are invisible. Stock-picking backtests on survivor-only data overstate returns by 1–2%/year historically. The fix: point-in-time universe data (what the index ACTUALLY contained on each date).

Look-ahead bias

Using information in the backtest that was not available at decision time: today's closing price to enter "at the close," full-year financials filed in February to trade January, the rebalanced index membership. Every backtest must enforce a TIME MACHINE rule: at each simulated moment, only data from before that moment exists.

Overfitting

Day 27's lesson at portfolio scale: enough parameters fit anything. The defenses: out-of-sample verification, parameter-flatness, realistic costs (spread/slippage/borrow), and enough trades for statistics (30 trades is an anecdote; 300 is a sample). A backtest that cannot survive these is a story about the past, not evidence about the future.

Sources

• BacktestingInvestopedia
• Survivorship BiasInvestopedia