Glossary
Overfitting
Tuning a rule so closely to past data that it fits noise as well as signal, so the in-sample result overstates what the rule does on new data.
Overfitting happens when a rule has many free settings, or many variants are tried, and the best on past data is kept. The winner partly reflects luck, so the result shrinks on new data.
The more trials, the larger the bias. It is measured by testing on data not used to choose the rule, and by correcting the best result for the number of trials (see the deflated Sharpe ratio).
Example: 20 rules with no edge, each tested at the 5% level, give a 64.15% chance that at least one looks significant (1 - 0.95^20).
Sources
- Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance. Notices of the American Mathematical Society 61(5), 458-471 (Bailey, Borwein, López de Prado, Zhu), 2014. Accessed 9 Nov 2026
- A Reality Check for Data Snooping. Econometrica 68(5), 1097-1126 (Halbert White), 2000. Accessed 9 Nov 2026