Quant Analytics: from scratch to pro · Machine learning in finance · Advanced
Leakage, purged cross-validation and embargo
Ordinary cross-validation assumes independent samples. Market labels overlap in time, so a random split puts near-copies of the test data into training and inflates the score. Purging and an embargo remove them.
In this lesson
- Recognise the common forms of data leakage in a model-building process
- Explain why a random k-fold split overstates performance on time-dependent data
- Apply purging and an embargo and count the training samples that remain
- Keep preprocessing inside each training fold
Read first: Features and labels: fixed horizons and the triple barrier, Look-ahead bias and other leaks into a backtest
A backtest or a model score is an estimate of how the method will do on data it has not seen. Leakage is anything that lets information from the test data reach the fitting, so the estimate is too good. In market data it is common, quiet and large.
Where leakage comes from
- Features that use the future: a rolling statistic centred on the sample, a bar's close used before it exists, revised data used as if it were first published.
- Preprocessing on the whole sample: scaling, filling gaps or choosing features with statistics from all data, including the test period.
- Overlapping labels: a label that spans several observations shares information with its neighbours (see features and labels).
- Repeated tuning: using the same test data to choose settings again and again turns the test data into training data.
- Selection after the fact: reporting only the best of many tries. See overfitting.
Why ordinary cross-validation fails
In k-fold cross-validation, each fold takes a turn as the test set, and the others train the model. It assumes samples are independent. Here they are not.
Take daily samples with 10-day labels. The sample on day 50 and the sample on day 51 share 9 of 10 days. If day 50 is in the test set and day 51 in training, the model has seen the answer to the test question, with a one-day shift.
Purging and embargo
López de Prado (2018) proposes two fixes. See purged cross-validation.
- Purge. Remove from the training set every sample whose label window overlaps the test fold.
- Embargo. Also remove a short stretch of training samples immediately after the test fold, because serial dependence carries information forward across the boundary. A common choice is a small percentage of the sample.
Folds stay contiguous blocks in time, not random picks, so the structure is respected. Both steps cost training data, which is the price of an honest score.
Worked example
There are 1,000 daily samples with 10-observation labels, and 5 folds of 200.
With a random split, each test sample has 2 x 9, which is 18, neighbours sharing part of its window. With about 80% of samples in training, the expected number of overlapping training samples per test sample is 14.4. The score is inflated by near-copies.
With purged k-fold on a middle fold:
- Test fold: 200 samples.
- Purge: the 9 samples before the fold, whose 10-observation labels reach into it.
- Embargo: 1% of 1,000, which is 10 samples after the fold.
- Training:
1,000 - 200 - 9 - 10, which is 781, or 78% of the sample.
The first fold has no purge before it, so it keeps 790. Compare the score with and without purging. A large drop when purging is switched on is the measure of how much of the earlier score was leakage.
Keep preprocessing inside the fold
Fit every data-dependent step (scaling, feature selection, imputation) on the training part of each fold, and only apply it to the test part. A workflow that does this by construction is safer than care. A final, locked holdout that no step touches until the end is the last guard; see walk-forward and the locked holdout.
Common mistakes
- Shuffling time-ordered data before splitting.
- Standardising on the full sample.
- Purging for the label length but forgetting the embargo.
- Choosing the number of folds or the embargo after seeing the score.
- Tuning on the test folds and then reporting their score.
- Believing a very high score. In market data, an excellent score is more often a leak than a discovery.
In practice
Treat any surprisingly good score as a bug until proven otherwise: switch off each possible leak in turn and watch the score. Report the scores with purging and embargo.
Worked numbers
Each result is recomputed from the formula shown.
| What | Formula | Result |
|---|---|---|
| samples | 1000 | 1,000 |
| test fold size | n / 5 | 200 |
| neighbours sharing a label window | 2 * 9 | 18 |
| expected overlapping training samples per test sample | neighbours * 0.8 | 14.4 |
| embargo at 1% of the sample | 0.01 * n | 10 |
| training samples, middle fold | n - fold - 9 - embargo | 781 |
| training samples, first fold | n - fold - embargo | 790 |
| share of the sample used for training, middle fold | train_mid / n | 78% |
Check yourself
Labels span 10 observations. A random split puts a test sample next to training samples. How many neighbours share part of its label window, and what fraction of them are in training with 5 folds?
9 neighbours on each side share some of the window, so 18. With 5 folds about 80% are in training: 18 x 0.8 = 14.4 overlapping training samples per test sample, on average.
1,000 samples, 5 folds, labels of length 10, embargo of 1%. How many training samples remain for a middle fold?
Test fold: 200. Purge: the 9 samples before the test fold whose 10-observation labels reach into it. Embargo: 1% of 1,000 = 10 after it. Training = 1,000 - 200 - 9 - 10 = 781.
Why is an embargo needed on top of purging?
Purging removes training samples whose labels overlap the test period. Serial dependence (volatility clustering, slow-moving features) also carries information from the test period into samples that start just after it. The embargo removes that stretch.
A feature is standardised using the mean and standard deviation of the whole data set before splitting. Is that leakage?
Yes. The test period contributes to the scaling used in training. The statistics must be computed on each training fold only and then applied to the test fold.
A score rises when the purge is switched off. What does that tell you?
That part of the score came from overlap between training and test labels, not from skill that carries to new data.
Key terms
Data leakageCross-validationPurged cross-validationTriple-barrier method
Sources
- Advances in Financial Machine Learning. Wiley (Marcos López de Prado), 2018. Accessed 9 Nov 2026
- The Elements of Statistical Learning (2nd edition). Springer (Trevor Hastie, Robert Tibshirani, Jerome Friedman), 2009. Accessed 9 Nov 2026
- Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance. Notices of the American Mathematical Society 61(5), 458-471 (Bailey, Borwein, López de Prado, Zhu), 2014. Accessed 9 Nov 2026