Skip to main content

Quant Analytics: from scratch to pro · Machine learning in finance · Beginner

Machine learning for market data: what it can and cannot do

Machine learning works well where signal is strong and the world stands still. Markets offer the opposite: weak signal, heavy noise and relationships that change. This lesson sets honest expectations and shows where the method helps.

3 min read Reviewed Revision 1

In this lesson

  • Explain why market data has a low signal-to-noise ratio
  • Compare a model's accuracy with the base rate before calling it skilful
  • Estimate how much a small accuracy gain is worth statistically
  • Describe where machine learning helps: as a filter on rules that already work

Read first: Returns: simple, log and compounded

Worked numbers

Each result is recomputed from the formula shown.

WhatFormulaResult
base rate of up days0.5353%
model accuracy0.5454%
standard error of accuracy on 500 days100 * sqrt(acc * (1 - acc) / 500)2.2
edge in standard errors(acc - base_rate) * 100 / se0.45
95% half-width of the accuracy1.96 * se4.4
variance explained at correlation 0.050.05 ** 20.25%
standard deviation of a noise correlation, 500 days100 / sqrt(500)4.5
best of 200 noise features, in standard deviationsphi_inv(1 - 0.5 / 200)2.8
typical best noise correlationz200 * noise_sd / 1000.126

Check yourself

Up days are 53% of all days. A model predicting 'up' every day is right how often?

53% of the time, with no skill at all. This is the base rate, and any accuracy figure must be read against it.

A model scores 54% on 500 test days. Is it better than the 53% base rate?

Not demonstrably. The standard error of an accuracy near 54% on 500 days is sqrt(0.54 x 0.46 / 500) = 2.2 points, so a 1-point edge is 0.45 standard errors, well inside noise.

A predictor's forecasts have a correlation of 0.05 with returns. How much variance does it explain?

0.05 squared = 0.25% of the variance. Small values like this can still be worth having, but they cannot be seen by eye.

Why does testing 200 candidate features on 500 days usually turn up one that looks useful?

Pure noise features have sample correlations with a standard deviation of 1/sqrt(500) = 4.5 points. The best of 200 is about 2.8 standard deviations out, a correlation near 0.126, by chance alone.

What does 'a filter on a rule that works' mean?

A rule already produces trades with positive expectancy for a reason that can be explained. The model only decides which of those trades to take. It does not invent the trades.

Key terms

Machine learningData leakageCross-validationConcept drift

Sources

Informational content. Not investment advice.