Glossary
Brier score
The mean squared difference between forecast probabilities and outcomes (0 or 1). Lower is better; 0 is perfect.
The Brier score, introduced by Glenn Brier in 1950, is mean((p - o)^2) where p is the forecast probability and o is 1 if the event occurred and 0 otherwise. It rewards both calibration and sharpness. Compare it with the score of always forecasting the base rate.
Example: forecasts 0.8, 0.6, 0.3, 0.7, 0.2 with outcomes 1, 1, 0, 0, 0 have squared errors 0.04, 0.16, 0.09, 0.49, 0.04. The mean is 0.164, against 0.24 for always forecasting the base rate of 0.4.