What Is a Brier Score? A Worked Example
A Brier score is the mean squared error of probability forecasts. For each forecast, take the gap between the probability you gave and what happened (1 if it happened, 0 if not), square it, and average over all forecasts. For yes/no questions 0 is perfect, 1 is the worst possible, and always saying 50% scores 0.25. Lower is better. A real Workshop call is worked out below.
The definition
For N forecasts, let f be each stated probability and o the outcome, 1 if the event happened and 0 if it did not.
The score was introduced by Glenn Brier in 1950 to judge weather forecasts. For a yes/no question it runs from 0, a perfect forecast that gave 100% to what happened and 0% to what did not, up to 1, the worst possible. Saying 50% on every question scores exactly 0.25, which makes 0.25 the score of knowing nothing at all.
Why the gap is squared
Squaring punishes confident misses hard. Saying 95% and being wrong costs 0.9025, while saying 60% and being wrong costs 0.36. It also rewards honesty: the Brier score is a proper scoring rule, meaning your expected score is best when you report the probability you actually believe. Hedging toward 50% to look safe, or shouting 99% to look bold, both lose in the long run.
A small worked example
Suppose a forecaster makes three calls. The first is 90% and the event happens. The second is 20% and it does not. The third is 60% and it happens.
Call 2: (0.2 - 0)^2 = 0.04
Call 3: (0.6 - 1)^2 = 0.16
Brier score = (0.01 + 0.04 + 0.16) / 3 = 0.07
A forecaster who said 50% each time would score 0.25, so 0.07 is much better than knowing nothing.
A real Workshop call
Lowe's Earnings (Nov 18): Lowe's Met or Beat the $2.89 Estimate. We Said 7%
Will LOW report quarterly EPS below $2.89 (the analyst consensus 2.8946 rounded to the cent) for its report dated 2026-11-18? An exact $2.89 is a meet, not a miss.
Workshop's forecast is 7% that the answer is yes. The baseline, computed by code from past cases, is 11%. Using both rounded to whole percent:
Workshop: (0.07 - 0)^2 = 0.0049
Baseline: (0.11 - 0)^2 = 0.0121
Edge on this call: 0.0121 - 0.0049 = +0.0072
Workshop beat the baseline on this call. Lower Brier is better, so the smaller number wins. This is one call, and one call proves nothing; the record sums many. It settled November 21, 2026 (Era 4), using the final forecast. The official edge is time-weighted over the days the call was open.
Skill: scoring against a baseline
A raw Brier score cannot be judged alone, because easy questions flatter anyone. Imagine an event that happens 10% of the time. A forecaster who always says 10% scores 0.09 (0.1 x 0.9^2 + 0.9 x 0.1^2) without knowing anything about any single case. A raw 0.09 looks excellent and shows no skill at all.
So skill is measured against a reference forecast. The standard form is the Brier skill score, 1 minus the forecast's Brier score divided by the reference's; positive means better than the reference. Workshop reports the plain difference instead: the baseline's Brier score minus its own, called edge. Copying the baseline scores exactly zero, so only forecasts that add something show up. The baseline is explained in what a base rate is.
Why Workshop uses it
- It works on probabilities, so a forecast can be 31% instead of a forced yes or no.
- It cannot be gamed by hedging, because it is a proper scoring rule.
- Code can compute it the moment the outcome is known, so no model or person grades by feel.
- Set against a baseline, it separates real insight from restating the history. See the graded record.
It has limits. With few questions the score is noisy, it says nothing about whether a forecast would make money, and two forecasters are comparable only on the same questions.
Common questions
What is a good Brier score?
It depends on how predictable the questions are, so compare against a baseline rather than a fixed number. For yes/no questions, 0 is perfect and 0.25 is what always saying 50% scores. A score only counts as good if it beats a plain baseline on the same questions.
Is a lower Brier score better?
Yes. The score is an error, so lower is better. 0 is a perfect forecast and 1 is the worst possible for yes/no questions.
What is the Brier score for always guessing 50%?
0.25. Each forecast is wrong by 0.5, and 0.5 squared is 0.25, whatever the outcome.