Workshop
Learn · ForecastingExplainer

What Is a Brier Score? A Worked Example

Workshop · Written by Workshop. Examples come from the live record.Updated November 21, 2026
Short answer

A Brier score is the mean squared error of probability forecasts. For each forecast, take the gap between the probability you gave and what happened (1 if it happened, 0 if not), square it, and average over all forecasts. For yes/no questions 0 is perfect, 1 is the worst possible, and always saying 50% scores 0.25. Lower is better. A real Workshop call is worked out below.

The definition

For N forecasts, let f be each stated probability and o the outcome, 1 if the event happened and 0 if it did not.

Brier score = (1/N) x sum of (f - o)^2

The score was introduced by Glenn Brier in 1950 to judge weather forecasts. For a yes/no question it runs from 0, a perfect forecast that gave 100% to what happened and 0% to what did not, up to 1, the worst possible. Saying 50% on every question scores exactly 0.25, which makes 0.25 the score of knowing nothing at all.

Why the gap is squared

Squaring punishes confident misses hard. Saying 95% and being wrong costs 0.9025, while saying 60% and being wrong costs 0.36. It also rewards honesty: the Brier score is a proper scoring rule, meaning your expected score is best when you report the probability you actually believe. Hedging toward 50% to look safe, or shouting 99% to look bold, both lose in the long run.

A small worked example

Suppose a forecaster makes three calls. The first is 90% and the event happens. The second is 20% and it does not. The third is 60% and it happens.

Call 1: (0.9 - 1)^2 = 0.01
Call 2: (0.2 - 0)^2 = 0.04
Call 3: (0.6 - 1)^2 = 0.16
Brier score = (0.01 + 0.04 + 0.16) / 3 = 0.07

A forecaster who said 50% each time would score 0.25, so 0.07 is much better than knowing nothing.

A real Workshop call

Lowe's Earnings (Nov 18): Lowe's Met or Beat the $2.89 Estimate. We Said 7%

Will LOW report quarterly EPS below $2.89 (the analyst consensus 2.8946 rounded to the cent) for its report dated 2026-11-18? An exact $2.89 is a meet, not a miss.

Workshop's forecast is 7% that the answer is yes. The baseline, computed by code from past cases, is 11%. Using both rounded to whole percent:

Result: no (outcome = 0)
Workshop: (0.07 - 0)^2 = 0.0049
Baseline: (0.11 - 0)^2 = 0.0121
Edge on this call: 0.0121 - 0.0049 = +0.0072

Workshop beat the baseline on this call. Lower Brier is better, so the smaller number wins. This is one call, and one call proves nothing; the record sums many. It settled November 21, 2026 (Era 4), using the final forecast. The official edge is time-weighted over the days the call was open.

Skill: scoring against a baseline

A raw Brier score cannot be judged alone, because easy questions flatter anyone. Imagine an event that happens 10% of the time. A forecaster who always says 10% scores 0.09 (0.1 x 0.9^2 + 0.9 x 0.1^2) without knowing anything about any single case. A raw 0.09 looks excellent and shows no skill at all.

So skill is measured against a reference forecast. The standard form is the Brier skill score, 1 minus the forecast's Brier score divided by the reference's; positive means better than the reference. Workshop reports the plain difference instead: the baseline's Brier score minus its own, called edge. Copying the baseline scores exactly zero, so only forecasts that add something show up. The baseline is explained in what a base rate is.

Why Workshop uses it

  • It works on probabilities, so a forecast can be 31% instead of a forced yes or no.
  • It cannot be gamed by hedging, because it is a proper scoring rule.
  • Code can compute it the moment the outcome is known, so no model or person grades by feel.
  • Set against a baseline, it separates real insight from restating the history. See the graded record.

It has limits. With few questions the score is noisy, it says nothing about whether a forecast would make money, and two forecasters are comparable only on the same questions.

Common questions

What is a good Brier score?

It depends on how predictable the questions are, so compare against a baseline rather than a fixed number. For yes/no questions, 0 is perfect and 0.25 is what always saying 50% scores. A score only counts as good if it beats a plain baseline on the same questions.

Is a lower Brier score better?

Yes. The score is an error, so lower is better. 0 is a perfect forecast and 1 is the worst possible for yes/no questions.

What is the Brier score for always guessing 50%?

0.25. Each forecast is wrong by 0.5, and 0.5 squared is 0.25, whatever the outcome.

Live calls on this topic

Keep reading