10 / 10 · Probability

Three meanings of probability

Counting, frequency and belief

You have met three ideas of what probability is, and mathematicians still argue about which one is fundamental.

Classical: probability is the fraction of equally likely outcomes. Clean, but you need to know the outcomes are equally likely first. Frequentist: probability is the long-run frequency, what the fraction settles to after endless repeats. Solid, but what does “the probability it rains tomorrow” mean when tomorrow happens once?

classical:    P = favourable ÷ total
frequentist:  P = limit of (successes ÷ tries)
bayesian:     belief after  ∝  belief before × likelihood

Try it · The coin may be unfair. Pick a starting belief, flip, and watch the curve of belief narrow around the true bias. Reveal the real value when you are ready.

Loading…

Bayesian: probability is a degree of belief, a number you hold about something uncertain and update with Bayes' theorem as evidence comes in. Two people with different starting beliefs can disagree at first, but enough evidence brings them together.

In practice all three agree when they apply. The coin below shows it: the classical guess, the observed frequency and the Bayesian belief all end up in the same place.

Key ideas

Words worth remembering

Prior: What you believe before seeing the evidence.
Posterior: What you believe after updating on the evidence.
Likelihood: How probable the evidence is under each possible answer.

Think about it

Is it rational for two people to start with different beliefs if they end up agreeing?