bayes() — Bayes’ theorem

Updates how sure you are after you get a new clue.

P(A|B) = P(A)·P(B|A) / P(B)

Key idea

• Prior = your starting guess before any clue. Posterior = your updated guess after the clue. Evidence = the clue itself. (e.g. prior → (evidence) → posterior)
• Bayes’ theorem is just the recipe that turns the prior into the posterior using the evidence.
• It weighs three things: how common the thing is (the base rate, i.e. the prior), how often the clue is right, and how often it gives a false alarm.
• When something is rare, even a positive test can mean a low real chance. (e.g. 1% base rate → ~10% truly sick)
• Easy trick: imagine 1,000 people and count the real alarms against the false alarms. (e.g. ≈ 9 real vs ≈ 79 false)
• The posterior (your updated chance) = real alarms ÷ (real alarms + false alarms).
• Adding up every way the clue can happen (real + false alarms) is called the law of total probability. (e.g. 0.9×0.01 + 0.08×0.99 = 0.0882)
• A higher base rate (prior) pushes the posterior up; a rarer one lets false alarms swamp it, pulling the posterior down. (e.g. 10% prior ≫ 1% prior)

Definition

$$\underset{\text{posterior}}{\underbrace{P(A\mid B)}}=\frac{\stackrel{\text{likelihood}}{\overbrace{P(B\mid A)}}\cdot \stackrel{\text{prior}}{\overbrace{P(A)}}}{\underset{\text{evidence}}{\underbrace{P(B)}}}$$

$P(A\mid B)$

the POSTERIOR — your updated chance of A after seeing the evidence (what you want)

$P(A)$

the PRIOR — your chance of A before any evidence (the base rate)

$P(B\mid A)$

the likelihood — how likely the evidence is when A is true (how strong the clue is)

$P(B)$

the EVIDENCE — the overall chance of seeing the clue either way (from the law of total probability)

In plain words

You think it might rain. Then you see dark clouds. Bayes helps you turn “maybe” into a better, updated chance of rain.

Where you'd use it

• Prior: a disease affects 1% of people. Evidence: a positive test that is right 90% of the time but false-alarms 8% of the time. Posterior: after the positive result, bayes shows your real chance of being sick is only about 10%, because the disease is so rare.
• Prior: 20% of emails are spam. Evidence: a flagged word that appears in 60% of spam but only 5% of normal mail. Posterior: an email containing that word is spam with probability roughly 75%.
• Prior: you think there is a 30% chance of rain. Evidence: dark clouds, which appear on 80% of rainy days but 20% of dry ones. Posterior: your updated chance of rain rises to about 63%.

Worked example 4

Prior P(sick): a disease hits 1% of people. Evidence: a positive test, which is right 90% of the time on real cases (the likelihood) but cries wolf 8% of the time. Posterior P(sick | positive): combining them, the real chance you are sick is only about 10%.

Frequently asked questions

What is the key idea of Bayes?

Update a prior belief with evidence to get a posterior. Crucially, P(A|B) ≠ P(B|A).

Why are most positive tests wrong for a rare disease?

The base rate is tiny, so false positives outnumber the true ones.