Update a probability with Bayes' theorem and recognize the base rate fallacy, without confusing P(A|B) and P(B|A).
A medical test: the Bayes calculation
10,000 people, prevalence 1%, sensitivity 99%, specificity 95%
Even with an accurate test, a positive result is rarely a true positive when the disease is rare. Bayes' theorem explains why.
Lesson notes
Bayes' theorem and the base rate fallacy
Bayes' theorem lets you update the probability of a hypothesis A when you get new data B: P(A|B) = P(B|A) · P(A) / P(B). Here P(A) is the initial (prior) probability before you see the data, P(B|A) is the probability of the data if the hypothesis is true (the sensitivity), and P(B) is the overall probability of the data.
Let's work through a medical example. A disease affects 1% of people (the prevalence). A test has a sensitivity of 99% — P(test+|disease) = 0.99 — and a specificity of 95% — P(test−|healthy) = 0.95, so P(test+|healthy) = 0.05. What does a positive test mean?
The calculation: out of 10,000 people, 100 are sick and 9,900 are healthy. Sick people with a positive test ≈ 99. Healthy people with a positive test ≈ 495. Total positive tests ≈ 594. P(disease|test+) = 99/594 ≈ 16.7%. Most positive tests are false, because the disease is rare.
This is the base rate fallacy: people intuitively think P(disease|test+) ≈ P(test+|disease) = 99%, but that's not true at all. P(disease|test+) depends on how rare the disease is. Always take the base rate — the original prevalence — into account.