Bayes’ theorem

Where it comes from

Real life constantly hands us probabilities facing the wrong way. A test manufacturer tells you P(positive | disease) — “our test catches 99% of infections.” But the patient does not want that number. The patient wants P(disease | positive) — “given that I tested positive, what are the odds I am actually sick?” The first is about the test; the second is about you. Bayes’ theorem is the machine that flips one into the other.

The naive approach is to treat the flip as free — to read P(A|B) and P(B|A) as the same thing:

Kill it with one concrete example. Suppose 1% of the population has a disease, and a test is 99% sensitive (catches 99% of the sick) and 99% specific (correctly clears 99% of the healthy). Run the numbers:

Why? Think in people, not probabilities. Out of 10,000: 100 are sick and the test catches 99 of them; 9,900 are healthy and the test wrongly flags 1% of them — another 99. So there are 198 positives on the table, and only 99 of them are actually sick: 99/198 = 50%. The false positives exactly match the true positives, because the healthy population is 99 times bigger. Ignoring that base rate is called base-rate neglect, and it is the single most common probability error in the wild.

Here is the intuition before the formalism. Read the formula as a belief update: you start with a prior P(A) — how likely A was before the evidence. Evidence B arrives. You multiply by how likely that evidence should be if A were true, P(B|A). And you divide by the total probability of seeing the evidence at all, P(B) — how likely B is across every hypothesis. Evidence that would be surprising under all alternatives is strong evidence; evidence that happens all the time anyway barely moves the needle. That division is the whole trick — it re-normalizes, so your updated beliefs still sum to 1.

Derivation

Bayes’ theorem needs no new axioms — it is two applications of the definition of conditional probability, back to back. The second line is the whole proof.

In practice the denominator P(B) is never given — you build it from the hypotheses with the law of total probability:

Why this form? It makes the normalizing role of the denominator visible: the numerator is one hypothesis’s “share” of the evidence, and the denominator is the sum of every hypothesis’s share. That is why the posteriors over a full partition always sum to 1.

How to use it

The procedure, every time:

1. Name A = the hypothesis, B = the evidence. A is what you want to know (“patient has the disease”); B is what you observed (“the test came back positive”). Write them down explicitly — most Bayes errors start with A and B swapped.
2. List the three inputs. The prior P(A) (base rate, before the evidence); the likelihood P(B|A) (how the evidence behaves if the hypothesis is true); and the false-positive rate P(B|¬A) (how the evidence behaves if the hypothesis is false).
3. Build P(B) with total probability. P(B) = P(B|A)·P(A) + P(B|¬A)·P(¬A). Never guess the denominator, never set it to 0.5 “by symmetry” — it is determined by your model.
4. Divide. P(A|B) = P(B|A)·P(A) / P(B). That is your posterior: the updated belief.
5. Sanity-check. The posterior must lie between 0 and 1, and posteriors over a full partition must sum to 1. Also check the direction: if the evidence is likelier under A than under not-A, the posterior should be above the prior. If it is not, recheck your inputs.

Chain it: yesterday’s posterior is today’s prior

Bayes is meant to be applied repeatedly. Got a second, independent test? Do not start over — the posterior from the first test becomes the prior for the second. Each round of evidence multiplies in, and beliefs converge toward the truth. See Example 3 below.

When the prior is shaky

Sometimes nobody knows the base rate. That does not break the method — it exposes the subjectivity debate: different priors give different posteriors. Two defenses: (1) use the best data you have and say so; (2) with enough evidence the likelihood overwhelms the prior and honest analysts converge anyway. One hard rule: never use a prior of exactly 0 or 1 (Cromwell’s rule) — 0 × likelihood stays 0 forever, so no evidence could ever move you.

Worked examples

Four problems, easiest first. In each one, read every step — the why of each move is the lesson.

Memorization tips

• Say it aloud: “posterior equals likelihood times prior, over evidence.” Three named parts, one division — the rhythm matches the formula.
• Name the parts every time: P(A) = prior, P(B|A) = likelihood, P(B) = evidence. A problem half-solved is a problem with A and B labeled.
• Draw the tree: first branches = hypotheses (priors), second branches = evidence (likelihoods). Bayes just re-weights the paths that end at B. If you can draw the tree, you can build P(B) — just add the B-paths.
• P(A|B) ≠ P(B|A) — the whole point of the theorem is that they differ. Confusing them is the prosecutor’s fallacy, and it costs marks, cases, and diagnoses.
• Respect the base rate: rare things stay unlikely even after positive evidence. If your posterior ignores the prior, you have committed base-rate neglect.
• The 50% test (your 5-second self-check): 1% base rate, 99%/99% test, one positive → exactly 50%. Run it in your head whenever a Bayes answer feels off — if your method cannot reproduce 50%, the method is wrong.

Final challenge

Five mixed questions — basics, applications, and the traps, all in one. Score 5/5 and Bayes’ theorem is yours.