Summary
Bayes’ theorem is a fundamental result in probability theory that describes how to update the probability of a hypothesis when new evidence becomes available. If $A$ represents a hypothesis and $B$ represents observed evidence, then
$P(A\mid B)=\frac{P(B\mid A)P(A)}{P(B)}$
where $P(A)$ is the prior probability of $A$, $P(B\mid A)$ is the likelihood of observing $B$ assuming $A$ is true, and $P(A\mid B)$ is the posterior probability after incorporating the evidence. The theorem follows directly from the definition of conditional probability and the identity
$P(A\cap B)=P(A\mid B)P(B)=P(B\mid A)P(A)$.
The importance of Bayes’ theorem lies in its ability to reverse conditional probabilities. Knowing, for example, how likely a positive medical test is when a patient has a disease does not immediately tell us how likely the patient is to have the disease after testing positive. Bayes’ theorem combines the test's accuracy with the disease's underlying prevalence to obtain that probability. This distinction is particularly important when dealing with rare events, because even a highly accurate test can produce a substantial proportion of false positives when the prior probability of the condition is very small.
The theorem is named after Thomas Bayes, whose work on conditional probability was published posthumously in 1763, although Pierre-Simon Laplace independently developed and greatly extended the approach. Today Bayes’ theorem forms the mathematical foundation of Bayesian inference, where beliefs about unknown parameters or competing hypotheses are repeatedly updated as new data arrive. Its applications extend across statistics, machine learning, medical diagnosis, genetics, cryptanalysis, decision theory, and many other areas involving reasoning under uncertainty.
Key takeaways
ARTICLE / ARTICLE [PDF]
Bayes’ theorem is a fundamental result in probability theory that describes how to update the probability of a hypothesis when new evidence becomes available. If $A$ represents a hypothesis and $B$ represents observed evidence, then
$P(A\mid B)=\frac{P(B\mid A)P(A)}{P(B)}$
where $P(A)$ is the prior probability of $A$, $P(B\mid A)$ is the likelihood of observing $B$ assuming $A$ is true, and $P(A\mid B)$ is the posterior probability after incorporating the evidence. The theorem follows directly from the definition of conditional probability and the identity
$P(A\cap B)=P(A\mid B)P(B)=P(B\mid A)P(A)$.
The importance of Bayes’ theorem lies in its ability to reverse conditional probabilities. Knowing, for example, how likely a positive medical test is when a patient has a disease does not immediately tell us how likely the patient is to have the disease after testing positive. Bayes’ theorem combines the test's accuracy with the disease's underlying prevalence to obtain that probability. This distinction is particularly important when dealing with rare events, because even a highly accurate test can produce a substantial proportion of false positives when the prior probability of the condition is very small.
The theorem is named after Thomas Bayes, whose work on conditional probability was published posthumously in 1763, although Pierre-Simon Laplace independently developed and greatly extended the approach. Today Bayes’ theorem forms the mathematical foundation of Bayesian inference, where beliefs about unknown parameters or competing hypotheses are repeatedly updated as new data arrive. Its applications extend across statistics, machine learning, medical diagnosis, genetics, cryptanalysis, decision theory, and many other areas involving reasoning under uncertainty.
Key takeaways
- Prior + evidence → posterior: Bayes’ theorem provides a systematic mathematical rule for updating probabilities.
- $P(A\mid B)\neq P(B\mid A)$: confusing these two conditional probabilities is a common error.
- Base rates matter: rare events can remain relatively unlikely even after apparently strong evidence.
- Bayes’ theorem is one of the central mathematical tools behind Bayesian statistics and probabilistic machine learning.
ARTICLE / ARTICLE [PDF]
┌────────────────────────────────┐
│ KONSTANTINOS MICHAILIDIS │
└────────────────────────────────┘
│ KONSTANTINOS MICHAILIDIS │
└────────────────────────────────┘