One of the main observations from our discussion about conditional probability and independence is that for two events $A$ and $B$, it is almost always the case that
$$ \text{Typically: } \,\, P\big(A \, | \, B\big) \neq P\big(B \, | \, A\big). $$
The irony is that, in many applications that involve collection of data, the term we are given is not the term that we are interested in.
The classic example occurs when taking a medical test to determine whether or not we have a certain condition. Let $C$ be the event that a person actually has the medical condition. Let $+$ be the event that the person tests positive. The critical question can be written mathematically like this:
$$ P\big(C \, | \, +\big) = \text{``The probability of having the condition given a positive test.''} $$
But the manufacturer’s of medical tests only know the following quantities:
Notice that the question we are naturally want to know is the reverse condition from the sensitivity.
We can think of Bayes’ Rule as the mathematical method for reversing the direction of conditional probability.
<aside> <img src="/icons/fleur-de-lis_purple.svg" alt="/icons/fleur-de-lis_purple.svg" width="40px" />
Suppose that a prior probability $P(C)$ is given. Then
$$ P\big(C \, | \, +\big) = \frac{P\big(+ \, | \, C) P(C)}{P(+)} $$
where $P(+)$ can be calculated through either of these two versions of the law of total probability:
$$ \begin{aligned} P(+) &= P(+\cap C) + P(+ \cap C') \\ &= P(+ \, | \, C) P(C) + P(+ \, | \, C') P(C') \end{aligned} $$
</aside>
This is a special case of the more general Bayes’ Rule given below. The easiest way to calculate this probability is through a tree diagram.
In the end, Bayes’ Rule is just a conditional probability calculation. The diagram helps isolate the necessary terms:
$$ P\big(C \, | \, +\big) = \frac{P(+ \cap C)}{P\big(+)} = \frac{P(+ \cap C)}{P\big(+ \cap \, C\big) + P\big(+ \cap \, C'\big)}. $$
Follow the branches that lead to $+$ events, and then see what proportion of those were due to the person genuinely having the condition.