Our text book assumes that you already know about probability or are willing to accept a few formulas. Let’s fill in some details. This will be review for many of you, but we want to make sure everyone is comfortable with a few important probability notions.
We will start with a somewhat silly example, but then see how it is related to Bayesian computaion.
Here is a table of hair and eye color for some men.
| Brown | Blue | Hazel | Green | |
|---|---|---|---|---|
| Black | 32 | 11 | 10 | 3 |
| Brown | 53 | 50 | 25 | 15 |
| Red | 10 | 10 | 7 | 7 |
| Blond | 3 | 30 | 5 | 8 |
| Brown | Blue | Hazel | Green | total | |
|---|---|---|---|---|---|
| Black | 32 | 11 | 10 | 3 | 56 |
| Brown | 53 | 50 | 25 | 15 | 143 |
| Red | 10 | 10 | 7 | 7 | 34 |
| Blond | 3 | 30 | 5 | 8 | 46 |
| total | 98 | 101 | 47 | 33 | 279 |
1. What proportion of these men have green eyes?
P(green eyes) = \(\frac{N(\mbox{green eyes})}{N} = \frac{3 + 15 + 7 + 8}{ 279 } = \frac{33}{279} = 0.118\)
2. What proportion of these men have red hair?
P(red hair) = \(\frac{N(\mbox{red hair})}{N} = \frac{10 + 10 + 7 + 7}{ 279 } = \frac{34}{279} = 0.122\)
3. What proportion of these men have green eyes and red hair?
P(green eyes, red hair) = \(\frac{N(\mbox{green eyes, red hair})}{N} = \frac{7}{279} = 0.025\)
4. What proportion of these men with green eyes have red hair?
P(red hair | green eyes) = \(\frac{N(\mbox{green eyes, red hair})}{N(\mbox{green eyes})} = \frac{7}{33} = 0.212\)
5. What proportion of these men with red hair have green eyes?
P(red hair | green eyes) = \(\frac{N(\mbox{green eyes, red hair})}{N(\mbox{red hair})} = \frac{7}{34} = 0.206\)
This time, the information is presented as proportions rather than counts.
| Brown | Blue | Hazel | Green | |
|---|---|---|---|---|
| Black | 0.115 | 0.029 | 0.016 | 0.006 |
| Brown | 0.211 | 0.109 | 0.093 | 0.045 |
| Red | 0.051 | 0.022 | 0.022 | 0.022 |
| Blond | 0.013 | 0.204 | 0.016 | 0.026 |
| Brown | Blue | Hazel | Green | total | |
|---|---|---|---|---|---|
| Black | 0.115 | 0.029 | 0.016 | 0.006 | 0.166 |
| Brown | 0.211 | 0.109 | 0.093 | 0.045 | 0.458 |
| Red | 0.051 | 0.022 | 0.022 | 0.022 | 0.117 |
| Blond | 0.013 | 0.204 | 0.016 | 0.026 | 0.259 |
| total | 0.39 | 0.364 | 0.147 | 0.099 | 1 |
1. What proportion of these women have green eyes?
2. What proportion of these women have red hair?
3. What proportion of these women have green eyes and red hair?
4. What proportion of these women with green eyes have red hair?
5. What proportion of these women with red hair have green eyes?
P(green eyes) = 0.006 + 0.045 + 0.022 + 0.026 = 0.099
P(red hair) = \(0.051 + 0.022 + 0.022 + 0.022 = 0.117\)
P(green eyes, red hair) = \(0.022\) – this is exactly what the table tells us.
\[\begin{align*} P(\mbox{red hair} \mid \mbox{green eyes}) & = \frac{N(\mbox{green eyes, red hair})}{N(\mbox{green eyes})} \\ & = \frac{N(\mbox{green eyes, red hair}) / N}{N(\mbox{green eyes}) / N} \\ & = \frac{P(\mbox{green eyes, red hair})}{P(\mbox{green eyes})} \\ & = \frac{0.022}{0.099} \\ & = 0.222 \end{align*}\]
Note: This gives us the standard formula for conditinal probability:
\[ P(A \mid B) = \frac{P(A ,B)}{P(B)} = \frac{P(\mbox{both})}{P(\mbox{condition})} \]
\[\begin{align*} P(\mbox{red hair} \mid \mbox{green eyes}) &= \frac{P(\mbox{red hair, green eyes})}{P(\mbox{green eyes})} \\ &= \frac{0.022}{0.117} \\ &= 0.188 \end{align*}\]
Now that we have two scenarios (one with men and one with women), we could adopt a more careful notation. Examples:
\[ P( \mbox{red hair} \mid \mbox{green eyes, male} ) \] \[ P( \mbox{green eyes, red hair} \mid \mbox{male} ) \]
Conditional Probabilty Summary
This is all really just a set up to get us thinking more generally about conditional probability, which will be critical for the Bayesian framework.
\[ P(A \mid B) = \frac{P(\mbox{both})}{P(\mbox{condition})} = \frac{P(\mbox{joint})}{P(\mbox{marginal})} = \frac{P(A, B)}{P(B)} \] If \(P(A \mid B) = P(A)\), then we say that \(A\) and \(B\) are independent. (Why is this a good name for this concept?)
Rearranging:
\[\begin{align*} P(A, B) &= P(B) \cdot P(A \mid B) \\ &= P(A) \cdot P(B \mid A) \end{align*}\]
More rearranging
\[\begin{align*} P(B) P(A \mid B) &= P(A) \cdot P(B \mid A) \\[5mm] P(A \mid B) &= \frac{P(A) \cdot P(B \mid A)}{P(B)} \end{align*}\]
Here are some more practice problems for you.
1. Among the women, what proportion of people have blond hair? What notation do we use for this? Is this a joint, marginal, or conditional probability?
2. Among the women, what proportion of people have blue eyes? What notation do we use for this? Is this a joint, marginal, or conditional probability?
3. Among the women, what proportion of people have blond hair and blue eyes? What notation do we use for this? Is this a joint, marginal, or conditional probability?
4. Among the women, what proportion of people with blue eyes have blond hair? What notation do we use for this? Is this a joint, marginal, or conditional probability?
5. Among the women, what proportion of people with blond hair have blue eyes? What notation do we use for this? Is this a joint, marginal, or conditional probability?
The proportion of men is 0.471. Use this to answer the following questions.
6. If you find out a person has blond hair and blue eyes, what is the probability that they are male?
7. If you find out a person has blond hair, what is the probability that they are male?