Suppose 3 students are randomly selected from a class. Assume they are all born in a non-leap year. What is the probability that none of them share a same birthday?
We will first identify the experiment, the outcomes, and the sample space of the given problem. The mechanism we are concerned about is the act of picking 3 random students. From this experiment, we observe a triple of birth days. Therefore, the sample space we are working with is all possible triples of birthdays in a non-leap year.
This is an example of multiple experiments with equally likely outcomes. The product of the sample space for each birth make up the whole sample space and we know that
$$\lvert\Omega\rvert=360\times365\times365=365^3.$$
We will define \(E_1\)
to be the event that the first two students share a birthday, \(E_2\)
the event that last two, and \(E_3\)
the event that the first and the last students share a birthday. We are interested in \(P(E_1^c\cap E_2^c\cap E_3^c)\)
, or equivalently, \(1-P(E_1\cup E_2\cup E_3)\)
.
To compute the probability, we may count the outcomes that satisfy the event of interest. Alternatively, we can compute the probability using a conditional probability.
Let’s first consider the first pair of birthdays. We can easily count the number of outcomes that satisfy \(E_1\)
to compute its probability:
$$P(E_1)=\frac{(\text{# of days the first pair can share a birthday})}{\lvert\Omega_1\times\Omega_2\rvert}$$
where \(\Omega_1\)
and \(\Omega_2\)
are the samples spaces for the first and second students respectively. Alternatively, we can think of the probability as
$$P(E_1)=\frac{\begin{pmatrix}\text{# of days the second birthday matches the first}\\ \text{regardless of when the first birthday is} \end{pmatrix}}{\lvert\Omega_2\rvert}.$$
In both cases, the probability is 1 out of 365. Therefore, the probability of the first pair having different birthdays is
$$P(E_1^c)=1-P(E_1)=\frac{364}{365}.$$
Now, let’s consider the third person. Suppose you know that the first two do not share a birthday. With the additional knowledge, we can count the number of ways that the third student shares a birthday with the first or the second student. That is,
$$\frac{\begin{pmatrix} \text{# of days the third person can share a birthday} \\ \text{with the first or second student} \\ \text{given that the first two do not share a birthday} \end{pmatrix}}{\lvert\Omega_3\rvert}=\frac{2}{365}.$$
The event that the third student shares a birthday not with the first student and not with the second person is the complement of the event above given the first two do not share a birthday. We can thus use the probability of the complement and get
$$1-\frac{2}{365} = \frac{363}{365}=P(E_2^c\cap E_3^c\left| E_1^c\right.).$$
The probability is an example of the conditional probability.
A conditional probability describes teh probability of an event, say \(A\)
, given that another event, say \(C\)
, occured. We also call the probability as the probability of event \(A\)
conditioned on event \(C\)
.
The conditional probability of \(A\)
given \(C\)
is defined as
$$P(A|C)=\frac{P(A\cap C)}{P(C)}$$
for any \(C\)
such that \(P(C)>0\)
.
Recall that one interpretation of the probability of an event is the long-term relative frequency of the event. Similarly, we can interpretate the conditional probability of \(A\)
given \(C\)
as the long-term relative frequency of event \(A\)
occuring among the times event \(C\)
occurs.
We derive the following rule by rearranging the definition of the conditional probability.
The multiplication rule states that for any event \(A\)
and \(C\)
, we have
$$P(A\cap C)=P(A|C)\cdot P(C).$$
Applying this rule to the birthdays example, we can answer the original question of the probability of all three selected students having different birthdays.
$$P\left(E_1^c\cap \left(E_2^c\cap E_3^c\right)\right) = P\left(\left(E_2^c\cap E_3^c\right)\left| E_1^c\right.\right) \cdot P\left(E_1^c\right) =\frac{363}{365}\cdot\frac{364}{365}$$
(This example has been adopted from (Dekking et al. 2005, Exercise 3.10, p. 38).)
Suppose Michael knows the answer to a multiple choice question with a probability of \(3/5\)
. When he does not know the answer, he picks an answer at random. The probability of picking the correct answer is \(1/4\)
. When Michael knows the answer, he answers correctly with a probability of \(1\)
. What is the probability that Michael correctly answers a multiple choice question?
Let’s \(K\)
be the event that Michael knows the answer and \(Y\)
the event that Michael answers correctly. Then, we can summarise the provided probabilities as follows:
$$P(K)=\frac{3}{5},\quad P\left(Y\left|K^c\right.\right)=\frac{1}{4},\quad \text{and}\quad P\left(Y\left|K\right.\right)=1.$$
$$\mathbf{K}$$
$$P(K)=3/5$$
$$\mathbf{Y}$$
$$P(Y|K)=1$$
$$\therefore P(Y\cap K)=1\cdot 3/5$$
$$\mathbf{Y^c}$$
$$P(Y^c|K)=0$$
$$\therefore P(Y^c\cap K)=0\cdot 3/5$$
$$\mathbf{K^c}$$
$$P(K^c)=2/5$$
$$\mathbf{Y}$$
$$P(Y|K^c)=1/4$$
$$\therefore P(Y\cap K^c)=1/4\cdot 2/5$$
$$\mathbf{Y^c}$$
$$P(Y^c|K^c)=3/4$$
$$\therefore P(Y^c\cap K^c)=3/4\cdot 2/5$$
Note that the event \(Y\)
can be divided into two disjoint subsets \(Y\cap K\)
and \(Y\cap K^c\)
. Therefore, we have
$$\begin{split}P(Y) &=& P(Y\cap K)+P(Y\cap K^c) \\ &=&P\left(Y\left|K\right.\right)P\left(K\right) + P\left(Y\left|K^c\right.\right)P\left(K^c\right) \\ &=&\frac{3}{5} + \frac{1}{4}\cdot\frac{2}{5}=\frac{7}{10}. \end{split}$$
Generalizing the previous result is the law of total probability.
Suppose \(C_1,C_2,\ldots,C_m\)
are disjoint events such that \(C_1\cup C_2\cup \cdots\cup C_m=\Omega\)
. The Law of Total Probability states that
$$P(A)=\sum_{i=1}^m\left[P\left(A\left|C_i\right.\right) P\left(C_i\right)\right]$$
for any arbitrary event \(A\)
.
Bayes’ rule is a result derived based on the law of tota probability and the multiplication rule.
Suppose \(C_1,C_2,\ldots,C_m\)
are disjoint events such that \(C_1\cup C_2\cup \cdots\cup C_m=\Omega\)
. Bayes’ Rule states that the conditional probability of \(C_i\)
given an arbitrary event \(A\)
is
$$P\left(\left. C_i\right| A\right) =\frac{P\left(A\left| C_i\right.\right)P\left(C_i\right)}{ \sum_{i=1}^m \left[P\left(A\left|C_i\right.\right) P\left(C_i\right)\right]}.$$
Let’s consider the multiple choice example again. Suppose you are the Michael’s teacher and grade a multiple question. Provided that he answered the question correctly, you can compute the probability that Michael knew the answer using Bayes’ rule.
$$P(K|Y)=\frac{P(Y|K)P(K)}{P(Y|K)P(K) + P\left(Y\left|K^c\right.\right)P\left(K^c\right)} = \frac{1\cdot3/5}{7/10}=\frac{6}{7}$$
An event \(A\)
is called independent of \(B\)
if
$$P(A|B)=P(A).$$
The definition expresses an event having no effect on another event using probabilities. From the mathematical representation, we can derive the following equivalent definitions.
To show that any two events \(A\)
and \(B\)
are independent, it is sufficient to prove any of the following
$$P(A|B)=P(A)$$
$$P(A\cap B)=P(A)P(B)$$
$$P(B|A) = P(B)$$
You may also replace \(A\)
with \(A^c\)
, \(B\)
with \(B^c\)
, or both. If you show that any one of them is not true, you show that the two events are dependent.
The alternative definition \(P(A\cap B)=P(A)P(B)\)
makes it easy to extene the definition to more than two events.
Events \(A_1,A_2,\ldots,A_m\)
are called independent if
$$P\left(A_1\cap A_2\cap\cdots\cap A_m\right) =\prod_{i=1}^m P\left(A_i\right).$$
The statement holds when any number of events are replaced by their complements.
Revisiting the multiple choice example, we note that
$$P(K)=\frac{3}{5}\neq\frac{6}{7}=P(K|Y).$$
Therefore, the two events are not independent. This may be intrepreted as follows.
\(P(K)=3/5\)
prior to any assessments.Dekking, Frederik Michel, Cornelis Kraaikamp, Hendrik Paul Lopuhaä, and Ludolf Erwin Meester. 2005. A Modern Introduction to Probability and Statistics: Understanding Why and How. Springer Science & Business Media.