STA237: Probability, Statistics, and Data Analysis I
PhD Student, DoSS, University of Toronto
Monday, May 15, 2023
flowchart LR
event((<font size=4em>Outcome))---fn(<font color=#386CB0><font size=4em>Random <br />Variable)-->value((<font color=#386CB0><font size=4em>Real<br />Number))
style event fill: #ffffff, stroke: #696969, stroke-width: 2px;
style fn fill: #ffffff, stroke: #696969, stroke-width: 2px;
style value fill: #ffffff, stroke: #696969, stroke-width: 2px;
Let \(\Omega\) be a sample space. A random variable \(X\) is a function that maps \(\Omega\) on to real number (\(\mathbb{R}\)), \[X:\Omega\to\mathbb{R}.\]
When a random variable \(X\) takes a countable number of values, it is called a discrete random variable.
Note that a discrete random variable may take an infinite number of values.
Suppose you play Rock Paper Scissors with a housemate. Assume both you and your friend pick each hand randomly.
In each round, the winner takes $1, $2, and $3 from the loser when the winner wins with a rock, paper, and scissors respectively.
When the round is a tie, you bot put $3 in a communal cash box that you share with other housemates. i.e., you both lose $3.
What is the probability that you win more than $5 after playing 3 rounds?
Let \(X_i\) be the amount you earn in \(i\)th round and denote each round’s outcome with
\((\) your hand, your housemate’s hand \()\).
e.g., \(X_i(\{\) (, ) \(\})=3\).
\(X_i\) maps each possible outcome from a round to a dollar amount.
Your hand
Their hand
\[-3\]
\[2\]
\[-1\]
\[-2\]
\[-3\]
\[3\]
\[1\]
\[-3\]
\[-3\]
Because the underlying mechanism is random, the value it takes is also random.
The underlying experiment also determines the probability associated with each possible value \(X_i\) can take.
Your hand
Their hand
\[-3\]
\[2\]
\[-1\]
\[-2\]
\[-3\]
\[3\]
\[1\]
\[-3\]
\[-3\]
Because the underlying mechanism is random, the value it takes is also random.
The underlying experiment also determines the probability associated with each possible value \(X_i\) can take.
\[x\]
\[-3\]
\[-2\]
\[-1\]
\[1\]
\[2\]
\[3\]
\[P\left(X=x\right)\]
\[\frac{4}{9}\]
\[\frac{1}{9}\]
\[\frac{1}{9}\]
\[\frac{1}{9}\]
\[\frac{1}{9}\]
\[\frac{1}{9}\]
\[P\left(X=x\right)=\begin{cases} \frac{4}{9} & x=-3 \\ \frac{1}{9} & x\in\left\{-2, -1, 1, 2, 3\right\} \\ 0 & \text{otherwise.} \end{cases}\]
We can omit \(\{\) inside probability functions with random variables.
Because the underlying mechanism is random, the value it takes is also random.
The underlying experiment also determines the probability associated with each possible value \(X_i\) can take.
The probability function defined for all possible values of a random variable describes the relative likelihoods or their distribution.
Such probability function is called probability mass function.
\[x\]
\[-3\]
\[-2\]
\[-1\]
\[1\]
\[2\]
\[3\]
\[P\left(X_i=x\right)\]
\[\frac{4}{9}\]
\[\frac{1}{9}\]
\[\frac{1}{9}\]
\[\frac{1}{9}\]
\[\frac{1}{9}\]
\[\frac{1}{9}\]
\[P\left(X_i=x\right)=\begin{cases} \frac{4}{9} & x=-3 \\ \frac{1}{9} & x\in\left\{-2, -1, 1, 2, 3\right\} \\ 0 & \text{otherwise.} \end{cases}\]
We can omit \(\{\) inside probability functions with random variables.
The probability mass function (pmf) uniquely defines (the behaviour of) a random variable.
The probability mass function \(p\) of a discrete random variable \(X\) is the function \[p:\mathbb{R}\to\left[0,1\right],\]
defined by
\[p(k)=P\left(X=k\right)\quad\] \[\quad\text{for }-\infty<k<\infty.\]
There are \(9^3\) equally likely outcomes from playing 3 rounds.
\(S=6\) when you win by one of
\((\$2, \$2, \$2)\),
\(3!\) arrangements of \((\$3, \$2, \$1)\).
\(S=7\) when you win by one of
\(3!/2!\) arrangements of \((\$3, \$3, \$1)\),
\(3\) arrangements of \((\$3, \$2, \$2)\).
\(S=8\) when you win by one of
\(3\) arrangements of \((\$3, \$3, \$2)\)
\(S=9\) when you win by \((\$3, \$3, \$3)\)
\[P(S>5)=\frac{7 + 6 + 3 + 1}{9^3}=\frac{17}{729}\]
The cumulative distribution function (cdf) also uniquely defines (the behaviour of) a random variable.
The cumulative distribution function, or distribution function \(F\) of a random variable \(X\) is the function
\[F:\mathbb{R}\to\left[0,1\right],\]
defined by
\[F(a)=P\left(X\le a\right)\quad\] \[\quad\text{for }-\infty < a<\infty.\]
\[P\left(S>5\right)\]
\[F_\color{red}{S}\left(\color{blue}{s}\right)\]
\(\theta\) is commonly written as \(p\) as it represents a probability. We will use \(\theta\) to avoid confusion with pmf.
A discrete random variable \(X\) has a Bernoulli distribution with parameter \(\theta\), \(0\le \theta\le 1\), if its probability mass function is given by
\[p_X(x)=\begin{cases} \theta & \text{when }x=1 \\ 1-\theta & \text{when }x=0.\end{cases}\]
We denote the distribution by \(\text{Ber}(\theta)\) and define the random variable by \(X\sim \text{Ber}(\theta)\).
In general, we can model experiments with exactly two possible outcomes with Bernoulli random variables.
Suppose you want ‘all-or-nothing’,
propose a 5-round game, and
play only scissors.
Assume that your friend still plays randomly.
What is the probability that you would win more than $0 after the 5 rounds?
$3
$3
$3
$3
$3
Let \(W_i\) be the \(\text{Ber}(\theta)\) random variable representing whether you win round \(i\).
We can denote the number of rounds you win as \(N=W_1 + W_2 + W_3 + W_4 + W_5\). You need to at least 3 rounds to win more than $0.
\[P(N \ge 3)=1 - P(N < 3) = 1 - P(N \le 2) = 1 - F_N(2)\]
Let’s consider \(F_N(2)=p_N(0) + p_N(1) + p_N(2)\)
\[p_N(x) = \binom{5}{x}\theta^x\left(1-\theta\right)^{5-x}\]
It’s important to remember the total number of objects, or often referred to as trials, \(n\) is a fixed parameter as well as \(\theta\) (or commonly \(p\) for the probability of event occurring, and sometimes \(q\) for the probability no event).
A discrete random variable \(X\) has a binomial distribution with parameters \(n\) and \(\theta\), \(n = 1, 2, 3, \ldots\), and \(0\le \theta \le 1\), if its probability mass function is given by
\[p_X(x)=\binom{n}{x}\theta^x(1-\theta)^{n-x} \quad\] \[\quad \text{for all }x=0, 1, 2, \ldots, n.\]
We denote the distribution by \(\text{Bin}(n,\theta)\).
The distribution describes a sum of \(n\) independent and identical Bernoulli trials.
The number of experiments (trials) is no longer fixed.
A discrete random variable \(X\) has a geometric distribution with parameter \(\theta\), \(0 < \theta \le 1\), if its probability mass function is given by
\[p_X(x)=(1-\theta)^{x-1}\theta\quad\] \[\quad\text{for } x=1,2,\ldots.\]
We denote this distribution by \(\text{Geo}(\theta)\).
The number of identical Bernoulli trials until the first event.
Suppose you take a high-resolution satellite picture the ocean and divide the picture into 9 lots as shown. The chance of one or more whales being captured in each lot is \(4/9\).
Observing whales in each lot is independent and all have the same probability.
Let \(Y_9\) be the number of lots with whales out of 9 lots.
\[ Y_9\sim\text{Binom}\left(9, 4/9\right) \]
You are interested in studying the the number of whales you capture in each picture, \(X\).
You realize there could be more than one whale in each lot and decide to make the lots smaller and smaller until they can fit only one per lot.
Assume the proportion of lots with whales remain constant at \(4/9\).
Q. How many lots do you need?
A. A very large number.
\[\vdots\]
\(\frac{\lambda^x}{x!}\) \(\cdot\)\(\left[\lim_{n\to\infty}\frac{n!}{\left(n-x\right)!n^x}\right]\) \(\cdot\)\(\left[\lim_{n\to\infty}\left(1-\lambda\left/n\right.\right)^{n}\right]\) \(\cdot\)\(\left[\lim_{n\to\infty}\left(1-\lambda\left/n\right.\right)^{-x}\right]\)
\(e=\lim_{x\to\infty}\left(1 + 1\left/x\right.\right)^x\)
The random variable captures the count of events in a fixed interval of Poisson process.
A discrete random variable \(X\) has a Poisson distribution with parameter \(\lambda\), \(\lambda > 0\), if its probability mass function is given by
\[p_X(x)=\frac{e^{-\lambda}\lambda^x}{x!}\quad\quad\text{for }x=0,1,2,\ldots.\]
We denote the distribution by \(\text{Pois}(\lambda)\).
(Dekking et al. Exercise 12.1)
Which of the following examples would
reasonably suit the Poisson process assumptions?
The times of bankruptcy of enterprises in the United States.
No. They tend to occur in clusters.
The times a chicken lays its eggs.
No. A chicken probably can’t lay a new egg immediately after laying one.
The times of airplane crashes in a worldwide registration.
Yes.
The locations of wrongly spelled words in a book.
Yes.
The times of traffic accidents at a crossroad.
Yes if you assume the accidents are minor and don’t affect the future traffic.
No if you assume major accidents occur and authorities block the traffic.
(Dekking et al. Exercise 12.2)
The number of customers that visit a bank on a day is modeled by a Poisson distribution. It is known that the probability of no customers at all is 0.00001. What is the expected number of customers?
\[\implies \lambda = -\log\left(0.00001\right)=11.513\]
learnr and run R worksheetClick here to install learnr on r.datatools.utoronto.ca
Follow this link to open the worksheet
If you see an error, try:
rlesson03 from Files paneOther steps you may try:
.Rmd and .R files on the home directory of r.datatools.utoronto.caTools > Global Optionsinstall.packages("learnr") in RStudio after the steps above or click hereChapter 4, Dekking et al.
Chapter 12, Dekking et al.
Quick Exercise 12.1
Exercises 12.3, 12.4, 12.5, 12.6
See a collection of corrections by the author here
© 2023. Michael J. Moon. University of Toronto.
Sharing, posting, selling, or using this material outside of your personal use in this course is NOT permitted under any circumstances.