L1-Fundamentals of Probability. gives fundamentals of statistics and probability.

Unit 2
Probability and Distributions

Outline
● Fundamentals of Probability
● Conditional probability
● Bayes theorem

Why Learn Probability?
• Nothing in life is certain. In everything we do, we gauge the
chances of successful outcomes, from business to medicine to
the weather
• A probability provides a quantitative description of the
chances or likelihoods associated with various outcomes
• It provides a bridge between descriptive and inferential
statistics
Population Sample
Probability
Statistics

Probabilistic vs Statistical Reasoning
• Suppose I know exactly the proportions of car makes in
California. Then I can find the probability that the first
car I see in the street is a Ford. This is probabilistic
reasoning as I know the population and predict the
sample
• Now suppose that I do not know the proportions of car
makes in California, but would like to estimate them. I
observe a random sample of cars in the street and then I
have an estimate of the proportions of the population.
This is statistical reasoning

What is Probability?
• We used graphs and numerical measures to describe
data sets which were usually samples.
• We measured “how often” using
Relative frequency = f/n
Sample
And “How often”
= Relative frequency
Population
Probability
• As n gets larger,

Basic Concepts
• An experiment is the process by which an
observation (or measurement) is obtained.
• An event is an outcome of an experiment,
usually denoted by a capital letter.
–The basic element to which probability is
applied
–When an experiment is performed, a
particular event either happens, or it doesn’t!

Experiments and Events
• Experiment: Record an age
– A: person is 30 years old
– B: person is older than 65
• Experiment: Toss a die
– A: observe an odd number
– B: observe a number greater than 2

Basic Concepts
• Two events are mutually exclusive if, when one
event occurs, the other cannot, and vice versa.
•Experiment: Toss a die
–A: observe an odd number
–B: observe a number greater than 2
–C: observe a 6
–D: observe a 3
Not Mutually
Exclusive
Mutually
Exclusive
B and C?
B and D?

Basic Concepts
• An event that cannot be decomposed is called a
simple event.
• Denoted by E with a subscript.
• Each simple event will be assigned a probability,
measuring “how often” it occurs.
• The set of all simple events of an experiment is
called the sample space, S.

Example
• The die rolls:
• Simple events: Sample space:
1
2
3
4
5
6
E1
E2
E3
E4
E5
E6
S ={E1, E2, E3, E4, E5, E6}
S
•E1
•E6
•E2
•E3
•E4
•E5

Basic Concepts
• An event is a collection of one or more
simple events.
•The die toss:
–A: an odd number
–B: a number > 2
S
A ={E1, E3, E5}
B ={E3, E4, E5, E6}
B
A
•E1
•E6
•E2
•E3
•E4
•E5

The Probability of an Event
• The probability of an event A measures “how often” A will
occur. We write P(A).
• Suppose that an experiment is performed n times. The relative
frequency for an event A is
• If we let n get infinitely large,

The Probability of an Event
• P(A) must be between 0 and 1.
– If event A can never occur, P(A) = 0. If event A always
occurs when the experiment is performed, P(A) =1.
• The sum of the probabilities for all simple events in S equals
1.
• The probability of an event A is found by adding
the probabilities of all the simple events contained in
A.

– Suppose that 10% of the U.S. population has red hair.
Then for a person selected at random,
Finding Probabilities
• Probabilities can be found using
– Estimates from empirical studies
– Common sense estimates based on equally likely
events.
P(Head) = 1/2
P(Red hair) = .10
• Examples:
–Toss a fair coin.

Using Simple Events
• The probability of an event A is equal to the sum
of the probabilities of the simple events contained
in A
• If the simple events in an experiment are equally
likely, you can calculate

Example 1
Toss a fair coin twice. What is the probability of
observing at least one head?
H
1st Coin 2nd Coin Ei P(Ei)
H
T
T
H
T
HH
HT
TH
TT
1/4
1/4
1/4
1/4
P(at least 1 head)
= P(E1) + P(E2) + P(E3)
= 1/4 + 1/4 + 1/4 = 3/4

Example 2
A bowl contains three Balls, one red, one blue and one
green. A child selects two Balls at random. What is the
probability that at least one is red?
1st Ball 2nd Ball Ei P(Ei)
RB
RG
BR
BG
1/6
1/6
1/6
1/6
1/6
1/6
P(at least 1 red)
= P(RB) + P(BR)+ P(RG)
+ P(GR)
= 4/6 = 2/3
R
B
G
B
G
G
R
R
B GB
GR

Example 3
The sample space of throwing a pair of dice is

Example 3
6/36
(1,1),(1,2),(1,3),(1,4),
(1,5),(1,6)
Red die show 1
6/36
(1,1),(2,1),(3,1),(4,1),
(5,1),(6,1)
Green die show 1
5/36
(1,5),(2,4),(3,3),(4,2),
(5,1)
Dice add to 6
2/36
(1,2),(2,1)
Dice add to 3
Probability
Simple events
Event

Counting Rules
• Sample space of throwing 3 dice has 216
entries, sample space of throwing 4 dice
has 1296 entries, …
• At some point, we have to stop listing and
start thinking …
• We need some counting rules

The mn Rule
• If an experiment is performed in two stages, with m ways
to accomplish the first stage and n ways to accomplish
the second stage, then there are mn ways to accomplish
the experiment.
• This rule is easily extended to k stages, with the number
of ways equal to
n1 n2 n3 … nk
Example: Toss two coins. The total number of
simple events is: 2 × 2 = 4

Examples
Example: Toss three coins. The total number of
simple events is: 2 × 2 × 2 = 8
Example: Two Balls are drawn from a bag
containing two red and two blue Balls. The total
number of simple events is:
6 × 6 = 36
Example: Toss two dice. The total number of
simple events is:
m
m
4 × 3 = 12
Example: Toss three dice. The total number of
simple events is: 6 × 6 × 6 = 216

Permutations
• The number of ways you can arrange
n distinct objects, taking them r at a time is
Example: How many 3-digit lock combinations can we make from the
numbers 1, 2, 3, and 4?
The order of the choice is
important!

Combinations
• The number of distinct combinations of n distinct
objects that can be formed, taking them r at a time is
Example: Three members of a 5-person committee must be chosen to
form a subcommittee. How many different subcommittees could be
formed?
The order of
the choice is
not important!

S
Event Relations
The beauty of using events, rather than simple events, is that
we can combine events to make other events using logical
operations: and, or and not.
The union of two events, A and B, is the event that either A
or B or both occur when the experiment is performed. We
write
A ∪ B
A B

S
A B
Event Relations
The intersection of two events, A and B, is the event
that both A and B occur when the experiment is
performed. We write A ∩ B.
• If two events A and B are mutually exclusive,
then P(A ∩ B) = 0.

S
Event Relations
The complement of an event A consists of all outcomes
of the experiment that do not result in event A. We write
AC
.
A
AC

Example
Select a student from the classroom and
record his/her hair color and gender.
– A: student has brown hair
– B: student is female
– C: student is male
What is the relationship between events B and C?
•AC
:
•B∩C:
•B C:
∪
Mutually exclusive; B = CC
Student does not have brown hair
Student is both male and female = ∅
Student is either male and female = all students = S

Calculating Probabilities for Unions and
Complements
• There are special rules that will allow you to calculate
probabilities for composite events.
• The Additive Rule for Unions:
• For any two events, A and B, the probability of their
union, P(A ∪ B), is
A B

Example: Additive Rule
Example: Suppose that there were 120 students in
the classroom, and that they could be classified as
follows:
Brown Not Brown
Male 20 40
Female 30 30
A: brown hair
P(A) = 50/120
B: female
P(B) = 60/120
P(A B) = P(A) + P(B) – P(A∩B)
∪
= 50/120 + 60/120 - 30/120
= 80/120 = 2/3 Check: P(A B)
∪
= (20 + 30 + 30)/120

Example: Two Dice
A: red die show 1
B: green die show 1
P(A B) = P(A) + P(B) – P(A∩B)
∪
= 6/36 + 6/36 – 1/36
= 11/36

A Special Case
When two events A and B are mutually
exclusive, P(A∩B) = 0
and P(A B) = P(A) + P(B).
∪
Brown Not Brown
Male 20 40
Female 30 30
A: male with brown hair
P(A) = 20/120
B: female with brown hair
P(B) = 30/120
P(A B) = P(A) + P(B)
∪
= 20/120 + 30/120
= 50/120
A and B are mutually
exclusive, so that

Example: Two Dice
A: dice add to 3
B: dice add to 6
A and B are mutually
exclusive, so that
P(A B) = P(A) + P(B)
∪
= 2/36 + 6/36
= 8/36

Calculating Probabilities
for Complements
• We know that for any event A:
– P(A ∩ AC
) = 0
• Since either A or AC
must occur,
P(A ∪ AC
) =1
• so thatP(A ∪ AC
) = P(A)+ P(AC
) = 1
P(AC
) = 1 – P(A)
A
AC

Example
Brown Not Brown
Male 20 40
Female 30 30
A: male
P(A) = 60/120
B: female
P(B) = ?
P(B) = 1- P(A)
= 1- 60/120 = 60/120
A and B are
complementary, so that
Select a student at random from the
classroom. Define:

Calculating Probabilities for
Intersections
In the previous example, we found P(A ∩ B) directly
from the table. Sometimes this is impractical or
impossible. The rule for calculating P(A ∩ B) depends
on the idea of independent and dependent events.
Two events, A and B, are said to be independent if
the occurrence or nonoccurrence of one of the
events does not change the probability of the
occurrence of the other event.

Outline
● Fundamentals of Probability
● Conditional probability
● Bayes theorem
● Random variables - discrete and continuous,
● Distributions
○ Frequency
○ Binomial
○ Poisson
○ Normal
● Sampling
● Central Limit Theorem

Conditional Probabilities
The probability that A occurs, given that event B
has occurred is called the conditional
probability of A given B and is defined as
“given”

Example 1
Toss a fair coin twice. Define
– A: head on second toss
– B: head on first toss
HT
TH
TT
1/4
1/4
1/4
1/4
P(A|B) = ½
P(A|not B) = ½
HH

Example 2
A bowl contains five M&Ms®
, two red and three
blue. Randomly select two candies, and define
– A: second candy is red.
– B: first candy is blue.
m
m
m
m
m
P(A|B) =P(2nd
red|1st
blue)= 2/4 = 1/2
P(A|not B) = P(2nd
red|1st
red) = 1/4

Defining Independence
• We can redefine independence in terms of conditional
probabilities:
Two events A and B are independent if and only if
P(A|B) = P(A) or P(B|A) = P(B)
Otherwise, they are dependent.
• Once you’ve decided whether or not two events are
independent, you can use the following rule to
calculate their intersection.

The Multiplicative Rule for Intersections
• For any two events, A and B, the probability that both A and
B occur is
P(A ∩ B) = P(A) P(B given that A occurred)
= P(A)P(B|A)
• If the events A and B are independent, then the probability
that both A and B occur is
P(A ∩ B) = P(A)
P(B)

Example 1
In a certain population, 10% of the people can be
classified as being high risk for a heart attack. Three people are
randomly selected from this population. What is the probability
that exactly one of the three are high risk?
Define H: high risk N: not high risk
P(exactly one high risk) = P(HNN) + P(NHN) + P(NNH)
= P(H)P(N)P(N) + P(N)P(H)P(N) + P(N)P(N)P(H)
= (.1)(.9)(.9) + (.9)(.1)(.9) + (.9)(.9)(.1)= 3(.1)(.9)2
= .243

Example 2
Suppose we have additional information in the
previous example. We know that only 49% of the population are
female. Also, of the female patients, 8% are high risk. A single person
is selected at random. What is the probability that it is a high risk
female?
Define H: high risk F: female
From the example, P(F) = .49 and P(H|F) = .08. Use
the Multiplicative Rule:
P(high risk female) = P(H∩F)
= P(F)P(H|F) =.49(.08) = .0392

Bayes’ Rule
Let S1 , S2 , S3 ,..., Sk be mutually exclusive and exhaustive events
with prior probabilities P(S1), P(S2),…,P(Sk). If an event A
occurs, the posterior probability of Si, given that A occurred is

We know:
P(F) =
P(M) =
P(H|F) =
P(H|M) =
Example
From a previous example, we know that 49% of the population are
female. Of the female patients, 8% are high risk for heart attack,
while 12% of the male patients are high risk. A single person is
selected at random and found to be high risk. What is the
probability that it is a male?
Define H: high risk F: female M: male
.12
.08
.51
.49

Example
Suppose a rare disease infects one out of every 1000
people in a population. And suppose that there is a good,
but not perfect, test for this disease: if a person has the
disease, the test comes back positive 99% of the time.
On the other hand, the test also produces some false
positives: 2% of uninfected people are also test
positive. And someone just tested positive.
What are his chances of having this disease?

We know:
P(A) = .001 P(Ac
) =.999
P(B|A) = .99 P(B|Ac
) =.02
Example
Define A: has the disease B: test positive
We want to know P(A|B)=?

Example
A survey of job satisfaction[2]
of teachers was taken,
giving the following results
If all the cells are divided by
the total number surveyed,
778, the resulting table is a
table of empirically derived
probabilities.

Example
For convenience, let C stand for the event that the teacher
teaches college, S stand for the teacher being satisfied and
so on. Let’s look at some probabilities and what they mean.
is the proportion of teachers who are college
teachers.
is the proportion of teachers who are
satisfied with their job.
is the proportion of teachers who are college
teachers and who are satisfied with their job.

Example
college teachers given they are satisfied.
Restated: This is the proportion of satisfied
that are college teachers.
satisfied given they are college teachers.
Restated: This is the proportion of college
teachers that are satisfied.

Example
P(C|S) ≠ P(C) so C and S are dependent events.
Are C and S independent events?

Example
P(C) = 0.150, P(S) = 0.545 and
P(C and S) = 0.095, so
P(C or S) = P(C)+P(S) - P(C and S)
= 0.150 + 0.545 - 0.095
= 0.600
P(C or S)?

L1-Fundamentals of Probability. gives fundamentals of statistics and probability.

More Related Content

Similar to L1-Fundamentals of Probability. gives fundamentals of statistics and probability. (20)

Recently uploaded (20)

L1-Fundamentals of Probability. gives fundamentals of statistics and probability.

Editor's Notes