1

Probability

Introduction to Probability, Conditional Probability

Introduction

2

• Probability is the study of randomness and uncertainty of any outcome.

• In the early days, probability was associated with games of chance (gambling).

3

Simple Games Involving Probability

Game: A fair die is rolled. If the result is 2, 3, or 4, you win $1; if it is 5, you win $2; but if it is 1 or 6, you lose $3.

Should you play this game?

Why do we need Probability?

• We have several graphical and numerical statistics for summarizing our data

• We want to make probability statements about the significance of our statistics

• Eg. In MSPH class, mean (height) = 5.4 feet • What is the chance that the true height of MSPH

students is between 5 feet and 6 feet ?

Deterministic vs. Random Processes• In deterministic processes, the outcome can be

predicted exactly in advance• Eg. Force = mass x acceleration. If we are given

values for mass and acceleration, we exactly know the value of force

• In random processes, the outcome is not known exactly, but we can still describe the probability distribution of possible outcomes • Eg. 10 coin tosses: we don’t know exactly how

many heads we will get, but we can calculate the probability of getting a certain number of heads

Random Experiment…• …a random experiment is an action or

process that leads to one of several possible outcomes/events. For example:

Experiment Outcomes

Flip a coin Heads, Tails

Exam Marks Numbers: 0, 1, 2, ..., 100

Assembly Time t > 0 seconds

Course Grades F, D, C, B, A, A+

Events• An event is an outcome or a set of outcomes of

a random process/experimentExample: Tossing a coin three times

Event A = getting exactly two heads = {HTH, HHT, THH}Example: Picking real number X between 1 and 20

Event A = chosen number is at most 8.23 = {X ≤ 8.23} Example: Tossing a fair dice

Event A = result is an even number = {2, 4, 6}

• Notation: P(A) = Probability of event A

Sample Space• The sample space S of a random process is

the set of all possible outcomes Example: one coin toss

S = {H,T} Example: three coin tosses

S = {HHH, HTH, HHT, TTT, HTT, THT, TTH, THH}Example: roll a six-sided dice

S = {1, 2, 3, 4, 5, 6}Example: Pick a real number X between 1 and 20

S = all real numbers between 1 and 20

Rules (Axioms) of ProbabilityRule 1: 0 ≤ P(A) ≤ 1 for any event A

Rule 2: The probability of the whole sample space is 1 P(S) = 1

Rule 3: P(Ac) = 1 - P(A)

Rule 4: If A and B are disjoint events then P(A or B) = P(A) + P(B)

Rule 5: If A and B are independent P(A and B) = P(A) x P(B)

11

Combinations of Events• The complement Ac of an event A is the event that A

does not occur• Probability Rule 3:

P(Ac) = 1 - P(A)• The union of two events A and B is the event that

either A or B or both occurs• The intersection of two events A and B is the event

that both A and B occur

Event A Complement of A Union of A and B Intersection of A and B

12

Disjoint Events• Two events are called disjoint if they can not

happen at the same time • Events A and B are disjoint means that the

intersection of A and B is zero • Example: coin is tossed twice

• S = {HH,TH,HT,TT}• Events A={HH} and B={TT} are disjoint • Events A={HH,HT} and B = {HH} are not disjoint

13

Independent events• Events A and B are independent if knowing that A

occurs does not affect the probability that B occurs

• Example: tossing two coinsEvent A = first coin is a head Event B = second coin is a head

• Disjoint events cannot be independent!• If A and B can not occur together (disjoint), then knowing that

A occurs does change probability that B occurs

• Probability Rule 5: If A and B are independent P(A and B) = P(A) x P(B)

Independent

multiplication rule for independent events

14

Equally Likely Outcomes Rule• If all possible outcomes from a random process

have the same probability, then• P(A) = (# of outcomes in A)/(# of outcomes in S) • Example: One Dice Tossed

P(even number) = |2,4,6| / |1,2,3,4,5,6|

The AND Rule of Probability• The probability of 2 independent events both happening

is the product of their individual probabilities.• Called the AND rule because “this event happens AND

that event happens”.• For example, what is the probability of rolling a 2 on one

die and a 2 on a second die? For each event, the probability is 1/6, so the probability of both happening is 1/6 x 1/6 = 1/36.

• Note that the events have to be independent: they can’t affect each other’s probability of occurring. An example of non-independence: you have a hat with a red ball and a green ball in it. The probability of drawing out the red ball is 1/2, same as the chance of drawing a green ball. However, once you draw the red ball out, the chance of getting another red ball is 0 and the chance of a green ball is 1.

The OR Rule of Probability

• The probability that either one of 2 different events will occur is the sum of their separate probabilities.

• For example, the chance of rolling either a 2 or a 3 on a die is 1/6 + 1/6 = 1/3.

NOT Rule

• The chance of an event not happening is 1 minus the chance of it happening.

• For example, the chance of not getting a 2 on a die is 1 - 1/6 = 5/6.

• This rule can be very useful. Sometimes complicated problems are greatly simplified by examining them backwards.

18

Conditional Probabilities• The notion of conditional probability can be

found in many different types of problems• Eg. imperfect diagnostic test for a disease

• What is probability that a person has the disease? Answer: 40/100 = 0.4

• What is the probability that a person has the disease given that they tested positive?More Complicated !

Disease + Disease - TotalTest + 30 10 40

Test - 10 50 60

Total 40 60 100

19

Definition: Conditional Probability

• Let A and B be two events in sample space

• The conditional probability that event B occurs given that event A has occurred is:

P(A|B) = P(A and B) / P(B)

• Eg. probability of disease given test positive

P(disease +| test +) = P(disease + and test +) / P(test +) = (30/100)/(40/100) =.75

20