1

Chapter 4

Basic Probability

David Chow

Sep 2014

2

Learning Objectives

In this chapter, you will learn:

Basic probability concepts,

Conditional probability, and

Bayes’ Theorem.

3

Definitions

Probability: the chance that an event will

occur, and 0 ≤ P ≤ 1

Event: Each possible type of outcome

Simple Event: an event that can be described

by a single characteristic

Sample Space: the collection of all possible

events

4

Three Probability Approaches

1. a priori classical probability:

based on prior knowledge of the

process involved.

2. empirical classical probability:

based on observed data.

3. subjective probability: based on

individual judgment (which may

come from past experience, personal

opinion, or analysis of a particular

situation).

Eg: A sailor assessing the probability

of raining is an example of ___.

5

Classical Probability

Calculating classical probability

1. a priori classical probability

2. empirical classical probability

outcomes possible ofnumber total

occurcan event the waysofnumber Occurrence ofy Probabilit

T

X

observed outcomes ofnumber total

observed outcomes favorable ofnumber Occurrence ofy Probabilit

6

Classical Probability

Two assumptions on classical probability

1. All outcomes are equally likely.

2. A procedure is repeated again and again, so

that the relative frequency probability (the

formulae on the previous slide) of an event

tends to approach the actual probability.

- This is called the Law of Large Numbers

- Probability is essentially a proportion

7

Eg: A Priori Classical Probability

Find the probability of selecting a face card (Jack, Queen,

or King) from a standard deck of 52 cards.

cards ofnumber total

cards face ofnumber Card Face ofy Probabilit

T

X

13

3

cards total52

cards face 12

T

X

8

Eg: Empirical Classical Probability

Taking Stats Not Taking Stats Total

Male 84 145 229

Female 76 134 210

Total 160 279 439

Find the probability of selecting a male taking statistics

from the population described in the following table:

191.0439

84

people ofnumber total

stats takingmales ofnumber Stats Taking Male ofy Probabilit

9

Eg: Sample Space

The Sample Space is the collection of all possible

events

Eg1: All 6 faces of a die:

Eg2: All possible outcomes when having a child: Boy or Girl

10

Events in Sample Space

Simple event

An outcome with one characteristic

Eg: A diamond card from a deck of cards

Complement of an event A (denoted A/)

All outcomes that are not part of event A

Eg: All cards that are not diamonds

Joint event

Involves two or more characteristics simultaneously

Eg: An ace that is also red from a deck of cards

11

Visualizing Events in

Sample Space

Contingency Tables:

Tree Diagrams:

Ace Not Ace Total

Black 2 24 26

Red 2 24 26

Total 4 48 52

Full Deck

of 52 Cards Sample

Space

2

24

2

24

12

Simple vs. Joint Probability

Simple (Marginal) Probability refers to the

probability of a simple event.

Eg: P(King)

Joint Probability refers to the probability of

an occurrence of two or more events.

Eg: P(King and Spade)

13

Mutually Exclusive Events

Mutually exclusive events are events that cannot occur

together (simultaneously).

Eg1: Drawing a card

A = queen of diamonds; B = queen of clubs

Events A and B are mutually exclusive if one card is selected

Eg2: New-born baby

B = having a boy; G = having a girl

Events B and G are mutually exclusive if one child is born

14

Collectively Exhaustive Events

Collectively exhaustive events

The set of events covers the entire sample space

Hence, one of the events must occur

Eg: A deck of cards

A = aces; B = black cards; C = diamonds; D = hearts

Events A, B, C and D are collectively exhaustive (but not mutually exclusive – a selected ace may also be a heart)

Events B, C and D are collectively exhaustive and also mutually exclusive

15

Computing Joint and

Marginal Probabilities

The probability of a joint event, A and B:

Computing a marginal (or simple) probability:

Where B1, B2, …, Bk are k mutually exclusive and collectively exhaustive events

outcomeselementaryofnumbertotal

BandAsatisfyingoutcomesofnumber)BandA(P

)BandP(A)BandP(A)BandP(AP(A) k21

Eg: B = Colleges

16

Joint Probability Using a

Contingency Table

P(A1 and B2) P(A1)

Total Event

P(A2 and B1)

P(A1 and B1)

Event

Total 1

Joint Probabilities Marginal (Simple) Probabilities

A1

A2

B1 B2

P(B1) P(B2)

P(A2 and B2) P(A2)

17

Eg: Joint Probability

P (Red and Ace)

52

2

cards of number total

ace and red are that cards of number

Ace Not

Ace

Total

Black 2 24 26

Red 2 24 26

Total 4 48 52

18

Eg: Marginal (Simple) Probability

P(Ace)

52

4

52

2

52

2)BlackandAce(P)dReandAce(P

Ace Not Ace Total

Black 2 24 26

Red 2 24 26

Total 4 48 52

19

Probability Summary So Far

What does “P (head) = ½” mean?

There must be a head out of the next 2 tosses.

The result must be head or no head, so P (head) must be either 1 or 0.

True or False?

Probability measures the likelihood that an event will occur.

0 ≤ P(A) ≤ 1 for any event A.

The sum of the probabilities of all mutually exclusive and collectively exhaustive events is 1.

P(A) + P(B) + P(C) = 1

A, B, and C are mutually exclusive and collectively exhaustive

Marginal probability = sum of joint probabilities if ____

Certain

Impossible

0.5

1

0

20

General Addition Rule

P(A or B) = P(A) + P(B) - P(A and B)

General Addition Rule:

If A and B are mutually exclusive, then

P(A and B) = 0, so the rule can be simplified:

P(A or B) = P(A) + P(B)

for mutually exclusive events A and B

21

Eg: General Addition Rule

Taking Stats Not Taking Stats Total

Male 84 145 229

Female 76 134 210

Total 160 279 439

Find the probability of selecting a male or a statistics student

from the population described in the following table:

P(Male or Stat) = P(M) + P(S) – P(M or S)

= 229/439 + 160/439 – 84/439 = 305/439

22

Eg: Titanic Mortality

1. P (a man or a boy) =

2. P (a man or a survivor) =

Men Women Boys Girls Total

Survived 332 318 29 27 706

Died 1360 104 35 18 1517

Total 1692 422 64 45 2223

23

Conditional Probability

24

Case Study: Genius & Maniac

Aristotle: There is no great genius without a mixture of madness

Scientists at King’s College London found that

Top graders at school were four times more likely to develop bipolar disorder (manic depression) than average students

The link was strongest among those who studied music or literature

25

Conditional Probability

A conditional probability is the probability of one event, given that another event has occurred:

P(B)

B)andP(AB)|P(A

P(A)

B)andP(AA)|P(B

where P(A and B) = joint probability of A and B,

P(A) = marginal probability of A, and P(B) = marginal probability of B

The Venn diagram provides an intuitive explanation.

The conditional

probability of A given

that B has occurred

The conditional

probability of B given

that A has occurred

26

Eg: Conditional Probability

Of the cars on a used car lot, 70% have air conditioning (AC) and 40% have a CD player (CD). 20% of the cars have both.

What is the probability that a car has a CD player, given that it has AC ?

I.e., find P(CD | AC).

27

Eg: Conditional Probability

CD No CD Total

AC 0.2 0.5 0.7

No

AC

0.2 0.1 0.3

Total 0.4 0.6 1.0

.2857.7

.2

P(AC)

AC)andP(CDAC)|P(CD

Given AC, we only consider the top row (70% of the cars). Of these, 20% have a CD player. 20% of 70% is about 28.57%.

28

Eg: Conditional Probability in

Decision Trees

P(CD and AC) = .2

P(CD and AC/) = .2

P(CD/ and AC/) = .1

P(CD/ and AC) = .5

4.

2.

6.

5.

6.

1.

All

Cars

4.

2.

Given CD or

no CD:

29

Eg: Conditional Probability in

Decision Trees

P(AC and CD) = .2

P(AC and CD/) = .5

P(AC/ and CD/) = .1

P(AC/ and CD) = .2

7.

5.

3.

2.

3.

1.

All

Cars

7.

2.

Given AC or

no AC:

30

Eg: Card Counting

What is the probability of

getting a high-valued card?

David Ho (何大一)

David Ho is a famous researcher of AIDS for the “AIDS cocktail” treatment

Movie: 21

Inspired by the true story of the MIT Blackjack Team, which is split into two groups:

1. "Spotters" play the minimum bet and do the counting. They send secret signals to the "big players“

2. “Big players“ place large bets when the count is favorable

31

Statistical Independence

Two events are independent if and only if:

Events A and B are independent when the

probability of one event is not affected by the

other event.

P(A)B)|P(A

32

Eg: Titanic Mortality Again

Probability of getting a women or child if a survivor

is randomly selected

P(man | died) =

Are the events “man” and “died” statistically

independent?

Men Women Boys Girls Total

Survived 332 318 29 27 706

Died 1360 104 35 18 1517

Total 1692 422 64 45 2223

33

Eg1: Statistical Independence

Given the following contingency table:

What is the probability of

(a) A|B ?

(b) A’|B’?

(c) A|B’ ?

(d) Are events A and B statistically independent?

B B’

A 10 30

A’ 25 35

34

Multiplication Rule

Multiplication rule for two events A and B:

If A and B are independent, then

and the multiplication rule simplifies to:

P(B)B)|P(AB)andP(A

P(B)P(A)B)andP(A

P(A)B)|P(A

35

Multiplication Rule

P (A and B) is the product of the probability

of event A and the probability of event B.

But the probability of B has to take into

account the previous occurrence of event A.

Hence the conditional probability P(B|A)

appears in the multiplication rule.

36

Multiplication Rule

Eg: Find P (2 and 2) if a die is rolled twice

Eg: Find P (2 Aces) if two cards are drawn

from a deck

with replacement or

without replacement

37

Multiplication Rule

Statistical independence simplifies the multiplication

rule, but sampling without replacement implies

dependence.

General rule: Assume stat. independence when

sample size (n) is within 5% of the population (N),

i.e., n =< 0.05 (N)

Eg: A sample of 12 camera are drawn from a population

of 1,000.

Given a 5% defect rate, find P (all 12 cameras are good).

38

Eg: Multiplication Rule

Suppose a city council is composed of 5

democrats, 4 republicans, and 3 independents.

Find the probability of randomly selecting a

democrat followed by an independent.

Note that after the democrat is selected (out of 12

people), there are only 11 people left in the

sample space.

.1145/442)(3/11)(5/1 P(D)D)|P(ID)and P(I

39

Marginal Probability Using

Multiplication Rules

)P(B)B|P(A)P(B)B|P(A)P(B)B|P(A P(A) kk2211

Marginal probability for event A:

Where B1, B2, …, Bk are k mutually exclusive and collectively exhaustive events

40

Bayes’ Theorem

Bayes’ Theorem was developed by Thomas

Bayes in the 18th Century.

It is used to revise previously calculated

probabilities based on new information.

It is an extension of conditional probability.

41

Bayes’ Theorem

))P(BB|P(A))P(BB|P(A))P(BB|P(A

))P(BB|P(AA)|P(B

kk2211

iii

where:

Bi = ith event of k mutually exclusive

and collectively exhaustive events

A = new event that might impact P(Bi)

42

Eg: Bayes’ Theorem

A drilling company has estimated a 40% chance of striking

oil for their new well. A detailed test has been scheduled for

more information.

Historically,

60% of successful wells have had detailed tests, and

20% of unsuccessful wells have had detailed tests.

Given that this well has been scheduled for a detailed test,

what is the probability that the well will be successful?

43

Eg: Bayes’ Theorem

Let S = successful well, U = unsuccessful well

P(S) = .4 , P(U) = .6 (prior probabilities)

Define the detailed test event as D

Conditional probabilities:

P(D|S) = .6 P(D|U) = .2

Find P(S|D)

44

Eg: Bayes’ Theorem

667.12.24.

24.

)6)(.2(.)4)(.6(.

)4)(.6(.

U)P(U)|P(DS)P(S)|P(D

S)P(S)|P(DD)|P(S

Apply Bayes’ Theorem:

So, the revised probability of success, given that this well has been scheduled for a detailed test, is .667

45

Eg: Bayes’ Theorem

Given the detailed test, the revised probability of a

successful well has risen to .667 from the original estimate

of 0.4.

Event Prior

Prob.

Conditional

Prob.

Joint

Prob.

Revised

Prob.

S (successful) .4 .6 .4*.6 = .24 .24/.36 = .667

U (unsuccessful) .6 .2 .6*.2 = .12 .12/.36 = .333

46

Review

47

Eg2: Addition Rule

P (planned to purchase) =

P (planned or actually purchased) =

Use the contingency table in this example to illustrate the addition rule.

Actually purchased

Planned to purchase Yes No Total

Yes 200 50 250

No 100 650 750

Total 300 700 1000

48

Eg3: Statistical Independence

A sample of 600 respondents was selected in Beijing to study

consumer behavior, with the following results:

Enjoys Shopping Gender

For Clothing Male Female Total

Yes 163 269 432

No 125 43 168

Total 288 312 600

49

Eg3: Continued

(a) Suppose the respondent chosen is a female. What is the probability that she does not enjoy shopping for clothing?

(b) Suppose the respondent chosen enjoys shopping for clothing. What is the probability that the individual is a male?

(c) Are enjoying shopping for clothing and the gender of the individual statistically independent? Explain.

50

Eg4: Bayes’ Theorem

1. If P(B) = 0.05, P(A | B) = 0.80, P(B’) = 0.95, and P(A | B’) = 0.40, find P(B | A).

2. An advertising executive is studying television viewing habits of married men and women during prime-time hours.

Based on past viewing records, the executive has determined that during prime time, husbands are watching television 60% of the time. When the husband is watching television, 40% of the time the wife is also watching. When the husband is not watching television, 30% of the time the wife is watching television.

Find the probability that

(a) If the wife is watching television, the husband is also watching television.

(b) The wife is watching television in prime time.

51

Video

52

賭仔自嘆 鄭君綿 1969

拎LUM六 長衫六

高腳七 仲有一隻大頭六

二三更 瓜老襯 輸到我木

日夜賭場黎侍候 生意唔撈我兩頭游

我o既錢輸晒 真係冇修

食梗青 頂肚癮 搵菜頭

畫則又怕彈番轉頭

問人地借 佢又拎下拎下頭

籌碼部佢又詐嬲 LEAM出碼頭把盤收

爛手錶 都當晒 HAM BAN LAN冇謀

祖先不開眼 令我好擔憂

我從前知道係咁醜 我都唔使MOU街頭

http://hk.youtube.com/watch?v=eN68W2fWwR0&feature=related

53

內地歹徒拍片推銷六合彩騙局

蘋果日報 2011-05-08

內地歹徒拍片推銷六合彩騙局 遭冒名欺詐馮程淑儀無奈

http://www.youtube.com/watch?v=Np4adVu-om0

六合彩在內地的地下賭風猖獗，不少人迷信六合彩貼士。

不法之徒假扮香港高官及賽馬會高層，拍攝宣傳片聲稱有方法可預測本港六合彩結果，在網上散播，引誘不知情內地人，入會購買所謂的「中獎號碼」。

馬會已報警，並要求有關網站刪除片段。警方表示正跟進事件。