Conditional Probability and Bayes - elanding.xyz · Outline 1 Bayes’ Theorem 2 Bayes’ Theorem...

Conditional Probability and BayesChapter 2 – Lecture 7

Yiren Ding

Shanghai Qibao Dwight High School

March 15, 2016

Yiren Ding Conditional Probability and Bayes 1 / 20

Outline

1 Bayes’ Theorem

2 Bayes’ Theorem in Odds FormApplication in Judicial DecisionsProbability, misleadingSearch and RescueTea House ScamGracia Murder Mystery

3 The O.J. Simpson Murder CaseBayesian Analysis

4 The Lost DiamondIntuitive Defense


Bayes’ Theorem

Introduction

Suppose in a court case, the defendant is either guilty (H, “hypothesisis true”) or not guilty (H). Before any evidence is presented, we canassign the prior probabilities P(H) and P(H) = 1− P(H).

The essential question we must ask is: How do the prior probabilitieschange once evidence (event E ) is revealed?

For instance, the event E could be the evidence that the accused’sfingerprint is found on the crime scene, or the accused has the sameblood type as the perpetrator’s.

The updated value of the probability is called the posteriorprobability, denoted by P(H |E ), and can be expressed via thestandard form of the Bayes’ Theorem:

P(H |E ) =P(HE )

P(E )=

P(E |H)P(H)

P(E |H)P(H) + P(E |H)P(H)


Bayes’ Theorem in Odds Form

Theorem 1 (Bayes’ Theorem in Odds Form).

The posterior probability P(H |E ) satisfies

P(H |E )

P(H |E )=

P(H)

P(H)

P(E |H)

P(E |H).

In words, Bayes’ theorem in odds form states that

posterior odds = prior odds × likelihood ratio .

The theorem easily follows by applying the definition of conditionalprobability to P(H |E ) and P(H |E ), but its implication is profound.

The factor P(H)

P(H)gives the prior odds in favor of the hypothesis H

before the evidence has been presented.

The factor P(E |H)

P(E |H)gives the likelihood ratio or the Bayes factor,

which represents the impact the evidence will have on the hypothesis.




Do not just memorize this theorem, but understand it first:

P(H |E )

P(H |E )=

P(H)

P(H)

P(E |H)

P(E |H).

The Bayes’ factor is large when P(E |H) is large compared toP(E |H). This means that it is very likely that the evidence will beobserved given that the hypothesis is true.

Therefore, with any two pieces of evidence E1 and E2, this theoremcan be used iteratively,

P(H |E1E2)

P(H |E1E2)=

P(H)

P(H)

P(E1 |H)

P(E1 |H)

P(E2 |HE1)

P(E2 |HE1).

Here lies the real power of Bayes’ theorem. You should be cryingright now at the beauty of probability theory!


Bayes’ Theorem in Odds Form Application in Judicial Decisions

Applying Bayes’ Theorem in Court

In judicial decision making, prior probability P(H) represents the personalopinion of the court before the evidence is presented.

The Bayes’ factor is often determined by an expert, who sometimesmistaken P(H |E ) with P(E |H)!

A classic example is the famous court case of People vs. Collins in LosAngeles in 1964. In this case, a couple matching the description of a couplethat had committed an armed robbery was arrested.

Based on “expert” testimony, the district attorney claimed that thefrequency of couples matching the description was roughly 1 in 12 million.

Although this was the estimate for P(E |H), the district attorney treatedthis as if it was P(H |E ) and incorrectly concluded that the couple wasguilty beyond reasonable doubt!

A low P(E |H) does not necessarily imply a high Bayes factor. The accusedis most likely guilty only if P(E |H) is significantly larger than P(E |H).


Bayes’ Theorem in Odds Form Probability, misleading

Probability can be misleading

Statement 1

Only 10% of traffic accidents are caused by drunk drivers. This meansthat 90% of traffic accidents are caused by sober drivers. Therefore, weshould only allow drunk drivers on the street.



Probability can be misleading!

Statement 2

For the past 10 years, only .1% of drunk people in Shanghai ended upkilling themselves while driving. Therefore, it is okay to get drunk whiledriving and still have a 99.9% probability of living.



English can trick you!

Be very careful when people say the following sentences:

1 Only ...% of X lead to Y .2 Only ...% of X end up as Y .3 Only ...% of X become Y .4 Only ...% of X are caused by Y .5 Only ...% of X are affected by Y .

...

These all refer to the conditional probability P(Y |X )!

Often people will try to trick you by subtly distorting the samplespace X in order to skew the probability, so you must be very carefulwhen interpreting these remarks.

Remember that a small % of X leading to Y does not necessarilymean that also a small % of Y leading to X !


Bayes’ Theorem in Odds Form Search and Rescue

Example 1 (Search and Rescue).

It is believed that a sought-after wreck will be in a certain sea area withprobability p = 0.4. A search in that area will detect the wreck withprobability d = 0.9 if it is there. What is the revised probability of thewreck being in the area when the area is searched and no wreck is found?

Here our hypothesis H is that the wreck is in the area and theevidence E is that the wreck has not been detected in that area.

The prior odds is P(H) : P(H) = 0.4 : 0.6 = 2 : 3.

The likelihood ratio is P(E |H) : P(E |H) = 0.1 : 1 = 1 : 10.

The posterior odds is P(H |E ) : P(H |E ) = 23 ×

110 = 1 : 15.

Therefore, the updated probability is

P(H) =1

1 + 15=

1

16.


Bayes’ Theorem in Odds Form Tea House Scam

Example 2 (Tea House Scam).

The Seven Virtues Tea House invites students to take part in the followingbetting game. Three cards are placed into a hat. One card is red on both sides,one is black on both sides, and one is red on one side and black on the other side.A student is asked to pick a card out of the hat at random, and reveal only oneside of the card. The owner of the tea house bets the student equal odds that theother side of the card will be the same color as the one shown. (If the other sideis the same color, I win $1, otherwise you win 1$.) Do you want to play?

Without loss of generality, suppose the chosen card reveals red on the visibleside. Let H be the hypothesis that both sides of the chosen card are red andlet E be the evidence that the visible side of the chosen card is red.

The prior odds is 1 : 2 since there is only one card with red on both sides.

The likelihood ratio is 1 : 14 = 4 : 1. (Seeing red is very likely!)

The posterior odds is therefore 12 ×

41 = 2 : 1, which means

P(H |E ) =2

2 + 1=

2

3.


Bayes’ Theorem in Odds Form Gracia Murder Mystery

Example 3 (Gracia Murder Mystery).

Gracia was murdered last night. Xena and Yang are the prime suspects.Both persons are on the run, and after an initial investigation, bothfugitives appear equally likely to be the murderer. Further investigationreveals that the actual perpetrator has blood type A. Ten percent of thepopulation belongs to the group having this blood type. Additional inquiryreveals that Xena has blood type A, but offers no information concerningYang’s blood type. In light of this new information, what is the probabilitythat Xena is the one who murdered Gracia?


Bayes’ Theorem in Odds Form Gracia Murder Mystery

Example 3 solution

Let H denote the event that Xena is the murderer. Let E representthe new evidence that Xena has blood type A.

The prior odds is 1 : 1, and the likelihood ratio is 1 : 110 = 10 : 1.

Hence the posterior odds is 11 ×

101 = 10 : 1, and the posterior

probability that Xena is the murderer is

P(H |E ) =10

1 + 10=

10

11.

The probability that Yang is the perpetrator is 1− 1011 = 1

11 and not,as many would think, 1

10 ×12 = 1

20 .

This is because that Yang is not a randomly chosen person becausehe has a 50% probability of being the perpetrator.

Bayesian analysis can sharpen our intuition!


The O.J. Simpson Murder Case

Example 4 (The O.J. Simpson Murder Case).

Nicole Brown was murdered at her home in Los Angeles on the night of June 12,1994. The prime suspect was her husband O. J. Simpson, at the time awell-known celebrity famous both as a TV actor as well as a retired professionalfootball star. This murder led to one of the most heavily publicized murder trialsin the United States during the last century. The fact that the murder suspecthad previously physically abused his wife played an important role in the trial.The famous defense lawyer Alan Dershowitz, a member of the team of lawyersdefending the accused, tried to belittle the relevance of this fact by stating thatonly 0.1% of the men who physically abuse their wives actually end up murderingthem. Was the fact that O. J. Simpson had previously physically abused his wifeirrelevant to the case?


The O.J. Simpson Murder Case Bayesian Analysis


The answer is no. To explain that, we define

E = the event that husband has physically abused his wife

M = the event that the wife has been murdered

G = the event that the husband is guilty of the murder of his wife

The crucial fact in this case is that Nicole Brown was already murdered!

The question, therefore, is not how likely does abuse lead to murder, i.e.,P(M |E ), but the probability that, given the wife is murdered, thehusband is guilty if he had previously abused his wife: P(G |EM).

Since the “size” of the event E is so much larger than the size of the eventEM, it is no surprise that the conditional probability P(M |E ) will besignificantly smaller than P(G |EM).

However, Alan Dershowitz’s argument that only 0.1% of physical abuse leadto murder was indeed falsely convincing!




To find the probability that Simpson is guilty we use the Bayes’ formula

P(G |EM)

P(G |EM)=

P(G |M)

P(G |M)

P(E |GM)

P(E |GM),

According to crime statistics, in 1992, 4, 936 women were murdered in theUnited States, of which roughly 1, 430 were murdered by their (ex)husbandsor boyfriends. This results gives an estimate of 1,430

4,936 = 0.29 for the prior

probability P(G |M), and an estimated probability 0.71 for P(G |M).

Furthermore, it is also known that roughly 5% of married women in theUnited States have at some point been physically abused by their husbands.If we assume that a woman who has been murdered by someone other thanher husband is a randomly selected woman, then P(E |GM) = 0.05.




The value of P(E |GM) will be estimated based on the reported remarksmade by Simpson’s famous defense attorney, Alan Dershowitz

Dershowitz admitted in a newspaper article, that a substantial percentage ofthe husbands who murder their wives have, previous to the murder, alsophysically abused their wives.

Given this statement, it is reasonable to assume that P(E |GM) = 0.5.

By substituting the various values we find the posterior odds to be

P(G |EM)

P(G |EM)=

0.29

0.71× 0.5

0.05= 4.08.

This means that P(G |EM) = 0.81. In other words, there is an estimatedprobability of 81% that the husband is the murderer of his wife in light ofthe knowledge that he had previously physically abused her.

Therefore this evidence is certainly very relevant to the case!



Visualizing the O.J. Simpson Murder


The Lost Diamond

Example 5 (The Lost Diamond).

A diamond merchant has lost a case containing a very expensive diamondsomewhere in a large city in an isolated area. The case has been foundagain but the diamond has vanished. However, the empty case containsDNA of the person who took the diamond. The city has 150,000inhabitants, and each is considered a suspect in the diamond theft. Anexpert declares that the probability of a randomly chosen person matchingthe DNA profile is 10−6. The police search a database with 5,120 DNAprofiles and find one person matching the DNA from the case. Apart fromthe DNA evidence, there is no additional background evidence related tothe suspect. On the basis of the extreme infrequency of the DNA profileand the fact that the population of potential perpetrators is only 150,000people, the prosecutor jumps to the conclusion that the odds of thesuspect not being the thief are practically nil and calls for a toughsentence. What do you think of this conclusion?


The Lost Diamond Intuitive Defense

Intuitive Defense

This is a textbook example of the faulty use of probabilities.

The real issue here is that the probability that the suspect is innocentof the crime given a DNA match is quite different from theprobability of a random DNA match alone!

What we should really look for is the probability that among allpersons matching the DNA, the arrested person is the perpetrator.

Therefore, the counsel of defense could reason as follows:

“Among the other 150, 000− 5, 120 = 144, 880individuals, the expected number of people matching theDNA profile is 144, 880× 10−6 = 0.14488. So theprobability that the suspect is guilty is 1/(1 + 0.14488)= 0.8735. It is not beyond reasonable doubt that thesuspect is guilty and thus the suspect must be released!”


Date post:	05-Jun-2019
Category:	Documents
Upload:	lethu
View:	228 times
Download:	1 times

Conditional Probability and Bayes - elanding.xyz · Outline 1 Bayes’ Theorem 2 Bayes’ Theorem...

Documents