Bayes‘ Theorem

Bayes‘ Theorem

• Situations 1. At beginning of baseball season, the fans of

last year’s season pennant winner thought their team had a good chance of winning again

However as season progressed because of team injuries the team began to lose even simple games

The team realized later that they must alter prior probability of winning

Bayes‘ Theorem

• Situations 2. A similar situation likely to occur in

business. A manager of a boutique finds that most of ski jacket of a famous brand finds that she thought would sell so well are hanging on the rack.

She must revise her prior probabilities and order something else

Bayes‘ Theorem

• In both the case above certain probabilities were altered after the people involved got additional information

• The new probabilities are known as revised or posterior probabilities

Bayes‘ Theorem

• The origin of obtaining posterior probabilities goes to Reverend Thomas Bayes and the basic formula for conditional probability under dependence is :-

P(BIA) =P(BA)

P(A)

Bayes‘ Theorem

• VALUE OF BAYES’ THEOREM Bayes’ Theorem offers a powerful statistical

method of evaluating new information and revising our prior estimates

If correctly used it makes it unnecessary to gather masses of data over long periods of time in order to make good decisions base on probabilities

Bayes‘ Theorem Example• We have equal numbers of two type of deformed

(biased) coins in a bowl• On one half the heads comes up 40% of times,

therefore P(heads) = 0.4 (Let us call this coin as type 1 coin)

• On the other half the heads comes up 70 % of times, therefore P(heads) = 0.7 ( Let us call this as type 2 coin)

Bayes‘ Theorem

Example contd• One coin is drawn and tossed once. It comes

up with an ‘head’. What is the probability that it is a type 1 coin ?

Bayes‘ Theorem Example contd• One coin is drawn and tossed once. It comes

up with an ‘head’. What is the probability that it is a type 1 coin ?

• We are bound to say 0.5 as the bowl contains half of type 1 coin and half of type 2 coin

• This is incorrect….

Bayes‘ Theorem

Example contd• We set up a table

Elementary Event

Probability of Elementary Event

P (Head I Elementary Event)

P(Head, Elementary event)

Type1Type 2

Bayes‘ Theorem

Example contd• We set up a table

Elementary Event

Probability of Elementary Event

P (Head I Elementary Event)

P(Head, Elementary event)

Type1 0.5 0.4 0.4x 0.5=0.20Type 2 0.5 0.7 0.7x 0.5=0.35

Total =1.0 P(Head)=0.55

Bayes‘ Theorem Example contd ( please see the table)• The sum of probabilities of the elementary event

(drawing either a type 1 or type 2 coin)is 1.0 because there are only two types of coins. The probability of each type is 0.5

• The sum of P(head I elementary event) column does not equal 1.0. The figures 0.4 and 0.7 simply represent the conditional probabilities of getting a head given type 1 and type 2 coin respectively

Bayes‘ Theorem Example contd ( please see the table)• The fourth column shows the joint probability of head and

type 1 coin occurring together (0.4 x 0.5 = 0.20) and the joint probability of head and type 2 coin occurring together (0.7 x 0.5 = 0.35)

• The sum of these joint probabilities is the marginal probability of getting a head

• We got joint probability by using the formula P(AB) = P(A I B) x P(B)

Bayes‘ Theorem Example contd ( please see the table)• To find the probability that the coin we have

drawn is type 2 P(B I A)= P(BA) / (P(A) P( type 2 I head) = P( type2, head)/ P (head) = 0.35/0.55 =0.636 Thus the probability that we have drawn a

type 2 coin is 0.636

Bayes‘ Theorem Example contd ( please see the table)• To find the probability that vthe coin we have drawn is

type 1 we use the formula for conditional probability under statistical dependence

P(B I A)= P(BA) / (P(A) P( type 1 I head) = P( type1, head)/ P (head) = 0.20/0.55 =0.364 Thus the probability that we have drawn a type 1

coin is 0.364

Bayes‘ Theorem Example contd ( please see the table)Analysis….What we have accomplished with one

additional piece of information made available to us?

What inference we have been able to draw from one toss of coin?

Bayes‘ Theorem Example contd ( please see the table)Analysis…Before we tossed the coin the best we could say was that there is 0.5 chance it is type 1 coin and 0.5 chance

it is a type2 coinHowever after tossing the coin we have been able to revise our

prior probability estimateOur new posterior estimate is that there is a higher probability

(0.636) that the coin we have in our hand is a type 2 than it is type 1 (0.364)

Bayes‘ Theorem• Tutorial Example Given the probabilities of three events A, B

and C occurring are P(A)=0.35, P(B)=0.45 and P(C)=0.2.

Assuming that A B and C has occurred , the probability of another event X occurring are P(X I A) = 0.8, P(X I B) = 0.65 and P(X I C)=0.3.

Find P(A I X), P(B I X) and P(C I X)

Bayes‘ Theorem• Tutorial Example Solution

Event

P(Event) P(X I Event) P(X and Event)

P(Event I X)

Bayes‘ Theorem• Tutorial Example Solution

Event

P(Event) P(X I Event)

P(X and Event)

P(Event I X)

A 0.35 0.80 0.28000 P(A I X) = 0.2800/0.6325 = 0.4427

B 0.45 0.65 0.2925 P(B I X) = 0.2925/0.6325 = 0.4625

C 0.20 0.30 0.0600 P(C I X) = 0.0600/0.6325 = 0.0949

P(X) = 0.6325

THANK YOU