Chapter 3 Conditional Probability and Independence

Part 1: TheoryVer. 110615Chapter 3 Conditional Probability and Independencethe probability that E occurs given that F occurs !

Definition

2Probability/Ch3

3Probability/Ch3QuizA box contains 3 cards. One is black on both sides. One is red on both sides. One is black on one side and red on the other.You draw a random card and see a black side. What are the chances the other side is red?A: 1/4B: 1/3C: 1/24Probability/Ch3SolutionF1F2F3B1B2B3S = { F1, B1, F2, B2, F3, B3 }The event you see a black side is SB = { F1, B1, F3 }The event the other side is red is OR = { F2, B2, F3 }P(OR | SB) =equally likely outcomesP(OR SB)P(SB)=1/|S|3/|S|= 1/35Probability/Ch36Probability/Ch3

Probability/Ch37

Probability/Ch38

Probability/Ch39

Probability/Ch310

Probability/Ch311

Law of total probabilityProbability/Ch312Bayess Rule

Independent Event

Probability/Ch313

Probability/Ch314

Problem of the point

1623 1662 Probability/Ch315

1601 1665 Probability/Ch316

Color a complete graphProbability/Ch317

Probability/Ch318

Then what ?Probability/Ch319

Probability/Ch320

Probability/Ch321Catalan numberBertrands ballot problemSuppose in an election candidate A receives n votes and B receives m where n>m. What is the probability that A always wins B during the counting process ?If we walk in a lattice and in each step we can choose going right or going up one unit. How many paths are there from (0,0) to (n,n), without crossing the diagonal line ?

(0,0)(n,n)

3. Conditional probabilitypart twoCause and effect123effect:Bcause:C1C2C323Probability/Ch3

Bayes ruleP(E|C) P(C)P(E)P(E|C) P(C)P(E|C) P(C) + P(E|Cc) P(Cc)=More generally, if C1,, Cn partition S then P(C|E) =P(Ci|E) =P(E|Ci) P(Ci)P(E|C1) P(C1) + + P(E|Cn) P(Cn)24Probability/Ch3Cause and effect123cause:C1C2C3effect:BP(Ci|B) =P(B|Ci) P(Ci)P(B|C1) P(C1) + P(B|C2) P(C2) + P(B|C3) P(C3)25Probability/Ch3Medical testsIf you are sick (S), a blood test comes out positive (P) 95% of the time.If you are not sick, the test is positive 1% of the time. Suppose 0.5% people in Hong Kong are sick.You take the test and come out positive. What are the chances that you are sick?P(P|S) P(S)P(P|S) P(S) + P(P|Sc) P(Sc)P(S|P) =95% 0.5%1% 99.5% 32.3%26Probability/Ch3Computer science vs. nursingTwo classes take place in Lady Shaw Building.One is a computer science class with 100 students, out of which 20% are girls.The other is a nursing class with 10 students, out of which 80% are girls. A girl walks out. What are the chances that she is from the computer science class? 27Probability/Ch3Two effectsCup one has 9 blue balls and 1 red ball.Cup two has 9 red balls and 1 blue ball.I choose a cup at random and draw a ball. It is blue.I draw another ball from the same cup (without replacement). What is the probability it is blue?28Probability/Ch3Russian roulette

AliceBob

BANGAlice and Bob take turns spinning the 6 hole cylinder and shooting at each other.What is the probability that Alice wins (Bob dies)?29Probability/Ch3Russian rouletteS = { H, MH, MMH, MMMH, MMMH, }E.g. MMH: Alice misses, then Bob misses, then Alice hitsA = Alice wins = { H, MMH, MMMMH, }Probability modeloutcomes are not equally likely!30Probability/Ch3Russian rouletteP(A)outcomeH MH MMH MMMH MMMHprobability1/6 5/6 1/6 (5/6)2 1/6 (5/6)3 1/6 (5/6)4 1/6 = 1/6 + (5/6)2 1/6 + (5/6)4 1/6 + = 1/6 (1 + (5/6)2 + (5/6)4 + )= 1/6 1/(1 (5/6)2)= 6/1131Probability/Ch3Russian rouletteSolution using conditional probabilities:P(A) = P(A|W1) P(W1) + P(A|W1c) P(W1c)A = Alice wins = { H, MMH, MMMMH, }W1 = Alice wins in first round = { H }Ac = Bob wins = { MH, MMMH, MMMMMH, }5/6 1/6 1P(Ac)P(A) = 1 1/6 + (1 P(A)) 5/611/6 P(A) = 1so P(A) = 6/1132Probability/Ch3Infinite sample spacesAxioms of probability:SE1. for every E, 0 P(E) 1S2. P(S) = 1SEF3. If EF = thenP(EF) = P(E) + P(F)3. If E1, E2, are pairwise disjoint:P(E1E2) = P(E1) + P(E2) + 33Probability/Ch3Problem for you to solveCharlie tosses a pair of dice. Alice wins if the sum is 7. Bob wins if the sum is 8.Charlie keeps tossing until one of them wins.What is the probability that Alice wins?34Probability/Ch3Hats againThe hats of n men are exchanged at random. What is the probability that no man gets back his hat?Solution using conditional probabilities:N = No man gets back his hatH1 = 1 gets back his own hatP(N) = P(N|H1) P(H1) + P(N|H1c) P(H1c)(n-1)/n 1/n 035Probability/Ch3Hats againN = No man gets back his hatH1 = 1 gets back his own hatS1 = 1 exchanges hats with another manP(N) = P(N|H1c)n 1nP(N|H1c) = P(NS1|H1c) + P(NS1c|H1c) = P(S1|H1c) P(N|S1H1c) + P(NS1c|H1c) 1/(n-1) pn-2let pn = P(N)pn-1pn = pn = pn-1 + pn-2 n 1n1n36Probability/Ch3Hats againN = No man gets back his hatpn = P(N)pn = pn-1 + pn-2 n 1n1np1 = 0pn pn-1 = (pn-1 pn-2)1np3 p2 = (p2 p1)1313!= p3 = 13!12p4 p3 = (p3 p2)1414!= p4 = + 13!1214!p2 =1237Probability/Ch3Summary of conditional probabilityConditional probabilities are used:to estimate the probability of a cause when we observe an effectConditioning on the right event can simplify the description of the sample spaceWhen there are causes and effects1To calculate ordinary probabilities238Probability/Ch3Independence of two events

Let E1 be first coin comes up H E2 be second coin comes up H Then P(E2 | E1) = P(E2)Events A and B are independent ifP(A B) = P(A) P(B) P(E2E1) = P(E2)P(E1)39Probability/Ch3Examples of (in)dependence

Let E1 be first die is a 4 S6 be sum of dice is a 6 S7 be sum of dice is a 7 P(E1) =P(S6) =P(E1S6) =E1, S6 are dependentP(S7) =P(E1S7) =E1, S7 are independentP(S6S7) =S6, S7 are dependent1/65/361/361/61/36040Probability/Ch30

40Algebra of independent eventsIf A and B are independent, then A and Bc are also independent.Proof: Assume A and B are independent.P(ABc)= P(A) P(AB) = P(A) P(A)P(B)= P(A)P(Bc) so Bc and A are independent.Taking complements preserves independence.41Probability/Ch3Independence of three eventsEvents A, B, and C are independent ifP(AB) = P(A) P(B) P(BC) = P(B) P(C) P(AC) = P(B) P(C) and P(ABC) = P(A) P(B) P(C). This is important!42Probability/Ch3(In)dependence of three events

Let E1 be first die is a 4 E2 be second die is a 3 S7 be sum of dice is a 7 P(E1E2) = P(E1) P(E2) P(E1S7) = P(E1) P(S7) P(E2S7) = P(E2) P(S7) P(E1E2S7) = P(E1) P(E2) P(S7) 1/61/61/61/36E1E2S71/61/61/61/3643Probability/Ch3Independence of many eventsEvents A1, A2, are independent if for every subset Ai1, , Air of the eventsP(Ai1Air) = P(Ai1) P(Air) Independence is preserved if we replace some event(s) by their complements, intersections, unionsAlgebra of independent events44Probability/Ch3

The proof can be done by induction.

45Probability/Ch3

PlayoffsAlice wins 60% of her ping pong matches against Bob. They meet for a 3 match playoff. What are the chances that Alice will win the playoff?Probability modelLet Wi be the event Alice wins match iWe assume P(W1) = P(W2) = P(W3) = 0.6We also assume W1, W2, W3 are independent46Probability/Ch3PlayoffsProbability modelTo convince ourselves this is a probability model, lets redo it as in Lecture 2:S = { AAA, AAB, ABA, ABB, BAA, BAB, BBA, BBB } the probability of AAA is P(W1W2W3) = 0.63AABP(W1W2W3c) = 0.62 0.4ABAP(W1W2cW3) = 0.62 0.4BBBP(W1cW2cW3c) = 0.43The probabilities add up to one.47Probability/Ch3PlayoffsA = { AAA, AAB, ABA, BAA } For Alice to win the tournament, she must win at least 2 out of 3 games. The corresponding event is0.630.62 0.4 eachP(A) = 0.63 + 3 0.62 0.4 = 0.648. Alice wins a p fraction of her ping pong games against Bob. What are the chances Alice beats Bob in an n match tournament (n is odd)?General playoff48Probability/Ch3PlayoffsSolutionProbability model similar as before.Let A be the event Alice wins playoff Ak be the event Alice wins exactly k matches P(Ak) = C(n, k) pk (1 p)n - kA = A(n+1)/2An P(A) = P(A(n+1)/2) + + P(An) (they are disjoint) number of arrangements of k As, n k Bsprobability of each such arrangement49Probability/Ch3PlayoffsP(A) = k = (n+1)/2 nC(n, k) pk (1 p)n - k

p = 0.6p = 0.7The probability that Alice wins an n game tournamentnn50Probability/Ch3Problem for youThe Lakers and the Celtics meet for a 7-game playoff. They play until one team wins four games.Suppose the Lakers win 60% of the time. What is the probability that all 7 games are played?

51Probability/Ch3Gamblers ruinYou have $100. You keep betting $1 on red at roulette.

You stop when you win $200, or when you run out of money.What is the probability you win $200?52Probability/Ch3Gamblers ruinProbability modelS = all infinite sequences of Reds and OthersLet Ri be the event of red in the ith round (there is an R in position i)Probabilities:P(R1) = P(R2) = = 18/37R1, R2, are independentcall this p53Probability/Ch3Gamblers ruinLet W be the event you win $200 and wn = P(W).You have $100. You stop when you win $200.$n= P(W|R1) P(R1) + P(W|R1c) P(R1c)wn = P(W)1-p pwn+1wn-1wn = (1-p)wn-1 + pwn+1w0 = 0w200 = 1.54Probability/Ch3Gamblers ruinp(wn+1 wn ) = (1-p)(wn wn-1)wn = (1-p)wn-1 + pwn+1w0 = 0w200 = 1.wn+1 wn = l (wn wn-1)let l = (1-p)/p = 19/18= l2 (wn-1 wn-2)= = ln (w1 w0)wn+1 = wn + lnw1= wn-1 + ln-1w1 + lnw1 = = w1 + lw1 + + lnw155Probability/Ch3Gamblers ruinwn = (1-p)wn-1 + pwn+1w0 = 0w200 = 1.l = (1-p)/p = 19/18wn+1 = w1 + + lnw1= (ln+1 1)/(l 1)w1w200 = (l200 1)/(l 1)w1wn+1 =ln+1 1l200 1You have $100. You stop when you win$200 or run out. The probability you win isw100 0.0045 56Probability/Ch3The general gamblers ruin problemTwo people play a game with each other. The loser pays one dollar to the winner at the end of the game. Initially they have r1 and r2 dollars and the winning rates are p and 1-p respectively. They continue to play until one of them goes bankrupt. Find the probability that the guy with r1 dollars initially loses all the money at the end.

Let Pr1 be the probability then

57Probability/Ch3

Consequently,

58Probability/Ch3Laplace rule of succession

As k is large,

This probability tends to 1 as n tends to infinity.59Probability/Ch3Laplace rule of succession

If in the first n flips, only r flips are heads, n-r are tails, what is the probability that the (n+1)st flip is head ? As k is large,

60Probability/Ch3

we can use integration by parts. That is,

As a result,

and

as k is large61Probability/Ch3