Date post: | 04-Jan-2016 |
Category: |
Documents |
Upload: | helena-mitchell |
View: | 213 times |
Download: | 1 times |
Probability and Independence
1
Krishna.V.PalemKenneth and Audrey Kennedy Professor of ComputingDepartment of Computer Science, Rice University
Probability
2
The classical definition of probability Pierre Simon
LaplaceThe probability of an event is the ratio of the number of cases favorable to it, to the number of all cases possible when nothing leads us to expect that any one of these cases should occur more than any other, which renders them, for us, equally possible.Let us elaborate…
Consider an event space
Event 1
Event 2
Event 3
Event 4
If there is no reasonbelieve that one event is more likely to occur
than another
Probabilityof favorableevents
Favorable events
No. of favorableevents
Total no. of events
Generalizing Probability
3
Consider another event space
Event 1
Event 2
Event 3
Event 4
Let us assume that each event is differently likelyto occur
Let us represent as a list of magnitude of“likeliness”
EventLikelines
s
Event 1 p1
Event 2 p2
Event 3 p3
Event 4 p4
Generalizing Probability (Contd.)
4
EventLikelines
s
Event 1 p1
Event 2 p2
Event 3 p3
Event 4 p4
Let us look at these values.
If these values have the following properties
1. All pi are in the range [0,1] 2. The sum of all pi = 1.
then these values are called the probabilities of theseevents.
For example, Event Probabiliti
es
Event 1 0.5
Event 2 0.3
Event 3 0.1
Event 4 0.1
Satisfies both the conditions
This means that event 1 occurs 1 out of 4 times … etc.
Specification
5
Consider the event space
Event 1
Event 2
Event 3
Event 4
Event Probabilities
Event 1 p1
Event 2 p2
Event 3 p3
Event 4 p4
Complete specification of the
behavior of the experiment
Both these together
6
Now we have this relationship
Event
Event 1
Event 2
Event 3
Event 4
Probabilities
p1
p2
p3
p4
This can be defined concisely as a function whose independent variablerepresents the event
The dependent variable is the valueof the probability
Let the variable ‘x’ represent the event
x Event
1Event
1
2Event
2
3Event
3
4Event
4
ipixp )(Probability (Event i)
Here ‘x’ is called a random variable.Where i={1,2,3,4}
Analysis of the event space of a coin tossCan you list the event space of an
experiment where an unbiased coin is being tossed ?
To complete the specification we need the probabilities of these events.
7
Event H
Event T
Event Probabilities
Event H ½
Event T ½
The method of union of eventsJoint Event space of
two coins
Event HH
Event TT
Event HTEvent TH
EventProbabili
ty
Event HH
1/4
Event HT
1/4
Event TH
1/4
Event TT
1/4
Consider the outcome
Heads in both the coins “OR”Tails in both the coins
From the previous definition,
Probability of the outcome = 2/4
= 1/2
Another way of representing the same probability is
Probability of (Heads in both the coins “OR” Tails in both the coins)
= Probability (Heads in both the coins) + Probability(Tails in both the coins)
Why do you think that the “OR” of two events is translated to a + for their probabilities ?8
Mutually Exclusive events and outcomes
Joint Event space of two coins
Event HH
Event TT
Event HTEvent TH
Two events are said to be mutually exclusive if
(i)Given one event as the outcome of the experiment it is assumed that all other outcomes are discarded.
For example, if Event HH is the outcome of the experimentthen Event TT “cannot” the outcome of the experiment
Let us consider another example,
Joint Event space of two coins
First coin is H
Second coin is T
First coin is TSecond coin is H
The event space of the two coins can also be represented like this because itcovers all possible events
But these two events are not mutually exclusive because if the eventthat the first coin is H occurs then the event that the second coin isH can also occur
Hence these events are “not” mutually exclusive9
The method of union of events
Joint Event space of two coins
Event HH
Event TT
Event HTEvent TH
The method of union of events states that “ If two events are mutually exclusive then the probabilitythat either of them will occur is the sum of probabilities of the two events”
10
Let us play the experiment ∞ times
Outcomes
HH HT TH TT HH HH TT HH HT TH HT TH TH …. ∞
Relative Frequency = (HH , HH, HH) + (TT,TT,TT,….)
All Outcomes All Outcomes
1/4 1/4+ = 1/2
For infinite trials For infinite trials
The method of union of events
11
RecallConsider an event space
Event 1
Event 2Event 3
Event 4
Favorable events
Probabilityof favorableevents
No. of favorableevents
Total no. of events
Event Probabilities
Event 1 p1
Event 2 p2
Event 3 p3
Event 4 p4
….Etc.
Given all the above events are mutually exclusive
Probabilityof favorableevents
.......42 Etcpp
….. Etc.
The method of union of events
Let us consider the case of two dice
Joint Event Space of Die 1 & 2
Event 11Event 12
Event 21Event 23
….. Etc.Using method of union of events, calculate the following
(i)Probability that the first DIE and the second die rolled a ‘3’ OR both the DICE rolled a ‘6’
A) P(Event 33 OR Event 66) = 1/36 + 1/36 = 1/18
(ii) Probability that 3 and 4 were rolled be either one of them
A) P(Event 34 OR Event 43) = 1/36 + 1/36 = 1/18
Are these events mutually exclusive ?
12
Mini-exercise 1 – Snakes and Ladders(S&L)Calculate the probability of landing in
square 6 OR square 8 in the following scaled-down version of snakes and ladders (no snake or ladder)?
13
7 8 9
6 5 4
1 2 3
We will start at square 1 Roll the die 50 times and
record the squares that you land in
Calculate the chance of landing in square 6 OR square 8
Note that favorable events could be derived both from landing on square 6 OR on square 8
Instructions for Mini-Exercise 1
The method of intersection of events
Joint Event space of two coins
Event HH
Event TT
Event HTEvent TH
Event Space of coin 1
Event H
Event T
Event Space of coin 2
Event H
Event T
Event
P
H 2/3
T 1/3
Event
P
H ½
T ½
Consider this joint experimentagain
What is the probability of the outcome of the experiment having HEAD in the first die and TAIL in the second die ?
All the events are not equally likely
Cannot use the earlier definition ofFavorable events
Total number of events14
The method of intersection of events
Joint Event space of two coins
Event HH
Event TT
Event HTEvent TH
Event Space of coin 1
Event H
Event T
Event Space of coin 2
Event H
Event T
Event
P
H 2/3
T 1/3
Event
P
H ½
T ½
Consider this joint experimentagain
What is the probability of the outcome of the experiment having HEAD in the first coin and TAIL in the second coin ?
= Probability of first coin having HEAD * Probability of second coin having TAIL = ½ * 1/3 = 1/6
H in coin 1 = ½ of the time
T in coin 2= 1/3 of this = ½ * 1/3 = 1/6 of the time
15
The method of intersection of events
Joint Event space of two coins
Event HH
Event TT
Event HTEvent TH
Event Space of coin 1
Event H
Event T
Event Space of coin 2
Event H
Event T
Event
P
H 2/3
T 1/3
Event
P
H ½
T ½
The method of intersection of events says “The probability of a joint event is equal to the product of probability of elementary events”
16
The method of intersection of eventsLet us consider the case of two dice
Using method of intersection of events,Calculate the following
(i) Probability that the first die rolled a ‘1’ and the second die rolled a ‘4’
A) P(Event 14) = P(Event 1) * P(Event 4) = 1/12 * 1/12 = 1/144
(ii) Probability that the first die rolled a ‘6’ and the second die rolled a ‘6’
A) P(Event 66) = P(Event 6) * P(Event 6) = 1/3 * 1/12 = 1/36
Outcome
Die 1 Die 2
1 1/12 1/3
2 1/12 1/3
3 1/12 1/12
4 1/12 1/12
5 1/3 1/12
6 1/3 1/12
Table of probabilities of the outcomes
17
Mini-exercise 2 – Snakes and Ladders(S&L)Calculate the probability of landing in
square 2 AND square 8 in the following scaled-down version of snakes and ladders (no snake or ladder)?
18
7 8 9
6 5 4
1 2 3
We will start at square 1 Roll the die for 50 times
and record the squares that you land in
Calculate the chance of landing in square 6 AND 8 in one cycle
In this case, favorable events requires that both events occur
A cycle is completed when you go from square 9 to square 1
Instructions for Mini-Exercise 2
Building a modelAn important component of this course
(including exercises and projects) would be building models, and these games are an example.
An example of a model would be
19
4
5
6
1/61/6
20
Solution:Probability of reaching 6 via 5 = 1/6Probability of reaching 6 via 4 = 1/6
Total probability of reaching 6 from 4 OR 5 = 1/6 + 1/6 = 1/3
4
5
6
1/61/6
21
4
5
6
1/6
1/6
1/6
You are at square 4 and want to compute the probability of reaching square 6
This can happen in two ways
(i) You roll a 1 and reach 5 and then roll a 1 again
(ii) You roll a 2 and reach 6
As we know the probabilities of the rolls we can calculate the probability of reaching 6 from 4
Solution:Probability of reaching 6 via 5 = (1/6 * 1/6 )Probability of reaching 6 directly = 1/6
Total probability of reaching 6 from 4 = 7/36
Mini-Exercise 3 Can you build a model for the snakes and
ladder game till square 6 ?In this game, if we cross square 6 with a die
roll we stay at 6That means, if you are at square 4 and you roll a 5,
then you still stay at 6
22
6 5 4
1 2 3
Remember the elements of a modelA node or a circle that indicates a particular square in
the gameAn arrow or an edge that indicates the possibility of
reaching another square from the current squareA label on each arrow that indicates the probability of
that transition taking place
23
4
5
6
3
21
1/6
3/6
1/6
1/6
1/6
2/6
1/61/6
1/6
24
This is a partial model of the scaled down snakes and ladder game
4
5
6
5/6
1/6
6/6 3
21
1/6
1/6
3/6
1/6
1/6
4/6
1/6
1/6
2/6
1/61/6
Mini-Exercise - 4 Using the model of the snakes and ladders,
calculate the following valuesThe probability of reaching square 4 from
square 1
The probability of reaching square 5 OR square 6 from square 2
The probability of reaching square 3 AND square 5 from square 1
25
26
6 5 4
1 2 3
4
5
6
5/6
1/6
6/6 3
21
1/6
1/6
3/6
1/6
1/64/6
1/6
1/6
2/6
1/61/6
Let us start with the transition model we built
Hint: New edge(s) might be added as an extension
Mini-Exercise - 5 Using the new model of the snakes and
ladders with the snake, calculate the following valuesThe probability of reaching square 4 from
square 1
The probability of reaching square 5 OR square 6 from square 2
The probability of reaching square 3 AND square 5 from square 1
27
Question ?
28
4
5
6
1/6
1/6
1/6 3
2
1
1/6
1/6
1/6
1/6
1/6
1/6
1/6
1/6
1/6
1/61/6
Say that you are at square 4 at a point in the game
You roll a die and you reach 6
What can we do to get that information ?
29
4
5
6
1/6
1/6
1/6 3
2
1
1/6
1/6
1/6
1/6
1/6
1/6
1/6
1/6
1/6
1/61/6
What if we place some sort of a token at each square we land ?
Let us say that we land at square 2
And then we reached square 4
So we leave a token at square 2
Thus after completing the game we have the additional “information” to determine where we have been
More information
30
4
5
6
1/6
1/6
1/6 3
2
1
1/6
1/6
1/6
1/6
1/6
1/6
1/6
1/6
1/6
1/61/6
So basically if we play this game multiple times, then we need more tokens (probably with different colors) to keep track of where we were each time.
Let us say that an all-knowing “Oracle” tells us that the previous square we were in was odd and not even.
Do we still need tokens to keep track ?
Will we need more tokens or less ?
Answers
31
4
5
6
1/6
1/6
1/6 3
2
1
1/6
1/6
1/6
1/6
1/6
1/6
1/6
1/6
1/6
1/61/6
We will analyze more precisely the relationship between two events in the next class.
Independence and DependenceTill now, we have been looking at events individually. But in practice, events are interrelated. Some events are dependent on some other event
taking place
Let us consider the following example
Let us play a small game. Bob is rolling a die. He asks Alice to guess what he rolled.
1.What is the probability that Bob is correct ?
2.Let us say that the Oracle told Danny that the outcome of the die is an even number or an odd number. Then what is the probability that Danny is correct ?
32
There is an underlying mathematical concept that can be explicitly stated to calculatethe answer to the question
Consider an experiment.
The outcome of the experiment can be specified in terms of two different event spaces
Event 1Event 2Event 3
…
Event EvenEvent Odd
Knowledge of the outcome of the roll of a die in terms of whether it is an even number or an odd number allows us to predict the actual outcome more precisely
p
p1
p2
p3
…
p
Peve
n
Podd
33 Let us calculate how much it improves our quality of our guess
There is an underlying mathematical concept that can be explicitly stated to calculatethe answer to the question
Consider an experiment.
The outcome of the experiment can be specified in terms of two different event spaces
Event 1Event 2Event 3
…
Event EvenEvent Odd
Knowledge of the outcome of the roll of a die in terms of whether it is an even number or an odd number allows us to predict the actual outcome more precisely
p
p1
p2
p3
…
p
Peve
n
Podd
34 Let us calculate how much it improves our quality of our guess