Cognitive Systems 300:
Probability and Causality (cont.)
David Poole and Peter Danielson
University of British Columbia
Fall 2013
1 David Poole and Peter Danielson Cognitive Systems 300: Probability and Causality (cont.)
The story so far....
Agents act in environments.
A controller considers:
what should the agent do nowwhat should the agent remember or believe
as a function of percepts and previous memory.
Memories are symbol structure; reasoning is search.
Hierarchical systems reduce complexity.
Acting is gambling.probabilities: possible worlds + conditioning
2 David Poole and Peter Danielson Cognitive Systems 300: Probability and Causality (cont.)
Learning Objectives
At the end of the class you should be able to:
know how to compute marginals and apply Bayes’theorem
build a belief network for a domain
predict the inferences for a belief network
explain the predictions of a causal model
3 David Poole and Peter Danielson Cognitive Systems 300: Probability and Causality (cont.)
Bayes’ theorem
The definition of conditioning and the commutativity ofconjunction (h ∧ e is equivalent to e ∧ h) gives us:
P(h ∧ e) =
P(h | e)× P(e)
= P(e | h)× P(h).
If P(e) 6= 0, divide the right hand sides by P(e):
P(h | e) =P(e | h)× P(h)
P(e).
This is Bayes’ theorem.
4 David Poole and Peter Danielson Cognitive Systems 300: Probability and Causality (cont.)
Bayes’ theorem
The definition of conditioning and the commutativity ofconjunction (h ∧ e is equivalent to e ∧ h) gives us:
P(h ∧ e) = P(h | e)× P(e)
=
P(e | h)× P(h).
If P(e) 6= 0, divide the right hand sides by P(e):
P(h | e) =P(e | h)× P(h)
P(e).
This is Bayes’ theorem.
4 David Poole and Peter Danielson Cognitive Systems 300: Probability and Causality (cont.)
Bayes’ theorem
The definition of conditioning and the commutativity ofconjunction (h ∧ e is equivalent to e ∧ h) gives us:
P(h ∧ e) = P(h | e)× P(e)
= P(e | h)× P(h).
If P(e) 6= 0, divide the right hand sides by P(e):
P(h | e) =
P(e | h)× P(h)
P(e).
This is Bayes’ theorem.
4 David Poole and Peter Danielson Cognitive Systems 300: Probability and Causality (cont.)
Bayes’ theorem
The definition of conditioning and the commutativity ofconjunction (h ∧ e is equivalent to e ∧ h) gives us:
P(h ∧ e) = P(h | e)× P(e)
= P(e | h)× P(h).
If P(e) 6= 0, divide the right hand sides by P(e):
P(h | e) =P(e | h)× P(h)
P(e).
This is Bayes’ theorem.
4 David Poole and Peter Danielson Cognitive Systems 300: Probability and Causality (cont.)
Why is Bayes’ theorem interesting?
Often we have causal knowledge:P(symptom | disease)P(light is off | status of switches and switch positions)P(alarm | fire)
P(image looks like | a tree is in front of a car)P(data | model)
and want to do evidential reasoning:P(disease | symptom)P(status of switches | light is off and switch positions)P(fire | alarm).
P(a tree is in front of a car | image looks like )P(model | data)
5 David Poole and Peter Danielson Cognitive Systems 300: Probability and Causality (cont.)
Exercise
A cab was involved in a hit-and-run accident at night. Twocab companies, the Green and the Blue, operate in the city.You are given the following data:
85% of the cabs in the city are Green and 15% are Blue.
A witness identified the cab as Blue. The court tested thereliability of the witness in the circumstances that existedon the night of the accident and concluded that thewitness correctly identifies each one of the two colours80% of the time and failed 20% of the time.
What is the probability that the cab involved in the accidentwas Blue?
From D. Kahneman, Thinking Fast and Slow, 2011, p. 166.
6 David Poole and Peter Danielson Cognitive Systems 300: Probability and Causality (cont.)
Exercise
A cab was involved in a hit-and-run accident at night. Twocab companies, the Green and the Blue, operate in the city.You are given the following data:
85% of the cabs in the city are Green and 15% are Blue.
A witness identified the cab as Blue. The court tested thereliability of the witness in the circumstances that existedon the night of the accident and concluded that thewitness correctly identifies each one of the two colours80% of the time and failed 20% of the time.
What is the probability that the cab involved in the accidentwas Blue?From D. Kahneman, Thinking Fast and Slow, 2011, p. 166.
6 David Poole and Peter Danielson Cognitive Systems 300: Probability and Causality (cont.)
Exercise
A cab was involved in a hit-and-run accident at night. Twocab companies, the Green and the Blue, operate in the city.You are given the following data:
The two companies operate the same number of cabs,but Green cabs are involved in 85% of the accidents.
A witness identified the cab as Blue. The court tested thereliability of the witness in the circumstances that existedon the night of the accident and concluded that thewitness correctly identifies each one of the two colours80% of the time and failed 20% of the time.
What is the probability that the cab involved in the accidentwas Blue?
From D. Kahneman, Thinking Fast and Slow, 2011, p. 167.Chapter 16 “Causes trump statistics”
7 David Poole and Peter Danielson Cognitive Systems 300: Probability and Causality (cont.)
Exercise
A cab was involved in a hit-and-run accident at night. Twocab companies, the Green and the Blue, operate in the city.You are given the following data:
The two companies operate the same number of cabs,but Green cabs are involved in 85% of the accidents.
A witness identified the cab as Blue. The court tested thereliability of the witness in the circumstances that existedon the night of the accident and concluded that thewitness correctly identifies each one of the two colours80% of the time and failed 20% of the time.
What is the probability that the cab involved in the accidentwas Blue?From D. Kahneman, Thinking Fast and Slow, 2011, p. 167.Chapter 16 “Causes trump statistics”
7 David Poole and Peter Danielson Cognitive Systems 300: Probability and Causality (cont.)
Tversky and Kahneman’s Linda [1983]
Linda is thirty-one years old, single, outspoken,and very bright. She majored in philosophy. As astudent, she was deeply concerned with issues ofdiscrimination and social justice, and alsoparticipated in antinuclear demonstrations.
Which is more probable:
(a) Linda is a bank teller.
(b) Linda is a bank teller and is active in the feministmovement.
85% to 95% of undergraduates at several major universitieschose the second option.From Tversky and Kahneman, Psychological Review, 1983.See D. Kahneman, Thinking Fast and Slow, 2011, p. 156.
8 David Poole and Peter Danielson Cognitive Systems 300: Probability and Causality (cont.)
Tversky and Kahneman’s Linda [1983]
Linda is thirty-one years old, single, outspoken,and very bright. She majored in philosophy. As astudent, she was deeply concerned with issues ofdiscrimination and social justice, and alsoparticipated in antinuclear demonstrations.
Which is more probable:
(a) Linda is a bank teller.
(b) Linda is a bank teller and is active in the feministmovement.
85% to 95% of undergraduates at several major universitieschose the second option.From Tversky and Kahneman, Psychological Review, 1983.See D. Kahneman, Thinking Fast and Slow, 2011, p. 156.
8 David Poole and Peter Danielson Cognitive Systems 300: Probability and Causality (cont.)
Tversky and Kahneman’s Linda [1983]
Linda is thirty-one years old, single, outspoken,and very bright. She majored in philosophy. As astudent, she was deeply concerned with issues ofdiscrimination and social justice, and alsoparticipated in antinuclear demonstrations.
Which is more probable:
(a) Linda is a bank teller.
(b) Linda is a bank teller and is active in the feministmovement.
85% to 95% of undergraduates at several major universitieschose the second option.From Tversky and Kahneman, Psychological Review, 1983.See D. Kahneman, Thinking Fast and Slow, 2011, p. 156.
8 David Poole and Peter Danielson Cognitive Systems 300: Probability and Causality (cont.)
Psychology of probability assessments
Which is more probable:
(a) A massive flood somewhere in North America next year,in which more than 1000 people drown.
(b) An earthquake in California sometime next year, causing aflood in which more than 1000 people drown.
From D. Kahneman, Thinking Fast and Slow, 2011, p. 159.
9 David Poole and Peter Danielson Cognitive Systems 300: Probability and Causality (cont.)
Psychology of probability assessments
Which is more probable:
(a) A massive flood somewhere in North America next year,in which more than 1000 people drown.
(b) An earthquake in California sometime next year, causing aflood in which more than 1000 people drown.
From D. Kahneman, Thinking Fast and Slow, 2011, p. 159.
9 David Poole and Peter Danielson Cognitive Systems 300: Probability and Causality (cont.)
Psychology of probability assessments
There is a regular six-sided die with four green faces and twored faces. Here are three sequences of greens (G) and reds (R)and you have to choose one. Which do you choose:
(a) RGRRR
(b) GRGRRR
(c) GRRRRR
From D. Kahneman, Thinking Fast and Slow, 2011, p. 162.almost two-thirds preferred (b)....until it was explained to them
10 David Poole and Peter Danielson Cognitive Systems 300: Probability and Causality (cont.)
Psychology of probability assessments
There is a regular six-sided die with four green faces and twored faces. Here are three sequences of greens (G) and reds (R)and you have to choose one. Which do you choose:
(a) RGRRR
(b) GRGRRR
(c) GRRRRR
From D. Kahneman, Thinking Fast and Slow, 2011, p. 162.almost two-thirds preferred (b)....until it was explained to them
10 David Poole and Peter Danielson Cognitive Systems 300: Probability and Causality (cont.)
Conditional independence
Random variable X is independent of random variable Y givenrandom variable Z if
P(X | Y ,Z )
= P(X | Z ).
That is, knowledge of Y ’s value doesn’t affect the belief in X ,given a value of Z .
11 David Poole and Peter Danielson Cognitive Systems 300: Probability and Causality (cont.)
Example domain — lights and switches in a house
light
two-wayswitch
switch
off
on
poweroutlet
circuit�breaker
outside power
�
l1
l2
w1
w0
w2
w4
w3
w6
w5
p2
p1
cb2
cb1s1
s2s3
12 David Poole and Peter Danielson Cognitive Systems 300: Probability and Causality (cont.)
Bayesian Belief networks
Directed acyclic graph where the nodes are randomvariables.
We generate the variables one at a time.When generating X , the parents of X are those alreadygenerated variables upon which X directly depends.
Represents the conditional (in)dependence assumption:a variable is independent of its non-descendants given itsparents.
13 David Poole and Peter Danielson Cognitive Systems 300: Probability and Causality (cont.)
Example: fire alarm belief network
Variables:
Fire: there is a fire in the building
Tampering: someone has been tampering with the firealarm
Smoke: what appears to be smoke is coming from anupstairs window
Alarm: the fire alarm goes off
Leaving: people are leaving the building en masse.
Report: a colleague says that people are leaving thebuilding en masse. (A noisy sensor for leaving.)
14 David Poole and Peter Danielson Cognitive Systems 300: Probability and Causality (cont.)
Components of a belief network
A belief network consists of:
a directed acyclic graph with nodes labeled with randomvariables
a domain for each random variable
a set of conditional probability tables for each variablegiven its parents (including prior probabilities for nodeswith no parents).
15 David Poole and Peter Danielson Cognitive Systems 300: Probability and Causality (cont.)
Example: wet grass on a summers day
Variables:
Shoes wet after walking on grass
Sprinkler was on last night
Grass wet
Rained last night
Grass shiny and appears to be wet
http://artint.info/tutorials/sprinkler.xml
16 David Poole and Peter Danielson Cognitive Systems 300: Probability and Causality (cont.)