Date post: | 27-Dec-2015 |
Category: |
Documents |
Upload: | wilfred-wilcox |
View: | 214 times |
Download: | 0 times |
Uncertain KR&R
Chapter 9
2
Outline
• Probability
• Bayesian networks
• Fuzzy logic
3
Probability
FOL fails for a domain due to:
Laziness: too much to list the complete set of rules, too hard to use the enormous rules that result
Theoretical ignorance: there is no complete theory for the domain
Practical ignorance: have not or cannot run all necessary tests
4
Probability
• Probability = a degree of belief
• Probability comes from:
Frequentist: experiments and statistical assessment
Objectivist: real aspects of the universe
Subjectivist: a way of characterizing an agent’s beliefs
• Decision theory = probability theory + utility theory
5
Probability
Prior probability: probability in the absence of any other information
P(Dice = 2) = 1/6
random variable: Dice
domain = <1, 2, 3, 4, 5, 6>
probability distribution: P(Dice) = <1/6, 1/6, 1/6, 1/6, 1/6, 1/6>
6
Probability
Conditional probability: probability in the presence of some evidence
P(Dice = 2 | Dice is even) = 1/3
P(Dice = 2 | Dice is odd) = 0
P(A | B) = P(A B)/P(B)
P(A B) = P(A | B).P(B)
7
Probability
Example:S = stiff neckM = meningitisP(S | M) = 0.5P(M) = 1/50000P(S) = 1/20
P(M | S) = P(S | M).P(M)/P(S) = 1/5000
8
Probability
Joint probability distributions:
X: <x1, …, xm> Y: <y1, …, yn>
P(X = xi, Y = yj)
9
Probability
Axioms:
• 0 P(A) 1
• P(true) = 1 and P(false) = 0
• P(A B) = P(A) + P(B) P(A B)
10
Probability
Derived properties:
• P(A) = 1 P(A)
• P(U) = P(A1) + P(A2) + ... + P(An)
U = A1 A2 ... An collectively exhaustive
Ai Aj = false mutually exclusive
11
Probability
Bayes’ theorem:
P(Hi | E) = P(E | Hi).P(Hi)/iP(E | Hi).P(Hi)
Hi’s are collectively exhaustive & mutually exclusive
12
Probability
Problem: a full joint probability distribution P(X1, X2, ..., Xn) is sufficient for computing any (conditional) probability on Xi's, but the number of joint probabilities is exponential.
13
Probability
• Independence:
P(A B) = P(A).P(B)
P(A) = P(A | B)
• Conditional independence:
P(A B | E) = P(A | E).P(B | E)
P(A | E) = P(A | E B)
Example:
P(Toothache | Cavity Catch) = P(Toothache | Cavity)
P(Catch | Cavity Toothache) = P(Catch | Cavity)
14
Probability
"In John's and Mary's house, an alarm is installed to sound in case of burglary or earthquake. When the alarm sounds, John and Mary may make a call for help or rescue."
15
Probability
"In John's and Mary's house, an alarm is installed to sound in case of burglary or earthquake. When the alarm sounds, John and Mary may make a call for help or rescue."
Q1: If earthquake happens, how likely will John make a call?
16
Probability
"In John's and Mary's house, an alarm is installed to sound in case of burglary or earthquake. When the alarm sounds, John and Mary may make a call for help or rescue."
Q1: If earthquake happens, how likely will John make a call?
Q2: If the alarm sounds, how likely is the house burglarized?
17
Bayesian Networks
• Pearl, J. (1982). Reverend Bayes on Inference Engines: A Distributed Hierarchical Approach, presented at the Second National Conference on Artificial Intelligence (AAAI-82), Pittsburgh, Pennsylvania
• 2000 AAAI Classic Paper Award
18
Bayesian Networks
Burglary Earthquake
Alarm
JohnCalls MaryCalls
P(B)
.001
P(E)
.002
B E P(A)
T TT FF TF F
.95
.94
.29
.001
A P(J)TF
.90
.05
A P(M)TF
.70
.01
19
Bayesian NetworksSyntax:
• A set of random variables makes up the nodes
• A set of directed links connects pairs of nodes
• Each node has a conditional probability table that quantifies the effects of its parent nodes
• The graph has no directed cycles
A
B C
D
A
B C
D
yes no
20
Bayesian Networks
Semantics:
• An ordering on the nodes: Xi is a predecessor of Xj i < j
• P(X1, X2, …, Xn)
= P(Xn | Xn-1, …, X1).P(Xn-1 | Xn-2, …, X1). … .P(X2 | X1).P(X1)
= iP(Xi | Xi-1, …, X1) = iP(Xi | Parents(Xi))
P (Xi | Xi-1, …, X1) = P(Xi | Parents(Xi)) Parents(Xi) {Xi-1, …, X1}
Each node is conditionally independent of its predecessors given its parents
21
Bayesian Networks
Example:
P(J M A B E)
= P(J | A).P(M | A).P(A | B E).P(B).P(E)
= 0.00062
B E
A
J M
22
• Why Bayesian Networks?
Bayesian Networks
23
P(Query | Evidence) = ?
Diagnostic (from effects to causes): P(B | J)
Causal (from causes to effects): P(J | B)
Intercausal (between causes of a common effect): P(B | A, E)
Mixed: P(A | J, E), P(B | J, E)
Uncertain Question Answering
B E
A
J M
24
• The independence assumptions in a Bayesian Network simplify computation of conditional probabilities on its variables
Uncertain Question Answering
25
Uncertain Question Answering
Q1: If earthquake happens, how likely will John make a call?
Q2: If the alarm sounds, how likely is the house burglarized?
Q3: If the alarm sounds, how likely both John and Mary make calls?
26
P(B | A)
= P(B A)/P(A)
= P(B A)
P(B | A)
= P(B A)
= 1P(B A) + P(B A))
Uncertain Question Answering
27
• A Bayesian Network implies all conditional independence among its variables
General Conditional Independence
28
General Conditional Independence
Y1 Yn
U1 Um
Z1j Znj
X
A node (X) is conditionally independent of its non-
descendents (Zij's), given its parents (Ui's)
29
General Conditional Independence
Y1 Yn
U1 Um
Z1j Znj
X
A node (X) is conditionally independent of all other
nodes, given its parents (Ui's), children (Yi's), and
children's parents (Zij's)
30
General Conditional Independence
X and Y are conditionally independent given E
Z
Z
Z
X YE
31
General Conditional Independence
Example:
Battery
GasIgnitionRadio
Starts
Moves
32
General Conditional Independence
Example:
Gas - (Ignition) - Radio
Gas - (Battery) - Radio
Gas - (no evidence at all) - Radio
Gas Radio | Starts
Gas Radio | Moves
Battery
GasIgnitionRadio
Starts
Moves
33
Vagueness
• The Oxford Companion to Philosophy (1995):
“Words like smart, tall, and fat are vague since in most contexts of use there is no bright line separating them from not smart, not tall, and not fat respectively …”
34
Vagueness
• Imprecision vs. Uncertainty:
The bottle is about half-full.
vs.
It is likely to a degree of 0.5 that the bottle is full.
35
Fuzzy Sets
• Zadeh, L.A. (1965). Fuzzy SetsJournal of Information and Control
36
Fuzzy Sets
37
Fuzzy Sets
38
Fuzzy Set Definition
A fuzzy set is defined by a membership function that maps elements of a given domain (a crisp set) into values in [0, 1].
A: U [0, 1]
A A
0
1
25
40
Age30
0.5
young
39
Fuzzy Set Representation
• Discrete domain:high-dice score: {1:0, 2:0, 3:0.2, 4:0.5, 5:0.9,
6:1}
• Continuous domain:
A(u) = 1 for u[0, 25]A(u) = (40 - u)/15 for u[25, 40]A(u) = 0 for u[40, 150]
0
1
25
40
Age30
0.5
40
Fuzzy Set Representation
• -cuts:
A = {u | A(u) }
A = {u | A(u) } strong -cut
A0.5 = [0, 30]
0
1
25
40
Age30
0.5
41
Fuzzy Set Representation
• -cuts:
A = {u | A(u) }
A = {u | A(u) } strong -cut
A(u) = sup { | u A}
A0.5 = [0, 30]
0
1
25
40
Age30
0.5
42
Fuzzy Set Representation
• Support: supp(A) = {u | A(u) > 0} = A0+
• Core: core(A) = {u | A(u) = 1} = A1
• Height: h(A) = supUA(u)
43
Fuzzy Set Representation
• Normal fuzzy set: h(A) = 1
• Sub-normal fuzzy set: h(A) 1
44
Membership Degrees
• Subjective definition
45
Membership Degrees
• Subjective definition
• Voting model:
Each voter has a subset of U as his/her own crisp definition of the concept that A represents.
A(u) is the proportion of voters whose crisp definitions include u.
46
Membership Degrees
• Voting model:
P1 P2 P3 P4 P5 P6 P7 P8 P9 P10
1
2
3 x x
4 x x x x x
5 x x x x x x x x x
6 x x x x x x x x x x
47
Fuzzy Subset Relations
A B iff A(u) B(u) for every uU
A is more “specific” than B
“X is A” entails “X is B”
48
Fuzzy Set Operations
• Standard definitions:
Complement: A(u) = 1 A(u)
Intersection: (AB)(u) = min[A(u), B(u)]
Union: (AB)(u) = max[A(u), B(u)]
49
Fuzzy Set Operations
• Example:
not young = young
not old = old
middle-age = not youngnot old
old = young
50
Fuzzy Relations
• Crisp relation:
R(U1, ..., Un) U1 ... Un
R(u1, ..., un) = 1 iff (u1, ..., un) R or = 0 otherwise
51
Fuzzy Relations
• Crisp relation:
R(U1, ..., Un) U1 ... Un
R(u1, ..., un) = 1 iff (u1, ..., un) R or = 0 otherwise
• Fuzzy relation: a fuzzy set on U1 ... Un
52
Fuzzy Relations
• Fuzzy relation:
U1 = {New York, Paris}, U2 = {Beijing, New York, London}
R = “very far”
R = {(NY, Beijing): 1, ...}
NY Paris
Beijing 1 .9
NY 0 .7
London .6 .3
53
Fuzzy Numbers
• A fuzzy number A is a fuzzy set on R:
A must be a normal fuzzy set
A must be a closed interval for every (0, 1]
supp(A) = A0+ must be bounded
54
Basic Types of Fuzzy Numbers
1
0
1
0
1
0
1
0
55
Basic Types of Fuzzy Numbers
1
0
1
0
56
Operations of Fuzzy Numbers
• Interval-based operations:
(A B)= A B
57
Operations of Fuzzy Numbers
• Arithmetic operations on intervals:
[a, b][d, e] = {fg | a f b, d g e}
58
Operations of Fuzzy Numbers
• Arithmetic operations on intervals:
[a, b][d, e] = {fg | a f b, d g e}
[a, b] + [d, e] = [a + d, b + e]
[a, b] [d, e] = [a e, b d]
[a, b]*[d, e] = [min(ad, ae, bd, be), max(ad, ae, bd, be)]
[a, b]/[d, e] = [a, b]*[1/e, 1/d] 0[d, e]
59
Operations of Fuzzy Numbers
+
1
about 2
about 2 + about 3 = ?
about 2 about 3 = ?
about 3
02 3
60
Operations of Fuzzy Numbers
• Discrete domains:
A = {xi: A(xi)} B = {yi: B(yi)}
A B= ?
61
Operations of Fuzzy Numbers
• Extension principle:
f: U1 U2 V
induces
g: U1 U2 V
[g(A, B)](v) = sup{(u1, u2) | v = f(u1, u2)}min{A(u1), B(u2)}
~~~
62
Operations of Fuzzy Numbers
• Discrete domains:
A = {xi: A(xi)} B = {yi: B(yi)}
(A B)(v)= sup{(xi, yj) | v = xi°yj)}min{A(xi), B(yj)}
63
Fuzzy Logic
if x is A then y is Bx is A*
------------------------ y is B*
64
Fuzzy Logic
• View a fuzzy rule as a fuzzy relation
• Measure similarity of A and A*
65
Fuzzy Controller
• As special expert systems
• When difficult to construct mathematical models
• When acquired models are expensive to use
66
Fuzzy Controller
IF the temperature is very high
AND the pressure is slightly low
THEN the heat change should be slightly negative
67
Fuzzy Controller
Controlledprocess
Defuzzificationmodel
Fuzzificationmodel
Fuzzy inference engine
Fuzzy rule base
actions
conditions
FUZZY CONTROLLER
68
Fuzzification
1
0
x0
69
Defuzzification
• Center of Area:
x = (A(z).z)/A(z)
70
Defuzzification
• Center of Maxima:
M = {z | A(z) = h(A)}
x = (min M + max M)/2
71
Defuzzification
• Mean of Maxima:
M = {z | A(z) = h(A)}
x = z/|M|
72
Exercises
• In Klir-Yuan’s textbook: 1.9, 1.10, 2.11, 2.12, 4.5