1
CS473:ArtificialIntelligence
Bayes’Nets:Independence
DieterFox[TheseslideswerecreatedbyDanKleinandPieterAbbeelforCS188IntrotoAIatUCBerkeley.AllCS188materialsareavailableathttp://ai.berkeley.edu.]
Recap:Bayes’Nets
§ ABayes’netisanefficientencodingofaprobabilisticmodelofadomain
§ Questionswecanask:
§ Inference:givenafixedBN,whatisP(X|e)?
§ Representation:givenaBNgraph,whatkindsofdistributionscanitencode?
§ Modeling:whatBNismostappropriateforagivendomain?
Bayes’ Nets
§ Representation
§ ConditionalIndependences
§ ProbabilisticInference
§ LearningBayes’ NetsfromData
ConditionalIndependence
§ XandYareindependentif
§ XandYareconditionallyindependent givenZ
§ (Conditional)independenceisapropertyofadistribution
§ Example:
BayesNets:Assumptions
§ AssumptionswearerequiredtomaketodefinetheBayesnetwhengiventhegraph:
§ Beyondabove“chainruleà Bayesnet” conditionalindependenceassumptions
§ Oftenadditionalconditionalindependences
§ Theycanbereadoffthegraph
§ Importantformodeling:understandassumptionsmadewhenchoosingaBayesnetgraph
P (xi|x1 · · ·xi�1) = P (xi|parents(Xi))
IndependenceinaBN
§ ImportantquestionaboutaBN:§ Aretwonodesindependentgivencertainevidence?§ Ifyes,canproveusingalgebra(tediousingeneral)§ Ifno,canprovewithacounterexample§ Example:
§ Question:areXandZnecessarilyindependent?§ Answer:no.Example:lowpressurecausesrain,whichcausestraffic.§ XcaninfluenceZ,ZcaninfluenceX(viaY)§ Addendum:theycouldbeindependent:how?
X Y Z
2
D-separation:Outline D-separation:Outline
§ Studyindependencepropertiesfortriples
§ Analyzecomplexcasesintermsofmembertriples
§ D-separation:acondition/algorithmforansweringsuchqueries
CausalChains§ Thisconfigurationisa“causalchain”
X:LowpressureY:RainZ:Traffic
§ GuaranteedXindependentofZ? No!
§ OneexamplesetofCPTsforwhichXisnotindependentofZissufficienttoshowthisindependenceisnotguaranteed.
§ Example:
§ Lowpressurecausesraincausestraffic,highpressurecausesnoraincausesnotraffic
§ Innumbers:
P(+y|+x)=1,P(-y|- x)=1,P(+z|+y)=1,P(-z|-y)=1
CausalChains§ Thisconfigurationisa“causalchain” § GuaranteedXindependentofZgivenY?
§ Evidencealongthechain“blocks” theinfluence
Yes!
X:LowpressureY:RainZ:Traffic
CommonCause§ Thisconfigurationisa“commoncause” § GuaranteedXindependentofZ? No!
§ OneexamplesetofCPTsforwhichXisnotindependentofZissufficienttoshowthisindependenceisnotguaranteed.
§ Example:
§ Projectduecausesbothforumsbusyandlabfull
§ Innumbers:
P(+x|+y)=1,P(-x|-y)=1,P(+z|+y)=1,P(-z|-y)=1
Y:Projectdue
X:Forumsbusy Z:Labfull
CommonCause§ Thisconfigurationisa“commoncause” § GuaranteedXandZindependentgivenY?
§ Observingthecauseblocksinfluencebetweeneffects.
Yes!
Y:Projectdue
X:Forumsbusy Z:Labfull
3
CommonEffect§ Lastconfiguration:twocausesofone
effect(v-structures)
Z:Traffic
§ AreXandYindependent?
§ Yes:theballgameandtheraincausetraffic,buttheyarenotcorrelated
§ Stillneedtoprovetheymustbe(tryit!)
§ AreXandYindependentgivenZ?
§ No:seeingtrafficputstherainandtheballgameincompetitionasexplanation.
§ Thisisbackwardsfromtheothercases
§ Observinganeffectactivatesinfluencebetweenpossiblecauses.
X:Raining Y:Ballgame
TheGeneralCase
TheGeneralCase
§ Generalquestion:inagivenBN,aretwovariablesindependent(givenevidence)?
§ Solution:analyzethegraph
§ Anycomplexexamplecanbebrokenintorepetitionsofthethreecanonicalcases
Reachability
§ Recipe:shadeevidencenodes,lookforpathsintheresultinggraph
§ Attempt1:iftwonodesareconnectedbyanundirectedpathnotblockedbyashadednode,thentheyarenotconditionallyindependent
§ Almostworks,butnotquite§ Wheredoesitbreak?§ Answer:thev-structureatTdoesn’tcount
asalinkinapathunless“active”
R
T
B
D
L
Active/InactivePaths
§ Question:AreXandYconditionallyindependentgivenevidencevariables{Z}?§ Yes,ifXandY“d-separated” byZ§ Considerall(undirected)pathsfromXtoY§ Noactivepaths=independence!
§ Apathisactiveifeachtripleisactive:§ CausalchainAà Bà CwhereBisunobserved(eitherdirection)§ CommoncauseAß Bà CwhereBisunobserved§ Commoneffect(akav-structure)
Aà Bß CwhereBoroneofitsdescendents isobserved
§ Allittakestoblockapathisasingleinactivesegment
ActiveTriples InactiveTriples § Query:
§ Checkall(undirected!)pathsbetweenand§ Ifoneormoreactive,thenindependencenotguaranteed
§ Otherwise(i.e.ifallpathsareinactive),thenindependenceisguaranteed
D-Separation
Xi �� Xj |{Xk1 , ..., Xkn}
Xi �� Xj |{Xk1 , ..., Xkn}
?
Xi �� Xj |{Xk1 , ..., Xkn}
4
Example
Yes R
T
B
T’
Example
R
T
B
D
L
T’
Yes
Yes
Yes
Example
§ Variables:§ R:Raining§ T:Traffic§ D:Roofdrips§ S:I’msad
§ Questions:
T
S
D
R
Yes
StructureImplications
§ GivenaBayesnetstructure,canrund-separationalgorithmtobuildacompletelistofconditionalindependencesthatarenecessarilytrueoftheform
§ Thislistdeterminesthesetofprobabilitydistributionsthatcanberepresented
Xi �� Xj |{Xk1 , ..., Xkn}
ComputingAllIndependences
X
Y
Z
X
Y
Z
X
Y
Z
X
Y
Z
XY
Z
{X �� Y,X �� Z, Y �� Z,
X �� Z | Y,X �� Y | Z, Y �� Z | X}
TopologyLimitsDistributions
§ GivensomegraphtopologyG,onlycertainjointdistributionscanbeencoded
§ Thegraphstructureguaranteescertain(conditional)independences
§ (Theremightbemoreindependence)
§ Addingarcsincreasesthesetofdistributions,buthasseveralcosts
§ Fullconditioningcanencodeanydistribution
X
Y
Z
X
Y
Z
X
Y
Z
{X �� Z | Y }
X
Y
Z X
Y
Z X
Y
Z
X
Y
Z X
Y
Z X
Y
Z
{}
5
BayesNetsRepresentationSummary
§ Bayesnetscompactlyencodejointdistributions
§ GuaranteedindependenciesofdistributionscanbededucedfromBNgraphstructure
§ D-separationgivespreciseconditionalindependenceguaranteesfromgraphalone
§ ABayes’ net’sjointdistributionmayhavefurther(conditional)independencethatisnotdetectableuntilyouinspectitsspecificdistribution
Bayes’ Nets
§ Representation
§ ConditionalIndependences
§ ProbabilisticInference§ Enumeration(exact,exponentialcomplexity)§ Variableelimination(exact,worst-case
exponentialcomplexity,oftenbetter)§ ProbabilisticinferenceisNP-complete§ Sampling(approximate)
§ LearningBayes’ NetsfromData