1

CS473:ArtificialIntelligence

Bayes’Nets:Independence

DieterFox[TheseslideswerecreatedbyDanKleinandPieterAbbeelforCS188IntrotoAIatUCBerkeley.AllCS188materialsareavailableathttp://ai.berkeley.edu.]

Recap:Bayes’Nets

§ ABayes’netisanefficientencodingofaprobabilisticmodelofadomain

§ Questionswecanask:

§ Inference:givenafixedBN,whatisP(X|e)?

§ Representation:givenaBNgraph,whatkindsofdistributionscanitencode?

§ Modeling:whatBNismostappropriateforagivendomain?

Bayes’ Nets

§ Representation

§ ConditionalIndependences

§ ProbabilisticInference

§ LearningBayes’ NetsfromData

ConditionalIndependence

§ XandYareindependentif

§ XandYareconditionallyindependent givenZ

§ (Conditional)independenceisapropertyofadistribution

§ Example:

BayesNets:Assumptions

§ AssumptionswearerequiredtomaketodefinetheBayesnetwhengiventhegraph:

§ Beyondabove“chainruleà Bayesnet” conditionalindependenceassumptions

§ Oftenadditionalconditionalindependences

§ Theycanbereadoffthegraph

§ Importantformodeling:understandassumptionsmadewhenchoosingaBayesnetgraph

P (xi|x1 · · ·xi�1) = P (xi|parents(Xi))

IndependenceinaBN

§ ImportantquestionaboutaBN:§ Aretwonodesindependentgivencertainevidence?§ Ifyes,canproveusingalgebra(tediousingeneral)§ Ifno,canprovewithacounterexample§ Example:

§ Question:areXandZnecessarilyindependent?§ Answer:no.Example:lowpressurecausesrain,whichcausestraffic.§ XcaninfluenceZ,ZcaninfluenceX(viaY)§ Addendum:theycouldbeindependent:how?

X Y Z

2

D-separation:Outline D-separation:Outline

§ Studyindependencepropertiesfortriples

§ Analyzecomplexcasesintermsofmembertriples

§ D-separation:acondition/algorithmforansweringsuchqueries

CausalChains§ Thisconfigurationisa“causalchain”

X:LowpressureY:RainZ:Traffic

§ GuaranteedXindependentofZ? No!

§ OneexamplesetofCPTsforwhichXisnotindependentofZissufficienttoshowthisindependenceisnotguaranteed.

§ Example:

§ Lowpressurecausesraincausestraffic,highpressurecausesnoraincausesnotraffic

§ Innumbers:

P(+y|+x)=1,P(-y|- x)=1,P(+z|+y)=1,P(-z|-y)=1

CausalChains§ Thisconfigurationisa“causalchain” § GuaranteedXindependentofZgivenY?

§ Evidencealongthechain“blocks” theinfluence

Yes!

X:LowpressureY:RainZ:Traffic

CommonCause§ Thisconfigurationisa“commoncause” § GuaranteedXindependentofZ? No!

§ OneexamplesetofCPTsforwhichXisnotindependentofZissufficienttoshowthisindependenceisnotguaranteed.

§ Example:

§ Projectduecausesbothforumsbusyandlabfull

§ Innumbers:

P(+x|+y)=1,P(-x|-y)=1,P(+z|+y)=1,P(-z|-y)=1

Y:Projectdue

X:Forumsbusy Z:Labfull

CommonCause§ Thisconfigurationisa“commoncause” § GuaranteedXandZindependentgivenY?

§ Observingthecauseblocksinfluencebetweeneffects.

Yes!

Y:Projectdue

X:Forumsbusy Z:Labfull

3

CommonEffect§ Lastconfiguration:twocausesofone

effect(v-structures)

Z:Traffic

§ AreXandYindependent?

§ Yes:theballgameandtheraincausetraffic,buttheyarenotcorrelated

§ Stillneedtoprovetheymustbe(tryit!)

§ AreXandYindependentgivenZ?

§ No:seeingtrafficputstherainandtheballgameincompetitionasexplanation.

§ Thisisbackwardsfromtheothercases

§ Observinganeffectactivatesinfluencebetweenpossiblecauses.

X:Raining Y:Ballgame

TheGeneralCase

TheGeneralCase

§ Generalquestion:inagivenBN,aretwovariablesindependent(givenevidence)?

§ Solution:analyzethegraph

§ Anycomplexexamplecanbebrokenintorepetitionsofthethreecanonicalcases

Reachability

§ Recipe:shadeevidencenodes,lookforpathsintheresultinggraph

§ Attempt1:iftwonodesareconnectedbyanundirectedpathnotblockedbyashadednode,thentheyarenotconditionallyindependent

§ Almostworks,butnotquite§ Wheredoesitbreak?§ Answer:thev-structureatTdoesn’tcount

asalinkinapathunless“active”

R

T

B

D

L

Active/InactivePaths

§ Question:AreXandYconditionallyindependentgivenevidencevariables{Z}?§ Yes,ifXandY“d-separated” byZ§ Considerall(undirected)pathsfromXtoY§ Noactivepaths=independence!

§ Apathisactiveifeachtripleisactive:§ CausalchainAà Bà CwhereBisunobserved(eitherdirection)§ CommoncauseAß Bà CwhereBisunobserved§ Commoneffect(akav-structure)

Aà Bß CwhereBoroneofitsdescendents isobserved

§ Allittakestoblockapathisasingleinactivesegment

ActiveTriples InactiveTriples § Query:

§ Checkall(undirected!)pathsbetweenand§ Ifoneormoreactive,thenindependencenotguaranteed

§ Otherwise(i.e.ifallpathsareinactive),thenindependenceisguaranteed

D-Separation

Xi �� Xj |{Xk1 , ..., Xkn}

Xi �� Xj |{Xk1 , ..., Xkn}

?

Xi �� Xj |{Xk1 , ..., Xkn}

4

Example

Yes R

T

B

T’

Example

R

T

B

D

L

T’

Yes

Yes

Yes

Example

§ Variables:§ R:Raining§ T:Traffic§ D:Roofdrips§ S:I’msad

§ Questions:

T

S

D

R

Yes

StructureImplications

§ GivenaBayesnetstructure,canrund-separationalgorithmtobuildacompletelistofconditionalindependencesthatarenecessarilytrueoftheform

§ Thislistdeterminesthesetofprobabilitydistributionsthatcanberepresented

Xi �� Xj |{Xk1 , ..., Xkn}

ComputingAllIndependences

X

Y

Z

X

Y

Z

X

Y

Z

X

Y

Z

XY

Z

{X �� Y,X �� Z, Y �� Z,

X �� Z | Y,X �� Y | Z, Y �� Z | X}

TopologyLimitsDistributions

§ GivensomegraphtopologyG,onlycertainjointdistributionscanbeencoded

§ Thegraphstructureguaranteescertain(conditional)independences

§ (Theremightbemoreindependence)

§ Addingarcsincreasesthesetofdistributions,buthasseveralcosts

§ Fullconditioningcanencodeanydistribution

X

Y

Z

X

Y

Z

X

Y

Z

{X �� Z | Y }

X

Y

Z X

Y

Z X

Y

Z

X

Y

Z X

Y

Z X

Y

Z

{}

5

BayesNetsRepresentationSummary

§ Bayesnetscompactlyencodejointdistributions

§ GuaranteedindependenciesofdistributionscanbededucedfromBNgraphstructure

§ D-separationgivespreciseconditionalindependenceguaranteesfromgraphalone

§ ABayes’ net’sjointdistributionmayhavefurther(conditional)independencethatisnotdetectableuntilyouinspectitsspecificdistribution

Bayes’ Nets

§ Representation

§ ConditionalIndependences

§ ProbabilisticInference§ Enumeration(exact,exponentialcomplexity)§ Variableelimination(exact,worst-case

exponentialcomplexity,oftenbetter)§ ProbabilisticinferenceisNP-complete§ Sampling(approximate)

§ LearningBayes’ NetsfromData