+ All Categories
Home > Documents > Recap: Bayes’ Nets · 2017. 5. 22. · 5 Bayes Nets Representation Summary §Bayes nets compactly...

Recap: Bayes’ Nets · 2017. 5. 22. · 5 Bayes Nets Representation Summary §Bayes nets compactly...

Date post: 18-Nov-2020
Category:
Upload: others
View: 0 times
Download: 0 times
Share this document with a friend
5
1 CS 473: Artificial Intelligence Bayes’ Nets: Independence Dieter Fox [These slides were created by Dan Klein and Pieter Abbeel for CS188 Intro to AI at UC Berkeley. All CS188 materials are available at http://ai.berkeley.edu.] Recap: Bayes’ Nets § A Bayes’ net is an efficient encoding of a probabilistic model of a domain § Questions we can ask: § Inference: given a fixed BN, what is P(X | e)? § Representation: given a BN graph, what kinds of distributions can it encode? § Modeling: what BN is most appropriate for a given domain? Bayes’ Nets § Representation § Conditional Independences § Probabilistic Inference § Learning Bayes’ Nets from Data Conditional Independence § X and Y are independent if § X and Y are conditionally independent given Z § (Conditional) independence is a property of a distribution § Example: Bayes Nets: Assumptions § Assumptions we are required to make to define the Bayes net when given the graph: § Beyond above “chain rule à Bayes net” conditional independence assumptions § Often additional conditional independences § They can be read off the graph § Important for modeling: understand assumptions made when choosing a Bayes net graph P (xi|x1 ··· xi-1)= P (xi|parents(Xi)) Independence in a BN § Important question about a BN: § Are two nodes independent given certain evidence? § If yes, can prove using algebra (tedious in general) § If no, can prove with a counter example § Example: § Question: are X and Z necessarily independent? § Answer: no. Example: low pressure causes rain, which causes traffic. § X can influence Z, Z can influence X (via Y) § Addendum: they could be independent: how? X Y Z
Transcript
Page 1: Recap: Bayes’ Nets · 2017. 5. 22. · 5 Bayes Nets Representation Summary §Bayes nets compactly encode joint distributions §Guaranteed independencies of distributions can be

1

CS473:ArtificialIntelligence

Bayes’Nets:Independence

DieterFox[TheseslideswerecreatedbyDanKleinandPieterAbbeelforCS188IntrotoAIatUCBerkeley.AllCS188materialsareavailableathttp://ai.berkeley.edu.]

Recap:Bayes’Nets

§ ABayes’netisanefficientencodingofaprobabilisticmodelofadomain

§ Questionswecanask:

§ Inference:givenafixedBN,whatisP(X|e)?

§ Representation:givenaBNgraph,whatkindsofdistributionscanitencode?

§ Modeling:whatBNismostappropriateforagivendomain?

Bayes’ Nets

§ Representation

§ ConditionalIndependences

§ ProbabilisticInference

§ LearningBayes’ NetsfromData

ConditionalIndependence

§ XandYareindependentif

§ XandYareconditionallyindependent givenZ

§ (Conditional)independenceisapropertyofadistribution

§ Example:

BayesNets:Assumptions

§ AssumptionswearerequiredtomaketodefinetheBayesnetwhengiventhegraph:

§ Beyondabove“chainruleà Bayesnet” conditionalindependenceassumptions

§ Oftenadditionalconditionalindependences

§ Theycanbereadoffthegraph

§ Importantformodeling:understandassumptionsmadewhenchoosingaBayesnetgraph

P (xi|x1 · · ·xi�1) = P (xi|parents(Xi))

IndependenceinaBN

§ ImportantquestionaboutaBN:§ Aretwonodesindependentgivencertainevidence?§ Ifyes,canproveusingalgebra(tediousingeneral)§ Ifno,canprovewithacounterexample§ Example:

§ Question:areXandZnecessarilyindependent?§ Answer:no.Example:lowpressurecausesrain,whichcausestraffic.§ XcaninfluenceZ,ZcaninfluenceX(viaY)§ Addendum:theycouldbeindependent:how?

X Y Z

Page 2: Recap: Bayes’ Nets · 2017. 5. 22. · 5 Bayes Nets Representation Summary §Bayes nets compactly encode joint distributions §Guaranteed independencies of distributions can be

2

D-separation:Outline D-separation:Outline

§ Studyindependencepropertiesfortriples

§ Analyzecomplexcasesintermsofmembertriples

§ D-separation:acondition/algorithmforansweringsuchqueries

CausalChains§ Thisconfigurationisa“causalchain”

X:LowpressureY:RainZ:Traffic

§ GuaranteedXindependentofZ? No!

§ OneexamplesetofCPTsforwhichXisnotindependentofZissufficienttoshowthisindependenceisnotguaranteed.

§ Example:

§ Lowpressurecausesraincausestraffic,highpressurecausesnoraincausesnotraffic

§ Innumbers:

P(+y|+x)=1,P(-y|- x)=1,P(+z|+y)=1,P(-z|-y)=1

CausalChains§ Thisconfigurationisa“causalchain” § GuaranteedXindependentofZgivenY?

§ Evidencealongthechain“blocks” theinfluence

Yes!

X:LowpressureY:RainZ:Traffic

CommonCause§ Thisconfigurationisa“commoncause” § GuaranteedXindependentofZ? No!

§ OneexamplesetofCPTsforwhichXisnotindependentofZissufficienttoshowthisindependenceisnotguaranteed.

§ Example:

§ Projectduecausesbothforumsbusyandlabfull

§ Innumbers:

P(+x|+y)=1,P(-x|-y)=1,P(+z|+y)=1,P(-z|-y)=1

Y:Projectdue

X:Forumsbusy Z:Labfull

CommonCause§ Thisconfigurationisa“commoncause” § GuaranteedXandZindependentgivenY?

§ Observingthecauseblocksinfluencebetweeneffects.

Yes!

Y:Projectdue

X:Forumsbusy Z:Labfull

Page 3: Recap: Bayes’ Nets · 2017. 5. 22. · 5 Bayes Nets Representation Summary §Bayes nets compactly encode joint distributions §Guaranteed independencies of distributions can be

3

CommonEffect§ Lastconfiguration:twocausesofone

effect(v-structures)

Z:Traffic

§ AreXandYindependent?

§ Yes:theballgameandtheraincausetraffic,buttheyarenotcorrelated

§ Stillneedtoprovetheymustbe(tryit!)

§ AreXandYindependentgivenZ?

§ No:seeingtrafficputstherainandtheballgameincompetitionasexplanation.

§ Thisisbackwardsfromtheothercases

§ Observinganeffectactivatesinfluencebetweenpossiblecauses.

X:Raining Y:Ballgame

TheGeneralCase

TheGeneralCase

§ Generalquestion:inagivenBN,aretwovariablesindependent(givenevidence)?

§ Solution:analyzethegraph

§ Anycomplexexamplecanbebrokenintorepetitionsofthethreecanonicalcases

Reachability

§ Recipe:shadeevidencenodes,lookforpathsintheresultinggraph

§ Attempt1:iftwonodesareconnectedbyanundirectedpathnotblockedbyashadednode,thentheyarenotconditionallyindependent

§ Almostworks,butnotquite§ Wheredoesitbreak?§ Answer:thev-structureatTdoesn’tcount

asalinkinapathunless“active”

R

T

B

D

L

Active/InactivePaths

§ Question:AreXandYconditionallyindependentgivenevidencevariables{Z}?§ Yes,ifXandY“d-separated” byZ§ Considerall(undirected)pathsfromXtoY§ Noactivepaths=independence!

§ Apathisactiveifeachtripleisactive:§ CausalchainAà Bà CwhereBisunobserved(eitherdirection)§ CommoncauseAß Bà CwhereBisunobserved§ Commoneffect(akav-structure)

Aà Bß CwhereBoroneofitsdescendents isobserved

§ Allittakestoblockapathisasingleinactivesegment

ActiveTriples InactiveTriples § Query:

§ Checkall(undirected!)pathsbetweenand§ Ifoneormoreactive,thenindependencenotguaranteed

§ Otherwise(i.e.ifallpathsareinactive),thenindependenceisguaranteed

D-Separation

Xi �� Xj |{Xk1 , ..., Xkn}

Xi �� Xj |{Xk1 , ..., Xkn}

?

Xi �� Xj |{Xk1 , ..., Xkn}

Page 4: Recap: Bayes’ Nets · 2017. 5. 22. · 5 Bayes Nets Representation Summary §Bayes nets compactly encode joint distributions §Guaranteed independencies of distributions can be

4

Example

Yes R

T

B

T’

Example

R

T

B

D

L

T’

Yes

Yes

Yes

Example

§ Variables:§ R:Raining§ T:Traffic§ D:Roofdrips§ S:I’msad

§ Questions:

T

S

D

R

Yes

StructureImplications

§ GivenaBayesnetstructure,canrund-separationalgorithmtobuildacompletelistofconditionalindependencesthatarenecessarilytrueoftheform

§ Thislistdeterminesthesetofprobabilitydistributionsthatcanberepresented

Xi �� Xj |{Xk1 , ..., Xkn}

ComputingAllIndependences

X

Y

Z

X

Y

Z

X

Y

Z

X

Y

Z

XY

Z

{X �� Y,X �� Z, Y �� Z,

X �� Z | Y,X �� Y | Z, Y �� Z | X}

TopologyLimitsDistributions

§ GivensomegraphtopologyG,onlycertainjointdistributionscanbeencoded

§ Thegraphstructureguaranteescertain(conditional)independences

§ (Theremightbemoreindependence)

§ Addingarcsincreasesthesetofdistributions,buthasseveralcosts

§ Fullconditioningcanencodeanydistribution

X

Y

Z

X

Y

Z

X

Y

Z

{X �� Z | Y }

X

Y

Z X

Y

Z X

Y

Z

X

Y

Z X

Y

Z X

Y

Z

{}

Page 5: Recap: Bayes’ Nets · 2017. 5. 22. · 5 Bayes Nets Representation Summary §Bayes nets compactly encode joint distributions §Guaranteed independencies of distributions can be

5

BayesNetsRepresentationSummary

§ Bayesnetscompactlyencodejointdistributions

§ GuaranteedindependenciesofdistributionscanbededucedfromBNgraphstructure

§ D-separationgivespreciseconditionalindependenceguaranteesfromgraphalone

§ ABayes’ net’sjointdistributionmayhavefurther(conditional)independencethatisnotdetectableuntilyouinspectitsspecificdistribution

Bayes’ Nets

§ Representation

§ ConditionalIndependences

§ ProbabilisticInference§ Enumeration(exact,exponentialcomplexity)§ Variableelimination(exact,worst-case

exponentialcomplexity,oftenbetter)§ ProbabilisticinferenceisNP-complete§ Sampling(approximate)

§ LearningBayes’ NetsfromData


Recommended