Date post: | 20-Jan-2016 |
Category: |
Documents |
Upload: | kevin-little |
View: | 215 times |
Download: | 0 times |
Independence, Decomposability and functions which take values into an
Abelian Group
Adrian Silvescu Vasant Honavar
Department of Computer ScienceIowa State University
Decomposition and Independence
Decomposition renders problems more tractable.
Apply recursively Decomposition is enabled
by “independence” Decomposition and
independence are dual notions
A B
A B
A B
A B
Conditional Decomposition and Independence
Seldom are the two sub-problems disjoint
All is not lost Conditional
Decomposition / Independence
Conditioning on C C a.k.a. separator
CA B
A BA C C B
CA B
C
=
Formalization of the intuitions
Problem P = (D, S, solP) D = Domain, S = Solutions solP : D S
A BsolP
Example: Determinant_Computation(M2, R, det)
Conditional Independence / Decomposition Formalization (Variable Based)
P = (D = A X B X C, S, solP)
P1 = (A X C, S1, solP1), P2 = (B X C, S2, solP2)
solP(A, B, C) = solP1(A, C) solP2(B, C)
PPP 2,1
Probabilities
I(A, B|C) iff P(A, B| C) = P(A|C) P(B|C) Equivalently P(A, B, C) = P(A, C) P(B|C) P(A, B, C) = f1(A, C) f2(B, C) Independencies can be represented by a
graph where we do not draw edges between variables that are independent conditioned on the rest of the variables.
A C B
The Hammersley-Clifford Theorem: From Pairwise to Holistic Decomposability
)(
)()(GMaxCliquesCC CfVp
Outline
Generalized Conditional Independence with respect to a function f and properties
Theorems Conclusions and Discussion
Conditional Independence with respect to a function f - If(A,B|C)
solP(A, B, C) = solP1(A, C) solP2(B, C)
Assumptions: – S = S1 = S2 [= G]– .
– A, B, C is a partition of the set of all variables– Saturated independence statements – from now on
PPP 2,1
PPP 2,1
f(A, B, C) = f1(A, C) f2(B, C)
If(A,B|C)
Conditional Independence with respect to a function f If(A,B|C) – cont’d
A B C f
0 0 0 .25
0 0 1 .3
… … … …
=
A C f1
0 0 .5
0 1 .3
… … …
B C f2
0 0 .5
0 1 .3
… … …
If(A,B|C) iff
f(A, B, C) = f1(A, C) f2(B, C)
Examples of If(A,B|C )
Multiplicative (probabilities)
Additive (fitness, energy, value functions)
Relational (relations)
Properties of If(A,B|C )
1.Trivial Independence If(A, Φ|C) 2. Symmetry If(A, B|C) => If(B, A|C) 3. Weak Union If(A, B U D|C) => If(A, B|C U D) 4. Intersection If(A, B|C U D) & If(A, D|C U B) => If(A, B U D|C)
A CD
B
Abelian Groups
(G, +, 0, -) is an Abelian Group iff– + is associative and commutative– 0 is a neutral element – - is an inversion operator
Examples:– (R, + , 0, - ) - additive (value func.)– ((0, ∞), · , 1, -¹) - multiplicative (prob.)– ({0, 1}, mod2, 0, id) - relational (relations)
Outline
Generalized Conditional Independence with respect to a function f
Properties and Theorems Conclusions and Discussion
Markov Properties [Pearl & Paz ‘87]
If Axioms 1-4 then the following are equivalent
Pairwise – (α,β) G => If(α, β|V\{α,β})
Local -
If(α, V\(N(α)U{α})| N(α)) Global – If C=V\{A, B}
separates A and B in G If(A, B| C=V\{A, B})
α βV\{α,β}
N(α)α
A BC
Factorization – Main Theorem
The Factorization Theorem: From Pairwise to Holistic Decomposability
)()()(
CfVf CGMaxCliquesC
Particular Cases - Factorization
Probabilistic – Hammersley-Clifford
Additive Decomposability
Relational Decomposability
Graph Separability and Independence [Geiger & Pearl ‘ 93]
If Axioms 1-4 hold then
SepG(A, B|C) If(A, B|C)
for all saturated independence statements
Completeness
Axioms 1-4 provide a complete axiomatic characterization of independence statements for functions which take values over Abelian groups
Outline
Generalized Conditional Independence with respect to a function f
Properties and Theorems Conclusions and Discussion
Conclusions (1)
Introduced a very general notion of Conditional Independence / Decomposability.
Developed it into a notion of Conditional Independence relative to a function f which takes values into an Abelian Group If(.,.|.).
We proved that If(.,.|.) satisfies the following important independence properties:
– 1. Trivial independence, – 2. Symmetry, – 3. Weak union – 4. Intersection
Conclusions (2)
Axioms 1-4 imply the equivalence of the Global, Local and Pairwise Markov Properties for our notion conditional independence relation If(.,.|.)) based on the result from [Pearl and Paz '87].
We proved a natural generalization of the Hammersley-Clifford which allows us to factorize the function f over the cliques of an associated Markov Network which reflects the Conditional Independencies of subsets of variables with respect to f.
Completeness Theorem, Graph Separability Eq. Theorem The theory developed in this paper subsumes: probability
distributions, additive decomposable functions and relations, as particular cases of functions over Abelian Groups.
Discussion: Relation to Graphoids
(-) Decomposition
(-) Contraction
(+) Weak Contraction
Graphoids – No finite axiomatic charact. [Studeny ’92]
Intersection Discussion – noninvertible elms.
Discussion – cont’d
Graph Separability Independence Completeness Seems that
– Trivial Independence– Symmetry– Weak Union– Intersection
Strong Axiomatic core for Independence
Applications