Oct 5 2004 ConsistencyProperties amp Algorithms
1
Foundations of Constraint Processing Fall 2004
Foundations of Constraint ProcessingCSCE421821 Fall 2004
wwwcseunledu~choueiryF04-421-821
Berthe Y Choueiry (Shu-we-ri)Avery Hall Room 123Bchoueirycseunledu
Tel +1(402)472-5444
Consistency Algorithms
Oct 5 2004 ConsistencyProperties amp Algorithms
2
Foundations of Constraint Processing Fall 2004
Lecture SourcesRequired reading1 Algorithms for Constraint Satisfaction Problems Mackworth and
Freuder AIJ852 Sections 31 32 33 Chapter 3 Constraint Processing Dechter
Recommended1 Sections 34mdash310 Chapter 3 Constraint Processing Dechter2 Networks of Constraints Fundamental Properties and Application
to Picture Processing Montanari Information Sciences 743 Consistency in Networks of Relations Mackworth AIJ77 4 Constraint Propagation with Interval Labels Davis AIJ875 Path Consistency on Triangulated Constraint Graphs Bliek amp Sam-
Haroud IJCAI99
Oct 5 2004 ConsistencyProperties amp Algorithms
3
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
4
Foundations of Constraint Processing Fall 2004
Consistency checkingMotivation bull CSP are solved with search (conditioning)bull Search performance is affected by
- problem size- amount of backtracking
bull Backtracking may yield
ThrashingExploring non-promising sub-trees and rediscovering the same inconsistencies over and over again
Malady of backtrack search
Oct 5 2004 ConsistencyProperties amp Algorithms
5
Foundations of Constraint Processing Fall 2004
(Some) causes of thrashingSearch order V1 V2 V3 V4 V5
What happens in search if webull do not check the constraint on V3 (node inconsistency)bull do not check constraint between V3 and V5 (arc
inconsistency)bull do not check constraints between V3mdashV4 V3mdashV5 and
V4mdashV5 (path inconsistency)
1 2V4
1 2 3
0 1 2
1 2V5
1 2 3V1 V2
V3ltV2
V3gt0
V3ltV4
V5ltV4
V3ltV5
V3ltV1
V3
Oct 5 2004 ConsistencyProperties amp Algorithms
6
Foundations of Constraint Processing Fall 2004
Consistency checkingGoal eliminate inconsistent combinations
Algorithms (for binary constraints)
bull Node consistencybull Arc consistency (AC-1 AC-2 AC-3 AC-7 AC-31 etc)bull Path consistency (PC-1 PC-2 DPC PPC etc)bull Constraints of arbitrary arity
ndash Waltz algorithm ancestor of ACsndash Generalized arc-consistency (Dechter Section 351 )ndash Relational m-consistency (Dechter Section 81)
Oct 5 2004 ConsistencyProperties amp Algorithms
7
Foundations of Constraint Processing Fall 2004
Overview of Recommended Reading
bull Davis AIJ77
ndash focuses on the Waltz algorithm ndash studies its performance and quiescence for given
bull constraint types (order relations bounded diff algebraic etc)bull domains types (continuous or finite)
bull Mackworth AIJ77ndash presents NC AC-1 AC-2 PC-1 PC-2
bull Mackworth and Freuder AIJ85
ndash studies their complexity Mackworth and Freuder concentrate on finite domains More people work on finite domains than on continuous ones Continuous domains can be quite tough
Oct 5 2004 ConsistencyProperties amp Algorithms
8
Foundations of Constraint Processing Fall 2004
Constraint Propagation with Interval Labels Ernest Davis
bull `Old paper (1987) terminology slightly differentbull Interval labels continuous domains
ndash Section 8 sign labels (discrete)bull Concerned with Waltz algorithm (eg
quiescence completeness)bull Constraint types vs domain types (Table 3)bull Addresses applications from reasoning
ndash about Physical Systems (circuit SPAM)ndash about time relations (TMM)
Advice read Section 3 more if you wish
Oct 5 2004 ConsistencyProperties amp Algorithms
9
Foundations of Constraint Processing Fall 2004
Waltz algorithm for label inference
bull Refine(C(Vi Vk Vm Vn) Vi) - finds a new label for Vi consistent with C
bull Revise(C (Vi Vk Vm Vn)) refines the domains of Vi Vk Vm Vn by iterating over these variables until quiescence (no domains is further refined)
bull Waltz Algorithm revises all constraints by iterating over each constraint connected to a variable whose domain has been revised
bull Waltz Algorithm ancestor of arc-consistency
C
Vk Vm Vn
Vi
Oct 5 2004 ConsistencyProperties amp Algorithms
10
Foundations of Constraint Processing Fall 2004
WARNINGrarr Completeness Running the Waltz Algorithm does not solve
the problem
A=2 and B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis provides a classification of problems in terms of completeness and quiescence of the Waltz algorithm
f(constraint types domain types) see Table 3
1 2 3 2 3 4
BA BA
2 3 2 3
Oct 5 2004 ConsistencyProperties amp Algorithms
11
Foundations of Constraint Processing Fall 2004
Importance of this paper Establishes that constraints of bounded differences
(temporal reasoning) can be efficiently solved by the Waltz algorithm (O(n3) n number of variables)
Early paper that attracts attention on the difficulty of label propagation in interval labels (continuous domains)
This work has been continued by Faltings (AIJ 92) who studied the early-quiescence problem of the Waltz algorithm
Oct 5 2004 ConsistencyProperties amp Algorithms
12
Foundations of Constraint Processing Fall 2004
Consistency checking algorithms
Examining finite binary CSPs and their complexity
Notation
Given variables i j k with values x y z the predicate
Pijk(x y z) is true iff the 3-tuple x y z Cijk
Node consistency checking Pi(x)
Arc consistency checking Pij(xy)
Path consistency bit-matrix manipulation
Oct 5 2004 ConsistencyProperties amp Algorithms
13
Foundations of Constraint Processing Fall 2004
Worst-case complexity as a function of the input parameters
Upper bound f(n) is O(g(n)) means that f(n) cg(n)
f grows as g or slower
Lower boundf(n) is (h(n)) means that f(n) ch(n)
f grows as g or faster
Input parameters for a CSP
n = number of variables
a = (max) size of a domain
dk = degree of Vk ( n-1)
e = number of edges (or constraints) [(n-1) n(n-1)2]
Worst-case asymptotic complexitytime
space
Oct 5 2004 ConsistencyProperties amp Algorithms
14
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node amp arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
15
Foundations of Constraint Processing Fall 2004
Node consistency (NC)Procedure NC(i)
Di Di x | Pi(x)
Begin
for i 1 until n do NC(i)
end
Oct 5 2004 ConsistencyProperties amp Algorithms
16
Foundations of Constraint Processing Fall 2004
Complexity of NC Procedure of NC(i)
Di Di x | Pi(x) Begin for i 1 until n do NC(i)end
bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)
Oct 5 2004 ConsistencyProperties amp Algorithms
17
Foundations of Constraint Processing Fall 2004
Arc-consistency Adapted from Dechter
Definition Given a constraint graph G
bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj
bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi
bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
18
Foundations of Constraint Processing Fall 2004
Procedure ReviseRevises the domains of a variable i
Procedure Revise(ij) Begin
DELETE false for each x Di do
if there is no y Dj such that Pij(x y) then begin
delete x from Di
DELETE true end return DELETE end
bull Revise is directionalbull What is the complexity of Revise
a2
Oct 5 2004 ConsistencyProperties amp Algorithms
19
Foundations of Constraint Processing Fall 2004
Revise example R Dechter
Apply the Revise procedure to the following example
5 6 15 1 2 10
YX+Y = 10X
Oct 5 2004 ConsistencyProperties amp Algorithms
20
Foundations of Constraint Processing Fall 2004
Solution DVi DVi Vi (CViVj DVi)
This is actually equivalent to DVi Vi (CViVj DVi)
Effect of Revise Adapted from Rina Dechter
Question Given
bull two variables Vi and Vj
bull their domains DVi and DVj and
bull the constraint CViVj
write the effect of the Revise procedure as a sequence of operations in relational algebra
Hintbull Think about the domain DVi as a unary constraint CVi and
bull consider the composition of this unary constraint and the binary one
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
21
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end
bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)
Oct 5 2004 ConsistencyProperties amp Algorithms
22
Foundations of Constraint Processing Fall 2004
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
23
Foundations of Constraint Processing Fall 2004
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Oct 5 2004 ConsistencyProperties amp Algorithms
24
Foundations of Constraint Processing Fall 2004
NC amp AC Example courtesy of M Fromherz
In the temporal problem below
1 AC propagates bound
2 Unary constraint xgt3 is imposed
3 AC propagates bounds again
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
Oct 5 2004 ConsistencyProperties amp Algorithms
25
Foundations of Constraint Processing Fall 2004
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Oct 5 2004 ConsistencyProperties amp Algorithms
26
Foundations of Constraint Processing Fall 2004
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Oct 5 2004 ConsistencyProperties amp Algorithms
27
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end
Oct 5 2004 ConsistencyProperties amp Algorithms
28
Foundations of Constraint Processing Fall 2004
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (i j) arcs(G) i j
4 While Q is not empty do
5 begin
6 select and delete any arc (k m) from Q
7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
2
Foundations of Constraint Processing Fall 2004
Lecture SourcesRequired reading1 Algorithms for Constraint Satisfaction Problems Mackworth and
Freuder AIJ852 Sections 31 32 33 Chapter 3 Constraint Processing Dechter
Recommended1 Sections 34mdash310 Chapter 3 Constraint Processing Dechter2 Networks of Constraints Fundamental Properties and Application
to Picture Processing Montanari Information Sciences 743 Consistency in Networks of Relations Mackworth AIJ77 4 Constraint Propagation with Interval Labels Davis AIJ875 Path Consistency on Triangulated Constraint Graphs Bliek amp Sam-
Haroud IJCAI99
Oct 5 2004 ConsistencyProperties amp Algorithms
3
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
4
Foundations of Constraint Processing Fall 2004
Consistency checkingMotivation bull CSP are solved with search (conditioning)bull Search performance is affected by
- problem size- amount of backtracking
bull Backtracking may yield
ThrashingExploring non-promising sub-trees and rediscovering the same inconsistencies over and over again
Malady of backtrack search
Oct 5 2004 ConsistencyProperties amp Algorithms
5
Foundations of Constraint Processing Fall 2004
(Some) causes of thrashingSearch order V1 V2 V3 V4 V5
What happens in search if webull do not check the constraint on V3 (node inconsistency)bull do not check constraint between V3 and V5 (arc
inconsistency)bull do not check constraints between V3mdashV4 V3mdashV5 and
V4mdashV5 (path inconsistency)
1 2V4
1 2 3
0 1 2
1 2V5
1 2 3V1 V2
V3ltV2
V3gt0
V3ltV4
V5ltV4
V3ltV5
V3ltV1
V3
Oct 5 2004 ConsistencyProperties amp Algorithms
6
Foundations of Constraint Processing Fall 2004
Consistency checkingGoal eliminate inconsistent combinations
Algorithms (for binary constraints)
bull Node consistencybull Arc consistency (AC-1 AC-2 AC-3 AC-7 AC-31 etc)bull Path consistency (PC-1 PC-2 DPC PPC etc)bull Constraints of arbitrary arity
ndash Waltz algorithm ancestor of ACsndash Generalized arc-consistency (Dechter Section 351 )ndash Relational m-consistency (Dechter Section 81)
Oct 5 2004 ConsistencyProperties amp Algorithms
7
Foundations of Constraint Processing Fall 2004
Overview of Recommended Reading
bull Davis AIJ77
ndash focuses on the Waltz algorithm ndash studies its performance and quiescence for given
bull constraint types (order relations bounded diff algebraic etc)bull domains types (continuous or finite)
bull Mackworth AIJ77ndash presents NC AC-1 AC-2 PC-1 PC-2
bull Mackworth and Freuder AIJ85
ndash studies their complexity Mackworth and Freuder concentrate on finite domains More people work on finite domains than on continuous ones Continuous domains can be quite tough
Oct 5 2004 ConsistencyProperties amp Algorithms
8
Foundations of Constraint Processing Fall 2004
Constraint Propagation with Interval Labels Ernest Davis
bull `Old paper (1987) terminology slightly differentbull Interval labels continuous domains
ndash Section 8 sign labels (discrete)bull Concerned with Waltz algorithm (eg
quiescence completeness)bull Constraint types vs domain types (Table 3)bull Addresses applications from reasoning
ndash about Physical Systems (circuit SPAM)ndash about time relations (TMM)
Advice read Section 3 more if you wish
Oct 5 2004 ConsistencyProperties amp Algorithms
9
Foundations of Constraint Processing Fall 2004
Waltz algorithm for label inference
bull Refine(C(Vi Vk Vm Vn) Vi) - finds a new label for Vi consistent with C
bull Revise(C (Vi Vk Vm Vn)) refines the domains of Vi Vk Vm Vn by iterating over these variables until quiescence (no domains is further refined)
bull Waltz Algorithm revises all constraints by iterating over each constraint connected to a variable whose domain has been revised
bull Waltz Algorithm ancestor of arc-consistency
C
Vk Vm Vn
Vi
Oct 5 2004 ConsistencyProperties amp Algorithms
10
Foundations of Constraint Processing Fall 2004
WARNINGrarr Completeness Running the Waltz Algorithm does not solve
the problem
A=2 and B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis provides a classification of problems in terms of completeness and quiescence of the Waltz algorithm
f(constraint types domain types) see Table 3
1 2 3 2 3 4
BA BA
2 3 2 3
Oct 5 2004 ConsistencyProperties amp Algorithms
11
Foundations of Constraint Processing Fall 2004
Importance of this paper Establishes that constraints of bounded differences
(temporal reasoning) can be efficiently solved by the Waltz algorithm (O(n3) n number of variables)
Early paper that attracts attention on the difficulty of label propagation in interval labels (continuous domains)
This work has been continued by Faltings (AIJ 92) who studied the early-quiescence problem of the Waltz algorithm
Oct 5 2004 ConsistencyProperties amp Algorithms
12
Foundations of Constraint Processing Fall 2004
Consistency checking algorithms
Examining finite binary CSPs and their complexity
Notation
Given variables i j k with values x y z the predicate
Pijk(x y z) is true iff the 3-tuple x y z Cijk
Node consistency checking Pi(x)
Arc consistency checking Pij(xy)
Path consistency bit-matrix manipulation
Oct 5 2004 ConsistencyProperties amp Algorithms
13
Foundations of Constraint Processing Fall 2004
Worst-case complexity as a function of the input parameters
Upper bound f(n) is O(g(n)) means that f(n) cg(n)
f grows as g or slower
Lower boundf(n) is (h(n)) means that f(n) ch(n)
f grows as g or faster
Input parameters for a CSP
n = number of variables
a = (max) size of a domain
dk = degree of Vk ( n-1)
e = number of edges (or constraints) [(n-1) n(n-1)2]
Worst-case asymptotic complexitytime
space
Oct 5 2004 ConsistencyProperties amp Algorithms
14
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node amp arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
15
Foundations of Constraint Processing Fall 2004
Node consistency (NC)Procedure NC(i)
Di Di x | Pi(x)
Begin
for i 1 until n do NC(i)
end
Oct 5 2004 ConsistencyProperties amp Algorithms
16
Foundations of Constraint Processing Fall 2004
Complexity of NC Procedure of NC(i)
Di Di x | Pi(x) Begin for i 1 until n do NC(i)end
bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)
Oct 5 2004 ConsistencyProperties amp Algorithms
17
Foundations of Constraint Processing Fall 2004
Arc-consistency Adapted from Dechter
Definition Given a constraint graph G
bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj
bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi
bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
18
Foundations of Constraint Processing Fall 2004
Procedure ReviseRevises the domains of a variable i
Procedure Revise(ij) Begin
DELETE false for each x Di do
if there is no y Dj such that Pij(x y) then begin
delete x from Di
DELETE true end return DELETE end
bull Revise is directionalbull What is the complexity of Revise
a2
Oct 5 2004 ConsistencyProperties amp Algorithms
19
Foundations of Constraint Processing Fall 2004
Revise example R Dechter
Apply the Revise procedure to the following example
5 6 15 1 2 10
YX+Y = 10X
Oct 5 2004 ConsistencyProperties amp Algorithms
20
Foundations of Constraint Processing Fall 2004
Solution DVi DVi Vi (CViVj DVi)
This is actually equivalent to DVi Vi (CViVj DVi)
Effect of Revise Adapted from Rina Dechter
Question Given
bull two variables Vi and Vj
bull their domains DVi and DVj and
bull the constraint CViVj
write the effect of the Revise procedure as a sequence of operations in relational algebra
Hintbull Think about the domain DVi as a unary constraint CVi and
bull consider the composition of this unary constraint and the binary one
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
21
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end
bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)
Oct 5 2004 ConsistencyProperties amp Algorithms
22
Foundations of Constraint Processing Fall 2004
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
23
Foundations of Constraint Processing Fall 2004
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Oct 5 2004 ConsistencyProperties amp Algorithms
24
Foundations of Constraint Processing Fall 2004
NC amp AC Example courtesy of M Fromherz
In the temporal problem below
1 AC propagates bound
2 Unary constraint xgt3 is imposed
3 AC propagates bounds again
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
Oct 5 2004 ConsistencyProperties amp Algorithms
25
Foundations of Constraint Processing Fall 2004
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Oct 5 2004 ConsistencyProperties amp Algorithms
26
Foundations of Constraint Processing Fall 2004
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Oct 5 2004 ConsistencyProperties amp Algorithms
27
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end
Oct 5 2004 ConsistencyProperties amp Algorithms
28
Foundations of Constraint Processing Fall 2004
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (i j) arcs(G) i j
4 While Q is not empty do
5 begin
6 select and delete any arc (k m) from Q
7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
3
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
4
Foundations of Constraint Processing Fall 2004
Consistency checkingMotivation bull CSP are solved with search (conditioning)bull Search performance is affected by
- problem size- amount of backtracking
bull Backtracking may yield
ThrashingExploring non-promising sub-trees and rediscovering the same inconsistencies over and over again
Malady of backtrack search
Oct 5 2004 ConsistencyProperties amp Algorithms
5
Foundations of Constraint Processing Fall 2004
(Some) causes of thrashingSearch order V1 V2 V3 V4 V5
What happens in search if webull do not check the constraint on V3 (node inconsistency)bull do not check constraint between V3 and V5 (arc
inconsistency)bull do not check constraints between V3mdashV4 V3mdashV5 and
V4mdashV5 (path inconsistency)
1 2V4
1 2 3
0 1 2
1 2V5
1 2 3V1 V2
V3ltV2
V3gt0
V3ltV4
V5ltV4
V3ltV5
V3ltV1
V3
Oct 5 2004 ConsistencyProperties amp Algorithms
6
Foundations of Constraint Processing Fall 2004
Consistency checkingGoal eliminate inconsistent combinations
Algorithms (for binary constraints)
bull Node consistencybull Arc consistency (AC-1 AC-2 AC-3 AC-7 AC-31 etc)bull Path consistency (PC-1 PC-2 DPC PPC etc)bull Constraints of arbitrary arity
ndash Waltz algorithm ancestor of ACsndash Generalized arc-consistency (Dechter Section 351 )ndash Relational m-consistency (Dechter Section 81)
Oct 5 2004 ConsistencyProperties amp Algorithms
7
Foundations of Constraint Processing Fall 2004
Overview of Recommended Reading
bull Davis AIJ77
ndash focuses on the Waltz algorithm ndash studies its performance and quiescence for given
bull constraint types (order relations bounded diff algebraic etc)bull domains types (continuous or finite)
bull Mackworth AIJ77ndash presents NC AC-1 AC-2 PC-1 PC-2
bull Mackworth and Freuder AIJ85
ndash studies their complexity Mackworth and Freuder concentrate on finite domains More people work on finite domains than on continuous ones Continuous domains can be quite tough
Oct 5 2004 ConsistencyProperties amp Algorithms
8
Foundations of Constraint Processing Fall 2004
Constraint Propagation with Interval Labels Ernest Davis
bull `Old paper (1987) terminology slightly differentbull Interval labels continuous domains
ndash Section 8 sign labels (discrete)bull Concerned with Waltz algorithm (eg
quiescence completeness)bull Constraint types vs domain types (Table 3)bull Addresses applications from reasoning
ndash about Physical Systems (circuit SPAM)ndash about time relations (TMM)
Advice read Section 3 more if you wish
Oct 5 2004 ConsistencyProperties amp Algorithms
9
Foundations of Constraint Processing Fall 2004
Waltz algorithm for label inference
bull Refine(C(Vi Vk Vm Vn) Vi) - finds a new label for Vi consistent with C
bull Revise(C (Vi Vk Vm Vn)) refines the domains of Vi Vk Vm Vn by iterating over these variables until quiescence (no domains is further refined)
bull Waltz Algorithm revises all constraints by iterating over each constraint connected to a variable whose domain has been revised
bull Waltz Algorithm ancestor of arc-consistency
C
Vk Vm Vn
Vi
Oct 5 2004 ConsistencyProperties amp Algorithms
10
Foundations of Constraint Processing Fall 2004
WARNINGrarr Completeness Running the Waltz Algorithm does not solve
the problem
A=2 and B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis provides a classification of problems in terms of completeness and quiescence of the Waltz algorithm
f(constraint types domain types) see Table 3
1 2 3 2 3 4
BA BA
2 3 2 3
Oct 5 2004 ConsistencyProperties amp Algorithms
11
Foundations of Constraint Processing Fall 2004
Importance of this paper Establishes that constraints of bounded differences
(temporal reasoning) can be efficiently solved by the Waltz algorithm (O(n3) n number of variables)
Early paper that attracts attention on the difficulty of label propagation in interval labels (continuous domains)
This work has been continued by Faltings (AIJ 92) who studied the early-quiescence problem of the Waltz algorithm
Oct 5 2004 ConsistencyProperties amp Algorithms
12
Foundations of Constraint Processing Fall 2004
Consistency checking algorithms
Examining finite binary CSPs and their complexity
Notation
Given variables i j k with values x y z the predicate
Pijk(x y z) is true iff the 3-tuple x y z Cijk
Node consistency checking Pi(x)
Arc consistency checking Pij(xy)
Path consistency bit-matrix manipulation
Oct 5 2004 ConsistencyProperties amp Algorithms
13
Foundations of Constraint Processing Fall 2004
Worst-case complexity as a function of the input parameters
Upper bound f(n) is O(g(n)) means that f(n) cg(n)
f grows as g or slower
Lower boundf(n) is (h(n)) means that f(n) ch(n)
f grows as g or faster
Input parameters for a CSP
n = number of variables
a = (max) size of a domain
dk = degree of Vk ( n-1)
e = number of edges (or constraints) [(n-1) n(n-1)2]
Worst-case asymptotic complexitytime
space
Oct 5 2004 ConsistencyProperties amp Algorithms
14
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node amp arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
15
Foundations of Constraint Processing Fall 2004
Node consistency (NC)Procedure NC(i)
Di Di x | Pi(x)
Begin
for i 1 until n do NC(i)
end
Oct 5 2004 ConsistencyProperties amp Algorithms
16
Foundations of Constraint Processing Fall 2004
Complexity of NC Procedure of NC(i)
Di Di x | Pi(x) Begin for i 1 until n do NC(i)end
bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)
Oct 5 2004 ConsistencyProperties amp Algorithms
17
Foundations of Constraint Processing Fall 2004
Arc-consistency Adapted from Dechter
Definition Given a constraint graph G
bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj
bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi
bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
18
Foundations of Constraint Processing Fall 2004
Procedure ReviseRevises the domains of a variable i
Procedure Revise(ij) Begin
DELETE false for each x Di do
if there is no y Dj such that Pij(x y) then begin
delete x from Di
DELETE true end return DELETE end
bull Revise is directionalbull What is the complexity of Revise
a2
Oct 5 2004 ConsistencyProperties amp Algorithms
19
Foundations of Constraint Processing Fall 2004
Revise example R Dechter
Apply the Revise procedure to the following example
5 6 15 1 2 10
YX+Y = 10X
Oct 5 2004 ConsistencyProperties amp Algorithms
20
Foundations of Constraint Processing Fall 2004
Solution DVi DVi Vi (CViVj DVi)
This is actually equivalent to DVi Vi (CViVj DVi)
Effect of Revise Adapted from Rina Dechter
Question Given
bull two variables Vi and Vj
bull their domains DVi and DVj and
bull the constraint CViVj
write the effect of the Revise procedure as a sequence of operations in relational algebra
Hintbull Think about the domain DVi as a unary constraint CVi and
bull consider the composition of this unary constraint and the binary one
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
21
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end
bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)
Oct 5 2004 ConsistencyProperties amp Algorithms
22
Foundations of Constraint Processing Fall 2004
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
23
Foundations of Constraint Processing Fall 2004
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Oct 5 2004 ConsistencyProperties amp Algorithms
24
Foundations of Constraint Processing Fall 2004
NC amp AC Example courtesy of M Fromherz
In the temporal problem below
1 AC propagates bound
2 Unary constraint xgt3 is imposed
3 AC propagates bounds again
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
Oct 5 2004 ConsistencyProperties amp Algorithms
25
Foundations of Constraint Processing Fall 2004
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Oct 5 2004 ConsistencyProperties amp Algorithms
26
Foundations of Constraint Processing Fall 2004
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Oct 5 2004 ConsistencyProperties amp Algorithms
27
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end
Oct 5 2004 ConsistencyProperties amp Algorithms
28
Foundations of Constraint Processing Fall 2004
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (i j) arcs(G) i j
4 While Q is not empty do
5 begin
6 select and delete any arc (k m) from Q
7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
4
Foundations of Constraint Processing Fall 2004
Consistency checkingMotivation bull CSP are solved with search (conditioning)bull Search performance is affected by
- problem size- amount of backtracking
bull Backtracking may yield
ThrashingExploring non-promising sub-trees and rediscovering the same inconsistencies over and over again
Malady of backtrack search
Oct 5 2004 ConsistencyProperties amp Algorithms
5
Foundations of Constraint Processing Fall 2004
(Some) causes of thrashingSearch order V1 V2 V3 V4 V5
What happens in search if webull do not check the constraint on V3 (node inconsistency)bull do not check constraint between V3 and V5 (arc
inconsistency)bull do not check constraints between V3mdashV4 V3mdashV5 and
V4mdashV5 (path inconsistency)
1 2V4
1 2 3
0 1 2
1 2V5
1 2 3V1 V2
V3ltV2
V3gt0
V3ltV4
V5ltV4
V3ltV5
V3ltV1
V3
Oct 5 2004 ConsistencyProperties amp Algorithms
6
Foundations of Constraint Processing Fall 2004
Consistency checkingGoal eliminate inconsistent combinations
Algorithms (for binary constraints)
bull Node consistencybull Arc consistency (AC-1 AC-2 AC-3 AC-7 AC-31 etc)bull Path consistency (PC-1 PC-2 DPC PPC etc)bull Constraints of arbitrary arity
ndash Waltz algorithm ancestor of ACsndash Generalized arc-consistency (Dechter Section 351 )ndash Relational m-consistency (Dechter Section 81)
Oct 5 2004 ConsistencyProperties amp Algorithms
7
Foundations of Constraint Processing Fall 2004
Overview of Recommended Reading
bull Davis AIJ77
ndash focuses on the Waltz algorithm ndash studies its performance and quiescence for given
bull constraint types (order relations bounded diff algebraic etc)bull domains types (continuous or finite)
bull Mackworth AIJ77ndash presents NC AC-1 AC-2 PC-1 PC-2
bull Mackworth and Freuder AIJ85
ndash studies their complexity Mackworth and Freuder concentrate on finite domains More people work on finite domains than on continuous ones Continuous domains can be quite tough
Oct 5 2004 ConsistencyProperties amp Algorithms
8
Foundations of Constraint Processing Fall 2004
Constraint Propagation with Interval Labels Ernest Davis
bull `Old paper (1987) terminology slightly differentbull Interval labels continuous domains
ndash Section 8 sign labels (discrete)bull Concerned with Waltz algorithm (eg
quiescence completeness)bull Constraint types vs domain types (Table 3)bull Addresses applications from reasoning
ndash about Physical Systems (circuit SPAM)ndash about time relations (TMM)
Advice read Section 3 more if you wish
Oct 5 2004 ConsistencyProperties amp Algorithms
9
Foundations of Constraint Processing Fall 2004
Waltz algorithm for label inference
bull Refine(C(Vi Vk Vm Vn) Vi) - finds a new label for Vi consistent with C
bull Revise(C (Vi Vk Vm Vn)) refines the domains of Vi Vk Vm Vn by iterating over these variables until quiescence (no domains is further refined)
bull Waltz Algorithm revises all constraints by iterating over each constraint connected to a variable whose domain has been revised
bull Waltz Algorithm ancestor of arc-consistency
C
Vk Vm Vn
Vi
Oct 5 2004 ConsistencyProperties amp Algorithms
10
Foundations of Constraint Processing Fall 2004
WARNINGrarr Completeness Running the Waltz Algorithm does not solve
the problem
A=2 and B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis provides a classification of problems in terms of completeness and quiescence of the Waltz algorithm
f(constraint types domain types) see Table 3
1 2 3 2 3 4
BA BA
2 3 2 3
Oct 5 2004 ConsistencyProperties amp Algorithms
11
Foundations of Constraint Processing Fall 2004
Importance of this paper Establishes that constraints of bounded differences
(temporal reasoning) can be efficiently solved by the Waltz algorithm (O(n3) n number of variables)
Early paper that attracts attention on the difficulty of label propagation in interval labels (continuous domains)
This work has been continued by Faltings (AIJ 92) who studied the early-quiescence problem of the Waltz algorithm
Oct 5 2004 ConsistencyProperties amp Algorithms
12
Foundations of Constraint Processing Fall 2004
Consistency checking algorithms
Examining finite binary CSPs and their complexity
Notation
Given variables i j k with values x y z the predicate
Pijk(x y z) is true iff the 3-tuple x y z Cijk
Node consistency checking Pi(x)
Arc consistency checking Pij(xy)
Path consistency bit-matrix manipulation
Oct 5 2004 ConsistencyProperties amp Algorithms
13
Foundations of Constraint Processing Fall 2004
Worst-case complexity as a function of the input parameters
Upper bound f(n) is O(g(n)) means that f(n) cg(n)
f grows as g or slower
Lower boundf(n) is (h(n)) means that f(n) ch(n)
f grows as g or faster
Input parameters for a CSP
n = number of variables
a = (max) size of a domain
dk = degree of Vk ( n-1)
e = number of edges (or constraints) [(n-1) n(n-1)2]
Worst-case asymptotic complexitytime
space
Oct 5 2004 ConsistencyProperties amp Algorithms
14
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node amp arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
15
Foundations of Constraint Processing Fall 2004
Node consistency (NC)Procedure NC(i)
Di Di x | Pi(x)
Begin
for i 1 until n do NC(i)
end
Oct 5 2004 ConsistencyProperties amp Algorithms
16
Foundations of Constraint Processing Fall 2004
Complexity of NC Procedure of NC(i)
Di Di x | Pi(x) Begin for i 1 until n do NC(i)end
bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)
Oct 5 2004 ConsistencyProperties amp Algorithms
17
Foundations of Constraint Processing Fall 2004
Arc-consistency Adapted from Dechter
Definition Given a constraint graph G
bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj
bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi
bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
18
Foundations of Constraint Processing Fall 2004
Procedure ReviseRevises the domains of a variable i
Procedure Revise(ij) Begin
DELETE false for each x Di do
if there is no y Dj such that Pij(x y) then begin
delete x from Di
DELETE true end return DELETE end
bull Revise is directionalbull What is the complexity of Revise
a2
Oct 5 2004 ConsistencyProperties amp Algorithms
19
Foundations of Constraint Processing Fall 2004
Revise example R Dechter
Apply the Revise procedure to the following example
5 6 15 1 2 10
YX+Y = 10X
Oct 5 2004 ConsistencyProperties amp Algorithms
20
Foundations of Constraint Processing Fall 2004
Solution DVi DVi Vi (CViVj DVi)
This is actually equivalent to DVi Vi (CViVj DVi)
Effect of Revise Adapted from Rina Dechter
Question Given
bull two variables Vi and Vj
bull their domains DVi and DVj and
bull the constraint CViVj
write the effect of the Revise procedure as a sequence of operations in relational algebra
Hintbull Think about the domain DVi as a unary constraint CVi and
bull consider the composition of this unary constraint and the binary one
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
21
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end
bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)
Oct 5 2004 ConsistencyProperties amp Algorithms
22
Foundations of Constraint Processing Fall 2004
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
23
Foundations of Constraint Processing Fall 2004
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Oct 5 2004 ConsistencyProperties amp Algorithms
24
Foundations of Constraint Processing Fall 2004
NC amp AC Example courtesy of M Fromherz
In the temporal problem below
1 AC propagates bound
2 Unary constraint xgt3 is imposed
3 AC propagates bounds again
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
Oct 5 2004 ConsistencyProperties amp Algorithms
25
Foundations of Constraint Processing Fall 2004
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Oct 5 2004 ConsistencyProperties amp Algorithms
26
Foundations of Constraint Processing Fall 2004
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Oct 5 2004 ConsistencyProperties amp Algorithms
27
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end
Oct 5 2004 ConsistencyProperties amp Algorithms
28
Foundations of Constraint Processing Fall 2004
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (i j) arcs(G) i j
4 While Q is not empty do
5 begin
6 select and delete any arc (k m) from Q
7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
5
Foundations of Constraint Processing Fall 2004
(Some) causes of thrashingSearch order V1 V2 V3 V4 V5
What happens in search if webull do not check the constraint on V3 (node inconsistency)bull do not check constraint between V3 and V5 (arc
inconsistency)bull do not check constraints between V3mdashV4 V3mdashV5 and
V4mdashV5 (path inconsistency)
1 2V4
1 2 3
0 1 2
1 2V5
1 2 3V1 V2
V3ltV2
V3gt0
V3ltV4
V5ltV4
V3ltV5
V3ltV1
V3
Oct 5 2004 ConsistencyProperties amp Algorithms
6
Foundations of Constraint Processing Fall 2004
Consistency checkingGoal eliminate inconsistent combinations
Algorithms (for binary constraints)
bull Node consistencybull Arc consistency (AC-1 AC-2 AC-3 AC-7 AC-31 etc)bull Path consistency (PC-1 PC-2 DPC PPC etc)bull Constraints of arbitrary arity
ndash Waltz algorithm ancestor of ACsndash Generalized arc-consistency (Dechter Section 351 )ndash Relational m-consistency (Dechter Section 81)
Oct 5 2004 ConsistencyProperties amp Algorithms
7
Foundations of Constraint Processing Fall 2004
Overview of Recommended Reading
bull Davis AIJ77
ndash focuses on the Waltz algorithm ndash studies its performance and quiescence for given
bull constraint types (order relations bounded diff algebraic etc)bull domains types (continuous or finite)
bull Mackworth AIJ77ndash presents NC AC-1 AC-2 PC-1 PC-2
bull Mackworth and Freuder AIJ85
ndash studies their complexity Mackworth and Freuder concentrate on finite domains More people work on finite domains than on continuous ones Continuous domains can be quite tough
Oct 5 2004 ConsistencyProperties amp Algorithms
8
Foundations of Constraint Processing Fall 2004
Constraint Propagation with Interval Labels Ernest Davis
bull `Old paper (1987) terminology slightly differentbull Interval labels continuous domains
ndash Section 8 sign labels (discrete)bull Concerned with Waltz algorithm (eg
quiescence completeness)bull Constraint types vs domain types (Table 3)bull Addresses applications from reasoning
ndash about Physical Systems (circuit SPAM)ndash about time relations (TMM)
Advice read Section 3 more if you wish
Oct 5 2004 ConsistencyProperties amp Algorithms
9
Foundations of Constraint Processing Fall 2004
Waltz algorithm for label inference
bull Refine(C(Vi Vk Vm Vn) Vi) - finds a new label for Vi consistent with C
bull Revise(C (Vi Vk Vm Vn)) refines the domains of Vi Vk Vm Vn by iterating over these variables until quiescence (no domains is further refined)
bull Waltz Algorithm revises all constraints by iterating over each constraint connected to a variable whose domain has been revised
bull Waltz Algorithm ancestor of arc-consistency
C
Vk Vm Vn
Vi
Oct 5 2004 ConsistencyProperties amp Algorithms
10
Foundations of Constraint Processing Fall 2004
WARNINGrarr Completeness Running the Waltz Algorithm does not solve
the problem
A=2 and B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis provides a classification of problems in terms of completeness and quiescence of the Waltz algorithm
f(constraint types domain types) see Table 3
1 2 3 2 3 4
BA BA
2 3 2 3
Oct 5 2004 ConsistencyProperties amp Algorithms
11
Foundations of Constraint Processing Fall 2004
Importance of this paper Establishes that constraints of bounded differences
(temporal reasoning) can be efficiently solved by the Waltz algorithm (O(n3) n number of variables)
Early paper that attracts attention on the difficulty of label propagation in interval labels (continuous domains)
This work has been continued by Faltings (AIJ 92) who studied the early-quiescence problem of the Waltz algorithm
Oct 5 2004 ConsistencyProperties amp Algorithms
12
Foundations of Constraint Processing Fall 2004
Consistency checking algorithms
Examining finite binary CSPs and their complexity
Notation
Given variables i j k with values x y z the predicate
Pijk(x y z) is true iff the 3-tuple x y z Cijk
Node consistency checking Pi(x)
Arc consistency checking Pij(xy)
Path consistency bit-matrix manipulation
Oct 5 2004 ConsistencyProperties amp Algorithms
13
Foundations of Constraint Processing Fall 2004
Worst-case complexity as a function of the input parameters
Upper bound f(n) is O(g(n)) means that f(n) cg(n)
f grows as g or slower
Lower boundf(n) is (h(n)) means that f(n) ch(n)
f grows as g or faster
Input parameters for a CSP
n = number of variables
a = (max) size of a domain
dk = degree of Vk ( n-1)
e = number of edges (or constraints) [(n-1) n(n-1)2]
Worst-case asymptotic complexitytime
space
Oct 5 2004 ConsistencyProperties amp Algorithms
14
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node amp arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
15
Foundations of Constraint Processing Fall 2004
Node consistency (NC)Procedure NC(i)
Di Di x | Pi(x)
Begin
for i 1 until n do NC(i)
end
Oct 5 2004 ConsistencyProperties amp Algorithms
16
Foundations of Constraint Processing Fall 2004
Complexity of NC Procedure of NC(i)
Di Di x | Pi(x) Begin for i 1 until n do NC(i)end
bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)
Oct 5 2004 ConsistencyProperties amp Algorithms
17
Foundations of Constraint Processing Fall 2004
Arc-consistency Adapted from Dechter
Definition Given a constraint graph G
bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj
bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi
bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
18
Foundations of Constraint Processing Fall 2004
Procedure ReviseRevises the domains of a variable i
Procedure Revise(ij) Begin
DELETE false for each x Di do
if there is no y Dj such that Pij(x y) then begin
delete x from Di
DELETE true end return DELETE end
bull Revise is directionalbull What is the complexity of Revise
a2
Oct 5 2004 ConsistencyProperties amp Algorithms
19
Foundations of Constraint Processing Fall 2004
Revise example R Dechter
Apply the Revise procedure to the following example
5 6 15 1 2 10
YX+Y = 10X
Oct 5 2004 ConsistencyProperties amp Algorithms
20
Foundations of Constraint Processing Fall 2004
Solution DVi DVi Vi (CViVj DVi)
This is actually equivalent to DVi Vi (CViVj DVi)
Effect of Revise Adapted from Rina Dechter
Question Given
bull two variables Vi and Vj
bull their domains DVi and DVj and
bull the constraint CViVj
write the effect of the Revise procedure as a sequence of operations in relational algebra
Hintbull Think about the domain DVi as a unary constraint CVi and
bull consider the composition of this unary constraint and the binary one
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
21
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end
bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)
Oct 5 2004 ConsistencyProperties amp Algorithms
22
Foundations of Constraint Processing Fall 2004
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
23
Foundations of Constraint Processing Fall 2004
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Oct 5 2004 ConsistencyProperties amp Algorithms
24
Foundations of Constraint Processing Fall 2004
NC amp AC Example courtesy of M Fromherz
In the temporal problem below
1 AC propagates bound
2 Unary constraint xgt3 is imposed
3 AC propagates bounds again
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
Oct 5 2004 ConsistencyProperties amp Algorithms
25
Foundations of Constraint Processing Fall 2004
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Oct 5 2004 ConsistencyProperties amp Algorithms
26
Foundations of Constraint Processing Fall 2004
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Oct 5 2004 ConsistencyProperties amp Algorithms
27
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end
Oct 5 2004 ConsistencyProperties amp Algorithms
28
Foundations of Constraint Processing Fall 2004
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (i j) arcs(G) i j
4 While Q is not empty do
5 begin
6 select and delete any arc (k m) from Q
7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
6
Foundations of Constraint Processing Fall 2004
Consistency checkingGoal eliminate inconsistent combinations
Algorithms (for binary constraints)
bull Node consistencybull Arc consistency (AC-1 AC-2 AC-3 AC-7 AC-31 etc)bull Path consistency (PC-1 PC-2 DPC PPC etc)bull Constraints of arbitrary arity
ndash Waltz algorithm ancestor of ACsndash Generalized arc-consistency (Dechter Section 351 )ndash Relational m-consistency (Dechter Section 81)
Oct 5 2004 ConsistencyProperties amp Algorithms
7
Foundations of Constraint Processing Fall 2004
Overview of Recommended Reading
bull Davis AIJ77
ndash focuses on the Waltz algorithm ndash studies its performance and quiescence for given
bull constraint types (order relations bounded diff algebraic etc)bull domains types (continuous or finite)
bull Mackworth AIJ77ndash presents NC AC-1 AC-2 PC-1 PC-2
bull Mackworth and Freuder AIJ85
ndash studies their complexity Mackworth and Freuder concentrate on finite domains More people work on finite domains than on continuous ones Continuous domains can be quite tough
Oct 5 2004 ConsistencyProperties amp Algorithms
8
Foundations of Constraint Processing Fall 2004
Constraint Propagation with Interval Labels Ernest Davis
bull `Old paper (1987) terminology slightly differentbull Interval labels continuous domains
ndash Section 8 sign labels (discrete)bull Concerned with Waltz algorithm (eg
quiescence completeness)bull Constraint types vs domain types (Table 3)bull Addresses applications from reasoning
ndash about Physical Systems (circuit SPAM)ndash about time relations (TMM)
Advice read Section 3 more if you wish
Oct 5 2004 ConsistencyProperties amp Algorithms
9
Foundations of Constraint Processing Fall 2004
Waltz algorithm for label inference
bull Refine(C(Vi Vk Vm Vn) Vi) - finds a new label for Vi consistent with C
bull Revise(C (Vi Vk Vm Vn)) refines the domains of Vi Vk Vm Vn by iterating over these variables until quiescence (no domains is further refined)
bull Waltz Algorithm revises all constraints by iterating over each constraint connected to a variable whose domain has been revised
bull Waltz Algorithm ancestor of arc-consistency
C
Vk Vm Vn
Vi
Oct 5 2004 ConsistencyProperties amp Algorithms
10
Foundations of Constraint Processing Fall 2004
WARNINGrarr Completeness Running the Waltz Algorithm does not solve
the problem
A=2 and B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis provides a classification of problems in terms of completeness and quiescence of the Waltz algorithm
f(constraint types domain types) see Table 3
1 2 3 2 3 4
BA BA
2 3 2 3
Oct 5 2004 ConsistencyProperties amp Algorithms
11
Foundations of Constraint Processing Fall 2004
Importance of this paper Establishes that constraints of bounded differences
(temporal reasoning) can be efficiently solved by the Waltz algorithm (O(n3) n number of variables)
Early paper that attracts attention on the difficulty of label propagation in interval labels (continuous domains)
This work has been continued by Faltings (AIJ 92) who studied the early-quiescence problem of the Waltz algorithm
Oct 5 2004 ConsistencyProperties amp Algorithms
12
Foundations of Constraint Processing Fall 2004
Consistency checking algorithms
Examining finite binary CSPs and their complexity
Notation
Given variables i j k with values x y z the predicate
Pijk(x y z) is true iff the 3-tuple x y z Cijk
Node consistency checking Pi(x)
Arc consistency checking Pij(xy)
Path consistency bit-matrix manipulation
Oct 5 2004 ConsistencyProperties amp Algorithms
13
Foundations of Constraint Processing Fall 2004
Worst-case complexity as a function of the input parameters
Upper bound f(n) is O(g(n)) means that f(n) cg(n)
f grows as g or slower
Lower boundf(n) is (h(n)) means that f(n) ch(n)
f grows as g or faster
Input parameters for a CSP
n = number of variables
a = (max) size of a domain
dk = degree of Vk ( n-1)
e = number of edges (or constraints) [(n-1) n(n-1)2]
Worst-case asymptotic complexitytime
space
Oct 5 2004 ConsistencyProperties amp Algorithms
14
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node amp arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
15
Foundations of Constraint Processing Fall 2004
Node consistency (NC)Procedure NC(i)
Di Di x | Pi(x)
Begin
for i 1 until n do NC(i)
end
Oct 5 2004 ConsistencyProperties amp Algorithms
16
Foundations of Constraint Processing Fall 2004
Complexity of NC Procedure of NC(i)
Di Di x | Pi(x) Begin for i 1 until n do NC(i)end
bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)
Oct 5 2004 ConsistencyProperties amp Algorithms
17
Foundations of Constraint Processing Fall 2004
Arc-consistency Adapted from Dechter
Definition Given a constraint graph G
bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj
bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi
bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
18
Foundations of Constraint Processing Fall 2004
Procedure ReviseRevises the domains of a variable i
Procedure Revise(ij) Begin
DELETE false for each x Di do
if there is no y Dj such that Pij(x y) then begin
delete x from Di
DELETE true end return DELETE end
bull Revise is directionalbull What is the complexity of Revise
a2
Oct 5 2004 ConsistencyProperties amp Algorithms
19
Foundations of Constraint Processing Fall 2004
Revise example R Dechter
Apply the Revise procedure to the following example
5 6 15 1 2 10
YX+Y = 10X
Oct 5 2004 ConsistencyProperties amp Algorithms
20
Foundations of Constraint Processing Fall 2004
Solution DVi DVi Vi (CViVj DVi)
This is actually equivalent to DVi Vi (CViVj DVi)
Effect of Revise Adapted from Rina Dechter
Question Given
bull two variables Vi and Vj
bull their domains DVi and DVj and
bull the constraint CViVj
write the effect of the Revise procedure as a sequence of operations in relational algebra
Hintbull Think about the domain DVi as a unary constraint CVi and
bull consider the composition of this unary constraint and the binary one
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
21
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end
bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)
Oct 5 2004 ConsistencyProperties amp Algorithms
22
Foundations of Constraint Processing Fall 2004
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
23
Foundations of Constraint Processing Fall 2004
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Oct 5 2004 ConsistencyProperties amp Algorithms
24
Foundations of Constraint Processing Fall 2004
NC amp AC Example courtesy of M Fromherz
In the temporal problem below
1 AC propagates bound
2 Unary constraint xgt3 is imposed
3 AC propagates bounds again
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
Oct 5 2004 ConsistencyProperties amp Algorithms
25
Foundations of Constraint Processing Fall 2004
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Oct 5 2004 ConsistencyProperties amp Algorithms
26
Foundations of Constraint Processing Fall 2004
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Oct 5 2004 ConsistencyProperties amp Algorithms
27
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end
Oct 5 2004 ConsistencyProperties amp Algorithms
28
Foundations of Constraint Processing Fall 2004
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (i j) arcs(G) i j
4 While Q is not empty do
5 begin
6 select and delete any arc (k m) from Q
7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
7
Foundations of Constraint Processing Fall 2004
Overview of Recommended Reading
bull Davis AIJ77
ndash focuses on the Waltz algorithm ndash studies its performance and quiescence for given
bull constraint types (order relations bounded diff algebraic etc)bull domains types (continuous or finite)
bull Mackworth AIJ77ndash presents NC AC-1 AC-2 PC-1 PC-2
bull Mackworth and Freuder AIJ85
ndash studies their complexity Mackworth and Freuder concentrate on finite domains More people work on finite domains than on continuous ones Continuous domains can be quite tough
Oct 5 2004 ConsistencyProperties amp Algorithms
8
Foundations of Constraint Processing Fall 2004
Constraint Propagation with Interval Labels Ernest Davis
bull `Old paper (1987) terminology slightly differentbull Interval labels continuous domains
ndash Section 8 sign labels (discrete)bull Concerned with Waltz algorithm (eg
quiescence completeness)bull Constraint types vs domain types (Table 3)bull Addresses applications from reasoning
ndash about Physical Systems (circuit SPAM)ndash about time relations (TMM)
Advice read Section 3 more if you wish
Oct 5 2004 ConsistencyProperties amp Algorithms
9
Foundations of Constraint Processing Fall 2004
Waltz algorithm for label inference
bull Refine(C(Vi Vk Vm Vn) Vi) - finds a new label for Vi consistent with C
bull Revise(C (Vi Vk Vm Vn)) refines the domains of Vi Vk Vm Vn by iterating over these variables until quiescence (no domains is further refined)
bull Waltz Algorithm revises all constraints by iterating over each constraint connected to a variable whose domain has been revised
bull Waltz Algorithm ancestor of arc-consistency
C
Vk Vm Vn
Vi
Oct 5 2004 ConsistencyProperties amp Algorithms
10
Foundations of Constraint Processing Fall 2004
WARNINGrarr Completeness Running the Waltz Algorithm does not solve
the problem
A=2 and B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis provides a classification of problems in terms of completeness and quiescence of the Waltz algorithm
f(constraint types domain types) see Table 3
1 2 3 2 3 4
BA BA
2 3 2 3
Oct 5 2004 ConsistencyProperties amp Algorithms
11
Foundations of Constraint Processing Fall 2004
Importance of this paper Establishes that constraints of bounded differences
(temporal reasoning) can be efficiently solved by the Waltz algorithm (O(n3) n number of variables)
Early paper that attracts attention on the difficulty of label propagation in interval labels (continuous domains)
This work has been continued by Faltings (AIJ 92) who studied the early-quiescence problem of the Waltz algorithm
Oct 5 2004 ConsistencyProperties amp Algorithms
12
Foundations of Constraint Processing Fall 2004
Consistency checking algorithms
Examining finite binary CSPs and their complexity
Notation
Given variables i j k with values x y z the predicate
Pijk(x y z) is true iff the 3-tuple x y z Cijk
Node consistency checking Pi(x)
Arc consistency checking Pij(xy)
Path consistency bit-matrix manipulation
Oct 5 2004 ConsistencyProperties amp Algorithms
13
Foundations of Constraint Processing Fall 2004
Worst-case complexity as a function of the input parameters
Upper bound f(n) is O(g(n)) means that f(n) cg(n)
f grows as g or slower
Lower boundf(n) is (h(n)) means that f(n) ch(n)
f grows as g or faster
Input parameters for a CSP
n = number of variables
a = (max) size of a domain
dk = degree of Vk ( n-1)
e = number of edges (or constraints) [(n-1) n(n-1)2]
Worst-case asymptotic complexitytime
space
Oct 5 2004 ConsistencyProperties amp Algorithms
14
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node amp arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
15
Foundations of Constraint Processing Fall 2004
Node consistency (NC)Procedure NC(i)
Di Di x | Pi(x)
Begin
for i 1 until n do NC(i)
end
Oct 5 2004 ConsistencyProperties amp Algorithms
16
Foundations of Constraint Processing Fall 2004
Complexity of NC Procedure of NC(i)
Di Di x | Pi(x) Begin for i 1 until n do NC(i)end
bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)
Oct 5 2004 ConsistencyProperties amp Algorithms
17
Foundations of Constraint Processing Fall 2004
Arc-consistency Adapted from Dechter
Definition Given a constraint graph G
bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj
bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi
bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
18
Foundations of Constraint Processing Fall 2004
Procedure ReviseRevises the domains of a variable i
Procedure Revise(ij) Begin
DELETE false for each x Di do
if there is no y Dj such that Pij(x y) then begin
delete x from Di
DELETE true end return DELETE end
bull Revise is directionalbull What is the complexity of Revise
a2
Oct 5 2004 ConsistencyProperties amp Algorithms
19
Foundations of Constraint Processing Fall 2004
Revise example R Dechter
Apply the Revise procedure to the following example
5 6 15 1 2 10
YX+Y = 10X
Oct 5 2004 ConsistencyProperties amp Algorithms
20
Foundations of Constraint Processing Fall 2004
Solution DVi DVi Vi (CViVj DVi)
This is actually equivalent to DVi Vi (CViVj DVi)
Effect of Revise Adapted from Rina Dechter
Question Given
bull two variables Vi and Vj
bull their domains DVi and DVj and
bull the constraint CViVj
write the effect of the Revise procedure as a sequence of operations in relational algebra
Hintbull Think about the domain DVi as a unary constraint CVi and
bull consider the composition of this unary constraint and the binary one
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
21
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end
bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)
Oct 5 2004 ConsistencyProperties amp Algorithms
22
Foundations of Constraint Processing Fall 2004
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
23
Foundations of Constraint Processing Fall 2004
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Oct 5 2004 ConsistencyProperties amp Algorithms
24
Foundations of Constraint Processing Fall 2004
NC amp AC Example courtesy of M Fromherz
In the temporal problem below
1 AC propagates bound
2 Unary constraint xgt3 is imposed
3 AC propagates bounds again
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
Oct 5 2004 ConsistencyProperties amp Algorithms
25
Foundations of Constraint Processing Fall 2004
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Oct 5 2004 ConsistencyProperties amp Algorithms
26
Foundations of Constraint Processing Fall 2004
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Oct 5 2004 ConsistencyProperties amp Algorithms
27
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end
Oct 5 2004 ConsistencyProperties amp Algorithms
28
Foundations of Constraint Processing Fall 2004
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (i j) arcs(G) i j
4 While Q is not empty do
5 begin
6 select and delete any arc (k m) from Q
7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
8
Foundations of Constraint Processing Fall 2004
Constraint Propagation with Interval Labels Ernest Davis
bull `Old paper (1987) terminology slightly differentbull Interval labels continuous domains
ndash Section 8 sign labels (discrete)bull Concerned with Waltz algorithm (eg
quiescence completeness)bull Constraint types vs domain types (Table 3)bull Addresses applications from reasoning
ndash about Physical Systems (circuit SPAM)ndash about time relations (TMM)
Advice read Section 3 more if you wish
Oct 5 2004 ConsistencyProperties amp Algorithms
9
Foundations of Constraint Processing Fall 2004
Waltz algorithm for label inference
bull Refine(C(Vi Vk Vm Vn) Vi) - finds a new label for Vi consistent with C
bull Revise(C (Vi Vk Vm Vn)) refines the domains of Vi Vk Vm Vn by iterating over these variables until quiescence (no domains is further refined)
bull Waltz Algorithm revises all constraints by iterating over each constraint connected to a variable whose domain has been revised
bull Waltz Algorithm ancestor of arc-consistency
C
Vk Vm Vn
Vi
Oct 5 2004 ConsistencyProperties amp Algorithms
10
Foundations of Constraint Processing Fall 2004
WARNINGrarr Completeness Running the Waltz Algorithm does not solve
the problem
A=2 and B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis provides a classification of problems in terms of completeness and quiescence of the Waltz algorithm
f(constraint types domain types) see Table 3
1 2 3 2 3 4
BA BA
2 3 2 3
Oct 5 2004 ConsistencyProperties amp Algorithms
11
Foundations of Constraint Processing Fall 2004
Importance of this paper Establishes that constraints of bounded differences
(temporal reasoning) can be efficiently solved by the Waltz algorithm (O(n3) n number of variables)
Early paper that attracts attention on the difficulty of label propagation in interval labels (continuous domains)
This work has been continued by Faltings (AIJ 92) who studied the early-quiescence problem of the Waltz algorithm
Oct 5 2004 ConsistencyProperties amp Algorithms
12
Foundations of Constraint Processing Fall 2004
Consistency checking algorithms
Examining finite binary CSPs and their complexity
Notation
Given variables i j k with values x y z the predicate
Pijk(x y z) is true iff the 3-tuple x y z Cijk
Node consistency checking Pi(x)
Arc consistency checking Pij(xy)
Path consistency bit-matrix manipulation
Oct 5 2004 ConsistencyProperties amp Algorithms
13
Foundations of Constraint Processing Fall 2004
Worst-case complexity as a function of the input parameters
Upper bound f(n) is O(g(n)) means that f(n) cg(n)
f grows as g or slower
Lower boundf(n) is (h(n)) means that f(n) ch(n)
f grows as g or faster
Input parameters for a CSP
n = number of variables
a = (max) size of a domain
dk = degree of Vk ( n-1)
e = number of edges (or constraints) [(n-1) n(n-1)2]
Worst-case asymptotic complexitytime
space
Oct 5 2004 ConsistencyProperties amp Algorithms
14
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node amp arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
15
Foundations of Constraint Processing Fall 2004
Node consistency (NC)Procedure NC(i)
Di Di x | Pi(x)
Begin
for i 1 until n do NC(i)
end
Oct 5 2004 ConsistencyProperties amp Algorithms
16
Foundations of Constraint Processing Fall 2004
Complexity of NC Procedure of NC(i)
Di Di x | Pi(x) Begin for i 1 until n do NC(i)end
bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)
Oct 5 2004 ConsistencyProperties amp Algorithms
17
Foundations of Constraint Processing Fall 2004
Arc-consistency Adapted from Dechter
Definition Given a constraint graph G
bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj
bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi
bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
18
Foundations of Constraint Processing Fall 2004
Procedure ReviseRevises the domains of a variable i
Procedure Revise(ij) Begin
DELETE false for each x Di do
if there is no y Dj such that Pij(x y) then begin
delete x from Di
DELETE true end return DELETE end
bull Revise is directionalbull What is the complexity of Revise
a2
Oct 5 2004 ConsistencyProperties amp Algorithms
19
Foundations of Constraint Processing Fall 2004
Revise example R Dechter
Apply the Revise procedure to the following example
5 6 15 1 2 10
YX+Y = 10X
Oct 5 2004 ConsistencyProperties amp Algorithms
20
Foundations of Constraint Processing Fall 2004
Solution DVi DVi Vi (CViVj DVi)
This is actually equivalent to DVi Vi (CViVj DVi)
Effect of Revise Adapted from Rina Dechter
Question Given
bull two variables Vi and Vj
bull their domains DVi and DVj and
bull the constraint CViVj
write the effect of the Revise procedure as a sequence of operations in relational algebra
Hintbull Think about the domain DVi as a unary constraint CVi and
bull consider the composition of this unary constraint and the binary one
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
21
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end
bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)
Oct 5 2004 ConsistencyProperties amp Algorithms
22
Foundations of Constraint Processing Fall 2004
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
23
Foundations of Constraint Processing Fall 2004
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Oct 5 2004 ConsistencyProperties amp Algorithms
24
Foundations of Constraint Processing Fall 2004
NC amp AC Example courtesy of M Fromherz
In the temporal problem below
1 AC propagates bound
2 Unary constraint xgt3 is imposed
3 AC propagates bounds again
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
Oct 5 2004 ConsistencyProperties amp Algorithms
25
Foundations of Constraint Processing Fall 2004
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Oct 5 2004 ConsistencyProperties amp Algorithms
26
Foundations of Constraint Processing Fall 2004
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Oct 5 2004 ConsistencyProperties amp Algorithms
27
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end
Oct 5 2004 ConsistencyProperties amp Algorithms
28
Foundations of Constraint Processing Fall 2004
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (i j) arcs(G) i j
4 While Q is not empty do
5 begin
6 select and delete any arc (k m) from Q
7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
9
Foundations of Constraint Processing Fall 2004
Waltz algorithm for label inference
bull Refine(C(Vi Vk Vm Vn) Vi) - finds a new label for Vi consistent with C
bull Revise(C (Vi Vk Vm Vn)) refines the domains of Vi Vk Vm Vn by iterating over these variables until quiescence (no domains is further refined)
bull Waltz Algorithm revises all constraints by iterating over each constraint connected to a variable whose domain has been revised
bull Waltz Algorithm ancestor of arc-consistency
C
Vk Vm Vn
Vi
Oct 5 2004 ConsistencyProperties amp Algorithms
10
Foundations of Constraint Processing Fall 2004
WARNINGrarr Completeness Running the Waltz Algorithm does not solve
the problem
A=2 and B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis provides a classification of problems in terms of completeness and quiescence of the Waltz algorithm
f(constraint types domain types) see Table 3
1 2 3 2 3 4
BA BA
2 3 2 3
Oct 5 2004 ConsistencyProperties amp Algorithms
11
Foundations of Constraint Processing Fall 2004
Importance of this paper Establishes that constraints of bounded differences
(temporal reasoning) can be efficiently solved by the Waltz algorithm (O(n3) n number of variables)
Early paper that attracts attention on the difficulty of label propagation in interval labels (continuous domains)
This work has been continued by Faltings (AIJ 92) who studied the early-quiescence problem of the Waltz algorithm
Oct 5 2004 ConsistencyProperties amp Algorithms
12
Foundations of Constraint Processing Fall 2004
Consistency checking algorithms
Examining finite binary CSPs and their complexity
Notation
Given variables i j k with values x y z the predicate
Pijk(x y z) is true iff the 3-tuple x y z Cijk
Node consistency checking Pi(x)
Arc consistency checking Pij(xy)
Path consistency bit-matrix manipulation
Oct 5 2004 ConsistencyProperties amp Algorithms
13
Foundations of Constraint Processing Fall 2004
Worst-case complexity as a function of the input parameters
Upper bound f(n) is O(g(n)) means that f(n) cg(n)
f grows as g or slower
Lower boundf(n) is (h(n)) means that f(n) ch(n)
f grows as g or faster
Input parameters for a CSP
n = number of variables
a = (max) size of a domain
dk = degree of Vk ( n-1)
e = number of edges (or constraints) [(n-1) n(n-1)2]
Worst-case asymptotic complexitytime
space
Oct 5 2004 ConsistencyProperties amp Algorithms
14
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node amp arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
15
Foundations of Constraint Processing Fall 2004
Node consistency (NC)Procedure NC(i)
Di Di x | Pi(x)
Begin
for i 1 until n do NC(i)
end
Oct 5 2004 ConsistencyProperties amp Algorithms
16
Foundations of Constraint Processing Fall 2004
Complexity of NC Procedure of NC(i)
Di Di x | Pi(x) Begin for i 1 until n do NC(i)end
bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)
Oct 5 2004 ConsistencyProperties amp Algorithms
17
Foundations of Constraint Processing Fall 2004
Arc-consistency Adapted from Dechter
Definition Given a constraint graph G
bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj
bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi
bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
18
Foundations of Constraint Processing Fall 2004
Procedure ReviseRevises the domains of a variable i
Procedure Revise(ij) Begin
DELETE false for each x Di do
if there is no y Dj such that Pij(x y) then begin
delete x from Di
DELETE true end return DELETE end
bull Revise is directionalbull What is the complexity of Revise
a2
Oct 5 2004 ConsistencyProperties amp Algorithms
19
Foundations of Constraint Processing Fall 2004
Revise example R Dechter
Apply the Revise procedure to the following example
5 6 15 1 2 10
YX+Y = 10X
Oct 5 2004 ConsistencyProperties amp Algorithms
20
Foundations of Constraint Processing Fall 2004
Solution DVi DVi Vi (CViVj DVi)
This is actually equivalent to DVi Vi (CViVj DVi)
Effect of Revise Adapted from Rina Dechter
Question Given
bull two variables Vi and Vj
bull their domains DVi and DVj and
bull the constraint CViVj
write the effect of the Revise procedure as a sequence of operations in relational algebra
Hintbull Think about the domain DVi as a unary constraint CVi and
bull consider the composition of this unary constraint and the binary one
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
21
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end
bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)
Oct 5 2004 ConsistencyProperties amp Algorithms
22
Foundations of Constraint Processing Fall 2004
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
23
Foundations of Constraint Processing Fall 2004
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Oct 5 2004 ConsistencyProperties amp Algorithms
24
Foundations of Constraint Processing Fall 2004
NC amp AC Example courtesy of M Fromherz
In the temporal problem below
1 AC propagates bound
2 Unary constraint xgt3 is imposed
3 AC propagates bounds again
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
Oct 5 2004 ConsistencyProperties amp Algorithms
25
Foundations of Constraint Processing Fall 2004
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Oct 5 2004 ConsistencyProperties amp Algorithms
26
Foundations of Constraint Processing Fall 2004
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Oct 5 2004 ConsistencyProperties amp Algorithms
27
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end
Oct 5 2004 ConsistencyProperties amp Algorithms
28
Foundations of Constraint Processing Fall 2004
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (i j) arcs(G) i j
4 While Q is not empty do
5 begin
6 select and delete any arc (k m) from Q
7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
10
Foundations of Constraint Processing Fall 2004
WARNINGrarr Completeness Running the Waltz Algorithm does not solve
the problem
A=2 and B=3 is still not a solution
rarr Quiescence The Waltz algorithm may go into infinite loops even if problem is solvable
x [0 100] x = y
y [0 100] x = 2y
rarr Davis provides a classification of problems in terms of completeness and quiescence of the Waltz algorithm
f(constraint types domain types) see Table 3
1 2 3 2 3 4
BA BA
2 3 2 3
Oct 5 2004 ConsistencyProperties amp Algorithms
11
Foundations of Constraint Processing Fall 2004
Importance of this paper Establishes that constraints of bounded differences
(temporal reasoning) can be efficiently solved by the Waltz algorithm (O(n3) n number of variables)
Early paper that attracts attention on the difficulty of label propagation in interval labels (continuous domains)
This work has been continued by Faltings (AIJ 92) who studied the early-quiescence problem of the Waltz algorithm
Oct 5 2004 ConsistencyProperties amp Algorithms
12
Foundations of Constraint Processing Fall 2004
Consistency checking algorithms
Examining finite binary CSPs and their complexity
Notation
Given variables i j k with values x y z the predicate
Pijk(x y z) is true iff the 3-tuple x y z Cijk
Node consistency checking Pi(x)
Arc consistency checking Pij(xy)
Path consistency bit-matrix manipulation
Oct 5 2004 ConsistencyProperties amp Algorithms
13
Foundations of Constraint Processing Fall 2004
Worst-case complexity as a function of the input parameters
Upper bound f(n) is O(g(n)) means that f(n) cg(n)
f grows as g or slower
Lower boundf(n) is (h(n)) means that f(n) ch(n)
f grows as g or faster
Input parameters for a CSP
n = number of variables
a = (max) size of a domain
dk = degree of Vk ( n-1)
e = number of edges (or constraints) [(n-1) n(n-1)2]
Worst-case asymptotic complexitytime
space
Oct 5 2004 ConsistencyProperties amp Algorithms
14
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node amp arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
15
Foundations of Constraint Processing Fall 2004
Node consistency (NC)Procedure NC(i)
Di Di x | Pi(x)
Begin
for i 1 until n do NC(i)
end
Oct 5 2004 ConsistencyProperties amp Algorithms
16
Foundations of Constraint Processing Fall 2004
Complexity of NC Procedure of NC(i)
Di Di x | Pi(x) Begin for i 1 until n do NC(i)end
bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)
Oct 5 2004 ConsistencyProperties amp Algorithms
17
Foundations of Constraint Processing Fall 2004
Arc-consistency Adapted from Dechter
Definition Given a constraint graph G
bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj
bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi
bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
18
Foundations of Constraint Processing Fall 2004
Procedure ReviseRevises the domains of a variable i
Procedure Revise(ij) Begin
DELETE false for each x Di do
if there is no y Dj such that Pij(x y) then begin
delete x from Di
DELETE true end return DELETE end
bull Revise is directionalbull What is the complexity of Revise
a2
Oct 5 2004 ConsistencyProperties amp Algorithms
19
Foundations of Constraint Processing Fall 2004
Revise example R Dechter
Apply the Revise procedure to the following example
5 6 15 1 2 10
YX+Y = 10X
Oct 5 2004 ConsistencyProperties amp Algorithms
20
Foundations of Constraint Processing Fall 2004
Solution DVi DVi Vi (CViVj DVi)
This is actually equivalent to DVi Vi (CViVj DVi)
Effect of Revise Adapted from Rina Dechter
Question Given
bull two variables Vi and Vj
bull their domains DVi and DVj and
bull the constraint CViVj
write the effect of the Revise procedure as a sequence of operations in relational algebra
Hintbull Think about the domain DVi as a unary constraint CVi and
bull consider the composition of this unary constraint and the binary one
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
21
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end
bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)
Oct 5 2004 ConsistencyProperties amp Algorithms
22
Foundations of Constraint Processing Fall 2004
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
23
Foundations of Constraint Processing Fall 2004
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Oct 5 2004 ConsistencyProperties amp Algorithms
24
Foundations of Constraint Processing Fall 2004
NC amp AC Example courtesy of M Fromherz
In the temporal problem below
1 AC propagates bound
2 Unary constraint xgt3 is imposed
3 AC propagates bounds again
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
Oct 5 2004 ConsistencyProperties amp Algorithms
25
Foundations of Constraint Processing Fall 2004
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Oct 5 2004 ConsistencyProperties amp Algorithms
26
Foundations of Constraint Processing Fall 2004
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Oct 5 2004 ConsistencyProperties amp Algorithms
27
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end
Oct 5 2004 ConsistencyProperties amp Algorithms
28
Foundations of Constraint Processing Fall 2004
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (i j) arcs(G) i j
4 While Q is not empty do
5 begin
6 select and delete any arc (k m) from Q
7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
11
Foundations of Constraint Processing Fall 2004
Importance of this paper Establishes that constraints of bounded differences
(temporal reasoning) can be efficiently solved by the Waltz algorithm (O(n3) n number of variables)
Early paper that attracts attention on the difficulty of label propagation in interval labels (continuous domains)
This work has been continued by Faltings (AIJ 92) who studied the early-quiescence problem of the Waltz algorithm
Oct 5 2004 ConsistencyProperties amp Algorithms
12
Foundations of Constraint Processing Fall 2004
Consistency checking algorithms
Examining finite binary CSPs and their complexity
Notation
Given variables i j k with values x y z the predicate
Pijk(x y z) is true iff the 3-tuple x y z Cijk
Node consistency checking Pi(x)
Arc consistency checking Pij(xy)
Path consistency bit-matrix manipulation
Oct 5 2004 ConsistencyProperties amp Algorithms
13
Foundations of Constraint Processing Fall 2004
Worst-case complexity as a function of the input parameters
Upper bound f(n) is O(g(n)) means that f(n) cg(n)
f grows as g or slower
Lower boundf(n) is (h(n)) means that f(n) ch(n)
f grows as g or faster
Input parameters for a CSP
n = number of variables
a = (max) size of a domain
dk = degree of Vk ( n-1)
e = number of edges (or constraints) [(n-1) n(n-1)2]
Worst-case asymptotic complexitytime
space
Oct 5 2004 ConsistencyProperties amp Algorithms
14
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node amp arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
15
Foundations of Constraint Processing Fall 2004
Node consistency (NC)Procedure NC(i)
Di Di x | Pi(x)
Begin
for i 1 until n do NC(i)
end
Oct 5 2004 ConsistencyProperties amp Algorithms
16
Foundations of Constraint Processing Fall 2004
Complexity of NC Procedure of NC(i)
Di Di x | Pi(x) Begin for i 1 until n do NC(i)end
bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)
Oct 5 2004 ConsistencyProperties amp Algorithms
17
Foundations of Constraint Processing Fall 2004
Arc-consistency Adapted from Dechter
Definition Given a constraint graph G
bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj
bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi
bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
18
Foundations of Constraint Processing Fall 2004
Procedure ReviseRevises the domains of a variable i
Procedure Revise(ij) Begin
DELETE false for each x Di do
if there is no y Dj such that Pij(x y) then begin
delete x from Di
DELETE true end return DELETE end
bull Revise is directionalbull What is the complexity of Revise
a2
Oct 5 2004 ConsistencyProperties amp Algorithms
19
Foundations of Constraint Processing Fall 2004
Revise example R Dechter
Apply the Revise procedure to the following example
5 6 15 1 2 10
YX+Y = 10X
Oct 5 2004 ConsistencyProperties amp Algorithms
20
Foundations of Constraint Processing Fall 2004
Solution DVi DVi Vi (CViVj DVi)
This is actually equivalent to DVi Vi (CViVj DVi)
Effect of Revise Adapted from Rina Dechter
Question Given
bull two variables Vi and Vj
bull their domains DVi and DVj and
bull the constraint CViVj
write the effect of the Revise procedure as a sequence of operations in relational algebra
Hintbull Think about the domain DVi as a unary constraint CVi and
bull consider the composition of this unary constraint and the binary one
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
21
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end
bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)
Oct 5 2004 ConsistencyProperties amp Algorithms
22
Foundations of Constraint Processing Fall 2004
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
23
Foundations of Constraint Processing Fall 2004
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Oct 5 2004 ConsistencyProperties amp Algorithms
24
Foundations of Constraint Processing Fall 2004
NC amp AC Example courtesy of M Fromherz
In the temporal problem below
1 AC propagates bound
2 Unary constraint xgt3 is imposed
3 AC propagates bounds again
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
Oct 5 2004 ConsistencyProperties amp Algorithms
25
Foundations of Constraint Processing Fall 2004
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Oct 5 2004 ConsistencyProperties amp Algorithms
26
Foundations of Constraint Processing Fall 2004
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Oct 5 2004 ConsistencyProperties amp Algorithms
27
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end
Oct 5 2004 ConsistencyProperties amp Algorithms
28
Foundations of Constraint Processing Fall 2004
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (i j) arcs(G) i j
4 While Q is not empty do
5 begin
6 select and delete any arc (k m) from Q
7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
12
Foundations of Constraint Processing Fall 2004
Consistency checking algorithms
Examining finite binary CSPs and their complexity
Notation
Given variables i j k with values x y z the predicate
Pijk(x y z) is true iff the 3-tuple x y z Cijk
Node consistency checking Pi(x)
Arc consistency checking Pij(xy)
Path consistency bit-matrix manipulation
Oct 5 2004 ConsistencyProperties amp Algorithms
13
Foundations of Constraint Processing Fall 2004
Worst-case complexity as a function of the input parameters
Upper bound f(n) is O(g(n)) means that f(n) cg(n)
f grows as g or slower
Lower boundf(n) is (h(n)) means that f(n) ch(n)
f grows as g or faster
Input parameters for a CSP
n = number of variables
a = (max) size of a domain
dk = degree of Vk ( n-1)
e = number of edges (or constraints) [(n-1) n(n-1)2]
Worst-case asymptotic complexitytime
space
Oct 5 2004 ConsistencyProperties amp Algorithms
14
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node amp arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
15
Foundations of Constraint Processing Fall 2004
Node consistency (NC)Procedure NC(i)
Di Di x | Pi(x)
Begin
for i 1 until n do NC(i)
end
Oct 5 2004 ConsistencyProperties amp Algorithms
16
Foundations of Constraint Processing Fall 2004
Complexity of NC Procedure of NC(i)
Di Di x | Pi(x) Begin for i 1 until n do NC(i)end
bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)
Oct 5 2004 ConsistencyProperties amp Algorithms
17
Foundations of Constraint Processing Fall 2004
Arc-consistency Adapted from Dechter
Definition Given a constraint graph G
bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj
bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi
bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
18
Foundations of Constraint Processing Fall 2004
Procedure ReviseRevises the domains of a variable i
Procedure Revise(ij) Begin
DELETE false for each x Di do
if there is no y Dj such that Pij(x y) then begin
delete x from Di
DELETE true end return DELETE end
bull Revise is directionalbull What is the complexity of Revise
a2
Oct 5 2004 ConsistencyProperties amp Algorithms
19
Foundations of Constraint Processing Fall 2004
Revise example R Dechter
Apply the Revise procedure to the following example
5 6 15 1 2 10
YX+Y = 10X
Oct 5 2004 ConsistencyProperties amp Algorithms
20
Foundations of Constraint Processing Fall 2004
Solution DVi DVi Vi (CViVj DVi)
This is actually equivalent to DVi Vi (CViVj DVi)
Effect of Revise Adapted from Rina Dechter
Question Given
bull two variables Vi and Vj
bull their domains DVi and DVj and
bull the constraint CViVj
write the effect of the Revise procedure as a sequence of operations in relational algebra
Hintbull Think about the domain DVi as a unary constraint CVi and
bull consider the composition of this unary constraint and the binary one
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
21
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end
bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)
Oct 5 2004 ConsistencyProperties amp Algorithms
22
Foundations of Constraint Processing Fall 2004
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
23
Foundations of Constraint Processing Fall 2004
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Oct 5 2004 ConsistencyProperties amp Algorithms
24
Foundations of Constraint Processing Fall 2004
NC amp AC Example courtesy of M Fromherz
In the temporal problem below
1 AC propagates bound
2 Unary constraint xgt3 is imposed
3 AC propagates bounds again
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
Oct 5 2004 ConsistencyProperties amp Algorithms
25
Foundations of Constraint Processing Fall 2004
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Oct 5 2004 ConsistencyProperties amp Algorithms
26
Foundations of Constraint Processing Fall 2004
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Oct 5 2004 ConsistencyProperties amp Algorithms
27
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end
Oct 5 2004 ConsistencyProperties amp Algorithms
28
Foundations of Constraint Processing Fall 2004
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (i j) arcs(G) i j
4 While Q is not empty do
5 begin
6 select and delete any arc (k m) from Q
7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
13
Foundations of Constraint Processing Fall 2004
Worst-case complexity as a function of the input parameters
Upper bound f(n) is O(g(n)) means that f(n) cg(n)
f grows as g or slower
Lower boundf(n) is (h(n)) means that f(n) ch(n)
f grows as g or faster
Input parameters for a CSP
n = number of variables
a = (max) size of a domain
dk = degree of Vk ( n-1)
e = number of edges (or constraints) [(n-1) n(n-1)2]
Worst-case asymptotic complexitytime
space
Oct 5 2004 ConsistencyProperties amp Algorithms
14
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node amp arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
15
Foundations of Constraint Processing Fall 2004
Node consistency (NC)Procedure NC(i)
Di Di x | Pi(x)
Begin
for i 1 until n do NC(i)
end
Oct 5 2004 ConsistencyProperties amp Algorithms
16
Foundations of Constraint Processing Fall 2004
Complexity of NC Procedure of NC(i)
Di Di x | Pi(x) Begin for i 1 until n do NC(i)end
bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)
Oct 5 2004 ConsistencyProperties amp Algorithms
17
Foundations of Constraint Processing Fall 2004
Arc-consistency Adapted from Dechter
Definition Given a constraint graph G
bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj
bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi
bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
18
Foundations of Constraint Processing Fall 2004
Procedure ReviseRevises the domains of a variable i
Procedure Revise(ij) Begin
DELETE false for each x Di do
if there is no y Dj such that Pij(x y) then begin
delete x from Di
DELETE true end return DELETE end
bull Revise is directionalbull What is the complexity of Revise
a2
Oct 5 2004 ConsistencyProperties amp Algorithms
19
Foundations of Constraint Processing Fall 2004
Revise example R Dechter
Apply the Revise procedure to the following example
5 6 15 1 2 10
YX+Y = 10X
Oct 5 2004 ConsistencyProperties amp Algorithms
20
Foundations of Constraint Processing Fall 2004
Solution DVi DVi Vi (CViVj DVi)
This is actually equivalent to DVi Vi (CViVj DVi)
Effect of Revise Adapted from Rina Dechter
Question Given
bull two variables Vi and Vj
bull their domains DVi and DVj and
bull the constraint CViVj
write the effect of the Revise procedure as a sequence of operations in relational algebra
Hintbull Think about the domain DVi as a unary constraint CVi and
bull consider the composition of this unary constraint and the binary one
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
21
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end
bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)
Oct 5 2004 ConsistencyProperties amp Algorithms
22
Foundations of Constraint Processing Fall 2004
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
23
Foundations of Constraint Processing Fall 2004
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Oct 5 2004 ConsistencyProperties amp Algorithms
24
Foundations of Constraint Processing Fall 2004
NC amp AC Example courtesy of M Fromherz
In the temporal problem below
1 AC propagates bound
2 Unary constraint xgt3 is imposed
3 AC propagates bounds again
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
Oct 5 2004 ConsistencyProperties amp Algorithms
25
Foundations of Constraint Processing Fall 2004
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Oct 5 2004 ConsistencyProperties amp Algorithms
26
Foundations of Constraint Processing Fall 2004
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Oct 5 2004 ConsistencyProperties amp Algorithms
27
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end
Oct 5 2004 ConsistencyProperties amp Algorithms
28
Foundations of Constraint Processing Fall 2004
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (i j) arcs(G) i j
4 While Q is not empty do
5 begin
6 select and delete any arc (k m) from Q
7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
14
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node amp arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
15
Foundations of Constraint Processing Fall 2004
Node consistency (NC)Procedure NC(i)
Di Di x | Pi(x)
Begin
for i 1 until n do NC(i)
end
Oct 5 2004 ConsistencyProperties amp Algorithms
16
Foundations of Constraint Processing Fall 2004
Complexity of NC Procedure of NC(i)
Di Di x | Pi(x) Begin for i 1 until n do NC(i)end
bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)
Oct 5 2004 ConsistencyProperties amp Algorithms
17
Foundations of Constraint Processing Fall 2004
Arc-consistency Adapted from Dechter
Definition Given a constraint graph G
bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj
bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi
bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
18
Foundations of Constraint Processing Fall 2004
Procedure ReviseRevises the domains of a variable i
Procedure Revise(ij) Begin
DELETE false for each x Di do
if there is no y Dj such that Pij(x y) then begin
delete x from Di
DELETE true end return DELETE end
bull Revise is directionalbull What is the complexity of Revise
a2
Oct 5 2004 ConsistencyProperties amp Algorithms
19
Foundations of Constraint Processing Fall 2004
Revise example R Dechter
Apply the Revise procedure to the following example
5 6 15 1 2 10
YX+Y = 10X
Oct 5 2004 ConsistencyProperties amp Algorithms
20
Foundations of Constraint Processing Fall 2004
Solution DVi DVi Vi (CViVj DVi)
This is actually equivalent to DVi Vi (CViVj DVi)
Effect of Revise Adapted from Rina Dechter
Question Given
bull two variables Vi and Vj
bull their domains DVi and DVj and
bull the constraint CViVj
write the effect of the Revise procedure as a sequence of operations in relational algebra
Hintbull Think about the domain DVi as a unary constraint CVi and
bull consider the composition of this unary constraint and the binary one
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
21
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end
bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)
Oct 5 2004 ConsistencyProperties amp Algorithms
22
Foundations of Constraint Processing Fall 2004
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
23
Foundations of Constraint Processing Fall 2004
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Oct 5 2004 ConsistencyProperties amp Algorithms
24
Foundations of Constraint Processing Fall 2004
NC amp AC Example courtesy of M Fromherz
In the temporal problem below
1 AC propagates bound
2 Unary constraint xgt3 is imposed
3 AC propagates bounds again
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
Oct 5 2004 ConsistencyProperties amp Algorithms
25
Foundations of Constraint Processing Fall 2004
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Oct 5 2004 ConsistencyProperties amp Algorithms
26
Foundations of Constraint Processing Fall 2004
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Oct 5 2004 ConsistencyProperties amp Algorithms
27
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end
Oct 5 2004 ConsistencyProperties amp Algorithms
28
Foundations of Constraint Processing Fall 2004
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (i j) arcs(G) i j
4 While Q is not empty do
5 begin
6 select and delete any arc (k m) from Q
7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
15
Foundations of Constraint Processing Fall 2004
Node consistency (NC)Procedure NC(i)
Di Di x | Pi(x)
Begin
for i 1 until n do NC(i)
end
Oct 5 2004 ConsistencyProperties amp Algorithms
16
Foundations of Constraint Processing Fall 2004
Complexity of NC Procedure of NC(i)
Di Di x | Pi(x) Begin for i 1 until n do NC(i)end
bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)
Oct 5 2004 ConsistencyProperties amp Algorithms
17
Foundations of Constraint Processing Fall 2004
Arc-consistency Adapted from Dechter
Definition Given a constraint graph G
bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj
bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi
bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
18
Foundations of Constraint Processing Fall 2004
Procedure ReviseRevises the domains of a variable i
Procedure Revise(ij) Begin
DELETE false for each x Di do
if there is no y Dj such that Pij(x y) then begin
delete x from Di
DELETE true end return DELETE end
bull Revise is directionalbull What is the complexity of Revise
a2
Oct 5 2004 ConsistencyProperties amp Algorithms
19
Foundations of Constraint Processing Fall 2004
Revise example R Dechter
Apply the Revise procedure to the following example
5 6 15 1 2 10
YX+Y = 10X
Oct 5 2004 ConsistencyProperties amp Algorithms
20
Foundations of Constraint Processing Fall 2004
Solution DVi DVi Vi (CViVj DVi)
This is actually equivalent to DVi Vi (CViVj DVi)
Effect of Revise Adapted from Rina Dechter
Question Given
bull two variables Vi and Vj
bull their domains DVi and DVj and
bull the constraint CViVj
write the effect of the Revise procedure as a sequence of operations in relational algebra
Hintbull Think about the domain DVi as a unary constraint CVi and
bull consider the composition of this unary constraint and the binary one
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
21
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end
bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)
Oct 5 2004 ConsistencyProperties amp Algorithms
22
Foundations of Constraint Processing Fall 2004
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
23
Foundations of Constraint Processing Fall 2004
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Oct 5 2004 ConsistencyProperties amp Algorithms
24
Foundations of Constraint Processing Fall 2004
NC amp AC Example courtesy of M Fromherz
In the temporal problem below
1 AC propagates bound
2 Unary constraint xgt3 is imposed
3 AC propagates bounds again
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
Oct 5 2004 ConsistencyProperties amp Algorithms
25
Foundations of Constraint Processing Fall 2004
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Oct 5 2004 ConsistencyProperties amp Algorithms
26
Foundations of Constraint Processing Fall 2004
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Oct 5 2004 ConsistencyProperties amp Algorithms
27
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end
Oct 5 2004 ConsistencyProperties amp Algorithms
28
Foundations of Constraint Processing Fall 2004
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (i j) arcs(G) i j
4 While Q is not empty do
5 begin
6 select and delete any arc (k m) from Q
7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
16
Foundations of Constraint Processing Fall 2004
Complexity of NC Procedure of NC(i)
Di Di x | Pi(x) Begin for i 1 until n do NC(i)end
bull For each variable we check a valuesbull We have n variables we do na checksbull NC is O(an)
Oct 5 2004 ConsistencyProperties amp Algorithms
17
Foundations of Constraint Processing Fall 2004
Arc-consistency Adapted from Dechter
Definition Given a constraint graph G
bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj
bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi
bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
18
Foundations of Constraint Processing Fall 2004
Procedure ReviseRevises the domains of a variable i
Procedure Revise(ij) Begin
DELETE false for each x Di do
if there is no y Dj such that Pij(x y) then begin
delete x from Di
DELETE true end return DELETE end
bull Revise is directionalbull What is the complexity of Revise
a2
Oct 5 2004 ConsistencyProperties amp Algorithms
19
Foundations of Constraint Processing Fall 2004
Revise example R Dechter
Apply the Revise procedure to the following example
5 6 15 1 2 10
YX+Y = 10X
Oct 5 2004 ConsistencyProperties amp Algorithms
20
Foundations of Constraint Processing Fall 2004
Solution DVi DVi Vi (CViVj DVi)
This is actually equivalent to DVi Vi (CViVj DVi)
Effect of Revise Adapted from Rina Dechter
Question Given
bull two variables Vi and Vj
bull their domains DVi and DVj and
bull the constraint CViVj
write the effect of the Revise procedure as a sequence of operations in relational algebra
Hintbull Think about the domain DVi as a unary constraint CVi and
bull consider the composition of this unary constraint and the binary one
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
21
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end
bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)
Oct 5 2004 ConsistencyProperties amp Algorithms
22
Foundations of Constraint Processing Fall 2004
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
23
Foundations of Constraint Processing Fall 2004
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Oct 5 2004 ConsistencyProperties amp Algorithms
24
Foundations of Constraint Processing Fall 2004
NC amp AC Example courtesy of M Fromherz
In the temporal problem below
1 AC propagates bound
2 Unary constraint xgt3 is imposed
3 AC propagates bounds again
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
Oct 5 2004 ConsistencyProperties amp Algorithms
25
Foundations of Constraint Processing Fall 2004
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Oct 5 2004 ConsistencyProperties amp Algorithms
26
Foundations of Constraint Processing Fall 2004
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Oct 5 2004 ConsistencyProperties amp Algorithms
27
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end
Oct 5 2004 ConsistencyProperties amp Algorithms
28
Foundations of Constraint Processing Fall 2004
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (i j) arcs(G) i j
4 While Q is not empty do
5 begin
6 select and delete any arc (k m) from Q
7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
17
Foundations of Constraint Processing Fall 2004
Arc-consistency Adapted from Dechter
Definition Given a constraint graph G
bull A variable Vi is arc-consistent relative to Vj iff for every value aDVi there exists a value bDVj | (a b)CViVj
bull The constraint CViVj is arc-consistent iff ndash Vi is arc-consistent relative to Vj and ndash Vj is arc-consistent relative to Vi
bull A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
18
Foundations of Constraint Processing Fall 2004
Procedure ReviseRevises the domains of a variable i
Procedure Revise(ij) Begin
DELETE false for each x Di do
if there is no y Dj such that Pij(x y) then begin
delete x from Di
DELETE true end return DELETE end
bull Revise is directionalbull What is the complexity of Revise
a2
Oct 5 2004 ConsistencyProperties amp Algorithms
19
Foundations of Constraint Processing Fall 2004
Revise example R Dechter
Apply the Revise procedure to the following example
5 6 15 1 2 10
YX+Y = 10X
Oct 5 2004 ConsistencyProperties amp Algorithms
20
Foundations of Constraint Processing Fall 2004
Solution DVi DVi Vi (CViVj DVi)
This is actually equivalent to DVi Vi (CViVj DVi)
Effect of Revise Adapted from Rina Dechter
Question Given
bull two variables Vi and Vj
bull their domains DVi and DVj and
bull the constraint CViVj
write the effect of the Revise procedure as a sequence of operations in relational algebra
Hintbull Think about the domain DVi as a unary constraint CVi and
bull consider the composition of this unary constraint and the binary one
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
21
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end
bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)
Oct 5 2004 ConsistencyProperties amp Algorithms
22
Foundations of Constraint Processing Fall 2004
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
23
Foundations of Constraint Processing Fall 2004
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Oct 5 2004 ConsistencyProperties amp Algorithms
24
Foundations of Constraint Processing Fall 2004
NC amp AC Example courtesy of M Fromherz
In the temporal problem below
1 AC propagates bound
2 Unary constraint xgt3 is imposed
3 AC propagates bounds again
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
Oct 5 2004 ConsistencyProperties amp Algorithms
25
Foundations of Constraint Processing Fall 2004
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Oct 5 2004 ConsistencyProperties amp Algorithms
26
Foundations of Constraint Processing Fall 2004
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Oct 5 2004 ConsistencyProperties amp Algorithms
27
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end
Oct 5 2004 ConsistencyProperties amp Algorithms
28
Foundations of Constraint Processing Fall 2004
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (i j) arcs(G) i j
4 While Q is not empty do
5 begin
6 select and delete any arc (k m) from Q
7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
18
Foundations of Constraint Processing Fall 2004
Procedure ReviseRevises the domains of a variable i
Procedure Revise(ij) Begin
DELETE false for each x Di do
if there is no y Dj such that Pij(x y) then begin
delete x from Di
DELETE true end return DELETE end
bull Revise is directionalbull What is the complexity of Revise
a2
Oct 5 2004 ConsistencyProperties amp Algorithms
19
Foundations of Constraint Processing Fall 2004
Revise example R Dechter
Apply the Revise procedure to the following example
5 6 15 1 2 10
YX+Y = 10X
Oct 5 2004 ConsistencyProperties amp Algorithms
20
Foundations of Constraint Processing Fall 2004
Solution DVi DVi Vi (CViVj DVi)
This is actually equivalent to DVi Vi (CViVj DVi)
Effect of Revise Adapted from Rina Dechter
Question Given
bull two variables Vi and Vj
bull their domains DVi and DVj and
bull the constraint CViVj
write the effect of the Revise procedure as a sequence of operations in relational algebra
Hintbull Think about the domain DVi as a unary constraint CVi and
bull consider the composition of this unary constraint and the binary one
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
21
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end
bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)
Oct 5 2004 ConsistencyProperties amp Algorithms
22
Foundations of Constraint Processing Fall 2004
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
23
Foundations of Constraint Processing Fall 2004
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Oct 5 2004 ConsistencyProperties amp Algorithms
24
Foundations of Constraint Processing Fall 2004
NC amp AC Example courtesy of M Fromherz
In the temporal problem below
1 AC propagates bound
2 Unary constraint xgt3 is imposed
3 AC propagates bounds again
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
Oct 5 2004 ConsistencyProperties amp Algorithms
25
Foundations of Constraint Processing Fall 2004
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Oct 5 2004 ConsistencyProperties amp Algorithms
26
Foundations of Constraint Processing Fall 2004
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Oct 5 2004 ConsistencyProperties amp Algorithms
27
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end
Oct 5 2004 ConsistencyProperties amp Algorithms
28
Foundations of Constraint Processing Fall 2004
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (i j) arcs(G) i j
4 While Q is not empty do
5 begin
6 select and delete any arc (k m) from Q
7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
19
Foundations of Constraint Processing Fall 2004
Revise example R Dechter
Apply the Revise procedure to the following example
5 6 15 1 2 10
YX+Y = 10X
Oct 5 2004 ConsistencyProperties amp Algorithms
20
Foundations of Constraint Processing Fall 2004
Solution DVi DVi Vi (CViVj DVi)
This is actually equivalent to DVi Vi (CViVj DVi)
Effect of Revise Adapted from Rina Dechter
Question Given
bull two variables Vi and Vj
bull their domains DVi and DVj and
bull the constraint CViVj
write the effect of the Revise procedure as a sequence of operations in relational algebra
Hintbull Think about the domain DVi as a unary constraint CVi and
bull consider the composition of this unary constraint and the binary one
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
21
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end
bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)
Oct 5 2004 ConsistencyProperties amp Algorithms
22
Foundations of Constraint Processing Fall 2004
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
23
Foundations of Constraint Processing Fall 2004
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Oct 5 2004 ConsistencyProperties amp Algorithms
24
Foundations of Constraint Processing Fall 2004
NC amp AC Example courtesy of M Fromherz
In the temporal problem below
1 AC propagates bound
2 Unary constraint xgt3 is imposed
3 AC propagates bounds again
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
Oct 5 2004 ConsistencyProperties amp Algorithms
25
Foundations of Constraint Processing Fall 2004
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Oct 5 2004 ConsistencyProperties amp Algorithms
26
Foundations of Constraint Processing Fall 2004
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Oct 5 2004 ConsistencyProperties amp Algorithms
27
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end
Oct 5 2004 ConsistencyProperties amp Algorithms
28
Foundations of Constraint Processing Fall 2004
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (i j) arcs(G) i j
4 While Q is not empty do
5 begin
6 select and delete any arc (k m) from Q
7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
20
Foundations of Constraint Processing Fall 2004
Solution DVi DVi Vi (CViVj DVi)
This is actually equivalent to DVi Vi (CViVj DVi)
Effect of Revise Adapted from Rina Dechter
Question Given
bull two variables Vi and Vj
bull their domains DVi and DVj and
bull the constraint CViVj
write the effect of the Revise procedure as a sequence of operations in relational algebra
Hintbull Think about the domain DVi as a unary constraint CVi and
bull consider the composition of this unary constraint and the binary one
1
22
3
1
2
3
1
2
3
Vi VjVjVi
Oct 5 2004 ConsistencyProperties amp Algorithms
21
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end
bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)
Oct 5 2004 ConsistencyProperties amp Algorithms
22
Foundations of Constraint Processing Fall 2004
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
23
Foundations of Constraint Processing Fall 2004
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Oct 5 2004 ConsistencyProperties amp Algorithms
24
Foundations of Constraint Processing Fall 2004
NC amp AC Example courtesy of M Fromherz
In the temporal problem below
1 AC propagates bound
2 Unary constraint xgt3 is imposed
3 AC propagates bounds again
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
Oct 5 2004 ConsistencyProperties amp Algorithms
25
Foundations of Constraint Processing Fall 2004
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Oct 5 2004 ConsistencyProperties amp Algorithms
26
Foundations of Constraint Processing Fall 2004
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Oct 5 2004 ConsistencyProperties amp Algorithms
27
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end
Oct 5 2004 ConsistencyProperties amp Algorithms
28
Foundations of Constraint Processing Fall 2004
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (i j) arcs(G) i j
4 While Q is not empty do
5 begin
6 select and delete any arc (k m) from Q
7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
21
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-1)Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j 4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE ( REVISE(i j) or CHANGE )8 end9 until not CHANGE10 end
bull AC-1 does not update Q the queue of arcsbull No algorithm can have time complexity below O(ea2)
Oct 5 2004 ConsistencyProperties amp Algorithms
22
Foundations of Constraint Processing Fall 2004
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
23
Foundations of Constraint Processing Fall 2004
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Oct 5 2004 ConsistencyProperties amp Algorithms
24
Foundations of Constraint Processing Fall 2004
NC amp AC Example courtesy of M Fromherz
In the temporal problem below
1 AC propagates bound
2 Unary constraint xgt3 is imposed
3 AC propagates bounds again
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
Oct 5 2004 ConsistencyProperties amp Algorithms
25
Foundations of Constraint Processing Fall 2004
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Oct 5 2004 ConsistencyProperties amp Algorithms
26
Foundations of Constraint Processing Fall 2004
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Oct 5 2004 ConsistencyProperties amp Algorithms
27
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end
Oct 5 2004 ConsistencyProperties amp Algorithms
28
Foundations of Constraint Processing Fall 2004
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (i j) arcs(G) i j
4 While Q is not empty do
5 begin
6 select and delete any arc (k m) from Q
7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
22
Foundations of Constraint Processing Fall 2004
Arc consistency
1 AC may discover the solution Example borrowed from Dechter
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
V2 V3
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
23
Foundations of Constraint Processing Fall 2004
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Oct 5 2004 ConsistencyProperties amp Algorithms
24
Foundations of Constraint Processing Fall 2004
NC amp AC Example courtesy of M Fromherz
In the temporal problem below
1 AC propagates bound
2 Unary constraint xgt3 is imposed
3 AC propagates bounds again
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
Oct 5 2004 ConsistencyProperties amp Algorithms
25
Foundations of Constraint Processing Fall 2004
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Oct 5 2004 ConsistencyProperties amp Algorithms
26
Foundations of Constraint Processing Fall 2004
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Oct 5 2004 ConsistencyProperties amp Algorithms
27
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end
Oct 5 2004 ConsistencyProperties amp Algorithms
28
Foundations of Constraint Processing Fall 2004
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (i j) arcs(G) i j
4 While Q is not empty do
5 begin
6 select and delete any arc (k m) from Q
7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
23
Foundations of Constraint Processing Fall 2004
Arc consistency
2 AC may discover inconsistency Example borrowed from Dechter
1 2 3
1 2 3 1 2 3 YltZ
ZltX
ZY
X
XltY
Oct 5 2004 ConsistencyProperties amp Algorithms
24
Foundations of Constraint Processing Fall 2004
NC amp AC Example courtesy of M Fromherz
In the temporal problem below
1 AC propagates bound
2 Unary constraint xgt3 is imposed
3 AC propagates bounds again
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
Oct 5 2004 ConsistencyProperties amp Algorithms
25
Foundations of Constraint Processing Fall 2004
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Oct 5 2004 ConsistencyProperties amp Algorithms
26
Foundations of Constraint Processing Fall 2004
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Oct 5 2004 ConsistencyProperties amp Algorithms
27
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end
Oct 5 2004 ConsistencyProperties amp Algorithms
28
Foundations of Constraint Processing Fall 2004
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (i j) arcs(G) i j
4 While Q is not empty do
5 begin
6 select and delete any arc (k m) from Q
7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
24
Foundations of Constraint Processing Fall 2004
NC amp AC Example courtesy of M Fromherz
In the temporal problem below
1 AC propagates bound
2 Unary constraint xgt3 is imposed
3 AC propagates bounds again
[0 10]
[0 10]
x y-3
x
y
[0 7]
[3 10]
x y-3
x
y
[4 7]
[7 10]
x y-3
x
y
Oct 5 2004 ConsistencyProperties amp Algorithms
25
Foundations of Constraint Processing Fall 2004
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Oct 5 2004 ConsistencyProperties amp Algorithms
26
Foundations of Constraint Processing Fall 2004
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Oct 5 2004 ConsistencyProperties amp Algorithms
27
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end
Oct 5 2004 ConsistencyProperties amp Algorithms
28
Foundations of Constraint Processing Fall 2004
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (i j) arcs(G) i j
4 While Q is not empty do
5 begin
6 select and delete any arc (k m) from Q
7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
25
Foundations of Constraint Processing Fall 2004
Complexity of AC-1Procedure AC-11 begin2 for i 1 until n do NC(i)3 Q (i j) | (ij) arcs(G) i j4 repeat5 begin6 CHANGE false7 for each (i j) Q do CHANGE (REVISE(i j) or CHANGE)8 end9 until not CHANGE10 end
Note Q is not modified and |Q| = 2ebull 4 9 repeats at most na timesbull Each iteration has |Q| = 2e calls to REVISEbull Revise requires at most a2 checks of Pij
AC-1 is O(a3 n e)
Oct 5 2004 ConsistencyProperties amp Algorithms
26
Foundations of Constraint Processing Fall 2004
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Oct 5 2004 ConsistencyProperties amp Algorithms
27
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end
Oct 5 2004 ConsistencyProperties amp Algorithms
28
Foundations of Constraint Processing Fall 2004
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (i j) arcs(G) i j
4 While Q is not empty do
5 begin
6 select and delete any arc (k m) from Q
7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
26
Foundations of Constraint Processing Fall 2004
AC versions
bull AC-1 does not update Q the queue of arcs
bull AC-2 iterates over arcs connected to at least one node whose domain has been modified Nodes are ordered
bull AC-3 same as AC-2 nodes are not ordered
Oct 5 2004 ConsistencyProperties amp Algorithms
27
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end
Oct 5 2004 ConsistencyProperties amp Algorithms
28
Foundations of Constraint Processing Fall 2004
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (i j) arcs(G) i j
4 While Q is not empty do
5 begin
6 select and delete any arc (k m) from Q
7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
27
Foundations of Constraint Processing Fall 2004
Arc consistency (AC-3)AC-3 iterates over arcs connected to at least one
node whose domain has been modified
Procedure AC-31 begin2 for i 1 until n do NC(i)3 Q (i j) | (i j) arcs(G) i j 4 While Q is not empty do5 begin6 select and delete any arc (k m) from Q7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m 8 end9 end
Oct 5 2004 ConsistencyProperties amp Algorithms
28
Foundations of Constraint Processing Fall 2004
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (i j) arcs(G) i j
4 While Q is not empty do
5 begin
6 select and delete any arc (k m) from Q
7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
28
Foundations of Constraint Processing Fall 2004
Complexity of AC-3Procedure AC-3
1 begin
2 for i 1 until n do NC(i)
3 Q (i j) | (i j) arcs(G) i j
4 While Q is not empty do
5 begin
6 select and delete any arc (k m) from Q
7 If Revise(k m) then Q Q (i k) | (i k) arcs(G) i k i m
8 end
9 end
bull First |Q| = 2e then it grows and shrinks 48
bull Worst case ndash 1 element is deleted from DVk per iteration
ndash none of the arcs added is in Q
bull Line 7 a(dk - 1)
bull Lines 4 - 8
bull Revise a2 checks of Pij
AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e n ndash 1) AC-3 is O(a3e)
If AC-p AC-3 is O(a2e) AC-3 is (a2e) Complete graph O(a3n2)
)2(2121
neaedaen
k k
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
29
Foundations of Constraint Processing Fall 2004
Example Apply AC-3Example Apply AC-3 Thanks to Xu Lin
DV1 = 1 2 3 4 5
DV2 = 1 2 3 4 5
DV3 = 1 2 3 4 5
DV4 = 1 2 3 4 5
CV2V3 = (2 2) (4 5) (2 5) (3 5) (2 3) (5 1) (1 2) (5 3) (2 1) (1 1)
CV1V3 = (5 5) (2 4) (3 5) (3 3) (5 3) (4 4) (5 4) (3 4) (1 1) (3 1)
CV2V4 = (1 2) (3 2) (3 1) (4 5) (2 3) (4 1) (1 1) (4 3) (2 2) (1 5)
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
V1
V3
V2
V4
V1
V3
V2
V4 1 3 5 1 2 3 5
1 2 3 4 1 3 5
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
30
Foundations of Constraint Processing Fall 2004
Applying AC-3 Thanks to Xu Lin
bull Queue = CV2 V4 CV4V2 CV1V3 CV2V3 CV3 V1 CV3V2
bull Revise(V2V4) DV2 DV2 5 = 1 2 3 4
bull Queue = CV4V2 CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V4 V2) DV4 DV4 4 = 1 2 3 5
bull Queue = CV1V3 CV2V3 CV3 V1 CV3 V2
bull Revise(V1 V3) DV1 1 2 3 4 5
bull Queue = CV2V3 CV3 V1 CV3 V2
bull Revise(V2 V3) DV2 1 2 3 4
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
31
Foundations of Constraint Processing Fall 2004
Applying AC-3 (cont) Thanks to Xu Lin
bull Queue = CV3 V1 CV3 V2
bull Revise(V3 V1) DV3 DV3 2 = 1 3 4 5
bull Queue = CV2 V3 CV3 V2
bull Revise(V2 V3) DV2 DV2
bull Queue = CV3 V2
bull Revise(V3 V2) DV3 1 3 5
bull Queue = CV1 V3
bull Revise(V1 V3) DV1 1 3 5 END
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
32
Foundations of Constraint Processing Fall 2004
Main ImprovementsMohr amp Henderson (AIJ 86) bull AC-3 O(a3e) AC-4 O(a2e)bull AC-4 is optimalbull Trade repetition of consistency-check operations with heavy
bookkeeping on which and how many values support a given value for a given variable
bull Data structures it usesndash m values that are active not filteredndash s lists all vvps that support a given vvpndash counter given a vvp provides the number of support provided by a
given variable
bull How it proceeds1 Generates data structures2 Prepares data structures3 Iterates over constraints while updating support in data structures
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
33
Foundations of Constraint Processing Fall 2004
Step 1 Generate 3 data structures
bull m and s have as many rows as there are vvprsquos in the problembull counter has as many rows as there are tuples in the constraints
TV
TV
TV
TV
TV
m
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
NILV
NILV
NILV
NILV
NILV
s
)4(
)2(
)3(
)2(
)5(
4
2
3
3
4
04)(
01)(
01)(
02)(
03)(
31
32
24
24
24
VV
VV
VV
VV
VV
counter
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
34
Foundations of Constraint Processing Fall 2004
Step 2 Prepare data structuresData structures s and counter
bull Checks for every constraint CViVj all tuples Vi=ai Vj=bj_)bull When the tuple is allowed then update
ndash s(Vjbj) s(Vjbj) (Vi ai) andndash counter(Vjbj) (Vjbj) + 1
Update counter ((V2 V3) 2) to value 1Update counter ((V3 V2) 2) to value 1Update s-htable (V2 2) to value ((V3 2))Update s-htable (V3 2) to value ((V2 2))
Update counter ((V2 V3) 4) to value 1Update counter ((V3 V2) 5) to value 1Update s-htable (V2 4) to value ((V3 5))Update s-htable (V3 5) to value ((V2 4))
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
35
Foundations of Constraint Processing Fall 2004
Constraints CV2V3 and CV3V2 S
(V4 5) Nil
(V3 2) ((V2 1) (V2 2))
(V3 3) ((V2 5) (V2 2))
(V2 2) ((V31) (V33) (V3 5) (V32))
(V3 1) ((V2 1) (V2 2) (V2 5))
(V2 3) ((V3 5))
(V1 2) Nil
(V2 1) ((V3 1) (V3 2))
(V1 3) Nil
(V3 4) Nil
(V4 2) Nil
(V3 5) ((V2 3) (V2 2) (V2 4))
(v4 3) Nil
(V1 1) Nil
(V2 4) ((V3 5))
(V2 5) ((V3 3) (V3 1))
(V4 1) Nil
(V1 4) Nil
(V1 5) Nil
(V4 4) Nil
Counter(V4 V2) 3 0
(V4 V2) 2 0
(V4 V2) 1 0
(V2 V3) 1 2
(V2 V3) 3 4
(V4 V2) 5 1
(V2 V4) 1 0
(V2 V3) 4 0
(V4 V2) 4 1
(V2 V4) 2 0
(V2 V3) 5 0
(V2 V4) 3 2
(V2 V4) 4 0
(V2 V4) 5 0
(V3 V1) 1 0
(V3 V1) 2 0
(V3 V1) 3 0
hellip hellip
(V3 V2) 4 0
Etchellip Etc
Updating m
Note that (V3 V2)4 0 thus
we remove 4 from the domain of V3
and update (V3 4) nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk l) | (V3 4) s[(Vk l)]
we decrement counter[(Vk V3) l] by 1
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
36
Foundations of Constraint Processing Fall 2004
Summary of arc-consistency algorithmsbull AC-4 is optimal (worst-case) [Mohr amp Henderson AIJ 86]
bull Warning worst-case complexity is pessimistic Better worst-case complexity AC-4
Better average behavior AC-3 [Wallace IJCAI 93]
bull AC-5 special constraints [Van Hentenryck Deville Teng 92]
functional anti-functional and monotonic constraints
bull AC-6 AC-7 general but rely heavily on data structures for bookkeeping [Bessiegravere amp Reacutegin]
bull Now back to AC-3 AC-2000 AC-2001 AC-31 AC33
bull Non-binary constraints ndash GAC (general) [Mohr amp Masini 1988]
ndash all-different (dedicated) [Reacutegin 94]
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
37
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison amp CSP
parameters bull Which AC algorithm to choose Project results
may tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
38
Foundations of Constraint Processing Fall 2004
CSP parameters lsaquon a t drsaquobull n is number of variablesbull a is maximum domain sizebull t is constraint tightness
bull d is constraint density
where e is the constraints emin=(n-1) and emax = n(n-1)2
Lately we use constraint ratio p = eemax
rarr Constraints in random problems often generated uniform
rarr Use only connected graphs (throw the unconnected ones away)
tuplesall
tuplesforbiddent
)(
)(
minmax
min
ee
eed
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
39
Foundations of Constraint Processing Fall 2004
Criteria for performance comparison
1 Bounding time and space complexity (theoretical)ndash worst-case ndash average-case ndash best- case
2 Counting the number of constraint checks (theoretical empirical)
3 Measuring CPU time (empirical)
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
40
Foundations of Constraint Processing Fall 2004
Performance comparison Courtesy of Lin XU
AC-7
AC-4
AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
41
Foundations of Constraint Processing Fall 2004
Wisdom () Free adaptation from Bessiegravere
bull When a single constraint check is very expensive to make use AC-7
bull When there is a lot of propagation use AC-4bull When little propagation and constraint checks
are cheap use AC-3x
Advice Instructors personal opinion
Use AC-3
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
42
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency minimality and decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
43
Foundations of Constraint Processing Fall 2004
AC is not enough Example borrowed from Dechter
Arc-consistentSatisfiable
seek higher levels of consistency
V1
a b
a b a bV2 V3
=
V1
b a
a b b aV2 V3
=
=
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
44
Foundations of Constraint Processing Fall 2004
Consistency of a pathA path (V0 V1 V2 hellip Vm) of length m is consistent iff bull for any value xDV0 and for any value yDVm that are consistent (ie PV0 Vm(x y))
a sequence of values z1 z2 hellip zm-1 in the domains of variables V1 V2 hellip Vm-1 such that all constraints between them (along the path not across it) are satisfied
(ie PV0 V1(x z1) PV1 V2(z1 z2) hellip PVm-1 Vm(zm-1 zm) )
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
45
Foundations of Constraint Processing Fall 2004
Note The same variable can appear more than once in the path
Every time it may have a different value
Constraints considered PV0Vm and those along the path
All other constraints are neglected
for all x DV0 for all y DVm
V0 Vm
Vm-1
V2
V1
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
46
Foundations of Constraint Processing Fall 2004
Example consistency of a path Check path length = 2 3 4 5 6
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
47
Foundations of Constraint Processing Fall 2004
Path consistency definition
A path of length m is path consistent A CSP is path consistent Property of a
CSP
Definition A CSP is path consistent (PC) iff every path is
consistent (ie any length of path)
Question should we enumerate every path of any length
Answer No only length 2 thanks to [Mackworth AIJ77]
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
48
Foundations of Constraint Processing Fall 2004
Making a CSP Path Consistent (PC)
Special case Complete graphTheorem In a complete graph if every path of length 2 is
consistent the network is path consistent [Mackworth AIJ77]
PC-1 two operations composition and intersection
Proof by induction
Special case Triangulated graphTheorem In a triangulated graph if every path of length 2 is
consistent the network is path consistent [Bliek amp Sam-Haroud lsquo99]
PPC (partially path consistent) PC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
49
Foundations of Constraint Processing Fall 2004
Tools for PC-1
Two operators
1 Constraint composition ( bull )
R13 = R12 bull R23
2 Constraint intersection ( )
R13 R13 old R13 induced
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
50
Foundations of Constraint Processing Fall 2004
Path consistency (PC-1)Achieved by composition and intersection (of binary relations
expressed as matrices) over all paths of length twoProcedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
51
Foundations of Constraint Processing Fall 2004
Properties of PC-1
Discrete CSPs [Montanari74]
1 PC-1 terminates
2 PC-1 results in a path consistent CSP
bull PC-1 terminates It is complete sound (for finding PC network)
bull PC-2 Improves to PC-1similarly to (AC-1 AC-3)
Complexity of PC-1
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
52
Foundations of Constraint Processing Fall 2004
Complexity of PC-1Procedure PC-11 Begin2 Yn R3 repeat4 begin5 Y0 Yn
6 For k 1 until n do7 For i 1 until n do8 For j 1 until n do
9 Ylij Yl-1
ij Yl-1ik bull Yl-1
kk bull Yl-1kj
10 end11 until Yn = Y0
12 Y Yn
10 end
Line 9 a3
Lines 6ndash10 n3 a3
Line 3 at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
53
Foundations of Constraint Processing Fall 2004
Some improvementsbull Mohr amp Henderson (AIJ 86)
ndash PC-2 O(a5n3) PC-3 O(a3n3)ndash Open question PC-3 optimal
bull Han amp Lee (AIJ 88)ndash PC-3 is incorrectndash PC-4 O(a3n3) space and time
bull Singh (ICTAI 95)ndash PC-5 uses ideas of AC-6 (support bookkeeping)
Note PC is seldom used in practical applications unless in presence of special type of constraints (eg bounded difference)
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
54
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Path consistency corresponds to inferring a new constraint
(alternatively tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
Considers all subgraphs of 3 variables
3-consistency
B
C
A
B lt C
A lt B
B
C
AA lt B
A lt C
B lt C
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
55
Foundations of Constraint Processing Fall 2004
Path consistency as inference of binary constraints
Another example
V4
V3
a b
V4
V2
V1
a b
V3
a b
a b
V2
a b
V1
a b
a b
=
=a b a b
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
56
Foundations of Constraint Processing Fall 2004
Question Adapted from Dechter
Given three variables Vi Vk and Vj and the constraints CViVk CViVj and CVkVj write the effect of PC as a sequence of operations in relational algebra
B
C
A
B lt C
A lt B
A + 3 gt C
B
C
A
A lt B
A + 3 gt C
A lt C
B lt C
B
C
A
A lt B
-3 lt A ndashC lt 0
B lt C
Solution CViVj CViVj ij(CViVk CVkVj)
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
57
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher
levels of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
58
Foundations of Constraint Processing Fall 2004
Important properties of a CSP
1 (Arc Consistency AC)
2 Path Consistency PC
3 (strong) K-consistency
4 Minimality
5 Decomposability (ref lossless join decomposition)
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
59
Foundations of Constraint Processing Fall 2004
Is this CSP 3-consistent is it 2-consistent
Definitions k-consistency vs strong k-consistencybull k-consistency any consistent assignment for (k-1)-variable can be
extended to any kth variablebull strong k-consistency has to be j-consistent for all j k
Warning Does 3-consistency guarantee 2 consistency
red blue
A
B
red
red blue
red
C
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
60
Foundations of Constraint Processing Fall 2004
MinimalityPC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Notebull Minimal path consistentbull The definition of minimal CSP is concerned with binary CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
61
Foundations of Constraint Processing Fall 2004
Minimal CSP
Minimal network aka central problem
Given two values for two variables if they are consistent then they appear in at least one solution
Informallybull In a minimal CSP the remainder of the CSP does not add any
further constraint to the direct constraint CVi Vj between the two variables Vi and Vj [Mackworth
AIJ77]
bull A minimal CSP is perfectly explicit as far as the pair Vi and Vj is concerned the rest of the network does not add any further constraint to the direct constraint CVi Vj [Montanari74]
bull The binary constraints are explicit as possible [Montanari74]
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
62
Foundations of Constraint Processing Fall 2004
Decomposabilitybull Any combination of values for k variables that satisfy the constraints
between them can be extended to a solution
bull Decomposability generalizes minimality
Minimality any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability any consistent combination of values for
any k variables is extendable to a solution
Decomposable Minimal Path Consistent
Strong n-consistent n-consistent Solvable
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
63
Foundations of Constraint Processing Fall 2004
DecomposabilityIn generalbull Decomposability minimality path consistentbull PC is used to approximate minimality (which is the central problem)
When is the approximation the real thingSpecial casesbull When composition distributes over intersection [Montanari74]
PC-1 on the completed graph guarantees minimality and decomposability
bull When constraints are convex [Bliek amp Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
64
Foundations of Constraint Processing Fall 2004
Relations to (theory of) DB
CSP DatabaseMinimal Pair-wise consistent
Decomposable Complete join
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
65
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSP0 Complete
the graph PC-1
CSP0 Triangulate the graph PPC
Result
bull CSP0 is not path consistent or
bull PC-1(CSP0) is path consistent
1 It is tight but could be tighter
2 The graph is complete
Result
bull CSP0 is not path consistent or
bull PPC(CSP0) is path consistent
1 It is generally less tight than PC-1(CSP0)
2 The graph is triangulatedWatch for PPC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
66
Foundations of Constraint Processing Fall 2004
Path ConsistencyCSPa Complete
the graph PC-124Result [Montanari 74]
bull CSPa is not path consistent or
bull PC-1(CSPa) is
1 Path consistent minimal amp decomposable
2 Graph is complete
With a(bc)=(ab)(bc)
CSPb Triangulatethe graph PPC
Constraint are convex
Result [BampS-H 99]
bull CSPb is not path consistent or
bull PPC(CSPb) is
1 Path consistent minimal decomposable
2 Existing edges are as tight as PC-1(CSPb)
3 Graph is triangulated
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
67
Foundations of Constraint Processing Fall 2004
PC guarantees minimality decomposability
In PC-1 two operations
R01 bull (R13 R13) = (R01 bull R13) (R01 bull Rrsquo13)
When ( bull ) distributes over ( ) then [Montanari74]
1 PC guarantees that CSP is minimal and decomposable
2 1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall)
Intersection
Composition bull
Rrsquo13
R13
R23R12
V1 V3
V2
V0 R01
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
68
Foundations of Constraint Processing Fall 2004
Temporal reasoning constraints of bounded difference
Variables X Y Z etc
Constraints a Y-X b ie Y-X = [a b] = I
Composition I 1 bull I2 = [a1 b1] bull [a2 b2] = [a1+ a2 b1+b2] Interpretation ndash intervals indicate distancesndash composition is triangle inequality
Intersection I1 I2 = [max(a1 a2) min(b1 b2)]
Distributivity I1 bull (I2 I3) = (I1 bull I2) (I1 bull I3)
Proof left as an exercise
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
69
Foundations of Constraint Processing Fall 2004
Composition of intervals + Rrsquo13 = R12 + R23 = [4 12]
R01 + R13 = [25] + [3 5] = [5 10]
R01 + R13 = [25] + [4 12] = [6 17]
Intersection of intervals R13 R13 = [4 12] [3 5] = [4 5]
R01 + (R13 R13) = (R01 + R13) (R01 + R13)
R01 + (R13 R13) = [2 5] + [4 5] = [6 10]
(R01 + R13) (R01 + R13) = [5 10] [617] = [6 10]
Path consistency guarantees minimality
Side note PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path) where composition is lsquoscalar additionrsquo and intersection is lsquoscalar minimalrsquo
Example constraints of bounded difference (temporal reasoning)
Rrsquo13
R13 =[35]
R23=[18]R12=[34]
V1 V3
V2
V0 R01=[25] [
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
70
Foundations of Constraint Processing Fall 2004
Convex constraints temporal reasoning (again)
Thanks to Xu Lin (2002)
bull Constraints of bounded difference are convex
bull We triangulate the graph (good heuristics exist)
bull Apply PPC restrict propagations in PC to triangles of the graph (and not in the complete graph)
bull According to [Bliek amp Sam-Haroud 99] PPC becomes equivalent to PC thus it guarantees minimality and decomposability
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
71
Foundations of Constraint Processing Fall 2004
Constraint propagation courtesy of Dechter
After Arc-consistency
After Path-consistency
bull Are these CSPs the samendash Which one is more explicitndash Are they equivalent
bull The more propagationndash the more explicit the constraintsndash the more search is directed towards a solution
1
2
3
1
2
3
1
2 2
3
( 0 1 )
( 0 1 )( 0 1 )
( 0 1 )
( 0 1 )( 0 1 )
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
72
Foundations of Constraint Processing Fall 2004
Arc-consistent
Path-consistent
PC can detect unsatisfiability
V3
a b
V4
a bV2
V1 a b
a b
a b
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
73
Foundations of Constraint Processing Fall 2004
PC is not enough
Arc-consistent
Path-consistent
Satisfiable
we should seek (even) higher levels of consistency
k-consistency k = 1 2 3 hellip
a b c
V2
a b c
V3
a b c
V1
a b c
V4
a b cV7
a b c V5
a b c
V6
All mutex constraints
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
74
Foundations of Constraint Processing Fall 2004
k- consistency
bull 2-consistency DVi DVi Vi (CViVj DVj)Projection of binary constraint is used to tighten unary constraint
bull 3-consistency CViVj CViVj ViVj(CViVk CVkVj)Projection of ternary constraint is used to tighten binary constraint
bull k-consistency [Freuder 78 82]
Projection of k-ary constraint is used to tighten (k-1)-ary constraint
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
75
Foundations of Constraint Processing Fall 2004
k-consistencyDefinition
A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables
Example (courtesy of Dechter)
Q
Q
Q
Q
Q
3-consistent 4-consistent
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
76
Foundations of Constraint Processing Fall 2004
Summary the higher the consistency level the better
bull New terms AC PC minimal CSP decomposable CSP complete graph triangulated graph Revise AC-1 AC-3 PC-1 PC-2 PPC
bull Complexity of AC PC algorithmsbull AC tightens domains PC tightens binary constraintsbull Consistency operations in terms of composition intersection in7
relational algebrabull Path consistency approximates minimality which approximates
decomposabilitybull Situations when path consistency guarantees minimality and
decomposabilitybull Criteria for performance comparisonbull Parameters for random CSPs
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
77
Foundations of Constraint Processing Fall 2004
Outline1 Motivation and material2 Node and arc consistency and their complexity 3 Criteria for performance comparison and CSP
parameters bull Which AC algorithm to choose Project results may
tell
4 Path consistency and its complexity5 K-consistency Minimality amp decomposability6 Other related results box consistency higher levels
of consistency (i j)-consistency etc
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC
Oct 5 2004 ConsistencyProperties amp Algorithms
78
Foundations of Constraint Processing Fall 2004
Related resultsbull Continuous domains box consistencybull Higher consistency levels
ndash (strong) k-consistency (ij)-consistency [Freuder]
bull Integration with searchndash AC
bull Forward Checking (FC) [Haralick amp Elliott]
bull Directional Arc-Consistency (DAC) [Dechter]
bull Maintaining arc consistency (MAC) [Sabin amp Freuder]ndash PC Directed path consistency (DPC) [Dechter]
bull Non-binary constraintsndash Generalized arc consistency (GAC) [Mohr amp Masini]
ndash Relational consistency [Dechter]
bull Non-binary constraints and integration with search nFC0 nFC1 hellip nFC5
bull etc
Tightness
Density
Low High
Low FC MAC
High FC FC