University of TorontoMechanical & Industrial Engineering
An Introduction to Constraint Programming
J. Christopher BeckDept. of Mechanical & Industrial EngineeringUniversity of TorontoCanada
University of TorontoMechanical & Industrial Engineering
University of TorontoMechanical & Industrial Engineering
Crystal Maze
• Place the numbers 1 through 8 in the nodes such that:– Each number appears exactly once
?
?
?
?
?
?
??
– No connected nodes have consecutive numbers
You have 5 minutes!
University of TorontoMechanical & Industrial Engineering
Constraint Satisfaction Problem (CSP)• Given:
– V, a set of variables {v0, v1, …, vn}
– D, a set of domains {D0, D1, …, Dn}
– C, a set of constraints {c0, c1, …, cm}
• Each constraint, ci, has a scope ci(v0, v2, v4, v117, …), the variables that it constrains
University of TorontoMechanical & Industrial Engineering
Constraint Satisfaction Problem (CSP)• A constraint, ci, is a mapping from the
elements of the Cartesian product of the domains of the variables in its scope to {T,F}– ci(v0, v2, v4, v117, …) maps:
(D0 X D2 X D4 X D117 X … ) {T,F}
• A constraint is satisfied if the assignment of the variables in its scope are mapped to T
University of TorontoMechanical & Industrial Engineering
Constraint Satisfaction Problem (CSP)• In a solution to a CSP:
– each variable is assigned a value from its domain: vi = di, di є Di
– each constraint is satisfied
University of TorontoMechanical & Industrial Engineering
Crystal Maze
• Place the numbers 1 through 8 in the nodes such that:– Each number appears exactly once
?
?
?
?
?
?
??
– No connected nodes have consecutive numbers
University of TorontoMechanical & Industrial Engineering
Modeling Crystal Maze
• Each node a variable
• {1, …, 8} values in the domain of each variable
• No consecutive numbers a constraint– (vi, vj) |vi – vj| > 1
• All values used all-different constraint
University of TorontoMechanical & Industrial Engineering
Heuristic Search
?
?
?
?
?
?
?? 1 8
{1, 2, 3, 4, 5, 6, 7, 8}
University of TorontoMechanical & Industrial Engineering
Inference/Propagation
?
?
?
?
?
?
?? 1 8
{1, 2, 3, 4, 5, 6, 7, 8}
{1, 2, 3, 4, 5, 6, 7, 8}
University of TorontoMechanical & Industrial Engineering
Inference/Propagation
?
?
?
?
?
?
?? 1 8
{3, 4, 5, 6}
7 2
{3, 4, 5, 6}
{3, 4, 5, 6} {3, 4, 5, 6}
64
3 5
{3, 4, 5, 6, 7} {2, 3, 4, 5, 6}
University of TorontoMechanical & Industrial Engineering
Generic CP Algorithm
AssertConstraint
Propagators
Start
SuccessSolution?
MakeHeuristicDecision
BacktrackTechnique
Failure
Nothing toretract?
Dead-end?
University of TorontoMechanical & Industrial Engineering
The Core of CP
• Modeling– How to represent the problem
• Heuristic search– How to branch– How much effort to find a good branch
• Backtracking
• Inference/propagation
University of TorontoMechanical & Industrial Engineering
Modeling
• How do you representthe problem with yourtools?
• Like MIP, it’s an art– Not as “advanced” on the theory side as MIP
modeling
• Different models of the same problem can have radically different performance
University of TorontoMechanical & Industrial Engineering
Modeling
• Often the first model is easy but poor– rich constraint language, “natural”– slow, scales poorly
• Need to think about:– symmetry, “what would inference do?”, search
heuristics, adding redundancy, multiple viewpoints, …
• May have to build custom constraints
University of TorontoMechanical & Industrial Engineering
Heuristic Search
• Traditional view:– variable-ordering &
value ordering
• More general view:– add constraints that partition the space
• Domain independent vs. domain dependent– min domain/degree, adaptive weights, …– e.g., heuristics for scheduling
University of TorontoMechanical & Industrial Engineering
The Importance of Heuristics
N = 30, |d| = 8, 45 constraints, 26 tuples/constraint(64-26)/64 nogoods => constraint tightness ~ 0.6
Forward checking search
means for 10 problems lexical min-domain
Searchtree 353,599 nodes 44 nodesTime 7.36s 0.01s
University of TorontoMechanical & Industrial Engineering
Backtracking
• Need to follow alternative branches– dead-end or a possible sub-optimal solution
• Chronological backtracking (aka DFS)– simplest, most popular, dumbest
• Constraint-directed backjumping– keep track of info to provably jump over some
branches
• Randomized restart– nice mathematical model
University of TorontoMechanical & Industrial Engineering
Inference/Propagation
• Probably the primary reason for the problem solving power of CP– main mechanism to reduce search
• Most mathematically/theoretically mature sub-area of CP
• 300+ “global constraints” in current catalog – domain-independent to specialized– stamp collecting?
University of TorontoMechanical & Industrial Engineering
Heuristic Search or Propagation?
?
?
?
?
?
?
?? 1 8
{1, 2, 3, 4, 5, 6, 7, 8}
There must be a 1 or 8here. Why?
University of TorontoMechanical & Industrial Engineering
Arc Consistency
• Fundamental notion in CP!
• Given: c1(v1,v2)
– a binary constraint
– e.g., v1 < v2
• Given: D1 = D2 = {0, 1, …, 5}
University of TorontoMechanical & Industrial Engineering
Arc Consistency
• c1 is arc consistent iff
– for all values d1 є D1 there exists a value d2 є D2 such that c1(v1=d1,v2=d2) T
– And similarly for all values d2 є D2
V1 V2<
{0,1,2,3,4,5} {0,1,2,3,4,5}
University of TorontoMechanical & Industrial Engineering
What Now?
V1 V2<
{0,1,2,3,4,5} {0,1,2,3,4,5}
V3
<<
{0,1,2,3,4,5}
University of TorontoMechanical & Industrial Engineering
Generalized Arc Consistency (GAC)• Given: c1(v1,..., vm)
• C1 is GAC iff
– for all variables di, for all values di є Di there exists a tuple of values [dj є Dj], j≠i such that C1(vi=di,[vj=dj]) T
• E.g., c1(v1,v2,v3,v4)
– for every value in d1 є D1 there must be a triple [d2 є D2, d3 є D3, d4 є D4] s.t. c1(v1=d1, v2=d2, v3=d3, v4=d4) T
University of TorontoMechanical & Industrial Engineering
All-different Constraint
• all-diff(v1, v2, …, vn)
– all vi’s must take on a different value
– logically equivalent to
• Useful in many, many combinatorial problems– travelling salesman, timetabling, …
jivv ji ,
University of TorontoMechanical & Industrial Engineering
All-Diff vs. Clique of ≠
• Logically, all-diff(v1, v2, …, vn) =def vi ≠ vj for 1 ≤ i < j ≤ n
• D1=D2=D3={1,3}
• Establish AC (or GAC) for– v1 ≠ v2, v1 ≠ v3, v2 ≠ v3
– all-diff(v1,v2,v3)
V1 V2≠
{1,3} {1,3}
V3≠≠
{1,3}
V1 V2
{1,3} {1,3}
V3{1,3}
all-diff
University of TorontoMechanical & Industrial Engineering
All-Diff vs. Clique of ≠
• So GAC on all-diff is stronger than AC on a clique of ≠
• But what about the computational complexity?
• All-diff is a “global constraint”– GAC on some global constraints is NP-
complete– What about all-diff?
University of TorontoMechanical & Industrial Engineering
GAC Propagation on All-Diff
• Value graph
• Matchings & graph theory
• Main Theorem
• Algorithm Sketch
University of TorontoMechanical & Industrial Engineering
Value Graph
• Example– D1 = {B,C,D,E},
D2 = {B,C}, D3 = {A,B,C,D}, D4 = {B,C}
– all-diff(v1,…,v4)
1
2
3
4
A
B
C
D
E
University of TorontoMechanical & Industrial Engineering
Definitions
• Given a graph G(V,E)
• A matching is a set of disjoint edges– disjoint = no common end-points
• M covers a set if all vertices in S are end-points of an edge in M
• A vertex is M-free if it isn’t covered by M
EM
VS
University of TorontoMechanical & Industrial Engineering
Value Graph
• In terms of matchings, provide a characterization of an assignment that satisfies the all-diff constraint
1
2
3
4
A
B
C
D
E
University of TorontoMechanical & Industrial Engineering
Theorem [Regin94]
• G is a value graph
• All-diff(v1,…,vn) is GAC iff every edge in G belongs to a matching in G covering {v1,…,vn}
University of TorontoMechanical & Industrial Engineering
1
2
3
4
A
B
C
D
E
University of TorontoMechanical & Industrial Engineering
1
2
3
4
A
B
C
D
E
University of TorontoMechanical & Industrial Engineering
1
2
3
4
A
B
C
D
E
University of TorontoMechanical & Industrial Engineering
Finding Initial Max-Matching
• Find a maximum matching– O(n√m), n = |V|, m = |Di|
– well-known graph theoryalgorithm
1
2
3
4
A
B
C
D
E
University of TorontoMechanical & Industrial Engineering
Finding More Max-Matchings 1
• Orient edges:– In M – from variables to values– Not in M – from values to variables
• Find strongly connected components– depth-first search – if you re-enter a node it is
in a strongly connected component
• M-alternating cycle
University of TorontoMechanical & Industrial Engineering
2
4
B
C
1
2
3
4
A
B
C
D
E
University of TorontoMechanical & Industrial Engineering
Finding More Max-Matchings 2
• Find an M-alternating path starting from an uncovered vertex
• Breadth first search
University of TorontoMechanical & Industrial Engineering
A
1
E
3
D
1
2
3
4
A
B
C
D
E
University of TorontoMechanical & Industrial Engineering
GAC Propagation
• All edges – in M– in an M-alternating circuit, or– in an M-alternating path
• are consistent
• Remove the rest
University of TorontoMechanical & Industrial Engineering
1
2
3
4
A
B
C
D
E
1
2
3
4
A
B
C
D
E
University of TorontoMechanical & Industrial Engineering
Value Graph
• D1 = {B,C,D,E}, D2 = {B,C}, D3 = {A,B,C,D}, D4 = {B,C}
• All-different:– D1 = {B,C,D,E}, D2 = {B,C},
D3 = {A,B,C,D}, D4 = {B,C}
University of TorontoMechanical & Industrial Engineering
GAC All-Diff Propagation Summary• Find a max-matching
– O(n√m), n = |V|, m = |Di|
• Find all M-alternating circuits– DFS to find strongly connected components:
O(n+m)
• Find all M-alternating paths starting from an M-free vertex– Breadth-first search: O(m)
University of TorontoMechanical & Industrial Engineering
Global Constraints
• Any number of variables
• Accompanying efficient propagation algorithm– to enforce some level of consistency (not
necessarily GAC)
• A custom global constraint becomes both an algorithmic and modeling object– extending the language– inference algorithm can be arbitrary– compositionality
University of TorontoMechanical & Industrial Engineering
Differences with MIP
• Richer, extensible language
• Traditionally no explicit use of lower bounds– no optimization function
• constraint satisfaction problems
– less focus on relaxations– less true now: LBs “inside” global constraints
University of TorontoMechanical & Industrial Engineering
CP Engineering Process
• Model, remodel, remodel, …– if you are lucky, you’re done– abstraction?
• Domain-specific heuristics
• Custom global constraints
• Hybrids– CP/LP, CP/MIP, CP/Metaheuristics
University of TorontoMechanical & Industrial Engineering
When to Choose CP
• <Handwaving mode on>
• Finding a feasible solution is hard– intricate set of complicated constraints
• No tight relaxations
• Existing global constraints
• Strong back-propagation