Counting CSP Solutions Using Generalized XOR Constraints

Carla P. Gomes, Willem-Jan van Hoeve,Ashish Sabharwal, Bart Selman

Cornell University

AAAI Conference, 2007Vancouver, BC

July 25, 2007 AAAI 2007 2

Problem Description

Constraint Satisfaction Problem (CSP) P:Input:

a set V of variablesa set of corresponding domains of variable values [finite]a set of constraints on V

Output:a solution, i.e. an assignment of values to variables in V such that all constraints are satisfied

Solution Counting: how many solutions does P have? #P-complete problem Many applications to probabilistic reasoning,

adversarial reasoning (bounded length), etc.[Roth ’96, Littman et al. ’01, Sang et al. ’04, Darwiche ’05, Domingos ’06]

July 25, 2007 AAAI 2007 3

Counting Techniques for CSPs

Best known generic method: exhaustive systematic searchContinue branch-and-bound search after first solution found

Simple implementation If terminates: obtain exact solution count If out of time: obtain lower bound

Not as advanced as systematic counters for SAT [Relsat, Cachet] No problem division into components No formula caching / component caching No conflict-based learning

Scalability issue

c

d

a

b

d

e e

0

0

0

1

1

1

1

1

1

1

c

dd0 1

… … 0

0

0

0

July 25, 2007 AAAI 2007 4

Specialized Counting Methods

E.g. for Binary CSPs [Angelsmark-Johnson ’03, Kask-Dechter-Gogate ’04] Count “blocks” of solutions at a time Similar to SAT model counter called Relsat Limited by scalability issues of high granularity exact counters

(counts often range in 1015 to 1060 and higher)

For integer domains [Morgado-Matos-Manquinho-MarquesSilva ’06] Solution count preserving translation into pseudo-Boolean form Optional further translation into Boolean form (SAT)

[Bailleux-Boufkhad-Roussel ’06]

Often better than direct integer domain counting Still limited scalability

July 25, 2007 AAAI 2007 5

This Work

A generic technique for efficiently counting CSP solutions

Provides lower and upper bounds on the solution count Provides a correctness confidence (e.g. a 99% guarantee) Often very fast

Builds upon the MBound framework for counting for SATXOR-streamlining [Gomes-Sabharwal-Selman AAAI-06]

Extends the idea to general CSPs by exploiting Richer structure : generalized XORs Modular solution techniques : specialized filtering / propagation

Background

July 25, 2007 AAAI 2007 7

What are XOR/Parity Constraints?Special constraints on Boolean variables

a b c d = 1 : satisfied if an odd number of a,b,c,d are set to 1

e.g. (a,b,c,d) = (1,1,1,0) satisfies it (1,1,1,1) does not

b d e = 0 : satisfied if an even number of b,d,e are set to 1

These translate into a small set of CNF clauses Used earlier in randomized reductions in Theoretical CS

[Valiant-Vazirani ’86]

July 25, 2007 AAAI 2007 8

XORs for Counting SATMBound approach:

choose constraint X uniformly at random from all XOR constraints of size k (i.e. with k variables)

Two crucial properties:

1. For any k, for every truth assignment A,Pr [ A satisfies X ] = 0.5

n When k=n/2, for every two truth assignments A and B,“A satisfies X” and “B satisfies X” are independent events(pairwise independence)

Good average behavior, some guarantees,

lower bounds

Provides low variation,stronger guarantees,upper bounds

July 25, 2007 AAAI 2007 9

XORs for Counting SAT

Key idea: instead of modifying the solver for counting, modify the problem and feed to the usual solver

Theorem: Algorithm is correct with probability 1 2t

[Gomes-Sabharwal-Selman AAAI-06][Gomes-Hoffmann-Sabharwal-Selman SAT-07]

s random XORconstraints

solution count 2s

orsolution count 2s+

SATinstance

Off-the-shelfSAT Solver

Iterate t times

July 25, 2007 AAAI 2007 10

The Desired Effect

M = 50 solutions 22 survive

7 survive

13 survive

3 surviveunique solution

If each XOR cut the solution space roughly in half, wouldget down to a unique solution in roughly log2 M steps

Counting CSP Solutions

July 25, 2007 AAAI 2007 12

Counting CSP Solutions

CSPinstance

Individualfiltering

Globalfiltering

based on watched vars

based on Gaussian elim.

Individualfiltering

based on dynamic prog.for knapsack constraints

Global filter.

a b c = 1Lower and upper bounds

a + b + c = r (mod D)Lower bounds only

Inefficient, incomplete

Binary XORson shadow model

Generalized XORson CSP vars

We propose 3 XOR constraintsbased approaches:

July 25, 2007 AAAI 2007 13

CSP Counting: Binary XORs

Relatively simple idea: Create a binary shadow model for the CSP

For each (x, v Domain(x)), create a new variable yx,v {0,1} Add constraint: yx,v = 1 iff x == v

Add random binary XORs over variables {yx,v} Apply Boolean XOR framework to obtain solution counts

Same correctness guarantees as for SAT Often works quite well!

random binary XORson shadow vars

solution count(bounds)CSP

CSP +binary shadow

modelCSP Solver

iterate

July 25, 2007 AAAI 2007 14

Filtering Binary XORs: Individual

Filtering possible iff XOR constraint has exactly 1 unbound variable

For efficiency, use watched variables technique from SAT Maintain watch on two yet unassigned variables Process constraint only when a watched variable becomes bound

Find new variable to watch If no such variable possible, do unit propagation

a c d f = 1

Variable assignment:Consider1 10

Unit propagation: f = 1(filter out 0 from Domain(f))

c = 0a = 1d = 1

July 25, 2007 AAAI 2007 15

Filtering Binary XORs: Global

Equivalent to a system of linear equations over F2 Use Gaussian-elimination style filtering algorithm

Diagonalize the matrix If a row has all 0’s and r.h.s. = 1, system unsatisfiable If a row has only one 1, assign value to variable based on r.h.s.

Can prove: achieves complete filtering each remaining free variable has support for both 0 and 1

a b c d = 1 b c e = 0a d e = 1

Consider the system of binary XOR constraints: a1 1 1 1 0 b 10 1 1 0 1 c = 01 0 0 1 1 d 1 e

July 25, 2007 AAAI 2007 16

No new binary shadow variables Instead, generalized XORs mod D

directly on CSP vars:

a + b + d + f + g = r (mod D), r {0, 1, …, D-1}

CSP Counting: Generalized XORs

random generalizedXORs mod D

solution count(bounds)CSP CSP Solver

iterate

Max. domainsize

July 25, 2007 AAAI 2007 17

CSP Counting: Generalized XORs

Solution counting algorithm:1. Independently repeat t times:

Add s random generalized XORs mod D to the CSP Solve streamlined CSP

1. If all t iterations have solution, output: num-solutions Ds Else Fail

Theorem: For every CSP, Pr [output is correct] 1 Dt Proof sketch:

Compute expected number of surviving solutions Apply Markov’s inequality and independence of iterations

Note: hybrid version: add XORs, count remaining solutions, take min/avg

July 25, 2007 AAAI 2007 18

Filtering Generalized XORs

Individual filtering for each generalized XOR User dynamic programming approach used for complete

filtering of the knapsack constraint [Trick ’03] E.g. a + b + c = 1 (mod 3), a {0,1}, b {1}, c {0,1,2}

0 0

1

2

0

1

2

0

1

2

a b c

Don’t include edges that cannotbe reached from the red node

Remove edges that do notlead to the green node

Filter variable valuesE.g. c cannot be 10

0

2

1 1

1

July 25, 2007 AAAI 2007 19

Counting CSP Solutions: Recap.

CSPinstance

Individualfiltering

Globalfiltering

based on watched vars

based on Gaussian elim.

Individualfiltering

based on dynamic prog.for knapsack constraints

Global filter.

Inefficient, incomplete

Binary XORson shadow model

Generalized XORson CSP vars

a b c = 1Lower and upper bounds

a + b + c = r (mod D)Lower bounds only

3 XOR constraints based approaches:

July 25, 2007 AAAI 2007 20

Experimental Highlights, #1

N-Queens problemorder 20 order 30

true count 3.9 x 1010 ---pure CSP search 3.5 x 106 4.1 x 106 1 hour

binary XORs, individual 1.1 x 106 5.4 x 108

binary XORs, global 2.1 x 106 5.4 x 108 13-196 secgeneralized XORs 6.6 x 108 9.2 x 1015

Pure CSP does not scale: limited by the number of search nodes that can be traversed within 1 hour

(99%

con

fiden

ce)

July 25, 2007 AAAI 2007 21

Experimental Highlights, #2

Graph coloring problems, e.g. “games120”:

Pure CSP 4.3 x 108 1 hr

Pseudo-Boolean counter 1.1 x 106 0.5 hrRelsat (SAT counter) on solution 1.4 x 106 0.5 hr

count preserving translation[Morgado et al. ’06, Bailleux et al. ’06]

Generalized XORs 4.5 x 1042 1 min

July 25, 2007 AAAI 2007 22

Experimental Highlights, #3

Spatially balanced Latin squares Impractical for SAT counters due to average distance computations Using streamlined version for experiments

order 15 order 17pure CSP 112, 1 hr --- 1 hr

generalized XORs 1748, 8 min 1058, 14 min

Note: order 17 instance cannot be solved at all by pure CSP in 1 hr!

July 25, 2007 AAAI 2007 23

Summary

3 approaches to extend the XOR-based counting framework from SAT to CSPs

Generalized XORs Advanced filtering / propagation techniques for XORs

Experimental results very promising Quite fast in practice Very high counts Provable correctness guarantees