SAT-solvingSAT-solving

An old AI technique becomes veryAn old AI technique becomes very

popular in modern A.I. popular in modern A.I.

Overview:Overview:

Basics on SATBasics on SAT

Unit propagation SAT-solvingUnit propagation SAT-solving

Local Search SAT-solvingLocal Search SAT-solving

3

What is SAT-solving?What is SAT-solving?

GivenGiven KR: a set of propositional formula’s KR: a set of propositional formula’s

FindFind a model for KR. a model for KR.

More specifically:More specifically:

LetLet X X11, X, X22, …, X, …, Xnn be all the variables in KR, be all the variables in KR,

FindFind an assignment I; X an assignment I; Xii -> {T,F} , for i=1,n , -> {T,F} , for i=1,n ,

such that all formula’s in KR become true.such that all formula’s in KR become true.

Useful ?Useful ?

Example: 3-queensExample: 3-queens

Representation:Representation:XX1,11,1 XX1,21,2 XX1,31,3

XX2,12,1 XX2,32,3XX2,22,2

XX3,13,1 XX3,23,2 XX3,33,3

represents: there is a queen on i,jrepresents: there is a queen on i,jXXi,ji,j

XXi,1 i,1 XXi,2i,2 XXi,3 i,3 (i=1,3) (i=1,3)

At least one queen on each row:At least one queen on each row:

XXi,1 i,1 ~~XXi,2i,2 ~~XXi,3 i,3 ,, XXi,2 i,2 ~~XXi,1i,1 ~~XXi,3i,3 , X , Xi,3 i,3 ~~XXi,1i,1 ~~XXi,2 i,2 (i=1,3)(i=1,3)

At most one queen on each row:At most one queen on each row:

Plus: similar formulas for columns and diagonals.Plus: similar formulas for columns and diagonals.

33 formula’s ! 33 formula’s ! Generalisation to q-queens ? Very many formula’s Generalisation to q-queens ? Very many formula’s !!

Example: personel rosteringExample: personel rosteringRepresentation:Representation: - For every employee, i,- For every employee, i, - For every shift in the day, j,- For every shift in the day, j, - For every day in the month, k :- For every day in the month, k :

represents: i works on shift j of day krepresents: i works on shift j of day kXXi,j,k i,j,k

Often this is more elegantly done with assignments to Often this is more elegantly done with assignments to {0,1}{0,1} and and sumssums instead of instead of

Generalise to higher numbers than just 1Generalise to higher numbers than just 1

At least one person works on every shift of every day:At least one person works on every shift of every day:

XXi,j,ki,j,k (j=1,5, k=1,30) (j=1,5, k=1,30) i=1,35i=1,35

An interpretation assigns to each Xi,j,k: true or falseAn interpretation assigns to each Xi,j,k: true or false

Is a personel assignment !Is a personel assignment !

6

SAT-solving and NP-completenessSAT-solving and NP-completeness SAT-solving: one of the first identified as SAT-solving: one of the first identified as NP-completeNP-complete

IfIf you find a P-algorithm for SAT: you get a P- you find a P-algorithm for SAT: you get a P-algorithm for all the others.algorithm for all the others.

Also means:Also means: most problems can be encoded as SAT- most problems can be encoded as SAT-problems and solved using SAT-techniques.problems and solved using SAT-techniques.

Means: all other NP-problems are Means: all other NP-problems are technically equivalent with this problemtechnically equivalent with this problem

7

But why practically useful?But why practically useful?

Although NP(-complete)Although NP(-complete)

Has led to:Has led to:- Many areas in CS and AI convert Many areas in CS and AI convert

problems problems to SAT to SAT - Then use SAT-solvers.Then use SAT-solvers.

Very efficient heuristic approaches exist Very efficient heuristic approaches exist that work well on certain classes of that work well on certain classes of problems.problems.

Example: Automated ReasoningExample: Automated Reasoning Compute a finite grounding of the predicate logic theory.Compute a finite grounding of the predicate logic theory. Marcus example: 2 constants: Marcus and Ceasar.Marcus example: 2 constants: Marcus and Ceasar.

1.1. man(Marcus) man(Marcus) is groundis ground

2.2. Pompeian(Marcus) Pompeian(Marcus) is groundis ground

3.3. x Pompeian(x)x Pompeian(x) Roman(x) Roman(x)Pompeian(Marcus) Pompeian(Marcus) Roman(Marcus) Roman(Marcus)Pompeian(Ceasar) Pompeian(Ceasar) Roman(Ceasar) Roman(Ceasar)

4.4. ruler(Caesar) ruler(Caesar) is groundis ground

5.5. x Roman(x) x Roman(x) loyal_to(x,Caesar) loyal_to(x,Caesar) hates(x,Caesar)hates(x,Caesar) Roman(Marcus) Roman(Marcus) loyal_to(Marcus,Caesar) loyal_to(Marcus,Caesar) hates(Marcus,Caesar)hates(Marcus,Caesar)

Roman(Ceasar) Roman(Ceasar) loyal_to(Ceasar,Caesar) loyal_to(Ceasar,Caesar) hates(Ceasar,Caesar)hates(Ceasar,Caesar)

Ect. ….Ect. ….

9

Example: continuedExample: continued Ground predicate logic is equivalent to propositional !Ground predicate logic is equivalent to propositional !

Example:Example:Pompeian(Marcus) Pompeian(Marcus) converts to converts to Pompeian_MarcusPompeian_Marcus

loyal_to(Marcus,Caesar) loyal_to(Marcus,Caesar) converts to converts to loyal_to_Marcus_Caesarloyal_to_Marcus_Caesar

The theorem follows if and only if this propositional The theorem follows if and only if this propositional

KB in unsatisfiable ! KB in unsatisfiable !

Add the propositional version of the negation of the Add the propositional version of the negation of the theoremtheorem

~loyal_to(Marcus,Caesar) ~loyal_to(Marcus,Caesar) converts to converts to loyal_to_Marcus_Caesarloyal_to_Marcus_Caesar

SAT-solvingSAT-solving

Conjunctive Normal FormConjunctive Normal Form

Every formula is equivalent to a formula of the Every formula is equivalent to a formula of the form:form:(A1 (A1 ... ... An) An) (B1 (B1 … … Bm) Bm) … … (C1 (C1 … … Ck) Ck)

where all where all AiAi, , BiBi, …, , …, CiCi are either are either atomicatomic or or ~atomic~atomic..

Idea:Idea:

p p q q p p q q push all ~ as deep as possiblepush all ~ as deep as possibleapply distributivity of apply distributivity of and and

p p q q q q p pq q ~p ~p

SAT-solving will work on a collection of disjunctions: SAT-solving will work on a collection of disjunctions:

X1 X1 … … Xn Xn ~Y1 ~Y1 … ~Ym … ~Ym

Naive SAT-solvingNaive SAT-solving

Depth-first left-to-right Depth-first left-to-right enumeration of all interpretationsenumeration of all interpretations

SAT - Standard backtrackingSAT - Standard backtrackingX~Y~Y~W,~XYZ,,~X~XWW

T~Y~Y~W,FYZ,,FFWW

XX==TT

YY==TT

TZ,,WW

ZZ==TT

T, , WW

WW==TT

T F

WW==FF

F, , WW

ZZ==FF

T, , WW

ZZ==TT

T

WW==TT

F

WW==FF

F, , WW

ZZ==FF

F~Y~Y~W,TYZ,,TTWW

XX==FF

YY==TT

FF~W,

WW==TT

F

WW==FF

T

FZ,,WW

YY==FF YY==FF

TT~W

SuccessSuccess SuccessSuccess

SuccessSuccess

SuccessSuccess

FailFail

FailFail

FailFail

FailFail FailFail

13

Naïve SAT-solving algorithmNaïve SAT-solving algorithm

NaiveSAT(NaiveSAT( i i))

ForFor TruthTruth = = TT, , FF dodo XXii := := TruthTruth;;

Remove all Dj containing Remove all Dj containing TT from S; from S; Remove all Remove all FF and and FF from all Dj;from all Dj;

IfIf no Dj in S is equal to no Dj in S is equal to FF thenthen

ElseElse ii := := ii + 1; + 1; NaiveSAT( NaiveSAT( i i )); ; ii := := ii - 1; - 1;

IfIf S= {} S= {} thenthen return( Xreturn( X11, X, X22, … , X, … , Xnn););

Form is a CNF-formula with variables XForm is a CNF-formula with variables X11, X, X22, ...X, ...Xnn

S:= { Dj | Dj is a disjunction in Form}S:= { Dj | Dj is a disjunction in Form}

Call: Call: NaiveSATNaiveSAT((11))

End-ForEnd-For

Davis-Putman (1960):Davis-Putman (1960):unit propagation unit propagation

The basis for VERY efficient SAT-solversThe basis for VERY efficient SAT-solvers

An early form of Dynamic Search An early form of Dynamic Search RearrangementRearrangement

15

1. Dealing with pure symbols1. Dealing with pure symbols

Example: Example:

SS = { = { X ~Y , ~Y ~Z, Z X } }

X X is pure: only occurs positiveis pure: only occurs positive

Y Y is pure: only occurs negativeis pure: only occurs negative

Z Z is not pure. is not pure.

A variable Xi is A variable Xi is purepure if it only appears with one sign in all disjunctions. if it only appears with one sign in all disjunctions.

Assign a pure variable the value that makes it true everywhere (don’t consider the Assign a pure variable the value that makes it true everywhere (don’t consider the other assignment).other assignment).

SAT + treating pure symbolsSAT + treating pure symbolsX~Y~Y~W,~XYZ,,~X~XWW

SuccessSuccess

ZZ==TT

X~Y~Y~W,~XY T,,~X~XWW

YY==FF

X TT ~W,~X~XWW

XX==FF

TT W

ZZ is pure is pure

YY is pure is pure

XX is pure is pure

17

The order of choosing variables has become The order of choosing variables has become dynamic.dynamic.

Effect of dealing with pure Effect of dealing with pure symbols:symbols:

Yet: we do not loose completeness:Yet: we do not loose completeness:If another solution exist, then the current If another solution exist, then the current assignment is ok too.assignment is ok too.+ If unsatisfiable: this will fail too.+ If unsatisfiable: this will fail too.

But not first-fail based.But not first-fail based.

We get a _much_ smaller search space !We get a _much_ smaller search space !

But we are no longer considering all assignments.But we are no longer considering all assignments.

18

1. Unit propagation1. Unit propagation

Example: Example:

SS = { = { X ~Y , ~Y, X } }

~Y and X ~Y and X are unitsare units

A disjunction is A disjunction is unitunit if it only contains one variable Xi. if it only contains one variable Xi.

Assign a unit the value that makes it true (don’t consider the other assignment).Assign a unit the value that makes it true (don’t consider the other assignment).

SAT – with unit propagationSAT – with unit propagationX~Y~Y~W,~XYZ,,~X~XWW

T~Y~Y~W,FYZ,,FFWW

XX==TTF~Y~Y~W,TYZ,,TTWW

XX==FF

YY==TT

FF~W

YY==FF

TT~W

SuccessSuccess

SuccessSuccess

SuccessSuccess

SuccessSuccess

YZ,,TT

WW==TT

YY==TT

TZ,, FZ

YY==FF

ZZ==TT

T

WW==FF

T

unit !unit !

unit !unit !

unit !unit !

20

Many more optimzations in real SAT-solvers!Many more optimzations in real SAT-solvers!

Effect of unit propagationEffect of unit propagation

Obviously: combine both optimizations!Obviously: combine both optimizations!

- Components analysis.- Components analysis.- More Dynamic Search Rearrangement- More Dynamic Search Rearrangement

- Eg.: take most frequent variable firstEg.: take most frequent variable first- - Intelligent backtracking, IndexingIntelligent backtracking, Indexing, …, …

Smaller search space again.Smaller search space again.

But example is not well suited for unit propagation.But example is not well suited for unit propagation.

Marcus as SAT-solving:Marcus as SAT-solving:

man_Marcus, ruler_Caesar, try_assassinate_Marcus_Ceasar, man_Marcus, ruler_Caesar, try_assassinate_Marcus_Ceasar, loyal_to_Marcus_Caesar, loyal_to_Marcus_Caesar, ~man_Marcus ~man_Marcus ~ruler_Ceasar ~ruler_Ceasar ~try_assassinate_Marcus_Ceasar ~try_assassinate_Marcus_Ceasar

~local_to_Marcus_Ceasar, ...~local_to_Marcus_Ceasar, ...

Part of the grounding of Marcus-examplePart of the grounding of Marcus-example plus the negation of the theorem:plus the negation of the theorem:

4 unit propagations: 4 unit propagations: try_assassinate_Marcus_Ceasar = try_assassinate_Marcus_Ceasar = TTman_Marcus = man_Marcus = TTruler_Caesar = ruler_Caesar = TTloyal_to_Marcus_Caesar = loyal_to_Marcus_Caesar = TT

TT,,TT, , TT, , TT, , FF FF FF FF, ..., ...

Fails !Fails !

Theorem Theorem proved.proved.

SAT-solvingSAT-solvingby Local Searchby Local Search

WalkSAT algorithmWalkSAT algorithm

Local Search representation:Local Search representation:

XXi i , i=1,n , i=1,n propositional variablespropositional variables

DDjj, j=1,m , j=1,m the disjunctions in the CNFthe disjunctions in the CNF

States are n-tuplesStates are n-tuples: (T,F,F,T, …,F): (T,F,F,T, …,F)

Objective functionObjective function: the number of : the number of DDjj ‘s that evaluate to F ‘s that evaluate to F

= an interpretation for the = an interpretation for the XXii ‘s ‘s

Find the global minimumFind the global minimum

NeighborsNeighbors: flip one truth value of an : flip one truth value of an XXii in failing in failing DDjj

To avoid local minimaTo avoid local minima: probabilistically do : probabilistically do NotNot take take the best flip the best flip

24

The WalkSAT algorithm:The WalkSAT algorithm:

ForFor i= 1 to i= 1 to Max_flipMax_flip dodo IFIF all Dj’s are true in all Dj’s are true in StateState ThenThen returnreturn StateState;; ElseElse

Disj:= random Dj that is false under Disj:= random Dj that is false under StateState;; With probability With probability PP flip random Xi in Dj; flip random Xi in Dj; ElseElse flip the Xi of Dj that minimizes false Dj’s;flip the Xi of Dj that minimizes false Dj’s;

Max_flipMax_flip:= some number;:= some number;PP:= some probability;:= some probability;S:= { Dj | Dj disjunction};S:= { Dj | Dj disjunction};StateState:= some interpretation for X:= some interpretation for X11, X, X22, ...X, ...Xnn;;

End-ForEnd-For

Report FailureReport Failure

Evaluation:Evaluation:

Unclear whether optimized SAT-solving or Unclear whether optimized SAT-solving or Local Search is better.Local Search is better. but today more people are using local searchbut today more people are using local search

Efficiency of methods depends on underconstraint Efficiency of methods depends on underconstraint versus overconstraintness of problemsversus overconstraintness of problems see Norvig & Russel for a discussionsee Norvig & Russel for a discussion

26

What about Disjunctive Normal What about Disjunctive Normal From?From?

DNF: a dual representation for propositional DNF: a dual representation for propositional formulas:formulas:

Satisfiability of DNF can be checked in linear Satisfiability of DNF can be checked in linear time!time! Find 1 conjunction that does not contain a Find 1 conjunction that does not contain a

variable and the negation of that variable.variable and the negation of that variable.But: conversion to DNF requires exponential time But: conversion to DNF requires exponential time and exponential space !and exponential space !

CNF and DNF are dual: so conversion to CNF is CNF and DNF are dual: so conversion to CNF is also exponential !!!also exponential !!!

(A1 (A1 ... ... An) An) (B1 (B1 … … Bm) Bm) … … (C1 (C1 … … Ck) Ck)

So why do we still prefer CNF ??So why do we still prefer CNF ??Think about it.Think about it.