Survey Propagation

Outline

• Survey Propagation: an algorithm for satisfiability1

– Warning Propagation– Belief Propagation– Survey Propagation

• Survey Propagation Revisited2

– SP’s relation to BP– Covers

1 Braunstein, A., Mézard, M., and Zecchina, R. 2005. Survey propagation: An algorithm for satisfiability. Random Struct. Algorithms 27, 2 (Sep. 2005), 201-226.2 Kroc, L.; Sabharwal, A.; and Selman, B. 2007. Survey propagation revisited. In UAl 2007.

Survey Propagation and Message Passing

• New threshold defining two regions of solutions, easy SAT and hard SAT– Easy SAT region has many, highly-clustered solutions,

and any two solutions are connected by a few number of steps

– Hard SAT region has many clusters of solutions, with fewer overall solutions than easy SAT

• SP is an efficient way of computing solutions in the Hard SAT region

New Model - Factor Graph

Message Passing

• Treat the Factor Graph as a network• Depending on the current state of the

network, messages will be passed between the nodes

• These messages will later be used to determine the final value of a variable

Cavity Fields

• Used to determine the value of a given message, and varies depending on the actual algorithm in use

• Calculated by removing a function node from consideration

Warning Propagation

• Messages are from the set {0,1}• Passed from Function nodes to Variable Nodes

– Product of cavity fields

• Initially randomly generate messages• Calculate new warnings for edges until we finish

the time, or we hit a fixed point for all edges– New warnings are calculated from the cavity fields

(∑b Є V+(j)\a ub→j) – (∑b Є V-(j)\a ub→j)

WP Example

Local Field

Contradiction Numbers

The whole algorithm is an iterative process involving multiple WP executions, simplifying the graph using local fields

Belief Propagation

• Finds the total number of solutions, as well as the fraction of assignments with variable Xi is true

• Two types of messages:– From Function node to Variable node

• Given a value for Xi, the probability that it satisfies the function clause

– From Variable node to Function node• Probability that variable i takes value Xi in the absence

of function clause a

Satisfiable and Unsatisfiable Sets

BP Example

Blue is the cavity field of the variable node, red is the message from function to variable. Feed these values into another function to compute probabilities of being true

WP and BP

• Work 100% of the time when the factor graph is a tree

• Can be used as heuristics when it is not a tree, but no guarantee on solution

• Goal: Create a more efficient scheme for the general case

Basics of Survey Propagation• Generally the same as BP, but deals better with

clusters• Details of the message change

– A message sent from function to variable is the probability that a warning is sent to the variable node

– A warning is sent if all other variable nodes connected to the function do not satisfy the function.

• Note: Behaves exactly like WP on tree factor graphs

Clustering of Configurations

SAT Assignments

The problem with BP’s approximations is that they do not hold globally, but do hold if you restrict the computation to a specific cluster. SP fixes this.

SP Results with Don’t Care State

Testing Data

Results

• New way to solve SAT using cavity fields and message passing

• Works well on generic input, as well as the tree structure

• Has not been rigorously examined, but is based off of similar work

Why does SP Work?

• As of 2008, SP is the only method of solving SAT problems with 1,000,000 variables (and larger)

• Does so in near linear time (even in the hardest region)

• SP behaves like backtrack searches, except it almost never has to backtrack

SP’s relation to BP

• BP works well when the number of SAT clusters is low

• SP seems to work well ALL the time

• Work in 2005 shows that SP is equivalent to BP search over a new object, covers

• This is a generalization of the clusters described previously

Covers

σ Є {0,1,*}ⁿ is a cover of F if:

1. Every clause has at least one satisfying literal, or two *-ed literals

2. No unsupported variables are assigned 0 or 1

• X =

• Y =

• Z =

(x V ¬y V ¬z)Λ(¬x V y V ¬z) Λ(¬x V ¬y V z)

Computing Covers

• New Factor Graphs:

Solving SAT using Covers

Let P(F) be a cover of F:The factor graph of P(F) can bee seen as a Bayesian Network, so marginals can be computed using Bayes’ Theorem

These marginals, when fed to the BP algorithm, computes a solution for the original SAT problem (and results in the SP algorithm).

The SP algorithm has been shown to compute marginals over these covers.

SP, BP and Marginals

Magnetization of a Variable: The difference between the marginal being positive vs. negative

Results

• The reason why SP works so well seems to be related to these covers

• These covers seem to expose hidden properties of a SAT instance, such as backbone variables

Interesting Aside

• SAT and Decision SAT are self-reducible

• Finding covers is NOT self-reducible

(x V ¬y V ¬z)Λ(¬x V y V ¬z) Λ(¬x V ¬y V z)

(y V ¬z) Λ(¬y V z)

(¬z)

x = 1? Oracle: yes

y = 0? Oracle: yes

(1,0,0) a cover of the original? No!