SAT-Based Model Checking · 1 A Short Intro to Model Checking Structures Properties Symbolic Model...

transcript

Intro to MC Solver Interface Invariants Beyond Safety

SAT-Based Model Checking

Fabio Somenzi

Department of Electrical, Computer, and Energy Engineering

University of Colorado at Boulder

SAT-SMT Summer School, 14 June 2012

Outline

1 A Short Intro to Model CheckingStructuresPropertiesSymbolic Model Checking

2 SAT Solver InterfaceTo The SolverFrom The Solver

3 Checking InvariantsBounded Model CheckingInterpolationProving Invariants by InductionIC3: Incremental Inductive Verification

4 Progress Properties and Branching TimeBounded Model CheckingIncremental Inductive Verification (FAIR)Model Checking CTL

Outline

Simple Synchronous Arbiter (Verilog)

module arbsim (input clock, input [1:2] r, output reg [1:2] g);initial g <= 0;always @ (posedge clock) begin

g[1] <= r[1] & (∼r[2] | g[2]);g[2] <= r[2] & (∼r[1] | ∼g[1]);

endendmodule // arbsim

Mutual Exclusion for the Simple Arbiter

10, 11

1001, 11

01, 11

I (g) = ¬g1 ∧ ¬g2

∃r1, r2 .T (r , g , g ′) = ¬g ′1 ∨ ¬g ′

P(g) = ¬g1 ∨ ¬g2

The Model Checking Question

Given a structure S and a property ϕ, is S a model of ϕ?

Written S |= ϕ

More in detail: does ϕ hold for all computations of S?

From all initial states

Finite-State Transition Systems

Symbolic representation of a system:

S : (i , x , I (x), T (i , x , x ′))

i : primary inputs

x : state variables

x ′: next state variables

I (x): initial states

T (i , x , x ′): transition relation

I and T define a finite transition structure (Kripke structure)

Every valuation of x is a state

∃i .T (i , x , x ′) = T (x , x ′) defines the transitions

Composition

Complex systems are composed of several modules

Each module is described as a finite state structure Si

The overall Kripke structure is obtained as the product of thestructures

State explosion!

The product can be either synchronous or asynchronous(interleaving)

Examples of Temporal Logic Properties

G p: p is invariably true (always along all paths)

p is an atomic propositionG is a temporal operator

F p: p is inevitably true (sometimes true along all paths)

p U q: q eventually holds and p holds up until then

G(p → X q): every p is immediately followed by a q

Only allowed if time is discrete

GF(p → q): if p is persistent, then q is inevitable

Properties

Properties are sets of behaviors

Various specification mechanisms are in use: Temporal logicsand automata are popular

The examples we have seen are formulae of the temporal logicLTL (Linear-Time Logic)

Syntactic sugar often useful (e.g., PSL, Property SpecificationLanguage)

Properties

Linear Time

Linear time logics reason about sets of computation paths

Branching Time

Branching time logics reason about computation trees

5 1 2 2

Invariance, Safety, and Progress

Invariance properties say that certain states are unreachable

Reachability analysis

Safety properties say that certain events never happen

Generalize invariants and can be reduced to them

Progress properties are the non-safety properties

Cycle detection (for finite state systems)

This can be made (a lot) more formal

Why is G(p → X q) a safety property, but G(p → F q) is not?Borel hierarchy, (Landweber [1969])

Automata

Properties can be described by automata that take thecomputation of the system as input and either accept it orreject it

For non-terminating computations and linear-time propertieswe need ω-automata

For linear-time model checking we need the automaton for thenegation of the property of interest

Model checking reduced to checking language emptiness of anω-automaton

Automata

Omega-Automata

ω-automata describe linear-time properties

In fact, nondeterministic Buchi automata recognize allω-regular properties

Examples of Buchi automata (an accepting run visits someaccepting state infinitely often)

They are more expressive than LTL

From Formula to Buchi Automaton

ψUϕ = ϕ ∨ [ψ ∧ X(ψ Uϕ)]

Expansion produces a DNF whose every term is theconjunction of:

1 a propositional formula that must hold now and2 a temporal formula that must hold from the next step

Branching Time Temporal Logic

Add path quantifiers to LTL to obtain CTL∗

A: for all pathsE: for at least one path

AGEF p: resetability

LTL is embedded in CTL∗ by prepending A to all formulae

AG(p → F q)

AG(p → F q) is equivalent to AG(p → AF q), but. . .

AFAG p is not equivalent to A FG p

Maidl [2000] for more info

In CTL every temporal operator must be immediatelypreceded by a path quantifier

AG(p → F q)

Linear vs. Branching Time

Branching time is more powerful, but also trickier

ResetabilityA FGϕ vs. AFAGϕ

Structure equivalence is finer-grained for branching time:

Linear time ↔ language (trace) equivalenceBranching time ↔ simulation relations

Linear time is more suitable for compositional verification andBounded Model Checking

Counterexample generation simpler for linear time

Characteristic Functions

Let V contain all states with either the first or the last bit setto 1

χV = x1 ∨ xn

Set V has 3 · 2n−2 elements

Great, but let us not get carried away, because it is notpossible to find a representation that is compact for mostfunctions

χV = x1 ∨ xn

Symbolic vs. Explicit Algorithms

Symbolic model checking uses characteristic functions

BDDs and CNF are the most popular choices

No explicit loop on the states or the transitions is used

Implicit enumeration

The cost of implicit enumeration is more affected by the sizeof the representation than by the cardinality of the set

Outline

From Hardware Description Language to CNF

From source code to CDFG

From CDFG to formulae over bit vectors and finite-domainvariables

May involve abstraction

Bit-blasting (binary encoding) to Boolean circuit plus memoryelements

Optimization of Boolean circuit

Often uses And-Inverter Graphs (AIGs) or similar datastructures

Conversion of circuit to CNF

Distributivity

Apply (a ∧ b)∨ c = (a ∨ c)∧ (b ∨ c) systematically along withsimplifications

Preserves equivalence and does not introduce new variables

Size may blow up

(a ∧ b) ∨ (c ∧ d) = (a ∨ c) ∧ (a ∨ d) ∧ (b ∨ c) ∧ (b ∨ d)(x1 ∧ x2 ∧ x3) ∨ (x4 ∧ x5 ∧ x6) ∨ · · ·

Seldom applied in its pure form

Distributivity

Size may blow up

Distributivity

Size may blow up

Distributivity

Size may blow up

Equisatisfiability

Two formulae F and G are equisatisfiable if

1 F is satisfiable iff G is satisfiable.

2 If ηF (ηG ) is a satisfying assignment for F (G ), there exists asatisfying assignment ηG (ηF ) for G (F ) that agrees with ηF(ηG ) on all the variables that F and G have in common.

A common case occurs when one of the two formulae, say G ,contains all the variables in the other formula. Then a satisfyingassignment for F can be easily derived from one for G by droppingthe extra variables.

Tseitin

Use definitions for subformulae

f ↔ g ∨ h

g ↔ a ∧ b

h ↔ c ∧ d

Then, from (a ∧ b) ∨ (c ∧ d), we get

(a ∨ ¬g) ∧ (b ∨ ¬g) ∧ (¬a ∨ ¬b ∨ g)

∧ (c ∨ ¬h) ∧ (d ∨ ¬h) ∧ (¬c ∨ ¬d ∨ h)

∧ (¬g ∨ f ) ∧ (¬h ∨ f ) ∧ (g ∨ h ∨ ¬f ) ∧ f

Simpler Equisatisfiable CNF Formulae

If the formula is in negation normal form, Tseitin’s translation canbe simplified (Plaisted and Greenbaum [1986])

f → g ∨ h

g → a ∧ b

h → c ∧ d

Then, from (a ∧ b) ∨ (c ∧ d), we get

(a ∨ ¬g) ∧ (b ∨ ¬g)

∧ (c ∨ ¬h) ∧ (d ∨ ¬h)

∧ (g ∨ h ∨ ¬f ) ∧ f

More Conversions to CNF

Wilson, Sheridan

Nice DAGs

Cut-based

BDD-based

SAT preprocessor

More Conversions to CNF

Wilson, Sheridan

Nice DAGs

Cut-based

BDD-based

SAT preprocessor

Proofs of Unsatisfiability

Different verification techniques require

Resolution proofs

UNSAT cores

Assumptions (unit clauses) in UNSAT cores

Can be extracted with minimal overhead (Een and Sorensson[2003])

Incremental Solving

Solve sequences of related SAT instances

Ability to push and pop clauses (efficiently)

Keep learned clauses that are still valid

All learned clauses remain valid if no clause is popped

Keep variable scores

Multiple solver objects

Outline

Bounded Model Checking

A technique to falsify invariants (“bug finding”)

Based on unrolling the transition relation

Looks for counterexamples of certain lengths

Foundation for complete methods

Checks for a counterexample to a property of a model

We assume finite state

Encodes the property checking problem as propositionalsatisfiability (SAT)

Constructs a propositional formula that is satisfiable iff thereexits a length-k counterexample, e.g.,

I (x0) ∧∧

0≤i<k

T (i i , x i , x i+1) ∧ ¬P(xk)

If no counterexample is found, BMC increases k until

a counterexample is found,the search becomes intractable, ork reaches a certain bound

I (x0) ∧∧

0≤i<k

T (i i , x i , x i+1) ∧ ¬P(xk)

I (x0) ∧∧

0≤i<k

T (i i , x i , x i+1) ∧ ¬P(xk)

I (x0) ∧∧

0≤i<k

T (i i , x i , x i+1) ∧ ¬P(xk)

Proving Properties with BMC

The original BMC algorithm (Biere et al. [1999]), althoughcomplete for finite state, is limited in practice to falsification

BMC can prove that an invariant ψ holds on a model S only ifa bound, κ, is known such that:

if no counterexample of length up to κ is found, then S |= ψ

Several methods exist to compute a suitable κ

The optimum value of κ, however, is usually very expensive toobtain

Finding it is at least as hard as checking whether S |= ψ

(Clarke et al. [2004])

Finding The Bound κ

Compute diameter of graph

Minimum d such that, if there is a path of length d + 1between two states, then there is a path of length at most dbetween the same states∀x0, . . . , xd+1 .

∧0≤i≤d T (x i , x i+1) →

∃x ′0, . . . , x′d .(

∧0≤i<d T (x ′i , x

′i+1)∧x

′0 = x0∧

∨0≤i≤d x

′i = xd+1)

If one end of the path is constrained to an initial (target)state, one obtains the forward (backward) recursive radius ofthe graph

Restrict search to simple paths (next slide)

∧0≤i≤d T (x i , x i+1) →

∃x ′0, . . . , x′d .(

∧0≤i<d T (x ′i , x

′i+1)∧x

′0 = x0∧

∨0≤i≤d x

′i = xd+1)

∧0≤i≤d T (x i , x i+1) →

∃x ′0, . . . , x′d .(

∧0≤i<d T (x ′i , x

′i+1)∧x

′0 = x0∧

∨0≤i≤d x

′i = xd+1)

Simple Paths

A counterexample to an invariant is a finite prefix path to astate that satisfies ¬P (bad state)

If a counterexample exists, then there is a simple path from aninitial state to a bad state that goes through no other initialor bad state

An invariant holds (Sheeran et al. [2000]) if:

there is no counterexample of length k to ¬P , andno simple path of length k + 1 to ¬P that does not gothrough any other states satisfying ¬P , orno simple path of length k + 1 from an initial state that doesnot go through any other initial states

Simple Paths

Checking for Simple Paths

Simple-minded check produces quadratic formula

0<i≤k

0≤j<i

(x i 6= x j)

Using a bitonic sorting network (Kroning and Strichman[2003]) reduces the complexity to O(k log2 k)

Lazy checking is more effective in practice (Sorensson’s thesis)

0<i≤k

0≤j<i

(x i 6= x j)

0<i≤k

0≤j<i

(x i 6= x j)

k-Induction

Sheeran et al. call their method k-induction

If all states on length-k paths from the initial states satisfy p,and

k consecutive states satisfying p are always followed by a statesatisfying p, then

all states reachable from the initial states satisfy p

The second premise is verified when there are no simple pathsof length k + 1

Abstraction Refinement

Assume abstract model Sa and abstraction of property ϕa

such that Sa |= ϕa implies S |= ϕ

Use complete method on abstract model Sa, but use BMC onthe concrete model S when a counterexample is found in Sa

Use the counterexample(s) found in Sa to constrain search in S

If concretization fails, use UNSAT core to refine abstractionOne-to-one and one-to-many concretization possible

It is possible to reverse the order: proof-based abstraction(Amla and McMillan [2004])

Use BMC and periodically extract abstract model fromUNSAT core and check it with complete model

Interpolation (McMillan [2003])

SupposeI (x0) ∧ T (x0, x1) ∧ T (x1, x2) ∧ · · · ∧ T (xk−1, xk) ∧ ¬P(xk) isunsatisfiable

Let F1 = I (x0) ∧ T (x0, x1) andF2 = T (x1, x2) ∧ · · · ∧ T (xk−1, xk) ∧ ¬P(xk)

Then F1(x0, x1) ∧ F2(x1, . . . , xk) is unsatisfiable

Interpolant I1(x1) is such that

F1(x0, x1) → I1(x1)I1(x1) ∧ F2(x1, . . . , xk) is unsatisfiable

I1(x1) can be computed in linear time from a resolution proofthat F1(x0, x1) ∧ F2(x1, . . . , xk) is unsatisfiable

∃x0 . I (x0) ∧ T (x0, x1) is the strongest interpolant

set of states reachable from I (x0) in one step

Interpolation

I1(x1) is a superset of the states reachable in one step suchthat no member state has a path of length k − 1 to a badstate

Replace I (x0) with I (x0) ∨ I1(x0) and repeat

If formula still unsatisfiable, interpolant I2(x1) is a superset ofstates reachable in one or two steps such that no memberstate has a path of length k − 1 to a bad state

A converging sequence of interpolants means that no statessatisfying ¬p (bad states) are reachable

Interpolation

Preimage Computation by Solution Enumeration

Let Pre(Q(x)) be the predicate describing the states that arepredecessors of the states described by Q

Repeated application of Pre from ¬P corresponds tobackward breadth-first search from the error states

Common approach with BDDs

Can be adapted to CNF (McMillan [2002])

Introduced the use of blocking clauses

Back to The Simple Arbiter

10, 11

1001, 11

01, 11

I (g) = ¬g1 ∧ ¬g2

∃r1, r2 .T (r , g , g ′) = ¬g ′1 ∨ ¬g ′

P(g) = ¬g1 ∨ ¬g2

Inductive Proofs for Transition Systems

Prove initiation (base case)

I (x) ⇒ P(x)All initial states satisfy P

(¬g1 ∧ ¬g2) ⇒ (¬g1 ∨ ¬g2)

Prove consecution (inductive step)

P(x) ∧ T (i , x , x ′) ⇒ P(x ′)All successors of states satisfying P satisfy P

(¬g1 ∨ ¬g2) ∧ (¬g ′1 ∨ ¬g ′

2) ⇒ (¬g ′1 ∨ ¬g ′

If both pass, all reachable states satisfy the property

S |= P

Visualizing Inductive Proofs

The inductive assertion (yellow) contains all initial (blue) statesand no arrow leaves it (it is closed under the transition relation)

Counterexamples to Induction: The Troublemakers

00 01 11 10

Counterexamples to Induction: The Troublemakers

00 01 11 10

Invariant Strengthening

00 01 11 10

Strong and Weak Invariants

000 001

011010

Induction is not restricted to:

the strongest inductive invariant (forward-reachable states)

. . . or the weakest inductive invariant (complement of thebackward-reachable states)

¬x1 is simpler than ¬x1 ∧ (¬x2 ∨ ¬x3) (strongest) and(¬x1 ∨ ¬x3) (weakest)

Completeness for Finite-State Systems

CTIs are effectively bad states

If a CTI is reachable so is at least one bad state

Remove CTI from P and try again

Eventually either:

An inductive strengthening of P resultsAn initial state is removed from P

In the latter case, a counterexample is obtained

Examples of Strengthening Strategies

Removing one CTI at a time is very inefficient!

Several strategies in use to avoid that

Fixpoint-based invariant checking: if νZ . p ∧ AXZ convergesin n > 0 iterations, then

∧0≤i<n AX

i p is an inductiveinvariant

In fact, the weakest inductive invariant

k-induction: if all states on length-k paths from the initialstates satisfy p, and k distinct consecutive states satisfying p

are always followed by a state satisfying p, then all statesreachable from the initial states satisfy p.

fsis algorithm: try to extract an inductive clause from CTI toexclude multiple CTIs

Relative Induction

010 000

001011

ϕ = ¬x1 ∧ (x1 ∨ ¬x2)

Relative Induction

010 000

001011

¬x1 is not inductive

Relative Induction

010 000

001011

x1 ∨ ¬x2 is inductive

Relative Induction

010 000

001011

¬x1 is inductive relative to x1 ∨ ¬x2

Shortcoming of Relative Induction

010 100

101011

P = (x1 ∨ x2 ∨ x3) ∧ (¬x1 ∨ ¬x2 ∨ x3)

ϕ = (x1 ∨ x2) ∧ (¬x1 ∨ ¬x2)

010 100

101011

(x1 ∨ x2) ∧ P ∧ T 6⇒ (x ′1 ∨ x ′2)

010 100

101011

(¬x1 ∨ ¬x2) ∧ P ∧ T 6⇒ (¬x ′1 ∨ ¬x ′2)

010 100

101011

(x1 ∨ x2) ∧ (¬x1 ∨ ¬x2) ∧ P ∧ T ⇒ (x ′1 ∨ x ′2) ∧ (¬x ′1 ∨ ¬x ′2)

010 100

101011

(x1 ∨ x2) and (¬x1 ∨ ¬x2) are mutually inductive

IC3: Basic Algorithm

IC3 (Bradley [2011]) stands for

1 Incremental Construction of

2 Inductive Clauses for

3 Indubitable Correctness

IC3 is an Incremental Inductive Verification (IIV) algorithm

Basic Tenets

Approximate reachability assumptions

Fi : contains at least all the states reachable in i steps or lessIf S |= P , Fi eventually becomes inductive for some i

Approximation is desirable: IC3 does not attempt to get themost precise Fi ’s

Stepwise relative induction

Learn useful facts via induction relative to reachabilityassumptions

Clausal representation

Learn clauses (lemmas) from CTIsA form of abstract interpretation

IC3 Invariants

The four main invariants of IC3:

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

Established if there are no counterexamples of length 0 or 1

The implicit invariant of the outer loop: no counterexamplesof length k or less

Reasonable Invariants

I ⇒ F0: F0 overapproximates the initial condition. (Inpractice, I = F0.)

Fi ⇒ Fi+1: a state believed to be reachable in i steps or lessis also believed to be reachable in i + 1 steps or less

Fi ⇒ P : no state believed to be reachable in i steps or lessviolates P

Fi ∧ T ⇒ F ′i+1: all the immediate successors of a state

believed to be reachable in i steps or less are believed to bereachable in i + 1 steps or less

Pseudo-Pseudocode

bool IC3 {if (I 6⇒ P or I ∧ T 6⇒ P ′)

return ⊥F0 = I ; F1 = P ; k = 1repeat {

while (there are CTIs in Fk) {either find a counterexample and return ⊥or refine F1, . . . ,Fk

}k ++set Fk = P and propagate clausesif (Fi = Fi+1 for some 0 < i < k)

return ⊤}

Example: Passing Property

No counterexamples of length 0 or 1

00 01 11 10I = ¬x1 ∧ ¬x2

P = ¬x1 ∨ x2

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

Does F1 ∧ T ⇒ P ′?

00 01 11 10F0 = I = ¬x1 ∧ ¬x2

F1 = P = ¬x1 ∨ x2

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

Found CTI s = x1 ∧ x2

00 01 11 10F0 = I = ¬x1 ∧ ¬x2

F1 = P = ¬x1 ∨ x2

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

Is ¬s = ¬x1 ∨ ¬x2 inductive relative to F1?

00 01 11 10F0 = I = ¬x1 ∧ ¬x2

F1 = P = ¬x1 ∨ x2

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

No. Is ¬s = ¬x1 ∨ ¬x2 inductive relative to F0?

00 01 11 10F0 = I = ¬x1 ∧ ¬x2

F1 = P = ¬x1 ∨ x2

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

Yes. Generalize ¬s at level 0 in one of the two possible ways: either¬x1 or ¬x2

00 01 11 10F0 = I = ¬x1 ∧ ¬x2

F1 = P = ¬x1 ∨ x2

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

Update F1

00 01 11 10F0 = I = ¬x1 ∧ ¬x2

F1 = (¬x1 ∨ x2) ∧ ¬x2

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

No more CTIs in F1. No counterexamples of length 2. InstantiateF2

00 01 11 10F0 = I = ¬x1 ∧ ¬x2

F1 = (¬x1 ∨ x2) ∧ ¬x2

F2 = P = ¬x1 ∨ x2

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

Propagate clauses from F1 to F2

00 01 11 10F0 = I = ¬x1 ∧ ¬x2

F1 = (¬x1 ∨ x2) ∧ ¬x2

F2 = (¬x1 ∨ x2) ∧ ¬x2

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

F1 and F2 are identical. Property proved

00 01 11 10F0 = I = ¬x1 ∧ ¬x2

F1 = (¬x1 ∨ x2) ∧ ¬x2

F2 = (¬x1 ∨ x2) ∧ ¬x2

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

What happens if we generalize ¬s = ¬x1 ∨ ¬x2 at level 0 in theother way (¬x1)?

00 01 11 10F0 = I = ¬x1 ∧ ¬x2

F1 = ¬x1 ∨ x2

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

Update F1

00 01 11 10F0 = I = ¬x1 ∧ ¬x2

F1 = (¬x1 ∨ x2) ∧ ¬x1

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

No more CTIs in F1. No counterexamples of length 2. InstantiateF2

00 01 11 10F0 = I = ¬x1 ∧ ¬x2

F1 = (¬x1 ∨ x2) ∧ ¬x1

F2 = P = ¬x1 ∨ x2

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

No clauses propagate from F1 to F2

00 01 11 10F0 = I = ¬x1 ∧ ¬x2

F1 = (¬x1 ∨ x2) ∧ ¬x1

F2 = P = ¬x1 ∨ x2

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

Remove subsumed clauses

00 01 11 10F0 = I = ¬x1 ∧ ¬x2

F1 = ¬x1

F2 = P = ¬x1 ∨ x2

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

Does F2 ∧ T ⇒ P ′?

00 01 11 10F0 = I = ¬x1 ∧ ¬x2

F1 = ¬x1

F2 = P = ¬x1 ∨ x2

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

Found CTI s = x1 ∧ x2 (same as before)

00 01 11 10F0 = I = ¬x1 ∧ ¬x2

F1 = ¬x1

F2 = P = ¬x1 ∨ x2

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

Is ¬s = ¬x1 ∨ ¬x2 inductive relative to F1?

00 01 11 10F0 = I = ¬x1 ∧ ¬x2

F1 = ¬x1

F2 = P = ¬x1 ∨ x2

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

No. We know it is inductive at level 0.

00 01 11 10F0 = I = ¬x1 ∧ ¬x2

F1 = ¬x1

F2 = P = ¬x1 ∨ x2

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

If generalization produces ¬x1 again, the CTI is not eliminated

00 01 11 10F0 = I = ¬x1 ∧ ¬x2

F1 = ¬x1

F2 = P = ¬x1 ∨ x2

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

Find predecessor t of CTI x1 ∧ x2 in F1 \ F0

00 01 11 10F0 = I = ¬x1 ∧ ¬x2

F1 = ¬x1

F2 = P = ¬x1 ∨ x2

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

Found t = ¬x1 ∧ x2

00 01 11 10F0 = I = ¬x1 ∧ ¬x2

F1 = ¬x1

F2 = P = ¬x1 ∨ x2

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

The clause ¬t = x1 ∨ ¬x2 is inductive at all levels

00 01 11 10F0 = I = ¬x1 ∧ ¬x2

F1 = ¬x1

F2 = P = ¬x1 ∨ x2

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

Generalization of ¬t = x1 ∨ ¬x2 produces ¬x2

00 01 11 10F0 = I = ¬x1 ∧ ¬x2

F1 = ¬x1

F2 = P = ¬x1 ∨ x2

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

Update F1 and F2

00 01 11 10F0 = I = ¬x1 ∧ ¬x2

F1 = ¬x1 ∧ ¬x2

F2 = (¬x1 ∨ x2) ∧ ¬x2

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

F1 and F2 are equivalent. Property (almost) proved

00 01 11 10F0 = I = ¬x1 ∧ ¬x2

F1 = ¬x1 ∧ ¬x2

F2 = (¬x1 ∨ x2) ∧ ¬x2

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

Example: Failing Property

No counterexamples of length 0 or 1

I = ¬x1 ∧ ¬x3 ∧ ¬x3

P = ¬x1 ∨ ¬x2 ∨ ¬x3

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

Does F1 ∧ T ⇒ P ′?

F0 = I = ¬x1 ∧ ¬x3 ∧ ¬x3

F1 = P = ¬x1 ∨ ¬x2 ∨ ¬x3

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

Found CTI s = ¬x1 ∧ x2 ∧ x3

F0 = I = ¬x1 ∧ ¬x3 ∧ ¬x3

F1 = P = ¬x1 ∨ ¬x2 ∨ ¬x3

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

The clause ¬s = x1 ∨ ¬x2 ∨ ¬x3 generalizes to ¬x2 at level 0

F0 = I = ¬x1 ∧ ¬x3 ∧ ¬x3

F1 = (¬x1 ∨ ¬x2 ∨ ¬x3) ∧ ¬x2

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

No CTI left: no counterexample of length 2. F2 instantiated, butno clause propagated

F0 = I = ¬x1 ∧ ¬x3 ∧ ¬x3

F1 = ¬x2

F2 = P = ¬x1 ∨ ¬x2 ∨ ¬x3

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

The clause ¬s = x1 ∨ ¬x2 ∨ ¬x3 generalizes again to ¬x2 at level 0

F0 = I = ¬x1 ∧ ¬x3 ∧ ¬x3

F1 = ¬x2

F2 = P = ¬x1 ∨ ¬x2 ∨ ¬x3

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

Suppose IC3 recurs on t = ¬x1 ∧ ¬x2 ∧ x3 in F1 \ F0

F0 = I = ¬x1 ∧ ¬x3 ∧ ¬x3

F1 = ¬x2

F2 = P = ¬x1 ∨ ¬x2 ∨ ¬x3

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

Clause ¬t = x1 ∨ x2 ∨ ¬x3 is not inductive at level 0: the propertyfails

F0 = I = ¬x1 ∧ ¬x3 ∧ ¬x3

F1 = ¬x2

F2 = P = ¬x1 ∨ ¬x2 ∨ ¬x3

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

Suppose now IC3 recurs on t = x1 ∧ ¬x2 ∧ x3 in F1 \ F0

F0 = I = ¬x1 ∧ ¬x3 ∧ ¬x3

F1 = ¬x2

F2 = P = ¬x1 ∨ ¬x2 ∨ ¬x3

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

Clause ¬t = ¬x1 ∨ x2 ∨ ¬x3 is inductive at level 1

F0 = I = ¬x1 ∧ ¬x3 ∧ ¬x3

F1 = ¬x2

F2 = P = ¬x1 ∨ ¬x2 ∨ ¬x3

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

Generalization of ¬t adds ¬x1 to F1 and F2

F0 = I = ¬x1 ∧ ¬x3 ∧ ¬x3

F1 = ¬x2 ∧ ¬x1

F2 = (¬x1 ∨ ¬x2 ∨ ¬x3) ∧ ¬x1

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

Only t = ¬x1 ∧ ¬x2 ∧ x3 remains in F1 \ F0

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

The same counterexample as before is found

I ⇒ F0

Fi ⇒ Fi+1 0 ≤ i < k

Fi ⇒ P 0 ≤ i ≤ k

Fi ∧ T ⇒ F ′i+1 0 ≤ i < k

Clause Generalization

A CTI is a cube (conjunction of literals)

e.g., s = x1 ∧ ¬x2 ∧ x3

The negation of a CTI is a clause

e.g., ¬s = ¬x1 ∨ x2 ∨ ¬x3

Conjoining ¬s to a reachability assumption Fi excludes theCTI from it

Generalization extracts a subclause from ¬s that excludesmore states that are “like the CTI”

e.g., ¬x3 may be a subclause of ¬s that excludes states that,like the CTI, are not reachable in i stepsEvery literal dropped doubles the number of states excluded bya clauseGeneralization is time-consuming, but critical to performance

Generalization

Crucial for efficiency

Generalization in IC3 produces a minimal inductive clause(MIC)

The MIC algorithm is based on DOWN and UP.

DOWN extracts the (unique) maximal subclause

UP finds a small, but not necessarily minimal subclause

MIC recurs on subclauses of the result of UP

Minimal Inductive Clause

123 124

13 14 23 24

123 124

13 14 23 24

123 124

13 14 23 24

123 124

13 14 23 24

123 124

13 14 23 24

Maximal Inductive Subclause (DOWN)

¬x1 ∨ x2 ∨ ¬x3

x2 ∨ ¬x3

Use of UNSAT Cores

¬s ∧ Fi ∧ T ⇒ ¬s ′ if and only if ¬s ∧ Fi ∧ T ∧ s ′ isunsatisfiable

The literals of s ′ are (unit) clauses in the SAT query

If the implication holds, the SAT solver returns anunsatisfiable core

Any literal of s ′ not in the core can be removed from s ′

because it does not contribute to the implication . . .

and from ¬s because strengthening the antecedent preservesthe implication

Use of UNSAT Core Example

¬s ∧ F0 ∧ T ⇒ ¬s ′ with

¬s = ¬x1 ∨ ¬x2

F0 = ¬x1 ∧ ¬x2

T = (¬x1 ∧ ¬x2 ∧ ¬x ′1 ∧ ¬x ′2) ∨ · · ·

The SAT query, after some simplification, is

¬x1 ∧ ¬x2 ∧ ¬x ′1 ∧ ¬x ′2 ∧ x ′1 ∧ x ′2

Two UNSAT cores are

¬x ′1 ∧ x ′1

¬x ′2 ∧ x ′2

from which the two generalizations we saw before follow

Clause Clean-Up

As IC3 proceeds, clauses may be added to some Fi thatsubsume other clauses

The weaker, subsumed clauses no longer contribute to thedefinition of Fi

However, a weaker clause may propagate to Fi+1 when thestronger clause does not

Weak clauses are eliminated by subsumption only betweenmajor iterations and after propagation

More Efficiency-Related Issues

State encoding determines what clauses are derived

Incremental vs. monolithic

Reachability assumptions carry global information. . . but are built incrementally

Semantic vs. syntactic approach

Generalization “jumps over large distances”

Long counterexamples at low k

Typically more efficient than increasing k

Consequences of no unrolling

Many cheap (incremental) SAT calls

Ability to parallelize

Clauses are easy to exchange

Outline

BMC: Translation from LTL

Various techniques have been devised to translate an LTLformula ϕ into a propositional formula that expresses theconstraints on a path that is a model of ¬ϕ. For instance:

[[¬FG¬p]] =∨

0≤l≤k

(T (xk , x l) ∧∨

l≤i≤k

p(x i))

k-induction can be extended to provide a termination criterion

BMC: Translation from LTL

Various techniques have been devised to translate an LTLformula ϕ into a propositional formula that expresses theconstraints on a path that is a model of ¬ϕ. For instance:

[[¬FG¬p]] =∨

0≤l≤k

(T (xk , x l) ∧∨

l≤i≤k

p(x i))

k-induction can be extended to provide a termination criterion

BMC: Liveness to Safety

Checking progress properties requires cycle detection

Augment model with shadow register

The augmented model can nondeterministically save asnapshot of the current state in the shadow register

If a state is subsequently reached that is identical to the onesaved, a cycle has been detected

Constraints can be added to make sure the cycle is anaccepting one

With this transformation an invariant checker suffices for allLTL properties

FAIR: Finding Rechable Fair Cycles

Check language nonemptiness of the composition of structureS and generalized Buchi automaton for ¬ϕ

Generalized means that multiple acceptance conditions (akafairness constraints may be given: each must be satisfied

FAIR (Bradley et al. [2011]) looks for a reachable fair cycle

Strongly Connected Components

A counterexample to a progress property is a lasso-shapedpath that satisfies fairness constraints

A lasso’s cycle is contained in a strongly connectedcomponent (SCC) of the state graph

A nonempty set of states is SCC-closed if every SCC is eithercontained in it or disjoint from it

A partition of the states into SCC-closed sets is a coarserpartition than the SCC partition; hence, . . .

Every cycle of a graph is contained in some SCC-closed set

Maintain a partition of the states into SCC-closed set

Refine it until a reachable fair cycle is found or none is provedto exist

FAIR: Finding Reachable Fair Cycles

Reduce search for reachable fair cycle to a set of safety problems:

Skeleton:•

◦ •

States of skeleton together satisfy all fairness constraints.

Task: Connect states to form lasso.•

◦ •

Reach Queries

Each connection task is a reach query.

Stem query: Connect initial condition to a state:

◦ •

Cycle query: Connect one state to another:

◦ •

(To itself if skeleton has only one state.)

Witness to Nonemptiness

If all queries are answered positively:

◦ •

Witness to nonemptiness of C.

Global Reachability

If a stem query is answered negatively: new inductive globalreachability information.

◦ •

Constrains subsequent selection of skeletons.

Constrains subsequent reach (stem and cycle) queries.

Improve proof by strengthening (using ideas from IC3).

Barriers: Discovering SCC-Closed Sets

If a cycle query is answered negatively: new information aboutSCC structure of state graph.

◦ •

Inductive proof: “one-way barrier”

Each “side” of the proof is SCC-closed.

Constrains subsequent selections of skeletons: all states onone side.

Example: Empty Language

000 001

101100

s0 s1sk1 010 110

000 001

101100

010 011

111110

s0 s1sk1 010 110

000 001

101100

010 011

111110

stem query produces x1 ∨ ¬x2

000 001

101100

s0 s1sk2 101 110

states satisfyx1 ∨ ¬x2

000 001

101100

s0 s1sk2 101 110

000 001

101100

stem query passes

s0 s1sk2 101 110

000 001

101100

reach(S , (x1 ∨ ¬x2), s0, s1) passes

s0 s1sk2 101 110

000 001

101100

reach(S , (x1 ∨ ¬x2), s1, s0) produces x2

s0 s1sk2 101 110

000 001

101100

because x1 ∧ x2 ∧ ¬x3 ⇒ x2 . . .

s0 s1sk2 101 110

000 001

101100

and x2 ∧ (x1 ∨ ¬x2) ∧ T ⇒ x ′2

000 001

101100

s0 s1sk3 101 100

states satisfy(x1 ∨ ¬x2) ∧ ¬x2

000 001

101100

s0 s1sk3 101 100

000 001

101100

stem query passes

s0 s1sk3 101 100

000 001

101100

reach(S , (x1 ∨ ¬x2) ∧ ¬x2, s0, s1) produces x2 ∨ x3

no skeletons left000 001

101100

Key Insights

Inductive assertions describe SCC-closed sets.

Arena: Set of states all on the same side of each barrier.

Unlike previous symbolic methods:

Barrier constraints on the transition relationcombined with the over-approximating nature ofIC3 enable the simultaneous (symbolic)consideration of all arenas.

A proof can provide information about many arenas eventhough the motivating skeleton comes from one arena.

Methodological Parallels with IC3

IC3 FAIR

Seed: CTI Skeleton

Lemma: Inductive clause Global reachability proofOne-way barrier

Relative to previously discovered lemmas.

CEX: CTI sequence Connected skeletonDiscovery guided by lemmas. Not minimal.

Proof: Inductive strengthening All arenas skeleton-freeSufficient set of lemmas.

IICTL: Incremental Inductive CTL Model Checking

Task-directed strategy

Maintains upper and lower bounds on states satisfying eachsubformula

States in between the bounds are undecided

Typically don’t need to decide all states to decide the property(Traditional symbolic CTL algorithms do)

Decide states by executing appropriate query:

EX: SAT queryEU: Safety model checker (e.g., IC3)EG: Fair cycle finder (e.g., FAIR)

Generalizing decisions (proofs or counterexamples) to otherstates and refining the bounds

IICTL Example

Property: AGEF p = ¬EF¬EFp

IICTL Example

[[EF p]]

IICTL Example

[[EF p]]

IICTL Example

[[EF¬EF p]]

[[EF p]]

IICTL Example

[[EF¬EF p]]

[[EF p]]

IICTL Example

⊇ initial states?[[EF¬EF p]]

[[EF¬EF p]]

[[EF p]]

IICTL Example

Yes: Property holdsNo: Property fails

⊇ initial states?[[EF¬EF p]]

[[EF¬EF p]]

[[EF p]]

IICTL Example

I ∧ ¬U0?⊤

IICTL Example

No: Property failsI ∧ ¬U0?⊤

IICTL Example

YesI ∧ ¬U0?⊤

IICTL Example

I ∧ ¬L0?

YesI ∧ ¬U0?⊤

IICTL Example

Yes: Property holdsI ∧ ¬L0?

YesI ∧ ¬U0?⊤

IICTL Example

NoI ∧ ¬L0?

YesI ∧ ¬U0?⊤

IICTL Example

s 6|= L0NoI ∧ ¬L0?

YesI ∧ ¬U0?⊤

IICTL Example

s |= U0

s 6|= L0NoI ∧ ¬L0?

YesI ∧ ¬U0?⊤

IICTL Example

s is undecided for node 0s |= U0

s 6|= L0NoI ∧ ¬L0?

YesI ∧ ¬U0?⊤

IICTL Example

s s is undecided for node 0s |= U0

s 6|= L0NoI ∧ ¬L0?

YesI ∧ ¬U0?⊤

IICTL Example

s 6|= L0NoI ∧ ¬L0?

YesI ∧ ¬U0?⊤

IICTL Example

s |= ψ1?s

s 6|= L0NoI ∧ ¬L0?

YesI ∧ ¬U0?⊤

IICTL Example

⇐⇒ s |= EFψ2?s |= ψ1?s

s 6|= L0NoI ∧ ¬L0?

YesI ∧ ¬U0?⊤

IICTL Example

⇐⇒ can s reach ψ2?⇐⇒ s |= EFψ2?s |= ψ1?s

s 6|= L0NoI ∧ ¬L0?

YesI ∧ ¬U0?⊤

IICTL Example

can s reach L2?⇐⇒ can s reach ψ2?⇐⇒ s |= EFψ2?s |= ψ1?s

s 6|= L0NoI ∧ ¬L0?

YesI ∧ ¬U0?⊤

IICTL Example

Yes: s can also reach ψ2can s reach L2?⇐⇒ can s reach ψ2?⇐⇒ s |= EFψ2?s |= ψ1?s

s 6|= L0NoI ∧ ¬L0?

YesI ∧ ¬U0?⊤

IICTL Example

Nocan s reach L2?⇐⇒ can s reach ψ2?⇐⇒ s |= EFψ2?s |= ψ1?s

s 6|= L0NoI ∧ ¬L0?

YesI ∧ ¬U0?⊤

IICTL Example

can s reach U2?

s 6|= L0NoI ∧ ¬L0?

YesI ∧ ¬U0?⊤

IICTL Example

No: s cannot reach ψ2can s reach U2?

s 6|= L0NoI ∧ ¬L0?

YesI ∧ ¬U0?⊤

IICTL Example

Yescan s reach U2?

s 6|= L0NoI ∧ ¬L0?

YesI ∧ ¬U0?⊤

IICTL Example

t |= U2Yescan s reach U2?

s 6|= L0NoI ∧ ¬L0?

YesI ∧ ¬U0?⊤

IICTL Example

t 6|= L2

s 6|= L0NoI ∧ ¬L0?

YesI ∧ ¬U0?⊤

IICTL Example

t is undecided for node 2t 6|= L2

Nocan s reach L2?s

s 6|= L0NoI ∧ ¬L0?

YesI ∧ ¬U0?⊤

IICTL Example

Nocan s reach L2?s

s 6|= L0NoI ∧ ¬L0?

YesI ∧ ¬U0?⊤

IICTL Example

Nocan s reach L2?s

s 6|= L0NoI ∧ ¬L0?

YesI ∧ ¬U0?⊤

IICTL Example

can t reach L4 (or U4)?t

Nocan s reach L2?s

s 6|= L0NoI ∧ ¬L0?

YesI ∧ ¬U0?⊤

IICTL Example

p ∨ tcan t reach L4 (or U4)?

Nocan s reach L2?s

s 6|= L0NoI ∧ ¬L0?

YesI ∧ ¬U0?⊤

IICTL Example

¬p ∧ ¬t

Nocan s reach L2?s

s 6|= L0NoI ∧ ¬L0?

YesI ∧ ¬U0?⊤

IICTL Example

¬p ∧ ¬t

Nocan s reach L2?s

s 6|= L0NoI ∧ ¬L0?

YesI ∧ ¬U0?⊤

IICTL Example

¬p ∧ ¬t

Nocan s reach L2?s

s 6|= L0NoI ∧ ¬L0?

YesI ∧ ¬U0?⊤

IICTL Algorithm

1 Construct the parse-graph of the formula

2 Initialize bounds

3 Are all initial states in lower bound of root node?Yes: property holds

4 Is any of the initial states not in upper bound of root?Yes: property fails

5 There is an undecided state s. Decide s recursively andgeneralize.

6 Repeat step 3

SAT-Based Model Checking · 1 A Short Intro to Model Checking Structures Properties Symbolic Model...

Documents