Model Checking with Proofs and Counterexamples

transcript

Model Checking withProofs and Counterexamples

Anvesh Komuravelli

Carnegie Mellon University

Joint work with Arie Gurfinkel, Sagar Chaki, and Edmund Clarke

Safety of Programs

Program P

+ Safety Assertions

Automatic verification for

assertion failures

Safe + Proof

Unsafe + Counterexample

0. x := 0;1. while (x < n) {

2. x := x + 1; }3. x := -x;4. assert (x ≤ 0)

// x ≥ 0

Loop Invariant

Safety of Programs

Program P

+ Safety Assertions

Automatic verification for

assertion failures

Safe + Proof

Unsafe + Counterexample

While-programs(unbounded variables,

no procedure calls,no memory operations)

0. x := 0;1. while (x < n) {2. x := x + 1; }3. x := -x;4. assert (x ≤ -1)

Counterexample Trace:pc=0. x=0, n=0pc=1. else branchpc=3. x=0pc=4. error

Algorithms for Safety

1. Safety is undecidable!• Reduction from Halting Problem to safety of a 2-counter machine

2. Existing algorithms use heuristics for verifying many programs in practice

3. Two broad classes of model checking algorithms:A. Generalize feasible and safe behaviors (Proof-Based)B. Eliminate infeasible and unsafe behaviors (Counterexample-based)

This talk: Improve (A) based on ideas from (B)

Backgroundon Proof-Based algorithms

should never hold

Transition System

Programs as Transition Systems

Variables

Init condition Transition relation

Error condition

0. x := 0;1. while (x < n) {2. x := x + 1; }3. x := -x;4. assert (x ≤ 0)

encodes how data and controlchange after every instruction

SAT-Based Model Checking

Counterexample of length 0?Counterexample of length 1?

SAT?SAT?

…Bounded Model Checking, Clarke et al., TACAS 1999

Transition System

1. Boolean SAT is NP-complete, but we have efficient solvers today2. SAT modulo theories (SMT) for handling arithmetic, etc.3. Eg: is unsatisfiable for integers x, y

…Bounded Model Checking, Clarke et al., TACAS 1999

Transition System

No upper bound on the length of a counterexample!Even for finite-state systems, the upper bound can be huge!

When do we stop?

Are initial states safe?

Are 1 step-reachable states safe?

Keep track of thereachable states!

Counterexample of length 0?Counterexample of length 1?

SAT?SAT?

Keep Track of the Reachable States

err(x)

reach(P)

Initial States

States reachable in

≤1 steps

States reachable in

≤2 steps

Usually Hopeless!

Reachable states can be diverging!

reachable statesat (pc=1)

n is a symbolic

(diverging)

converged!

err(x)

Generalize

(Heuristics usingCraig Interpolation[1,2])

Generalize the reachable states!

[1] McMillan, Interpolation and SAT-Based Model Checking, CAV 2003[2] McMillan, Lazy Abstraction with Interpolants, CAV 2006

err(x)

Generalize the reachable states!

reach(P)

Proofs and Invariants

reachable statesat (pc=1)

(diverging)

x ≥ 0 is aloop invariant!

The actual set of reachable states

is stronger:0 ≤ x ≤ n

Proof of SafetyProof of

“Bounded” Safety

Many heuristics for generalizations!

err(x)

• No unique generalization!• Today’s best algorithms for hardware verification are SAT-based• Several competitive algorithms exist for software

One possible generalization

Another possible

generalization

The ProblemGeneralizations are not always sufficient

Generalizations can suffer from local view

x = y = z = w = 0;while (*) {

// loop invariant:// (x ≥ 4 => y ≥ 100) && (z ≤ 10w)if (*) {x++; y += 100;}else if (*)

if (x ≥ 4) {x++; y++;}else if (y > 10w && z ≥ 100x) {

y = −y;}t = 1;w += t; z += 10t;

}assert (!(x ≥ 4 && y ≤ 2))

State-of-the-art Tool Z3cannot verify in an hour

Source: Automatically Refining Abstract Interpretations, Gulavani, Chakraborty, Nori and Rajamani, TACAS ‘08.

Proofs ofBounded Safety never connect

z and w

Abstractions for better generalizations!

x = y = z = w = 0;while (*) {

// loop invariant:// (x ≥ 4 => y ≥ 100) && (z ≤ 10w)if (*) {x++; y += 100;}else if (*)

y = −y;}t = 1;w += t; z += 10t;

}assert (!(x ≥ 4 && y ≤ 2))

Verifies the abstractionin < 1 sec.

t = *;

Abstractions only add behaviors

How to obtain helpful abstractions automatically?

1. An abstraction of the program can dramatically improve generalizations!

2. How to obtain helpful abstractions?

3. How to efficiently and automatically maintain abstractions?

Spurious counterexample

CounterExample-Guided Abstraction

Refinement(CEGAR)[1]

err(x)

[1] Clarke et al., Counterexample-Guided Abstraction Refinement, CAV 2000.

reach(P)

Abstractions are great, but not always!

Reachable states of an abstraction

The second class of

algorithms

Our algorithm Spacer

Spacer(Software Proof-based Abstraction with CounterExample-based Refinement)

Program

Fix a Bound

Check Safety

Feasible?Invariants?

Abstract Refine

Proof-Based Abstraction CEGARNo No

Yes Yes

Safety Proof Counterexample

Program

Fix a Bound

Check Safety

Abstract Refine

Yes Yes

Proofs from Abstractions

Program

Fix a Bound

Check Safety

Abstract Refine

Yes Yes

Refinement using Spurious

Counterexamples

Program

Fix a Bound

Check Safety

Abstract Refine

Yes Yes

Proof-Based Abstraction

Program

Fix a Bound

Check Safety

Abstract Refine

Yes Yes

Initial Statesof P Generalization/Proof

err(x)

reach(P)

reach(A1)

Spacer at a high level

Initial Statesof A1

Abstraction

err(x)

reach(P)

reach(A1)

Reachable states of A1 in ≤1 steps

Spurious counterexample

err(x)

reach(P)

reach(A2)

Refine A1 to A2 eliminating the

spurious counterexample

Generalization/Proof Reachable states

of P in ≤1 steps

err(x)

reach(P)

reach(A3)

Reachable states of P in ≤1 steps

Fresh abstraction,to avoid bias

Key Ideas of Spacer

1. Abstractions help obtain (hopefully) more general proofs

2. First integration of Proof-Based Abstraction with SAT/SMT-Based Model Checking

3. Orthogonal to heuristics for Interpolation/Generalization

Implementation and Experimental Evidence on C Programs

Abstractions add a new dimension

……

Abstract

SAT-Based Model Checking with Abstractions

……

Under-approximations

Abstract

need not be monotonic

SAT-Based Model Checking with Abstractions

……

Under-approximations

Abstract

non-trivial abstraction

Program

Fix a Bound

Check Safety

Abstract Refine

Yes Yes

Spacer on Example

x = y = z = w = 0;c = 0;

while (*) {// (y > 10w) => (z < 100x), z ≤ 100x,// x ≤ 2, c ≤ 0 => x ≤ 0, c ≤ 1 => x ≤ 1assume (c < 2);if (*) {x++; y += 100;}else if (* && x ≥ 4) {x++; y++;}else if (y > 10w && z ≥ 100x) {y = −y;}else assume (false);w++; z += 10;c += 1;

assert (!(x ≥ 4 && y ≤ 2));

Add Counters

Bound Solve

Loop Invariants

Spacer on Example

x = y = z = w = 0;c = 0;

while (*) {// (y > 10w) => (z < 100x), z ≤ 100x,// x ≤ 2, c ≤ 0 => x ≤ 0, c ≤ 1 => x ≤ 1assume (c < 2);if (*) {x++; y += 100;}else if (* && x ≥ 4) {x++; y++;}else if (y > 10w && z ≥ 100x) {y = −y;}else assume (false);w++; z += 10;c += 1;

assert (!(x ≥ 4 && y ≤ 2));

Bound Solve Unbounded?

Preserved!Specific to

under-approx.

Depend on counter

Extract UnboundedInvariants

Treat as conjecturedunbounded invariants.

(as in Houdini[1]).

[1] Houdini, an annotationassistant for ESC/Java,C. Flanagan and K.R.M. Leino, 2001

Spacer on Example

x = y = z = w = 0;c = 0;

while (*) {// (y > 10w) => (z < 100x), z ≤ 100x,// x ≤ 2, c ≤ 0 => x ≤ 0, c ≤ 1 => x ≤ 1

if (*) {x++; y += 100;}else if (* && x ≥ 4) {x++; y++;}else if (y > 10w && z ≥ 100x) {y = −y;}else assume (false);w++; z += 10;c += 1;

assert (!(x ≥ 4 && y ≤ 2));

Bound Solve Unbounded? NO

Invariants aretoo weak!

Abstract

Spacer on Example

x = y = z = w = 0;c = 0;assume (y > 10w => z < 100x, z ≤ 100x);while (*) {

// (y > 10w) => (z < 100x), z ≤ 100x,// x ≤ 2, c ≤ 0 => x ≤ 0, c ≤ 1 => x ≤ 1assume (c < 2);if (*) {x++; y += 100;}else if (* && x ≥ 4) {x++; y++;}else if (y > 10w && z ≥ 100x) {y = −y;}else assume (false);w++; z += 10;c += 1;assume (y > 10w => z < 100x, z ≤ 100x);

assert (!(x ≥ 4 && y ≤ 2));

Abstract

Redundant

Bound Solve Unbounded? NO

Strengthenwith

Invariants

Spacer on Example

// (y > 10w) => (z < 100x), z ≤ 100x,// x ≤ 2, c ≤ 0 => x ≤ 0, c ≤ 1 => x ≤ 1assume (c < 2);if (*) {x++; y = *;}else if (* && x ≥ 4) {x++; y = *;}else if (y > 10w && z ≥ 100x) {y = *;}else assume (false);w = *; z = *;c += 1;assume (y > 10w => z < 100x, z ≤ 100x);

assert (!(x ≥ 4 && y ≤ 2));

Bound AbstractSolve Unbounded? NO

Proof-BasedAbstraction

Spacer on Example

assume (c < 4);if (*) {x++; y = *;}else if (* && x ≥ 4) {x++; y = *;}else if (y > 10w && z ≥ 100x) {y = *;}else assume (false);w = *; z = *;c += 1;assume (y > 10w => z < 100x, z ≤ 100x);

assert (!(x ≥ 4 && y ≤ 2));

Bound Solve

Counterexample!

Increment x to 4Choose y arbitrarily

Feasible?

Concrete controlpath is infeasible

NO Refine

Concretize

Spacer on Example

assume (c < 4);if (*) {x++; y += 100;}else if (* && x ≥ 4) {x++; y++;}else if (y > 10w && z ≥ 100x) {y = −y;}else assume (false);w = *; z = *;c += 1;assume (y > 10w => z < 100x, z ≤ 100x);

assert (!(x ≥ 4 && y ≤ 2));

Bound Solve Feasible? NO Refine

Spacer on Example

// (y > 10w) => (z < 100x), z ≤ 100x// y > 0, (x > 0) => (y ≥ 100)assume (c < 4);if (*) {x++; y += 100;}else if (* && x ≥ 4) {x++; y++;}else if (y > 10w && z ≥ 100x) {y = −y;}else assume (false);w = *; z = *;c += 1;assume (y > 10w => z < 100x, z ≤ 100x);

assert (!(x ≥ 4 && y ≤ 2));

Bound Solve Unbounded? YES

Invariants

Implementation Details

Three Key Components

1. Extracting Unbounded Invariants

2. Proof-Based Abstraction

3. Counterexample Analysis and Refinement

How can we efficiently use today’s SAT/SMT solvers?

An invariant for the transition systemis a formula that holds for the initial states and after every transition

Extracting Unbounded Invariants

φ is an invariant iff

Given: A set L of conjectures for invariants, each including “initial states”.

Goal: A maximal I L s.t.

(initial)

(transition)

Re-start from scratch!

unsat sat, making true

…until fixed point

Introduce assumption

variables

unsat sat, making true

Proof ofBounded Safety

Not all of is necessary!

A proof of “bounded” safety foris a formula that holds of the initial states and after every transitionup to the given bound, and excludes error states.

(initial)

(transition)

(safety)

unsat UNSAT core

Iteratively minimize

What have we seen so far?

1. Generalizing reachable states can be hard!

2. Abstractions can really help!

3. Algorithm Spacer that combines abstraction refinement with SAT-based model checking

4. How Spacer can be efficiently automated

Tool andExperimental Evaluation

Spacer Tool

C Program

(Horn-SMT) Logical Encoding

Spacer Backend(using Z3’s framework)

Existing Front-end based on LLVM

Proof-Based Abstraction, CEGAR, etc.

Theories handled:Linear Arithmetic

(Rationals and Integers),Bitvectors

Spacer Tool

Program

Under-Approximate

Check Safety

Feasible?Feasible?

Abstract Refine

Yes Yes

SMT-BasedModel Checker in Z3

The hard example mentioned in the beginning

x = y = z = w = 0;while (*) {

if (*) {x++; y += 100;}else if (*)

y = −y;}t = 1;w += t; z += 10t;

}assert (!(x ≥ 4 && y ≤ 2))

Spacerautomatically

verifies in under a minute!

Results on SV-COMP’13 Benchmarks

0 100 200 300 400 500 600 700 800 9000

No abstraction (secs)

Advantage!

Time-out

Mem-out

Summary

Conclusion

Focused Proofs: Abstractions guide Interpolation towards certain generalizations

Combine Proof-Based Abstraction and Counterexample-Guided Refinement

General framework independent of the underlying model checker

Works in practice!

Future Directions

Verification in presence of assumptions

Different kinds of bounding/abstraction

Synthesizing ghost code to help verification

Questions?

For more details, read our CAV’13 paper!

Concrete:

Abstract:

Counterexample Analysis and Refinement

An “abstract counterexample” is a finite length path consistent with error states

Feasibility Check

Model Checking with Proofs and Counterexamples

Documents