Efficient Modular Glass Box Software Model Checking

Michael RobersonChandrasekhar Boyapati

The University of Michigan

Software Model Checking Exhaustively test programs

�On all possible inputs�On all possible schedules�Up to finite bounds

Binary Tree State Space

Initial State

State Space Explosion

State Space Reduction Many software model checkers

� Verisoft, JPF, CMC, SLAM, Blast, Magic, …

Many state space reduction techniques� Partial order reduction� Predicate abstraction� Effective for control-oriented properties

Our work is on Glass Box Software Model Checking� Effective for data-oriented properties� Significantly more efficient than previous model checkers

Modular Glass Box Checking Check modules against abstractions Check program replacing modules with abstractions

Modular glass box model checking is important� Further improve scalability of glass box checking

Modular glass box model checking is nontrivial

Modular Checking

Initial State of Module

Initial State of Abstraction

Check outputs at each step

Modular checking in traditional model checkers

Modular Glass Box Checking

We can't reach this transition!

We cannot use reachability through transitions Programmers must provide a class invariant State space includes all states that satisfy the invariant Programmers must provide an abstraction function We use it to generate the abstraction of each state

Equal

Modular Glass Box Checking Choose a state Generate abstraction Run one operation Check outputs Generate abstraction

on post-state Check for equality

c1 c2

a1 a2

a2'

Operation

Operation

Abstraction

Abstraction

a1_output

c1_output

Outline Motivation Example Approach Experimental Results Related Work Conclusions

Integer Counterclass IntegerCounter { Map map = new SearchTree(); int max_frequency = 0; int most_frequent = 0;

void count(int i) { Integer frequency = (Integer)map.get(i); if (frequency == null) frequency = new Integer(0); map.insert(i, new Integer(frequency+1)); if (frequency >= max_frequency) { max_frequency = frequency; most_frequent = i; } } int get_most_frequent() { return most_frequent; } int get_max_frequency() { return max_frequency; }}

new AbstractMap();

Count an integer

Return most frequent integer

Return frequency of most frequent integer

Frequencies are stored in a Map

Modular Approach:Replace Module with

Abstraction

Module vs Abstraction SearchTree

� Implements Map get, insert, delete

� Balanced binary tree� Efficient execution� Larger state space

AbstractMapImplements Map

get, insert, delete

Linked listSimple executionSmaller state space

5

2 6

1 41 2 4 5 6

vs

Checking SearchTree Choose a state Generate abstraction Run one operation Check outputs Generate abstraction

on post-state Check for equality

Equal

c2

a1 a2

a2'

Operation

Operation

Abstraction

Abstraction

52 6

1 4c1

Checking SearchTree Choose a state Generate abstraction Run one operation Check outputs Generate abstraction

on post-state Check for equality

52 6

1 4

Equal

c2

a1 a2

a2'

Operation

Operation

Abstraction

Abstraction

1 2 4 5 6

Equal

a2

insert(3,x)

Checking SearchTree Choose a state Generate abstraction Run one operation Check outputs Generate abstraction

on post-state Check for equality

52 6

1 4

insert(3,x)

52 6

1 43

1 2 3 4 5 6

c2

a2'

Operation

Operation

Abstraction

Abstraction

1 2 4 5 6

Checking SearchTree Choose a state Generate abstraction Run one operation Check outputs Generate abstraction

on post-state Check for equality

Equal

insert(3,x)52 6

1 4

insert(3,x)

52 6

1 43

1 2 3 4 5 6

a2'

Abstraction

Abstraction

1 2 4 5 6

a1_output

c1_output

Checking SearchTree Choose a state Generate abstraction Run one operation Check outputs Generate abstraction

on post-state Check for equality

Equal

insert(3,x)52 6

1 4

insert(3,x)

52 6

1 43

1 2 3 4 5 6

a2'

Abstraction

Abstraction

1 2 4 5 6

1 2 3 4 5 6

Checking SearchTree Choose a state Generate abstraction Run one operation Check outputs Generate abstraction

on post-state Check for equality

Equal

insert(3,x)52 6

1 4

insert(3,x)

52 6

1 43

1 2 3 4 5 6

Abstraction

Abstraction

1 2 4 5 6

1 2 3 4 5 6

Glass Box Pruning

5

2 6

1 4

insert(3,x)

5

2 6

1 4

3

insert(3,x)

5

2

1 4

5

2

1 4

3

insert(3,x)

5

2 6

4

3

7

5

2 6

4 7

An insert operation only touches one path Insert behaves similarly on many states

insert(3,x)

5

2 6

4

3

7

5

2 6

4 7

insert(3,x)

5

2

1 4

5

2

1 4

3

Glass Box Pruning

5

2 6

1 4

insert(3,x)

5

2 6

1 4

3

PRUNED

We don't need to check more than one of these We can prune all others from the state space

insert(3,x)

5

2 6

4

3

7

5

2 6

4 7

insert(3,x)

5

2

1 4

5

2

1 4

3

Glass Box Pruning

5

2 6

1 4

insert(3,x)

5

2 6

1 4

3

PRUNED

We check each tree path, not each tree This reduces the state space dramatically

Checking IntegerCounter

52 6

1 4

IntegerCounter'

1 2 4 5 6

IntegerCounter with SearchTree IntegerCounter with AbstractMap

Smaller state spaceBetter state space reduction

Faster analysis

vs

241

65

451

62

42 5

1 6

52 6

1 4

52 6

14

42 6

1 5

1 2 4 5 6

IntegerCounter

Outline Motivation Example Approach

� Program specification� Search algorithm� State space representation� State space reduction

Experimental Results Related Work Conclusions

Module Implementation

class SearchTree implements Map { class Node { int key; Object value; Node left; Node right; }

Node root;

Object get(int key) { /* ... */ }

void insert(int key, Object value) { /* ... */ }

void remove(int key) { /* ... */ }}

Map interface methods

c1 c2Operation

Equal

a2

a2'

Abstractionclass AbstractMap implements Map { class Node { Object key; Object value; Node next; }

Node head;

Object get(int key) { /* ... */ }

void insert(int key, Object value) { /* ... */ }

void remove(int key) { /* ... */ }

@Declarative boolean equalTo(AbstractMap m) { /* ... */ }}

Map interface methods

Equality test

Declarative methods� Subset of Java� Free of side effects� Used for specification� Aid our analyses

a1 a2Operation

Module Specification

class SearchTree implements Map { class Node { int key; Object value; Node left; Node right; }

Node root;

/* ... Map operations ... */

@Declarative boolean repOk() { /* ... */ } AbstractMap abstraction() { /* ... */ }}

Module Invariant

Abstraction function

c1

a1

Abstraction

Approach Program specification Search algorithm State space representation State space reduction

Search Algorithm

5

2 6

1 4

Choose an unchecked valid state

insert(3,x)

@Declarativeboolean repOk() { /* ... */}

Search Algorithm

5

2 6

1 4

Generate its abstraction

insert(3,x)

1 2 4 5 6

AbstractMap abstraction() { /* ... */}

Search Algorithm

5

2 6

1 4

Run the operation on both states

insert(3,x)5

2 6

1 4

3

1 2 3 4 5 6

void insert(int key, Object value) { /* ... */}

1 2 4 5 6

Search Algorithm Generate the post-state abstraction

insert(3,x)

AbstractMap abstraction() { /* ... */}

5

2 6

1 4

5

2 6

1 4

3

1 2 3 4 5 61 2 4 5 6

1 2 3 4 5 6

Search Algorithm Check invariant and abstraction equality

insert(3,x)@Declarativeboolean repOk() { /* ... */}

@Declarativeboolean equalTo(AbstractMap m) { /* ... */}

5

2 6

1 4

5

2 6

1 4

3

1 2 3 4 5 61 2 4 5 6

1 2 3 4 5 6

insert(3,x)

5

2

1 4

5

2

1 4

3

insert(3,x)

5

2 6

4

3

7

5

2 6

4 7

State Space Reduction

5

2 6

1 4

insert(3,x)

5

2 6

1 4

3

PRUNED

Identify and prune similar states

Search Algorithm

Let S be the states that satisfy repOk()

While S is not empty

Choose a state s in S.

Check s.

Let P be the set of states similar to s

S = S - P

Need efficient representation and operations for these

sets!

Approach Program specification Search algorithm State space representation State space reduction

Representation Represent a set as a boolean formula

� Encode each field as bits (b0, b1, …)� Constrain the bits using boolean operations

n1.left = null || n1.key > n2.key

Øb4 Ú (b1 Ù Øb7) Ú ((b1 Ú Øb7) Ù b0 Ù Øb6)

key = {b0,b1}value = {b2,b3}

left = {b4}right = {b5}

n1

key = {b6,b7}value = {b8,b9}

left = {}right = {}

n2 ...

Representation Initialize to set of states that satisfy invariant

� Construct a formula describing invariant

boolean formula

@Declarativeboolean repOk() { /* ... */}

Representation Initialize to set of states that satisfy invariant

� Construct a formula describing invariant

Declarative methods� No assignment, object creation, or loops� Declarative methods allow efficient translation� Declarative methods produce compact formulas

@Declarativeboolean repOk() { /* ... */}

Search Algorithm

Use a SAT solver

Add ¬P to the SAT solver

Let S be the states that satisfy repOk()

While S is not empty

Choose a state s in S.

Check s.

Let P be the set of states similar to s

S = S - P

Approach Program specification Search algorithm State space representation State space reduction

� Dynamic analysis� Static analysis

Dynamic Analysis Discover and prune states that are similar Symbolic execution

� Generates a path constraint, P� P holds for states that traverse the same code path� P is the set of similar states to be pruned

5

2 6

1 4

insert(3,x)

Dynamic Analysis Discover and prune states that are similar Symbolic execution

� Generates a path constraint, P� P holds for states that traverse the same code path� P is the set of similar states to be pruned

n1

n2 n3

n4 n5

op(key,value)

op = insert&& root = n1 && key < n1.key

&& n1.left = n2 && key > n2.key&& n2.right = n5 && key < n5.key

&& n5.left = null

Operation is insert

Node exists, is greater/less

than key

Final node does not exist (yet)

Static Analysis Dynamic analysis finds P, the similar states Pruning these states is not always correct!class WhyStaticAnalysis { boolean a, b;

void operation() { if (a) a = false; else a = true; }

@Declarative boolean repOk() { return a || b; }}

a = true

b = true

a = false

b = true

a = true

b = false

a = false

b = false

a = trueP :=

We use a static analysis to ensure correctness� All pruned transitions checked together� Any error within finite bounds is caught

Static Analysis

class WhyStaticAnalysis { boolean a, b;

void operation() { if (a) a = false; else a = true; }

@Declarative boolean repOk() { return a || b; }}

a = true

b = true

a = false

b = true

a = true

b

a = false

b

a = trueP :=

Prestateof a pruned transition

Poststateof a pruned transition

repOk = (a || b)

repOkPre = (a || b)a=true = true

repOkPost = (a || b)a=false = b

repOkPre ® repOkPost = b

Not valid when b = false!

InvariantPrestate Invariant

Poststate Invariant

Correct Transition

Static Analysis For every valid prestate in P, the following hold

� The invariant is maintained in the poststate� Equality of abstractions

repOkpre ® repOkpost && abs_post.equalTo(abs_post')

Use a SAT solver to check� If it holds then pruning is sound� If not, we have a counterexample

Equal

pre post

abspre

abspost

abspost'

Operation

Operation

Abstraction

Abstraction

boolean formula

Outline Motivation Example Approach Experimental Results Related Work Conclusions

Checking Modules vs Abstractions TreeMap

� Implemented with a red-black tree HashMap

� Implemented with a hash table AbstractMap

� Implemented with a linked list of (key, value) pairs

Checking Modules vs Abstractions TreeSet

� Implemented with a TreeMap HashSet

� Implemented with a HashMap AbstractSet

� Implemented with a linked list of set items

Maps vs AbstractMapBenchmark Max Number

of Nodes JPF Korat Blast Glass Box Checker

TreeMapvs

AbstractMap

1234

…89

…153163

1.2185.556

memory out

0.6080.6130.6760.732

…2202.31timeout

aborted 0.1880.2440.3920.485

…1.1241.491

…4.571

40.405787.411

HashMapvs

AbstractMap

1234

…101112…163264

0.6746.514

memory out

0.4650.4970.5390.810

…150.932

1203.986timeout

aborted 0.1760.2270.2560.305

…0.7800.9531.162

…2.879

75.1392004.723

We check over 235 trees in under 15 minutes

Sets vs AbstractSet

Benchmark Max Number of Nodes JPF Korat Blast Glass Box

Checker

TreeSetvs

AbstractSet

1234

…131415…3163

0.5370.950

26.816memory out

0.6380.6480.6931.020

…615.5241706.41timeout

aborted 0.1950.2460.3930.489

…3.0653.3994.481

…39.789

796.955

HashSetvs

AbstractSet

1234

…13141516…3264

0.7280.7816.574

memory out

0.5200.5110.7160.570

…238.420593.045952.317timeout

aborted 0.1710.2210.2480.299

…1.3441.6082.0292.816

…68.011

2543.034

Checking Clients

Benchmark Max Size OriginalProgram

Maps Replaced with AbstractMap

IntegerCounter

123

…7

153163

127255

0.1980.2790.469

…1.1825.591

41.5632.488timeout

0.1130.1540.164

…0.2670.5391.867

10.79493.276

946.091

DualCache

123

…7

153163

127255

0.2030.2830.503

…1.2075.765

53.434723.267timeout

0.2220.3230.327

…0.5291.0154.057

26.192215.521

2180.506

Checking Clients

Benchmark Max Number of Nodes

OriginalProgram

Maps Replaced with AbstractMap

TreeSet

123

…7

153163

127255511

0.1950.2460.393

...0.9614.481

39.789796.955timeout

0.1220.1530.167

…0.2510.4290.9913.388

16.690184.827425.328

HashSet

12345

…163264

128256512

0.1710.2210.2480.2990.363

…2.816

68.0112543.034

timeout

0.1060.1410.1530.1690.193

…0.4510.9893.464

17.07191.629

754.426

Related Work State space reduction techniques

� Abstraction & refinement [SLAM; Blast; Magic]� Partial order reduction [Godefroid97; Flanagan05]� Heap canonicalization [Musuvathi05; Iosif02]� Symmetry reduction [Ip93]

Related Work Software model checkers

� Verisoft [Godefroid97]� Java Pathfinder [Visser00]� CMC [Musuvathi02]� Bandera [Corbett00]� Bogor [Dwyer05]� SLAM [Ball01]� Blast [Henzinger02]� Magic [Chaki03]� Jalloy [Vaziri03]� Miniatur [Dolby07]

Conclusions

Significant improvement over traditional model checkers for checking complex data dependent properties

A promising approach to checking much larger programs and broader classes of program properties than is currently possible

Modular Glass Box Model Checking Offers: