1
Distributed BDD-based Model Checking
Orna GrumbergTechnion, Israel
Joint work with Tamir Heyman, Nili Ifergan, and Assaf Schuster
CAV00, FMCAD00, CAV01, CAV03, CHARME05,ATVA05
PDMC July 14, 2011
2
Model Checking
Specification
Model Checker
Counter Example+
System Model
3
BDD-Based Model Checking
Model checking often suffers from the state-explosion problem which refers to its high space requirements
One of the first solution proposed for this problem:•symbolically representing the model using BDDs•Model checking is then done by manipulations on BDDs
4
Binary decision diagrams (BDDs)
• Data structure for representing Boolean functions– Often concise in memory
• They are suitable for representing sets of states
• Most Boolean operations can be done efficiently with BDDs
Yet, they still cannot handle large systems
5
Our solution:Distributed Model checking
• Using the accumulative computation power and memory of a number of machines working in parallel
• Enables model checking of models with higher memory requirements– For large models, time requirements
become a problem as well
6
Next solution:Asynchronous Distributed Model Checking
• Obtaining high speedups by using an asynchronous distributed algorithm for Reachability Analysis – Computing Reachability for models
with high time and memory requirements
7
Reachability Analysis
• Reachability Analysis - computing the set of states that can be reach from the set of initial states
• The verification of most temporal
safety properties can be reduced to Reachability Analysis
8
Sequential Reachability Algorithm
9
R
Sequential Reachability Algorithm
S0
Image Computation
• The computation of the set of successors of a given set of states
• Image can be obtained by micro steps – Partitioned transition relations
• Each partition defines the transition for one variable
• The conjunction of all the partitions give the transition of all variables.
– Each micro step adds to the intermediate results one more partition
11
The Distributed Approach
12
• The state space on which the Reachability analysis is performed is partitioned into slices
• Each slice is owned by one process
• A set of Boolean window functions: w1…wn, defines for each process the slice it owns
State Space Partition
13
State Space Partition – Cont.
• The set of window functions is complete and disjoint:
• The state space S is partitioned so that
,i i ii
S S w S S S
W1 W2
W3 W4 W5
11, : 0
n
ii i jw i j w w
The Basic Distributed Algorithm
• The algorithm works iteratively• Each iteration consists of two
phases
The Basic Distributed Algorithm
• The algorithm works iteratively• Each iteration consists of two
phases– Phase 1 – Image Computation
Each of the active workers applies image computation• to states it owns, • found in the previous iteration
The Basic Distributed Algorithm
• The algorithm works iteratively• Each iteration consists of two
phases– Phase 1 – Image Computation
Each of the active workers applies image computation• to states it owns, • found in the previous iteration
Some of the computed states may not own by it
The Basic Distributed Algorithm
• The algorithm works iteratively• Each iteration consists of two
phases– Phase 1 – Image Computation
Each of the active workers applies image computation
– Phase 2 – ExchangeThe active workers exchange the states they do not own with the other workers
The Basic Distributed Algorithm
• The algorithm works iteratively• Each iteration consists of two phases
– Phase 1 – Image Computation Each of the active workers applies image computation
– Phase 2 – ExchangeThe active workers exchange the states they do not own with the other workers
• Phase 2 starts only after all the active workers finished phase 1
19
The Basic Algorithm
Image Iteration 1Exchange
P1
P2
. . . . . . . . . . .
Using workers “by need”
There is a pool of non-active free workers•Workers join and leave the computation as needed
20
Splitting
If the worker’s memory overflows during image computation •The image computation stops with an intermediate results •The overflowed worker gets a free worker from the pool and splits the computation into two parts
– Each worker then continues the computation starting from its part of the intermediate result
Splitting (cont.)
Overflow may occur also in the exchange phase•Splitting is applied and then exchange continues among the new set of workers, according to their new windows
23
Collect Small Workers
• If a worker does not have enough work – It delivers its ownership and owned
states to a colleague worker– It joins the pool of free workers (and
may participate in the computation later on, if needed)
24
High Time Requirements
Observations:The iterations are synchronized •“Fast” workers have to wait until all workers complete the image phase before they can proceed to the exchange phase
– Unnecessary idle time for “fast” processes
25
High Time Requirements
Observations:The iterations are synchronized •“Fast” workers have to wait until all workers complete the image phase before they can proceed to the exchange phase
– Unnecessary idle time for “fast” processes
•In reachability analysis states can be found in any order as long as they are all found
26
The Asynchronous Approach
• The iterations are no longer synchronized among processes
• Each process can execute the image computation at its own pace while sending and receiving states "in the background“
– Image computation and state exchange become concurrent operations
27
Challenge 1 – Concurrency
• Computing image and sending/receiving non-owed states is done concurrently – Receiving and sending non-owned states
packages is done “in the background” (asynchronous send/receive)
– Transforming packages back to BDDs and packaging BDDs to be sent, is done between micro steps
28
Challenge 2 – Package Forwarding
Because of splitting during the exchange phase, some non-owned states may reach the wrong process
Solution:•Each process “takes” from the package the states it owns and forwards it to other processes which also have states in this package
29
P2, W2P2,W’2P3,W’3
Package Forwarding
P1
P2,W2
P3,W2W’3
30
Package Forwarding (cont.)
• The algorithm assures that each non-owned state will eventually reach its owner
31
Challenge 3 - Termination
• Asynchronous distributed termination detection in an environment in which processes may join and leave the computation – Extension to the two phase Dijkstra
termination detection algorithm with dynamic number of processes
• The termination detection algorithm works “in the background”
32
Implementation • Division, developed by Tamir Heyman, is
a generic platform for the study of distributed symbolic model checking– Division requires a model checker as an
external module• We use Intel’s model checker, Forecast
• The distributed asynchronous algorithm has been developed and implemented by Nili Ifergan, on top of the Division framework
33
Implementation issues
• The communication between the processes was done through MPI over fast Ethernet– The communication module allows for true
asynchronous message passing• We conducted our experiments on a
parallel testbed – It included a maximum of 55 PC machines,
each consisting of dual 2.4GHz Xeon processors with 4GB memory
Summary
• We developed a distributed BDD-based model checking– The state space is partitioned, not the
BDDs
• We obtained significant memory and time reductions
• We verified huge industrial designs, mostly taken from Intel
34
Summary (cont.)
• The project stopped because SAT-based (bounded) model checking became a standard tool– Fast and easy to use
• The advantage of our approach was (and maybe still is) in the verification of extremely large hardware designs– At the time there was no interest
35
The future
• Modern technologies and infrastructures may make distributed BDD-based model checking attractive again
?
36
37
The End
