Modernized Computation Modernized Computation Engines for Tomorrow's Engines for Tomorrow's

Formal VerificationFormal Verification

Robert Brayton Niklas Een Alan MishchenkoRobert Brayton Niklas Een Alan Mishchenko

Berkeley Verification and Synthesis Research CenterBerkeley Verification and Synthesis Research CenterDepartment of EECSDepartment of EECS

UC BerkeleyUC Berkeley

Task OverviewTask Overview

SRC task ID:SRC task ID: 2265.001 2265.001 Start date:Start date: April 1, 2012 April 1, 2012 Thrust area:Thrust area: Verification Verification Task leaders:Task leaders:

Robert Brayton, Nilkas Een, Alan Mishchenko Robert Brayton, Nilkas Een, Alan Mishchenko (Univ. of California/Berkeley)(Univ. of California/Berkeley)

Industrial liaisons:Industrial liaisons: See next slideSee next slide

Students:Students: Jiang Long, Sayak Ray, Baruch SterinJiang Long, Sayak Ray, Baruch Sterin

Industrial LiaisonsIndustrial Liaisons FreescaleFreescale

Himyanshu AnandHimyanshu Anand IBMIBM

Jason BaumgartnerJason Baumgartner IntelIntel

Timothy KamTimothy Kam, Ranan Fraer, , Ranan Fraer, Alexander NadelAlexander Nadel, , Murali TalupurMurali Talupur

Mentor GraphicsMentor GraphicsJeremy LevittJeremy Levitt, , Christian StangierChristian Stangier

(Source: (Source: http://www.src.org/library/research-catalog/2265.001/http://www.src.org/library/research-catalog/2265.001/))

Anticipated ResultAnticipated Resultss

Methodology and algorithms for next-generation improvements in formal verification, addressing SAT solving hybrid simulation counter-example handling invariant generation

Public software implementation of the above methodology and algorithms.

Experimental evaluation on industrial benchmarks.

Task DescriptionTask Description We propose to leverage the unique expertise of our group, Berkeley Verification We propose to leverage the unique expertise of our group, Berkeley Verification

and Synthesis Research Center (BVSRC), and our previous SRC contracts for and Synthesis Research Center (BVSRC), and our previous SRC contracts for solving hard industrial problems arising in formal verification. The goal would be a solving hard industrial problems arising in formal verification. The goal would be a new level in the state-of-the-art of logic formal verification engines, which adds new level in the state-of-the-art of logic formal verification engines, which adds the following to the design flow:the following to the design flow: Application-specific SAT solversApplication-specific SAT solvers to improve performance of key verification engines. to improve performance of key verification engines.

Two new design decisions will be explored for developing SAT solvers, which are Two new design decisions will be explored for developing SAT solvers, which are specifically geared to solving numerous, related, and relatively easy problems, on the specifically geared to solving numerous, related, and relatively easy problems, on the one hand, and monolithic, large, and hard problems, on the other hand.one hand, and monolithic, large, and hard problems, on the other hand.

Hybrid simulationHybrid simulation based on new heuristics to improve state space coverage. New based on new heuristics to improve state space coverage. New ideas will be explored for improving bit-level simulation and combining it with symbolic ideas will be explored for improving bit-level simulation and combining it with symbolic simulation, handled by adding symbolic variables or exhaustively simulating selected simulation, handled by adding symbolic variables or exhaustively simulating selected input subspaces.input subspaces.

Counter-example minimizationCounter-example minimization to shorten the counter-examples produced by some to shorten the counter-examples produced by some algorithms, such as random and hybrid simulation. A counter-example minimizer will be algorithms, such as random and hybrid simulation. A counter-example minimizer will be developed based on hybrid simulation and bounded model checking. Another aspect to developed based on hybrid simulation and bounded model checking. Another aspect to be explored is the use of concurrency to speed up the minimization process.be explored is the use of concurrency to speed up the minimization process.

Various methods for automated inductive invariant generationVarious methods for automated inductive invariant generation. Several ways of . Several ways of generating inductive invariants will be explored. One of them is based on using high-generating inductive invariants will be explored. One of them is based on using high-level information about the design; another is an extension of a previous method based level information about the design; another is an extension of a previous method based on the structural analysis of the AIG.on the structural analysis of the AIG.

The new methods developed while working on this proposal will be tested on The new methods developed while working on this proposal will be tested on industrial designs in our synthesis and verification tool, ABC, and made available industrial designs in our synthesis and verification tool, ABC, and made available in source code, which can be customized to specific applications.in source code, which can be customized to specific applications.

Task Deliverables Task Deliverables 2013 Annual review presentation (27-Mar-2013) Report on a software release of a circuit-based SAT solver.

Evaluation on industrial problems (30-Jun-2013)

2014 Annual review presentation (30-Apr-2014) Report on a software release of a counter-example generator.

Evaluation on industrial problems (30-Jun-2014)

2015 Report on a software release of a hybrid simulator and invariant

generator. Evaluation on industrial problems (30-Apr-2015) Final report summarizing research accomplishments and future

direction (30-Jun-2015)

Current State of the ProjectCurrent State of the Project Covered in this presentationCovered in this presentation

Advances in application-specific SAT solving (work in progress)Advances in application-specific SAT solving (work in progress) Advances in simulation (IWLS’12)Advances in simulation (IWLS’12) Counter-example analysis (submitted to IWLS’13)Counter-example analysis (submitted to IWLS’13) Towards new invariant generation (work in progress)Towards new invariant generation (work in progress)

Other developmentsOther developments Semi-canonical form for sequential AIGs (DATE’13)Semi-canonical form for sequential AIGs (DATE’13) Automated gate-level abstraction (DATE’13)Automated gate-level abstraction (DATE’13) Solving multiple-output properties (work in progress)Solving multiple-output properties (work in progress)

Advances in SAT SolvingAdvances in SAT Solving

PunySATPunySAT: An application-specific SAT solver: An application-specific SAT solver Geared to small, hard SAT instancesGeared to small, hard SAT instances Similar to MiniSAT in everything, exceptSimilar to MiniSAT in everything, except Clauses are bit-strings, not integer arraysClauses are bit-strings, not integer arrays

BCP can be more efficient (see next slide)BCP can be more efficient (see next slide) Experimental results inconclusiveExperimental results inconclusive

Tied with MiniSAT on 50 problems from SAT competitionTied with MiniSAT on 50 problems from SAT competition

PunySAT: BCP Using Bit-StringsPunySAT: BCP Using Bit-Stringsbool bool conflconfl(( BitVec iclauseBitVec iclause, , BitVec assignBitVec assign )) {{ return (iclause & ~assign) == 0;return (iclause & ~assign) == 0;}}

bool bool bcpbcp (BitVec iclause, BitVec assign, lit_t& prop_lit)(BitVec iclause, BitVec assign, lit_t& prop_lit) {{ BitVec mask = (iclause & ~assign);BitVec mask = (iclause & ~assign); if ((mask & (mask-1)) == 0)if ((mask & (mask-1)) == 0) {{ prop_lit = neg(prop_lit = neg(__builtin_ctz__builtin_ctz(mask));(mask)); return true; return true; }} elseelse return false;return false;}}

ccla_id la_id propagatepropagate()() {{        while (qhead < trail_sz)while (qhead < trail_sz) {{                lit_t lit_t q, q, p = trail[qheadp = trail[qhead++++];];                for (uint i = 0; i < occurs[p].size(); i++){for (uint i = 0; i < occurs[p].size(); i++){                        cla_id from = occurs[p][i];cla_id from = occurs[p][i];                        BitVec cl = clauses[from];BitVec cl = clauses[from];                        if (cl.if (cl.conflconfl(assign))  (assign))  return from;return from; // returns conflicting clause// returns conflicting clause                        if (cl.if (cl.bcpbcp(assign, q) && !assign.has(q))(assign, q) && !assign.has(q))                                enqueueQ(q, from);enqueueQ(q, from);                }}        }}        return 0;return 0; // returns ”no conflict// returns ”no conflict””} }

BitVecBitVec is a bit-vector of the is a bit-vector of the appropriate size for the appropriate size for the problem (sayproblem (say,, 256 bits) 256 bits)..

iclauseiclause is is a clause with all a clause with all literals invertedliterals inverted..

lit_tlit_t is a typedef for an is a typedef for an unsigned intunsigned int

(mask & (mask-1)) == 0(mask & (mask-1)) == 0 checks if checks if the the mask has at most mask has at most one bit setone bit set

__builtin_ctz(mask)__builtin_ctz(mask) returns the returns the index of the lowermost "1" in index of the lowermost "1" in mask (supported natively by mask (supported natively by most processors)most processors)

PunySAT: ExperimentPunySAT: Experiment

Green: ZZMiniSATRed: PunySAT-2013-03-27

Considered 50+ problems from SAT competitions with less than Considered 50+ problems from SAT competitions with less than 256 256 variables, solved by either MiniSAT or PunySAT in less than 5 minvariables, solved by either MiniSAT or PunySAT in less than 5 min

MiniSAT solved 48 problems; PunySAT solved 50 problemsMiniSAT solved 48 problems; PunySAT solved 50 problems

Advances in Simulation: Key IdeaAdvances in Simulation: Key Idea

Rarity simulationRarity simulation is random simulation with is random simulation with prioritizing reachable statesprioritizing reachable statesGracefully handles resets by skipping Gracefully handles resets by skipping

frequently visited statesfrequently visited statesVisits rare reachable states where hard-to-Visits rare reachable states where hard-to-

detect failures may be founddetect failures may be foundMore efficient than naïve random More efficient than naïve random

simulation at detecting property failuressimulation at detecting property failures

Rarity Simulation: ImplementationRarity Simulation: Implementation Divide flops into fixed-size groupsDivide flops into fixed-size groups in the order of in the order of

their appearance in the designtheir appearance in the design Groups of 8 flops are used by defaultGroups of 8 flops are used by default

Maintain a record of observed flop valuesMaintain a record of observed flop values For each group, 256 (=2^8) counters are usedFor each group, 256 (=2^8) counters are used

After simulating a fixed number (by default, 20) After simulating a fixed number (by default, 20) frames, recompute the frequency of having a frames, recompute the frequency of having a given value in each flop group, and given value in each flop group, and choose next choose next states for simulation based on the rarity of valuesstates for simulation based on the rarity of values By default, 1 out of 64 states is chosenBy default, 1 out of 64 states is chosen

R. Brayton, N. Een, and A. Mishchenko, "Using speculation for sequential equivalence checking", Proc. IWLS'12, pp. 139-145.

Rarity Simulation: IllustrationRarity Simulation: Illustration

Start with initial state

Accumulate info about reached states

Decide what next states to simulate from

Accumulate info about reached states

Decide what next states to simulate from

etc

Initial stateRandom PI values

Counter-Example (CE) AnalysisCounter-Example (CE) Analysis CE is a set of PI values in each time frame, which, starting CE is a set of PI values in each time frame, which, starting

from the initial state, leads to the property failurefrom the initial state, leads to the property failure Given a CE, PIs can be divided into three categoriesGiven a CE, PIs can be divided into three categories

Essential PIsEssential PIs whose values are needed for the property failure whose values are needed for the property failure Don’t-care PIsDon’t-care PIs whose values are not important whose values are not important Optional PIsOptional PIs (all the remaining ones) (all the remaining ones)

We introduce the notion of We introduce the notion of CE-induced networkCE-induced network This network, composed of two-input AND-/OR-gates, has unate This network, composed of two-input AND-/OR-gates, has unate

Boolean function in terms of PI variables, which Boolean function in terms of PI variables, which represents represents allall subsets of the PIs implying the property failure subsets of the PIs implying the property failure according to the CEaccording to the CE

ApplicationsApplications Design debugging, CE minimization, abstraction refinementDesign debugging, CE minimization, abstraction refinement

A. Mishchenko, N. Een, and R. Brayton, "A toolbox for counter-example analysis and optimization", Submitted to IWLS'13.

Construction of CE-Induced NetworkConstruction of CE-Induced Network

Unfold the original network for the Unfold the original network for the depth indicated by the CEdepth indicated by the CE

Assign values of primary inputs and Assign values of primary inputs and internal nodes according to the CEinternal nodes according to the CE

Replace all primary inputs of the Replace all primary inputs of the unfolding by free variablesunfolding by free variables

Replace each AND of the unfolding Replace each AND of the unfolding by AND, OR or BUF using the rulesby AND, OR or BUF using the rules

Rehash and sweep dangling nodesRehash and sweep dangling nodes

1

11

0

00

0

10

Unfolding CE-induced network

Experiment: CE Bit ProfilingExperiment: CE Bit ProfilingExample Engine PIs FFs Frames Total bits DC, % Opt, % Essen, % Min, % Time, s

6s1 SIM 45 291 1247 56115 53.95 27.50 18.55 29.72 173.33 6s5 SIM 141 2519 63 8883 60.32 37.95 1.73 25.69 11.48 6s14 SIM 439 811 869 381491 83.16 15.25 1.59 4.38 141.72 6s17 SIM 450 819 1084 487800 83.50 15.23 1.27 4.43 213.70 6s18 SIM 450 819 496 223200 83.14 15.41 1.44 4.56 83.53 6s133 SIM 450 819 1084 487800 83.50 15.23 1.27 4.44 209.30 6s134 SIM 36 571 610 21960 90.83 7.02 2.15 7.19 3.83 Bob12s05 SIM 437 3956 43 18791 65.60 31.48 2.92 30.09 28.44 bobtuttt SIM 2807 111 1582 4440674 98.74 1.24 0.02 0.09 57.50 6s41 BMC 19 959 74 1406 58.46 33.57 7.97 23.26 0.61 6s134 BMC 36 571 169 6084 92.11 5.34 2.55 5.85 0.59 6s162 BMC 73 156 74 5402 79.67 13.16 7.16 13.92 0.42 6s172 BMC 403 422 46 18538 66.71 29.48 3.81 10.42 1.48 6s172 PDR 403 422 111 44733 47.06 49.10 3.83 17.34 7.70

Geo 1.000 0.731 0.157 0.021 0.076

Engine: Formal verification engine that produced counter-exampleTotal bits: The total number of primary inputs in the unrolled testcaseDC/Opt/Essen: Percentage of don’t-care, optional, and essential bitsMin: Percentage of bits in the minimized counter-exampleTime: Runtime of bit profiling in seconds

Experiment: Bounded Unfolding vs. Experiment: Bounded Unfolding vs. CE-Induced NetworkCE-Induced Network

Example CE Depth PI AND Level Time, s PI AND Level Time, s

6s1 1247 56115 3514473 43546 1.91 56115 575536 9243 0.63 6s5 63 8883 1571421 2595 0.55 8883 548554 1596 0.38 6s14 869 381491 12102021 318775 6.66 381491 2666982 112934 5.78 6s17 1084 487800 12906342 330886 7.31 487800 2925648 114392 6.05 6s18 496 223200 5572818 151699 3.17 223200 1292920 53502 2.33 6s133 1084 487800 12906261 330885 7.16 487800 2926433 114394 6.14 6s134 610 21960 103586 6430 0.09 21960 18033 2492 0.06 bob12s05 43 18791 607028 743 0.38 18791 196375 548 0.30 bobtuttt 1582 4440674 14682128 64774 6.30 4440674 369044 13142 3.34 6s41 74 1406 103884 549 0.05 1406 27600 259 0.02 6s134 169 6084 21844 1495 0.02 6084 3925 639 0.02 6s162 74 5402 67990 1666 0.02 5402 16427 477 0.06 6s172 46 18538 120433 822 0.03 18538 45363 484 0.01 6s172 111 44733 389403 2382 0.11 44733 153616 1472 0.08

Geo 1.000 1.000 1.000 1.000 1.000 0.212 0.397 0.711

CE Depth: The timeframe where the property fails according to the CEPI/AND/Level: The number of PIs, AIG nodes, and AIG node levelsTime: Runtime of unfolding vs. constructing CE-induced network, in seconds

Rediscovery of High-Level Structure in Rediscovery of High-Level Structure in a Bit-Level AIG via Support Hashinga Bit-Level AIG via Support Hashing AlgorithmAlgorithm

InputInput: Sequential AIG: Sequential AIG OutputOutput: Sequential AIG annotated with high-level information: Sequential AIG annotated with high-level information

ComputationComputation Select a subset of inputsSelect a subset of inputs (or internal nodes) with high fanout (or internal nodes) with high fanout Iterate combinational support computation to Iterate combinational support computation to derive sequential derive sequential

support of every nodesupport of every node in the AIG in terms of the selected nodes in the AIG in terms of the selected nodes Hash nodes by their supportHash nodes by their support to find their equivalence classes to find their equivalence classes Group equivalence classesGroup equivalence classes of small cardinality and/or with similar of small cardinality and/or with similar

support to create well-balanced partitionssupport to create well-balanced partitions (optional) Iterate support hashing to break large partitions into (optional) Iterate support hashing to break large partitions into

smaller onessmaller ones ApplicationsApplications

Circuits partitioningCircuits partitioning Invariant computationInvariant computation

Support Hashing: IllustrationSupport Hashing: Illustration

Blocks A, B, and C are structurally different because Blocks A, B, and C are structurally different because Nodes in A depend on XNodes in A depend on X Nodes in B depend on X and YNodes in B depend on X and Y Nodes in C depend on YNodes in C depend on Y

Block A Block B Block CSupport = X- Support = XY Support = -Y

Node X Node Y

Related PublicationsRelated Publications Hybrid simulation

- R. Brayton, N. Een, and A. Mishchenko, "Using speculation for sequential equivalence checking", Proc. IWLS'12, pp. 139-145.

Toolbox for counter-example manipulation- A. Mishchenko, N. Een, and R. Brayton, "A toolbox for counter-example analysis and optimization", Submitted to IWLS'13.

Gate-level abstraction - A. Mishchenko, N. Een, R. Brayton, J. Baumgartner, H. Mony, and P. Nalla, "GLA: Gate-level abstraction revisited", Proc. DATE'13.

Computing canonical form for sequential AIGs - A. Mishchenko, N. Een, R. Brayton, M. Case, P. Chauhan, and N. Sharma, "A semi-canonical form for sequential AIGs", Proc. DATE'13.

Novel logic representation for fast synthesis and mapping - A. Mishchenko and R. Brayton, "Faster logic manipulation for large designs", To be submitted to IWLS'13.

Ongoing and Future WorkOngoing and Future Work

Application-specific SAT solving Run additional experiments with PunySAT

Hybrid simulation Combine rarity simulation with BMC

Counter-example analysis Develop a CE minimization procedure

Invariant detection Develop invariant mining based on the proposed

annotation of the AIG with high-level information

Other Work DirectionsOther Work Directions Improving multi-output property solver Exploring new algorithms for abstraction refinement

Refine multiple counter-examples Use UNSAT core computationin a seprate SAT solver Handle segments of counter

Speeding up isomorphism detection Add incremental signature propagation Avoid separate handling of each PO sequential logic cone

New memory-efficient BMC engine based on Constant-memory unfolding engine Low-memory AIG package Fast logic synthesis to reduce logic cloud given to the SAT solver

Continuing to release ABC with new features

ConclusionsConclusions Reviewed the SRC Task 2265.001 2265.001 (first year)

“Modernized Computation Engines for Tomorrow's Formal Verification”

Discussed new findings Reviewed recent publications