SpecDiff: Differencing LTSs

Zhenchang Xing*, Jun Sun+, Yang Liu* and Jin Song Dong*

*National University of Singapore+Singapore University of Technology and Design

Differencing LTSs? WHY?

Program Behaviors Change!

Program behavior changes

Program evolution

Different behavioral exploration

methods

The Evolution of Specification

An Evolved Concurrent Stack Spec in CSP#

An earlier version A later version

•Violating the linearizibility

!

Evolve

The Differences Lead to Program Fault?

The LTS of the earlier version The LTS of the later version

•Diagnosing faulty evolution?

The Application of Partial Order Reduction

A Dinning-Philosophers Spec in CSP#

Dinning-Philosophers

“Reduced” States and Transitions?

No Partial Order Reduction(118 states/300 transitions)

Partial Order Reduction(116 states/248 transitions)

•Impact of partial order reduction?

NOTE: There is nothing wrong with Spec, and Spec remains unchanged!

The Application of Process Counter Abstraction

Parameterized Readers-Writer Lock

A Readers-Writer Lock Spec in CSP#

Cutoff number = 2Cutoff number = 1

Recurring Changes as Cutoff Number Increases?

Cutoff number = 3 Cutoff number = 4

•Behavioral patterns of parameterized systems?

NOTE: There is nothing wrong with Spec, and Spec remains unchanged!

WHY Do We Want to Differencing LTSs?

• Analyzing Changing Program Behaviors – Diagnosing faulty evolution

– Evaluating impact of different behavior exploration methods

– Revealing behavioral change patterns of parameterized systems

– ……

Differencing LTSs? HOW?

SpecDiff Architecture

An Overview of Our SpecDiff Approach1. Describing program behavior in CSP# specification language2. Generating the LTSs of CSP# program(s) with PAT Simulator3. Applying GenericDiff to compare two LTSs4. Visualization and query-based analysis

Differencing LTSs By GenericDiff• Input: LTSs to be compared– LTS1 and LTS2

• GenericDiff: A generic graph differencing technique1. Parsing and quantifying the inputs LTSs

• Typed Attributed Graphs (TAGs)2. Capturing the graph structure and the matching candidates

• PairUpGraph (i.e. a product of two TAGs)3. Traversing the model graphs and computing the similarities

• Random walk on PairUpGraph4. Select an “optimal” matching

• Bipartite graph matching

• Output: Symmetric difference– One set of matched states and transitions– Two sets of unmatched states and transitions

Analyzing LTS Differences• Merging the two LTSs into a unified LTS– Creating the matched parts of two LTSs– Appending the unmatched states and transitions

• Visually inspecting the unified LTS– Normal view of the whole unified LTS– Fragmented views of maximally-connected

matched (or unmatched) subgraphs

• Searching for change patterns– User-defined queries

A fragment of the unified LTS of the evolved concurrent stack example (returned by the query searching for “matched states with unmatched same-label transitions”,

for example, matched states 6/22 with unmatched push.0.1)

Diagnosing Faulty Program Evolution• The second process pops nothing (pop.1.0) after the first

process has pushed an item (push.0.1) into the stack!

Black: matched states/transitions in both LTSs; Green: unmatched states/transitions in the earlier-version LTS; Red: unmatched states/transitions in the later-version LTS

SpecDiff? DOES IT WORK?

Tool Support & Usage

A short DEMO of SpecDiff in PAT!http://www.patroot.com

Formal Tool Demonstration, ASE’10

Initial Evaluation

• The evolution of a concurrent stack• The application of partial order reduction• The application of process counter abstraction

Three pilot-study scenarios

• Debugging faulty program evolution• Understanding the impacts of state reduction

techniques• Revealing behavioral change patterns in the verification

of parameterized systems

SpecDiff assists in

How to Scale it Up?• Differencing LTSs “smartly”– Syntactic differences to “guide” the differencing

process of large LTSs– Interactive visualization techniques to “select”

which part(s) of the LTSs to differentiate

• Optimizing SpecDiff implementation– Direct comparison of the internal data structures

of LTSs instead of the LTSs renderedin the GUI

• Identifying “important” differences– Important differences (e.g. program fault) would

be reflected in the differences of small LTSs

Why Not Use Counter Examples?

• SpecDiff is complementary to counter-example analysis– Contextual information– Highlighted differences

• SpecDiff is useful in other scenarios, such as– Nothing wrong with specification– Specification remains unchanged

Effectiveness & Applicability

SpecDiff: Differencing LTSs

Conclusions and Future Work

Tool Support & Usage

Scale it Up!