+ All Categories
Home > Documents > Verification of Evolving Software via Component...

Verification of Evolving Software via Component...

Date post: 02-Apr-2019
Category:
Upload: vanlien
View: 216 times
Download: 0 times
Share this document with a friend
32
Verification of Evolving Software via Component Substitutability Analysis Natasha Sharygina 1,2 Joint work with Edmund Clarke 1 and Nishant Sinha 1 1 Carnegie Mellon University, USA 2 The University of Lugano, Switzerland
Transcript
Page 1: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Verification of Evolving Software via Component Substitutability

Analysis

Natasha Sharygina1,2

Joint work with Edmund Clarke1

and Nishant Sinha1

1Carnegie Mellon University, USA2The University of Lugano, Switzerland

Page 2: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

What is Model Checking?

RequirementsRequirements SpecificationSpecification

CodeCode

AutomatedConformance

Check

AutomatedConformance

Check

ProgramProgram

Page 3: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Model Checking Notation

Program

True

Model Checker

Property

Preprocessor

Finite State Model

False + Counterexamples

•Model Checking Problem:

•If Yes then property holds•If Not Counterexample is generated

M P

M ² P ?

M ² P

C

Page 4: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Abstraction Refinement for Program Verification

VerificationYes

System OKAbstraction

Finite Model

CounterexampleValid?

Program

AbstractionGuidance

Yes

No

Counterexample

AbstractionRefinement

ImprovedAbstractionGuidance

No

SpuriousCounterexample

Page 5: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Motivation

• Model checking is a highly time consuming, labor intensive effort

• For example, a system of 25 components (~20K LOC) and 100+ properties might take up to a month of verification effort

• Discourages its widespread use when system evolves

Page 6: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Software Evolution

• Software evolution is inevitable in any real system:– Changing requirements

– Bug fixes

– Product changes (underlying platform, third-party components, etc.)

Page 7: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Motivation

• Component-based Software– Software modules shipped by separate developers– Undergo several updates/bug-fixes during their

lifecycle

• Component assembly verification– Necessary on upgrade of any component– High costs of complete global verification– Instead check for substitutability of new component

Page 8: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Substitutability Check

• Incremental in nature• Two phases:

– Containment check• All local behaviors (services) of the previous

component contained in new one

– Compatibility check• Safety with respect to other components in

assembly: all global specifications still hold

Page 9: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Containment, Compatibility Duality

ComponentC

Containment Check(local)

Compatibility Check(global)

Identical Behaviors NewBehaviors

Lost Behaviors

Upgraded Component

C’

Page 10: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Substitutability Check

• Approaches– Obtain a finite behavioral model of all

components by abstraction: Labeled Kripkestructures

– Containment: • Use under- and over- approximations

– Compatibility: • Use dynamic assume-guarantee reasoning

Page 11: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Predicate Abstraction into LKS

• Labeled Kripke Structures– <Q,Σ,T,P,L>

• Composition semantics– Synchronize on shared actions

• Represents abstractions

p

!q

q

β

α

γ

!p

C Component Component LKSAbstraction

Predicate Abstraction

Page 12: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Component Assembly• A set of communicating concurrent C programs

– No recursion, procedures inlined

• Each component abstracted into a Component LKS– Communication between components is abstracted into interface

actions

C1 C2 C3

M1 M2 M3

Component Assembly C

Abstraction M

Predicate Abstraction

Page 13: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Predicate Abstraction into LKSL1

lock = 0

if (x < y)

lock=1

x < y

if (x >= y)

lock = 0x < y

if (x < y) x >= y

lock=1void OSSemPend(…) {

L1: lock = 1;if (x < y) {

L2: lock = 0;…

}if (x >= y) {…

L3: lock = 0;…

} else {…

}}

L2

τ

if (x >= y) x >= y

τ

τ

L3

τ

Page 14: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Containment Check

• Goal: Check C µ C’ (or C \ Bug µ C’)– All behaviors retained after upgrade– Cannot check directly: need approximations

• Idea: Use both under- and over-approximations

• Solution: – Compute M: C µ M– Compute M’: M’ µ C’– Check for M µ M’

C

Containment Check

Identical New Lost

C’

Page 15: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Containment (contd.)

C C’

M M’

over-approx under-approx

µ ?True

C µ C’False, CE

CE 2 C ?False,Refine M

CE 2 C’ ?

True,Refine M’

C * C’,CE provided as feedback False

True

M

C

C’

M’

Page 16: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Containment (contd.)

• Computing over-approximation– Conventional predicate abstraction

• Computing under-approximation– Modified predicate abstraction– Compute Must transitions instead of May

Page 17: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Compatibility CheckC

Compatibility Check

Identical New Lost

C’

• Assume-guarantee to verify assembly properties

• Automatically generate assumption A – Cobleigh et. al. at NASA Ames

• Use learning algorithm for regular languages, L*

M1 || A ² PM2 ² A

M1 || M2 ² P

AG - Non Circular

• Goal: Reuse previous verification results

Page 18: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

L* learner

Learning Regular languages: L*• Proposed by D. Angluin, improved by Rivest et al.

– Learning regular sets from queries and counterexamples, Information and Computation, 75(2), 1987.

• Polynomial in the number of states and length of max counterexample

Minimally adequate Teacher

IsMember( trace ρ )

IsCandidate( DFA D )

a

b

a

b

UnknownRegular Language, U

±Counterexample/ Yes

Modelchecker

Yes/No

MinimumDFA

Page 19: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Learning for Verification

• Model checker as a Teacher– Possesses information about concrete components– Model checks and returns true/counterexample

• Learner builds a model sufficient to verify properties• Relies on both learner and teacher being efficient

• Finding wide applications– Adaptive Model Checking: Groce et al.– Automated Assume-Guarantee Reasoning: Cobleigh et al.– Synthesize Interface Specifications for Java Programs: Alur et al.– Regular Model Checking: Vardhan et al., Habermehl et al.

Page 20: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Compatibility Check

R1: M1 || A ² P

R2: M2 ² A

trueL* Assumption

GenerationA

CE

CE AnalysisActual CEM1 || M2 2 P

-CE for A

+CE for A

Teacher

M1 || M2 ² P

true

Page 21: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Handling Multiple Components

• AG-NC is recursive– (Cobleigh et al.)

R1: M1 || A ² PR2: M2 ² A

M1 || M2 ² P

M1 k A1 ² P

M2 k A2 ² A1 M3 ² A2

M2 k M3 ² A1

M1 k M2 k M3 ² P

• Each Ai computed by a separate L* instantiation

Page 22: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Compatibility of Upgrades• Suppose assumptions are available from the old assembly • Dynamic AG: Reuse previous verification results

C

Identical New Lost

C’

• Can we reuse previous assumptions directly?• NO: upgrades may change the unknown U to be learned

• Requires Dynamic L*

M1 k A1 ² P M2 ² A1

M1 k M2 ² P

M’1 k A’1 ² P M2 ² A’1

M’1 k M2 ² P

Upgrade

Reuse?

Page 23: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Dynamic L*

• Learn DFA A corresponding to U

• Unknown language U changes to U’

• Goal: Continue learning from previous model A

• Central Idea: Re-validate A to A’ which agrees with U’

Page 24: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Dynamic L*• L* maintains a table data-structure to store samples

• Definition: Valid Tables– All table entries agree with U

• Theorem – L* terminates with any valid observation table OT

• When U changes to U’, – Suppose the last candidate w.r.t. U is A– Re-validate OT of A w.r.t. U’– Obtain A’ from OT’– Continue learning from A’

Page 25: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Dynamic AG

M1 k A1 ² P M2 ² A1

M1 k M2 ² P

M’1 k A’1 ² P M2 ² A’1

M’1 k M2 ² P

Re-Validate! and Reuse

Upgrade

Page 26: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Implementation

• ComFoRT framework• Industrial benchmark

– ABB Inter-process Communication (IPC) software– 7 main components – CriticalSection, IPCQueue, ReadMQ, WriteMQ +

environment processes

• Evaluated on single and simultaneous upgrades – WriteMQ and IPCQueue components

• Properties– P1: Write after obtaining CS lock– P2: Correct protocol to write to IPCQueue

Page 27: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Experimental Results

10805232Ipc2 (P2)

173624363Ipc3 (P1)

141649258Ipc3 (P2)

241102355Ipc4 (P1)

173286358Ipc2 (P1)

141694308Ipc1 (P2)

132260279Ipc1 (P1)

Tupgrade (msec)Torig (msec)#Mem QueriesUpgrade# (Property)

Page 28: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

ComFoRT Schema

VerificationYes

System OKAbstraction

Model

CounterexampleValid?

System

AbstractionGuidance

Yes

No

Counterexample

AbstractionRefinement

ImprovedAbstractionGuidance

No

SpuriousCounterexample

DynamicAssume-Guarantee Reasoning

Page 29: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Conclusion

• Automated Substitutability Checking– Containment and Compatibility– Reuses previous verification results– Handles multiple upgrades– Built upon CEGAR framework

• Implementation– ComFoRT framework– Promising results on an industrial example

Page 30: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Related Work (1)

• Use of learning in formal verification – Adaptive Model Checking: Groce, Peled, Yannakakis,

TACAS’02

– Automated Assume-Guarantee Reasoning: Cobleigh, Giannakopoulou, Pasareanu, TACAS’03

– Synthesis of Interface Specifications for Java Programs: Alur, Cerny, Gupta, Madhusudan, Nam, Srivastava, POPL’05

Page 31: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Related Work (2)

• Use of learning and predicate abstraction– Abstraction and assume-guarantee reasoning for

automated software verification, Chaki, Clarke, Giannakopoulou, Pasareanu, NASA (RIACS)´04

• Component compatibility– Interface compatibility checking for software modules

Chakrabarti, Alfaro, Henzinger, Jurdzinski, Mang, CAV´02

– Early identification of incompatibilities in multi-component Upgrades, McCamant, Ernst, ECOOP 2004

Page 32: Verification of Evolving Software via Component ...liacs.leidenuniv.nl/~bonsanguemm/fmco/2005/sharygina.pdfAnalysis Natasha Sharygina1,2 Joint work with Edmund Clarke1 and Nishant

Future Directions

• Assume-Guarantee for Liveness

• Other AG Rules, e.g., Circular


Recommended