MOP: An Efficient and Generic Runtime Verification Framework Feng Chen and Grigore Rosu University...

MOP: An Efficient and Generic Runtime Verification Framework

Feng Chen and Grigore Rosu

University of Illinois at Urbana-Champaign

OOPSLA’07 – 25 Oct 2007

MOP – Grigore Rosu 2

Monitoring of FormalSpecifications in Software

Runtime verification (RV 2001 – 2007)– Observe a run of the system

• Instrumentation

– Check it against desired properties• Explicit or implicit properties

– React /report if needed• Error messages, throw exceptions, recover

OOPSLA’07 – 25 Oct 2007


Runtime Verification Systems

• ≤ 2001– MAC (UPenn), PAX* (NASA), TimeRover (commercial)

• 2002-2004– HAWK/Eagle (NASA), MOP* (UIUC), POTA (UTA)

• ≥ 2005:– PQL (Stanford), Tracematches (Oxford), PTQL– PEX (Microsoft)– MOP* (added universal parameters, more logics)

OOPSLA’07 – 25 Oct 2007


Overview

• What is MOP– Philosophy, Examples, Features

• Decentralized Monitors and Indexing

• Experiments and Evaluation

• Current limitations

• Conclusions and future work

OOPSLA’07 – 25 Oct 2007


(MOP)*Monitoring-Oriented Programming

http://fsl.cs.uiuc.edu/mop- proposed in 2003 –

RV’03, ICFEM’04, RV’05, CAV’05,TACAS’05, CAV’06, CAV’07…

*(not to be confused with Meta-Object Protocol)

http://fsl.cs.uiuc.edu/mop

OOPSLA’07 – 25 Oct 2007


What is MOP?

• Framework where the benefits of monitoring specifications may encourage us to use them– Monitoring is basic design discipline– Recovery allowed and encouraged– Provides to programmers and hides under the hood a

large body of formal methods knowledge/techniques– Generic for different application domains

• Tool supported– JavaMOP – “compiles” MOP for Java into AspectJ

OOPSLA’07 – 25 Oct 2007


Overview


• Decentralized Indexing




OOPSLA’07 – 25 Oct 2007


class Resource {/*@ scope = class logic = PTLTL { Event authenticate: end(exec(* authenticate())); Event use: begin(exec(* access())); Formula : use -> <*> authenticate } violation Handler { @this.authenticate(); }@*/void authenticate() {...}void access() {...}...}

JavaMOP Example: “Enforce authentication before use”

OOPSLA’07 – 25 Oct 2007


/*@ scope = class logic = ERE {

Event openRegKey: end(exec(* openRegKey()));Event closeHandle: begin(exec(* closeHandle()));Formula : (openRegKey closeHandle)*

} validation handler { @this.closeRegKey(); return; }@*/

Method openRegKey should be followed b

y closeRegKey, not by closeHandle

JavaMOP Example: Correcting method matching

OOPSLA’07 – 25 Oct 2007


/*@ scope = class logic = ERE [ int count = 0; int writes = 0;] {

Event open : end(call(* open(..))) {writes = 0;};Event write : end(call(* write(..))) {writes ++;} ;Event close : end(call(* close(..)));Formula : (open write+ close)*

} violation handler{ @RESET; } validation handler{

count ++; File2.log(count + ": " + writes);

}@*/

JavaMOP Example: ProfilingHow many times a file is open, written to, and then closed?

OOPSLA’07 – 25 Oct 2007


Multi-Parameter Specifications

• Until now, MOP was limited to one-parameter specifications; the examples above showed only such specifications: parameter = current object

• Because of that restrictionmonitor states were always stored as fields into

the monitored objects in MOP

• Situation is more complex for multiple-parameter specifications …

OOPSLA’07 – 25 Oct 2007


Fail Fast Iterators in Java

Vector v = new Vector();v.add(new Integer(1));Iterator i = v.iterator();v.add(new Integer(2));

• Following code throws exception in Java:

• No exception raised if one uses Enumeration instead of Iterator– Java language decision, showing that properties

referring to sets of objects are important

OOPSLA’07 – 25 Oct 2007


JavaMOP Example:Safe Enumeration

• (borrowed from Tracematches)– We thank Tracematches team for their useful

collection of regular pattern properties!

• Safety property:– If nextElement() invoked on an enumeration

object, then the corresponding collection (vector) is not allowed to change after the creation of the enumeration object

create next* updatesource+ next

OOPSLA’07 – 25 Oct 2007


/*@scope = globallogic = ERESafeEnum(Vector v, Enumeration+ e) {Event create<v,e> : end(call(Enumeration+.new(v, ..))) with e;Event updatesource<v> : end(call(* v.add*(..))) \/ … ;Event next<e> : begin(call(Object e.nextElement()));Formula : create next* updatesource+ next}validation handler { System.out.println("Error ...");}@*/

Multiple-Parameters in JavaMOPSafe Enumeration

JavaMOP generates >200 AspectJ LOC from above

OOPSLA’07 – 25 Oct 2007


Overview






OOPSLA’07 – 25 Oct 2007


MOP Features: Extensible logic framework

• Observation: no silver-bullet logic for specs• MOP logic plugins (the “How”): encapsulate moni

tor synthesizers; so far we have plugins for– ERE (extended regular expressions)– PtLTL (Past-time LTL) and FtLTL (Future-time LTL)– ATL (Allen temporal logic),– JML (fragment of Java modeling language);– NEW: CFG and ptCaRet plugins (need stack)

• Generic universal parameters– Allow monitor instances per groups of objects

OOPSLA’07 – 25 Oct 2007


MOP Features: Configurable monitors

Working scope (the “Where”)– Check point: check spec at defined place– Method: pre-post condition– Class: check spec everywhere during obj lifetime– Interface: check spec at boundaries of methods– Global: may refer to more than one object

Running mode– Inline: shares resources with application– Outline: communicates with application via sockets– Offline: generated monitor has random access to log

OOPSLA’07 – 25 Oct 2007


MOP Features: Programmable Actions

• Monitor variables

• Event-triggered actions (any code)

• Violation/validation handlers (any code)

• Logic-based AOP instance– Aspects are formal specifications– MOP aim: generate optimal AOP code

OOPSLA’07 – 25 Oct 2007


Overview






OOPSLA’07 – 25 Oct 2007


Decentralized Monitoring/Indexing(works only for inline MOP)

• The problem: how to monitor a universally quantified specification efficiently!

create<v,e>udatesource<v>next<e>

create next* updatesource+ next

( v,e)

OOPSLA’07 – 25 Oct 2007


“Super-Logic” Approach

• Eagle, PQL, PTQL, Tracematches, …• Choose a universally quantified logic• Device monitor synthesizers for it:

• Doable in MOP, too, but against its spirit

– Logic plugins only know how to generate monitors for – No relationship between monitors for and for (p)

(p) Mon(p)

OOPSLA’07 – 25 Oct 2007


MOP’s Decentralized Monitors

Monitor instances(one per parameter instance)

Mp1

Mp2

Mp3

… Mp1000

OOPSLA’07 – 25 Oct 2007


Indexing …

• The next problem: how can we retrieve all needed monitor instances efficiently?

Mp1

Mv,e1

Mv,e2

… Mp1000

udatesource<v>

Naïve implementation can be very inefficient (both time- and memory-wise)

OOPSLA’07 – 25 Oct 2007


MOP’s Centralized Indexing

• One index tree per parameter set

• Weak references

SafeEnum events


OOPSLA’07 – 25 Oct 2007


MOP’s Decentralized Indexing

• Monitors scattered all over the code

• Monitor states piggybacked to object states

SafeEnum events


OOPSLA’07 – 25 Oct 2007


Overview






OOPSLA’07 – 25 Oct 2007


The JavaMOP Implementation

Short story:

1. Parse the MOP specifications– Generate all monitors using the logic plugins– Find instrumentation points

2. Generate standard AspectJ code

3. Wave everything (observation, monitoring, handlers) together using AspectJ compiler

OOPSLA’07 – 25 Oct 2007


MOP Evaluation on DaCapo:Statistics

OOPSLA’07 – 25 Oct 2007


MOP Evaluation on DaCapo:Runtime Overhead

Summary:MOP runtime overhead larger than 10% in only 8% of programs

OOPSLA’07 – 25 Oct 2007


Evaluation of MOP on TM

• MOP is generic (logic, etc.), but its generality does not come at a performance cost!

• Following programs part of Tracematches’ (TM) benchmarks (hand optimized by TM)

OOPSLA’07 – 25 Oct 2007


Overview






OOPSLA’07 – 25 Oct 2007


Limitations of the Current Implementation of JavaMOP

• No nested or existential parameters

• Monitor creating events must contain all the parameters of the specification

• No support for real-time

• Inherit the limitations of AspectJ– Would like to have dynamic aspects …– Though shouldn’t miss events that trigger actions

OOPSLA’07 – 25 Oct 2007


Conclusions and Future Work

• MOP is a generic, yet efficient RV framework• Experiments showed that RV is feasible• Proposed a logic-independent approach to

support universally quantified specifications– With both centralized and decentralized indexing

• Future work– Use MOP for improved and rigorous testing

• Test case generation, sliced causality

– Real-time logic plugins– Static analysis (plugins) to reduce runtime overhead

OOPSLA’07 – 25 Oct 2007


Related Work

Extensively discussed in paper• Programming languages

– PQL (Lam et al., OOPSLA ’05)– Tracematches (de Moor et al., OOPSLA ’05, ‘07)– PTQL (Goldsmith et al., OOPSLA’05)

• Runtime verification– MAC (Lee et al., RV’01)– PAX* (Havelund, Rosu, RV’01)– TimeRover (Drusinski, RV’02)– HAWK/Eagle (Barringer et al., VMCAI’04)– PEX (Schulte, Tilman; Microsoft)

OOPSLA’07 – 25 Oct 2007


OOPSLA’07 – 25 Oct 2007


Backup slides

OOPSLA’07 – 25 Oct 2007


Application to Verification

isSorted?

No

Yes

FancySort

InsertionSort

/*@ logic = JML {

ensures isSorted(array); } Violation Handler {

insertionSort(array); }@*/

Here is an MOP approach to validate a sorting program:

OOPSLA’07 – 25 Oct 2007


Monitoring-based Verification?

Monitor & Recover: presumably reduces the complexity of program verification

Monitor

Incorrect

CorrectComplex

Component

SimpleComponent

Architecture

φ p= φc & (φm \/ φs )

φcφm

φs

Is sorted?Correct sorting

Array elementsunchanged

OOPSLA’07 – 25 Oct 2007


Why Monitoring I

• Monitoring is well-adopted in many engineering disciplines– Fuses, watchdogs, fire-alarms, etc.

• Monitoring adds redundancy– Increases reliability, robustness and

confidence in correct behavior, reduces risk

OOPSLA’07 – 25 Oct 2007


Why Monitoring II

• Ideally, one would like to prove a program correct with respect to its specification …– Known to be hard

• Provably correct systems can fail, too– Unexpected environment, wrong/strong

assumptions, hardware or OS errors, etc.

OOPSLA’07 – 25 Oct 2007


Monitoring of FormalSpecifications in Software

Runtime verification (RV 2001 – 2007)– Observe a run of the system

• Instrumentation

– Check it against desired properties• Explicit or implicit properties

– React /report if needed• Error messages, throw exceptions, recover

OOPSLA’07 – 25 Oct 2007


MOP Conceptual InstancesMOP[logic, working-scope, running-mode]

• JPaX = MOP[LTL, class, outline/offline]– NASA (joint work with Havelund)

• TemporalRover = MOP[MiTL, class, inline]– Commercial (Drusinski)

• MAC = MOP[PastLTL, class, inline/outline]– UPenn (Lee, Sokolski, ...)

• Hawk = MOP[Eagle, global, inline]– NASA (Havelund and collaborators)

• Tracematches = MOP[RegExp, global, inline]– Oxford (de Moor and students)

• PQL = MOP[CFG, global, inline]– Stanford (Lam and students)

• PTQL = MOP[SQL, global, outline]– Berkeley, Stanford (Aiken and students)

OOPSLA’07 – 25 Oct 2007


Why “Monitoring-Oriented” ?

• We think one can write reliable programs if one adopts a “monitor and recover” style– Not a substitute for writing good code upfront– Write good code, but, “just in case” also say

what to do if requirements are violated/validated

• MOP tools provide support for that

• … may reduce complexity of verification: prove what is easy, monitor the rest

OOPSLA’07 – 25 Oct 2007


MOP Approach to Monitoring: Divide & Conquer

• What properties/specifications to check?– Combine formal specification and implementation

• Where to check it?– Automatic monitor integration

• Currently using Aspect-Oriented Programming

• How to express the property?– Declarative, automatic monitor synthesis

• What if specification violated or validated?– Execute user provided “recovery” code

Date post:	13-Dec-2015
Category:	Documents
Upload:	amy-patterson
View:	213 times
Download:	0 times

MOP: An Efficient and Generic Runtime Verification Framework Feng Chen and Grigore Rosu University...

Documents