Concern-specific annotations for finding bugs in Java-like programs

Lieven Desmet, Bart Jacobs, Frank Piessens, Jan Smans and Dries Vanoverberghe

K.U. Leuven, Belgium

Wolfram SchulteMicrosoft Research, Redmond

2

Type systems are great!

lightweight

sufficiently expressive

decidable

fast

modular

the most successful formal method!

3

Type systems for Java-like languages

concurrency Type-Based Race Detection for Java

Cormac Flanagan, Stephen N. Freund (PLDI 00) Ownership Types for Safe Programming: Preventing Data Races and

Deadlocks Chandrasekhar Boyapati, Robert Lee, Martin Rinard (OOPSLA 02)

encapsulation Modular Specification and Verification of Object-Oriented Programs

Peter Müller, PhD Simple Loose Ownership Domains

Jan Schäfer, Arnd Poetsch-Heffter (FTFJP 06) non-null

Declaring and checking non-null types in an object-oriented languageK. Rustan M. Leino,Manuel Fähndrich (OOPSLA 03)

security A systematic approach to static access control

François Pottier, Christian Skalka, Scott Smith (ACM TOPLAS 05) sessions

Session Types for Object-Oriented LanguagesMariangiola Dezani-Ciancaglini,Dimitris Mostrous, Nobuko Yoshida and Sophia Drossopoulou (FMCO 06)

...

so, lots of type systems thataddress a specific concern!

4

but...

... how can we combine them in a way they can (re-)use each other’s info?

... type systems can be too stringent.

5

Existing practice

type system/concern type checker/type system verification ≈

run type checkers in sequence

advantages: easy, simple, fast

disadvantages: hard to (re-)use info from other type systems more or less fixed expressiveness mandatory; all or nothing run-time effects

6

Our proposal

(optional) annotations/concern vc generator/annotation verification ≈

generate vc condition use theorem prover

advantages: info can be reused by multiple concerns flexible precision more expressive, less stringent

disadvantages: slower too expressive, then undecidable

7

Three Applications

concurrency-related bugs Ownership Types for Safe Programming: ...

Chandrasekhar Boyapati, Robert Lee, Martin Rinard (OOPSLA 02)

stack inspection-related bugs A Systematic Approach To Static Access Control

François Pottier, Christian Skalka, Scott Smith (ACM TOPLAS 05)

data dependency-related bugs for frameworks using a blackboard style repository

8

Concurrency-related bugs?

data races

deadlocks

9

Data race

A data race occurs when a shared variable is written by one thread and concurrently read or written by another.

class Counter {

int dangerous;

void Inc() {

dangerous++;

}

}

Counter ct = new Counter();

new Thread(ct.Inc).Start();

new Thread(ct.Inc).Start();

//What is the value of the field ct.dangerous?

How to guarantee there are nodata races?

10

Rules and conceptual model

rule: A thread t can only access the fields of an object

o if either the object is thread-local or t holds o’s lock.

conceptual model: Each thread has corresponding access set,

consisting of all objects it has created but not shared yet or whose lock it holds.

Threads can only access (fields of) objects in their access set.

Prevent data races by ensuring that access sets remain disjoint!

11

Annotations and ghost state

Threads have access sets. t.A is a new ghost field per thread describing the set of

objects accessible to that thread.

Objects can be shared. o.shared is a boolean ghost field in each object o share(o); is new operation that shares an unshared o

Fields can be declared to be shared. Shared fields can only be assigned shared objects.

12

Object life cycle

1. A new o is unshared, and added to tid.A.2. An unshared o can be made accessible for other threads by sharing

it; o is taken out of tid.A3. A shared o can be exclusively acquired by locking it; when locking

succeeds o is added to tid.A4. A locked o can be released for others by unlocking it; o is taken out

of tid.A.

free lockednew share

acquire

release

shared

unshared

13

Translation to VC’s

vc(x := o.f; s, Q) ≡

o tid.A && vc(s, Q)[H(o, f)/x]

vc(share (o); s, Q) ≡

o tid.A

&& !o.shared

&& vc(s, Q)[(tid.A\{o})/tid.A, H[o.shared→true]/H]

...

14

Example for Data Race Freedom

class Session {

shared Counter c;

Session(Counter c)

requires c.shared;

ensures this tid.A;

ensures !this.shared;

{ this.c = c; }

void Run()

requires tid.A == {this};

{

lock(this.c) { this.c.x++; }

}

}

Counter c = new Counter();

share(c);

Session s = new Session(c);

Session s2 = new Session(c);

new Thread(s.Run).Start();

new Thread(s2.Run).Start();

15

Example for Data Race Freedom

class Session {

shared Counter c;

threadlocal Session(shared Counter c)

{ this.c = c; }

void Run()

requires tid.A == {this};

{

lock(this.c) { this.c.x++; }

}

}

Counter c = new Counter();

share(c);

Session s = new Session(c);

Session s2 = new Session(c);

new Thread(s.Run).Start();

new Thread(s2.Run).Start();

16

Deadlock

3

21

Fork 1

Fork 2

Fork 3

Dining Philosophers

1 has F1, waits for F22 has F2, waits for F33 has F3, waits for F1

A deadlock occurs when a set of threads each wait for a lock (i.e shared object) that another thread holds.

How to guarantee there are nodeadlocks?

17

Rule

rule: partial order over all shared objects each thread must acquire shared

objects in descending order

18

Annotations and ghost state

We construct a partial order on shared objects, called ≺.

When o is shared, we add edges to the partial order as specified in the share command’s where clause.

Each thread has a new ghostfield lockstack, holding the set of acquired locks.

19

Translation to VC’s

vc(share (o) where p<o<q; s, Q) ≡

o tid.A

&& !o.shared

&& p q≺

&& p o q ≺ ≺

vc(s, Q)[(tid.A\{o})/A, H[o.shared→true]/H]

vc(lock(o) { B }; s, Q) ≡

o.shared

&& (tid.lockstack.Empty || o tid.lockstack.top)≺

&& vc(B; unlock(o); s, Q)[(tid.A\{o})/A, ...]

20

Example for Deadlock Freedom

3

21

Fork 1

Fork 2

Fork 3

left

right left

right

right

left

Fork f1 = new Fork();

share(f1);

Fork f2 = new Fork();

share(f2) where f1 f2;≺

Fork f3 = new Fork();

share(f3) where f2 f3;≺

new Thread( delegate() { lock (f2) { lock (f1) { /*eat*/ }}}

).Start();

new Thread( delegate() { lock (f3) { lock (f2) { /*eat*/ }}}

).Start();

new Thread( delegate() { lock (f3) { lock (f1) { /*eat*/ }}}

).Start();

21

Comparison with Boyapati

advantages: support for more complex ownership

transfer patterns

disadvantages: undecidable slower

22

Conclusion: concurrency-related bugs

data race freedom access set per thread access sets disjoint

deadlock freedom partial order over shared objects lock in descending order

A Statically Verifiable Programming Model for Concurrent Object-Oriented ProgramsBart Jacobs, Jan Smans, Frank Piessens, Wolfram Schulte (ICFEM 06)

Safe Concurrency for Aggregate Objects with InvariantsBart Jacobs, K. Rustan M. Leino, Frank Piessens, Wolfram Schulte (SEFM 05)

23

Stack inspection-related bugs

what is stack inspection? technology for safely executing untrusted code

how? at load-time, each component is assigned a static

permission set

at run-time, each thread maintains a dynamic permission set

∩ of static permission sets of methods on the call stack

before sensitive operation, check dynamic permission set (Demand)

if ok, no-op; otherwise, SecurityException

24

What are stack inspection-related bugs?

SecurityExceptions! similar to Esc/Java’s or Spec#’s treat-

ment of NullPointerExceptions

25

Rule

rule: invoke a sensitive operation (and the

corresponding Demand) only if sufficient permissions are present

26

Annotations and ghost state

Threads have dynamic permission sets. t.Dynamic is a new ghost field per thread describing

t’s dynamic permission set.

Components have static permission sets. staticc is a ghost field per class, describing the static

permissions associated with that class.

Underspecify static permission sets using the Minimum attributes.

27

Translation to VC’s*

vc_sps(p.Demand(); s, Q) ≡

p tid.Dynamic && vc(s, Q)

vc_sps(o.Method(); s, Q) ≡

vc(o.Method(tid.Dynamic); s, Q[...])

for every method body s in a class C:

vc_sps(s, Q) ≡

vc(s, Q[(tid.Dynamic ∩ staticC)/tid.Dynamic])

* essentially a Security-passing Style Transformation(Dan S. Wallach)

28

An example

29

Example (2)class Assembly{

static Assembly LoadFrom(String url)

requires url.StartsWith(“file:”) ==>

tid.Dynamic.Contains(new FileIOPermission(url));

requires ! url.StartsWith(“file:”) ==>

tid.Dynamic.Contains(new WebPermission());

{

if(url.StartsWith(“file:”)){

new FileIOPermission(url).Demand();

//open and return the assembly

} else{

new WebPermission().Demand();

//open and return the assembly

}

}

}

30

Comparison with Pottier

advantages: path-sensitive permission parameters

disadvantages: in general, undecidable slower

31

Conclusion: stack inspection-related bugs

SecurityException freedom keep track of dynamic permission set Demand only if present

Static Verification of Code Access Security Policy Compliance of .NET ApplicationsJan Smans, Bart Jacobs, Frank Piessens (JOT April 06)

Static Verification of Code Access Security Policy Compliance of .NET Applications

Jan Smans, Bart Jacobs, Frank Piessens (.NET Technologies 05)

32

Data Dependency-related bugs

Static Verification of Indirect Data Sharing in Loosely-coupled ComponentSystemsLieven Desmet, Frank Piessens, Wouter Joosen en Pierre Verbaeten (SC 06)

Bridging the Gap between Web Application Firewalls andWeb ApplicationsLieven Desmet, Frank Piessens, Wouter Joosen en Pierre Verbaeten (FMSE 06)

33

Generalization: a methodology for designing concern-specific annotations

1. Pick a concern.

2. Design a set of rules to address the concern.

3. Decide on the conceptual model.

4. Decide on the ghost state to be tracked and corresponding annotations.

5. Define the transformation from annotated programs to VC’s.

6. Prove soundness.

34

Examples of reuse

aliasing

method framing

...

35

Related Work

similar projects:

Enforcing Resource Bounds via Static Verification of Dynamic Checks (JRes)Ajay Chander, David Espinosa, Nayeem Islam, Peter Lee, George Necula (ESOP 05)

...

36

Related Work (2)

Static Typing Where Possible, Dynamic Typing When Needed

Hybrid Type Checking Cormac Flanagan (POPL 06)

Static Typing Where Possible, Dynamic Typing When NeededErik Meijer, Peter Drayton (RDL 04)

Soft TypingRobert Cartwright, Mike Fagan (PLDI 91)

Universes: Lightweight Ownership for JMLWerner Dietl, Peter Müller (JOT 05)

37

Related Work (3)

Pluggable/Optional Type Systems

Pluggable Type SystemsGilad Bracha (RDL 04)

...

38

Future Work

type checking where possible, theorem proving when needed

more case studies e.g. Java type system?

39

Conclusion

type systems → verification system +: more expressive +: (re)-use of information between concerns -: slower -: possibly undecidable

evidence: 3 case studies

40

Thx for listening!

Questions?