Testing Atomicity of Composed Concurrent Operations

Ohad Shacham Tel Aviv University

Nathan Bronson Stanford University

Alex Aiken Stanford University

Mooly Sagiv Tel Aviv University

Martin Vechev ETH & IBM Research

Eran Yahav Technion

Concurrent Data Structures

• Writing highly concurrent data structures is complicated

• Modern programming languages provide efficient concurrent collections with atomic semantics

.

.

…

……

…

…

………

Challenge

Testing the atomicity of composed operations

…

……

…

…

………

TOMCAT Motivating Example

attr = new HashMap();…

Attribute removeAttribute(String name){ Attribute val = null; synchronized(attr) { found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } } return val;}

TOMCAT 5*.TOMCAT 6*.

Invariant: removeAttribute(name) returns the value it removes from attr or null

attr = new ConcurrentHashMap();…

Attribute removeAttribute(String name){ Attribute val = null; /* synchronized(attr) { */ found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } /* } */ return val;}

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”); if (found) {

val = attr.get(“A”);

attr.remove(“A”); } return val;

attr.put(“A”, o);

attr.remove(“A”);

o

Linearizability removeAttribute(“A”) {

Attribute val = null;

found = attr.containsKey(“A”); if (found) {

val = attr.get(“A”);

attr.remove(“A”); } return val;

attr.put(“A”, o);

attr.remove(“A”);

attr.put(“A”, o);

attr.remove(“A”);

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”); if (found) { return val;

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”); if (found) { return val; attr.put(“A”, o);

attr.remove(“A”);

attr.put(“A”, o);

attr.remove(“A”);

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”) ; if (found) {

val = attr.get(“A”); attr.remove(“A”);

} return val;

o

o

null

null

o

null

null

o

null

null

o

null

Challenge

Testing the linearizabiliy of composed operations

…

……

…

…

………

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”); if (found) {

val = attr.get(“A”);

attr.remove(“A”); } return val;

attr.put(“A”, o);

attr.remove(“A”);

• Reconstruction in TOMCAT extremely challenging– Large traces– Large number of traces– Bugs occur in rare cases with specific key values

Our Solution

ModularityGenerates simple traces

Base linearizability Restrict generated traces

Influence Restrict generated traces

Enables Env control

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”); if (found) {

val = attr.get(“A”);

attr.remove(“A”); } return val;

attr.put(“A”, o);

attr.remove(“A”);

Modular Checking

Challenge

Testing Linearizability of composed operation in a modular fashion

…

……

…

…

………

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”) ;if (found) {return val;

removeAttribute(“B”) {Attribute val = null;

found = attr.containsKey(“B”) ;if (found) {return val;

removeAttribute(“GGG”) {Attribute val = null;

found = attr.containsKey(“GGG”) if (found) {return val;

attr.put(“A”, o);

removeAttribute(“GGG”) {Attribute val = null;

found = attr.containsKey(“GGG”) if (found) {

val = attr.get(“GGG”) ; attr.remove(“GGG”) ;

}return val;

attr.put(“GGG”, o);

removeAttribute(“K”) {Attribute val = null;

found = attr.containsKey(“K”) if (found) {

val = attr.get(“K”) ; attr.remove(“K”) ;

}return val;

attr.put(“K”, o’);

removeAttribute(“K”) {Attribute val = null;

found = attr.containsKey(“K”)

if (found) {return val;

attr.put(“K”, o’);

removeAttribute(“P”) {Attribute val = null;

found = attr.containsKey(“P”) if (found) {

val = attr.get(“P”) ; attr.remove(“P”) ;

}return val;

attr.put(“P”, o’);

removeAttribute(“P”) {Attribute val = null;

found = attr.containsKey(“P”) if (found) {

val = attr.get(“P”) ; attr.remove(“P”) ;

}return val;

attr.put(“P”, o’);

removeAttribute(“O”) {Attribute val = null;

found = attr.containsKey(“O”) if (found) {

val = attr.get(“O”) ; attr.remove(“O”) ;

}return val;

attr.put(“O”, o’);

removeAttribute(“LL”) {Attribute val = null;

found = attr.containsKey(“LL”) if (found) {

val = attr.get(“LL”) ; attr.remove(“LL”) ;

}return val;

attr.put(“LL”, o’);

removeAttribute(“L”) {Attribute val = null;

found = attr.containsKey(“L”) if (found) {

val = attr.get(“L”) ; attr.remove(“L”) ;

}return val;

attr.put(“L”, o’);

removeAttribute(“G”) {Attribute val = null;

found = attr.containsKey(“G”) if (found) {

val = attr.get(“G”) ; attr.remove(“G”) ;

}return val;

attr.put(“G”, o’);

removeAttribute(“R”) {Attribute val = null;

found = attr.containsKey(“R”) if (found) {

val = attr.get(“R”);

attr.remove(“R”) ;}

return val;

attr.put(“R”, o’);

attr.remove(“R”);

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”); if (found) {

val = attr.get(“A”);

attr.remove(“A”); } return val;

attr.put(“A”, o);

attr.remove(“A”);

Influence Base Environment

Running Example

Attribute removeAttribute(String name){ Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val;}

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”); if (found) {

val = attr.get(“A”);

attr.remove(“A”); } return val;

attr.put(“A”, o);

attr.remove(“A”);

Attribute removeAttribute(String name){ Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val;}

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”); if (found) {

val = attr.get(“A”);

attr.remove(“A”); } return val;

attr.put(“A”, o);

attr.remove(“A”);

attr.put(“A”, o);

attr.remove(“A”);

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”); if (found) { return val;

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”); if (found) { return val; attr.put(“A”, o);

attr.remove(“A”);

attr.put(“A”, o);

attr.remove(“A”);

removeAttribute(“A”) {Attribute val = null;

found = attr.containsKey(“A”) ; if (found) {

val = attr.get(“A”); attr.remove(“A”);

} return val;

o

o

null

null

o

null

null

o

null

null

o

null

COLT

program

CO extractor

candidateCOs

Timeout

instrument linearizability

checking

CO

key/value driver

Non-Lin

Execution

library spec

influence driver

Benchmark

• Used simple static analysis to extract composed operations– 19% needed manual modification

• Extracted 112 composed operations from 55 applications– Apache Tomcat, Cassandra, MyFaces – Trinidad, etc…

• Extracted all composed operations per application

• We did not find additional composed operations– Using Google Code and Koders

112Unknown

59Non

Linearizable

53Unknown

53Unknown

42Non

Linearizable

17Open Non

Linearizable

17Open Non

Linearizable

42Non

Linearizable

27Linearizable

26Globals

Globals

…

…

……

…

……

…

…

……

…

…

…

…

……

…

m.put(k,v)

…

27Linearizable

85Non-linearizable

Easy Detection

Attribute removeAttribute(String name){ Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val;}

Current Work

• Define a class of data-independent composed operations

• Use small model reduction to prove linearizability of data-independent composed operations

• Empirical study for the ratio of real life data-independent composed operations

Summary

• Writing concurrent data structures is hard• Employing atomic library operations is error prone• Modular linearizability checking• Leverage influence

• Sweet spot– Identify many important bugs together with a traces showing and

explaining the violations– Hard to find– Simple and efficient technique