Concurrency Theory vs Concurrent Languages · Concurrency Theory vs Concurrent Languages Silvia...

transcript

Concurrency Theory vs

Concurrent Languages

Silvia Crafa!

Universita’ di Padova!

Bertinoro, OPCT 2014

Concurrency Theory vs

Concurrent Languages

Silvia Crafa!

Universita’ di Padova!

Bertinoro, OPCT 2014

Bisimulation inside

The Quest for good Abstractions

✤ When a language has been invented VS when became popular?!

✤ Why has been invented VS why became popular?

Fortran!Lisp!

CobolPascal C!

MLC++!

HaskellJava JavaScript

Ruby !Python

X10ScalaGoC#

Fortran!Lisp!

CobolPascal C!

MLC++!

Ruby !Python

X10ScalaGoC#

Add structure!to the code

Fortran!Lisp!

CobolPascal C!

MLC++!

Ruby !Python

X10ScalaGoC#

OOP to handle!industrial complex !software systems!

Encapsulation/Modularity Interfaces/Code Reuse

Fortran!Lisp!

CobolPascal C!

MLC++!

Ruby !Python

X10ScalaGoC#

INTERNET

Fortran!Lisp!

CobolPascal C!

MLC++!

Ruby !Python

X10ScalaGoC#

INTERNET

less Efficiency !more Portability!Security (Types)!

GUIs and IDEs

Fortran!Lisp!

CobolPascal C!

MLC++!

Ruby !Python

X10ScalaGoC#

INTERNET

GUIs and IDEsProductivity!

Types are burdensome

Fortran!Lisp!

CobolPascal C!

MLC++!

Ruby !Python

X10ScalaGoC#

INTERNET

GUIs and IDEs

CONCURRENCY

Productivity!Types are burdensome

✤ Changes need a catalyser

Fortran!Lisp!

CobolPascal C!

MLC++!

Ruby !Python

X10ScalaGoC#

CONCURRENCYINTERNET

Fortran!Lisp!

CobolPascal C!

MLC++!

Ruby !Python

X10ScalaGoC#

CONCURRENCYINTERNET

✤ new hardware can only be parallel!✤ new software must be concurrent

Fortran!Lisp!

CobolPascal C!

MLC++!

Ruby !Python

X10ScalaGoC#

Popular Parallel Programming

Grand Challenge

CONCURRENCYINTERNET

✤ new hardware can only be parallel!✤ new software must be concurrent

How hard is Concurrent Programming?

✤ (correct) concurrent programming is difficult!

✤ Adding concurrency to sequential code is even harder

Intrinsic reasons!nondeterminism

Accidental reasons!improper !

programming model

Think concurrently!(Concurrent Algorithm)

Translate into !a concurrent code

programming model

DESIGN of !concurrent language

programming model

High-level Concurrency Abstraction

DESIGN of !concurrent language

Expressiveness!Performance

Easy to think!Easy to reason about

✤ OOP!✤ encapsulation !✤ memory management !✤ multiple inheritance

C++ —> Java —> Scala

✤ Types!✤ documentation vs verbosity

C++ —> Java —> Ruby —>Scala

✤ Types!✤ documentation vs verbosity

C++ —> Java —> Ruby —>Scala

✤ Functional Programming!✤ composing and passing behaviours!✤ sometimes imperative style is easier to reason about

C#—> Scala C++11, Java8

✤ OOP

✤ Types

✤ Functional Programming

which abstractions !interoperate!

productively?

Concurrency Abstractions?!Many Concurreny Models…

✤ Shared Memory Model and “Java Threads”

synchronized(lock)!lock.wait()!lock.notify()

atomic {…}!when(cond){…}

async{}!finish{}

Java STM X10

async{}!finish{}

Java STM X10

✤ logical threads distinguished from executorsScalability!

(activities/tasks) (pool of thread workers)

new Thread().start()!

JVM threadLightweight threads in the program!

Pool of Executors in the runtime

async{}!finish{}

Java STM X10

✤ logical threads distinguished from executorsScalability!

(activities/tasks) (pool of thread workers)

Many Concurrency Models

✤ GPU Concurrency Model!✤ Massive data parallelism!✤ integration with high-level concurrent language

(X10, Nova, Scala heterogeneous compiler)

✤ Shared Memory !✤ is very natural for “centralised algorithms”

and components operating on shared data!✤ is error-prone when the sole purpose of SM

is thread communication

✤ Message Passing Model !✤ It is the message that carries the state!!✤ Channel based: Google’s GO!✤ Actor Model: Erlang, Scala. It fits well

both OOP and FP!✤ Sessions

Many Concurrency Models

✤ GPU Concurrency Model

✤ Shared Memory

✤ Message Passing Model

which abstractions !interoperate!

productively?

Fortran!Lisp!

CobolPascal C!

MLC++!

Ruby !Python

X10ScalaGoC#

CONCURRENCYINTERNET

✤ New catalyser:

Fortran!Lisp!

CobolPascal C!

MLC++!

Ruby !Python

X10ScalaGoC#

CONCURRENCYINTERNET

✤ multicore —> concurrent programming!

✤ cloud computing —> distributed programming

DISTRIBUTION

✤ New catalyser:

Fortran!Lisp!

CobolPascal C!

MLC++!

Ruby !Python

X10ScalaGoC#

Reactive Programming

CONCURRENCYINTERNET

DISTRIBUTION

✤ New catalyser:

Reactive Programming✤ react to events!

✤ react lo load!!

✤ react to failures

✤ react lo load!!

✤ futures!

✤ push data to consumers when available rather than polling

instead of issuing a command that asks for a change, react to an event that indicates

that something has changed!✤ event - driven!✤ asynchronous

✤ react lo load!!

✤ futures!

✤ scalability!✤ up/down +/- CPU nodes!✤ in/out +/- server

✤ react lo load!!

✤ futures!

✤ resiliency

✤ scalability!✤ up/down +/- CPU nodes!✤ in/out +/- server

Fortran!Lisp!

CobolPascal C!

MLC++!

Ruby !Python

X10ScalaGoC#

CONCURRENCYINTERNET

DISTRIBUTION✤ New catalyser:

Fortran!Lisp!

CobolPascal C!

MLC++!

Ruby !Python

X10ScalaGoC#

CONCURRENCYINTERNET

DISTRIBUTION✤ New catalyser:

✤ big data application —> High Performance Computing

BIG DATA

High Performance Computing✤ scale-out on massively parallel hardware!

✤ high-performance computing on supercomputers!✤ analytic computations on big data!

✤ a single program !✤ runs on a collection of places on a cluster of computers!✤ can create global data-structures spanning multiple places!✤ can spawn tasks at remote places, detecting termination of arbitrary trees of

spawned tasks

✤ Big Data Application Framework !✤ Map - Reduce Model!✤ Bulk Synchronous Parallel Model

spawned tasks

✤ Big Data Application Framework !✤ Map - Reduce Model!✤ Bulk Synchronous Parallel Model

spawned tasks

“Concurrent Patterns” with their

distinctive abstractions

What about Theory ?

The X10 experience

The X10 programming language✤ open-source language for HPC programming!

!✤ key design features: !

✤ scaling: code running on 100 - 10.000 multicore nodes (up to 50millions core)!

✤ productivity: high level abstractions (Java-like, Scala-like) + typing (constrained dependent types as contracts). !

✤ performance on heterogeneous hardware: it compiles to Java, to C++, to CUDA. Resilient extension!

✤ concurrent abstractions: place-centric, asynchronous computing

The X10 programming language

// double in parallel all the array elements!

val a:Array[Int]= …!

for(i in 0..(a.size-1))!

async { a(i)*=2 }!

println (“The End”)

Spawns an asynchronous !lightweight activity !running in parallel

for(i in 0..(a.size-1))!

async { a(i)*=2 }!

Spawns an asynchronous !lightweight activity !running in parallel

waits for the termination!of all the spawned activities

finish

var b=0!

finish for(i in 0..(a.size-1))!

async { a(i)*=2 !

atomic { b=b+a(i) }!

STM when(cond) s!clocks

class HelloWholeWorld {!

public static def main(args:Rail[String]) {!

finish for (p in Place.places())!

async at(p)!

Console.OUT.printnl(“Hello from place “+p)!

Console.OUT.printnl(“Hello from everywhere”)!

async at(p)!

%X10_NPLACES=4!Hello from place 1!Hello from place 2!Hello from place 0!Hello from place 3!Hello from everywhere

async at(p)!

%X10_NPLACES=4!Hello from place 1!Hello from place 2!Hello from place 0!Hello from place 3!Hello from everywhere

Async Partitioned Global Address Space

✤ A global address space is divided into multiple places (=computing nodes)!✤ Each place can contain activities and objects!

✤ An object belongs to a specific place, but can be remotely referenced!✤ DistArray is a data structure whose elements are scattered over multiple places

DistArray

Immutable data, class, struct, function

async at at at

asyncActivity

Object

Address space Place 0 Place MAX_PLACES-1...Place 1

Remote ref

Resilient X10: if a node fails….

✤ it is relatively easy to localize the impact of place death!✤ Objects in other places are still alive, but remote references become inaccessible!✤ Execution continues using the remaining nodes!✤ Happens Before Relation between remaining statements is preserved (HB

Invariance) – no new race conditions, or sequentialization induced by failure.

DistArray

async at at at

asyncActivity

Object

Remote ref

Resilient X10: if a node fails….

✤ it is relatively easy to localize the impact of place death!✤ Objects in other places are still alive, but remote references become inaccessible!✤ Execution continues using the remaining nodes!✤ Happens Before Relation between remaining statements is preserved (HB

Invariance) – no new race conditions, or sequentialization induced by failure.

DistArray

async at at at

asyncActivity

Object

Remote ref

finish async at atomic clock!local/global references!

place failures!

can be mixed in any way!

SEMANTICS !!!!

TX10object id global object id

exception

error propagation !and handling

Semantics of (Resilient) X10 [ECOOP 2014] S.Crafa, D.Cunningham, V.Saraswat, A.Shinnar, O.Tardieu

Values v ::= o | o$p | E | DPE

Expressions e ::= v | x | e.f | {f :e, . . . , f :e} | globalref e | valof e

Statements s ::= skip; | throw v | valx = e s | e.f = e; | {s t}

at(p)valx = e in s | async s | finish s | try s catch t

at(p) s | async s | finishµ s

Configurations k ::= hs, gi | g

Local heap h ::= ; | h · [o 7! (fi : vi)]Global heap g ::= ; | g · [p 7! h]

Semantics of (Resilient) X10✤ Small-step transition system, mechanised in Coq!

✤ non in ChemicalAM style (better fits the centralised view of the distributed program)

hs, gi E⌦�!p hs0, g0i | g0hs, gi E⇥�!p hs0, g0i | g0hs, gi �!p hs0, g0i | g0

Async failures arise in parallel threads!and are caught by the inner finish waiting for their termination!

finish {async throw E async s2}

Synch failures lead to the failure of any sync continuation!leaving async (remote) running code free to terminate!

{async at(p)s1 throw E s2}

Synch failures lead to the failure of any sync continuation!leaving async (remote) running code free to terminate!

{async at(p)s1 throw E s2}

Proved

in Coq

Proved

in Coq

Proved

in Coq Absence of stuck states !

(the proof can be run, yielding an interpreter for TX10)

Semantics of Resilient X10smoothly scales to node failure, with!✤ global heap is a partial map: dom(g) collects non failed places!✤ executing a statement at failed place results in a DPE!✤ place shift at failed place results in a DPE!✤ remote exceptions flow back at the remaining finish masked as DPE

p 2 dom(g)

hs, gi �!p hs, g \ {(p, g(p)}i

(Place Failure)p /2 dom(g)

hskip, giDPE⌦��!p g

hat(p) s, giDPE⌦��!q g

hasync s, giDPE⌦��!p g

contextual rules !modified accordingly

Semantics of Resilient X10✤ Happens Before Invariance!

✤ failure of place q does not alter the happens before relationship between statement instances at places other than q

at(0) { at(p) finish at(q) async s1 s2}

at(0) finish { at(p){at(q) async s1} s2}

s2 runs at 0 after s1

s2 runs at 0 in parallel with s1

p fails while s1 !is running at q

same behaviour!p fails while s1 !is running at q

throws v

flows at place 0 while s2 is running

flows at place 0 discarding s1DPE⌦

Equational theory for (Resilient) X10equivalent configurations when hs, gi ⇠= ht, gi

✤ transition steps are weakly bi-simulated!

✤ under any modification of the shared heap by current activities (object field update, object creation, place failure)

hs, gi R ht, gi whenever

1. `isSync s i↵ `isSync t

2. 8p,8� environment move

if hs,�(g)i ��!p hs0, g0i then 9t0. ht, �(g)i �=)p ht0, g0i

with hs0, g0i R ht0, g0i and viceversa

models the update of g:dom(�(g)) = dom(g) and

8p2dom(g) dom(g(p)) ✓ dom(�(g)(p))

Bisimulation whose Bisimilarity is a congruence

models the update of g:dom(�(g)) = dom(g) and

8p2dom(g) dom(g(p)) ✓ dom(�(g)(p))

Equational theory for (Resilient) X10{{s t} u} ⇠= {s {t u}}

` isAsync s try {s t} catchu ⇠= {try s catchu try t catchu}

at(p){s t} ⇠= {at(p)s at(p)t}

at(p)at(q)s ⇠= at(q)s

async at(p)s ⇠= at(p) async s

finish {s t} ⇠= finish s finish t

finish {s async t} ⇠= finish {s t}

finish at(p) s ⇠= at(p) finish s

if s throws a sync exc. and home is failed, then l.h.s. throws a masked DPEx while r.h.s. re-throws vx since synch exc are not masked by DPE

Conclusions✤ Concurrecy is critical for Programming Languages!

✤ heterogeneous concurrency models (Distribution)!

✤ What is the right level of abstraction? !✤ What are good abstractions? Expressive, flexible, easy to reason

about, easy to implement in a scalable/resilient way!

✤ Formal method to experiment! !✤ test new primitive, new mix of primitives!✤ tool to reason about programs

�=✏, v⇥ h{s t}, gi ��!p h{s0 t}, g0i | ht, g0i

� = v⌦ h{s t}, gi ��!p hs0, g0i | g0

hs, gi ��!p hs0, g0i | g0

` isAsync t hs, gi ��!p hs0, g0i | g0

h{t s}, gi ��!p h{t s0}, g0i | ht, g0i

(Par Left)

(Par Right)

(v0, g0) = copy(v, q, g)

hat(q)valx = v in s, gi �!p

hat(q){s[v0/

] skip}, g0i

(Place Shift)

hs, gi ��!q hs0, g0i | g0

hat(q) s, gi ��!p hat(q) s0, g0i | g0

hasync s, gi �!p hasync s, gi

(Spawn)

hs, gi ��!p hs0, g0i | g0

� = ✏ hasync s, gi ��!p hasync s0, g0i | g0

�=v⇥, v⌦ hasync s, giv⇥

��!p hasync s0, g0i | g0

(Async)

hs, gi ��!p hs0, g0i

hfinishµ s, gi �!p hfinishµ[� s0, g0i

(Finish)

hfinishµ s, gi �0�!p g0

hs, gi ��!p g0 �0 = E⌦ if �[µ6=; else ✏

(End Finish)

(Exception)

hs, gi ��!p hs0, g0i | g0(Try)

(Skip)

�=✏, v⇥ htry s catch t, gi ��!p htry s0 catch t, g0i | g0

�=v⌦ htry s catch t, gi �!p h{s0 t}, g0i | ht, g0i

hthrow v, gi v⌦�!p g

hskip, gi �!p g Plus rules for expression evaluation

Concurrency Theory vs Concurrent Languages · Concurrency Theory vs Concurrent Languages Silvia...

Documents