Actor Frameworks for the JVM Platform
Rajesh Karmani*, Amin Shali, Gul AghaUniversity of Illinois at Urbana-Champaign
08/27/2009
2
Multi-core, many-core programming Erlang E Axum Stackless Python Theron (C++) RevActor (Ruby) … still growing
3
and on the JVM Scala Actors ActorFoundry SALSA Kilim Jetlang Actor’s Guild Clojure Fan Jsasb … still growing
4
Actor model of programming
Autonomous, concurrent actorsInherently concurrent model of programming
No shared stateNo data races
Asynchronous message-passing Uniform, high-level primitive for both data exchange
and synchronization
“Actors: A Model of Concurrent Computation in Distributed Systems," Gul Agha. MIT Press, 1986.
5
Actor anatomy
Actors = encapsulated state + behavior + independent control + mailbox
Object
6
Standard Actor Semantics
Encapsulation Fairness Location Transparency Mobility
7
Actor Encapsulation
1. There is no shared state among actors Local (private) state
2. Access another actor’s state only by sending messages
Messages have send-by-value semantics Implementation can be relaxed on shared
memory platforms, if “safe”
8
Why Encapsulation?
Reasoning about safety properties becomes “easier”
JVM memory safety is not sufficient for Actor semantics
Libraries like Scala and Kilim impose conventions instead of enforcement Makes it easier to make mistakes
9
Fairness – even the playing field Permanently Disabled Actor: An actor which is
executing an infinite loop, blocked on a external call, or in a deadlock
Enabled Actor: An actor that Is not permanently disabled, AND Has a pending message
Scheduling fairness: An enabled actor is eventually scheduled
10
Fairness – even the playing field Non-cooperating actors can occupy a native
thread indefinitely I/O and System calls, Infinite loops
Non-cooperating actors can starve enabled actors, in the absence of scheduling fairness
Fairness specially critical in libraries for existing languages Interaction with existing code-base, plug-ins, 3rd
party components
11
Why Location Transparent Naming? Enables certain load-balancing and fault-
tolerance mechanisms Run-time can exploit resources available on
cluster, grid or scalable multicores (distributed memory)
Potentially better performance Uniform model for multicore and distributed
programming
12
Why Mobility?
Weak mobility Allow system to run-time to move actors around based on
run-time conditions (also system mobility) Previous work has shown good performance improvement
when augmented with dynamic load balancing techniques1
Strong mobility Allow programmers to exploit heterogeneous resources
declaratively (also programmer mobility) Privacy of data Large amounts of data
Note: Requires encapsulation for efficiency1 “Efficient Support for Location Transparency in Concurrent Object-Oriented Programming Languages”, Kim et al., SC 1995.
13
Comparison of Semantics
Scala Actors
Kilim JavAct Jetlang SALSA AA AF
State Encapsulation No No Yes Yes Yes Yes Yes
Safe Messaging No No No No Yes Yes Yes
Fair Scheduling Yes No No No Yes Yes Yes
Location Transparency No No Yes Yes Yes Yes Yes
MobilityNo No Yes No Yes Yes Yes
14
ActorFoundry
Off-the-web library for Actor programming Major goals: Usability and Extensibility Other goals: Reasonable performance
Supports standard Actor semantics
15
Actor
Actor Control
Mailbox
dequeue and invoke
send
create
Actor Anatomy (Encapsulation)
Actor Name
Actor NameActor Name
Actor NameActor Name
16
ActorFoundry – Programming Model Actors are like objects
Implicit ‘receive’ Run-time provides fetch-decode loop
One-to-one correspondence between message and Java method
Primitives for asynchronous (send) as well as synchronous (call) messages
Wraps standard IO objects as actors (stdout,
stdin, stderr actors)
17
ActorFoundry - basic API
create(node, class, params) Locally or at remote nodes
send(actor, method, params) call(actor, method, params)
Request-reply messages destroy(reason)
Explicit memory management
18
ActorFoundry - Implementation Maps each actor onto a Java thread Actor = Java Object + Java Thread + Mailbox + ActorName
ActorName provides encapsulation as well as location transparency
Message contents are deep copied Fairness: Reliable delivery (but unordered
messages) and fair scheduling Support for distribution and actor migration
19
ActorFoundry - Architecture
TCP, UDP
Broker, Shell
20
Motivation Revisited – Why ActorFoundry? Extensibility
Modular and hence extensible Usability
Actors as objects + small set of library calls Leverage Java libraries and expertise
Performance Let’s check…
21
Great Language Shootout Ported ActorFoundry to Java6 The Computer Language Benchmarks Game [2]
Implemented a concurrent benchmark in ActorFoundry: Thread-ring
Thread-ring: Pass a token around 503 concurrent entities 107 times
Platform Time (s)
Java threads 134.98
Haskell threads 6.70
Erlang light-weight processes 7.49
Scala actors 56.5
ActorFoundry actors 13 minutes[2] http://shootout.alioth.debian.org/
22
and now in a Chart
Intel Core2 Duo, 2.4GHz, 4GB RAM, Java Heapsize: 256M
Coarse-grained actors + Support for standard semantics
23
The quest for Continuations
Kilim - Actor library2 for Java Consists of a run-time and a “Weaver”
(bytecode post-processor) for CPS transform
With a custom continuations based scheduler M:N architecture Credit to the extensible architecture
Threadring performance: ~ 7 minutes
2 http://kilim.malhar.net
24
M:N Runtime Architecture
OS
1 2 3 4 5 6
Worker Threads
Run-time (+ Scheduler)
JVM
Cores
25
M:N Runtime Architecture
OS
1 2 3 4 5 6
Worker Threads
Run-time (+ Scheduler)
JVM
Cores
26
M:N Runtime Architecture
OS
1 2 3 4 5 6
Worker Threads
Run-time (+ Scheduler)
JVM
Cores
27
To copy or not to copy?
Another major bottleneck: deep copying of message contents
By serializing/de-serializing object
28
To copy or not to copy?
1. Exclude immutable types
2. Introduced new run-time functions: sendByRef (actor, method, params) callByRef (actor, method, params)
Threadring performance: ~ 30s
Scala, Kilim provide zero-copy messages Onus on programmers to copy contents, if needed Breaks encapsulation, infeasible in general
29
Fairness – ActorFoundry approach Scheduler thread periodically monitors
“progress” of worker threads Progress means executing an actor from
schedule queue If no progress has been made by any worker
thread => possible starvation Launch a new worker thread Conservative but not incorrect
30
Performance – crude comparison
Threadring benchmark on Intel Core2 Duo, 2.4GHz, 4GB RAM, Heapsize: 256M
31
Performance – crude comparison
Chameneos-redux benchmark on Intel Core2 Duo, 2.4GHz, 4GB RAM, Heapsize: 256M
32
Actor Foundry - Discussion
Elegant, object-like syntax and implicit control Leverage existing Java code and libraries Zero-copy as well as by-copy message
passing Reasonably efficient run-time
With encapsulation and fair scheduling Support for distributed actors, migration
Modular and extensible
33
Using Actor Foundry
Download binary distribution3 Win32, Solaris, Linux, MacOS Bundled with a bunch of example programs
Ant build.xml file included Launch foundry node manager
Specify the first actor and first message Launch ashell for command-line interaction
with foundry node (optional)
3 http://osl.cs.uiuc.edu/af
34
Performance - Can we do better? Possibly, with a locality-aware scheduler
Distribute shared memory among worker threads ..each with a separate scheduling queue Reduces contention and improves locality
Augment with migration logic for dynamic load-balancing
35
Proposed Runtime Architecture
OS
1 2 3 4 5 6
OS
Worker Threads
Run-time (+ Scheduler)
JVM
Cores
36
Promise for scalable performance? Over-decompose application into fine-grained
actors As the number of cores increase, spread out
the actors
Of course, speed-up is constrained by parallelism in application Parallelism is bounded by # of actors
37
Other Future Directions
Garbage collection Debugging Automated testing Model checking
Type-checking messages Support for ad-hoc actor definitions
38
References
[1] Gul Agha. Actors: a model of concurrent computation in distributed systems. MIT Press, Cambridge, MA, USA, 1986.
[2] Rajesh K. Karmani, Amin Shali, and Gul Agha. Actor frameworks for the JVM platform: a comparative analysis. In PPPJ ’09: Proceedings of the 7th International Conference on Principles and Practiceof Programming in Java, 2009. ACM.
[3] Gul Agha, Ian A. Mason, Scott Smith, and Carolyn Talcott. AFoundation for Actor Computation. Journal of Functional Programming,1997.
[4] Rajendra Panwar and Gul Agha. A methodology for programmingscalable architectures. In Journal of Parallel and Distributed Computing,1994.