Jobim: an Actors Library for the Clojure Programming Language

Antonio GarroteMaría N. Moreno García

Clojure

• New Lisp dialect

• Hosted: JVM, CLR

• Focus on concurrency: (STM, Agents, futures, fork-join, java.util.concurrent.*)

Distributed concurrent applications in Clojure?

“Given the diversity, sophistication, maturity, interoperability,robustness etc of these options, it's unlikely I'm going to fiddle around with some language-specific solution.”

Rich Hickey

Distributed applications on the JVM

• JINI

• Java Spaces

• JMS

• Terracota

• RabbitMQ

• ZeroMQ

• Gearman

Problems

• Different computational models

• Hard to port code from one solution to other

• Friction with Clojure semantics

Extending Clojure

• Computational model?

• Suitable notation?

• Underlying implementation?

Computational Model

Actors: components

• Named channel (PID)

• Message box

• Execution context

PID

MBox ExecutionContext

Computation based on the exchange of messages

Actors: mobile processes

PID1

MBox ExecutionContext

PID2

MBox ExecutionContext

PID3

MBox ExecutionContext

PID3 PID3

Msg

Notation

Actors: minimal interface

(def *pid* (spawn f))

(send! *pid* msg)

(def msg (receive *pid*))

(is (= *pid* (self)))

Selective Reception(let [p (promise) pid (spawn #(let [a (receive odd?) b (receive even?)] (deliver p [a b])))]

(send! pid 2) (send! pid 1)

(is (= @p [1 2])))

Implementation

Message Box

• Lamina - Z. Tellman (https://github.com/ztellman/lamina)

• Transforms Clojure sequences into event-driven channels

• Synchronous and asynchronous interface

PIDs/Channel Names

• Plain Java strings: easy to exchange

• Globally unique identifiers for actors across all the nodes in a Jobim cluster

• Generated by Jobim’s runtime

• GUID node + process counter

Execution context

• Threaded actors:

- Java thread per actor

- End of thread execution, releases resources

- Threaded actors do not scale (~2000 threads per node max.)

Execution context

• Evented actors (Scala):

- Actors context = closure + callback functions

- Small number of reactor threads execute all the evented threads

(spawn #(loop [] (let [[from msg] (receive)] (send! from msg) (recur))))

(spawn-evented #(react-loop [] (react [[from msg]] (send! from msg) (react-recur))))

Threaded

Evented

Multiplexer

Reactor1 ReactorN...

Events

Messages

Callbacks+Contexts queues react-loop

react-recur

listen-once

publish

react-future

thread-pool handler

publish

Distribution

3 problems

Messaging Coordination Serialization

Modular solution

Messaging Coordination Serialization

Protocols: jobim.services.*

Plugins

• Name service

• Membership groups: presence

• Distributed agreement: 2PC protocol

Coordination

Coordination

• Apache ZooKeeper plugin

- Light-weight

- Scalable

- Small set of primitives to build sophisticated coordination protocols

Coordination: name service - group membership

(nodes)

(resolve-node-name node-name)

(register-name name *pid*)

(def *pid* (resolve-name name))

Coordination: 2PC - group membership

(link *pid1* *pid2*)

Signal

Messaging

• TCP plugin: Netty, Z.Tellman’s Aleph [https://github.com/ztellman/aleph]

• RabbitMQ plugin

• ZeroMQ plugin

Serialization

• Java Serialization plugin

• JSON plugin

• Kryo serialization library plugin (Yahoo S4)

Behaviours

Reusing distributed components?

• Encapsulate distributed patterns

• Hide message passing logic

• Building blocks for larger distributed systems

• Built using Clojure protocols

• Threaded and evented versions

Behaviours

• Supervisor

• Generic Server

• FSM

• Event Manager / Event Handler

• Generic TCP server

;; FSM Lock type

(def-fsm Lock (init [this code] [:locked {:so-far [] :code code}]) (next-transition [this state-name state-data message] (let [[topic _] message] (condp = [state-name topic] [:locked :button] handle-button [:open :lock] handle-lock action-ignore))) (handle-info [this current-state current-data message] (do (cond-match [[?from :state] message] (send! from current-state)) (action-next-state current-state current-data))))

Demo

Future work

• Benchmarking + Performance

• Packaging / deployment / managing of distributed apps

• Missing functionality

Code+deps

• https://github.com/antoniogarrote/jobim

[jobim-core "0.1.2-SNAPSHOT"]

[jobim-rabbitmq "0.1.1-SNAPSHOT"]

[jobim-zeromq "0.1.1-SNAPSHOT"] *