CONCURRENCYAND DISTRIBUTED SYSTEMS IN JRUBY

@iconara

speakerdeck.com/u/iconara(real time!)

Theo / @iconara

Chief Architect at

big data with Ruby since 2009

MAKING THIS

INTO THIS

big data with Ruby since 2009

APRIL 2011stuck

MAY 2011let’s try JRuby

JUNE 2011wrapping Java libraries for fun and for profit

HOTBUNNIESnatural Ruby API for RabbitMQ

JULY 2011threads are better than processes

JRUBY

JRUBY

J.U.C(java.util.concurrent)

SHORTCUTJava Concurrency in Practice

EXECUTORSconfigurable thread pools, a better

abstraction than using Thread directly

thread_pool = Executors.new_fixed_thread_pool(16)

all_the_things.each do |thing| thread_pool.submit do crunch_crunch_crunch(thing) endend

BLOCKING QUEUESeasy producer/consumer patterns

J.U.C.ATOMICoptimistic locking primitives,

compare-and-swap, thread safe non-blocking counters

MORE AWESOMESConcurrentHashMap, CountDownLatch,

ForkJoinPool, Google Guava, LMAX distruptor

TL; DRuse j.u.c and avoid mutable state

and you’re safe, mostly

enough ropeTHREADS

AUGUST 2011threads are a mess, let’s try actors

ACTORSa higher level abstraction of concurrency

AKKAconcurrency library for Scala,famous actor implementation

MIKKAAkka wrapper for JRuby

DO ALL THE THINGSProgramming Concurrency on the JVM

AUGUST 2011async systems with no back pressure

mechanism will die of memory starvation

STATE

AGGREGATION

INPUT

PERSISTENCE

OUTPUT

TIMER

STATE

AGGREGATION

INPUT

PERSISTENCE

OUTPUT

TIMER

!

AUGUST 2011actors are cool, but not really

worth it when all the arrows in your flow chart point in the same direction

DIVIDE DIVIDE DIVIDErun multiple independent, synchronous workers,

tune to the number of CPUs & IO wait

SEPTEMBER 2011blocking all the way down,

back pressure all the way up

AUGUST 2011(short rant about MongoDB’s global write lock)

NEW SHINY TOYSfor building new shiny products

CASSANDRAdistributed database, mostly magic

CASSANDRAvery verbose drivers

ColumnFamilyUpdater<String, String> updater = template.createUpdater("a key");updater.setString("domain", "www.datastax.com");updater.setLong("time", System.currentTimeMillis());

try { template.update(updater);} catch (HectorException e) { // hurgh}

EURYDICEJRuby nails it

new_data = {:domain => 'www.datastax.com', :time => Time.now.to_i}column_family.update('a key', new_data)

STORMstream processing framework written in Clojure

STORM(it does the arrows, you do the blobs)

REDSTORMStorm for JRuby

DON’T DO THIS AT HOME KIDS!the key to building concurrent and distributed

systems is to separate what you want to do from the concurrent- and distributed-ness, and let someone

else take care of that

YOUR CODE

JRUBY IS A SLIPPERY SLOPE TO JBOSS

let there be no doubt about it

SCALING IS HARDgo to Srdan’s talk tomorrow

KTHXBAI@iconara

github.com/iconaraarchitecturalatrocities.com

burtcorp.com