CELLULOID, CELLULOID::IO AND FRIENDS

Marcelo Correia Pinheiro@salizzar

Monday, July 15, 13

TOPICS

• Reactor Pattern / Actor Model revisited

• Celluloid

• Celluloid::IO

•DCell

• Reel

• #TODO

Monday, July 15, 13

REACTOR PATTERN REVISITED

• Event Handling for concurrent requests

•Multiplex

• X inputs are combined to a single channel

•Demultiplex

• Single channel is converted to X inputs

• Aka Synchronous Event Loop

Monday, July 15, 13

REACTOR PATTERN REVISITED

•Dispatcher

•Dispatch resources from Demultiplexer to related request handler

• Request Handler

• An app that handles request

Monday, July 15, 13

ACTOR MODEL REVISITED

• Carl Hewitt paper from 1973

•Mathematical model of Concurrent Computation

• Known first languages:

• Cosmic Cube

• J-Machine

•Most popular implementation: Erlang

Monday, July 15, 13

ACTOR MODEL REVISITED

• Actor is a entity that interact with other actors sending / receiving messages (mailbox)

• Each actor runs as a independent process

•No shared state

Monday, July 15, 13

CELLULOID

• Ruby Actor Model implementation

• Created by Tony Arcieri - @bascule

• Need Fibers support

• MRI 1.9

• Rubinius / JRuby with 1.9 mode enabled

• Use. Only. Thread. Safe. Libs. For. Your. Sanity.

• Heavily inspired on Erlang concurrency approach

Monday, July 15, 13

CELLULOID

• Automatic Synchronization

• Don’t worry with semaphores / mutex, Celluloid manages :)

• Remember: each actor runs in a thread

• Method dispatch using Fibers

• If method call other actors, Fiber is suspended until call chain returns something

• Example: I/O waiting

Monday, July 15, 13

CELLULOID

# -*- encoding: UTF-8 -*-

require 'celluloid'

class FredFlinstone include Celluloid

def scream(to) @scream = "#{to}#{to[-1] * 10}" @screamed_at = Time.now end

def resume "Screamed [#{@scream}] at #{@screamed_at}" endend

Monday, July 15, 13

CELLULOID

irb(main):001:0> fred = FredFlinstone.new=> #<Celluloid::ActorProxy(FredFlinstone:0x9cd7cc)>irb(main):002:0> fred.async.scream "Wilma"=> nilirb(main):003:0> fred.resume=> "Screamed [Wilmaaaaaaaaaaa] at 2013-07-10 23:01:29 -0300"

Monday, July 15, 13

CELLULOID

• Fault-tolerance

• Erlang philosophy: let it crash

• Celluloid handles crashed actors with these mechanisms:

• Supervisors

• Supervision groups

• Linking

Monday, July 15, 13

CELLULOID

• Supervisors

• How actors crash? Simple: unhandled exceptions

• Warning #1: async calls that raises an error crashes the message receiver ; posterior calls NOT RAISES ANYTHING.

• Warning #2: actors spawns a native Thread, that are not automatically cleaned by GC; you *must* explicitly terminate them if not crashed.

• Supervise to the rescue

Monday, July 15, 13

CELLULOID

# -*- encoding: UTF-8 -*-

require 'celluloid'

class Devops include Celluloid

def initialize(name) @name = name end

def up_to_no_good @bad_cmd = 'rm-f /' @command = `#{@bad_cmd}`, @executed_at = Time.now endend

Monday, July 15, 13

CELLULOID

irb(main):001:0> supervisor = Devops.supervise "@salizzar"=> #<Celluloid::ActorProxy(Celluloid::SupervisionGroup:0x91e6b4) (...)irb(main):002:0> salizzar = supervisor.actors.first=> #<Celluloid::ActorProxy(Devops:0x907c84) @name="@salizzar">irb(main):003:0> salizzar.async.up_to_no_good=> nilE, [2013-07-10T23:26:54.507455 #2467] ERROR -- : Devops crashed!Errno::ENOENT: No such file or directory - rm-f /! /vagrant/examples/supervisor.rb:14:in ``'! /vagrant/examples/supervisor.rb:14:in `up_to_no_good'! (...)irb(main):004:0> salizzar=> #<Celluloid::ActorProxy(Devops) dead>irb(main):005:0> salizzar.terminateCelluloid::DeadActorError: actor already terminated(...)irb(main):006:0> salizzar = supervisor.actors.first=> #<Celluloid::ActorProxy(Devops:0x4e99b8) @name="@salizzar">

Monday, July 15, 13

CELLULOID

• Supervision Groups

• Supervise many actors at once

• Able to supervise other groups too

• You can create pools of supervised actors

• Transparent GC cleaning (automatic terminate all supervised actors)

Monday, July 15, 13

CELLULOID

# -*- encoding: UTF-8 -*-

require 'celluloid'

class EyeOfSauron < Celluloid::SupervisionGroup supervise FredFlinstone, as: :fred pool Devops, as: :devops_poolend

Monday, July 15, 13

CELLULOID

irb(main):001:0> eye_of_sauron = EyeOfSauron.run!=> #<Celluloid::ActorProxy(EyeOfSauron:0xecaab8) @members=[#<Celluloid::SupervisionGroup::Member:0x00000001d89720(...)irb(main):002:0> fred = Celluloid::Actor[:fred]=> #<Celluloid::ActorProxy(FredFlinstone:0xec39d4)>irb(main):003:0> devops = Celluloid::Actor[:devops_pool]=> #<Celluloid::ActorProxy(Devops:0xe3f0e4) @name="hipster">irb(main):004:0>

Monday, July 15, 13

CELLULOID

• Linking

• Suppose that you have two interdependent actors and want to be notified if one fails

• Association by linking actor that commonly dies and the receiver enables a simple callback when failure occurs

• Very useful to terminate broken actors manually

Monday, July 15, 13

CELLULOID

# -*- encoding: UTF-8 -*-

require 'celluloid'

class RobertoBaggio include Celluloid

class KickedFarAwayError < StandardError; end

def kick_penalty raise KickedFarAwayError, "OH MAMMA MIA! :'(" endend

Monday, July 15, 13

CELLULOID

# -*- encoding: UTF-8 -*-

require 'celluloid'

class GalvaoBueno include Celluloid

trap_exit :penalty_kick

def penalty_kick(player, reason) puts "#{player.inspect} will kick and... #{reason.class}!" 2.times { puts "ACABOOOOOOU! "; sleep(1) } 3.times { puts "EH TETRAAAA! "; sleep(1) } endend

Monday, July 15, 13

CELLULOID

irb(main):001:0> galvao = GalvaoBueno.new=> #<Celluloid::ActorProxy(GalvaoBueno:0x12e6aac)>irb(main):002:0> baggio = RobertoBaggio.new=> #<Celluloid::ActorProxy(RobertoBaggio:0x1312e68)>irb(main):003:0> galvao.link baggio=> #<Celluloid::ActorProxy(RobertoBaggio:0x1312e68)>irb(main):004:0> baggio.async.kick_penalty=> nilE, [2013-07-11T00:05:36.212336 #2586] ERROR -- : RobertoBaggio crashed!RobertoBaggio::KickedFarAwayError: OH MAMMA MIA! :'(! /vagrant/examples/baggio.rb:11:in `kick_penalty'! (...)irb(main):005:0>#<Celluloid::ActorProxy(RobertoBaggio) dead> will kick and... RobertoBaggio::KickedFarAwayError!ACABOOOOOOU!ACABOOOOOOU!EH TETRAAAA!EH TETRAAAA!EH TETRAAAA!

Monday, July 15, 13

CELLULOID

• Futures

• Kind of lazy computation: request a future on method call and only execute it when needed

•When value is required, Celluloid internal threadpool executes method synchronously and returns the result

• Transparent error raising

•No need to explicitly clean up pool, let GC work

Monday, July 15, 13

CELLULOID

# -*- encoding: UTF-8 -*-

require 'celluloid'require 'restclient'

class LazyConsumer include Celluloid

def retrieve RestClient.get('http://www.locaweb.com.br').body endend

Monday, July 15, 13

CELLULOID

irb(main):001:0> consumer = LazyConsumer.new=> #<Celluloid::ActorProxy(LazyConsumer:0x12f4210)>irb(main):002:0> future = consumer.future.retrieve=> #<Celluloid::Future:0x000000025957e8>irb(main):003:0> future.value=> "<!DOCTYPE html>\n<html dir=\"ltr\" lang=\"pt-BR\">\n<head>\n\t <meta http-equiv=\"content-type\" content=\"text/html; charset=utf-8\" />\n <meta name=\"robots\" content=\"index, follow\" />\n(...)”

Monday, July 15, 13

CELLULOID

• Pools

• You can define a pool of actors (ORLY?);

• Default size: cores available on machine (Celluloid.cores)

• Delegates method call to a worker on pool to execute it

• Not sooo great due for GIL on MRI, but is OK when you have async I/O :)

• Main tips:

• Synchronous calls if concurrent access to a resource (via Actor.<#method> or Actor.future.<#method>)

• Asynchronous calls if parallel computation (via Actor.async.<#method>)

Monday, July 15, 13

CELLULOID

# -*- encoding: UTF-8 -*-

require 'celluloid'require 'restclient'

class LazyConsumer include Celluloid

def retrieve RestClient.get('http://www.locaweb.com.br').body endend

Monday, July 15, 13

CELLULOID

irb(main):001:0> Celluloid::Actor.all.size=> 0irb(main):002:0> consumer = LazyConsumer.pool size: 4=> #<Celluloid::ActorProxy(LazyConsumer:0xc69bc0)>irb(main):003:0> Celluloid::Actor.all.size=> 5

Monday, July 15, 13

CELLULOID::IO

• Celluloid plus Evented I/O = Celluloid::IO

• Celluloid with steroids =P

• Uses nio4r (libev native extension) as a Reactor to manage Celluloid Actor Mailboxes

• Great with most-idle connections (sockets, websockets and friends)

•Multiplex message processing and I/O in a transparent way

Monday, July 15, 13

CELLULOID::IO

• Stream-based:

• Celluloid::IO::TCPSocket

• Celluloid::IO::UnixSocket

• Celluloid::IO::TCPServer

• Celluloid::IO::UnixServer

• SSL

• Celluloid::IO::SSLSocket

• Celluloid::IO::SSLServer

• UDP

• Celluloid::IO::UDPSocket

Monday, July 15, 13

CELLULOID::IO# -*- encoding: UTF-8 -*-

require 'celluloid/io'

class WhoisServer include Celluloid::IO

def initialize(host, port) @server = TCPServer.new host, port end

def start ; run ; end def stop ; @server.close if @server ; end def run ; loop { async.handle_connection @server.accept } ; end

def handle_connection(socket) _, port, host = socket.peeraddr domain_id = socket.read.strip socket.write("I received a query to #{domain_id} at #{Time.now}\n") ensure socket.close endend

Monday, July 15, 13

CELLULOID::IO

irb(main):001:0> ws = WhoisServer.new '0.0.0.0', 4343=> #<Celluloid::ActorProxy(WhoisServer:0xa2f404) @server=#<Celluloid::IO::TCPServer:0x00000001652358 @server=#<TCPServer:fd 10>>>irb(main):002:0> ws.async.start=> nil

vagrant@vagrant-debian-wheezy:~$ whois -h localhost -p 4343 xalala.com.brI received a query to xelele.com.br at 2013-07-11 01:00:17 -0300vagrant@vagrant-debian-wheezy:~$ whois -h localhost -p 4343 xirubiru.com.brI received a query to xirubiru.com.br at 2013-07-11 01:01:09 -0300

Monday, July 15, 13

DCELL

•Distributed Ruby (wat) objects as network services

•DCell != DRb (Distributed Ruby)

•DRb comes with Ruby STDLIB

• Ruby specific, not interoperatable with CORBA, RMI, etc

•DCell is built on top of Celluloid::ZMQ

•ØMQ protocol implementation with Celluloid Actors

Monday, July 15, 13

DCELL# -*- encoding: UTF-8 -*-# example from https://github.com/celluloid/dcell :)

require 'dcell'

DCell.start id: 'itchy', addr: 'tcp://127.0.0.1:9001'

class Itchy include Celluloid

def initialize puts "Ready for mayhem!" @n = 0 end

def fight @n = (@n % 6) + 1

puts(@n <= 3 ? "Bite!" : "Fight!") endend

Itchy.supervise_as :itchy ; sleep

Monday, July 15, 13

DCELL

# -*- encoding: UTF-8 -*-# example from https://github.com/celluloid/dcell :)

require 'dcell'

DCell.start id: 'scratchy', addr: 'tcp://127.0.0.1:9002'

itchy_node = DCell::Node['itchy']

puts "Fighting itchy! (check itchy's output)"

6.times do itchy_node[:itchy].fight sleep 1end

Monday, July 15, 13

REEL

• Celluloid::IO web server powered

• Similar syntax to EventMachine

• And, of course, weird and potentially ugly after some time

• Rack support is experimental

• Good with websockets

• Not so fast:

• Goliath < Reel <<<< Thin <<< Node.js

Monday, July 15, 13

REEL

• Let’s show code from https://github.com/salizzar/reel-example

Monday, July 15, 13

#TODO

• Great opportunity to create a DCell similar gem using AMQP

• Stable Rack support for Reel

• Not sure, low usage at this time

• Other wrappers are welcome

• Celluloid::Redis is a great example

• Celluloid wrap != EventMachine wrap

• Use Dependency Injection API (if possible) to wrap sockets with Celluloid::IO instead of STDLIB sockets

Monday, July 15, 13

DOUBTS?

•Questions are free as beer :)

Monday, July 15, 13

THANK YOU! :)

Monday, July 15, 13