Concurrency: Rubies, Plural

Elise Huard & Eleanor McHugh

RubyConf 2010

manifestoconcurrency matters

multicoreit’s a revolution in mainstream computing

and you want to exploit it in Ruby

MULTIPROCESSOR/MULTICORE

NETWORK ON CHIP(50..96..100 CORES)

diminishing returns

• communication takes finite time

• so doubling processors never doubles a system’s realworld performance

• realtime capacity ~70% theoretical capacity

• or less!!!

• independence = performance

“... for the first time in history, no one is building a much faster sequential processor. If you want your programs to run significantly faster (...) you’re going to have to parallelize your program.”

Hennessy and Patterson “Computer Architectures” (4th edition, 2007)

concurrencywhy it really matters

an aide to good design

• decouples independent tasks

• encourages data to flow efficiently

• supports parallel execution

• enhances scalability

• improves program comprehensibility

finding green pasturesadopting concurrency idioms from other languages

victims of choice

Erlang Actors

Go Concurrent Sequential Processes

Clojure Software Transactional Memory

Icon Coexpressions

coroutinessynchronising via transfer of control

icon

• a procedural language

• with a single thread of execution

• generators are decoupled coexpressions

• and goal-directed evaluation

• creates flexible flow-of-control

icon coexpressionsprocedure main()

n := create(seq(1)\10) s := create(squares())c := create(cubes())while write(@n, “\t”, @s, “\t”, @c)

end

procedure seq(start)repeat {

suspend startstart += 1

}end

procedure squares()odds := 3sum := 1repeat {

suspend sum sum +:= oddsodds +:= 2

}end

procedure cubes()odds := 1sum := 1i := create(seq(2))repeat {

suspend sum sum +:= 1 + (odds * 6)odds +:= @i

}end

1 1 1

2 4 8

3 9 27

4 16 64

5 25 125

6 36 216

7 49 343

8 64 512

9 81 729

10 100 1000

ruby fibers

• coexpressions

• bound to a single thread

• scheduled cooperatively

• the basis for enumerators

• library support for full coroutines

ruby coexpressionsdef seq start = 1

Fiber.new do loop do

Fiber.yield start start += 1

endend

end

n = seq()

s = Fiber.new dosum, odds = 1, 3loop do

Fiber.yield sum sum += oddsodds += 2

endend

c = Fiber.new dosum, odds, i = 1, 1, seq(2)loop do

Fiber.yield sum sum += 1 + (odds * 6)odds += i.resume

endend

10.times doputs “#{n.resume}\t#{s.resume}\t#{c.resume}”

end

1 1 1

2 4 8

3 9 27

4 16 64

5 25 125

6 36 216

7 49 343

8 64 512

9 81 729

10 100 1000

ruby coroutinesrequire 'fiber'

def login Fiber.new do |server, name, password| puts "#{server.transfer(Fiber.current)} #{name}" puts "#{server.transfer name} #{password}" puts “login #{server.transfer password}” endend

def authenticate Fiber.new do |client| name = client.transfer "name:" password = client.transfer "password:" if password == "ultrasecret" then client.transfer "succeeded" else client.transfer "failed" end endend

login.transfer authenticate, "jane doe", "ultrasecret"login.transfer authenticate, "john doe", "notsosecret"

name: jane doepassword: ultrasecretlogin succeededname: john doepassword: notsosecretlogin failed

icon revisitedprocedure powers()

repeat {while e := get(queue) do

write(e, “\t”, e^ 2, “\t”, e^ 3)e @&source

}end

procedure process(L)consumer := get(L) every producer := !L do

while put(queue, @producer) doif *queue > 3 then @consumer

@consumer end

global queue

procedure main()queue := []process{ powers(), 1 to 5, seq(6, 1)\5 }

end

1 1 1

2 4 8

3 9 27

4 16 64

5 25 125

6 36 216

7 49 343

8 64 512

9 81 729

10 100 1000

a ruby equivalentrequire 'fiber'

def process consumer, *fibersq = consumer.transfer(Fiber.current)fibers.each do |fiber|

while fiber.alive?q.push(fiber.resume)q = consumer.transfer(q) if q.length > 3

endendconsumer.transfer q

end

powers = Fiber.new do |caller|loop do

caller.transfer([]).each do |e|puts "#{e}\t#{e ** 2}\t#{e ** 3}" rescue nil

endend

end

low_seq = Fiber.new do5.times { |i| Fiber.yield i + 1 }nil

end

high_seq = Fiber.new do(6..10).each { |i| Fiber.yield i }

end

process powers, low_seq, high_seq

1 1 1

2 4 8

3 9 27

4 16 64

5 25 125

6 36 216

7 49 343

8 64 512

9 81 729

10 100 1000

processesthe traditional approach to concurrency

language VM

OS(kernel processes, other processes)

Your program

multicore - multiCPU

processes + threads

Process 2

RAMmemory space

Process 1

thread1

scheduler (OS)

CPU CPU

memory space

thread2 t1 t2 t3

cooperative preemptive

active task has full control scheduler controls task activity

runs until it yields control switches tasks automatically

scheduler activates new task a task can still yield control

blocking I/O blocks the system blocking I/O blocks the task

Classic MacOS MacOS X

schedulers

3 threads, 2 cores

1

2

3

1

1

2

3 1

23

core1 core2 core1 core2

process thread

address space private shared

kernel resources private + shared shared

scheduling kernel varies

communication IPC via kernel in process

control children in process

feature comparison

process creation

• unix spawns

• windows cuts from whole cloth

• ruby wraps this many ways

• but we’re mostly interested in fork

pipes

• creates an I/O channel between processes

• unnamed pipes join two related processes

• posix named pipes

• live in the file system

• have user permissions

def execute &blockchild_input, parent_input = IO.pipepid = fork dochild_input.closeresult = block.callparent_input.write result.to_jsonparent_input.close

endparent_input.closesorted = JSON.parse child_input.readchild_input.closeProcess.waitpid pidreturn sorted

end

forking

context switchingOperating System Benchmark Operation Time (ms)

Linux

spawn new process fork() / exec() 6.000

clone current process fork() 1.000

spawn new thread pthread_create 0.300

switch current process sched_yield() 0.019

switch current thread sched_yield() 0.019

Windows NT

spawn new process spawnl() 12.000

clone current process N/A ---

spawn new thread pthread_create() 0.900

switch current process Sleep(0) 0.010

switch current thread Sleep(0) 0.006

C Benchmarks by Gregory Travis on a P200 MMXhttp://cs.nmu.edu/~randy/Research/Papers/Scheduler/

fork

• exploit OS’s efficiency in spawning processes

• ruby enterprise = patched for COW

• not on Ruby 1.9 (yet)

persistent pipesprocess 1

File.umask 0 MKFIFO = 132syscall MKFIFO, fifo_name, 0666fd = IO.sysopen “server”, File::RDONLYserver = File.new fd, "r"client_name = server.gets.chompputs "#{Time.now}: [#{client_name}]"fd = IO.sysopen client_name, File::WRONLYclient = IO.new fd, "w"message = server.gets.chompclient.puts message.reverseclient.closeserver.closeFile.delete “server”

process 2

File.umask 0 MKFIFO = 132syscall MKFIFO, “client”, 0666fd = IO.sysopen "server", File::WRONLYserver = IO.new fd, "w"server.puts fifo_nameserver.puts "hello world!"server.closefd = IO.sysopen “client”, File::RDONLYclient = IO.new fd, "r"puts client.getsclient.closeFile.delete “client”

shared state hurts

• non-determinism

• atomicity

• fairness/starvation

• race conditions

• locking

• transactional memory

semaphores

• exist independently of processes

• provide blocking access

• allowing processes to be synchronised

• nodes in the file system

• usable from Ruby with syscall

synchronising processesrequire ‘dl’require ‘fcntl’libc = DL::dlopen ‘libc.dylib’ open = libc[‘sem_open’, ‘ISII’]try_wait = libc[‘sem_trywait’, ‘II’]wait = libc[‘sem_wait’, ‘II’]post = libc[‘sem_post’, ‘II’]close = libc[‘sem_close’, ‘II’]

process 1s = open.call(“/tmp/s”, Fcntl::O_CREAT, 1911)[0]wait.call sputs “locked at #{Time.now}”sleep 50puts “posted at #{Time.now}”post.call sclose.call s

process 2s = open.call(“/tmp/s”)t = Time.nowif try_wait.call(s)[0] == 0 then

puts “locked at #{t}”else

puts “busy at #{t}”wait.call sputs “waited #{Time.now - t} seconds”

end

locked at Thu May 28 01:03:23 +0100 2009 busy at Thu May 28 01:03:36 +0100 2009posted at Thu May 28 01:04:13 +0100 2009 waited 47.056508 seconds

complexity

• file locking

• shared memory

• message queues

• transactional data stores

threadsthe popular approach to concurrency

threads under the hood

from http://www.igvita.com/2008/11/13/concurrency-is-a-myth-in-ruby/ @igrigorik

the global lockdown

• compatibility for 1.8 C extensions

• only one thread executes at a time

• scheduled fairly with a timer thread

• 10 μs for Linux

• 10 ms for Windows

the macruby twist

• grand central despatch

• uses an optimal number of threads

• state is shared but not mutable

• object-level queues for atomic mutability

synchronisation

• locks address race conditions

• mutex

• condition variable

• monitor

• deadlocks

• livelocks

RUBY THREADSrequire 'thread'threads = []account = UnsafeAccount.newmutex = Mutex.new10.times dothreads << Thread.new domutex.synchronize doaccount.receive 10

endend

end

threads.each {|t| t.join }

puts account.balance

THREADS + SOCKETSrequire 'socket'require 'thread'require 'mutex_m'

class UDPServerinclude Mutex_m

def start address, port, *options@socket = UDPSocket.new@socket.bind address, port@socket.setsockopt *optionsevent_loop

end

def stoplock@socket.close@socket = nilunlock

end

def serve request["hello", 0]

end

private

def event_looploop do

if sockets = select([@socket]) thensockets[0].each do |s|

spawn_handler send

endend

endend

def spawn_handler sockett = Thread.new(socket) do |s|

message, peer = *s.recvfrom 512reply, status = *serve messageUDPSocket.open.send reply, status, peer[2], peer[1]

endt.join

end

clojure

lisp dialect for the JVM

refs software transactional memory

agents independent, asynchronous change

vars in-thread mutability

check out Tim Bray’s Concur.next series

parallel banking(ns account)

  ; ref  (def transactional-balance (ref 0))

  ; transfer: within a transaction  (defn parallel-transfer [amount]     (dosync        (alter transactional-balance transfer amount)))

  ; many threads adding 10 onto account  (defn parallel-stm [amount nthreads]     (let [threads (for [x (range 0 nthreads)] (Thread. #(parallel-transfer amount)))]        (do           (doall (map #(.start %) threads))           (doall (map #(.join %) threads))))     @transactional-balance)

ruby can do that toorequire 'clojure'

include Clojure

def parallel_transfer(amount)  Ref.dosync do    @balance.alter {|b| b + amount }  endend

def parallel_stm(amount, nthreads)  threads = []  10.times do    threads << Thread.new do      parallel_transfer(amount)    end  end  threads.each {|t| t.join }  @balance.derefend

@balance = Ref.new(0)puts parallel_stm(10,10)

enumerableseveryday ruby code is often naturally concurrent

map/reduce

• decompose into independent elements

• process each element separately

• use functional code with side-effects

• recombine the elements

• intrinsically suited to parallel execution

a two-phase operationconcurrent sequential

(0..5).to_a.each { |i| puts i }

x = 0(0..5).to_a.each { |i| x = x + i }

(0..5).to_a.map { |i| i ** 2 }

(0..5).to_a.inject { |sum, i| sum + i }

the parallel gemrequire 'brute_force'require 'parallel'

# can be run with :in_processes as wellmapped = Parallel.map((0..3).to_a, :in_threads => 4) do |num|  map("english.#{num}") # hash the whole dictionaryend

hashed = "71aa27d3bf313edf99f4302a65e4c042"

puts reduce(hashed, mapped) # returns “zoned”

TECHNIQUES

• event-driven I/O

algebra, actors + eventssynchronising concurrency via communication

process calculi

• mathematical model of interaction

• processes and events

• (a)synchronous message passing

• named channels with atomic semantics

go

• statically-typed compiled systems language

• class-free object-orientation

• garbage collection

• independent lightweight coroutines

• implicit cross-thread scheduling

package mainimport "syscall"

func (c *Clock) Start() {if !c.active {

go func() {c.active = truefor i := int64(0); ; i++ {

select {case status := <- c.Control:

c.active = statusdefault:

if c.active {c.Count <- i

}syscall.Sleep(c.Period)

}}

}()}

}

type Clock struct {Period int64Count chan int64Control chan boolactive bool

}

func main() {c := Clock{1000, make(chan int64), make(chan bool), false}c.Start()

for i := 0; i < 3; i++ {println("pulse value", <-c.Count, "from clock")

}

println("disabling clock")c.Control <- falsesyscall.Sleep(1000000)println("restarting clock")c.Control <- trueprintln("pulse value", <-c.Count, "from clock")

}

produces: pulse value 0 from clock pulse value 1 from clock pulse value 2 from clock disabling clock restarting clock pulse value 106 from clock

a signal generator

RUBY SIGNAL GENERATOR

actor model

• named actors: no shared state

• asynchronous message passing (fire and forget)

erlang

• Actor model: Actors, asynchronous message passing

• actors = “green processes”

• efficient VM (SMP enabled since R12B)

• high reliability

© ericsson 2007

Text

erlang-module(brute_force).-import(plists).-export(run/2).

map(FileName) ->    {ok, Binary} = file:read_file(FileName),    Lines = string:tokens(erlang:binary_to_list(Binary), "\n"),    lists:map(fun(I) -> {erlang:md5(I), I} end, Lines).

reduce(Hashed, Dictionary) ->    dict:fetch(Hashed, Dictionary).

run(Hashed, Files) ->    Mapped = plists:map(fun(I) -> map(I) end, Files),    Values = lists:flatten(Mapped),    Dict = dict:from_list(Values),    reduce(Hashed, Dict).

rubinius: actors

• actors in the language: threads with inbox

• (VM actors to communicate between actors in different VMs)

rubinius actorsrequire 'quick_sort'require 'actor'

class RbxActorSort

def execute(&block)current = Actor.currentActor.spawn(current) {|current|

current.send(block.call) }Actor.receive

end

end

puts q = QuickSort.new([1,7,3,2,77,23,4,2,90,100,33,2,4], RbxActorSort).sort

ruby: revactor

• erlang-like semantics: actor spawn/receive, filter

• Fibers (so cooperative scheduling)

• Revactor::TCP for non-blocking network access (1.9.2) (rev eventloop)

ruby: futuresLazy.rb gem (@mentalguy)require 'lazy'require 'lazy/futures'

def fib(n)  return n if (0..1).include? n  fib(n-1) + fib(n-2) if n > 1end

puts "before first future"future1 = Lazy::Future.new { fib(40) }puts "before second future"future2 = Lazy::Future.new { fib(40) }puts "and now we're waiting for results ... getting futures fulfilled is blocking"

puts future1puts future2

kernel stuff

Some of these problems have been solved before ...

GOLDEN RULES

• beware of shared mutable state

• but: sane ways to handle concurrency

• they are all possible in Ruby

fun

http://www.delicious.com/elisehuard/concurrency

http://www.ecst.csuchico.edu/~beej/guide/ipc/

http://wiki.netbsd.se/kqueue_tutorial

http://www.kegel.com/c10k.html

Elise Huard @elise_huard http://jabberwocky.eu

Eleanor McHugh @feyeleanor http://slides.games-with-brains.net

Further Reading