Hacking with ruby2rubyMarc Chung

OpenRainblog.marcchung.comblog.openrain.com@heisenthought

Hacking with ruby2rubyMarc Chung

OpenRainblog.marcchung.comblog.openrain.com@heisenthought

ruby2ruby

Ruby2RubyWalks the Ruby AST

To emit Ruby

sudo gem install ruby2ruby

Limitations

• Only Ruby 1.8

• Not 1.9, yet

• Built into Rubinius

function rhino_rocks() { return “Rhino, rocks!”;}

js> rhino_rocks();

Rhino, rocks!

js> rhino_rocks.toString();

function rhino_rocks() { return “Rhino, rocks!”;}

js> eval(rhino_rocks.toString()).call();

Rhino, rocks!

ruby_rocks = proc { return “Ruby, rocks”;}

irb> ruby_rocks.call;

Ruby, rocks!

irb> require “ruby2ruby”irb> require “parse_tree_extensions”irb> ruby_rocks.to_ruby

proc { return \“Ruby, rocks\” }

irb> eval(ruby_rocks.to_ruby).call

Ruby, rocks!

define_methodclass Voice define_method(:shout) do |word| word.upcase! endend

puts Ruby2Ruby.translate(Voice)

class Voice < Object def yell(word) word.upcase! endend

aliasclass AliasClass def old_busted "42" end

alias_method :new_hotness, :old_bustedend

puts Ruby2Ruby.translate(AliasClass)

class AliasClass < Object def new_hotness "42" end def old_busted "42" endend

#module_functionmodule HelperModule def help puts "Help me" end module_function : helpend

puts Ruby2Ruby.translate(HelperModule)

module HelperModule def help puts("Help me") end def self. help puts("Help me") endend

ActiveRecord Models

class Doctor < ActiveRecord::Base has_many :patientsend

See, Ruby, Run!

• Drmap

• git clone git://github.com/mchung/drmap.git

• Optionally, install Journeta

Under the hood

• Abstract syntax trees

• S-expressions

• ParseTree

class Book def title “Ruby programming” endend

case NODE_BREAK:case NODE_NEXT: if (node->nd_stts) add_to_parse_tree(self, current, node->nd_stts, locals);

break;

case NODE_YIELD: if (node->nd_stts) add_to_parse_tree(self, current, node->nd_stts, locals);

if (node->nd_stts && (nd_type(node->nd_stts) == NODE_ARRAY || nd_type(node->nd_stts) == NODE_ZARRAY) && !node->nd_state) rb_ary_push(current, Qtrue);

break;

[:class, :Book, [:const, :Object], [:defn, :title, [:scope, [:block, [:args], [:str, "Ruby programming"]]]]]

class Book < Object def title “Ruby programming” endend

/opt/local/usr/bin

• parse_tree_show

• r2r_show

class Book def title “Ruby programming” endend

$ parse_tree_show book.rb

s(:class, :Book, nil, s(:scope, s(:defn, :title, s(:args), s(:scope, s(:block, s(:str, "Ruby programming"))))))

$ r2r_show book.rb

class Book def title() “Ruby programming” endend

Operator Precedence with Guy Decoux: a b c, d

[[:fcall, :a, [:array, [:fcall, :b, [:array, [:vcall, :c], [:vcall, :d]]]]]]

ParseTree.new.parse_tree_for_string(“a b c, d”)

ruby2java

class JavaClass def self.main puts “Java, rocks!” endend

[:class, :JavaClass, [:const, :Object], [:defs, [:self], :main, [:scope, [:block, [:args], [:fcall, :puts, [:array, [:str, "Hello, Java"]]]]]]]

class Ruby2Java < SexpProcessor def process_class(exp) “public class #{exp.shift} #{next_token(exp, true)}” end

def process_const(exp) “extends #{exp.shift}” end....end

public class JavaClass { public static void main(String argv[]) { return “Hello, Java”; }}

Distributing Ruby

• drmap - multi-machine worker queue

• gisting - map/reduce framework

drmapirb> fn = proc {|x| x**10}

irb> Marshal.dump(fn)TypeError: no marshal_dump is defined for class Proc

irb> YAML::dump(fn)=> "--- !ruby/object:Proc {}\n\n"

irb> YAML::dump(fn.to_ruby)=> "--- proc { |x| (x ** 10) }\n"

irb> Marshal.dump(fn.to_ruby)=> "\004\b\"\eproc { |x| (x ** 10) }"

Enumerable#drmap

module Enumerable

def drmap(&block) pool = Drmap::BeanstalkPool.new

jid = rand(100) each_with_index do |element, idx| pool.put_job(jid, block.to_ruby, element) end

results = [] while results.size < length results << pool.next_result(jid) end results end

end

def process loop do begin job_payload = next_job fn = job_payload[:proc] data = job_payload[:data] result = eval(fn).call(data) save(job_payload[:jid], result) success! rescue => e fail!(e) end endend

drmap

• Trivial

• Distributed

• Powerful

Demo time!

Gisting

• MapReduce in Ruby

• http://github.com/mchung/gisting

map / reduce

• Functional programming

• Iterating over collections

• (5..10).map {|x| x+1} #=> [6, 7, 8, 9, 10, 11]

• (5..10).reduce(:+) #=> 45

Google MapReduce

• Modeled after map() and reduce()

• Programming model

• Programs are trivially parallelizable

Demo time!

Gisting::Spec

input.file_pattern = "file.txt" input.map do |map_input| words = map_input.strip.split("\t") Emit(words[1], "1") end

output.reduce do |reduce_input| count = 0 reduce_input.each do |value| count += value.to_i end Emit(count)end

inputs = args spec = Gisting::Spec.new inputs.each do |file_input| input = spec.add_input input.file_pattern = file_input input.map do |map_input| # 2722 mailbox 2006-05-23 00:08:39 words = map_input.strip.split("\t") Emit(words[1], "1") end end output = spec.output output.filebase = "/Users/mchung/Public/datasets/output" output.num_tasks = 2 output.reduce do |reduce_input| count = 0 reduce_input.each do |value| count += value.to_i end Emit(count) end

result = MapReduce(spec) pp result

Gisting

• MapReduce programming model in Ruby

• Uses ruby2ruby to serialize procs

• Uses EventMachine for Map/ReduceServer

• -: 94978

• mailbox: 20872

• google: 6758

• ebay: 2832

• yahoo: 2674

• yahoo.com: 2198

• myspace.com: 1916

• google.com: 1882

self.give(:thanks)OpenRain

openrain.com

My Git Repogithub.com/mchung

Marc Chungblog.marcchung.com

Ruby::AZrubyaz.org

Resources

Code Generation: The Safety Scissors Of Metaprogrammingmwrc2008.confreaks.com/03bowkett.html

Forbidden Fruit: A Taste of Ruby's Parse Treegoruco2008.confreaks.com/03_wanstrath.html

Ruby2Rubyseattlerb.rubyforge.org/ruby2ruby/

ParseTreeparsetree.rubyforge.org