Hacking with ruby2rubyMarc Chung
OpenRainblog.marcchung.comblog.openrain.com@heisenthought
Hacking with ruby2rubyMarc Chung
OpenRainblog.marcchung.comblog.openrain.com@heisenthought
ruby2ruby
Ruby2RubyWalks the Ruby AST
To emit Ruby
sudo gem install ruby2ruby
Limitations
• Only Ruby 1.8
• Not 1.9, yet
• Built into Rubinius
function rhino_rocks() { return “Rhino, rocks!”;}
js> rhino_rocks();
Rhino, rocks!
js> rhino_rocks.toString();
function rhino_rocks() { return “Rhino, rocks!”;}
js> eval(rhino_rocks.toString()).call();
Rhino, rocks!
ruby_rocks = proc { return “Ruby, rocks”;}
irb> ruby_rocks.call;
Ruby, rocks!
irb> require “ruby2ruby”irb> require “parse_tree_extensions”irb> ruby_rocks.to_ruby
proc { return \“Ruby, rocks\” }
irb> eval(ruby_rocks.to_ruby).call
Ruby, rocks!
define_methodclass Voice define_method(:shout) do |word| word.upcase! endend
puts Ruby2Ruby.translate(Voice)
class Voice < Object def yell(word) word.upcase! endend
aliasclass AliasClass def old_busted "42" end
alias_method :new_hotness, :old_bustedend
puts Ruby2Ruby.translate(AliasClass)
class AliasClass < Object def new_hotness "42" end def old_busted "42" endend
#module_functionmodule HelperModule def help puts "Help me" end module_function : helpend
puts Ruby2Ruby.translate(HelperModule)
module HelperModule def help puts("Help me") end def self. help puts("Help me") endend
ActiveRecord Models
class Doctor < ActiveRecord::Base has_many :patientsend
See, Ruby, Run!
• Drmap
• git clone git://github.com/mchung/drmap.git
• Optionally, install Journeta
Under the hood
• Abstract syntax trees
• S-expressions
• ParseTree
class Book def title “Ruby programming” endend
case NODE_BREAK:case NODE_NEXT: if (node->nd_stts) add_to_parse_tree(self, current, node->nd_stts, locals);
break;
case NODE_YIELD: if (node->nd_stts) add_to_parse_tree(self, current, node->nd_stts, locals);
if (node->nd_stts && (nd_type(node->nd_stts) == NODE_ARRAY || nd_type(node->nd_stts) == NODE_ZARRAY) && !node->nd_state) rb_ary_push(current, Qtrue);
break;
[:class, :Book, [:const, :Object], [:defn, :title, [:scope, [:block, [:args], [:str, "Ruby programming"]]]]]
class Book < Object def title “Ruby programming” endend
/opt/local/usr/bin
• parse_tree_show
• r2r_show
class Book def title “Ruby programming” endend
$ parse_tree_show book.rb
s(:class, :Book, nil, s(:scope, s(:defn, :title, s(:args), s(:scope, s(:block, s(:str, "Ruby programming"))))))
$ r2r_show book.rb
class Book def title() “Ruby programming” endend
Operator Precedence with Guy Decoux: a b c, d
[[:fcall, :a, [:array, [:fcall, :b, [:array, [:vcall, :c], [:vcall, :d]]]]]]
ParseTree.new.parse_tree_for_string(“a b c, d”)
ruby2java
class JavaClass def self.main puts “Java, rocks!” endend
[:class, :JavaClass, [:const, :Object], [:defs, [:self], :main, [:scope, [:block, [:args], [:fcall, :puts, [:array, [:str, "Hello, Java"]]]]]]]
class Ruby2Java < SexpProcessor def process_class(exp) “public class #{exp.shift} #{next_token(exp, true)}” end
def process_const(exp) “extends #{exp.shift}” end....end
public class JavaClass { public static void main(String argv[]) { return “Hello, Java”; }}
Distributing Ruby
• drmap - multi-machine worker queue
• gisting - map/reduce framework
drmapirb> fn = proc {|x| x**10}
irb> Marshal.dump(fn)TypeError: no marshal_dump is defined for class Proc
irb> YAML::dump(fn)=> "--- !ruby/object:Proc {}\n\n"
irb> YAML::dump(fn.to_ruby)=> "--- proc { |x| (x ** 10) }\n"
irb> Marshal.dump(fn.to_ruby)=> "\004\b\"\eproc { |x| (x ** 10) }"
Enumerable#drmap
module Enumerable
def drmap(&block) pool = Drmap::BeanstalkPool.new
jid = rand(100) each_with_index do |element, idx| pool.put_job(jid, block.to_ruby, element) end
results = [] while results.size < length results << pool.next_result(jid) end results end
end
def process loop do begin job_payload = next_job fn = job_payload[:proc] data = job_payload[:data] result = eval(fn).call(data) save(job_payload[:jid], result) success! rescue => e fail!(e) end endend
drmap
• Trivial
• Distributed
• Powerful
Demo time!
Gisting
• MapReduce in Ruby
• http://github.com/mchung/gisting
map / reduce
• Functional programming
• Iterating over collections
• (5..10).map {|x| x+1} #=> [6, 7, 8, 9, 10, 11]
• (5..10).reduce(:+) #=> 45
Google MapReduce
• Modeled after map() and reduce()
• Programming model
• Programs are trivially parallelizable
Demo time!
Gisting::Spec
input.file_pattern = "file.txt" input.map do |map_input| words = map_input.strip.split("\t") Emit(words[1], "1") end
output.reduce do |reduce_input| count = 0 reduce_input.each do |value| count += value.to_i end Emit(count)end
inputs = args spec = Gisting::Spec.new inputs.each do |file_input| input = spec.add_input input.file_pattern = file_input input.map do |map_input| # 2722 mailbox 2006-05-23 00:08:39 words = map_input.strip.split("\t") Emit(words[1], "1") end end output = spec.output output.filebase = "/Users/mchung/Public/datasets/output" output.num_tasks = 2 output.reduce do |reduce_input| count = 0 reduce_input.each do |value| count += value.to_i end Emit(count) end
result = MapReduce(spec) pp result
Gisting
• MapReduce programming model in Ruby
• Uses ruby2ruby to serialize procs
• Uses EventMachine for Map/ReduceServer
• -: 94978
• mailbox: 20872
• google: 6758
• ebay: 2832
• yahoo: 2674
• yahoo.com: 2198
• myspace.com: 1916
• google.com: 1882
self.give(:thanks)OpenRain
openrain.com
My Git Repogithub.com/mchung
Marc Chungblog.marcchung.com
Ruby::AZrubyaz.org
Resources
Code Generation: The Safety Scissors Of Metaprogrammingmwrc2008.confreaks.com/03bowkett.html
Forbidden Fruit: A Taste of Ruby's Parse Treegoruco2008.confreaks.com/03_wanstrath.html
Ruby2Rubyseattlerb.rubyforge.org/ruby2ruby/
ParseTreeparsetree.rubyforge.org
