Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 1
One VM to Rule Them All Christian Wimmer, Chris Seaton VM Research Group, Oracle Labs
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 3
The following is intended to provide some insight into a line of research in Oracle Labs. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described in connection with any Oracle product or service remains at the sole discretion of Oracle. Any views expressed in this presentation are my own and do not necessarily reflect the views of Oracle.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 4
One Language to Rule Them All? Let’s ask Google…
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 5
One Language to Rule Them All? Let’s ask Stack Overflow…
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 6
“Write Your Own Language”
Prototype a new language
Parser and language work to build syntax tree (AST), AST Interpreter
Write a “real” VM
In C/C++, still using AST interpreter, spend a lot of time implementing runtime system, GC, …
People start using it
Define a bytecode format and write bytecode interpreter
People complain about performance
Write a JIT compiler Improve the garbage collector
Performance is still bad
Prototype a new language in Java
Parser and language work to build syntax tree (AST) Execute using AST interpreter
People start using it
And it is already fast
Current situation How it should be
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 7
@Specialization( rewriteOn=ArithmeticException.class) int add(int l, int r) { return Math.addExact(l, r); } @Specialization double add(double l, double r) { return l + r; } @Specialization(guards = "isString") String doString(Object l, Object r) { return l.toString() + r.toString(); }
Truffle Requirements
L1: decl rax jz L2 movl rcx, rdx[16+4*rax] cvtsi2sd xmm1, rcx addsd xmm0, xmm1 jmp L1 L2:
+ Generality
Ruby, JavaScript, Python, R, J, Java, Groovy,
Clojure, Scala ...
function f(a, n) { var x = 0; while (n-‐-‐ > 0) { x = x + a[n]; } return x; }
Simplicity
+ Performance
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 8
System Structure
Low-footprint VM, also suitable for embedding
Common API between language implementation and optimization system
Language agnostic dynamic compiler
Your language here!
Integrate with Java applications Substrate VM
Truffle Graal
Ruby
JavaScript Python
R
Graal VM
…
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 9
AST Interpreter Uninitialized Nodes
Compiled Code
Automatic Partial Evaluation
Truffle Approach
Node Rewriting for Type Feedback
AST Interpreter Rewritten Nodes
Syntax tree nodes are “stable”
Aggressive constant folding, method inlining, escape analysis
Deoptimize compiled code on tree rewrite
Eliminate dynamic type checks
Eliminate boxing of primitive values
AST Inlining
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 10
More Details on Truffle Accepted for Onward! 2013, October 26-31 2013, Indianapolis, IN
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 11
Ruby Prototype: High Performance
Fastest Ruby implementation …
… for the few benchmarks that
we looked at
Substrate VM
Truffle Graal
Ruby
Graal VM
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 12
Ruby Prototype: Low Footprint
Startup time (“Hello World”)
comparable to MRI
Substrate VM
Truffle Graal
Ruby
Graal VM
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 13
Ruby Prototype: Completeness
§ RubySpec § A library of executable assertions that
covers the language, core library and standard library
§ This is the defacto Ruby spec § Gives us a quantifiable result for how much
of Ruby we implement correctly
Over 45% of RubySpec
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 14
Completeness
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 15
Completeness: Informally
Charles Nutter: ‘So You Want to Optimize Ruby’ http://blog.headius.com/2012/10/so-you-want-to-optimize-ruby.html
Language Feature Implemented Notes
Fixnum to Bignum promotion ✓
Support for floating point ✓
Closures ✓
Bindings and eval ✓
callcc and Continuation ✓ Very limited support, the same as JRuby
Fibers ✓ Slightly limited support, the same as JRuby
Frame local variables ✓
C extensions
Ruby 1.9 encoding ✓
Garbage collection ✓
Concurrency and parallelism ✓ We currently use a GIL
Tracing and debugging ✓
ObjectSpace ✓
Method invalidation ✓
Constant invalidation ✓
Ruby on Rails
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 16
Completeness: More formally via RubySpec Running language tests
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
Topaz RubyTruffle JRuby
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 17
Low Footprint
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 18
Ahead-of-Time Compilation
Static Analysis
Substrate VM Execution Model
Substrate VM
Java Application
JDK
Reachable methods, fields, and classes
Initial Heap
Machine Code
OS
All Java classes from application, JDK, and Substrate VM
Application running without compilation
or class loading
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 19
Startup Performance Running "Hello World"
Execution time:
Memory footprint:
time -‐f "%e"
time -‐f "%M"
13
353
688
140
200
400
600
800
MRI JRuby Truffle onJVM
Truffle onSVM
[msec] Execution Time
5
35
53
9
0
10
20
30
40
50
60
MRI JRuby Truffle onJVM
Truffle onSVM
[MByte] Memory Footprint
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 20
High Performance
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 21
Why is Ruby Slow?
-b + (Math.sqrt(b**2 - 4*a*c)) / 2*a
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 22
Why is Ruby Slow?
-b + (Math.sqrt(b**2 - 4*a*c)) / 2*a execute b check that b is a Float check that the negate method in Float has not changed calculate negation check the result of that is a Float execute b check that b is a Float check that the power method in Float has not changed calculate power check the result of that is a Float execute a check that a is a Float check that the multiply method in Float has not changed calculate multiplication check the result of that is a Float execute c check that c is a Float check that the multiply method in Float has not changed calculate multiplication check the result of that is a Float check that Math has not changed check that the sqrt method in Math has not changed calculate sqrt check the result of that is a Float execute a check that a is a Float check that the multiply method in Float has not changed calculate multiplication check the result of that is a Float check that the division method in Float has not changed calculate division
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 23
Improving Performance Using Truffle
-b + (Math.sqrt(b**2 - 4*a*c)) / 2*a execute b check that b is a Float check that the negate method in Float has not changed calculate negation check the result of that is a Float execute b check that b is a Float check that the power method in Float has not changed calculate power check the result of that is a Float execute a check that a is a Float check that the multiply method in Float has not changed calculate multiplication check the result of that is a Float execute c check that c is a Float check that the multiply method in Float has not changed calculate multiplication check the result of that is a Float check that Math has not changed check that the sqrt method in Math has not changed calculate sqrt check the result of that is a Float execute a check that a is a Float check that the multiply method in Float has not changed calculate multiplication check the result of that is a Float check that the division method in Float has not changed calculate division
+
- /
b sqrt *
... 2 a
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 24
Improving Performance Using Truffle
-b + (Math.sqrt(b**2 - 4*a*c)) / 2*a execute b check that b is a Float check that the negate method in Float has not changed calculate negation check the result of that is a Float execute b check that b is a Float check that the power method in Float has not changed calculate power check the result of that is a Float execute a check that a is a Float check that the multiply method in Float has not changed calculate multiplication check the result of that is a Float execute c check that c is a Float check that the multiply method in Float has not changed calculate multiplication check the result of that is a Float check that Math has not changed check that the sqrt method in Math has not changed calculate sqrt check the result of that is a Float execute a check that a is a Float check that the multiply method in Float has not changed calculate multiplication check the result of that is a Float check that the division method in Float has not changed calculate division
+
- /
b sqrt *
... 2 a
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 25
Improving Performance Using Truffle
-b + (Math.sqrt(b**2 - 4*a*c)) / 2*a execute b check that b is a Float check that the negate method in Float has not changed calculate negation check the result of that is a Float execute b check that b is a Float check that the power method in Float has not changed calculate power check the result of that is a Float execute a check that a is a Float check that the multiply method in Float has not changed calculate multiplication check the result of that is a Float execute c check that c is a Float check that the multiply method in Float has not changed calculate multiplication check the result of that is a Float check that Math has not changed check that the sqrt method in Math has not changed calculate sqrt check the result of that is a Float execute a check that a is a Float check that the multiply method in Float has not changed calculate multiplication check the result of that is a Float check that the division method in Float has not changed calculate division
+
- /
b sqrt *
... 2 a
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 26
Improving Performance Using Truffle
-b + (Math.sqrt(b**2 - 4*a*c)) / 2*a execute b check that b is a Float check that the negate method in Float has not changed calculate negation check the result of that is a Float execute b check that b is a Float check that the power method in Float has not changed calculate power check the result of that is a Float execute a check that a is a Float check that the multiply method in Float has not changed calculate multiplication check the result of that is a Float execute c check that c is a Float check that the multiply method in Float has not changed calculate multiplication check the result of that is a Float check that Math has not changed check that the sqrt method in Math has not changed calculate sqrt check the result of that is a Float execute a check that a is a Float check that the multiply method in Float has not changed calculate multiplication check the result of that is a Float check that the division method in Float has not changed calculate division
+
- /
b sqrt *
... 2 a
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 27
Improving Performance Using Truffle
-b + (Math.sqrt(b**2 - 4*a*c)) / 2*a execute b check that b is a Float check that the negate method in Float has not changed calculate negation check the result of that is a Float execute b check that b is a Float check that the power method in Float has not changed calculate power check the result of that is a Float execute a check that a is a Float check that the multiply method in Float has not changed calculate multiplication check the result of that is a Float execute c check that c is a Float check that the multiply method in Float has not changed calculate multiplication check the result of that is a Float check that Math has not changed check that the sqrt method in Math has not changed calculate sqrt check the result of that is a Float execute a check that a is a Float check that the multiply method in Float has not changed calculate multiplication check the result of that is a Float check that the division method in Float has not changed calculate division
+
- /
b sqrt *
... 2 a
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 28
Improving Performance Using Truffle
-b + (Math.sqrt(b**2 - 4*a*c)) / 2*a execute b check that b is a Float check that the negate method in Float has not changed calculate negation check the result of that is a Float execute b check that b is a Float check that the power method in Float has not changed calculate power check the result of that is a Float execute a check that a is a Float check that the multiply method in Float has not changed calculate multiplication check the result of that is a Float execute c check that c is a Float check that the multiply method in Float has not changed calculate multiplication check the result of that is a Float check that Math has not changed check that the sqrt method in Math has not changed calculate sqrt check the result of that is a Float execute a check that a is a Float check that the multiply method in Float has not changed calculate multiplication check the result of that is a Float check that the division method in Float has not changed calculate division
+
- /
b sqrt * ... 2 a
+
- /
b * ... 2 a
sqrt
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 29
Improving Performance Using Truffle
-b + (Math.sqrt(b**2 - 4*a*c)) / 2*a
execute b check that the negate method in Float has not changed calculate negation execute b check that the power method in Float has not changed calculate power execute a check that the multiply method in Float has not changed calculate multiplication execute c check that the multiply method in Float has not changed calculate multiplication check that Math has not changed check that the sqrt method in Math has not changed calculate sqrt execute a check that the multiply method in Float has not changed calculate multiplication check that the division method in Float has not changed calculate division
execute b check that b is a Float check that the negate method in Float has not changed calculate negation check the result of that is a Float execute b check that b is a Float check that the power method in Float has not changed calculate power check the result of that is a Float execute a check that a is a Float check that the multiply method in Float has not changed calculate multiplication check the result of that is a Float execute c check that c is a Float check that the multiply method in Float has not changed calculate multiplication check the result of that is a Float check that Math has not changed check that the sqrt method in Math has not changed calculate sqrt check the result of that is a Float execute a check that a is a Float check that the multiply method in Float has not changed calculate multiplication check the result of that is a Float check that the division method in Float has not changed calculate division
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 30
Improving Performance Using Graal
-b + (Math.sqrt(b**2 - 4*a*c)) / 2*a
execute b check that the negate method in Float has not changed calculate negation execute b check that the power method in Float has not changed calculate power execute a check that the multiply method in Float has not changed calculate multiplication execute c check that the multiply method in Float has not changed calculate multiplication check that Math has not changed check that the sqrt method in Math has not changed calculate sqrt execute a check that the multiply method in Float has not changed calculate multiplication check that the division method in Float has not changed calculate division
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 31
Improving Performance Using Graal
-b + (Math.sqrt(b**2 - 4*a*c)) / 2*a
execute b check that the negate method in Float has not changed calculate negation execute b check that the power method in Float has not changed calculate power execute a check that the multiply method in Float has not changed calculate multiplication execute c check that the multiply method in Float has not changed calculate multiplication check that Math has not changed check that the sqrt method in Math has not changed calculate sqrt execute a check that the multiply method in Float has not changed calculate multiplication check that the division method in Float has not changed calculate division
class Float
⚑ modified?
module Math
⚑ modified?
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 32
Improving Performance Using Graal
-b + (Math.sqrt(b**2 - 4*a*c)) / 2*a
execute b check that the negate method in Float has not changed calculate negation execute b check that the power method in Float has not changed calculate power execute a check that the multiply method in Float has not changed calculate multiplication execute c check that the multiply method in Float has not changed calculate multiplication check that Math has not changed check that the sqrt method in Math has not changed calculate sqrt execute a check that the multiply method in Float has not changed calculate multiplication check that the division method in Float has not changed calculate division
class Float
⚑ modified?
module Math
⚑ modified?
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 33
Improving Performance Using Graal
-b + (Math.sqrt(b**2 - 4*a*c)) / 2*a
execute b check that the negate method in Float has not changed calculate negation execute b check that the power method in Float has not changed calculate power execute a check that the multiply method in Float has not changed calculate multiplication execute c check that the multiply method in Float has not changed calculate multiplication check that Math has not changed check that the sqrt method in Math has not changed calculate sqrt execute a check that the multiply method in Float has not changed calculate multiplication check that the division method in Float has not changed calculate division
class Float
⚑ modified?
module Math
⚑ modified?
java object InstalledCode
.invalidate()
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 34
Improving Performance Using Graal
-b + (Math.sqrt(b**2 - 4*a*c)) / 2*a
execute b check that the negate method in Float has not changed calculate negation execute b check that the power method in Float has not changed calculate power execute a check that the multiply method in Float has not changed calculate multiplication execute c check that the multiply method in Float has not changed calculate multiplication check that Math has not changed check that the sqrt method in Math has not changed calculate sqrt execute a check that the multiply method in Float has not changed calculate multiplication check that the division method in Float has not changed calculate division
class Float
⚑ modified?
module Math
⚑ modified?
java object InstalledCode
.invalidate()
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 35
Improving Performance Using Graal
unmodified = new Assumption();
unmodified.check();
unmodified.invalidate();
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 36
Improving Performance Using Graal
-b + (Math.sqrt(b**2 - 4*a*c)) / 2*a
execute b calculate negation execute b calculate power execute a calculate multiplication execute c calculate multiplication calculate sqrt execute a calculate multiplication calculate division
execute b check that the negate method in Float has not changed calculate negation execute b check that the power method in Float has not changed calculate power execute a check that the multiply method in Float has not changed calculate multiplication execute c check that the multiply method in Float has not changed calculate multiplication check that Math has not changed check that the sqrt method in Math has not changed calculate sqrt execute a check that the multiply method in Float has not changed calculate multiplication check that the division method in Float has not changed calculate division
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 37
Improving Performance
-b + (Math.sqrt(b**2 - 4*a*c)) / 2*a
execute b calculate negation execute b calculate power execute a calculate multiplication execute c calculate multiplication calculate sqrt execute a calculate multiplication calculate division
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 38
Peak Performance
0"
20"
40"
60"
80"
100"
120"
140"
1.8.7*p374"
1.9.3*p448"
2.0.0*p247"
topaz*daily"
jruby*1.7.4*server*invokedynamic"
truffle*server*nograal"
truffle*server"
Speedu
p&Speedup&Rela*ve&to&1.8.72p374&
Fannkuch"
N*Body"
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 39
Peak Performance
0"
1"
2"
3"
4"
5"
6"
7"
jruby/1.7.4/server/interpreter"
jruby/1.7.4/server"
jruby/1.7.4/server/invokedynamic"
topaz/dev"
truffle/server/nograal"
truffle/server"
Speedu
p&Speedup&Rela*ve&to&jruby21.7.42server2invokedynamic&
Fannkuch"
N/Body"
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 40
Simplicity
§ One intern working for five months on the Ruby implementation § New to Truffle, Graal and Ruby
§ Written using Eclipse § Debugged as a normal Java program using the server compiler § Run using Graal for testing and performance numbers
§ No mention in the implementation of bytecode, classloaders, assembly, system calls, OSR
§ One very minor use of Unsafe, one very minor use of reflection
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 41
Oracle Labs Laurent Daynès Michael Haupt Peter Kessler Christos Kotselidis David Leibs Roland Schatz Chris Seaton Doug Simon Michael Van De Vanter Christian Wimmer Christian Wirth Mario Wolczko Thomas Würthinger Laura Hill (Manager)
JKU Linz Gilles Duboscq Matthias Grimmer Christian Häubl Josef Haider Christian Humer Christian Huber Manuel Rigger Lukas Stadler Bernhard Urban Andreas Wöß Prof. Hanspeter Mössenböck
Acknowledgments
Oracle Labs Interns Danilo Ansaloni Daniele Bonetta Shams Imam Stephen Kell Helena Kotthaus Gregor Richards Rifat Shariyar Codrut Stancu Wei Zhang
Purdue University Tomas Kalibera"Floreal Morandat"Petr Maj Prof. Jan Vitek University of California, Irvine University of Dortmund
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 42
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 43
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 44
Truffle Approach (Details)
U
U U
U
U I
I I
G
G I
I I
G
G
AST Rewriting for Profiling Feedback
AST InterpreterRewritten Nodes
AST InterpreterUninitialized Nodes
Compilation usingPartial Evaluation
Compiled Code
Node Transitions
S
UI
D
G
Unin it ia lized Integer
Generic
DoubleString
I
I I
G
G I
I I
G
G
Deoptimizationto AST Interpreter
D
I D
G
G D
I D
G
G
AST Rewriting to Update Profiling Feedback
Recompilation usingPartial Evaluation
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 45
System Structure (Details)
Guest Language Application
OS
AOT Optimization: using Graal for static analysis and AOT compilation
Language Parser AST InterpreterGuest Language Implementation
Truffle API Framework for Node Rewriting
Truffle Optimizer Partial Evaluation using Graal
VM Runtime Services Garbage Collector Graal CompilerStack Walking Deoptimization
Hosted on any Java VM
Hosted on Graal VM
(slow, for guest language development and debugging only)
(fast, for integration of guest language code with existing Java applications)