Reusing JITs are from Mars, Dynamic Scripting Languages ......- PHP: Facebook, LAMP - Python:...

© 2012 IBM Corporation

Reusing JITs are from Mars, Dynamic Scripting Languages are from Venus

Peng Wu, IBM T.J. Watson Research Center

May 9, 2012

© 2012 IBM Corporation2 Reusing JITs are from Mars, and Dynamic Scripting Languages are from Venus

Trends in Workloads, Languages, and Architectures

newArchitecture

Appl

icat

ion/

prog

ram

min

gtra

ditio

nal

Accelerators(GPGPU, FPGA, SIMD)

Dynamic scripting Languages

(javascript, python, php)

Streaming model(Hadoop, CUDA, OpenCL, SPL, …)

multi-core,general-purpose

C/C++, Fortran, Java, …

SPEC, HPC, Database, Webserver

Big data workload(distributed)

mixed workloads(data center)

Programming by examples

traditional

new

System programmers HPC CS programmers Domain experts Non-programmers

Demographic evolution of programmers


Popularity of Dynamic Scripting LanguagesTrend in emerging programming paradigms

– Dynamic scripting languages are gaining popularity and emerging in production deployment

Commercial deployment- PHP: Facebook, LAMP- Python: YouTube,

InviteMedia, Google AppEngine

- Ruby on Rails: Twitter, ManyEyes

“Python helped us gain a huge lead in features and a majority of early market share over our competition using C and Java.”

- Scott BeckerCTO of Invite Media Built on Django, Zenoss, Zope

“Python helped us gain a huge lead in features and a majority of early market share over our competition using C and Java.”

- Scott BeckerCTO of Invite Media Built on Django, Zenoss, Zope

2.387%Perl10

2.879%Javascript9

3.665%Python8

4.962%Visual Basic7

5.288%PHP6

7.348%C#5

8.236%Objective-C4

8.896%C++3

17.026%Java2

17.555%C1

ShareNameRank

TIOBE Language Index

Education- Increasing adoption of

Python as entry-level programming language

Demographics- Programming becomes a

everyday skill for many non-CS majors

1.510%Ruby11


Language Interpreter Comparison (Shootout)

4

Benchmarks: shootout (http://shootout.alioth.debian.org/) measured on NehalemLanguages: Java (JIT, steady-version); Python, Ruby, Javascript, Lua (Interpreter)Standard DSL implementation (interpreted) can be 10~100 slower than Java (JIT)

fast

er

http://shootout.alioth.debian.org/

© 2012 IBM Corporation5 Reusing JITs are from Mars, and Dynamic Scripting Languages are from Venus5

Dynamic Scripting Language JIT Landscape

NitroCrankShaft

IonMonkey

Javascript

Unladen- swallow

PyPy

Python

Ruby

RubiniusDaVinciMachine

Client

PHP

Server

Client/Server

Client/Server

ChakraDaVinciMachine

JVM based– Jython– JRuby– Rhino

CLR based– IronPython– IronRuby– IronJscript– SPUR

Add-on JIT– Unladen-

swallow– Fiorano– Rubinius

Add-on trace JIT– PyPy– LuaJIT– TraceMonkey– SPUR

Significant difference in JIT effectiveness across languages– Javascript has the most effective JITs– Ruby JITs are similar to Python’s

Fiorano

HipHopP9


Scripting Languages Compilers: A Tale of Two Worlds

Customary VM and JIT design targeting one scripting language

– in-house VM developed from scratch and designed to facilitate the JIT

– in-house JIT that understands target language semantics

Heavy development investment, most noticeably in Javascript

– where performance transfers to competitiveness

Such VM+JIT bundle significantly reduces the performance gap between scripting languages and statically typed ones

– Sometimes more than 10x speedups over interpreters

The reusing JIT phenomenon– reuse the prevalent interpreter

implementation of a scripting language– attach an existing mature JIT– (optionally) extend the “reusing” JIT to

optimize target scripting languages

Considerations for reusing JITs– Reuse common services from mature

JIT infrastructure– Harvest the benefits of mature

optimizations– Compatibility with standard

implementation by reusing VM

Willing to sacrifice some performance, but still expect substantial speedups from compilation


Scripting Languages Compilers: A Tale of Two Worlds

Customary VM and JIT design targeting one scripting language

– in-house VM developed from scratch and designed to facilitate the JIT

– in-house JIT that understands target language semantics

Heavy development investment, most noticeably in Javascript

– where performance transfers to competitiveness

Such VM+JIT bundle significantly reduces the performance gap between scripting languages and statically typed ones

– Sometimes more than 10x speedups over interpreters

The reusing JIT phenomenon– reuse the prevalent interpreter

implementation of a scripting language– attach an existing mature JIT– (optionally) extend the “reusing” JIT to

optimize target scripting languages

Considerations for reusing JITs– Reuse common services from mature

JIT infrastructure– Harvest the benefits of mature

optimizations– Compatibility with standard

implementation by reusing VM

Willing to sacrifice some performance, but still expect substantial speedups from compilation


Outline

Let’s take an in-depth look at the reusing JIT phenomenon

We focus on the world of Python JIT1. PyPy: customary VM + trace JIT based on RPython2. Fiorano JIT: based on Testarossa JIT from IBM J9 VM (our own)3. Jython: translating Python codes into Java codes4. Unladen-swallow JIT: based on LLVM JIT (google)5. IronPython: translating Python codes into CLR (Microsoft)

The rest of the talk– The state-of-the-art of reusing JIT approach– Understanding Jython, Fiorano JIT, and PyPy– Recommendation of Reusing JIT designers– Conclusions


Python Language and Implementation

Python is an object-oriented, dynamically typed language– Monolithic object model (every data is an object, including integer or method frame)– support exception, garbage collection, function continuation– CPython is Python interpreter in C (de factor standard implementation of Python)

LOAD_GLOBAL (name resolution)– dictionary lookup

CALL_FUNCTION (method invocation)– frame object, argument list processing,

dispatch according to types of calls

BINARY_ADD (type generic operation)– dispatch according to types, object creation

def foo(list):return len(list)+1

0 LOAD_GLOBAL 0 (len)3 LOAD_FAST 0 (list)6 CALL_FUNCTION 19 LOAD_CONST 1 (1)12 BINARY_ADD13 RETURN_VALUE

foo.py

python bytecode


Overview on Jython

A clean implementation of Python on top of JVM

Generate JVM bytecodes from Python 2.5 codes– interface with Java programs– true concurrence (i.e., no global interpreter lock)– but cannot easily support standard C modules

Runtime rewritten in Java, JIT optimizes user programs and runtime– Python built-in objects are mapped to Java class hierarchy– Jython 2.5.x does not use InvokeDynamic in Java7 specification

Jython is an example of JVM languages that share similar characteristics– e.g., JRuby, Clojure, Scala, Rhino, Groovy, etc– similar to CLR/.NET based language such as IronPython, IronRuby


0

0.5

1

1.5

2

2.5

3

django

float

nbodynqueenspys

tone

richard

srie

tveld

slowpick

leslo

wspitf

ireslo

wunpickle

spambayesgeomean

Exec

utio

n Ti

me

Nor

mal

ized

to C

pyth

on

Execution Time of Jython 2.5.2 Normalized over CPythonspeedup


Jython: An Extreme case of Reusing JITs

Jython has minimal customization for the target language Python– It does a “vanilla” translation of a Python

program to a Java program – The (Java) JIT has no knowledge of Python

language nor its runtime

private static PyObject calc$1(PyFrame frame) {frame.setlocal(3, i$0);frame.setlocal(2, i$0);while(frame.getlocal(3)._lt(frame.getlocal(0)).__nonzero__()) {

frame.setlocal(2, frame.getlocal(2)._add(frame.getlocal(1)));frame.setlocal(3, frame.getlocal(3)._add(i$1));

}return frame.getlocal(2);

}

def calc1(self,res,size):x = 0while x < size:

res += 1x += 1

return res


Jython Runtime Profile

# Java bytecode

path length per Python loop iteration(a) localvar-

loop(b) getattr-

loop(c) call-loop

heap-read 47 80 131

heap-write 11 11 31

heap-alloc 2 2 5

branch 46 70 101

invoke (JNI) 70(2) 92(2) 115(4)return 70 92 115

arithmetic 18 56 67

local/const 268 427 583

Total 534 832 1152


res += self.ax += 1

return res

(b) getattr-loop

def foo(self):return 1


res += self.foo()x += 1

return res

(c) call-loop

In an ideal code generation

Critical path of 1 iteration include:

• 2 integer add• 1 integer compare• 1 conditional branch

On the loop exit• box the accumulated value into PyInteger• store boxed value to res


res += 1x += 1

return res

(a) localvar-loop

100x path length explosion


Why is the Java JIT Ineffective?

What does it take to optimize this example effectively?

Massive inlining to expose all computation within the loop to the JIT– for integer reduction loop, 70 ~ 110 call sites need to be inlined

Precise data-flow information in the face of many data-flow join– for integer reduction loop, between 40 ~ 100 branches

Ability to remove redundant allocation, heap-read, and heap-write– require precise alias/points-to information

Let’s assume that the optimizer can handle local accesses effectively


Python bytecode

CPython

The Fiorano JIT

Python bytecode -> Intermediate representation

VM

JIT

binary

Optimizations and code generation

profile information IR

code cache

New component

IBM production-qualityJust-In-Time (JIT) compiler for Java as a base

CPython as a languagevirtual machine (VM)

– de facto standard of Python

Same structure as Unladen Swallow

CPython with LLVM

Existing component

Overview of our Approach

Python-specific Optimizations

Profiler Selector

Python program

Presenter

Presentation Notes

Now, I show you high level overview of our runtime. We choose Python as a target language. We use IBM production-quality JIT compiler as a base. Then, we use CPython as a base, which is defacto Python VM. The advantage is that we can use existing libraries such as mod_wsgi, which uses internal structure of CPython. The structure is the same as Unladen Swallow. That uses CPython and LLVM compiler infrastructure.


What’s Added to the Fiorano JIT?No-opt level compilation support– Translated CPython bytecode into Testaross IR (IRGEN)– Added method hotness profiling and compilation trigger

Python-specific optimization support– Runtime profiling in CPython interpreter– A lot of IRGEN level specialization for Python

• Caching the results of LOAD_GLOBAL (watch invalidation)• Fast path versioning for LOAD_ATTR/STORE_ATTR/CALL• Guard-based specialization for arithmetic & compare • Specialization for built-ins such as instanceof, xrange, sin, cos• Guard-based & fast path versioning for

GET_ITER/FOR_ITER,UNPACK_SEQUENCE– Unboxing optimization for some integer and float

• Extending the escape analysis optimization in the Testarossa JIT

VEE 2011: Adding Dynamically-Typed Language Support to a Statically-Typed Language Compiler: Performance Evaluation, Analysis, and Tradeoffs


0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

djan

go

float

nbod

ynq

ueen

spy

ston

eric

hard

srie

tvel

dslo

wpick

leslo

wspitf

ireslo

wunpi

ckle

spam

baye

sge

omea

n

Execu

tion

Tim

e N

orm

ali

zed t

o C

pyth

on

fiorano-hot unladen-swallow pypy_18

Normalized Execution Time of Python JITs over CPython

speedup


PyPy (Customary Interpreter + JIT)

A Python implementation written in RPython– interface with CPython modules may take a big performance hit

RPython is a restricted version of Python, e.g., (after start-up time)– Well-typed according to type inference rules of RPython– Class definitions do not change– Tuple, list, dictionary are homogeneous (across elements)– Object model implementation exposes runtime constants– Various hint to trace selection engine to capture user program scope

Tracing JIT through both user program and runtime– A trace is a single-entry-multiple-exit code sequence (like long extended basic block)– Tracing automatically incorporates runtime feedback and guards into the trace

The optimizer fully exploit the simple topology of a trace to do very powerful data-flow based redundancy elimination


Number/Percentage of Ops Removed by PyPy Optimization

Such degree of allocation removal was not seen in any

general-purpose JIT

PEPM 2011: Allocation Removal by Partial Evaluation in a Tracing JIT


Common Pitfalls of Existing Reusing JIT Approaches

1. Over-reliance on the JIT alone to improve the performance and underestimating the importance of optimizing the runtime

For example, a) optimizing named lookup by analyzing hashtable implementations vs. b) implementing named lookup as hidden classes and using runtime feedback to them to indexed lookup

2. Over-reliance on traditional redundancy elimination optimizations to reduce path length of the fat runtime

Fat runtime imposes two major hurdles to effective dataflowLong call-chain requires excessive inlining capacityExcessive redundant heap operations

3. Not emphasizing enough on, specialization, a unique and abundant optimization opportunity in scripting language runtime


Effect of Different Optimization Levels: Fiorano JIT

0

0.2

0.4

0.6

0.8

1

1.2

djan

go

float

nbod

y

nque

ens

pyst

one

richa

rds

rietv

eld

slowpi

ckle

slowsp

itfire

slowun

pickl

esp

amba

yes

geom

ean

Ex

ecu

tio

n T

ime

No

rma

lize

d t

o C

pyth

o

noOpt cold warm hot

spee

dup


Tips for Reusing JIT Designers

1. Understand characteristics of your runtime– identify dominant operations w/ high overhead– understand the nature of excessive computation (e,g, heap, branch, call)

2. Remove excessive path lengths in the runtime as much as possible

3. Inside the reusing JIT, focus on the JIT’s ability to specialize

4. Boosting existing optimizations in reusing JIT


Typical Profile of a “Fat” Scripting Language Runtime

# Java Bytecode

Instruction path length per python bytecodeLOAD_LOCAL BINARY_ADD

(int+int)LOAD_ATTR

(self.x)COMPARE

(int > 0)CALL_FUNCT

(self.op())heap-read 3 5 29 17 53

heap-write 0 2 4 2 16

heap-alloc 0 1 1 0 2

branch 2 8 19 18 34

invoke (JNI) 0 17(0) 23(0) 26(2) 23(2)

return 0 17 23 26 23

arithmetic 0 5 38 8 11

local/const 6 60 152 96 154

Total 12 115 289 191 313

CPython runtime exhibits similar characteristics

Instruction path length profile of a typical Python bytecode in Jython runtime




2. Remove excessive path lengths in the runtime as much as possible– adopt best practice of VM implementation– re-evaluate the improved runtime (Step 1)

3. Inside the reusing JIT, focus on the JIT’s ability to specialize



Effect of Runtime Improvement: Jython 2.5.1 to 2.5.2

0

0.5

1

1.5

2

2.5

3

3.5

djan

go

float

nbod

y

nque

ens

pyst

one

richa

rds

rietv

eld

slow

pick

leslo

wsp

itfire

slow

unpi

ckle

spam

baye

s

geom

ean

Execu

tion T

ime N

orm

ali

zed t

o C

pyth

o

jython252_ojdk jython251_ojdk

Improvements from Jython 2.5.1 to 2.5.2– more than 50% reduction in path length of CALL_FUNCTION– significant speedups on large benchmarks with frequent calls

spee

dup





3. Inside the reusing JIT, focus on the JIT’s ability to specialize– Coverage: how many are specialized and specialized successfully– Degree of strength reduction: how fast is the fast version of specialization



Pybench: Speedup of JITs on Common Python Idioms

0%

100%

200%

300%

400%

500%

CALLS

LOOKUP

ARITHMETICNEW_IN

STANCECONTROL_

FLOW

STRING

UNICODEDICTIO

NARY

LIST

TUPLES

Spee

dup

over

CPy

thon

pypy_18fioranojython 2.5.2

37x 122x 29x 98x 35x23x 136x

spee

dup


Breakdown of Dynamic Python Bytecode Execution

0%

20%

40%

60%

80%

100%

djan

go

float

nbod

y

nque

ens

pyst

one

richa

rds

rietv

eld

slowpi

ckle

slowsp

itfire

slowun

pick

lesp

amba

yes

% B

yte

cod

es

Interpreted Interpreted-guard-failedCompield-unspecializable Compiled-unspecializedCompiled-specialization-succeeded Compiled-specialization-failed





3. Inside the reusing JIT, focus on the JIT’s ability to specialize– Coverage: how many are specialized and specialized successfully– Degree of strength reduction: how fast is the fast version of specialization



Effective Boosting Techniques in Fiorano JIT

Runtime feedback driven specialization– Types are typically quite stable to rely on simple runtime feedback– Achieve much higher coverage than analysis based approach

Focus on early path length reduction, especially during translation to IR

Guard-based specialization– Compared to versioning based specialization, guard eliminates data-flow join– Need to monitor guard failure and need de-optimization support


Concluding Remarks

Whenever an interpreted language emerges, reusing an existing JIT (LLVM, Java JIT) to compile the language becomes an economic option

Many reusing JITs for scripting languages do not live up to the expectation. Why?– The root cause of scripting language overhead is the excessive path length

explosion in the language runtime (10~100x compared to static language)– Traditional JITs are not capable of massive path length reduction in language

runtime permeated with heap/pointer manipulation and control-flow join

We offer lessons learned and recommendations to reusing JITs designers– Focus on path length reduction as the primary metrics to design your system– Do not solely rely on the JIT, improving the language runtime is as important– When reusing optimizations in the JIT, less is more– Instead, focus on specialization, runtime feedback, and guard-based approach


BACK UP


InvokeDynamics and JVM Languages

Performance of pilot implementation of Jython using invokedynamics

By Shashank Bharadwaj, University of Coloradohttp://wiki.jvmlangsummit.com/images/8/8d/Indy_and_Jython-Shashank_Bharadwaj.pdf


Evolution of Javascript JITsGoogle

– V8:• efficient object representation• hidden classes• GC

– Crankshaft: “traditional” optimizer (Dec 2010)• adaptive compilation• aggressive profiling• optimistic assumptions• SSA, invariant code motion, register

allocation, inlining• Overall, improved over V8 by 50%

– Beta release of Chrome with native client integrated

• C/C++ codes executed inside browser with security restrictions close to Javascripts

Mozilla– TraceMonkey

• trace-JIT, aggressive type specialization– JaegerMonkey (Sept, 2010, Firefox 4)

• method-JIT, inlining – IonMonkey (2011)

Apple – Nitro JIT (Safari 5)– “ 30% faster than Safari 4, 3% faster than

Chrome 5, 2X faster than Firefox 3.6”

Microsoft– Chakra JIT (IE9)

• async compilation• type optimization • fast interpreter• library optimization

JIT compilation for Javascript is a realityall major browser/mobile vendors have their own Javascript engine!Nodejs: server-side Javascript using asynchronous event driven model


Marco Cornero (ST Ericsson): http://www.hipeac.net/system/files/2011-04-06_compilation_for_mobile.pdf


Google: Crankshaft JIT

A new JIT compiler for V8 (Dec 2010)– Performance improvement by 50%, upto 2X (V8 benchmark)– Mostly benefits codes with hot loops, not for very short scripts (SunSpider)– Improved start-up time for web apps, e.g., gmail

Crankshaft JIT (adaptive compilation):– Base compiler: simple code generation– Runtime profiler: identify hot codes and collect type info– Optimizing compiler (hot codes only): SSA, loop invariant code motion, linear-

scan RA, inlining, using runtime type info– Deoptimization support: can bail out of optimized codes if runtime assumption

(e.g., type) is no longer valid


Performance of Javascript implementations

37

spee

dup


Performance of Ruby Implementations

38

spee

dup


IronPython: DynamicSitesOptimize method dispatch (including operators)Incrementally create a cache of method stubs and guards in response to VM queries public static object Handle(object[],

FastDynamicSite<object, object, object> site1,object obj1, object obj2) {

if (((obj1 != null) && (obj1.GetType() == typeof(int))) && ((obj2 != null) && (obj2.GetType() == typeof(int)))) { return Int32Ops.Add(Converter.ConvertToInt32(obj1),

Converter.ConvertToInt32(obj3)); }if (((obj1 != null) && (obj1.GetType() == typeof(string)))

&& ((obj2 != null) && (obj2.GetType() == typeof(string)))) { return = StringOps.Add(Converter.ConvertToString(obj1),

Converter.ConvertToString(obj2)); } return site1.UpdateBindingAndInvoke(obj1, obj3);

}

Propagate types when UpdateBindingAndInvoke recompiles stub

Presenter

Presentation Notes

Example could be animated/explained incrementally to show how the stub is built up step by step

Date post:	28-Mar-2020
Category:	Documents
Upload:	others
View:	8 times
Download:	0 times

Reusing JITs are from Mars, Dynamic Scripting Languages ......- PHP: Facebook, LAMP - Python:...

Documents