Dynamic Interpretation for Dynamic Scripting Languages
Dynamic Interpretation for DynamicScripting Languages
Kevin Williams, Jason McCandless, David Gregg
Trinity College Dublin
April 28, 2010
1 /42
Dynamic Interpretation for Dynamic Scripting Languages
Outline
Introduction
� Novel intermediate representation for scriptinglanguages that explicitly encodes types ofvariables.
2 /42
Dynamic Interpretation for Dynamic Scripting Languages
Outline
1 Motivation
2 Background
3 Motivating Example
4 Dynamic Intermediate Representation
5 Experimental Evaluation
6 Scope for Optimization
7 General Applicability
8 Conclusions3 /42
Dynamic Interpretation for Dynamic Scripting Languages
Motivation
Motivation
� Scripting languages growing really fast!� Run slower than statically typed languages (type
checks)� Lua: a popular portable scripting language amongst
many developers
4 /42
Dynamic Interpretation for Dynamic Scripting Languages
Motivation
Motivation
� Standard interpretation loop often used to executearray of opcodes
� Run-time type checks a significant portion ofprogram execution
� Research shows advantages of type specializationin JIT, little research on specialization of interpreters
5 /42
Dynamic Interpretation for Dynamic Scripting Languages
Motivation
Our Idea
� A Dynamic Intermediate Representation thatexplicitly encodes types of variables at eachexecution point
� Dynamic IR is a flow graph, each edge directsprogram flow based on control and type changes inthe program
� Therefore specialized path in the graph for everysequence of control and type changes
6 /42
Dynamic Interpretation for Dynamic Scripting Languages
Motivation
This Paper
� Present the initial development of our prototypeimplementation in the Lua VM
� Design of dynamic representation as well astechniques used for its efficient interpretation
7 /42
Dynamic Interpretation for Dynamic Scripting Languages
Background
Background
� High level languages have type systems. Theyenable detection and prevention of invalidoperations...
� Dynamically typed languages – each time anoperation is applied to a variable during programexecution, the type of the variable must be checked
� Goal of much research to reduce or eliminatedynamic type checks in compilers or virtualmachines
8 /42
Dynamic Interpretation for Dynamic Scripting Languages
Background
Background
� Lua VM has nine variable types: {nil, boolean,lightuserdata, number, string, table, function,thread, userdata}
� Register based machine� Thirty-eight instructions� Interpreter accesses array of instructions through
program counter
9 /42
Dynamic Interpretation for Dynamic Scripting Languages
Motivating Example
Static IR
Static IR
1 local N = 1002 local flags = {}3 for i = 2, N do4 if not flags[i] then5 for k = i+i, N, i do6 flags[k] = true7 end8 end9 end
10 /42
Dynamic Interpretation for Dynamic Scripting Languages
Motivating Example
Static IR
Static IR
1 loadk r0 k0 ; reg0 = constant02 newtable r1 0 0 ; reg1 = new table (0 ,0)3 loadk r2 k1 ; reg2 = constant14 move r3 r0 ; reg3 = reg05 loadk r4 k2 ; reg4 = constant26 forprep r2 L15 ; perform forloop prep , goto[15]7 gettable r6 r1 r5 ; reg6 = reg1[reg5]8 test r6 L15 ; i f reg6 , goto[9] else goto[15]9 add r6 r5 r5 ; reg6 = reg5 + reg5
10 move r7 r0 ; reg7 = reg011 move r8 r5 ; reg8 = reg512 forprep r6 r1 ; perform forloop prep , goto[14]13 settable r1 r9 k3 ; reg9[reg1] = constant314 forloop r6 L13 ; i f loop , goto[13] else goto[15]15 forloop r2 L7 ; i f loop , goto[7] else [end]
11 /42
Dynamic Interpretation for Dynamic Scripting Languages
Motivating Example
Static IR
‘Imaginary’ Static IR graph(a) 1 loadk
(b) 2 newtable
(c) 3 loadk
(d) 4 move
(e) 5 loadk
(f) 6 forprep
(o) 15 forloop
(g) 7 gettable
(h) 8 test
(i) 9 add
(j) 10 move
(k) 11 move
(l) 12 forprep
(n) 14 forloop
(m) 13 settable
An ‘imaginary’ control flow graph for the staticinterpretation of the Sieve of Eratosthenes algorithm.
12 /42
Dynamic Interpretation for Dynamic Scripting Languages
Motivating Example
Dynamic IR
Dynamic IR graph(a) 1 loadk_number
(b) 2 newtable
(c) 3 loadk_number
(d) 4 move_number
(e) 5 loadk_number
(f) 6 forprep_number_number_number
(g) 7 gettable_table_number
nil
(h) 8 test_nil_1
(i) 9 add_number_number
(j) 10 move_number
(k) 11 move_number
(l) 15 forloop_lt_number_number_number
loop branch
(m) 12 forprep_number_number_number (n) 12 forprep_number_number_number
(o) 13 settable_table_number
(p) 14 forloop_lt_number_number_number
loop branch
(q) 14 forloop_lt_number_number_number_number
loop branch
(r) 7 gettable_table_number
nil
boolean
(s) 7 gettable_table_number
nil
boolean
(t) 8 test_nil_1
cond branch
(u) 8 test_boolean_1
cond branch
(v) 9 add_number_number
(w) 10 move_number_number
(x) 11 move_number_number
(y) 15 forloop_lt_number_number_number_number
loop branch
(z) 15 forloop_lt_number_number_number_number
loop branch
Outer loop later iterations
Outer loop first iteration
Initialization
13 /42
Dynamic Interpretation for Dynamic Scripting Languages
Dynamic Intermediate Representation
Dynamic IR
� Dynamic IR a flow graph� Nodes represent specialized instructions� Edges represent either control-flow or type-flow
� Every type change results in a new path -> allvariable types are known at every point inexecution
� Construction of the dynamic flow graph guided bythe Static IR
14 /42
Dynamic Interpretation for Dynamic Scripting Languages
Dynamic Intermediate Representation
Standard Node
Standard Node
int opcode;char* type;Node* next;
� Always result in same control and type flow
15 /42
Dynamic Interpretation for Dynamic Scripting Languages
Dynamic Intermediate Representation
Conditional Node
Conditional Node
int opcode;char* type;Node* fallthrou;Node* target;
� Instructions that result in two paths of control flowbut no type changes
16 /42
Dynamic Interpretation for Dynamic Scripting Languages
Dynamic Intermediate Representation
Type-Directed Node
Type-Directed Node
int opcode;char* type;Node* target[9];
� Dispatch the next node based on the type of theoperation’s result variable
17 /42
Dynamic Interpretation for Dynamic Scripting Languages
Dynamic Intermediate Representation
Call Node
Call Node
int opcode;char* type;Table<Instr*,Node*> call_table;Node* mru_call_node;Instr* mru_call_key;Table<Node*,Node*>return_table;Node* mru_return_node;Node* mru_return_key;
� Represent a call site and known set of parametertypes
18 /42
Dynamic Interpretation for Dynamic Scripting Languages
Dynamic Intermediate Representation
Return Node
Return Node
� Represent a return instruction and a set of returntypes
19 /42
Dynamic Interpretation for Dynamic Scripting Languages
Dynamic Intermediate Representation
Instruction Specialization
Instruction Specialization
1 Register Loads2 Arithmetic Operations3 Table Access4 Conditional Branches
20 /42
Dynamic Interpretation for Dynamic Scripting Languages
Experimental Evaluation
Benchmarks
� No real standard for scripting language benchmarks
21 /42
Dynamic Interpretation for Dynamic Scripting Languages
Experimental Evaluation
Micros
� We use a set of kernel benchmarks taken fromCLBG and GWCLS
22 /42
Dynamic Interpretation for Dynamic Scripting Languages
Experimental Evaluation
Experimental Setup
Experimental Setup
� Intel Xeon dual core 2.13GHz
23 /42
Dynamic Interpretation for Dynamic Scripting Languages
Experimental Evaluation
Performance
PerformanceStaticDynamic
0
2
4
6
8
10
12
14bi
nary
−tr
ees
fann
kuch
fast
a
k−nu
cleo
tide
man
delb
rot
n−bo
dy
nsie
ve
nsie
ve−
bits
part
ial−
sum
s
recu
rsiv
e
spec
tral
−no
rm
reve
rse−
com
p
thre
ad−
ring
acke
rman
n
arra
y−A
cces
s
hash
−A
cces
s
fibon
acci
heap
sort
mat
rix−
mul
nest
ed−
loop
s
rand
omnu
mge
n
strin
g−co
ncat
mea
n
Exe
cutio
n tim
e in
sec
onds
24 /42
Dynamic Interpretation for Dynamic Scripting Languages
Experimental Evaluation
Performance
PerformanceStaticDynamic
0x
0.5x
1x
1.5x
2x
2.5xbi
nary
−tr
ees
fann
kuch
fast
ak−
nucl
eotid
em
ande
lbro
tn−
body
nsie
vens
ieve
−bi
tspa
rtia
l−su
ms
recu
rsiv
esp
ectr
al−
norm
reve
rse−
com
pth
read
−rin
gA
cker
man
nA
rray
−A
cces
sH
ash−
Acc
ess
Fib
onac
ciH
eaps
ort
Mat
rix−
Mul
Nes
ted−
Loop
sR
ando
m−
Num
−G
enS
trin
g−C
onca
tm
ean
Per
form
ance
Spe
edup
25 /42
Dynamic Interpretation for Dynamic Scripting Languages
Experimental Evaluation
Machine Instructions Issued
x86 Machine Instructions IssuedStaticDynamic
0%
20%
40%
60%
80%
100%
120%bi
nary
−tr
ees
fann
kuch
fast
ak−
nucl
eotid
em
ande
lbro
tn−
body
nsie
vens
ieve
−bi
tspa
rtia
l−su
ms
recu
rsiv
esp
ectr
al−
norm
reve
rse−
com
pth
read
−rin
gac
kerm
ann
arra
y−A
cces
sha
sh−
Acc
ess
fibon
acci
heap
sort
mat
rix−
mul
nest
ed−
loop
sra
ndom
num
gen
strin
g−co
ncat
mea
n
Inst
ruct
ions
issu
ed
26 /42
Dynamic Interpretation for Dynamic Scripting Languages
Experimental Evaluation
Branch Instructions
Conditional Branch InstructionsStaticDynamic
0%
20%
40%
60%
80%
100%
120%
140%bi
nary
−tr
ees
fann
kuch
fast
ak−
nucl
eotid
em
ande
lbro
tn−
body
nsie
vens
ieve
−bi
tspa
rtia
l−su
ms
recu
rsiv
esp
ectr
al−
norm
reve
rse−
com
pth
read
−rin
gac
kerm
ann
arra
y−A
cces
sha
sh−
Acc
ess
fibon
acci
heap
sort
mat
rix−
mul
nest
ed−
loop
sra
ndom
num
gen
strin
g−co
ncat
mea
n
Con
ditio
nal b
ranc
h in
stru
ctio
ns
27 /42
Dynamic Interpretation for Dynamic Scripting Languages
Experimental Evaluation
Cache Access
L1 Data CacheStaticDynamic
0%
20%
40%
60%
80%
100%
120%bi
nary
−tr
ees
fann
kuch
fast
ak−
nucl
eotid
em
ande
lbro
tn−
body
nsie
vens
ieve
−bi
tspa
rtia
l−su
ms
recu
rsiv
esp
ectr
al−
norm
reve
rse−
com
pth
read
−rin
gac
kerm
ann
arra
y−A
cces
sha
sh−
Acc
ess
fibon
acci
heap
sort
mat
rix−
mul
nest
ed−
loop
sra
ndom
num
gen
strin
g−co
ncat
mea
n
Leve
l 1 d
ata
cach
e ac
cess
es
28 /42
Dynamic Interpretation for Dynamic Scripting Languages
Experimental Evaluation
Cache Access
L1 Instruction CacheStaticDynamic
0%
20%
40%
60%
80%
100%
120%bi
nary
−tr
ees
fann
kuch
fast
ak−
nucl
eotid
em
ande
lbro
tn−
body
nsie
vens
ieve
−bi
tspa
rtia
l−su
ms
recu
rsiv
esp
ectr
al−
norm
reve
rse−
com
pth
read
−rin
gac
kerm
ann
arra
y−A
cces
sha
sh−
Acc
ess
fibon
acci
heap
sort
mat
rix−
mul
nest
ed−
loop
sra
ndom
num
gen
strin
g−co
ncat
mea
n
Leve
l 1 in
stru
ctio
n ca
che
acce
sses
29 /42
Dynamic Interpretation for Dynamic Scripting Languages
Experimental Evaluation
Processor Stalls
Cycles Stalled On Any ResourceStaticDynamic
0%
20%
40%
60%
80%
100%
120%
140%
160%
180%bi
nary
−tr
ees
fann
kuch
fast
ak−
nucl
eotid
em
ande
lbro
tn−
body
nsie
vens
ieve
−bi
tspa
rtia
l−su
ms
recu
rsiv
esp
ectr
al−
norm
reve
rse−
com
pth
read
−rin
gac
kerm
ann
arra
y−A
cces
sha
sh−
Acc
ess
fibon
acci
heap
sort
mat
rix−
mul
nest
ed−
loop
sra
ndom
num
gen
strin
g−co
ncat
mea
n
Cyc
les
stal
led
on a
ny r
esou
rce
30 /42
Dynamic Interpretation for Dynamic Scripting Languages
Experimental Evaluation
Memory
Memory UseStaticDynamic
0%
50%
100%
150%
200%
250%
300%
350%bi
nary
−tr
ees
fann
kuch
fast
ak−
nucl
eotid
em
ande
lbro
tn−
body
nsie
vens
ieve
−bi
tspa
rtia
l−su
ms
recu
rsiv
esp
ectr
al−
norm
reve
rse−
com
pth
read
−rin
gac
kerm
ann
arra
y−A
cces
sha
sh−
Acc
ess
fibon
acci
heap
sort
mat
rix−
mul
nest
ed−
loop
sra
ndom
num
gen
strin
g−co
ncat
mea
n
Mem
ory
Use
31 /42
Dynamic Interpretation for Dynamic Scripting Languages
Scope for Optimization
Interpreter Dispatch
� Switch based currently used in Lua. Othertechniques equally applicable to our representation
� As specialization reduces the bottleneck of typechecks, instruction dispatch becomes moreimportant to overall performance
32 /42
Dynamic Interpretation for Dynamic Scripting Languages
Scope for Optimization
Register Caching
� Popular among stack based virtual machines
33 /42
Dynamic Interpretation for Dynamic Scripting Languages
Scope for Optimization
Super Nodes
� Concatenate commons pairs of instructions to forma single instruction
34 /42
Dynamic Interpretation for Dynamic Scripting Languages
Scope for Optimization
Loop Optimizations
� Leverage the type knowledge of variables insideloops to eliminate array bounds checking and movememory allocation outside loop body
� Specialize loop counters to integers when loopbound is constant
35 /42
Dynamic Interpretation for Dynamic Scripting Languages
Scope for Optimization
Dead Node Removal
� Specialization can lead to redundant nodes.� Can be removed and all entry nodes directed to
target exit node
36 /42
Dynamic Interpretation for Dynamic Scripting Languages
Scope for Optimization
Dynamic IR graph(a) 1 loadk_number
(b) 2 newtable
(c) 3 loadk_number
(d) 4 move_number
(e) 5 loadk_number
(f) 6 forprep_number_number_number
(g) 7 gettable_table_number
nil
(h) 8 test_nil_1
(i) 9 add_number_number
(j) 10 move_number
(k) 11 move_number
(l) 15 forloop_lt_number_number_number
loop branch
(m) 12 forprep_number_number_number (n) 12 forprep_number_number_number
(o) 13 settable_table_number
(p) 14 forloop_lt_number_number_number
loop branch
(q) 14 forloop_lt_number_number_number_number
loop branch
(r) 7 gettable_table_number
nil
boolean
(s) 7 gettable_table_number
nil
boolean
(t) 8 test_nil_1
cond branch
(u) 8 test_boolean_1
cond branch
(v) 9 add_number_number
(w) 10 move_number_number
(x) 11 move_number_number
(y) 15 forloop_lt_number_number_number_number
loop branch
(z) 15 forloop_lt_number_number_number_number
loop branch
Outer loop later iterations
Outer loop first iteration
Initialization
37 /42
Dynamic Interpretation for Dynamic Scripting Languages
Scope for Optimization
Library Nodes
� Stack incrementing and decrementing required forexecuting standard library operations
38 /42
Dynamic Interpretation for Dynamic Scripting Languages
Scope for Optimization
JIT Compilation
� Mixed-mode� Dynamic IR will improve current model of static
interpretation followed by speculated specialization
39 /42
Dynamic Interpretation for Dynamic Scripting Languages
General Applicability
General Applicability
� DIR implemented in Lua� Applicable to other scripting languages
40 /42
Dynamic Interpretation for Dynamic Scripting Languages
Conclusions
Conclusions
� Presented a novel approach to scripting languagespecialization
� First stage of a project to bring specialization to alllevels of interpretation
� Not addressed some potential scaling issues withrepresentation
41 /42