A New Transpant JAVA Thread Migration System Using Just-in-

Time Recompilation

Wenzhang Zhu, Cho-Li Wang, Weijian Fang, and Francis C.M. Lau

The University of Hong Kong

10 Nov 2004 16th IASTED PDCS 2004, Cambridge, USA 2

Java Thread

Lightweight computation unit Supports concurrency Supported by Java at language level Shared memory paradigm Multithreading used in large Java programs –

e.g. Tomcat, JBoss

10 Nov 2004 16th IASTED PDCS 2004, Cambridge, USA 3

True Parallelism for Threads

Multiple threads of a program spreading over multiple compute nodes in a cluster

A “global object space” to support data sharing

JESSICA2 [8]

Heap Heap

threads

CPU1 CPU2 CPU3

objects

10 Nov 2004 16th IASTED PDCS 2004, Cambridge, USA 4

Dynamic Thread Migration

To achieve load balancing and best performance for applications

overloaded underloaded balanced workload

10 Nov 2004 16th IASTED PDCS 2004, Cambridge, USA 5

Example of Thread Migration

We need to carry the code and its execution state to remote node

What is in the execution state? At least

the program counter the intermediate

values of variables

i=1;j=2;

k=i+j;

Node 1

Node 2

Program:

1: i=1;

2: j=2;

3: k=i+j;

10 Nov 2004 16th IASTED PDCS 2004, Cambridge, USA 6

Previous Approaches

User level Bytecode level [6][7] JVM level [4][5], with JIT [9] Normal thread execution suffers

time: checking flags and flushing registers to memory space: code size expanded

A multithreaded Java application will have >95% normal execution, <5% migration

What we present will help optimize the 95%

10 Nov 2004 16th IASTED PDCS 2004, Cambridge, USA 7

JIT Compilation

Can we migrate native code? No, machines are not the same

Javabytecodeinexecution

JITC

Native code

Migrate?

10 Nov 2004 16th IASTED PDCS 2004, Cambridge, USA 8

New Approach

Derive portable bytecode-oriented thread context (BTC) from native (raw) thread context (RTC)

Restore BTC to RTC

Re-run JIT compilation in both phases to transform the thread context

Target nodeTarget node

Source nodeSource node

10 Nov 2004 16th IASTED PDCS 2004, Cambridge, USA 9

The Detailed Steps

Very quickly …

10 Nov 2004 16th IASTED PDCS 2004, Cambridge, USA 10

Just-in-Time Recompilation (source node)

Stack walk

main()

compute()

JVM-kernel function()

10 Nov 2004 16th IASTED PDCS 2004, Cambridge, USA 11

Just-in-Time Recompilation (source node)

Stack walk

Frame segmentation

main()

compute()

JVM-kernel function()

Javaframesto migrate

10 Nov 2004 16th IASTED PDCS 2004, Cambridge, USA 12

Just-in-Time Recompilation (source node)

Stack walk

Frame segmentation

main()

compute()Bytecode PC positioning

incl %ebx

cmpl $0x1e, %ebx

jl 0x82512432

iinc 1, 1

iload_1

bipush 30

if_icmplt 5Mapping

10 Nov 2004 16th IASTED PDCS 2004, Cambridge, USA 13

Just-in-Time Recompilation (source node)

Stack walk

Frame segmentationcompute()

Bytecode PC positioningincl %ebx

cmpl $0x1e, %ebx

jl 0x82512432

iinc 1, 1

iload_1

bipush 30

if_icmplt 5

Breakpoint selection

getstatic #2return

breakpoint

10 Nov 2004 16th IASTED PDCS 2004, Cambridge, USA 14

Just-in-Time Recompilation (source node)

Stack walk

Frame segmentationcompute()

Bytecode PC positioning

Breakpoint selection

Type derivation

local 0: java.lang.threadlocal 1: intstack 0: floatstack 1: int

main()

local 0: java.lang.threadstack 0: doublestack 2: double

10 Nov 2004 16th IASTED PDCS 2004, Cambridge, USA 15

Just-in-Time Recompilation (source node)

Stack walk

Frame segmentationcompute()

Bytecode PC positioning

Breakpoint selection

Type derivation

Translation

re-generate native code with breakpoints

getstatic #2Iload_1

inc %ebxcmpl $1e, %ebxjl 0x82512432

0x82512432:… capture frames…--- save local var’s and their typereturn

main()...other native code...jmp start_migration

10 Nov 2004 16th IASTED PDCS 2004, Cambridge, USA 16

Just-in-Time Recompilation (source node)

Stack walk

Frame segmentation compute()

Bytecode PC positioning

Breakpoint selection

Type derivation

Translation

replace saved return address with the newlygenerated code

inc %ebxcmpl $1e, %ebxjl 0x82512432

0x82512432:… capture frames…--- save local var’s and their typereturn main()

...other native code...jmp start_migration

Native code patching

saved return address

activation record ofcompute()

10 Nov 2004 16th IASTED PDCS 2004, Cambridge, USA 17

Just-in-Time Recompilation (target node)

Thread creation

Frame

PC

10 Nov 2004 16th IASTED PDCS 2004, Cambridge, USA 18

Just-in-Time Recompilation (target node)

Thread creation

Dynamic register patchingcompute()

1. use JIT compiler to derive register2. mapping at restoration point generate

code stub

Register recovering stub:movl %0x1234, %eaxmovl %0x5678, %ebx...

main()

Register recovering stub:movl %0x1111, %eax...

10 Nov 2004 16th IASTED PDCS 2004, Cambridge, USA 19

Just-in-Time Recompilation (target node)

Stack growth

%ebp

bootstrap frame

trampoline frame

Ret addr

frame 0

reg1 <- value1reg2 <- value2

jmp restore_point0

Ret addr

%ebp

%ebp

frame 1

reg1 <- value1jmp restore_point1

Compiled methods:

compute(){...retore_point1:}

main(){...retore_point10:}

trampoline

bootstrap(){ trampoline();closing handler();}

Stack rebuilding

Thread creation

Dynamic register patching

Linking code stub and native frames

10 Nov 2004 16th IASTED PDCS 2004, Cambridge, USA 20

Experimentation

Cluster of 2GHz P4 PCs, running Linux kernel 2.4.22

Normal runtime overhead = 0 Migration latency hiding

Pre-loading classes needed at target node when the frames have been segmented

Overlapping class loading with stack transform at source node

10 Nov 2004 16th IASTED PDCS 2004, Cambridge, USA 21

Migration Overhead

0

5000

10000

15000

20000

25000

30000

35000

40000

CPI SOR ASP NBody

Tim

e(u

s)

DNCI

JITR

JITR + Preload

0

5000

10000

15000

20000

25000

30000

35000

40000

CPI SOR ASP NBody

Tim

e(u

s)

DNCI

JITR

JITR + Preload

DNIC: Dynamic Native Code Instrumentation approachJITR: Just-in-Time recompilation approachJITR+Preload: Just-in-Time recompilation with class preloading enabled

10 Nov 2004 16th IASTED PDCS 2004, Cambridge, USA 22

Multithreaded Java Application Server Simulation

10 Nov 2004 16th IASTED PDCS 2004, Cambridge, USA 23

Parallel Mesh Refinement Simulation

10 Nov 2004 16th IASTED PDCS 2004, Cambridge, USA 24

Conclusion

This paper presents an efficient transparent Java thread migration system using the technique of JIT recompilation without code instrumentation

The idea of using JIT recompilation demonstrates a new use of the JIT compiler to gather runtime information

10 Nov 2004 16th IASTED PDCS 2004, Cambridge, USA 25

JESSICA2 Website

www.csis.hku.hk/~clwang/projects/JESSICA2.html