Aarhus University, 2005 Esmertec AG 1

ImplementingObject-OrientedVirtual Machines

Lars Bak & Kasper LundEsmertec AG{lbak,klund}@esmertec.com

Aarhus University, 2005 Esmertec AG 2

What Is A Virtual Machine?

A virtual machine is a software abstraction layer implemented on top of a "real" hardware platform and operating system

Aarhus University, 2005 Esmertec AG 3

Why Are VMs Interesting?

Platform independenceCode migrationWorks for small embedded systems

and large server systemsMost modern programming

languages are expecting a virtual machine

Enables aggressive optimization

Aarhus University, 2005 Esmertec AG 4

What Is Performance?

Fast executionFast startup timeFast warm-up timeSmall pausesScalable

• Heap size• CPUs

Aarhus University, 2005 Esmertec AG 5

Obstacles To Performance?

Platform independent byte codesObject-oriented

• Virtual calls are the default• High call frequency• High allocation rate

Garbage collectionSynchronization in languageDynamic class loading

Aarhus University, 2005 Esmertec AG 6

… But That Is Not Enough

Preserve the illusion of bytecode interpretation

Important for platform independent debugging

Provides serviceability for systems in production systems

Aarhus University, 2005 Esmertec AG 7

The Agenda

Fast virtual dispatchRuntime compilationAdaptive compilation

• Applying aggressive optimizationsOSVM + demonstration

Aarhus University, 2005 Esmertec AG 8

Our Background

Beta runtime systemSelf virtual machine and IDEStrongtalk virtual machine Java HotSpot virtual machineCLDC HI virtual machineOSVM embedded platform

20+ years of experience in designing and implementing object oriented virtual

machines

Aarhus University, 2005 Esmertec AG 9

Dynamic Dispatch

Core part of execution in most OO systems

Target method at a call site depends on receiver type

Runtime lookup is needed to find the method

Aarhus University, 2005 Esmertec AG 10

Lookup Function

f(receiver type, name) -> methodImplementation techniques for

making the lookup function efficient• Virtual dispatch tables• Inline caches• Hash table

Aarhus University, 2005 Esmertec AG 11

Virtual Dispatch Tables

Indirect method tableOften used in statically typed languages

like Java and C++Technique

• Object points to dispatch table• Method gets index in table of defining class• Call

vtbl = obj->vtable()target = vtbl[index]call target

Aarhus University, 2005 Esmertec AG 12

Problems With Vtable

Indirect calls are very expensive on modern CPUs

Can only handle single inheritance in strongly typed systems

Does not work with interface callsMakes incremental execution very

hard

Aarhus University, 2005 Esmertec AG 13

Inline Caching

Save the result from last lookupDeusch-Schiffman Smalltalk, 1984>85% of all calls are monomorphic in

most object oriented systemsUse self modifying code for

implementationWhere should the cache result reside?

Aarhus University, 2005 Esmertec AG 14

Inline Caching

Call site has several modes• Empty (no targets)• Monomorphic (one target)• Megamorphic

Inline caching used in monomorphic case

Aarhus University, 2005 Esmertec AG 15

Code For Inline Cache

Caller// receiver is in ecx

move eax, <address of receiver type>

call target

Calleecmp eax, ecx[class offset]

bne _inline_cache_miss

… code for callee method …

Aarhus University, 2005 Esmertec AG 16

Inline Caching

Fast on modern CPUsUsed in most OO systems where self

modifying code is permissible Side benefit: type information is

collected

Aarhus University, 2005 Esmertec AG 17

Hash Tables

Often needed with dynamic typingUsed in Smalltalk and SelfCache for f(receiver type, name) ->

methodUsed as secondary lookup for inline

cache

Aarhus University, 2005 Esmertec AG 18

Hash Table Problems

Many collisions result in slow downBut, what about a 2-way associative

cacheHard to update in multi threaded

executionUse thread local cache

Aarhus University, 2005 Esmertec AG 19

Combined Solutions

Inline caching + hash table• Self + Smalltalk

Inline caching + virtual table• Java

Aarhus University, 2005 Esmertec AG 20

Dynamic Compilation

The process of compiling at runtimeGoals

• Fast startup• Great performance• Good interactive performance• Minimize memory footprint

Aarhus University, 2005 Esmertec AG 21

Execution Model

Interpreter and compiled code combination

Allows the systems to decide:• When to compile?• What to compile?

Aarhus University, 2005 Esmertec AG 22

Adaptive Compilation

InterpreterInterpreter bytecodedmethods

bytecodedmethods

compiledmethodscompiledmethods

Profiler / Recompiler

Profiler / Recompiler

CompilerCompiler

Aarhus University, 2005 Esmertec AG 23

Should We Compile That Method?

Compile the method if the total execution time is reduced

How do we predict if it is beneficial?Use the past to predict the future

Aarhus University, 2005 Esmertec AG 24

Learning From The Past

Maintain a sliding window of the past to answer:• What to compile?• When to compile?

Two approaches:• Invocation counters• Sample based profiling

Aarhus University, 2005 Esmertec AG 25

Why We Need Inlining

Too many dynamic dispatchInlining will:

• Create bigger basic blocks• Give compiler more to work on• Might avoid allocation

Aarhus University, 2005 Esmertec AG 26

Aggressive Inlining

Use class hierarchy analysis if language has static types

Inline based on the current state of the class hierarchy

Back out if class loading violates assumptions• Use deoptimization as described later

Aarhus University, 2005 Esmertec AG 27

Deoptimization

The act of converting a compiled activation into a set of interpreter activations

Allow the system to• Back out of inlining decisions• Preserve the bytecode illusion

Aarhus University, 2005 Esmertec AG 28

Needed Information

Debugging information stored with compiled code• Enough information to reconstruct the

interpreter state at each interrupt pointsAccess to compiled frames

Aarhus University, 2005 Esmertec AG 29

What Can Deoptimization Be Used For?

Backing out of aggressive inlining Source level inspection Support for source code level

debugging Support for incremental execution

Aarhus University, 2005 Esmertec AG 30

The Process Of Deoptimization

1) Identify activations to deoptimize2) Transform the compiled activations into

an off-stack interpreter based representation

3) Insert trap and remove compiled method4) When returning to activation unpack the

off-stack representation to the stack5) Continue execution in interpreter

Aarhus University, 2005 Esmertec AG 31

Virtual Machine Performance Summary

Object oriented optimizations• Agressive inlining• Subtype check (instanceof & cast)

Efficient memory management system• Fast allocation• Efficient garbage collection

Ultra fast synchronization... and a good compiler

Aarhus University, 2005 Esmertec AG 32

The OSVM Product

OSVM IDE OSVM Embedded Platform

A middleware platform for creating secure, object-oriented, real-time software for embedded devices• Always-on platform• Requires minimal memory• Increases development productivity• Enables analysis and correction of software problems in the

field• Enables minimal updates of deployed software

Aarhus University, 2005 Esmertec AG 33

OSVM Components OSVM IDE

• Plugin to Eclipse 3.0• Integrated with CVS• Configuration of services• Complete tool chain

OSVM Libraries• Core classes• Debugging and reflection

OSVM Virtual Machine• Executes platform-independent code• Runs on various platforms

OSVM RTOS• Small hard real time operating system• Allows tight hardware integration

Aarhus University, 2005 Esmertec AG 34

OSVM Demonstration

Intelligent tank demonstration

OSVM IDE

Aarhus University, 2005 Esmertec AG 35

Summary

Discussed various virtual machine optimizations

However, the real reason for being here:• Hire part time student programmers for

our cool project