Is Java ReadyIs Java ReadyFor For
Real-Time Embedded Systems?Real-Time Embedded Systems?
Is Java ReadyIs Java ReadyFor For
Real-Time Embedded Systems?Real-Time Embedded Systems?
Angelo [email protected]
DOC GroupWashington University
St. Louis, USA
David Sharp, Jim Urnes, [email protected],
james.m.urnes-jr @boeing.comBoeing St. Louis USA
2
Table of ContentsTable of Contents
Real Time Java Real Time Java
General PerformanceGeneral Performance JVM98 SPEC BenchmarkJVM98 SPEC Benchmark FilteringFiltering Hanoi TowerHanoi Tower Garbage CollectionGarbage Collection
RT PerformanceRT Performance Thread Preemption BehaviorThread Preemption Behavior Priority Inversion AvoidancePriority Inversion Avoidance Jitter in Periodic Event HandlingJitter in Periodic Event Handling Timing in Periodic Event HandlingTiming in Periodic Event Handling
Concluding RemarksConcluding Remarks
3
What’s Going On?What’s Going On?
During Y2K two specification for extending Java with Real-Time capabilities have appeared, specifically RTSJ from the Real Time Java Expert Group (headed by
Greg Bollella) J-Consortium Specification for Real Time Java
Sun supports the RTSJ, and the market seems to head in the same direction Ajile had already implemented a subset of the spec on HW
already last August IBM provided us with the first “implementation” of a RTJVM
TimeSys has been selected by Sun to implement the reference JVM that will implement the RTSJ specification
Real Tim
e Java
Real Tim
e Java
4
What is RTJavaWhat is RTJava
Real Time Java is an extension of the Java platform to allow the development of Real Time Application
The RTJava specification enhance 7 areas Thread Scheduling and Dispatching Memory Management Synchronization and Resource Sharing Asynchronous Event Handling Asynchronous Transfer of Control Asynchronous Thread Termination Physical Memory AccessR
eal Tim
e Java
Real Tim
e Java
5
JVM98 SPEC Benchmark 1/3JVM98 SPEC Benchmark 1/3
_200_check
_201_compress
_202_jess
_209_db
_213_javac
_222_mpegaudio
_227_mtrt
_228_jack
Genera
l Perf
orm
ance
Genera
l Perf
orm
ance
The JVM98 SPEC Benchmark is composed by the following applications
6
JVM98 SPEC BenchmarkJVM98 SPEC Benchmark
0 20 40 60 80 100 120Grade
_227_mtrt
_202_jess
_201_compress
_209_db
_222_mpegaudio
_228_jack
_213_javac
Ap
plic
ati
on
MS-Max
IBM-Max
Sun-Max
Genera
l Perf
orm
ance
Genera
l Perf
orm
ance
J9 JVM98 Results JView JVM98 Results Javac JVM98 Results
7
Application DescriptionApplication Description
Finite Impulse Response (FIR) Filter with 21 tap
The filter implement a noise attenuation on audio samples
Filters coefficient and sample data are double
Genera
l Perf
orm
ance
Genera
l Perf
orm
ance
FilterAudio Sample Filtered Audio Sample
x(k) y(k)
8
Filter PerformanceFilter PerformanceG
enera
l Perf
orm
ance
Genera
l Perf
orm
ance
0
20
40
60
80
100
120
140
160
180
200
30000 60000 90000 120000Samples
ms
ec
C++
Jview
Sun-JDK
IBM-J9
9
Application DescriptionApplication Description
Java implementation of the Algorithm that solves the famous Hanoi Tower game
Genera
l Perf
orm
ance
Genera
l Perf
orm
ance
10
Hanoi TowerHanoi TowerG
enera
l Perf
orm
ance
Genera
l Perf
orm
ance
Hanoi
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
20 25 26 27Disks
mse
c
Sun-Jit
J9-Jit
J9-AOC
JView
11
Application DescriptionApplication Description
Build a full binary tree of height H Each node of the binary tree allocates a slot of memory of
B bytes
Analyze the following case Once the tree is build, remove the reference from the
root, and force the garbage collection Create and release while building the tree to investigate if
the JVM have some sort of incremental GC
Genera
l Perf
orm
ance
Genera
l Perf
orm
ance
12
Garbage Collection Performance Garbage Collection Performance 1/51/5
Genera
l Perf
orm
ance
Genera
l Perf
orm
ance
0 200 400 600 800 1000 1200 1400 1600 1800 2000
msec
14
16
18
Tree Depth
Garbage Collection Test (J9)
Tot Time
Method Call
Imposed GC
Tot Time 78 438 1902
Method Call 62 407 1782
Imposed GC 16 31 125
14 16 18
13
Genera
l Perf
orm
ance
Genera
l Perf
orm
ance
Garbage Collection Performance Garbage Collection Performance 2/52/5
0 100 200 300 400 500 600 700 800 900 1000
ms
ec
14
16
18
20
Tree Depth
Garbage Collection Test J9-RAC
Total Time
Method Call
Imposed GC
Total Time 16 63 235 906
Method Call 16 63 235 906
Imposed GC 0 0 0 0
14 16 18 20
14
Garbage Collection Performance Garbage Collection Performance 3/53/5
Genera
l Perf
orm
ance
Genera
l Perf
orm
ance
C++ vs. Java Memory Alloc/Dealloc
0
200
400
600
800
1000
1200
1400
1600
1800
2000
14 16 18 20Depth
mse
c.
C++-RAC
C++
J9-RAC
Sun-RAC
15
0
20
40
60
80
100
120
140
160
14 16 18
Depth
msec
J9-GC Time
Sun-GC Time
Sun-Xincgc GC Time
J9-GC-rac
Sun-GC-rac
Garbage Collection Performance Garbage Collection Performance 4/54/5
Genera
l Perf
orm
ance
Genera
l Perf
orm
ance
16
Genera
l Perf
orm
ance
Genera
l Perf
orm
ance
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
20000
14 16 18 20Depth
ms
ec
C++
J9
Sun-JVM
C++RAC
J9-RAC
Sun-RAC
Garbage Collection Performance Garbage Collection Performance 5/55/5
17
Test CasesTest Cases
Real-time determinism test cases RealtimeThread preemption handling (PreemptTest) Priority inversion avoidance (PriInvertTest) Dispatching of AsyncEvents (EventDispatchTest2) Jitter in periodic event handling (EventJitterTest) Timing of periodic event handling (EventTimingTest)
RT P
erf
orm
an
ceR
T P
erf
orm
an
ce
18
PreemptTest ScenarioPreemptTest Scenario
Purpose: Measure whether priority preemption occurs correctly for
multiple RealtimeThreads of different priorities
Method: Stagger the start of fixed-duration, processor-holding
RealtimeThreads of increasing or decreasing priority. Using timestamp logging, see when threads enter and exit in relation to each other
RT P
erf
orm
an
ceR
T P
erf
orm
an
ce
19
PreemptTest ResultsPreemptTest Results
Result: Thread 1 worked correctly, it kept the processor until it was done. But threads 2, 3, and 4, all started immediately upon thread 1 finishing. They ran simultaneously and when complete, threads 5, 6, and 7 all started and ran simultaneously. Thread preemption did not work correctly.
Starting at priority 7, start the treads every 2 seconds in decreasing priority order. Threads try to keep theprocessor for 7 seconds.
This works!
Problem!Enter and leavein blocks of 3
RT P
erf
orm
an
ceR
T P
erf
orm
an
ce
20
PreemptTest AnalysisPreemptTest Analysis Problem
The IBM j9 groups multiple Java thread levels to a single underlying RTOS thread level
RTSJ Requires at least 28 unique priority levels
Suggestion Allow underlying RTOS threads to be FIFO as well as Round
Robin
RT P
erf
orm
an
ceR
T P
erf
orm
an
ce
Java Thread Priority QNX Neutrino Thread Priority28, 29, 30 15r25, 26, 27 14r22, 23, 24 13r19, 20, 21 12r16, 17, 18 11r13, 14, 15 10r10, 11, 12 9r
7, 8, 9 8r4, 5, 6 7r1, 2, 3 6r
21
PriInvertTest ScenarioPriInvertTest Scenario
Purpose: Measure whether priority inversion is properly avoided.
Method: A low priority thread obtains a lock on a synchronized
method, a medium thread preempts the low, and a high priority thread preempts the medium and attempts to obtain a lock on the same synch’d method. Log entry and exit times for the threads and synchronized method.
RT P
erf
orm
an
ceR
T P
erf
orm
an
ce
22
PriInvertTest ResultsPriInvertTest Results
Result: The low priority thread did NOT get elevated to high priority to finish the shared method. Therefore, the medium priority thread got to finish before the high priority thread. Priority inversion occurred.
RT P
erf
orm
an
ceR
T P
erf
orm
an
ce
23
PriInvertTest AnalysisPriInvertTest Analysis
Problem Priority Inheritance does not currently work RTSJ specifies Priority Inheritance as the default priority
inversion avoidance method for synchronized blocks
RT P
erf
orm
an
ceR
T P
erf
orm
an
ce
24
EventDispatchTest ScenarioEventDispatchTest Scenario
Purpose: Measure the execution order for multiple
AsyncEventHandlers of different priority when an AsyncEvent fires for which they are all registered.
Method: Set up three AsyncEventHandlers of different priority, all
registered to the same AsyncEvent. Issue the event and have the handlers log timestamps on entry and exit.
RT P
erf
orm
an
ceR
T P
erf
orm
an
ce
25
EventDispatchTest ResultsEventDispatchTest Results
Result: This is correct. The highest priority handler runs first, then next highest, and then lowest.
In this tests the Java priorities are such that the resulting QNX priorities are different.
Result: This is incorrect. The handlers don’t start in priority order and preempt each other.
In this tests the Java priorities are such that the resulting QNX priorites are the same.
RT P
erf
orm
an
ceR
T P
erf
orm
an
ce
26
EventDispatchTest AnalysisEventDispatchTest Analysis
Problem The OTI j9 groups multiple Java thread levels to a single
underlying RTOS thread level RTSJ requires 28 unique priority levels
Java Thread Priority QNX Neutrino Thread Priority28, 29, 30 15r25, 26, 27 14r22, 23, 24 13r19, 20, 21 12r16, 17, 18 11r13, 14, 15 10r10, 11, 12 9r
7, 8, 9 8r4, 5, 6 7r1, 2, 3 6r
RT P
erf
orm
an
ceR
T P
erf
orm
an
ce
27
EventJitterTest ScenarioEventJitterTest Scenario
Purpose: Measure the amount of time variation between runs of a
PeriodicTimer driven AsyncEventHandler with various other activity occurring.
Method: Setup a PeriodicTimer object to fire an AsyncEventHandler at
a fixed rate, while other lower priority, processor-keeping RealtimeThreads run. Log a timestamp each time the handler runs. After the run, import the data to Excel for analysis.
RT P
erf
orm
an
ceR
T P
erf
orm
an
ce
28
EventJitterTest ResultsEventJitterTest Results
In this 1 second test, our AsyncEventHandler runs at java priority 30. Another RealtimeThread runs at java priority 6. PeriodicTimer event fires every 50 msecs (20 Hz.).
Result: Quite good - jitter between runs of handler within RTOS timer resolution.
RT P
erf
orm
an
ceR
T P
erf
orm
an
ce
29
EventJitterTest Results (cont)EventJitterTest Results (cont)
In this 1 second test, our AsyncEventHandler runs at java priority 30. 200 other RealtimeThreads runs at java priority 6. PeriodicTimer event fires every 50 msecs (20 Hz.).
Result: Not bad - some jitter (+/- 1.1 msec) between runs, but lower priority threads do seem to affect jitter.
RT P
erf
orm
an
ceR
T P
erf
orm
an
ce
30
EventJitterTest Results (cont)EventJitterTest Results (cont)
In this 1 second test, our AsyncEventHandler runs at java priority 30. Another RealtimeThread runs at java priority 10. PeriodicTimer event fires every 50 msecs (20 Hz.).
Result: Bad - the periodic events never get to the handler, even though the handler has higher priority than the other RealtimeThread.
RT P
erf
orm
an
ceR
T P
erf
orm
an
ce
31
Handlerthread
Java pri=30QNX pri=15r
EventJitterTest AnalysisEventJitterTest Analysis
Apparent AsyncEventHandler mechanization At least for PeriodicTimer driven AsyncEvents
“Spawner”thread
QNX pri=8r
Handlerthread
Java pri=30QNX pri=15r
Handlerthread
Java pri=30QNX pri=15r
PeriodicTimer’s fire() method called
Handlerthread
Java pri=30QNX pri=15r
handler thread of desired priority spawned for each firing
Another threadJava pri = 10QNX pri = 9r
Problem!If this thread is using the processor,the “Spawner” thread will not get the opportunity to spawn the handlerthread. Priority inversion occurs.
RT P
erf
orm
an
ceR
T P
erf
orm
an
ce
32
EventJitterTest Analysis EventJitterTest Analysis (cont)(cont)
Results for AsyncEventHandler with priority 30 (highest)
All data points are 100 second runs at 20 Hz (50 msecs)
3 sets of data analyzed with other RealtimeThreads run at priority 1, 6, and 9 respectively
Other threads are processor-keeping andrun time-sliced at the same priority.Dip at exactly 50 threads -
repeatable
Dip at exactly 50 threads - repeatable
Num of Other Threads vs. Error Range (i.e. Jitter)
00.010.020.030.040.050.06
0 50 100 150 200 250 300 350
Number of other threads
Err
or
ran
ge
(s
ec
s)
pri=1
pri=6
pri=9
For priority 9, jitter is off the scale due to contention with “spawner”. (previous slide)
For priority 9, jitter is off the scale due to contention with “spawner”. (previous slide)
RT P
erf
orm
an
ceR
T P
erf
orm
an
ce
33
EventJitterTest Analysis EventJitterTest Analysis (cont)(cont)
Absolute period measurements
For other numbers of RT threads, some jitter outside +/- 1 msec occurs.
For other numbers of RT threads, some jitter outside +/- 1 msec occurs.
Period for Each Sample30 other theads, pri=6
0.02
0.04
0.06
0.08
0 500 1000 1500 2000
Samples
Per
iod
(se
cs)
Period for Each Sample50 other theads, pri=6
0.020.030.040.050.060.070.08
0 500 1000 1500 2000
Samples
Per
iod
(se
cs)
Period for Each Sample50 other theads, pri=6
0.04850.049
0.04950.05
0.05050.051
0.0515
0 500 1000 1500 2000
Samples
Per
iod
(se
cs)
0 or50 other RT threads results in jitter values around +/- 1 msec, probably due to RTOS timer resolution
0 or50 other RT threads results in jitter values around +/- 1 msec, probably due to RTOS timer resolution
34
EventJitterTest Analysis EventJitterTest Analysis (cont)(cont)
–Standard deviation and Mean
Std Dev trend increases with number of other RT threads
Std Dev trend increases with number of other RT threads
Degradation above 250 threads, perhaps hitting processor throughput limit
Degradation above 250 threads, perhaps hitting processor throughput limit
RT P
erf
orm
an
ceR
T P
erf
orm
an
ce
Number of Other Threads vs. Std. Dev.
0
0.0005
0.001
0.0015
0.002
0.0025
0 50 100 150 200 250 300 350
Number of other threads
std
de
vpri=1
pri=6
Number of Other Threads vs. Mean Period
00.010.020.030.040.050.060.070.080.09
0.1
0 50 100 150 200 250 300 350
Number of other threads
Me
an
pe
rio
d (
se
cs
)
pri=1
pri=6
pri=9
35
EventTimingTest ScenarioEventTimingTest Scenario
Purpose: Measure the time it takes for an AsyncEvent to get from
firing to its handler. Vary priority and other processor activity.
Method: Set up an AsyncEvent and associated AsyncEvent handler at
specified priority. Over a number of iterations, log the timestamp before firing and upon entry into the handler. Run variable number of other processor-keeping RealtimeThreads. Import results to Excel for analysis.
RT P
erf
orm
an
ceR
T P
erf
orm
an
ce
36
EventTimingTest ResultsEventTimingTest Results
Result plotted.
High priority (30) AsynchEventHandler with 50 other processor-keeping RT threads of priority 1. 100 samples taken.
Latency for each sample50 other threads, pri=1
0.001960.00198
0.0020.002020.002040.00206
0 20 40 60 80 100
Sample Number
Lat
ency
(se
cs)RT P
erf
orm
an
ceR
T P
erf
orm
an
ce
37
EventTimingTest Analysis EventTimingTest Analysis (cont)(cont)
Still under investigation If handler priority lowered to 15, the average latency improves to
about 0.6 msec– Does not exhibit the “spawner” thread priority inversion of the
PeriodicTimer driven AsyncEventHandler
High priority (30) AsynchEventHandler with various number of other processor-keeping RT threads. 100 samples taken for each point.
Num of Other Threads vs. Avg. Event LatencyHandler pri = 30, other pri=1, 6, 12
0.001
0.002
0.003
0.004
0 100 200 300 400 500 600 700 800 900
Number of other threads
Av
g.
La
ten
cy
(s
ec
s)
pri=1
pri=6
pri=12
About 2 msec averagelatency for this conditionbelow 300 other threads
RT P
erf
orm
an
ceR
T P
erf
orm
an
ce
38
Summary Of Experimental Summary Of Experimental ResultsResults
Java thread priority preemption must be maintained for each of the 28 required thread levels.
AsyncEventHandler threads driven by PeriodicTimers must not be prevented from running by lower priority threads.
Priority inheritance should work by default.
Programmer needs better control of Java thread to underlying RTOS mapping.
RT P
erf
orm
an
ceR
T P
erf
orm
an
ce
39
Concluding RemarksConcluding Remarks
JIT provide acceptable performance to Java, and improvement in this area will close the performance gap with C++ (which is already within a factor 2)
J9 RT-Java extension are still very minimal, and cannot be considered yet for industrial strength development
The J9 RT-Java extensions suffer of some major problem, mainly Priority Mapping Priority Inversion Effect induced by the “Spawner”
This investigation has provided us with Test-Bed that will make easier to evaluate the “performance” of a RT-JVM once mature (complete) implementation will start to appear
