Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java DebuggersA Peek Under the Hood
David BuckPrincipal Member of Technical StaffJava SE Sustaining EngineeringSeptember, 2016
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
4
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
About Me
David Buck
• Java SE Sustaining Engineering
• Mostly JRockit fixes
• OpenJDK 8 Updates
Project Maintainer
• Hobbies: Programming
5
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
Introduction
Java Platform Debug Architecture
Java Debug Wire Protocol
JVM Tool Interface
Wrap-up
1
2
3
4
5
6
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Introduction
7
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
The Elephant in the Room
8
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
• 2 JavaOne Debugger “under the hood talks” this year
9
The Elephant in the Room
By Peter H. Wrege (Own work) [CC BY-SA 3.0 (http://creativecommons.org/licenses/by-sa/3.0) or GFDL (http://www.gnu.org/copyleft/fdl.html)], via Wikimedia Commons
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
• 2 JavaOne Debugger “under the hood talks” this year
• Martin’s talk is really good!
10
The Elephant in the Room
By Peter H. Wrege (Own work) [CC BY-SA 3.0 (http://creativecommons.org/licenses/by-sa/3.0) or GFDL (http://www.gnu.org/copyleft/fdl.html)], via Wikimedia Commons
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 11
Buck’s Talk Martin’s talk
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 12
JVM TIJDWP
JDI
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 13
JVM TIJDWP
JDI
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
• Both talks needed for full picture
• Video and slides from a past version of Martin’s talk are available online
• No particular order to the content
14
JDI
AKA Martin’s Talk
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 15
JDIJVM TIJDWP
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 16
JDI?JVM TI?JDWP?
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 17
JDI?JVM TI?JDWP?
Java Platform Debugger Architecture
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java Platform Debugger Architecture
18
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java Platform Debugger Architecture
• Facilitate writing a debugger in Java
• Impact Debuggee application as little as possible
• Allow native-level / in-process tooling
19
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Heisenbugs
Bug Heisenberg
Heisenbug
picture: Bundesarchiv, Bild 183-R57262 / Unknown / CC-BY-SA 3.0 [CC BY-SA 3.0 de (http://creativecommons.org/licenses/by-sa/3.0/de/deed.en)]
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Observer Effect
picture: Christian Schirm (Own work) [CC0]
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
• Debugging, at the same level of abstraction (Java), in the same process, will cause problems
– Stop-the-world
– debugger heap overhead
22
Strange Loop Problems
Picture By Maurizio Codogno, it:Utente:.mau. (Own work) [GFDL (http://www.gnu.org/copyleft/fdl.html), CC-BY-SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0/) or CC BY 2.5 (http://creativecommons.org/licenses/by/2.5)], via Wikimedia Commons
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java Platform Debugger Architecture
• Facilitate writing a debugger in Java
– Java API
• Impact Debuggee application as little as possible
–Out-of-process debugger
• Allow native-level / in-process tooling– Native in-process debugger
23
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 24
Java vs. Native
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 25
In-process vs. Out-of-process
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 26
Why not both?
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 27
By Libertinus (http://www.flickr.com/photos/libertinus/5807760586) [CC BY-SA 2.0 (http://creativecommons.org/licenses/by-sa/2.0)], via Wikimedia Commons
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java Platform Debugger Architecture
28
TargetJVM
Debugger JVM
Events
JDI
DebuggerTarget Application
(Debuggee)
JVM TI
AgentJDWP
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java Platform Debugger Architecture
• Since Java 1.3
• Modular design
• 3 components
– JVM Tool Interface
– Java Debug Wire Protocol
– Java Debug Interface
29
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java Platform Debugger Architecture
30
TargetJVM
Debugger JVM
JDI
DebuggerTarget Application
(Debuggee)
JVM TI
AgentJDWP
Commands
Events
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java Platform Debugger Architecture
31
TargetJVM
Debugger JVM
JDI
DebuggerTarget Application
(Debuggee)
JVM DI
AgentJDWP
Commands
Events
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java Platform Debugger Architecture
32
TargetJVM
Native Debugger Tool
Target Application(Debuggee)
JVM TI
AgentJDWP
Commands
Events
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java Platform Debugger Architecture
33
TargetJVM
Debugger JVM
JDI
DebuggerTarget Application
(Debuggee)
JVM TI
AgentJDWP
Commands
Events
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java Platform Debugger Architecture
34
TargetJVM
Target Application(Debuggee)
JVM TI
Agent
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java Platform Debugger Architecture
35
TargetJVM
Debugger JVM
JDI
DebuggerTarget Application
(Debuggee)
JVM TI
AgentJDWP
Commands
Events
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java Platform Debugger Architecture
36
TargetJVM
Debugger JVM
JDI
JShellTarget Application
(Debuggee)
JVM TI
AgentJDWP
Commands
Events
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java Debug Interface (JDI)
• Reference implementation lives in
– tools.jar (JDK 6/7/8)
– jdk.jdi module (JDK 9)
• Pure Java implementation
• “speaks” JDWP on behalf of Java client code
37
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java Debug Wire Protocol
38
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java Debug Wire Protocol (JDWP)
• Binary protocol between target JVM and debugger processes
• Does not specify transport
• Asynchronous
• Tailored to JDI
39
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java Debug Wire Protocol
40
TargetJVM
Debugger JVM
JDI
DebuggerTarget Application
(Debuggee)
JVM TI
AgentJDWP
Commands
Events
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java Debug Wire Protocol
41
TargetJVM
Debugger
Target Application(Debuggee)
JDWP
Commands
Events
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
JDWP Handshake
42
TargetJVM
Debugger
Target Application(Debuggee)
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
JDWP Handshake
43
TargetJVM
Debugger
Target Application(Debuggee)
JDWP-Handshake
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
JDWP Handshake
44
TargetJVM
Debugger
Target Application(Debuggee)
JDWP-Handshake
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
JDWP
Java Debug Wire Protocol
45
TargetJVM
Debugger
Target Application(Debuggee)
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java Debug Wire Protocol
• All Packets have 11 byte header
• Some packets have data
–Optional
– Variable length
46
Header
Data
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
• Command • Reply
47
JDWP Packets
Length
ID
FlagsCommand
setCommand
Data(optional, variable length)
Length
ID
Flags Error Code
Data(optional, variable length)
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
JDWP Header Fields
• length (4 bytes)
• id (4 bytes)
• flags (1 byte)
• command set (1 byte)
• command (1 byte)
• error code (2 bytes)
48
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
JDWP Length Field
• 32-bit value
• Size of entire packet in bytes (including header)
• Up to a 4GB packet size!
49
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
JDWP ID Field
• 32-bit value
• Only used to match command and reply packets
• Each “direction” (debugger <-> target) has it’s own ID-space
• Over 4 billion in flight packets per direction
50
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
JDWP Flags Field
• 8-bit value (but only one bit currently used)
• Only used to distinguish between Command / Reply packets
– 0x80 == Reply Packet
– 0x00 == Command Packet
• Additional flags may be added in the future
51
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
JDWP Command Fields
• Only in command packets
• Command Set Field
– 8 bit
– Each set corresponds to a JDI class• 0 - 63 debugger -> target VM
• 64 - 127 target VM -> debugger
• 128 - 256 Vendor-defined
• Command Field– 8 bit
– Each command corresponds to a JDI method
52
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
JDWP Error Code Field
• 16 bit value
• Only in reply packets
53
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
JDWP Data
• Variable size
• Optional
• Examples:
– Requested field ID (command)
– Requested field vale (reply)
54
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
JDWP Commands
55
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
What about transport?!
• In practice, this somewhat couples debugger / debuggee
• Socket transport
– All Oracle JDK supported platforms
– Supports remote debugging
– JVM TI agent currently only support IPv4
• Shared memory transport
–Only on Windows
• Service provider interface available for implementing your own transport / connectors
56
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Security Implications of JDWP
• Zero Authentication
• JDWP access == Remote Code Execution
• As of yesterday www.shodan.io reports two IP sending JDWP handshake
• Many more are passively waiting for incoming connections
57
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
JVM Tool Interface
58
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
• JVM Debug Interface (JVM DI)
• JDK 1.1 ~ JDK 1.5
JVM Profiler Interface (JVM PI)
• JDK 1.1 ~ JDK 1.5
• Never officially supported
• Stability issues (esp. with HS)
• Hard to use with JNI
• Very invasive
– Not all GC supported
– Performance impact
59
A Tail of Two APIs
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
• JVM Debug Interface (JVM DI)
• JDK 1.1 ~ JDK 1.5
JVM Profiler Interface (JVM PI)
• JDK 1.1 ~ JDK 1.5
• Never officially supported
• Stability issues (esp. with HS)
• Hard to use with JNI
• Very invasive
– Not all GC supported
– Performance impact
60
A Tail of Two APIs
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
JVM Tool Interface (JVM TI)
• Introduced in JDK 5
• Replaced
– JVM PI (Removed in JDK 6)
– JVM DI (Removed in JDK 6)
• Many use cases outside of JPDA
• Fully supported
• Pay-as-you-go performance impact
• Based on JNI
61
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
JVM TI Agents
• Native code that runs in JVM’s address space
• Multiple concurrent agents supported
• Can be
– Statically linked into JVM
– Loaded at startup
– Loaded during runtime
• Can take command line arguments
-agentlib:<agent_lib_name>=<name1>[=<value1>],<name2>[=<value2>]...
62
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Native Agent
• JVM TI
• Command line options:
– agentlib
– agentpath
Java Agent
• java.lang.instrument
• Comand line option:
– javaagent
63
A note about “agents”
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Native Agent
• JVM TI
• Command line options:
– agentlib
– agentpath
Java Agent
• java.lang.instrument
• Comand line option:
– javaagent
64
A note about “agents”
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java Platform Debugger Architecture
65
TargetJVM
Debugger JVM
JDI
DebuggerTarget Application
(Debuggee)
JVM TI
AgentJDWP
Commands
Events
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java Platform Debugger Architecture
66
TargetJVM
Debugger JVM
JDI
DebuggerTarget Application
(Debuggee)
JVM TI
AgentJDWP
Commands
Events
libjdwp.sojdwp.dll
libjdwp.dylib
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
JVM TI based on Java Native Interface
• JVM TI Callbacks have JNI environment pointer
• Very hard to make a useful JVM TI agent without JNI calls
• JNI data types used (e.g. jint, jdouble)
• Same “modified UTF-8” for strings
• Not simply an extension of JNI, many important differences
67
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java Native Interface
68
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java Native Interface
jdouble Java_pkg_Cls_f__ILjava_lang_String_2 (JNIEnv *env,
jobject obj, jint i, jstring s) {
const char *str = (*env)->GetStringUTFChars(env, s, 0);
...
(*env)->ReleaseStringUTFChars(env, s, str);
return ...
}
69
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java Native Interface
jdouble Java_pkg_Cls_f__ILjava_lang_String_2 (JNIEnv *env,
jobject obj, jint i, jstring s) {
const char *str = (*env)->GetStringUTFChars(env, s, 0);
...
(*env)->ReleaseStringUTFChars(env, s, str);
return ...
}
70
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
JVM TI Environment
• event callbacks
• enabled events
• enabled capabilities
• memory allocation/deallocation hooks
71
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
JVM TI
Created dynamically and works across threads
JNI
One environment per thread
72
JVM TI Environment vs. JNI Environment
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Capabilities
• Flags that represent JVM TI functionality
• Allows “negotiation” between agent and JVM
– Lets agent tell JVM what it needs
– Lets JVM tell agent what it supports
73
Picture: David Monniaux / Mai-Linh Doan (GFDL / cc-by-sa-2.0-fr)
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
• Only “pay” for the features you want / use
• Can be set dynamically
– Some can only be enabled before startup
• 2 levels of cost
– Enabling
– Using
74
Capabilities
Picture: David Monniaux / Mai-Linh Doan (GFDL / cc-by-sa-2.0-fr)
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Bytecode Insertion (BCI)
• Many JVM PI use-cases replaced by BCI in JVM TI
• Instrumentation at the bytecode level is automatically compatible with all JVM technologies.
– Allows instrumented code to be JIT-compiled
– Use any GC collector
75
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
JVM TI Agent Lifecycle
• Initialize
– Agent_OnLoad()
– Agent_OnAttach()
• Probe / request capabilities
• Set callback functions
• Enable events
• Process events via callbacks
• Shutdown on Agent_OnUnload()
76
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
JVM TI Example
• Agent_OnLoad()
– Setup global data for agent’s private use
– Probe/request capabilities
–Modify JRE environment
– Set callback functions
– Enable few events, including VMInit
• VMInit callback– Enable rest of events
• Process events via callbacks
• Cleanup on Agent_OnUnload()
77
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Peaking under the hood of JVM TI:
78
-XX:TraceJVMTI
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Peaking under the hood of JVM TI:
79
-XX:TraceJVMTI
XX:TraceJVMTI= desc {, desc } ...
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Peaking under the hood of JVM TI:
80
-XX:TraceJVMTI
XX:TraceJVMTI= desc {, desc } ...
domain action kind
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Peaking under the hood of JVM TI:
81
-XX:TraceJVMTI
XX:TraceJVMTI= desc {, desc } ...
domain action kind
name a particular function or event name
all all functions and events
func all major functions
allfunc all functions
event all events
ec event controller
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Peaking under the hood of JVM TI:
82
-XX:TraceJVMTI
XX:TraceJVMTI= desc {, desc } ...
domain action kind
+ add
- remove
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Peaking under the hood of JVM TI:
83
-XX:TraceJVMTI
XX:TraceJVMTI= desc {, desc } ...
domain action kind
i input parameters
e error returns
o output
t event triggered (posted)
s event sent
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Full Speed Debugging
84
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Full Speed Debugging
85
• 1.4.0 with client (c1) JIT compiler
• 1.4.1 with both client (c1) and server (c2) JIT compilers
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Full Speed Debugging
86
O RLY?
By pe_ha45 - originally posted to Flickr as Snowy Owl - Schnee-Eule, CC BY 2.0, https://commons.wikimedia.org/w/index.php?curid=11400326
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Full Speed Debugging
Depends on your definition of “Full Speed”…
Pre-1.4: -Xdebug == -Xint
87
By Charles Homler - Own work, CC BY 3.0, https://commons.wikimedia.org/w/index.php?curid=29896748
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Requiredcan_access_local_variables
can_generate_single_step_events
can_generate_exception_events
can_generate_frame_pop_events
can_generate_breakpoint_events
can_suspend
can_generate_method_entry_events
can_generate_method_exit_events
can_generate_garbage_collection_events
can_maintain_original_method_order
can_generate_monitor_events
can_tag_objects
can_get_source_debug_extension
can_get_source_file_name
can_get_line_numbers
Optionalcan_force_early_return
can_force_early_return
can_generate_field_modification_events
can_generate_field_access_events
can_get_bytecodes
can_get_synthetic_attribute
can_get_owned_monitor_info
can_get_current_contended_monitor
can_get_monitor_info
can_pop_frame
can_redefine_classes
can_redefine_any_class
can_get_owned_monitor_stack_depth_info
can_get_constant_pool
can_signal_thread
88
Capabilities Used by JDWP Agent
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Full Speed Debugging
• Only methods with breakpoints are required to be interpreted
• But JVM TI capabilities used by jdwp agent do have a cost
89
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Full Speed Debugging
• Only methods with breakpoints are required to be interpreted
• But JVM TI capabilities used by jdwp agent do have a cost
• Usually not a noticeable impact, but YMMV. There is no free lunch.
90
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Wrap-up
91
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Takeaways
• These APIs are
– Stable
–Officially supported
– Very flexible
• There are limitless new tools you can write and still a lot of room for new ideas
• Not simply for debugging
92
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Resources
[ Martin Skarsaune’s JDI talk ]
video: https://www.youtube.com/watch?v=lNR4bEzYaH4
slides: https://prezi.com/ahspfg2wdkwt/jvm-debugging-under-the-hood-jdays/
[ JDWP Connection and Invocation Details ]
https://docs.oracle.com/javase/8/docs/technotes/guides/jpda/conninv.html
[ JDWP Specification ]
https://docs.oracle.com/javase/8/docs/technotes/guides/jpda/jdwp-spec.html
[ JVM TI Specification ]
https://docs.oracle.com/javase/8/docs/platform/jvmti/jvmti.html
[ Kelly O'Hair's Weblog ]
Use the search tool to find posts about “jvmti”
https://blogs.oracle.com/kto/
93
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 94
Thank You!
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
95
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 96