Download - Living in the Matrix with Bytecode Manipulation (New Relic)

© 2014 SpringOne 2GX. All rights reserved. Do not distribute without permission.

Living in the Matrix with Bytecode Manipulation

Ashley Puls

Wednesday, September 10, 14

2

Ashley PulsSenior Software Engineer

New Relic, Inc.


Outline

• What is bytecode• Why manipulate bytecode• In depth examination of 2 bytecode

manipulation frameworks• Applications• Lessons Learned

3


4Source: http://www.techlila.com/write-programs-linux/

What is Java bytecode?


http://www.techlila.com/write-programs-linux/







6













9

javac -verbose src/com/example/spring2gx/BankTransactions.java



10




11




12




13



14



15


Source: http://www.techlila.com/write-programs-linux/Wednesday, September 10, 14



16


Source: http://www.techlila.com/write-programs-linux/Wednesday, September 10, 14



17


Source: http://www.techlila.com/write-programs-linux/

Instruction set of the Java Virtual Machine




18







19







Why learn about bytecode manipulation?

20



21

• A bytecode manipulation framework is likely on your stack

• Spring• Hibernate

• Groovy• Clojure

• Eclipse• JRuby




• It is really fun!


22






• It is really fun!

• Many applications. Can become a valuable tool.


23




Why manipulate bytecode?

24



25

• Program analysis • find bugs in code• examine code complexity



26


• Class generation• proxies• remove access to certain APIs• compiler for another language like Scala



27



• Transform classes without Java source code• profilers• optimization and obfuscation• additional logging



28



• Transform classes without Java source code• profilers• optimization and obfuscation• additional logging


Logging

29Source: http://www.vforteachers.com/About_NetSupport.htm


http://www.vforteachers.com/About_NetSupport.htm


Logging

30Source: http://www.vforteachers.com/About_NetSupport.htm, http://inzolo.com/blog/tutorials/bank-accounts




http://inzolo.com/blog/tutorials/bank-accounts


Logging

31Source: http://www.vforteachers.com/About_NetSupport.htm, http://upload.wikimedia.org/wikipedia/commons/d/d3/49024-SOS-ATM.JPG






Logging


A message should be logged every time an important action occurs




Logging


A message should be logged every time an important action occurs

Log important values to identify the action




34

public class BankTransactions {

public static void main(String[] args) { BankTransactions bank = new BankTransactions(); // login and withdraw i dollars from account for (int i = 0; i < 100; i++) { String accountId = "account" + i; bank.login("password", accountId, "Ashley");

bank.unimportantProcessing(accountId); bank.withdraw(accountId, Double.valueOf(i)); } System.out.println(“Transactions completed”); }

}


35

public class BankTransactions {

public static void main(String[] args) { BankTransactions bank = new BankTransactions(); // login and withdraw i dollars from account for (int i = 0; i < 100; i++) { String accountId = "account" + i; bank.login("password", accountId, "Ashley");

bank.unimportantProcessing(accountId); bank.withdraw(accountId, Double.valueOf(i)); } System.out.println(“Transactions completed”); }

}


36

/** * A method annotation which should be used to indicate important methods whose * invocations should be logged. * */public @interface ImportantLog {

/** * The method parameter indexes whose values should be logged. For example, * if we have the method hello(int paramA, int paramB, int paramC), and we * wanted to log the values of paramA and paramC, then fields would be ["0", * "2"]. If we only want to log the value of paramB, then fields would be * ["1"]. */ String[] fields();}


37

public void login(String password, String accountId, String userName) { // login logic }

public void withdraw(String accountId, Double moneyToRemove) { // transaction logic }


38

@ImportantLog(fields = { "1", "2" }) public void login(String password, String accountId, String userName) { // login logic }

@ImportantLog(fields = { "0", "1" }) public void withdraw(String accountId, Double moneyToRemove) { // transaction logic }


39



A call was made to "login" on "com/example/spring2gx/BankTransactions" Important params: Index 1 value: ${accountId} Index 2 value: ${userName}


40



A call was made to "login" on "com/example/spring2gx/BankTransactions" Important params: Index 1 value: ${accountId} Index 2 value: ${userName}

A call was made to "withdraw" on "com/example/spring2gx/BankTransactions" Important params: Index 0 value: ${accountId} Index 1 value: ${moneyToAdd} Wednesday, September 10, 14

Java Agent

41

• Ability to modify bytecode without modifying or accessing the application’s source code

• Feature added in Java 1.5• Docs: http://docs.oracle.com/javase/8/docs/api/java/lang/instrument/package-summary.html


http://docs.oracle.com/javase/8/docs/api/java/lang/instrument/package-summary.html




Typical Java process

42

JVM



43

java com/example/spring2gx/BankTransactions

JVM



44


ClassloaderJVM



45


BankTransactions.class

public static void main(String[] args)

ClassloaderJVM



46


Transactions completed

BankTransactions.class

public static void main(String[] args)

ClassloaderJVM


Process with Java agent

47

JVM



48

JVM

java -javaagent:/to/agent.jar com/example/spring2gx/BankTransactions

JVM



49

JVM


JVM



50

JVM


JVM Agent

void premain(String args, Instrumentation inst)

Agent.class1. call Agent premain in manifest



51

JVM


JVM Agent


Agent.class

MyTransformer.class

byte[] transform( . .. , byte[] bankTransBytes)

1. call Agent premain in manifest

2. JVM registers my transformer



52

JVM JVM Agent


Agent.class

MyTransformer.class




3. Give BankTransactions bytes to MyTransformer




53

JVM JVM Agent


Agent.class

MyTransformer.class





4. MyTransformer provides bytes to load




54

JVM JVM Agent


Agent.class

MyTransformer.class


Transactions completed

BankTransactions.classpublic static void main(String[] args)




4. MyTransformer provides bytes to load

5. BankTransactions loaded and main runs



Code for Java agent

55

JVM


Code for Java agent

56

JVM

Manifest: Premain-Class: com.example.spring2gx.agent.Agent


Code for Java agent

57

JVM

package com.example.spring2gx.agent;

public class Agent {

public static void premain(String args, Instrumentation inst) { System.out.println("Starting the agent"); inst.addTransformer(new ImportantLogClassTransformer()); }}



Code for Java agent

58

JVM



public static void premain(String args, Instrumentation inst) { System.out.println("Starting the agent"); inst.addTransformer(new ImportantLogClassTransformer()); }}



Code for Java agent

59

JVM package com.example.spring2gx.agent;

public class ImportantLogClassTransformer implements ClassFileTransformer {

public byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) throws IllegalClassFormatException { //TODO return null; }}


Code for Java agent

60





Code for Java agent

61





Code for Java agent

62



public byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) throws IllegalClassFormatException { System.out.println(" Loading class: " + className); return null; }}


Code for Java agent

63

JVM

Starting the agent Loading class: java/lang/invoke/MethodHandleImpl Loading class: java/lang/invoke/MemberName$Factory Loading class: java/lang/invoke/LambdaForm$NamedFunction Loading class: java/lang/invoke/MethodType$ConcurrentWeakInternSet Loading class: java/lang/invoke/MethodHandleStatics Loading class: java/lang/invoke/MethodHandleStatics$1 Loading class: java/lang/invoke/MethodTypeForm Loading class: java/lang/invoke/Invokers Loading class: java/lang/invoke/MethodType$ConcurrentWeakInternSet$WeakEntry Loading class: java/lang/Void Loading class: java/lang/IllegalAccessException Loading class: sun/misc/PostVMInitHook Loading class: sun/launcher/LauncherHelper Loading class: java/util/concurrent/ConcurrentHashMap$ForwardingNode Loading class: sun/misc/URLClassPath$FileLoader$1 Loading class: com/example/spring2gx/mains/BankTransactions


Code for Java agent

64





Code for Java agent

65





Code for Java agent

66



public byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) throws IllegalClassFormatException { // manipulate the bytes here return null; }}


Code for Java agent

67



public byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) throws IllegalClassFormatException { // manipulate the bytes here return modified bytes; }}


Bytecode Manipulation Frameworks

68

• ASM: http://asm.ow2.org/

• BCEL: http://commons.apache.org/proper/commons-bcel/

• CGLib: https://github.com/cglib/cglib

• Javassist: http://www.csg.ci.i.u-tokyo.ac.jp/~chiba/javassist/

• Serp: http://serp.sourceforge.net/

• Cojen: https://github.com/cojen/Cojen/wiki

• AspectJ: http://www.eclipse.org/aspectj/


http://asm.ow2.org/

http://asm.ow2.org/

http://commons.apache.org/proper/commons-bcel/


https://github.com/cglib/cglib


http://www.csg.ci.i.u-tokyo.ac.jp/~chiba/javassist/


http://serp.sourceforge.net/


https://github.com/cojen/Cojen/wiki


http://www.eclipse.org/aspectj/


Bytecode Manipulation Frameworks• ASM: http://asm.ow2.org/

• BCEL: http://commons.apache.org/proper/commons-bcel/

• CGLib: https://github.com/cglib/cglib

• Javassist: http://www.csg.ci.i.u-tokyo.ac.jp/~chiba/javassist/

• Serp: http://serp.sourceforge.net/

• Cojen: https://github.com/cojen/Cojen/wiki

• AspectJ: http://www.eclipse.org/aspectj/69


http://asm.ow2.org/

http://asm.ow2.org/













• Java Programming Assistant

• Subproject of JBoss

• Object based API

• Actively updated

• Version 3.18.2 released May 2014

• Source code: https://github.com/jboss-javassist/javassist

• Docs: http://www.csg.ci.i.u-tokyo.ac.jp/~chiba/javassist/tutorial/tutorial.html

Javassist

70


https://github.com/jboss-javassist/javassist

https://github.com/jboss-javassist/javassist

http://www.csg.ci.i.u-tokyo.ac.jp/~chiba/javassist/tutorial/tutorial.html

http://www.csg.ci.i.u-tokyo.ac.jp/~chiba/javassist/tutorial/tutorial.html

Javassist

71

CtClass:compile time classCtClass Name

BankTransactions


Javassist

72

CtClass:compile time classCtClass Name

Bank TransactionsBankTransactions


Javassist

73

CtClass:compile time class

ClassPool: container of CtClass objects

Bank Transactions


Javassist

74

CtClass:compile time classClass Name

Bank Transactions



Javassist

75

CtClass:compile time classClass Name

Bank TransactionsBankTransactions



Javassist

76

Java Class: Bank Transactions



Javassist

77

CtClass

Fields

Constructors

Methods



Javassist

78

CtClass

Fields

Constructors

Methods

CtClass



Javassist

79

CtClass

Fields

Constructors

Methods

CtClass

CtFields

CtConstructors

CtMethods


Javassist

80

CtClass

CtField

ClassPool

CtClass

CtClass

CtClass

CtConst

CtMethod

CtMethod


81

Javassist

CtMethod


82

Javassist

line 1 of code

line 2 of code

line 3 of code

line 4 of code

line 5of code

CtMethod


83

Javassist

line 1 of code

line 2 of code

line 3 of code

line 4 of code

line 5 of code

putMethod.insertBefore(“System.out.println(\“before method\”);”);

System.out.println(“before method”);

CtMethod


84

Javassist

line 2 of code

line 3 of code

line 4 of code

line 5 of code

System.out.println(“after method”);

line 1 of code

putMethod.insertAfter(“System.out.println(\”after method\”);”);

CtMethod


85

Javassist

line 2 of code

line 3 of code

line 4 of code

catch (IOException e) { Sys.out.prln( . . .

line 5 of code

line 1 of code

CtClass exceptionType = classpool.get(“java.io.IOException”);putMethod.addCatch(“{System.out.println($e); throw $e}”, exceptionType);

CtMethod


86

Javassist

line 2 of code

System.out.println(“middle method”);

line 3 of code

line 4 of code

line 5 of code

line 1 of code

putMethod.insertAt(3, true, “System.out.println(\“middle method\”);”);

CtMethod


Javassist

87

CtClass

CtField

ClassPool

CtClass

CtClass

CtClass

CtConst

CtMethod

CtMethod

insertBefore

insertAfter

catchEx

insertAt


88

JVM



public static void premain(String args, Instrumentation inst) { System.out.println("Starting the agent"); inst.addTransformer(new ImportantLogClassTransformer()); }

}


Code for Java agent

89





90

public byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] cfbuffer) throws IllegalClassFormatException { 1. convert byte array to a ct class object 2. check each method of ct class for annotation @ImportantLog 3. if @ImportantLog annotation is present on method, then a. get important method parameter indexes

b. add logging statement to beginning of the method return null;}Wednesday, September 10, 14

91

public byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] cfbuffer) throws IllegalClassFormatException { pool.insertClassPath(new ByteArrayClassPath(className,classfileBuffer)); CtClass cclass = pool.get(className.replaceAll("/", ".")); 2. check each method of ct class for annotation @ImportantLog 3. if @ImportantLog annotation present on method, then a. get important method parameter indexes

b. add logging statement to beginning of the method return null;}Wednesday, September 10, 14

92

public byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] cfbuffer) throws IllegalClassFormatException { pool.insertClassPath(new ByteArrayClassPath(className,classfileBuffer)); CtClass cclass = pool.get(className.replaceAll("/", ".")); if (!cclass.isFrozen()) { for (CtMethod currentMethod : cclass.getDeclaredMethods()) { Annotation annotation = getAnnotation(currentMethod); 3. if @ImportantLog annotation present on method, then a. get important method parameter indexes

b. add logging statement to beginning of the method } } } return null;}Wednesday, September 10, 14

93

public byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] cfbuffer) throws IllegalClassFormatException { pool.insertClassPath(new ByteArrayClassPath(className,classfileBuffer)); CtClass cclass = pool.get(className.replaceAll("/", ".")); if (!cclass.isFrozen()) { for (CtMethod currentMethod : cclass.getDeclaredMethods()) { Annotation annotation = getAnnotation(currentMethod); if (annotation != null) { List<String> parameterIndexes = getParamIndexes(annotation);

b. add logging statement to beginning of the method } } } return null;}Wednesday, September 10, 14

94

public byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] cfbuffer) throws IllegalClassFormatException { pool.insertClassPath(new ByteArrayClassPath(className,classfileBuffer)); CtClass cclass = pool.get(className.replaceAll("/", ".")); if (!cclass.isFrozen()) { for (CtMethod currentMethod : cclass.getDeclaredMethods()) { Annotation annotation = getAnnotation(currentMethod); if (annotation != null) { List<String> parameterIndexes = getParamIndexes(annotation); currentMethod.insertBefore(createJavaString( currentMethod, className, parameterIndexes)); } } return cclass.toBytecode(); } return null;}Wednesday, September 10, 14

95


96

private Annotation getAnnotation(CtMethod method) { MethodInfo mInfo = method.getMethodInfo(); // the attribute we are looking for is a runtime invisible attribute // use Retention(RetentionPolicy.RUNTIME) on the annotation to make it // visible at runtime AnnotationsAttribute attInfo = (AnnotationsAttribute) mInfo .getAttribute(AnnotationsAttribute.invisibleTag); if (attInfo != null) { // this is the type name meaning use dots instead of slashes return attInfo.getAnnotation("com.example.spring.mains.ImportantLog"); } return null; }


97


98


99

private List<String> getParamIndexes(Annotation annotation) { ArrayMemberValue fields = (ArrayMemberValue) annotation .getMemberValue(“fields”); if (fields != null) { MemberValue[] values = (MemberValue[]) fields.getValue(); List<String> parameterIndexes = new ArrayList<String>(); for (MemberValue val : values) { parameterIndexes.add(((StringMemberValue) val).getValue()); } return parameterIndexes; } return Collections.emptyList(); }


100


101


102


103

private String createJavaString(CtMethod currentMethod, String className, List<String> indexParameters) { StringBuilder sb = new StringBuilder(); sb.append("{StringBuilder sb = new StringBuilder"); sb.append("(\"A call was made to method '\");"); sb.append("sb.append(\""); sb.append(currentMethod.getName()); sb.append("\");sb.append(\"' on class '\");"); sb.append("sb.append(\""); sb.append(className); sb.append("\");sb.append(\"'.\");"); sb.append("sb.append(\"\\n Important params:\");"); . . .


104

private String createJavaString(CtMethod currentMethod, String className, List<String> indexParameters) { StringBuilder sb = new StringBuilder(); sb.append("{StringBuilder sb = new StringBuilder"); sb.append("(\"A call was made to method '\");"); sb.append("sb.append(\""); sb.append(currentMethod.getName()); sb.append("\");sb.append(\"' on class '\");"); sb.append("sb.append(\""); sb.append(className); sb.append("\");sb.append(\"'.\");"); sb.append("sb.append(\"\\n Important params:\");"); . . .

Put multiple statements inside brackets {}


105

private String createJavaString(CtMethod currentMethod, String className, List<String> indexParameters) { . . . for (String index : indexParameters) { try { int localVar = Integer.parseInt(index) + 1; sb.append("sb.append(\"\\n Index: \");"); sb.append("sb.append(\""); sb.append(index); sb.append("\");sb.append(\" value: \");"); sb.append("sb.append($" + localVar + ");"); } catch (NumberFormatException e) { e.printStackTrace(); } } sb.append("System.out.println(sb.toString());}"); return sb.toString();}


106

private String createJavaString(CtMethod currentMethod, String className, List<String> indexParameters) { . . . for (String index : indexParameters) { try { int localVar = Integer.parseInt(index) + 1; sb.append("sb.append(\"\\n Index: \");"); sb.append("sb.append(\""); sb.append(index); sb.append("\");sb.append(\" value: \");"); sb.append("sb.append($" + localVar + ");"); } catch (NumberFormatException e) { e.printStackTrace(); } } sb.append("System.out.println(sb.toString());}"); return sb.toString();} $0, $1, $2, ... used to access “this” and method parametersWednesday, September 10, 14

107


• Positives

• Simplicity

• Documentation

• Negatives

• Limited functionality

• Slow

108

Javassist


ASM

109

• Released in Open Source in 2002

• Actively updated

• Version 5.0.3 released May 2014

• Two ASM libraries

• Event based (SAX like)

• Object based (DOM like)

• Documentation: http://download.forge.objectweb.org/asm/asm4-guide.pdf


http://download.forge.objectweb.org/asm/asm4-guide.pdf


110

ASM

Source: http://download.forge.objectweb.org/asm/asm4-guide.pdfWednesday, September 10, 14



111

ASM




112

ASM




113

ASM

ClassReader: event producer

ClassWriter: event consumer

ClassVisitor: event filter

BankTrans


ClassReader: given a byte[], parses a compiled class

114

ASM




BankTrans


ClassWriter: produces output byte[]

115

ASM





BankTrans


116

ASM




BankTrans


BankTrans


ClassVisitor: delegates class events, event filter

117

ASM



BankTrans


BankTrans


R:B:

118

ASM



BankTrans


visitField

visitMethod

BankTrans



119

ASM


import java.lang.instrument.Instrumentation;


public static void premain(String args, Instrumentation inst) { System.out.println("Starting the agent"); inst.addTransformer(new ImportantLogClassTransformer()); }

}


120




ASM


121

public byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) throws IllegalClassFormatException {

ClassReader cr = new ClassReader(classfileBuffer); ClassWriter cw = new ClassWriter(cr, ClassWriter.COMPUTE_FRAMES);

cr.accept(cw, 0); return cw.toByteArray();

}

ASM


122

public byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) throws IllegalClassFormatException {

ClassReader cr = new ClassReader(classfileBuffer); ClassWriter cw = new ClassWriter(cr, ClassWriter.COMPUTE_FRAMES); ClassVisitor cv = new LogMethodClassVisitor(cw, className); cr.accept(cv, 0); return cw.toByteArray();

}

ASM


R:B:

123

ASM



BankTrans


visitField

visitMethod

BankTrans


R:B:

124

ASM



BankTransvisitField

visitMethod

BankTrans

LogMethodClassVisitor:event filter


125

ASM


BankTrans


BankTrans

LogMethodClassVisitor:event filter

visitMethod


126

ASM




127

ASMpublic class LogMethodClassVisitor extends ClassVisitor {

private String className;

public LogMethodIfAnnotationVisitor(ClassVisitor cv, String className) { super(Opcodes.ASM5, cv); this.className = className; }

@Override public MethodVisitor visitMethod(int access, String name, String desc, String signature, String[] exceptions) { // put our logic in here }}


128

ASMpublic class LogMethodClassVisitor extends ClassVisitor {

private String className;


@Override public MethodVisitor visitMethod(int access, String name, String desc, String signature, String[] exceptions) { // put our logic in here }}


129

ASM




130

ASM




131

ASM




132

ASM

Method Visitor

visitAnnotation

Method Visitor

visitCode

visitEnd

Login Method Login Method


133

ASMpublic class LogMethodClassVisitor extends ClassVisitor { private String className;


@Override public MethodVisitor visitMethod(int access, String name, String desc, String signature, String[] exceptions) { // put our logic in here }}Wednesday, September 10, 14

134

ASMpublic class LogMethodClassVisitor extends ClassVisitor { private String className;


@Override public MethodVisitor visitMethod(int access, String name, String desc, String signature, String[] exceptions) { MethodVisitor mv = super.visitMethod(access, name, desc, signature, exceptions); return new PrintMessageMethodVisitor(mv, name, className); }}Wednesday, September 10, 14

135

ASM




136

ASMpublic class PrintMessageMethodVisitor extends MethodVisitor {

@Override public AnnotationVisitor visitAnnotation(String desc, boolean visible) { }

@Override public void visitCode() { }

}


137


@Override public AnnotationVisitor visitAnnotation(String desc, boolean visible) { 1. check method for annotation @ImportantLog 2. if annotation present, then get important method param indexes }

@Override public void visitCode() { 3. if annotation present, add logging to beginning of the method }

}


138

ASM public AnnotationVisitor visitAnnotation(String desc, boolean visible) { if ("Lcom/example/spring2gx/mains/ImportantLog;".equals(desc)) { isAnnotationPresent = true; return new AnnotationVisitor(Opcodes.ASM5, super.visitAnnotation(desc, visible)) { public AnnotationVisitor visitArray(String name) { if (“fields”.equals(name)) { return new AnnotationVisitor(Opcodes.ASM5, super.visitArray(name)) { public void visit(String name, Object value) { parameterIndexes.add((String) value); super.visit(name, value); } }; } else { return super.visitArray(name); } } }; } return super.visitAnnotation(desc, visible); }


139


@Override public AnnotationVisitor visitAnnotation(String desc, boolean visible) { 1. check method for annotation @ImportantLog 2. if annotation present, then get important method param indexes }

@Override public void visitCode() { 3. if annotation present, add logging to beginning of the method }

}


140

ASM public void visitCode() { if (isAnnotationPresent) { // create string builder mv.visitFieldInsn(Opcodes.GETSTATIC, "java/lang/System", "out", "Ljava/io/PrintStream;"); mv.visitTypeInsn(Opcodes.NEW, "java/lang/StringBuilder"); mv.visitInsn(Opcodes.DUP); // add everything to the string builder mv.visitLdcInsn("A call was made to method \""); mv.visitMethodInsn(Opcodes.INVOKESPECIAL, "java/lang/StringBuilder", "<init>", "(Ljava/lang/String;)V", false); mv.visitLdcInsn(methodName); mv.visitMethodInsn(Opcodes.INVOKEVIRTUAL, "java/lang/StringBuilder", "append", "(Ljava/lang/String;)Ljava/lang/StringBuilder;", false);

. . .Wednesday, September 10, 14

141

Javap


142

Javappublic class PrintMessage { private String methodName; private String className; public PrintMessage(String mName, String cName) { methodName = mName; className = cName; } public void print() { StringBuilder sb = new StringBuilder(); sb.append("A call was made to method \""); sb.append(methodName); sb.append("\" on class \""); sb.append(className); sb.append("\"."); System.out.println(sb.toString()); }


143

Javap

javap -c com/example/spring2gx/mains/PrintMessage


public void print(); Code: 0: new #38 // class java/lang/StringBuilder 3: dup 4: invokespecial #40 // Method java/lang/StringBuilder."<init>":()V 7: astore_1 8: aload_1 9: ldc #41 // String A call was made to method \" 11: invokevirtual #43 // Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder; 14: pop 15: aload_1 16: aload_0 17: getfield #14 // Field methodName:Ljava/lang/String; 20: invokevirtual #43 // Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;. . .

144

Javap


public void print(); Code: 0: new #38 // class java/lang/StringBuilder 3: dup 4: invokespecial #40 // Method java/lang/StringBuilder."<init>":()V 7: astore_1 8: aload_1 9: ldc #41 // String A call was made to method \" 11: invokevirtual #43 // Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder; 14: pop 15: aload_1 16: aload_0 17: getfield #14 // Field methodName:Ljava/lang/String; 20: invokevirtual #43 // Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;. . .

145

Javap


146


147


148

Javap


149

Javap


150

Javap


151

ASM public void visitCode() { if (isAnnotationPresent) { // create string builder mv.visitFieldInsn(Opcodes.GETSTATIC, "java/lang/System", "out", "Ljava/io/PrintStream;"); mv.visitTypeInsn(Opcodes.NEW, "java/lang/StringBuilder"); mv.visitInsn(Opcodes.DUP); // add everything to the string builder mv.visitLdcInsn("A call was made to method \""); mv.visitMethodInsn(Opcodes.INVOKESPECIAL, "java/lang/StringBuilder", "<init>", "(Ljava/lang/String;)V", false); mv.visitLdcInsn(methodName); mv.visitMethodInsn(Opcodes.INVOKEVIRTUAL, "java/lang/StringBuilder", "append", "(Ljava/lang/String;)Ljava/lang/StringBuilder;", false);

. . .Wednesday, September 10, 14

152

ASM• Positives

• Speed and Size

• Documentation

• Depth of functionality

• Number of people using the framework

• Negatives

• Developer ramp-up

• Writing byte code


153

Applications


154

Hibernate


155

Hibernate

LazyInitializationException


156

Hibernate



157

Hibernate

9 instances of “throw new LazyInitializationException”



158

Hibernate

Name Location Team

Joe Smith Dallas, TX Team 1

Jane Doe Portland, OR

Triathlon Club

Tim Doe San Fran, CA

Running Club

Sue Smith Austin, TX Red Lizards

Name Location Description

Team 1 Dallas, TX Casual runners

Triathlon Club

Beaverton, OR

Swim, bike, run

Running Club

Santa Clara, CA

For fun

Red Lizards Austin, TX Fast

Runner Team


159

Hibernate

Conference AttendeeRunner


160

Hibernate

Name: Ashley Puls

Runner


161

Hibernate

Name: Ashley Puls

Location: Portland, OR

Runner


162

Hibernate

Name: Ashley Puls

Team: Hibernate Proxy

Name: Ashley Puls


Runner


163

Hibernate

Name: Ashley Puls

Company:

164

Name: Ashley Puls

Team

Hibernate Proxy

Name: Triathlon Club

Location: null

Description: null

Name: Ashley Puls


Runner


164

Hibernate

Name: Ashley Puls

Company:

164

Name: Ashley Puls

Hibernate Proxy


Description:

Name: Ashley Puls


Runner


165

Hibernate

Name: Ashley Puls

Company:

164

Name: Ashley Puls

Hibernate Proxy


Description:

Name: Ashley Puls


Runner


166

Hibernate

Name: Ashley Puls

Company:

164

Name: Ashley Puls

Team

Hibernate Proxy

call a getter on the hibernate proxy:runner.getTeam().getLocation();Name: Ashley Puls


Runner


Location: null

Description: null


167

Hibernate

Name: Ashley Puls

Company:

164

Name: Ashley Puls

Team

Hibernate Proxy Team

Name: Ashley Puls


Runner


Location: null

Description: null


Location: Beaverton, OR

Description: swim, bike, run

call a getter on the hibernate proxy:runner.getTeam().getLocation();


168

Hibernate

Name: Ashley Puls

Company:

164

Name: Ashley Puls

Hibernate Proxy


Description:

Name: Ashley Puls


Runner


169

Hibernate

Name: Ashley Puls

Company:

164

Name: Ashley Puls

Team

Hibernate Proxy Team


Location: Beaverton, OR

Description: swim, bike, run

Trying to access uninitialized field outside a session


Name: Ashley Puls


Runner


Location: null

Description: null


170

Hibernate

package org.hibernate.proxy.pojo.javassist;

/** * A Javassist-based lazy initializer proxy. * * @author Muga Nishizawa */public class JavassistLazyInitializer extends BasicLazyInitializer implements MethodHandler {


171

Hibernate

public static HibernateProxy getProxy( . . . ) { . . . ProxyFactory factory = new ProxyFactory(); factory.setSuperclass(interfaces.length == 1 ? persistentClass : null); factory.setInterfaces(interfaces); factory.setFilter(FINALIZE_FILTER); Class cl = factory.createClass(); final HibernateProxy proxy = (HibernateProxy) cl.newInstance(); ((Proxy) proxy).setHandler(instance); instance.constructed = true; return proxy; . . . }


172

Hibernatepublic Object invoke(final Object proxy, final Method thisMethod, final Method proceed, final Object[] args) throws Throwable { if (this.constructed) { Object result = this.invoke(thisMethod, args, proxy); if (result == INVOKE_IMPLEMENTATION) { Object target = getImplementation(); final Object returnValue = thisMethod.invoke(target, args); if (returnValue == target) { if (returnValue.getClass().isInstance(proxy)) { return proxy; } return returnValue; } else { return result; } } else { if (thisMethod.getName().equals("getHibernateLazyInitializer")) { return this; } else { return proceed.invoke(proxy, args); } }}


173

Hibernate

Hibernate Proxy

Why do we need a proxy?

Name: Ashley Puls


Name: Ashley Puls


Runner


174

Hibernate

Name: Ashley Puls

Title: Software Engineer

Company: New Relic

Conference Attendee

1. Is the entity dirty? Have we changed any of the data?

2. What fields on the entity have been loaded?

Name: Ashley Puls


Name: Ashley Puls


Runner


Hibernate - Build Time Tasks

• Schema Tasks• Schema Export• Schema Update• Schema Validator

175




• ByteCode Tasks• EnhancementTask• InstrumentTask

176




• ByteCode Tasks• EnhancementTask• InstrumentTask

177


Hibernate - Ant

178

<taskdef name="enhance" classname="org.hibernate.tool.enhance.EnhancementTask" classpathref="enhancement.classpath" />

<enhance> <fileset dir="${ejb-classes}/org/hibernate/auction/model" includes="**/*.class"/></enhance>


Hibernate - Gradle

179

apply plugin: 'java'apply plugin: 'maven'apply plugin: 'enhance'buildscript { repositories { mavenCentral() } dependencies { classpath 'org.hibernate:hibernate-gradle-plugin:VERSION' }}dependencies { compile group: 'org.hibernate.javax.persistence', name: 'hibernate-jpa-[SPEC-VERSION]-api', version: '[IMPL-VERSION]' compile group: 'org.hibernate', name: 'hibernate-gradle-plugin', version: 'VERSION'}


180

Hibernatepublic class EnhancementTask extends Task implements EnhancementContext { private List<FileSet> filesets = new ArrayList<FileSet>(); private final ClassPool classPool = new ClassPool( false ); private final Enhancer enhancer = new Enhancer( this ); @Override public void execute() throws BuildException { for (FileSet fileSet : filesets) { final File fileSetBaseDir = fileSet.getDir(project);

for (String relativeIncludedFileName :scanner.getIncludedFiles()) { final File javaClassFile = new File( fileSetBaseDir, relativeIncludedFileName ); processClassFile( javaClassFile); } } }Wednesday, September 10, 14

181



182



183



184

Hibernate

/** * Atm only process files annotated with either @Entity or @Embeddable */ private void processClassFile(File javaClassFile) {

final CtClass ctClass = classPool.makeClass( new FileInputStream( javaClassFile ) );

processEntityClassFile(javaClassFile, ctClass); }





185

Hibernate





186

Hibernate


187

Hibernate

private void processEntityClassFile(File javaClassFile, CtClass ctClass ) {

byte[] result = enhancer.enhance(ctClass.getName(),ctClass.toBytecode()); if(result != null) writeEnhancedClass(javaClassFile, result); }

}


188

Hibernate



}


private void enhance(CtClass managedCtClass, boolean isComposite) {

// 1. stop if managedCtClass is an interface// 2. stop if managedCtClass is already enhanced

// 3. stop if managedCtClass is not an entity }

189

Hibernate




// 3. stop if managedCtClass is not an entity // 4. enhance managedCtClass.addInterface( managedEntityCtClass ); enhancePersistentAttributes( managedCtClass ); addEntityInstanceHandling( managedCtClass ); addEntityEntryHandling( managedCtClass ); addLinkedPreviousHandling( managedCtClass ); addLinkedNextHandling( managedCtClass );}

190

Hibernate





191

Hibernate





192

Hibernate


193

Hibernateprivate CtField addField(CtClass targetClass, CtClass fieldType, String fieldName, boolean makeTransient) { final ConstPool constPool = targetClass.getClassFile().getConstPool(); final CtField theField = new CtField( fieldType, fieldName, targetClass ); targetClass.addField( theField ); // make that new field (1) private, (2) transient and (3) @Transient if ( makeTransient ) { theField.setModifiers( theField.getModifiers() | Modifier.TRANSIENT ); } theField.setModifiers( Modifier.setPrivate( theField.getModifiers() ) );

final AnnotationsAttribute annotationsAttribute = getVisibleAnnotations(theField.getFieldInfo()); annotationsAttribute.addAnnotation( new Annotation( Transient.class.getName(), constPool ) ); return theField;}Wednesday, September 10, 14

194



195



196



197

Hibernate

private void addGetter(CtClass targetClass, CtField theField, String getterName) { targetClass.addMethod( CtNewMethod.getter( getterName, theField ) );}

private void addSetter(CtClass targetClass, CtField theField, String setterName) { targetClass.addMethod( CtNewMethod.setter( setterName, theField ) );}


198

Hibernate

private void addGetter(CtClass targetClass, CtField theField, String getterName) { targetClass.addMethod( CtNewMethod.getter( getterName, theField ) );}

private void addSetter(CtClass targetClass, CtField theField, String setterName) { targetClass.addMethod( CtNewMethod.setter( setterName, theField ) );}


199

Hibernate

private byte[] getByteCode(CtClass managedCtClass) { final ByteArrayOutputStream byteStream = new ByteArrayOutputStream(); final DataOutputStream out = new DataOutputStream( byteStream ); try { managedCtClass.toBytecode( out ); return byteStream.toByteArray(); } finally { out.close(); }}


200

Hibernate

private byte[] getByteCode(CtClass managedCtClass) { final ByteArrayOutputStream byteStream = new ByteArrayOutputStream(); final DataOutputStream out = new DataOutputStream( byteStream ); try { managedCtClass.toBytecode( out ); return byteStream.toByteArray(); } finally { out.close(); }}


201

Hibernate



}


202

Hibernate



}


203

Hibernate private void writeEnhancedClass(File javaClassFile, byte[] result) {

if ( javaClassFile.delete() ) { if ( ! javaClassFile.createNewFile() ) { log( "Unable to recreate class file [" + javaClassFile.getName() + "]", Project.MSG_INFO ); } }

FileOutputStream outputStream = new FileOutputStream( javaClassFile, false ); try { outputStream.write( result); outputStream.flush(); } finally { outputStream.close(); } }


204





205





206

Spring


207

Spring


208

Spring

https://github.com/scottfrederick/spring-music




209

Springpackage org.cloudfoundry.samples.music.config.web;

@Configuration@ComponentScan(basePackageClasses = { AlbumController.class })public class WebMvcConfig extends WebMvcConfigurerAdapter {

@Bean public InternalResourceViewResolver internalResourceViewResolver() { InternalResourceViewResolver internalResourceViewResolver = new InternalResourceViewResolver(); internalResourceViewResolver.setPrefix("/WEB-INF/"); . . . Wednesday, September 10, 14

210




211




212

Spring

@Controller@RequestMapping(value = "/albums")public class AlbumController { private static final Logger logger = LoggerFactory.getLogger(AlbumController.class);

private final AlbumRepository repository; private SongsService songs;


213

Spring

@Controller@RequestMapping(value = "/albums")public class AlbumController { private static final Logger logger = LoggerFactory.getLogger(AlbumController.class);

private final AlbumRepository repository; private SongsService songs;


214

Spring

@Component: A generic Spring component

@Repository: Class at the persistence layer

@Controller: Class which receives HTTP servlet requests and responses

@Service: Class at the service level

AlbumRepositoryPopulator

MongoAlbumRepository, JpaAlbumRepository

ErrorController, InfoController, AlbumController


215

Springpackage org.springframework.context.annotation;

/** * A component provider that scans the classpath from a base package. * It then applies exclude and include filters to the resulting * classes to find candidates. * * <p>This implementation is based on Spring's * {@link org.springframework.core.type.classreading.MetadataReader * MetadataReader} * facility, backed by an ASM * {@linkorg.springframework.asm.ClassReader ClassReader}. */public class ClassPathScanningCandidateComponentProvider implements EnvironmentCapable, ResourceLoaderAware {


216

Spring

public Set<BeanDefinition> findCandidateComponents(String basePackage) { Set<BeanDefinition> candidates = new LinkedHashSet<BeanDefinition>(); String packageSearchPath = ResourcePatternResolver.CLASSPATH_ALL_URL_PREFIX + resolveBasePackage(basePackage) + "/" + resourcePattern; Resource[] resources = resourcePatternResolver.getResources(packageSearchPath); for (Resource resource : resources) { MetadataReader metadataReader = metadataReaderFactory.getMetadataReader(resource); if (isCandidateComponent(metadataReader)) { ScannedGenericBeanDefinition sbd = new ScannedGenericBeanDefinition(metadataReader); sbd.setResource(resource); sbd.setSource(resource); if (isCandidateComponent(sbd)) { candidates.add(sbd); } } } return candidates;}


217

Spring

public Set<BeanDefinition> findCandidateComponents(String basePackage) { Set<BeanDefinition> candidates = new LinkedHashSet<BeanDefinition>(); String packageSearchPath = ResourcePatternResolver.CLASSPATH_ALL_URL_PREFIX + resolveBasePackage(basePackage) + "/" + resourcePattern; Resource[] resources = resourcePatternResolver.getResources(packageSearchPath); for (Resource resource : resources) { MetadataReader metadataReader = metadataReaderFactory.getMetadataReader(resource); if (isCandidateComponent(metadataReader)) { ScannedGenericBeanDefinition sbd = new ScannedGenericBeanDefinition(metadataReader); sbd.setResource(resource); sbd.setSource(resource); if (isCandidateComponent(sbd)) { candidates.add(sbd); } } } return candidates;}


218

SpringSimpleMetadataReader

Resource


R:B:

219


Resource


R:B:

220


metadataResource


221

Spring

package org.springframework.core.type.classreading;

final class SimpleMetadataReader implements MetadataReader {

SimpleMetadataReader(Resource resource, ClassLoader classLoader) throws IOException { InputStream is = new BufferedInputStream(resource.getInputStream()); ClassReader classReader = new ClassReader(is); AnnotationMetadataReadingVisitor visitor = new

AnnotationMetadataReadingVisitor(classLoader); classReader.accept(visitor, ClassReader.SKIP_DEBUG);

this.annotationMetadata = visitor; this.classMetadata = visitor; this.resource = resource; }


222

Spring







223

Spring







224

Spring







225

Spring







R:B:

226

Spring

ClassReader

SimpleMetadataReader

metadata

AnnotationMetadataReadingVisitor

Resource


R:B:

227

ASM



BankTrans


visitField

visitMethod

BankTrans



228

Springpublic class AnnotationMetadataReadingVisitor extends ClassMetadataReadingVisitor implements AnnotationMetadata {

public MethodVisitor visitMethod(int access, String name, String desc, String signature, String[] exceptions) { if ((access & Opcodes.ACC_BRIDGE) != 0) { return super.visitMethod(access, name, desc, signature, exceptions); } return new MethodMetadataReadingVisitor(name, access, getClassName(), classLoader, methodMetadataSet); }

public AnnotationVisitor visitAnnotation(final String desc, boolean visible) { String className = Type.getType(desc).getClassName(); this.annotationSet.add(className); return new AnnotationAttributesReadingVisitor(className, attributesMap,

metaAnnotationMap, classLoader); }Wednesday, September 10, 14

229





230





R:B:

231

Spring

ClassReader


metadata


Resource

visitMethod


232





233





234





R:B:

235

Spring

ClassReader


metadata


Resource

visitMethod

visitAnnotation


R:B:

236

Spring

ClassReader


metadata


Resource

visitMethod

visitAnnotation

visitAnnotation

visitvisitAnnotation

visitArrayvisitEnum


237

Springrepackaging ASM in spring-core

Source: http://www.clker.com/clipart-3085.html


https://jira.spring.io/browse/SPR-10292


http://www.clker.com/clipart-3085.html


238

Springrepackaging ASM in spring-core

Source: http://www.clker.com/clipart-3085.html









239

Lessons Learned


240

Lessons Learned

Source: http://www.mindbodygreen.com/0-4965/How-Baby-Steps-Lead-to-Big-Results.html


http://www.mindbodygreen.com/0-4965/How-Baby-Steps-Lead-to-Big-Results.html

http://www.mindbodygreen.com/0-4965/How-Baby-Steps-Lead-to-Big-Results.html

241

Lessons Learned

Source: http://www.dishesandsocks.com/2011/06/16/teacher-classroom-organization-part-1/


http://www.dishesandsocks.com/2011/06/16/teacher-classroom-organization-part-1/

http://www.dishesandsocks.com/2011/06/16/teacher-classroom-organization-part-1/

242

Lessons Learned

Source: http://4hdwallpapers.com/transformer-isolated-on-white-background-3d-render.html


http://4hdwallpapers.com/transformer-isolated-on-white-background-3d-render.html

http://4hdwallpapers.com/transformer-isolated-on-white-background-3d-render.html

243

Applications

Obfuscate so users can not view your source code

Find bugs

Write your own compiler for a language that runs on the JVM

Add logging to an application

Perform preprocessing of code at build time

Restrict API methods for specific users

Determine if a collection is accessed by multiple threads

Lazy load data from a databaseFind differences between jars


244

Make bytecode manipulation frameworks your friend. They might save your job one day.

Source: http://www.kulfoto.com/dog-pictures/15799/mans-best-friend


http://www.kulfoto.com/dog-pictures/15799/mans-best-friend

http://www.kulfoto.com/dog-pictures/15799/mans-best-friend

245


Source: http://alangregerman.typepad.com/.a/6a00d83516c0ad53ef017c3329aba8970b-800wi

Questions?

246


http://alangregerman.typepad.com/.a/6a00d83516c0ad53ef017c3329aba8970b-800wi

http://alangregerman.typepad.com/.a/6a00d83516c0ad53ef017c3329aba8970b-800wi