Living in the Matrix with Bytecode Manipulation (New Relic)

transcript

Living in the Matrix with Bytecode Manipulation

Ashley Puls

Wednesday, September 10, 14

Ashley PulsSenior Software Engineer

New Relic, Inc.

Outline

• What is bytecode• Why manipulate bytecode• In depth examination of 2 bytecode

manipulation frameworks• Applications• Lessons Learned

4Source: http://www.techlila.com/write-programs-linux/

What is Java bytecode?

javac -verbose src/com/example/spring2gx/BankTransactions.java

Source: http://www.techlila.com/write-programs-linux/Wednesday, September 10, 14

Source: http://www.techlila.com/write-programs-linux/

Instruction set of the Java Virtual Machine

Why learn about bytecode manipulation?

• A bytecode manipulation framework is likely on your stack

• Spring• Hibernate

• Groovy• Clojure

• Eclipse• JRuby

• It is really fun!

• Many applications. Can become a valuable tool.

Why manipulate bytecode?

• Program analysis • find bugs in code• examine code complexity

• Class generation• proxies• remove access to certain APIs• compiler for another language like Scala

• Transform classes without Java source code• profilers• optimization and obfuscation• additional logging

Logging

29Source: http://www.vforteachers.com/About_NetSupport.htm

Logging

30Source: http://www.vforteachers.com/About_NetSupport.htm, http://inzolo.com/blog/tutorials/bank-accounts

Logging

31Source: http://www.vforteachers.com/About_NetSupport.htm, http://upload.wikimedia.org/wikipedia/commons/d/d3/49024-SOS-ATM.JPG

Logging

A message should be logged every time an important action occurs

Logging

A message should be logged every time an important action occurs

Log important values to identify the action

public class BankTransactions {

public static void main(String[] args) { BankTransactions bank = new BankTransactions(); // login and withdraw i dollars from account for (int i = 0; i < 100; i++) { String accountId = "account" + i; bank.login("password", accountId, "Ashley");

bank.unimportantProcessing(accountId); bank.withdraw(accountId, Double.valueOf(i)); } System.out.println(“Transactions completed”); }

public class BankTransactions {

public static void main(String[] args) { BankTransactions bank = new BankTransactions(); // login and withdraw i dollars from account for (int i = 0; i < 100; i++) { String accountId = "account" + i; bank.login("password", accountId, "Ashley");

bank.unimportantProcessing(accountId); bank.withdraw(accountId, Double.valueOf(i)); } System.out.println(“Transactions completed”); }

/** * A method annotation which should be used to indicate important methods whose * invocations should be logged. * */public @interface ImportantLog {

/** * The method parameter indexes whose values should be logged. For example, * if we have the method hello(int paramA, int paramB, int paramC), and we * wanted to log the values of paramA and paramC, then fields would be ["0", * "2"]. If we only want to log the value of paramB, then fields would be * ["1"]. */ String[] fields();}

public void login(String password, String accountId, String userName) { // login logic }

public void withdraw(String accountId, Double moneyToRemove) { // transaction logic }

@ImportantLog(fields = { "1", "2" }) public void login(String password, String accountId, String userName) { // login logic }

@ImportantLog(fields = { "0", "1" }) public void withdraw(String accountId, Double moneyToRemove) { // transaction logic }

A call was made to "login" on "com/example/spring2gx/BankTransactions" Important params: Index 1 value: ${accountId} Index 2 value: ${userName}

A call was made to "withdraw" on "com/example/spring2gx/BankTransactions" Important params: Index 0 value: ${accountId} Index 1 value: ${moneyToAdd} Wednesday, September 10, 14

Java Agent

• Ability to modify bytecode without modifying or accessing the application’s source code

• Feature added in Java 1.5• Docs: http://docs.oracle.com/javase/8/docs/api/java/lang/instrument/package-summary.html

Typical Java process

java com/example/spring2gx/BankTransactions

ClassloaderJVM

BankTransactions.class

public static void main(String[] args)

ClassloaderJVM

Transactions completed

BankTransactions.class

public static void main(String[] args)

ClassloaderJVM

Process with Java agent

java -javaagent:/to/agent.jar com/example/spring2gx/BankTransactions

JVM Agent

void premain(String args, Instrumentation inst)

Agent.class1. call Agent premain in manifest

JVM Agent

Agent.class

MyTransformer.class

byte[] transform( . .. , byte[] bankTransBytes)

1. call Agent premain in manifest

2. JVM registers my transformer

JVM JVM Agent

Agent.class

MyTransformer.class

3. Give BankTransactions bytes to MyTransformer

JVM JVM Agent

Agent.class

MyTransformer.class

4. MyTransformer provides bytes to load

JVM JVM Agent

Agent.class

MyTransformer.class

Transactions completed

BankTransactions.classpublic static void main(String[] args)

4. MyTransformer provides bytes to load

5. BankTransactions loaded and main runs

Code for Java agent

Manifest: Premain-Class: com.example.spring2gx.agent.Agent

Code for Java agent

package com.example.spring2gx.agent;

public class Agent {

public static void premain(String args, Instrumentation inst) { System.out.println("Starting the agent"); inst.addTransformer(new ImportantLogClassTransformer()); }}

Code for Java agent

public static void premain(String args, Instrumentation inst) { System.out.println("Starting the agent"); inst.addTransformer(new ImportantLogClassTransformer()); }}

Code for Java agent

JVM package com.example.spring2gx.agent;

public class ImportantLogClassTransformer implements ClassFileTransformer {

public byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) throws IllegalClassFormatException { //TODO return null; }}

Code for Java agent

public byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) throws IllegalClassFormatException { System.out.println(" Loading class: " + className); return null; }}

Code for Java agent

Starting the agent Loading class: java/lang/invoke/MethodHandleImpl Loading class: java/lang/invoke/MemberName$Factory Loading class: java/lang/invoke/LambdaForm$NamedFunction Loading class: java/lang/invoke/MethodType$ConcurrentWeakInternSet Loading class: java/lang/invoke/MethodHandleStatics Loading class: java/lang/invoke/MethodHandleStatics$1 Loading class: java/lang/invoke/MethodTypeForm Loading class: java/lang/invoke/Invokers Loading class: java/lang/invoke/MethodType$ConcurrentWeakInternSet$WeakEntry Loading class: java/lang/Void Loading class: java/lang/IllegalAccessException Loading class: sun/misc/PostVMInitHook Loading class: sun/launcher/LauncherHelper Loading class: java/util/concurrent/ConcurrentHashMap$ForwardingNode Loading class: sun/misc/URLClassPath$FileLoader$1 Loading class: com/example/spring2gx/mains/BankTransactions

Code for Java agent

public byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) throws IllegalClassFormatException { // manipulate the bytes here return null; }}

Code for Java agent

public byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) throws IllegalClassFormatException { // manipulate the bytes here return modified bytes; }}

Bytecode Manipulation Frameworks

• ASM: http://asm.ow2.org/

• BCEL: http://commons.apache.org/proper/commons-bcel/

• CGLib: https://github.com/cglib/cglib

• Javassist: http://www.csg.ci.i.u-tokyo.ac.jp/~chiba/javassist/

• Serp: http://serp.sourceforge.net/

• Cojen: https://github.com/cojen/Cojen/wiki

• AspectJ: http://www.eclipse.org/aspectj/

Bytecode Manipulation Frameworks• ASM: http://asm.ow2.org/

• BCEL: http://commons.apache.org/proper/commons-bcel/

• CGLib: https://github.com/cglib/cglib

• Javassist: http://www.csg.ci.i.u-tokyo.ac.jp/~chiba/javassist/

• Serp: http://serp.sourceforge.net/

• Cojen: https://github.com/cojen/Cojen/wiki

• AspectJ: http://www.eclipse.org/aspectj/69

• Java Programming Assistant

• Subproject of JBoss

• Object based API

• Actively updated

• Version 3.18.2 released May 2014

• Source code: https://github.com/jboss-javassist/javassist

• Docs: http://www.csg.ci.i.u-tokyo.ac.jp/~chiba/javassist/tutorial/tutorial.html

Javassist

CtClass:compile time classCtClass Name

BankTransactions

Javassist

CtClass:compile time classCtClass Name

Bank TransactionsBankTransactions

Javassist

CtClass:compile time class

ClassPool: container of CtClass objects

Bank Transactions

Javassist

CtClass:compile time classClass Name

Bank Transactions

Javassist

CtClass:compile time classClass Name

Bank TransactionsBankTransactions

Javassist

Java Class: Bank Transactions

Javassist

CtClass

Fields

Constructors

Methods

Javassist

CtClass

Fields

Constructors

Methods

CtClass

Javassist

CtClass

Fields

Constructors

Methods

CtClass

CtFields

CtConstructors

CtMethods

Javassist

CtClass

CtField

ClassPool

CtClass

CtConst

CtMethod

Javassist

CtMethod

Javassist

line 1 of code

line 2 of code

line 3 of code

line 4 of code

line 5of code

CtMethod

Javassist

line 1 of code

line 2 of code

line 3 of code

line 4 of code

line 5 of code

putMethod.insertBefore(“System.out.println(\“before method\”);”);

System.out.println(“before method”);

CtMethod

Javassist

line 2 of code

line 3 of code

line 4 of code

line 5 of code

System.out.println(“after method”);

line 1 of code

putMethod.insertAfter(“System.out.println(\”after method\”);”);

CtMethod

Javassist

line 2 of code

line 3 of code

line 4 of code

catch (IOException e) { Sys.out.prln( . . .

line 5 of code

line 1 of code

CtClass exceptionType = classpool.get(“java.io.IOException”);putMethod.addCatch(“{System.out.println($e); throw $e}”, exceptionType);

CtMethod

Javassist

line 2 of code

System.out.println(“middle method”);

line 3 of code

line 4 of code

line 5 of code

line 1 of code

putMethod.insertAt(3, true, “System.out.println(\“middle method\”);”);

CtMethod

Javassist

CtClass

CtField

ClassPool

CtClass

CtConst

CtMethod

insertBefore

insertAfter

catchEx

insertAt

public static void premain(String args, Instrumentation inst) { System.out.println("Starting the agent"); inst.addTransformer(new ImportantLogClassTransformer()); }

Code for Java agent

public byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] cfbuffer) throws IllegalClassFormatException { 1. convert byte array to a ct class object 2. check each method of ct class for annotation @ImportantLog 3. if @ImportantLog annotation is present on method, then a. get important method parameter indexes

b. add logging statement to beginning of the method return null;}Wednesday, September 10, 14

public byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] cfbuffer) throws IllegalClassFormatException { pool.insertClassPath(new ByteArrayClassPath(className,classfileBuffer)); CtClass cclass = pool.get(className.replaceAll("/", ".")); 2. check each method of ct class for annotation @ImportantLog 3. if @ImportantLog annotation present on method, then a. get important method parameter indexes

b. add logging statement to beginning of the method return null;}Wednesday, September 10, 14

public byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] cfbuffer) throws IllegalClassFormatException { pool.insertClassPath(new ByteArrayClassPath(className,classfileBuffer)); CtClass cclass = pool.get(className.replaceAll("/", ".")); if (!cclass.isFrozen()) { for (CtMethod currentMethod : cclass.getDeclaredMethods()) { Annotation annotation = getAnnotation(currentMethod); 3. if @ImportantLog annotation present on method, then a. get important method parameter indexes

b. add logging statement to beginning of the method } } } return null;}Wednesday, September 10, 14

public byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] cfbuffer) throws IllegalClassFormatException { pool.insertClassPath(new ByteArrayClassPath(className,classfileBuffer)); CtClass cclass = pool.get(className.replaceAll("/", ".")); if (!cclass.isFrozen()) { for (CtMethod currentMethod : cclass.getDeclaredMethods()) { Annotation annotation = getAnnotation(currentMethod); if (annotation != null) { List<String> parameterIndexes = getParamIndexes(annotation);

b. add logging statement to beginning of the method } } } return null;}Wednesday, September 10, 14

public byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] cfbuffer) throws IllegalClassFormatException { pool.insertClassPath(new ByteArrayClassPath(className,classfileBuffer)); CtClass cclass = pool.get(className.replaceAll("/", ".")); if (!cclass.isFrozen()) { for (CtMethod currentMethod : cclass.getDeclaredMethods()) { Annotation annotation = getAnnotation(currentMethod); if (annotation != null) { List<String> parameterIndexes = getParamIndexes(annotation); currentMethod.insertBefore(createJavaString( currentMethod, className, parameterIndexes)); } } return cclass.toBytecode(); } return null;}Wednesday, September 10, 14

private Annotation getAnnotation(CtMethod method) { MethodInfo mInfo = method.getMethodInfo(); // the attribute we are looking for is a runtime invisible attribute // use Retention(RetentionPolicy.RUNTIME) on the annotation to make it // visible at runtime AnnotationsAttribute attInfo = (AnnotationsAttribute) mInfo .getAttribute(AnnotationsAttribute.invisibleTag); if (attInfo != null) { // this is the type name meaning use dots instead of slashes return attInfo.getAnnotation("com.example.spring.mains.ImportantLog"); } return null; }

private List<String> getParamIndexes(Annotation annotation) { ArrayMemberValue fields = (ArrayMemberValue) annotation .getMemberValue(“fields”); if (fields != null) { MemberValue[] values = (MemberValue[]) fields.getValue(); List<String> parameterIndexes = new ArrayList<String>(); for (MemberValue val : values) { parameterIndexes.add(((StringMemberValue) val).getValue()); } return parameterIndexes; } return Collections.emptyList(); }

private String createJavaString(CtMethod currentMethod, String className, List<String> indexParameters) { StringBuilder sb = new StringBuilder(); sb.append("{StringBuilder sb = new StringBuilder"); sb.append("(\"A call was made to method '\");"); sb.append("sb.append(\""); sb.append(currentMethod.getName()); sb.append("\");sb.append(\"' on class '\");"); sb.append("sb.append(\""); sb.append(className); sb.append("\");sb.append(\"'.\");"); sb.append("sb.append(\"\\n Important params:\");"); . . .

Put multiple statements inside brackets {}

private String createJavaString(CtMethod currentMethod, String className, List<String> indexParameters) { . . . for (String index : indexParameters) { try { int localVar = Integer.parseInt(index) + 1; sb.append("sb.append(\"\\n Index: \");"); sb.append("sb.append(\""); sb.append(index); sb.append("\");sb.append(\" value: \");"); sb.append("sb.append($" + localVar + ");"); } catch (NumberFormatException e) { e.printStackTrace(); } } sb.append("System.out.println(sb.toString());}"); return sb.toString();}

private String createJavaString(CtMethod currentMethod, String className, List<String> indexParameters) { . . . for (String index : indexParameters) { try { int localVar = Integer.parseInt(index) + 1; sb.append("sb.append(\"\\n Index: \");"); sb.append("sb.append(\""); sb.append(index); sb.append("\");sb.append(\" value: \");"); sb.append("sb.append($" + localVar + ");"); } catch (NumberFormatException e) { e.printStackTrace(); } } sb.append("System.out.println(sb.toString());}"); return sb.toString();} $0, $1, $2, ... used to access “this” and method parametersWednesday, September 10, 14

• Positives

• Simplicity

• Documentation

• Negatives

• Limited functionality

• Slow

Javassist

• Released in Open Source in 2002

• Actively updated

• Version 5.0.3 released May 2014

• Two ASM libraries

• Event based (SAX like)

• Object based (DOM like)

• Documentation: http://download.forge.objectweb.org/asm/asm4-guide.pdf

Source: http://download.forge.objectweb.org/asm/asm4-guide.pdfWednesday, September 10, 14

ClassReader: event producer

ClassWriter: event consumer

ClassVisitor: event filter

BankTrans

ClassReader: given a byte[], parses a compiled class

BankTrans

ClassWriter: produces output byte[]

BankTrans

ClassVisitor: delegates class events, event filter

BankTrans

visitField

visitMethod

BankTrans

import java.lang.instrument.Instrumentation;

public static void premain(String args, Instrumentation inst) { System.out.println("Starting the agent"); inst.addTransformer(new ImportantLogClassTransformer()); }

public byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) throws IllegalClassFormatException {

ClassReader cr = new ClassReader(classfileBuffer); ClassWriter cw = new ClassWriter(cr, ClassWriter.COMPUTE_FRAMES);

cr.accept(cw, 0); return cw.toByteArray();

public byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) throws IllegalClassFormatException {

ClassReader cr = new ClassReader(classfileBuffer); ClassWriter cw = new ClassWriter(cr, ClassWriter.COMPUTE_FRAMES); ClassVisitor cv = new LogMethodClassVisitor(cw, className); cr.accept(cv, 0); return cw.toByteArray();

BankTrans

visitField

visitMethod

BankTrans

BankTransvisitField

visitMethod

BankTrans

LogMethodClassVisitor:event filter

BankTrans

LogMethodClassVisitor:event filter

visitMethod

ASMpublic class LogMethodClassVisitor extends ClassVisitor {

private String className;

public LogMethodIfAnnotationVisitor(ClassVisitor cv, String className) { super(Opcodes.ASM5, cv); this.className = className; }

@Override public MethodVisitor visitMethod(int access, String name, String desc, String signature, String[] exceptions) { // put our logic in here }}

ASMpublic class LogMethodClassVisitor extends ClassVisitor {

private String className;

@Override public MethodVisitor visitMethod(int access, String name, String desc, String signature, String[] exceptions) { // put our logic in here }}

Method Visitor

visitAnnotation

Method Visitor

visitCode

visitEnd

Login Method Login Method

ASMpublic class LogMethodClassVisitor extends ClassVisitor { private String className;

@Override public MethodVisitor visitMethod(int access, String name, String desc, String signature, String[] exceptions) { // put our logic in here }}Wednesday, September 10, 14

ASMpublic class LogMethodClassVisitor extends ClassVisitor { private String className;

@Override public MethodVisitor visitMethod(int access, String name, String desc, String signature, String[] exceptions) { MethodVisitor mv = super.visitMethod(access, name, desc, signature, exceptions); return new PrintMessageMethodVisitor(mv, name, className); }}Wednesday, September 10, 14

ASMpublic class PrintMessageMethodVisitor extends MethodVisitor {

@Override public AnnotationVisitor visitAnnotation(String desc, boolean visible) { }

@Override public void visitCode() { }

@Override public AnnotationVisitor visitAnnotation(String desc, boolean visible) { 1. check method for annotation @ImportantLog 2. if annotation present, then get important method param indexes }

@Override public void visitCode() { 3. if annotation present, add logging to beginning of the method }

ASM public AnnotationVisitor visitAnnotation(String desc, boolean visible) { if ("Lcom/example/spring2gx/mains/ImportantLog;".equals(desc)) { isAnnotationPresent = true; return new AnnotationVisitor(Opcodes.ASM5, super.visitAnnotation(desc, visible)) { public AnnotationVisitor visitArray(String name) { if (“fields”.equals(name)) { return new AnnotationVisitor(Opcodes.ASM5, super.visitArray(name)) { public void visit(String name, Object value) { parameterIndexes.add((String) value); super.visit(name, value); } }; } else { return super.visitArray(name); } } }; } return super.visitAnnotation(desc, visible); }

@Override public AnnotationVisitor visitAnnotation(String desc, boolean visible) { 1. check method for annotation @ImportantLog 2. if annotation present, then get important method param indexes }

@Override public void visitCode() { 3. if annotation present, add logging to beginning of the method }

ASM public void visitCode() { if (isAnnotationPresent) { // create string builder mv.visitFieldInsn(Opcodes.GETSTATIC, "java/lang/System", "out", "Ljava/io/PrintStream;"); mv.visitTypeInsn(Opcodes.NEW, "java/lang/StringBuilder"); mv.visitInsn(Opcodes.DUP); // add everything to the string builder mv.visitLdcInsn("A call was made to method \""); mv.visitMethodInsn(Opcodes.INVOKESPECIAL, "java/lang/StringBuilder", "<init>", "(Ljava/lang/String;)V", false); mv.visitLdcInsn(methodName); mv.visitMethodInsn(Opcodes.INVOKEVIRTUAL, "java/lang/StringBuilder", "append", "(Ljava/lang/String;)Ljava/lang/StringBuilder;", false);

. . .Wednesday, September 10, 14

Javappublic class PrintMessage { private String methodName; private String className; public PrintMessage(String mName, String cName) { methodName = mName; className = cName; } public void print() { StringBuilder sb = new StringBuilder(); sb.append("A call was made to method \""); sb.append(methodName); sb.append("\" on class \""); sb.append(className); sb.append("\"."); System.out.println(sb.toString()); }

javap -c com/example/spring2gx/mains/PrintMessage

public void print(); Code: 0: new #38 // class java/lang/StringBuilder 3: dup 4: invokespecial #40 // Method java/lang/StringBuilder."<init>":()V 7: astore_1 8: aload_1 9: ldc #41 // String A call was made to method \" 11: invokevirtual #43 // Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder; 14: pop 15: aload_1 16: aload_0 17: getfield #14 // Field methodName:Ljava/lang/String; 20: invokevirtual #43 // Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;. . .

ASM public void visitCode() { if (isAnnotationPresent) { // create string builder mv.visitFieldInsn(Opcodes.GETSTATIC, "java/lang/System", "out", "Ljava/io/PrintStream;"); mv.visitTypeInsn(Opcodes.NEW, "java/lang/StringBuilder"); mv.visitInsn(Opcodes.DUP); // add everything to the string builder mv.visitLdcInsn("A call was made to method \""); mv.visitMethodInsn(Opcodes.INVOKESPECIAL, "java/lang/StringBuilder", "<init>", "(Ljava/lang/String;)V", false); mv.visitLdcInsn(methodName); mv.visitMethodInsn(Opcodes.INVOKEVIRTUAL, "java/lang/StringBuilder", "append", "(Ljava/lang/String;)Ljava/lang/StringBuilder;", false);

. . .Wednesday, September 10, 14

ASM• Positives

• Speed and Size

• Documentation

• Depth of functionality

• Number of people using the framework

• Negatives

• Developer ramp-up

• Writing byte code

Applications

Hibernate

LazyInitializationException

Hibernate

9 instances of “throw new LazyInitializationException”

Hibernate

Name Location Team

Joe Smith Dallas, TX Team 1

Jane Doe Portland, OR

Triathlon Club

Tim Doe San Fran, CA

Running Club

Sue Smith Austin, TX Red Lizards

Name Location Description

Team 1 Dallas, TX Casual runners

Triathlon Club

Beaverton, OR

Swim, bike, run

Running Club

Santa Clara, CA

For fun

Red Lizards Austin, TX Fast

Runner Team

Hibernate

Conference AttendeeRunner

Hibernate

Name: Ashley Puls

Runner

Hibernate

Name: Ashley Puls

Location: Portland, OR

Runner

Hibernate

Name: Ashley Puls

Team: Hibernate Proxy

Name: Ashley Puls

Runner

Hibernate

Name: Ashley Puls

Company:

Name: Ashley Puls

Hibernate Proxy

Name: Triathlon Club

Location: null

Description: null

Name: Ashley Puls

Runner

Hibernate

Name: Ashley Puls

Company:

Name: Ashley Puls

Hibernate Proxy

Description:

Name: Ashley Puls

Runner

Hibernate

Name: Ashley Puls

Company:

Name: Ashley Puls

Hibernate Proxy

Description:

Name: Ashley Puls

Runner

Hibernate

Name: Ashley Puls

Company:

Name: Ashley Puls

Hibernate Proxy

call a getter on the hibernate proxy:runner.getTeam().getLocation();Name: Ashley Puls

Runner

Location: null

Description: null

Hibernate

Name: Ashley Puls

Company:

Name: Ashley Puls

Hibernate Proxy Team

Name: Ashley Puls

Runner

Location: null

Description: null

Location: Beaverton, OR

Description: swim, bike, run

call a getter on the hibernate proxy:runner.getTeam().getLocation();

Hibernate

Name: Ashley Puls

Company:

Name: Ashley Puls

Hibernate Proxy

Description:

Name: Ashley Puls

Runner

Hibernate

Name: Ashley Puls

Company:

Name: Ashley Puls

Hibernate Proxy Team

Location: Beaverton, OR

Description: swim, bike, run

Trying to access uninitialized field outside a session

Name: Ashley Puls

Runner

Location: null

Description: null

Hibernate

package org.hibernate.proxy.pojo.javassist;

/** * A Javassist-based lazy initializer proxy. * * @author Muga Nishizawa */public class JavassistLazyInitializer extends BasicLazyInitializer implements MethodHandler {

Hibernate

public static HibernateProxy getProxy( . . . ) { . . . ProxyFactory factory = new ProxyFactory(); factory.setSuperclass(interfaces.length == 1 ? persistentClass : null); factory.setInterfaces(interfaces); factory.setFilter(FINALIZE_FILTER); Class cl = factory.createClass(); final HibernateProxy proxy = (HibernateProxy) cl.newInstance(); ((Proxy) proxy).setHandler(instance); instance.constructed = true; return proxy; . . . }

Hibernatepublic Object invoke(final Object proxy, final Method thisMethod, final Method proceed, final Object[] args) throws Throwable { if (this.constructed) { Object result = this.invoke(thisMethod, args, proxy); if (result == INVOKE_IMPLEMENTATION) { Object target = getImplementation(); final Object returnValue = thisMethod.invoke(target, args); if (returnValue == target) { if (returnValue.getClass().isInstance(proxy)) { return proxy; } return returnValue; } else { return result; } } else { if (thisMethod.getName().equals("getHibernateLazyInitializer")) { return this; } else { return proceed.invoke(proxy, args); } }}

Hibernate

Hibernate Proxy

Why do we need a proxy?

Name: Ashley Puls

Runner

Hibernate

Name: Ashley Puls

Title: Software Engineer

Company: New Relic

Conference Attendee

1. Is the entity dirty? Have we changed any of the data?

2. What fields on the entity have been loaded?

Name: Ashley Puls

Runner

Hibernate - Build Time Tasks

• Schema Tasks• Schema Export• Schema Update• Schema Validator

• ByteCode Tasks• EnhancementTask• InstrumentTask

Hibernate - Ant

Hibernate - Gradle

apply plugin: 'java'apply plugin: 'maven'apply plugin: 'enhance'buildscript { repositories { mavenCentral() } dependencies { classpath 'org.hibernate:hibernate-gradle-plugin:VERSION' }}dependencies { compile group: 'org.hibernate.javax.persistence', name: 'hibernate-jpa-[SPEC-VERSION]-api', version: '[IMPL-VERSION]' compile group: 'org.hibernate', name: 'hibernate-gradle-plugin', version: 'VERSION'}

Hibernatepublic class EnhancementTask extends Task implements EnhancementContext { private List<FileSet> filesets = new ArrayList<FileSet>(); private final ClassPool classPool = new ClassPool( false ); private final Enhancer enhancer = new Enhancer( this ); @Override public void execute() throws BuildException { for (FileSet fileSet : filesets) { final File fileSetBaseDir = fileSet.getDir(project);

for (String relativeIncludedFileName :scanner.getIncludedFiles()) { final File javaClassFile = new File( fileSetBaseDir, relativeIncludedFileName ); processClassFile( javaClassFile); } } }Wednesday, September 10, 14

Hibernate

/** * Atm only process files annotated with either @Entity or @Embeddable */ private void processClassFile(File javaClassFile) {

final CtClass ctClass = classPool.makeClass( new FileInputStream( javaClassFile ) );

processEntityClassFile(javaClassFile, ctClass); }

Hibernate

private void processEntityClassFile(File javaClassFile, CtClass ctClass ) {

byte[] result = enhancer.enhance(ctClass.getName(),ctClass.toBytecode()); if(result != null) writeEnhancedClass(javaClassFile, result); }

Hibernate

private void enhance(CtClass managedCtClass, boolean isComposite) {

// 1. stop if managedCtClass is an interface// 2. stop if managedCtClass is already enhanced

// 3. stop if managedCtClass is not an entity }

Hibernate

// 3. stop if managedCtClass is not an entity // 4. enhance managedCtClass.addInterface( managedEntityCtClass ); enhancePersistentAttributes( managedCtClass ); addEntityInstanceHandling( managedCtClass ); addEntityEntryHandling( managedCtClass ); addLinkedPreviousHandling( managedCtClass ); addLinkedNextHandling( managedCtClass );}

Hibernate

Hibernateprivate CtField addField(CtClass targetClass, CtClass fieldType, String fieldName, boolean makeTransient) { final ConstPool constPool = targetClass.getClassFile().getConstPool(); final CtField theField = new CtField( fieldType, fieldName, targetClass ); targetClass.addField( theField ); // make that new field (1) private, (2) transient and (3) @Transient if ( makeTransient ) { theField.setModifiers( theField.getModifiers() | Modifier.TRANSIENT ); } theField.setModifiers( Modifier.setPrivate( theField.getModifiers() ) );

final AnnotationsAttribute annotationsAttribute = getVisibleAnnotations(theField.getFieldInfo()); annotationsAttribute.addAnnotation( new Annotation( Transient.class.getName(), constPool ) ); return theField;}Wednesday, September 10, 14

Hibernate

private void addGetter(CtClass targetClass, CtField theField, String getterName) { targetClass.addMethod( CtNewMethod.getter( getterName, theField ) );}

private void addSetter(CtClass targetClass, CtField theField, String setterName) { targetClass.addMethod( CtNewMethod.setter( setterName, theField ) );}

Hibernate

private void addGetter(CtClass targetClass, CtField theField, String getterName) { targetClass.addMethod( CtNewMethod.getter( getterName, theField ) );}

private void addSetter(CtClass targetClass, CtField theField, String setterName) { targetClass.addMethod( CtNewMethod.setter( setterName, theField ) );}

Hibernate

private byte[] getByteCode(CtClass managedCtClass) { final ByteArrayOutputStream byteStream = new ByteArrayOutputStream(); final DataOutputStream out = new DataOutputStream( byteStream ); try { managedCtClass.toBytecode( out ); return byteStream.toByteArray(); } finally { out.close(); }}

Hibernate

private byte[] getByteCode(CtClass managedCtClass) { final ByteArrayOutputStream byteStream = new ByteArrayOutputStream(); final DataOutputStream out = new DataOutputStream( byteStream ); try { managedCtClass.toBytecode( out ); return byteStream.toByteArray(); } finally { out.close(); }}

Hibernate

Hibernate private void writeEnhancedClass(File javaClassFile, byte[] result) {

if ( javaClassFile.delete() ) { if ( ! javaClassFile.createNewFile() ) { log( "Unable to recreate class file [" + javaClassFile.getName() + "]", Project.MSG_INFO ); } }

FileOutputStream outputStream = new FileOutputStream( javaClassFile, false ); try { outputStream.write( result); outputStream.flush(); } finally { outputStream.close(); } }

Spring

https://github.com/scottfrederick/spring-music

Springpackage org.cloudfoundry.samples.music.config.web;

@Configuration@ComponentScan(basePackageClasses = { AlbumController.class })public class WebMvcConfig extends WebMvcConfigurerAdapter {

@Bean public InternalResourceViewResolver internalResourceViewResolver() { InternalResourceViewResolver internalResourceViewResolver = new InternalResourceViewResolver(); internalResourceViewResolver.setPrefix("/WEB-INF/"); . . . Wednesday, September 10, 14

Spring

@Controller@RequestMapping(value = "/albums")public class AlbumController { private static final Logger logger = LoggerFactory.getLogger(AlbumController.class);

private final AlbumRepository repository; private SongsService songs;

Spring

@Controller@RequestMapping(value = "/albums")public class AlbumController { private static final Logger logger = LoggerFactory.getLogger(AlbumController.class);

private final AlbumRepository repository; private SongsService songs;

Spring

@Component: A generic Spring component

@Repository: Class at the persistence layer

@Controller: Class which receives HTTP servlet requests and responses

@Service: Class at the service level

AlbumRepositoryPopulator

MongoAlbumRepository, JpaAlbumRepository

ErrorController, InfoController, AlbumController

Springpackage org.springframework.context.annotation;

/** * A component provider that scans the classpath from a base package. * It then applies exclude and include filters to the resulting * classes to find candidates. * * <p>This implementation is based on Spring's * {@link org.springframework.core.type.classreading.MetadataReader * MetadataReader} * facility, backed by an ASM * {@linkorg.springframework.asm.ClassReader ClassReader}. */public class ClassPathScanningCandidateComponentProvider implements EnvironmentCapable, ResourceLoaderAware {

Spring

public Set<BeanDefinition> findCandidateComponents(String basePackage) { Set<BeanDefinition> candidates = new LinkedHashSet<BeanDefinition>(); String packageSearchPath = ResourcePatternResolver.CLASSPATH_ALL_URL_PREFIX + resolveBasePackage(basePackage) + "/" + resourcePattern; Resource[] resources = resourcePatternResolver.getResources(packageSearchPath); for (Resource resource : resources) { MetadataReader metadataReader = metadataReaderFactory.getMetadataReader(resource); if (isCandidateComponent(metadataReader)) { ScannedGenericBeanDefinition sbd = new ScannedGenericBeanDefinition(metadataReader); sbd.setResource(resource); sbd.setSource(resource); if (isCandidateComponent(sbd)) { candidates.add(sbd); } } } return candidates;}

Spring

public Set<BeanDefinition> findCandidateComponents(String basePackage) { Set<BeanDefinition> candidates = new LinkedHashSet<BeanDefinition>(); String packageSearchPath = ResourcePatternResolver.CLASSPATH_ALL_URL_PREFIX + resolveBasePackage(basePackage) + "/" + resourcePattern; Resource[] resources = resourcePatternResolver.getResources(packageSearchPath); for (Resource resource : resources) { MetadataReader metadataReader = metadataReaderFactory.getMetadataReader(resource); if (isCandidateComponent(metadataReader)) { ScannedGenericBeanDefinition sbd = new ScannedGenericBeanDefinition(metadataReader); sbd.setResource(resource); sbd.setSource(resource); if (isCandidateComponent(sbd)) { candidates.add(sbd); } } } return candidates;}

SpringSimpleMetadataReader

Resource

metadataResource

Spring

package org.springframework.core.type.classreading;

final class SimpleMetadataReader implements MetadataReader {

SimpleMetadataReader(Resource resource, ClassLoader classLoader) throws IOException { InputStream is = new BufferedInputStream(resource.getInputStream()); ClassReader classReader = new ClassReader(is); AnnotationMetadataReadingVisitor visitor = new

AnnotationMetadataReadingVisitor(classLoader); classReader.accept(visitor, ClassReader.SKIP_DEBUG);

this.annotationMetadata = visitor; this.classMetadata = visitor; this.resource = resource; }

Spring

ClassReader

SimpleMetadataReader

metadata

AnnotationMetadataReadingVisitor

Resource

BankTrans

visitField

visitMethod

BankTrans

Springpublic class AnnotationMetadataReadingVisitor extends ClassMetadataReadingVisitor implements AnnotationMetadata {

public MethodVisitor visitMethod(int access, String name, String desc, String signature, String[] exceptions) { if ((access & Opcodes.ACC_BRIDGE) != 0) { return super.visitMethod(access, name, desc, signature, exceptions); } return new MethodMetadataReadingVisitor(name, access, getClassName(), classLoader, methodMetadataSet); }

public AnnotationVisitor visitAnnotation(final String desc, boolean visible) { String className = Type.getType(desc).getClassName(); this.annotationSet.add(className); return new AnnotationAttributesReadingVisitor(className, attributesMap,

metaAnnotationMap, classLoader); }Wednesday, September 10, 14

Spring

ClassReader

metadata

Resource

visitMethod

Spring

ClassReader

metadata

Resource

visitMethod

visitAnnotation

Spring

ClassReader

metadata

Resource

visitMethod

visitAnnotation

visitvisitAnnotation

visitArrayvisitEnum

Springrepackaging ASM in spring-core

Source: http://www.clker.com/clipart-3085.html

Springrepackaging ASM in spring-core

Source: http://www.clker.com/clipart-3085.html

https://jira.spring.io/browse/SPR-10292

Lessons Learned

Source: http://www.mindbodygreen.com/0-4965/How-Baby-Steps-Lead-to-Big-Results.html

Lessons Learned

Source: http://www.dishesandsocks.com/2011/06/16/teacher-classroom-organization-part-1/

Lessons Learned

Source: http://4hdwallpapers.com/transformer-isolated-on-white-background-3d-render.html

Applications

Obfuscate so users can not view your source code

Find bugs

Write your own compiler for a language that runs on the JVM

Add logging to an application

Perform preprocessing of code at build time

Restrict API methods for specific users

Determine if a collection is accessed by multiple threads

Lazy load data from a databaseFind differences between jars

Make bytecode manipulation frameworks your friend. They might save your job one day.

Source: http://www.kulfoto.com/dog-pictures/15799/mans-best-friend

Source: http://alangregerman.typepad.com/.a/6a00d83516c0ad53ef017c3329aba8970b-800wi

Questions?

Living in the Matrix with Bytecode Manipulation (New Relic)

Documents