Date post: | 12-May-2015 |
Category: |
Technology |
Upload: | jaxlondonconference |
View: | 469 times |
Download: | 1 times |
© Copyright Waratek 2013
!
Put your Java Apps to Sleep? Find out How… John Ma(hew Holt | JAX London 2013
© Copyright Waratek 2013
John Matthew Holt!• CTO and co-‐Founder of Waratek • Been doing JVM engineering for 10 years • Invented “Java VirtualizaCon” to transform how JVMs operate in virtualized and cloud-‐compuCng environments
• Lead the design of the first Type-‐1 VMM for a JVM
© Copyright Waratek 2013
What will we cover?!• First some background…
– Why virtualize INSIDE the JVM? – Think “VMware for Java”
• PuSng you Java Apps to Sleep… – Why are Java apps Cred? (or restated: why should they be slept?)
– Let your apps nap: “wake-‐on-‐LAN for JVCs”
© Copyright Waratek 2013
First some background…!
• Why virtualize INSIDE the JVM?
© Copyright Waratek 2013
Because…!• Because JVMs make bad bedfellows with HW hypervisors (HV)
• A JVM thwarts HW hypervisor features like: – The GC in a JVM thwarts memory-‐overcommit in a HW HV
– The GC’s heap memory, and the JIT’s permgen/codecache, thwarts ‘transparent page sharing’ (memory de-‐dup) in a HW HV
– Dedicated GC and JIT for EVERY Java applicaCon wastes lots and lots of CPU Cme and memory capacity doing the same thing over, and over, and over, and…
© Copyright Waratek 2013
Today’s JVMs are OLD!• Were designed ~20 years ago, and haven’t changed design since then
• JVM design PREDATES the major 2 trends of last decade: virtualizaCon and cloud-‐compuCng
• Only 2 innovaCons in JVMs in 20 years: – Beder JIT compilers – Concurrent GCs – ….all blindly focused on single-‐tenant performance at the expense of mulC-‐tenant efficiency
© Copyright Waratek 2013
Server Java has problems!• JVMs are grossly inefficient users of CPU and memory
– A JVM assumes it can use all available CPU and memory capacity for itself and its single applicaCon, eg:
– 1 JBoss can consume up to 500MB RSS with no apps – 1 Tomcat can consume up to 250MB RSS with no webapps – Running mulCple JVMs on a server rapidly exhausts physically
memory while CPU uClisaCon averages 5% • >100 different JVM versions makes compliance/support a
NIGHTMARE • Security vulnerabiliCes and compromises are EVERYWHERE
inside Java
© Copyright Waratek 2013
VMWare does not virtualize Java well!
• Realworld stats using VMware: – >80% of Server JVMs have <=2GB of heap memory
• Yet actual applicaCon heap uClisaCon averages <20% – A 2GB JVM heap needs ~3GB of OS RSS memory, which in turn needs ~4GB of VMWare tenant memory • …reducing actual applicaCon memory uClisaCon to <10%
– CPU uClisaCon running Java on Vmware averages <7% • Yet physical memory is exhausted because of running mulCple JVMs
© Copyright Waratek 2013
Java is a compliance nightmare!• Realworld stats:
– 93% of organisaCons are running versions of Java at least 5 years old
– The average organisaCon has 51 disCnct versions of Java installed
– <1% of installed Java is upgraded to the latest version • Recurring enterprise conversaCon:
– QuesCon: “what version of Java do you run?” – Answer: “every version except the latest version!”
© Copyright Waratek 2013
Java is a security nightmare!• Realworld stats:
– In last 12 months alone, >250 CVE alerts issued by US-‐CERT/NIST for Java
– The most prevalent version of Java has 96 “perfect 10” vulnerabiliCes
– “There are virtually no modern versions of Java without any known severe vulnerabiliCes” (source: Bit9)
• CriCcal Security Patch Updates are released so oven that developers and admins can’t keep up
© Copyright Waratek 2013
Virtualization is the answer…!• …but HIGHER up the Server Stack • VirtualizaCon is creeping up the Server Stack already – x86 HW got virtualized in 1999 (VMware et al) – x86 OS got virtualized in 2006 (OpenVZ, LXC, et al) – …now it’s Java’s turn in 2013
• Virtualizing Java means virtualizing at the TOP of the Server Stack, whereas VMware virtualizes at the BOTTOM
© Copyright Waratek 2013
Cost-per-tentant to zero !• The higher up virtualizaCon goes up the Server Stack, the cheaper a “tenant” becomes
• Virtualizing INSIDE the JVM gets close to “zero-‐cost tenants”: – Helloworld in <1MB of tenant memory – Examples.war (Tomcat demo webapps) in <5MB of tenant memory
– …Java becomes CHEAPER than python/ruby/perl when the JVM itself gets virtualized!
© Copyright Waratek 2013
Virtualization == Security!• Virtualizing the JVM makes Server Java secure and compliant – Virtualized Java Apps run as “tenants” in a shared JVM – Tenants run in Java Virtual Container (JVC) “jails” – Tenants are isolated and “locked-‐down” – Tenants can run any Java version
• Doesn’t mader what “Java version” a tenant uses, it gets the latest JVM
• Means a “Java version” becomes just a tenant API for the convenience of developers
• Admins control the REAL version at the JVM level so that tenants (developers) can stop caring about “versions” altogether
© Copyright Waratek 2013
First some background…!
• Think “VMWare for Java”
© Copyright Waratek 2013
A hypervisor INSIDE a JVM!• Waratek has wriden a type-‐1 VMM that runs INSIDE the
Oracle HotSpot and OpenJDK JVMs – >300,000 lines of Java code, <5K lines of C-‐code – Can easily run on any OS/CPU pair (only Linux/x86 supported so
far) – Does EVERYTHING VMware does, but INSIDE the JVM
• Virtual FS • Virtual networks/IPs/NICs • CPU pinning and quotas • Memory overcommit, elasCc memory, memory quotas • I/O QoS, rate-‐limiCng • Etc…
– …all with ZERO code change to Java applicaCons
© Copyright Waratek 2013
A real VMM inside the JVM!
© Copyright Waratek 2013
JVC
Java Code
C/C++ Code
VMWare for Java!
Waratek JVM
java.* APIs
SecurityManager
Waratek Java Hypervisor
Java App
• Waratek virtualizes Java exactly like VMware virtualizes HW – Waratek inserts a “Java
Hypervisor” between the JVM and the Java App + java.* APIs + SecurityManager
– The Java app PLUS java.* APIs and SecurityManager run inside a Java Virtual Container (JVC) “jail”
– For a Java App to go outside of its JVC “jail” it has to “trap” to the Java Hypervisor
JNI Libs
Process
© Copyright Waratek 2013
JVC
Java Code
C/C++ Code
VMWare for Java!
Waratek JVM
java.* APIs
SecurityManager
• Every incoming and outgoing access by a JVC is intercepted and checked by the Java Hypervisor, e.g: – Class loading intercepted and checked – JNI invocaCon intercepted – Parameter and return value checks – File access (delete/copy/move) checks – Network operaCon checks – IO rate-‐limiCng and flow-‐control – CPU Cme control – Memory consumpCon control – Buffer overflow checks – Null pointer checks – Type safety checks – OS access checks – JNI Libraries isolated – …
Waratek Java Hypervisor
Java App
JNI Libs
Process
© Copyright Waratek 2013
Slots into existing VM tools!• You can manage the Java hypervisor exactly like you manage
exisCng HW hypervisors: – SSH CLI – RedHat LibVirt/KVM integraCon (beta) – VMWare vCAC integraCon shortly – OpenStack integraCon shortly
• Plus other management interfaces – HTTP REST API – Local and remote JMX APIs – /proc/PID/* pseudo-‐filesystem for JVM and each JVC – Vmware CloudFoundry integraCon (beta) – RedHat OpenShiv integraCon soon
© Copyright Waratek 2013
A HelloWorld JVC in LibVirt XML!
© Copyright Waratek 2013
JVC
Java Code
C/C++ Code
JVCs make Java secure!
Waratek JVM
java.* APIs
SecurityManager
• If a Java App inside a JVC compromises either/both of java.* APIs or SecurityManager, then only that JVC is effected and NOT the Java Hypervisor or JVM or other JVCs
• JNI Libraries are isolatable in their own OS process
• Thus a compromised JVC cannot:
– Insert malicious code outside its JVC – Delete/move/copy files outside its VROOT – Trigger buffer overflows – Do null-‐pointer segmentaCon faults – Perform unsafe heap memory access – Fork OS processes – etc…
Waratek Java Hypervisor
Java App
JNI Libs
Process
© Copyright Waratek 2013
“Come in, we’re open”!• The Java Hypervisor has an open API for unlimited customizaCon and extensibility
• Anyone can write their own PURE-‐JAVA plugins to the Java hypervisor to customize ANYTHING about JVCs, eg: – Make a DropBox.com account the virtual filesystem for a JVC
– Create a virtual LAN between distributed JVCs anywhere in the world
– Clone live network I/O of an acCve JVC to a test JVC for side-‐by-‐side applicaCon tesCng
© Copyright Waratek 2013
Example hypervisor plugin!• Waratek wrote ElasCcat as one example of a Java Hypervisor plugin
• ElasCcat is open source, is <1000 lines of Java code, and loads into the Java hypervisor at startup
• ElasCcat virtualizes Apache Tomcat so it becomes the world’s first mulC-‐tenant JavaEE container: – Tomcat runs in one JVC and each webapp runs in it’s own dedicated JVC
– If one webapp crashes it won’t crash any other JVC – …all this achieved with ZERO code change to Tomcat or webapps, and without them knowing they’ve been virtualized!
© Copyright Waratek 2013
© Copyright Waratek 2013
Putting Java to jSleep…!
• Why are Java apps Cred? (or: why should they be slept?)
© Copyright Waratek 2013
Java Apps are tired!!• Realworld stats:
– >50% of PaaS apps are “dead bone idle” at any given Cme
– The average Dev/Test environment runs at <10% sustained uClizaCon
– Lots and lots of enterprise Java apps are only used periodically, e.g. end-‐of-‐week, end-‐of-‐month…
• But these apps have SLAs and can’t be shut-‐off!
© Copyright Waratek 2013
Fact from the field:!• So frustrated was one Fortune500 admin that he shut-‐off a bunch of Java apps that had been idle for 2 weeks (no TCP traffic) and waited to see what would happen… the results were: – …a small number of app-‐owners noCced immediately (1-‐2 days)
– …the largest number of app-‐owners noCced within 1-‐2 weeks
– …and a small number of app-‐owners NEVER noCced at all!
© Copyright Waratek 2013
Wouldn’t it be great if…!• …you could “compress” an idle app and re-‐awaken it instantly WITHOUT THE APP KNOWING?
• …and without the END-‐POINTS knowing? • …and without ever RESTARTING the app later?
© Copyright Waratek 2013
Actually it exists…almost!!• The technology for this nearly already exists:
– It’s called “wake-‐on-‐LAN” (WoL)
• …but there’s a problem: WoL only works at the HARDWARE level, so is of NO USE in mulCtenant virtualized cloud-‐compuCng environments
• Think about it: – As long as ONE tenant on a server is acCve then the server HAS to stay alive and WoL CANNOT be used!
© Copyright Waratek 2013
Putting Java to jSleep…!
• Let your apps nap: wake-‐on-‐LAN for JVCs
© Copyright Waratek 2013
“Wake-on-LAN for JVCs”!• What’s needed is a NEW kind of WoL that “naps” single idle tenants on a mulCtenant server…
• …and reclaims the memory/CPU/IO quotas that were provisioned for the now-‐napped app, to be used to run other new apps
• This is what Waratek jSleep does for Java apps!
© Copyright Waratek 2013
jSleep in a Slide!• Waratek jSleep works inside the Java Hypervisor to do four things: – 1) monitor the acCvity of individual apps (JVCs) – 2) put idle JVCs to sleep aver defined inacCvity thresholds and reclaim their CPU/mem/IO quotas
– 3) monitor the I/O connecCons (TCP/UDP Sockets, ServerSockets, Files, etc) of a sleeping JVC
– 4) and awaken a sleeping JVC when I/O acCvity is detected, reinstaCng reclaimed quotas
• …it’s wake-‐on-‐LAN for JVCs!
© Copyright Waratek 2013
jSleep is Very Powerful!• Lots of “knobs and dials” make it possible to define sophisCcated sleep/awake profiles that respond to: – CPU acCvity – Network/IO acCvity – IO Cmeouts – Wall-‐clock Cme – …or just about anything else
• THERE’S ALSO a built-‐in network profile that allows for immediate configuraCon via the JIRSH CLI
© Copyright Waratek 2013
Simple jSleep Example!• Set a jSleep policy based on network traffic of defined
sockets/ports, eg: – jirsh # sleepinfo 1 –-set network=120000,include_local=:8080,10.1.0.8:4444
– Above command: • Suspends JVC 1 aver the network interface is idle 120s • The include_local param is a series of host:port combinaCons that specify what traffic of remote hosts and/or local ports is allowed to wake the JVC, so in above:
– Traffic from any remote host to local port 8080 will wake the JVC – Traffic from remote host 10.1.0.8 to local port 4444 will wake the JVC
© Copyright Waratek 2013
jSleep API in 60 Seconds!• 2 packages, 8 classes:
– cloudvm.jsleep.event.* • BytesReceivedEvent • CpuEvent • NetworkIdleEvent • TimeoutEvent
– cloudvm.jsleep.profile.* • JSleepProfile • JSleepProfile.MemoryType • JSleepProfileState • NetworkProfile
© Copyright Waratek 2013
jSleep API in 60 Seconds!• Main class: cloudvm.jsleep.profile.JSleepProfile
– Every JVC can have one of these – Defines the two CORE methods:
• checkSuspend( VirtualContainerMXBean ) • checkResume( VirtualContainerMXBean )
– Is an abstract class so anyone can write their own custom sleep/resume profile
– Loaded and configured using JMX (local or remote)
© Copyright Waratek 2013
Custom jSleep API Example!• Here’s a custom jSleep profile…
public boolean checkSuspend(VirtualContainerMXBean paramVirtualContainerMXBean) { Date currentDate = new Date(); if(currentDate.aver(sleepDate) && currentDate.before(resumeDate)) { return true; } else { return false; } } public boolean checkResume(VirtualContainerMXBean paramVirtualContainerMXBean) { Date currentDate = new Date(); if(currentDate.before(sleepDate) || currentDate.aver(resumeDate)) { return true; } else { return false; } }
© Copyright Waratek 2013
What happens when…!• …a JVC sleeps?:
– The threads of the JVC are suspended on the OS kernel (but NOT destroyed)
– The JVC’s dead objects are asynchronously reclaimed by the GC
– If the JSleepProfile was configured with MemoryType.VARIABLE, the JVCs unused memory quota will be reclaimed for use by other JVCs
– All open file descriptors will be kept open – …all this in <1 second
© Copyright Waratek 2013
What happens when…!• …a JVC wakes up?:
– The threads of the JVC are resumed on the OS kernel
– If the JSleepProfile was configured with MemoryType.VARIABLE, the JVM will reinstate the prior memory quota of the JVC
– Any file descriptors with received but unread data will now be “readable” by the applicaCon
– …all this in <1 second
© Copyright Waratek 2013
Why you should care…!• Dev/Test, PaaS, and middleware environments are
characterized by LOTS of “applicaCon idleness” • Idle apps today are treated NO DIFFERENTLY than acCve,
busy apps • Years ago H/W wake-‐on-‐LAN was an effecCve technology for
powering down idle servers… but it no longer works in mulCtenant cloud infrastructure
• jSleep is “wake-‐on-‐LAN for JVCs” and can increase the number of Java apps-‐per-‐server by up to 10x – …this means a reducCon in server sprawl and OpEx up to 90%
© Copyright Waratek 2013
Summary!
• Recap and links…
© Copyright Waratek 2013
Biggest change to Java in yrs!• A hypervisor inside the JVM transforms Java with: – Ultra-‐liteweight JVCs for near zero-‐cost tenants – JVC “jails” for Java security and isolaCon – Mixed-‐version JVCs for version consolidaCon and legacy compliance
– jSleep (“Wake-‐On-‐LAN for JVCs”) for near-‐zero footprint of idle tenants
© Copyright Waratek 2013
Links!• General informaCon about the Waratek JVM
– hdp://www.waratek.com
• jSleep, JVI (Java VirtualizaCon Interface), and JMX APIs – hdp://javadoc.waratek.com
• ElasCcat source code – hdps://github.com/waratek/elasCcat-‐driver
© Copyright Waratek 2013
!