1 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Insert Information Protection Policy Classification from Slide 8
2 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Insert Information Protection Policy Classification from Slide 8
Java in the Database – The Sky’s the Limit: Lessons
from the Largest Deployment Kuassi Mensah
Group Product Manager
Oracle Corporation
Rune Lilleng
Database Manager
Norwegian Labor and Welfare Admin..
3 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
4 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Latin America 2011 December 6–8, 2011
Tokyo 2012 April 4–6, 2012
5 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Oracle OpenWorld Bookstore
• Visit the Oracle OpenWorld Bookstore for a fabulous
selection of books on many of the conference topics
and more!
• Bookstore located at Moscone West, Level 2
• All Books at 20% Discount
6 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Who is Kuassi Mensah …
• Group Product Manager for Database Access Frameworks (Net
Services, DRCP, etc) and database APIs (Java, C/C++, PHP, Ruby,
Python, Perl).
• Hold a MS in Computer Sciences from the Programming Institute
(University of Paris VI).
• Has published several articles and a book @
http://www.amazon.com/exec/obidos/ASIN/1555583296
• Is a frequent speaker at Oracle and IT events and maintain a blog @
http://db360.blogspot.com, as well as Facebook, Linkedin, and Twitter
(http://twitter.com/kmensah) pages.
7 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Who is Rune Lilleng …
• Database Manager @ Norwegian Labour and Welfare
Administration (NAV)
• Worked with Oracle products for 11 years
– Database
– Weblogic
– Exadata
• NAV’s IT department
– 700 employees (IT Operations and IT Development)
8 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Program Agenda
• Why Java in the database
• Architecture & best practices
• Use Cases
• Lessons from the largest deployment
• Q/A
9 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Insert Information Protection Policy Classification from Slide 8
Java in the Database
Why?
10 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Java
JDBC Calls
SQL
Data Shipping/Caching vs Function Shipping
Java
Multiple Roundtrips
Java SE/EE in Mid-Tier
SQL
Java in the Database
Up to 10 times faster!
Java + any language
1 Roundtrip
11 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Rationales for Java in the Database
• Why Java?
– #1 programming language (TIOBE index)
• Java VM
– Ensures Java Portability
– Not just Java, any language that compiles to Java byte codes
• http://www.is-research.de/info/vmlanguages/
• Why in the database
– In-place data processing
– Java methods as triggers and stored procedures
– Ability to extend database capabilities with Java libraries
12 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Decide When to Adopt Java in the Database
• Java EE belongs to Middle-tier
• Java SE with little/no SQL works better in the mid-tier
• Java SE with SQL (JDBC) works better/faster in the
database
– Avoid the round trip cost for massive data processing
– Extreme Scalability: system and user defined application classes
shared across the instance
– Relatively simple to move code from client to database and back
13 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Why Not PL/SQL
• PL/SQL is better for direct manipulation of SQL types
• Java more effective for algorithmic (data logic)
• There are more Java developers
• Java has superior built-in functionalities
• Java has a wide range of libraries
• There are things you cannot do (easily) in PL/SQL
14 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Insert Information Protection Policy Classification from Slide 8
Java in the Database
Architecture & Best Practices
15 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Tight Integration with the Database The Sky is the limit
UGA
Newspace Newspace Newspace
Oldspace Oldspace Oldspace
Class Objmems
UGA UGA Sessionspace Sessionspace Sessionspace
Internspace
PGA
SGA
16 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Specialized Heaps
• Newspace & Oldspace
– GC’d space
– Holds Java objects and most internal VM structures
– All Java objects are allocated from here
• Sessionspace
– Database session duration: objects which survive call boundary
• SGA class objmems
– Shared heaps for each loaded class
– Lifetime tied to duration for which class is in use
• Other internal heaps - UGA class objmems
– Call/session duration unscanned spaces
– Intern space
17 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
JIT Compilation
• Automatic upon class loading
• Data dictionary views allow monitoring methods that
have been compiled SQL> select method_name, method_index, is_compiled from user_java_methods
2 where name = 'java/lang/String' and method_name = 'substring';
METHOD_NAME METHOD_INDEX IS_
---------------- ------------ ---
substring 48 YES
substring 49 YES
18 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Manual Compilation
Shouldn’t be necessary, but …
SQL> select dbms_java.compile_method('Test','foo','()V') from dual;
DBMS_JAVA.COMPILE_METHOD('TEST','FOO','()V')
--------------------------------------------
1
Total of 1 methods compiled in 25 ms.
SQL> select method_name, is_compiled from user_java_methods
where name='Test';
METHOD_NAME IS_
---------------- ---
<init> NO
foo YES
19 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Memory Management
• It’s better to handle an OutOfMemoryError than to get an ORA-
exception that can’t be handled in Java
• Find a good value for max oldspace size through experimentation
• To get or set the oldspace memory limit equivalent to –Xmx <value>
Static long oracle.aurora.OracleRuntime.getMaxMemorySize()
Static void oracle.aurora.OracleRuntime.setMaxMemorySize(long value)
If it’s huge it may mask a leak, but if it is too small you will get out of
memory errors.
20 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Dealing With Out of Memory Errors
import oracle.aurora.vm.OracleRuntime; ...
while (keepGoing) {
try {
run();
}
catch (OutOfMemoryError e) {
long currentMax = OracleRuntime.getMaxMemorySize();
resetAndCleanup();
if (currentMax < MAX_MEMORY_THRESHOLD)
OracleRuntime.setMaxMemorySize(currentMax +
MAX_MEMORY_INCREMENT);
else
keepGoing = false;
}
}
21 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Monitoring Memory Usage through JMX
• Built-in MBeans monitor each heap area
22 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Monitoring Memory Usage through JMX
• Set notification thresholds, per heap
23 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Monitoring Memory Usage through JMX
• Monitor finer-grained GC statistics
24 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Monitoring through the Data Dictionary SQL> select s.sid, n.name p_name, st.value
from v$session s, v$sesstat st, v$statname n
where s.sid=102 and s.sid=st.sid and n.statistic# = st.statistic# and n.name like
'java%';
SID P_NAME VALUE
--- ---------------------------------------- ----------
102 java call heap total size 6815744
102 java call heap total size max 6815744
102 java call heap used size 668904
102 java call heap used size max 846920
102 java call heap live size 667112
102 java call heap live size max 704312
102 java call heap object count 13959
102 java call heap object count max 17173
102 java call heap live object count 13907
102 java call heap live object count max 14916
102 java call heap gc count 432433
102 java call heap collected count 123196423
25 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Memory Usage Best Practices
• Configure appropriate heap size via: oracle.aurora.vm.OracleRuntime.setMaxMemorySize(long)
• Avoid retaining unnecessary state across calls
• Avoid very very large contiguous allocations
• Avoid repeated use of user-defined classloaders
• Typical Java best-practices
– Use finalizers sparingly
– Avoid System.gc()
26 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Insert Information Protection Policy Classification from Slide 8
Java in the Database
Use Cases
27 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Java in the Database: What for?
• Portable Data Logic
• Custom Alert applications that monitor business data
• Parsers for various File Formats (txt, zip, xml, binary)
• Custom Md5 CRC
• Trigger-based Notification System using RMI
• Sending emails with attachment from within the database
• Execute external OS commands and external procedures
• Implement image transformation and
format conversion (GIF, PNG, JPEG,
etc)
• Implement database-resident CMS
• HTTP Call-Out
• JDBC Call-Out to Non-Oracle RDBMS
• RMI Call-Out to SAP
• Web Services Call-Out
• RESTful Database Web Services
• Lucene Domain Index
28 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
JDBC Callout to Non-Oracle RDBMS
29 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
SAP Callout
30 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Database as Web Services Consumer
31 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
32 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Java in the Database Summary
• Business Logic Runs Directly in the Database and
Process Data In Situ
– Code Shipping in Cheaper/Faster than Data Shipping
– Eliminates Network Roundtrip
– Less Moving Parts
• Perfect for Data-Intensive Problems
• Good for Data-and-Compute intensive
33 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Insert Information Protection Policy Classification from Slide 8
Java in the Database
Lessons from the largest
deployment
34 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Agenda
• About NAV
• About ARENA
– History
– Technical Architecture
– System Architecture
– Experiences
• Q & A
35 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
About
• Rune Lilleng • Database Manager @ NAV
– Oracle, DB2, MS SQL – 9 DBA
• Worked with Oracle products for 11 years – Database – Weblogic – Exadata
• NAV’s IT department – 700 employees (IT Operations and IT Development)
36 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
About NAV
• The Norwegian Labour and Welfare Administration (NAV) was established on 1 July 2006.
• Handles a broad range of state services – Pension services – Sickness benefits – Unemployment benefits – Occupational rehabilitation – Social Services
• 450 local offices all across Norway – 20.000 Employees
• Administers one third of the national budget – 300+ billion NOK
• Has the entire population as its users
37 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
About ARENA
– Development started in late 1999 and the system was delivered in 2003.
– It replaced several old mainframe systems, and at the time it’s main functions were related
to unemployment and work related services.
– 2003: 1500 concurrent users, estimated and tested for a maximum load of 2500 concurrent
users. Processed about 14 Billion NOK in benefits.
– 2006: Aetat (work) and RTV (pensions and social security) merger into NAV. HW upgrades
and application changes made to handle 5500 concurrent users.
– 2010: Social reform, and system changes made to handle a new major benefit (AAP) and
potentially 3000 new users.
– 2011: Close to 8000 concurrent users, processing 75 Billion NOK.
38 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
System Architecture
39 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Technical Architecture
– Oracle database 11g for all data and as JAVA VM for the system’s “beans framework”
– OAS 10g for the runtime environment.
– User Interface is developed in Oracle Developer/Forms Builder.
40 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Technical Architecture Oracle Application Server 10g Oracle Forms PL/SQL
Workbench GUI
Validation logic
Business logic
Presentation
Presentation-
logic
Oracle database 11g EE PL/SQL JAVA/XML
Frame Solution
Beans Facade Managers
Workflow Processes
Organization Rules
Document Process Data
JAVA
Business Data
Business Logic Batch
Batch $
Oracle Weblogic
11g
Integration Components
41 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Experiences
- Highly Scalable
- From 1500 concurrent users (2003) to 8000 (2011)
- Application performance remains unchanged
- Added more CPU cores and memory over the years (12 to 46)
- Highly Reliable
- Prior to 11g upgrade < 1h unplanned downtime due to non human errors (eg. bugs)
- 11g
- Issues with mutex contention
- Application accesses certain objects (PL/SQL packages JAVA) very frequently which again causes a high
degree of mutex contention.
- Oracle Forms
- Decreased in popularity, difficult to recruit
- Memory hungry on the database server, one connection per users
- Looking into connection pooling options
42 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Future
- Today this runs on an HP Superdome
- Was about to be replaced with a new one due to age and high support costs, but that is no longer a viable
option.
- Migration project is planned for 2012, and Oracle Exadata is an alternative.
- Done some tests already, on one of our X2-2 machines, but not full scale.
- Forms
- Although the application is closing on 12 years, it’s projected to live another 10 to 15 years
- Would like to migrate away from Forms, to eg. Weblogic
43 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Insert Information Protection Policy Classification from Slide 8
Q&A
44 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
For More Information
search.oracle.com
or oracle.com/technetwork/database/enterprise-edition/index-098579.html
Oracle Net Services
45 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
46 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
The preceding is intended to outline our general product direction.
It is intended for information purposes only, and may not be
incorporated into any contract. It is not a commitment to deliver
any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and
timing of any features or functionality described for Oracle’s
products remains at the sole discretion of Oracle.
47 Copyright © 2011, Oracle and/or its affiliates. All rights
reserved.
Insert Information Protection Policy Classification from Slide 8