10th June 2014 UKOUG ScotlandMatt Brasier

Monitoring Oracle SOA Suite

About me

• Head of Consulting at C2B2• 14 years WebLogic experience• 12 in a consultancy role• Author– Oracle SOA Suite 11g Performance Tuning

Cookbook

About C2B2

• Middleware professional services– Consultancy– Support

• Oracle partner• Oracle SOA Suite 12c upgrade campaign

Agenda

• Intro• Why Monitor?• How to Monitor?• What to Monitor?

Introduction

Oracle SOA Suite

• Evolved rather than designed– BPMN– BPEL– Mediator– Rules– Workflow– BAM/B2B/Event Processor

Oracle SOA Suite

• Runs on a stack– Hardware– OS– JVM– WebLogic– SOA Suite

Monitoring

• Monitoring– Capturing of metrics– Visualising of metric trends

• Alerting– Generating a notification when a condition is met• Out of bounds metric

The Java Monitoring world

• Still developing• Lots of tools• Monitoring for deployments of all sizes• Monitoring frameworks vs Alerting

frameworks• Java based frameworks vs frameworks that

can monitor java

Why Monitor?

Five reasons to monitor SOA Suite

• Detect problems early• Capacity planning• Resolve problems faster• Understand your system better• Save money

Detect problems early

• Locate areas of resource contention• Identify unusual workloads• Identify failed components or services• Detect problems in other systems– Middleware is often the best place to start

• Alerting is key– Alerts to the right people at the right time

Capacity planning

• Detect trends in usage• Understand how changes in use will affect

capacity• Increase capacity before it causes a problem– Hardware and upgrade lead times

Resolve problems faster

• Post incident analysis• What resources were at their limits?• What was the system doing before it failed?• What were other systems doing at the time?

• What can we do to alert before failure next time?

Understand your system better

• What does a normal day look like on the system?– Resource usage– Use case load

• Which resources are key to system operation?• What improvements can you make?

Save money

• The bottom line– Less outages– Faster resolutions– Less repeated failures– Target capacity correctly– More knowledgeable and better prepared

administrators

How to Monitor?

Ways to monitor Oracle SOA Suite

• Manual monitoring• Scripts• Monitoring tools• Log scraping tools• Alerting

Manual Monitoring

• Someone sits and looks at a console– Watching log files– Reviewing output from basic tools• jVisualVM• jstat• JRMC• DMS

Scripts

• Collect data from manual tools and store it somewhere using a script– WLST– jstat

• Analyse data in tools such as excel when required

Monitoring Tools

• Usually server/agent based– Install agent on the host to be monitored

• Wide range of tools– Oracle Enterprise Manager Cloud Control– Nagios– ManageEngine– HP OpenView

Monitoring Tools

• Specific vs Generic– Specific tools often provide more out of the box– Generic tools often need some configuration

• Auto discovery vs manual configuration• Where do they store the data?• How much information do they provide– OS– Hardware

Log Scraping Tools

• Splunk is the best known example• Send metrics to a log file• Log scraping tool reads and parses the

metrics• Log scraping server integrates metrics

together and displays them

Monitoring best practices

• Capture key metrics (see later)• Don’t alert on everything you capture• Consider data retention periods• Be open with access to monitoring data

Alerting best practices

• Who is in a position to do something about the alert

• The people who have to deal with it should own the alert– Thresholds– Enabling/disabling– Damping

Enterprise Monitoring

• Architectural options– Generic enterprise monitoring tool– Specialist monitoring tool that sends alerts up

stream– Multiple monitoring tools• Network• Hardware• OS• Middleware

What to Monitor?

What to monitor?

• Hardware• OS• JVM• WebLogic• SOA Suite• Log files

* Indicates alert candidate

Hardware

• Disk usage*• Network IO*• CPU Usage*• Memory Usage

Operating System

• File handles*• User CPU vs System CPU• Modification dates on key files*• Boot time *

Java Virtual Machine

• Garbage Collection*• Memory Usage– Old– New– Perm…

• Threads

WebLogic

• Datasource connection pool size*• JMS Queue/Topic depth*• JMS Consumer count*• Stuck threads*• Work managers

SOA Suite

• Composite response times

Log Files

• Key exception or error types– java.io.IOException– oracle.jdbc.– java.lang.OutOfMemoryError– .– .– .

What Next?

What Next?

• Write a monitoring policy• Review your existing monitoring against it• Update your monitoring• Improve your availability• Resolve issues faster

What Next?

• Follow C2B2 on twitter– @c2b2consulting

• Follow me on twitter– @mbrasier

• View our SOA Suite resources page– http://info.c2b2.co.uk/soa-suite-resources

Questions?