Syer Monitoring Integration And Batch

Chicago, October 19 - 22, 2010Chicago, October 19 - 22, 2010

Deploying and Monitoring Spring Integration and Spring Batch Applications

Dave Syer - SpringSource

SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.

Agenda

• Spring Batch and Spring Integration• Spring and JMX• Spring Integration

– Metrics– Notifications and alerts– Control

• Metametrics and Metaoperations: e.g. deployment and provisioning• Spring Batch

– Control– Metrics– Notifications and alerts

• Using SpringSource Hyperic• More Future Directions


Spring Batch

Application

Batch Core

Batch Infrastructure

Re-usable low level stuff: flat files, XML files, database keys

Quality of service, auditability, management information, Job, Step, status

Business logic


Job Configuration

<job><step id="load" next="analyse"/>

<step id="analyse"><next on="COMPLETED" to="summarize"/><next on="SKIPS" to="logSkips"/><end on="FAILED"/>

</step>

<step id="logSkips" next="summarize"/><step id="summarize" />

</job>

<job><step id="load" next="analyse"/>

<step id="analyse"><next on="COMPLETED" to="summarize"/><next on="SKIPS" to="logSkips"/><end on="FAILED"/>

</step>

<step id="logSkips" next="summarize"/><step id="summarize" />

</job>


Quality of Service

• Stuff happens: – Item fails– Job fails

• Failures can be– Transient – try again and see if you succeed– Skippable – ignore it and maybe come back to it later– Fatal – need manual intervention

• Mark a job execution as FAILED• When it restarts, pick up where you left off• Monitor status of running and historic executions• All framework concerns: not business logic


Batch Domain Diagram

Job

Steplocates

Infrastructure dependencies

executes

JobLauncher

starts and stops

JobParameters

JobExecution

StepExecution*

JobRepository

stored in

used to identify and manage jobs

executes

BatchOperator

uses

Application Developer

configures


Item-Oriented Processing

• Input-output can be grouped together = Item-Oriented Processing

ItemReader

read()

ExitStatus

ItemWriter

write(item)

Step

item

execute()

repeat, retry, etc.


Demo


Spring Integration

Application

Integration Adapters

Integration Core

Enterprise Integration Patterns, message, poller, router, splitter, aggregator, etc.

File, JMS, JDBC, JMX, TCP, etc.

Business logic


Message

• Payload can be any object

• Header values are stored in a Map

Headers

Payload


Message Channel

• Decouples Producers from Consumers

• Provides extension point for interceptors• May be Point-to-Point

• Or Publish/Subscribe

Producer

Consumer


Low-level Components

MessageSource

MessageHandler

Endpoint• Passive• Poller• Lifecycle

MessageChannel


The Big Picture


Demo


Spring Batch and Spring Integration

• Message triggers Job• Step wraps gateway (e.g. background processing)• Remote chunking• Partitioning• Automatic retry• Flat file and XML processing• QoS: once only processing, restart• Audit: track status of execution

• http://www.springsource.org/spring-batch-admin/spring-batch-integration


JMX

Application

Client

MBeanServer

MBean MBean MBean

Connector Connector

Client


JMX

• MBeanServer provides services to MBeans and clients (through connectors)– Registration (ObjectName -> MBean)– Attributes– Operations– Notifications– Monitors (thresholds, value changes etc.)


Spring and JMX

@ManagedResourcepublic class TransferServiceImpl implements TransferSe rvice {

@ManagedOperationpublic void transfer(String source, String target, do uble amount) {

…}

@ManagedAttributepublic double getTotalBalance() {

…}

}

@ManagedResourcepublic class TransferServiceImpl implements TransferService {

@ManagedOperationpublic void transfer(String source, String target, double amount) {

…}

@ManagedAttributepublic double getTotalBalance() {

…}

}


Spring and JMX

<context:mbean-server id="mbeanServer" />

<context:mbean-export server="mbeanServer" />

<bean id="transferService" class="com.foo.TransferSe rviceImpl"/>

<context:mbean-server id="mbeanServer" />

<context:mbean-export server= "mbeanServer" />

<bean id="transferService" class= "com.foo.TransferServiceImpl" />

MBean

com.foo:type=TransferServiceImpl,name=transferService

ObjectName

Automatically unregistered whenApplicationContext closes


Spring 3.0 and JMX: Hyperic Metrics

@ManagedResourcepublic class TransferServiceImpl implements TransferSe rvice {

…

@ManagedMetric(metricType = MetricType.GAUGE)public double getTotalBalance() {

…}

}

<context:mbean-export server="mbeanServer" default-

domain="spring.application"/>

@ManagedResourcepublic class TransferServiceImpl implements TransferService {

…

@ManagedMetric( metricType = MetricType. GAUGE)public double getTotalBalance() {

…}

}

<context:mbean-export server= "mbeanServer" default-

domain= "spring.application" />


Spring Integration Metric Use Cases

• How many messages have been processed?

• How long did it take?

• Are things getting any worse?

• Were there any errors?

• Can I shutdown the system now without losing any messages?


Spring Integration Metrics

• MessageChannel– send count, receive count (if polling)– execution times– error rates and ratios

• MessageHandler– handle count– execution times– error rates

• MessageSource– message count

• Global– active handler count– queued message count


MessageHandler: Naming Problems

• MessageHandler is end user API (usually POJO) and location of interesting metrics

• Endpoint is the thing with a name (bean id)• Many endpoints are anonymous• But channels are not

widgets


Spring Integration JMX

<int-jmx:mbean-export server="mbeanServer“

default-domain="spring.application"/>

<int:channel id="widgets"/>

<int:service-activator input-channel="widgets"expression="payload.length()" />

<int-jmx:mbean-export server= "mbeanServer“

default-domain= "spring.application" />

<int:channel id="widgets" />

<int:service-activator input-channel= "widgets"expression= "payload.length()" />

MBean

spring.application:type=MessageChannel,name=widgets

MBean

spring.application:type=MessageHandler,name=widgets,bean=anonymous


Demo


Spring Integration Metric Types

ExponentialMovingAverageHandleDurationMessageHandler

IntegerHandleCountMessageHandler

ExponentialMovingAverageRatioErrorRatioMessageChannel

ExponentialMovingAverageRateSendErrorRateMessageChannel

ExponentialMovingAverageSendDurationMessageChannel

IntegerQueueSizeMessageChannel

IntegerSendCountMessageChannel

AlgorithmMetricComponent


Spring Integration Control

• Alerts on hotspots: slow handling, high error rates• Lifecycle

– start, stop, running

• Throttle messages on downstream error• Graceful shutdown


Demo


Metametrics and Metaoperations

• Deployment and provisioning: get me one instance of this application.

• What have I got deployed?• Is it working?• Scale and elastic scale: increase the number of workers.• Dynamic load: re-balance thread pools.• Highlight 4 approaches

– Command line– Local service– Build and deployment tool– Private or public cloud


Command Line

• Write a Java main() method• Use one off the shelf, e.g. CommandLineJobRunner• Maybe use Maven, Ant, Gradle etc.• Pros:• Lowest common denominator• High granularity:

– Tweak JVM parameters (heap size etc.)– Tweak global transaction settings

• Cons:• Setting up classpath and dependencies• Inventory: tracing and managing versions


Local Service: Servlet Container

• Write a servlet / web service for the business features• Use one off the shelf, e.g. Spring Batch Admin• Pros:• Readily available container• Standard packaging and dependency management• Tools for monitoring and management, e.g. Hyperic• Cons:• Some overhead creating container instance• Low granularity: module boundaries have to be drawn


Remote Build and Deployment Tool

• Write a remote build and deployment tool• Use one off the shelf, e.g. Capistrano, Hudson• Pros:• Granularity of command line• Can be used to manage remote server as well• Handles versioning• Cons:• New technology for some users• Still quite low level: requires script maintainence


Private or Public Cloud (PaaS)

• Make deployment and provisioning a service• Pros:• Commoditisation / standardisation• Public services easily accessible• Private cloud products emerging (e.g. Napa)• Very efficient once running: “blueprint” applies to all

deployments• Elastic and dynamic scale are built in features• Cons:• Extra layer in process and implementation• Some restrictions on application, especially in public

clouds


Spring Batch Control

• Basic use cases– Stop a job execution– Abort a job execution– Restart a job execution*

– Start new or next execution*

• Exposed through Spring Batch Admin and CommandLineJobRunner

• SBA can be deployed as a standalone service (just needs a JobRepository)

• *WithSBA only for Jobs that are configured inside SBA application


Spring Batch Metrics

• Latest JobExecution status, duration• StepExecution status and history, items processed,

skipped, failure analysis• Implementation: Spring Batch Admin

– web application– services (REST and Java)– JMX endpoints, e.g. consume with Hyperic


Notifications and Alerts

• Spring Batch SLA: step overrun– When a step execution takes too long, downstream

processing can fail– Use MBean Monitor features to send notifications– JMX client can enable and consume the notifications as

desired


Step Execution SLA

<bean id="serviceLevel"

class="org...admin.jmx.StepExecutionServiceLevelMon itor">

<constructor-arg ref="batchMBeanExporter"/>

<property name="timeout" value=“60000" />

<property name="jobName" value="job1" />

<property name="stepName" value="j1.s1" />

<property name="enabled" value="true" />

</bean>

<bean id="serviceLevel"

class= "org...admin.jmx.StepExecutionServiceLevelMonitor" >

<constructor-arg ref="batchMBeanExporter" />

<property name="timeout" value= “60000" />

<property name="jobName" value= "job1" />

<property name="stepName" value= "j1.s1" />

<property name="enabled" value= "true" />

</bean>MBean

spring.application:type=StepExecutionMonitor/WARN,name=job1/j1.s1

spring.application:type=StepExecutionMonitor/ERROR,name=job1/j1.s1


Demo


JMX Notifications and Spring Integration

• Spring Integration provides adapters for JMX notifications:– inbound: notification -> Message– outbound: Message -> notification

<jmx:notification-listening-channel-adapter

channel="input"

object-name="com.foo:type=TestBean,name=testBean"/>

<jmx:notification-publishing-channel-adapterchannel="input"object-name="com.foo:type=TestBean,name=testBean“default-notification-type="com.foo.test"/>

<jmx:notification-listening-channel-adapter

channel= "input"

object-name= "com.foo:type=TestBean,name=testBean" />

<jmx:notification-publishing-channel-adapterchannel= "input"object-name= "com.foo:type=TestBean,name=testBean“default-notification-type= "com.foo.test" />


Spring Integration ControlBus

• MBean operations on Message* components triggered by Message

<jmx:control-bus mbean-exporter="mbeanExporter"

channel="control"/>

<int-jmx:mbean-export server="mbeanServer"/>

<jmx:control-bus mbean-exporter= "mbeanExporter"

channel= "control" />

<int-jmx:mbean-export server= "mbeanServer" />

control

ControlBus.TARGET_BEAN_NAME=“myEndpoint"

JmxHeaders.OPERATION_NAME="stop"


JMX Notifications and Hyperic

• Hyperic can also consume JMX notifications– treated like log files (regex pattern on message)– generate alerts (email, SMS etc.)– trigger control actions (e.g. switch off upstream messages)

• Some bugfixes not yet released support auto-detection with no XML (should be in 4.5.x)


Demo?


More Future Directions

• Tracing and Business Transaction Monitoring• Replay failed transaction• Automated scheduling and restart• Performance summary, test harness


Q&A

Source Code for demos:

http://git.springsource.org/s2gx-2010/monitoring-spring-batch-integration

Date post:	19-Oct-2014
Category:	Technology
View:	4,380 times
Download:	1 times

Syer Monitoring Integration And Batch

Technology