© 2006 IBM Corporation
GSE Nordic WebSphere User Group 2006
®
Building an ESB with WebSphere MQ and Message Broker
Jonathan [email protected]
Message Broker Development
IBM Software Group
2Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Agenda
Service Oriented Architectures• What is a service and why do I need SOA?
Enterprise Service Bus• What is it and what does it mean for SOA?• Functions of an ESB
Protocols, Route, Transform, Eventing• What is an Advanced ESB?
Building an ESB with WebSphere MQ and Message Broker• Features and functions to build an ESB
Summary & References
IBM Software Group
3Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Service Oriented Architectures
IBM Software Group
4Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
What is a Service?
(Ideally) a self-contained, stateless business function that accepts one or more requests and returns one or more responses through a well-defined, standard interface. Services perform
discrete units of work such as editing and processing a transaction. Services should not
depend on the state of other functions or processes. The technology used to provide a
service, such as a programming language, does not form part of its definition.
IBM Software Group
5Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Applications implement most of today’s Services• There’s nothing wrong with this!
How are services and applications related?
• Application can implement the Service exactly
• Service can be implemented within part of an Application
− Including multiple services
• Combination of Applications can implement the service
Services and Applications
Application ServiceKEY:
IBM Software Group
6Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Identifying Services
Technology reality•“Where I am right now”•“Bottom Up” approach•Example: CICS transactions, MQ applications, Files, JSP, EJB…
Business driven•“Top Down”•“Where I’d like to be in the future”
−Explicitly user and business focussed
Mixed Business and Technology•“Middle Out”•Emphasis on cooperative and evolutionary approach
IBM Software Group
7Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
What is a SOA?
A software architecture which
• …defines the use of services to support the requirements of software users. Resources are available to other participants as independent services accessed in a standardized way.
• … comprises loosely coupled, high interoperable application services. These services interoperate based on a formal definition independent of the underlying platform and programming language. The interface definition hides the vendor and language-specific implementation.
• … is independent of development technology (e.g. Java, .NET, COBOL, C, ASM). The software components become very reusable because the interface is defined in a standards-compliant manner.
IBM Software Group
8Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
How do you know when you’ve got SOA?
+ = + =
Easy to connect services together without knowing that they had to be connected initially
• Take components / services and add them together• Then, later add more services / components
IBM Software Group
9Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
What are the benefits of SOA?
Asset reuse• Service is component of reuse
Lower Development Cost • Eliminate duplicate services
Faster Time-to-Value• Compose new applications from existing services
Extend Business Reach• Allow new clients to exploit existing services without change
Providing a new front end
Business Responsiveness• Dynamically reconfigure services to meet new business opportunities
Governance• Standardize, mandate and measure service usage
Allows you to introduce control
IBM Software Group
10Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Enterprise Service Bus
IBM Software Group
11Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
What is an ESB?
A software infrastructure for SOA which
• … comprises a set of interaction points to which services are connected. Services are able to invoke other services which are connected to the bus.
• ... simplifies an SOA by minimizing the explicit connectivity between services.
• … allows the creation of dynamic and flexible connectivity between services during service invocation without changing service consumers or providers.
IBM Software Group
12Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
SOA with an ESB
Simplification of Infrastructure• It DOES simplify the way you think
about service connectivity. • It DOESN’T change the way you think
about services.
Dynamic and Flexible Infrastructure• New connectivity
e.g. add customer request audit.• Flexible connectivity
e.g. prioritize customer request. • Service replacement
e.g. New service upgrade without…
IBM Software Group
13Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Functions of an ESB
Four functions in an ESB• Protocol support• Routing• Transformation• Eventing, including CEP
Other functions…• Security
can be performed in ESB, but not core functionality
Shape = ProtocolColor = Data type
IBM Software Group
14Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
What are the different kinds of ESB?
Basic ESB• XML as only supported data format• Web Services protocol emphasis (SOAP/HTTP)• Adapters for everything
XML conversion on and off the busProtocol conversion on and off the bus
Advanced ESB• All formats supported natively
XML, COBOL, C, SWIFT, EDI, MIME, Acord, X12• All protocols supported natively
Web Services PLUS MQ, JMS, File, SCADA device, user defined…• Minimizes need for adapters
Improves performance and manageability
IBM Software Group
15Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Building an Advanced ESB with MQ and MB
IBM Software Group
16Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
The Advanced ESB
WebSphere MQ and Message Broker form an Advanced ESB
MQ is transport protocol of choice for high performance, transactional messaging
• Very high quality of service transport• Loosely coupled
Message Broker provides higher level ESB functions• Comprehensive range of native transport protocols
FROM anywhere, TO anywhere• Perform suite of routing actions on service messages• Transformation of message formats between services• Broad range of Event Processing capabilities
IBM Software Group
17Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Use MQ to Build SOA Enabled Applications
WebSphere MQ
• An excellent SOA interface for applicationsSimple and powerful interaction styleVery high quality of service transport
• View your applications as servicesSelf contained, well defined, standard interfaceLoosely coupled requests and responsesDiscrete, transactional units of work
MQ
Application ServiceKEY:
IBM Software Group
18Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Add Message Broker for an Advanced ESB
WebSphere Message Broker
1… creates services from applications.
2 … provides service interactions for a broad range of protocol endpoints, including MQ.
3 … provides flexible and dynamic connectivity between service endpoints.
Provide routing and transformation capability
4 … combines with other brokers to form a scaleable ESB backbone.
1
24
3
Application Service
KEY:
Input & Output nodes
Message Flow
Protocol
IBM Software Group
19Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
ESB Function 1 – Protocol Support
Consume and generate service request messages• “FROM anywhere, TO anywhere”
Ability to bridge between application services which use different transport protocols
Extensive range of transports means only need to use protocol adapters if necessary
MQ is transport protocol of choice for high performance or transactional interactions
IBM Software Group
20Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Message Broker provides a comprehensive range of protocol endpoints using input and output nodes.
Also including…
Protocols - Message Broker Input and Output Nodes
UUser DefinedMQ HTTP SCADA
JMS 1.1 MQe IP/Multicast
Database
FLAT FILES VSAM/QSAM
SMTP
CICS TCP/IPTIBCO RV
IBM Software Group
21Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Protocol Support
Grouped into different categories of protocols
de facto Standard protocols• MQ, HTTP, JMS, TCP/IP
Native input nodes
“Standards” protocols• Web Services via SOAP/HTTP, SOAP/JMS
Native HTTP(S), Any JMS 1.1 connectivitySOAP and MIME parsers provided
Technology interactions• Files, VSAM, QSAM & CICS to access traditional data sources
Industry protocols• SCADA, MQe for mobile devices and sensors
User Defined protocols• For User ISV provided extensions, full user defined node interface.
IBM Software Group
22Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
ESB Function 2 – Routing
Routing can take on different forms• Route, Filter and Order service request messages
Dynamically direct service requests within the enterprise
Support for many types of routing within different interaction patterns
• One-way• Request-response• Publications• Service decomposition and reassembly
IBM Software Group
23Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Message Broker provides a powerful suite of routing nodes.
Flexible and wide range of routing metaphors supported.
Routing – Message Broker routing nodes
Route and Filter
Order
AggregationPublication Trace
Exceptions User Logic
IBM Software Group
24Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Message Broker provides an Eclipse based Message Flow Editor.
User friendly tool to express routing connectivity.• Drag and drop nodes onto the canvas• Input and output nodes represent the protocols• Lines connecting nodes show the ‘route’ through the flow
Routing – Building a routing flow
IBM Software Group
25Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Routing
Full range of messages can be routed and filtered• XML – for XML payloads (full schema support)• Record based – for COBOL, C payloads• Industry standard – SWIFT, EDI…• User defined – RYO formats
Different routing and filtering rules• Message based
Message Header and Body routing are supported• External reference
Database lookup, PLUS in-memory cache for infrequently changing data• User algorithm
Partial Parsing for efficient routing of large messages
IBM Software Group
26Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
ESB Function 3 - Transformation
Connected services may require different message formats• Shape• Data format
Ability to transform BETWEEN service message formats requires• Model of message formats• Transformation language to map between formats
Wide range of Message formats supported natively• No need for extra XML normalization• Model created using importers (COBOL, C, DTD, Schema) or Eclipse tooling
Wide range of Transformation languages• Allowing you to choose the right tool for the job• Matches existing or future skill sets
IBM Software Group
27Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Transformation – Graphical Mapping and Java
IBM Software Group
28Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Transformation – ESQL and XSLT
IBM Software Group
29Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Transformation
Message Broker models all data via “Message Tree”• All data formats (XML and non-XML) are understood natively by Message Broker• Message Tree created by a native parser from the source input message format• Transformed using appropriate transformation language• Output tree used to generate required target output message format• …leads to high performance
Data is only transformed when it is necessaryPartial parsing only reads required fields
Use transformation languages to match skill set• Graphical mapping for non-programmers• XSLT for XML orientation• SQL for database and declarative orientation• Java for widely available skill set benefits
• Transformation languages can be mixed and matched within same message flow
IBM Software Group
30Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
ESB Function 4 – Event Handling
Processing external events• Ability to distribute information to interested parties
Publish Subscribe and more complex event processing (CEP)
Timer based processing• Periodically start flows at preconfigured and programmatic time interval
IBM Software Group
31Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Eventing – Message Broker event nodes
Message Broker provides a rich set of event nodes.Publish Subscribe Timer Complex Event Processing
IBM Software Group
32Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Complex Event Processing - What is it?
Active Systems are systems that contain active (event-driven) components• Reactive Systems – React to something that happens in the system (server failed, direct
requests to other servers)• Proactive Systems – Use predictive methods to redirect the system towards better results
and or eliminate problems (server utilization is high, direct request to other servers)
Processing of action triggered not by a single event, but by a complex composition of events, happening at different times, and within different contexts
Examples: compliance checks, fraud detection, monitoring of service level agreements, etc
IBM Software Group
33Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Complex Event Processing Nodes
SituationManager Node
Detected situations result in message being written to queueDetected Situations
IntelligentFilter Node
Detected situations result in alternate path through messageflow
IBM stock gained 3% and NASDAQ lost 1% within the last hour
Route messages to a compliance officer if a client performs more than a certain number of messages that cause a credit breach in one day
IBM Software Group
34Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Security Considerations for an ESB
Can secure the link between the service and ESB• Transport protocol security, e.g. SSL
All security cannot be done in ESB• Co-operation from the protocol / transport• Extreme is that none of the security is
handled within the ESB
ESB needs to “see” relevant data to perform routing and transformation• Security versus flexibility
IBM Software Group
35Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Performance Considerations for an ESB
Flexibility AND performance are metrics of success• ESB can affect latency, possibly throughput
Partial parsing for efficient routing and transformation of large messages• Only parse as much as necessary
50% average throughput improvement for Message Broker v6• Same flows, etc.
Important to use performance enhancing features where possible• In-memory caches• Queues, rather than databases, for transient data
IBM Software Group
36Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Summary
ESB is an important infrastructure component for SOA• Protocols, Routing, Transformation, Eventing
WebSphere MQ and Message Broker form Advanced ESB• Integrate standard and non-standards-based applications and services
IBM Software Group
37Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
References
WebSphere MQ and Message Broker
http://www.ibm.com/software/integration/wmqhttp://www.ibm.com/software/integration/wbimessagebroker
IBM Enterprise Service Bus web site:
http://www.ibm.com/software/info1/websphere/index.jsp?tab=landings/esbbenefits
ESB Primer
http://submit.boulder.ibm.com/dd/wsdd/asis/501957/501957.html
Red BookPatterns: Implementing an SOA Using an Enterprise Service Bus: SG24-6346
IBM Software Group
38Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Summary: Resources … IBM’s ESB Web Site
IBM Enterprise Service Bus web site:http://www.ibm.com/software/info1/websphere/index.jsp?tab=landings/esbbenefits
IBM Software Group
39Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Any Questions?
IBM Software Group
40Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Extra foils
IBM Software Group
41Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
MQ, MB and the SOA Lifecycle
IBM Software Group
42Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
SOA Lifecycle and ESB
SOA Lifecycle is model, compose, deploy, monitor
ESB is infrastructure component of SOA• Model service endpoints• Compose service endpoint connectivity• Deploy endpoints and connectivity to ESB• Monitor ESB
IBM Software Group
43Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
SOA Lifecycle – Model and Compose
Application
Service
KEY:
Input & Output nodes
Message Flow
Protocol
CICS TXN
MQHTTP EXCI
CONSUMER
MQ
CONSUMER
HTTP
IBM Software Group
44Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
SOA Lifecycle - Model
How to model a service endpoint• Service is an application, e.g. CICS, IMS transaction• ESB makes application endpoint available to other services
Formal Service definition• None (implicit)• Paper • COBOL/PLI copybooks, C header files• WSDL• SCA• …other
Tools for service endpoint definition and interactions• Importers to create ESB service endpoint definition from application• Exporters to enable ESB endpoint invocation from application
IBM Software Group
45Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
SOA Lifecycle - Compose
Two types of composition• Service creation
Composing services from the applications you already have• New service creation
How to (de) compose a service from other services• Both aspects are important
Work with what you haveKnow how to evolve
Comprehensive samples in Message Broker version 6• Most common scenarios available for import directly into work space
Request-reply, Web Service, Audit, Aggregation…
IBM Software Group
46Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
SOA Lifecycle - Deploy
Make an ESB service endpoint active• Message Flow is made live
Servicing requests, and generating responses• Hot deploy is possible
Existing endpoints are not affected• Deployment tools
Eclipse, Command Line, Programmatic (Java or PCF)
Runtime Versioning• Ability to determine which level of ESB artefact is active
Message Flows, Message Sets etc…• Trace-ability• Problem determination
IBM Software Group
47Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
SOA Lifecycle - Deploy
IBM Software Group
48Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
SOA Lifecycle - Monitor
Monitor ESB connectivity layer• Monitor Flows, Execution Groups, Brokers
Application monitoring is outside scope of ESB• Need integrated monitoring tools• This cannot be delivered by individual products
SOA based monitoring tools e.g. ITCAM for SOA
End to end tracking becoming increasingly important• Correlation between messages through Application and infrastructure• Message Broker V6 Message Flow tracking feature
Monitoring tools enabled• Tools to consolidate across enterprise wide interactions
IBM Software Group
49Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
SOA Lifecycle – Monitor ESB
IBM Software Group
50Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Monitoring – the wider pictureAs SOA’s get deployed you need to see all of the “bits” using one view, you don’t have
time to hop from monitor to monitor when your business is effectively stopped.
You have to implement things such as ITIL service desk functions to restore normal operations as soon as possible – using tools such as ITM 6.1 and OMEGAMON Reflex Automation.
You will have to integrate components into your infrastructure that do not have traditional monitors – Tivoli’s UA Technology can build these and interface directly into the TEP (Tivoli Enterprise Portal).
As Z Series continues to grow it hosts the major components of SOA’s – DB2, CICS, WMQ, Broker, WAS, WPS, ESB
Business users will want to charge for Services – which means SLA’s and SLA compliance and Business Views – “How many widgets did we sell today ?” – not how is CICS response time.
IBM Software Group
51Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Managing an ESB
Tivoli will exploit the brokering runtimes, mediation frameworks provided by WebSphere• Create and manage the mediations for management
Tivoli management server displays aggregated data, provides notifications, alerts for IT Operators
• .Net, WebLogic, WAS
ServicesIntermediary
Service Requestor
Service Provider
RoutingTransformation
Monitoring Enforcement
TEP Views provided by:ITCAM for SOAITCAM for RTTITCAM for WAS
OMEGAMON XE for WBI
= ESB functions provided by WebSphere
= Management functions provided by Tivoli
IBM Software Group
52Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Service Operations View
Service Operations View
Service Inventory View
Service Inventory View
Service Performance View
Service Performance View
Integrated views of web services data through a centralized portal
Aggregate web services data with other IT infrastructure monitoring
Customize workspaces to tailor views to specific roles
Use Situations to create thresholds, alerts and take actions
Provide message logging as well as message rejection
Manage Service Relationships
IBM Software Group
53Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Summary
IBM Software Group
54Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
Summary
ESB is important infrastructure component for SOA• Protocols, Routing, Transformation, Eventing
WebSphere MQ and Message Broker form Advanced ESB• Integrate standard and non-standards-based applications and services
IBM Software Group
55Building an Advanced ESB for SOA with WebSphere Message Broker Version 6
References