WebSphere_UsrGrp_PubSub.ppt

© 2005 IBM Corporation

SOA on your terms and our expertise

Software

Publish / Subscribe

Tim PickrellIT SpecialistBusiness Integration Architect, Lab Services

2 SOA on your terms and our expertise

Software

Agenda

IBM’s Publish / Subscribe Product set

Publish / Subscribe Concepts and facilities

Ease of implementation

Why use Publish / Subscribe?

Experiences and some Gotchas


Software

Publish / Subscribe Product Set - History

MA0C Supportpac

MQSeries Integrator V2.0

– Topic Based Security and contect based filters

– RFH2 Header

MQSeries Integrator V2.0.x

– Telemetry and Mobile

WebSphere MQSeries Event Broker V2.1

– Real-time transport

WebSphere MQSeries Integration Message/Event Broker V5

– Multicast


Software

WebSphere Business Integration Brokers

WebSphere Publish / Subscribe

WebSphere Business Integrator Event Broker V6.0

WebSphere Business Integrator Message Broker V6.0

WebSphere Service Integration Bus



Software

Publish / Subscribe Concepts and Capabilities


Software

What is Publish / Subscribe?

Publisher 1

Train Schedules

Publisher 2

Flight Schedules

Subscriber 1

Train Schedules

Subscriber 2

Train and flight Schedules

Broker


Software

What is a Subscriber?

An application that is interested in receiving data of a certain type or category

– matches a specific category or “topic”

Need not know where the data comes from

Does not know how the data was delivered to the subscriber

Identifies the topic(s) on which it is interested in receiving associated data

Example

– Interested in price of IBM stock


Software

What is a Publisher?

Application that owns data to be made available to other applications (subscribers)

Does not know which or how many subscribers are interested in the data

Does not know how to deliver the data to the subscribers

Associates the data with a “topic”

Example: stock trading

– New stock price


Software

What is a topic?

A topic is an arbitrary string

Can be part of a hierarchy

– Levels delimited by /

Wildcards are allowed on subscriptions

– # matches any number of levels

– + matches a single level

– One # and any number of + permitted in a subscription

– Wildcards must be delimited by /

– # as topic matches everything

Support for multiple subscription points (streams) – MQ

– Similar to high level topic but on a separate tree


Software

What is a Topic - Example

MarketData

RealTime Delayed

EQ FI OP

SYMBOL

EQ FI OP

Special characters in topics…

Topic level separator

Single-level wildcard +

Multi-level wildcard #

e.g Subscription - MD/R/EQ/IBM


Software

Publish / Subscribe characteristics

Subscriber population changing rapidly

All users are interested in an identical message

Push applications

Minimize management overhead

– No central control or no control at all

Examples of good fit

– Streaming video

– Stock tickers

Multicast/Broadcast (if used) saves network bandwidth

– Good for one to many

– Logging more efficient


Software

Publish / Subscribe Application Styles

Events

– Every published message is significant

– Subscribers receive as available

State

– Information that is being updated/replaced

– Brokers retain latest publications

– Subscribers receive when updated and/or when desired

Conferencing

– Publisher also subscriber

– Option to suppress own publications


Software

Programming Interfaces

MQI

– Uses RFH2 headers for pub/sub information

JMS

– Durable versus non-durable

– API, not a protocol or implementation

XMS

AMI

– Recommend you don’t use this for new development

WebSphere Message Broker compute node



Software

MQI Publish / Subscribe Capabilities


Software

Publish / Subscribe using the MQ transport

BrokerPublisher Subscriber

Publish

Delete Publication

Response Response

Register Subscriber

DeRegister Subscriber

Request Update

Publish


Software

Simple Publish / Subscribe

Publisher

Subscriber

Subscriber Queue

Broker

Input Queue

Control Queue

1 - Publish

3 - Publish

4 - Publish

3 - DeRegSub

2 - Subscribe

5


Software

Retained publications

Publisher

Subscriber

Subscriber Queue

Broker

Input Queue

Control Queue

1 - Publish 3 - Publish

2 - Subscribe

4


Software

Publish on request

Publisher

Subscriber

Subscriber Queue

Broker

Input Queue

Control Queue

1 - Publish5 - Publish

2 - Subscribe

6

3 - Publish

4 – Request Update


Software

Subscriptions

A subscription consists of:

One or more topics – wildcards can be used

An optional subscription point or Stream

An optional filter on the message content

A subscriber queue, queue manager and Correlation-id


Software

The psc folder – registering a subscriber

<psc>

<Command>RegSub</Command>

<Topic>Sport/Soccer/Score</Topic>

<QMgrName>SPORT1</QMgrName

<QName>RESULTS</QName>

<PubOpt>CorrelAsId</PubOpt>

</psc>

The combination of queue manager name, queue name and correlation-Id is known as the subscriber-Id


Software

Mapping published messages to subscribers

MQMD RFH2 Data

<Topic/Sport/Soccer/Score</Topic>

Subscriber-Id SubscriptionsQMGR: SPORT1

Queue: RESULTS, CorrelAsId

Topic: Sport/Soccer/#

Topic: Sport/+/Score

QMGR: SPORT1

Queue:Events, CorrelAsId

Topic: Sport/Events/#

MQMD: correlid from subscriber

QMGR: SPORT1

Queue: RESULTS

Publication message


Software

Publishing using the MQ transport

Broker needs a Message Flow to tell it how to publish

Message flow names input queue for publish message source

Recommend failure and log queues


Software

Publication Node Properties


Software

Shared Subscription Queues

Shared subscription queues

– Correlid sent on subscribe request

– Published messages have same correlation id

– Requires use of correlasid option

Non-shared subscription queues

– Multiple subscriptions match same publication

– Only a single message is returned to subscriber


Software

Subscription DeRegistration

Subscriptions can be deregistered using the DeRegister subscription command message

A subscription may expire

A subscription can be removed by an administrator from the subscriptions view of the broker toolkit

The broker automatically deregisters subscriptions on closure of a temporary dynamic subscription queue


Software

WebSphere Transports

High Performance

Broker

WebSphere MQ Enterprise

WebSphere MQ Mobile

WebSphere MQ Telemetry

WebSphere MQ Real-time

WebSphere MQ Enterprise

WebSphere MQ Mobile

WebSphere MQ Telemetry

WebSphere MQ Real-time

WebSphere MQ Multicast

WebSphere MQ Web Services WebSphere MQ Web Services

Send and Receive using Any transport

Transports Optimised for your unique application needs



Software

JMS and XMS Publish / Subscribe Capabilities


Software

Sample JMS Code

// Obtain a TCF TopicConnectionFactory tcf = new MQTopicConnectionFactory();

// Obtain MQ-specific referenceMQTopicConnectionFactory mqtcf = (MQTopicConnectionFactory)tcf; // Create a topic connectionTopicConnection conn = tcf.createTopicConnection();

// start the connectionconn.start();

// Create a topic session from the connectionTopicSession topicsess = conn.createTopicSession(false, Session.AUTO_ACKNOWLEDGE);

// Create the TopicString topicstring = "TEST/TOPIC";Topic t = topicsess.createTopic(topicstring);

// Create the publisher pub = topicsess.createPublisher(t);


Software

Sample JMS Code continued

// Set up the publication messageTextMessage msg = null;msg = topicsess.createTextMessage();String msgdata = "Sample Publication message";msg.setText(msgdata);

// Publish !pub.publish(msg);

// tidy up by closing the publisher, session and connection objectspub.close();sess.close();conn.close();


Software

Sample JMS Code

package jms11;

import javax.naming.*; // The JNDI classes

import javax.jms.*; // The JMS 1.1 classes

public class JMS11UnifiedTransactionExample {

/**

* Transfer a message from a source destination

* to a target destination. The destinations may

* both be queues, both be topics, or may be from a

* queue to a topic, or from a topic to a queue.

*/

public void transferMessage(String connectionFactoryName, String sourceName, String targetName)

throws NamingException, JMSException {

// Get the specified connection factory and destinations

Context jndiContext = new InitialContext();

ConnectionFactory factory = (ConnectionFactory)

jndiContext.lookup(connectionFactoryName);

Destination source = (Destination) jndiContext.lookup(sourceName);

Destination target = (Destination) jndiContext.lookup(targetName);

// Create the connection and session

Connection connection = factory.createConnection();

Session session = connection.createSession(true,

Session.AUTO_ACKNOWLEDGE);


Software

Sample JMS Code - continued

// Use the session and destinations to

// create the consumer and producer

MessageConsumer consumer = session.createConsumer(source);

MessageProducer producer = session.createProducer(target);

// Transfer the next message on the source

// destination to the target destination

// in a single transaction

try {

producer.send(consumer.receive());

session.commit();

}

catch (JMSException ex) {

session.rollback();

}

// Release all resources

producer.close();

consumer.close();

session.close();

connection.close();

}

}


Software

XMS Client Overview A C++ implementation of the JMS Messaging API

Simple API to code, simplifies the programming and manages all connections

Common messaging API across IBM integration products– Current (WMQ) and future (WPM etc.)

High level Message classes (JMS 1.1)

Simplified Point-to-Point and Publish/Subscribe messaging– Send/Receive messages to/from queues or topics

Decoupled Administration – Admin tooling to define Connection Factories and Destinations– Directory can be LDAP, COS naming, or file system based JNDI– XMS and JMS can both use the same administered objects


Software

Sample Message Service Client for C/C++ code (IA94)using IBM.XMS;namespace IBM.XMS.LeggiXMS{ public class LeggiXMS {

/// <summary>/// The main entry point for the application./// </summary>[STAThread]static void Main(string[] args){ try { Hashtable hash = new Hashtable(); hash.Add(XMSC.IC_URL, "file://C:/JNDI-Directory/.bindings"); IContext ctx = new InitialContext(hash); IConnectionFactory connectionFactory = (IConnectionFactory)ctx.Lookup("CF_XMS"); IConnection connection = connectionFactory.CreateConnection(); ISession session = connection.CreateSession(false, AcknowledgeMode.AutoAcknowledge); IDestination topic = (IDestination)ctx.Lookup(“TOPIC_XMS"); IMessageConsumer consumer = session.CreateConsumer(topic); IMessage msg = consumer.Receive(0); ITextMessage msgTxt = msg as ITextMessage; if (msgTxt != null) { System.Console.WriteLine( msgTxt.Text ); } else { System.Console.WriteLine( "Empty message" ); } consumer.Close(); queue.Dispose(); session.Close(); connection.Close();

} catch (XMSException e) { System.Console.WriteLine(e.Message); System.Console.WriteLine(e.LinkedException.Message); }}


Software

WebSphere Real-time transport

High-performance protocol designed for publish/subscribe– Developed at IBM Research (Hawthorne)

Non-persistent (non-durable) messages only

Uses direct TCP/IP connection– Does not use WebSphere MQ

Supports JMS and XMS clients

Integrated in Message Broker V6.0 and Event Broker V6.0– First released in WMQ Event Broker V2.1

Number of connections may be limited by operating system– Requires multiple brokers in collective for more connections

Unicast and Multicast


Software

Unicast vs Multicast

Multiple transmissions to send the same message.

Single transmission to send same message to all clients.

All clients must be on the same LAN segment.

Unicast

Multicast

4 Transmissions

1 Transmission


Software

Multicast Advantages

Scaleable

– Conserves network bandwidth

– Minimises broker resource utilisation

– Particularly effective with high numbers of subscribers

Fairness (simultaneous reception)

– Only within the same LAN segment

Minimises latency

Neat Mapping with publish / subscribe


Software

Publish / Subscribe nodes

Receives Subscription and Publication Requests

Receives Subscription and Publication requests. Publishes

directly.

Receives Publication requests that include an RFH header


Software

Real-timeOptimizedFlow node


Software

Real-Time Input node


Software

Real-timeInput Node properties


Software

Broker Networks

Broker Broker

BrokerBroker

Broker Broker

Broker Broker

Broker

London New York

Tokyo

Multiple brokers can be interconnected

Performance & scalability

Availability

Geographical distributionConfigMgr – not shown


Software

Broker Network configuration


Software

Broker Properties


Software

Broker – Multicast properties


Software

Broker Toolkit - Subscription view


Software

Cloned Subscriptions – High Availability

QM

QMSUBQ

BrokerPUBQ

CTRLQ

P

S

QM

BrokerPUBQ

CTRLQ

QMSUBQ

S


Software

Message Broker – Topic Based Security


Software

Best Practices

Deciding on a transport

Queuing vs messaging

Is the application really a pub/sub application?

Logging of published messages

High fan-out vs high fan-in


Software

Why use Publish / Subscribe?

All applications are interested in an identical message

Subscriber population is dynamic and changes rapidly


Software

Market Data Solution Configuration

Broker 1 Broker 2

RMMToEB RMMToEBWAS

Fat Pipe

Tick Receiver

WAS

Real Time Loader

MQ

Trade

PipePosition Server

Trade Feed

External

Trades

JMSJMS

RMM

RMM

RMM

TAM ITM

DJ30,DJ30 Delay,Options, Position blotter, History request,

etc


Software

Design of the Topic Hierarchy

Design of the topic hierarchy requires careful consideration !

Must be flexible enough to satisfy business requirements when matching publications to subscribers

Must provide sufficient granularity for topic based security

Has a potential impact on performance – including assignment of multicast groups


Software

Design of the Topic Hierarchy Continued

/[R|D]/[E|O|F]/B[1..100]/Symbol/Exchange

Real-time or Delay Service

Equities, Fixed-Income or Options

Multicast Bucket

Symbol e.g. IBM

Exchange e.e Nasdaq


Software

Real Time Messaging – Example architecture

FBSI

MAUI

SDIST

Application Server

Web Server

FENS

EDB

Exc

hang

es

XML

URL Get RequestXML Document Reply

HTTP

TIBCo RVUnsecure UDP

XpressoXML

TIBCoRemote RVRD

Alert data &Notification Info

Alerts or Tickle messages that user info is available

Inbound: · Trades & · Data RequestsOutbound: · Replies

Alerts or Tickle messages that user info is available

as well asReal time market data

RMDS

TIBCoRV

Real time market data

Trade status &Notificaiton info

Outbound: · TradesInbound: · Execution status

ATPClient

Inbound: Requests for alert data & notification infoOutbound: Replies

Proxy Server

URL Get RequestXML Document Reply

HTTPS

HTTPS

Fire

wal

l

Fire

wal

l

DMZ


Software

Solution Components

IBM Out of the Box Components

– WMQ

– Message Broker

– Replay Server

– WebSphere Application Server

– DB2

– XMS Client API

Custom Applications

– WebSphere Application Server App

– Heartbeat Monitor App


Software

S S

XMS

UDB

Broker Replay

Server

CommandHandler

Solution Server

Replay Proxy

Monitor App


Software

WAS Core Components

S

WebSphere Application Server

Application Server Cluster

ATP Client

Clustered Messaging Endpoint

Messaging Endpoint

Resource Status Table


Software

Register Client Subscription

S

Replay Proxy: Simple class that subscribes to replay server and publishes to client topics. Serves multiple clients.

MDB: Client

Command Handler

Lookup Table

•---------

•---------

Client Command Handler: Implemented as a Message Driven Bean, handles subscription and description requests from the client.

Message Broker

Replay Server

1

2

3

1. Register subscription request

2. Add reference to lookup table

3. Register client with replay proxy

Replay Proxy

Sub ID/WS 00:34:45

Verb Topic Timestamp

Lookup Table: Client to Replay server topic mapping


Software

Related material

Redbook - MQSeries Publish/Subscribe Applications

– http://publib-b.boulder.ibm.com/abstracts/sg246282.html


Software


Software

WebSphere Event Broker

Subset of Message Broker V6

– Transformation capability removed

• Message repository• Most processing nodes (e.g. compute, filter)

– Includes Generic XML parser only

• Content based limited to XML messages

Both Message Broker and Event Broker include

– Gryphon technology from IBM Research for real-time support

– Multicast support developed by IBM Haifa


Software

SOA Reference Architecture

Ap

ps

&

Info

As

sets

Business Innovation & Optimization Services

Dev

elo

pm

ent

Ser

vice

s

Interaction Services Process Services Information Services

Partner Services Business App Services Access Services

Integrated environment for design

and creation of solution

assets

Manage and secure services,

applications &

resources

Facilitates better decision-making with real-time business information

Enables collaboration between people,

processes & information

Orchestrate and automate business

processes

Manages diverse data and content in a

unified manner

Connect with trading partners

Build on a robust, scaleable, and secure services environment

Facilitates interactions with existing information and application assets

ESBFacilitates communication between services

IT S

ervi

ceM

anag

emen

t

Infrastructure Services

Optimizes throughput, availability and performance

Date post:	10-May-2015
Category:	Documents
Upload:	zubin67
View:	1,019 times
Download:	1 times

WebSphere_UsrGrp_PubSub.ppt

Documents