An Introduction to the Message Queuing

Technology & IBM WebSphere MQ

Ravi YogeshWeb Performance Engineer, Wells Fargo

(https://www.linkedin.com/in/raviyogesh)

Agenda:

Basics of MQ Technology

(Persistence, Types, Styles)

Message Queuing in Banking

Different Message Queuing Implementations

In Focus : IBM WebSphere MQ

Message Queues in Action around Us

MQ Technology: Basics

• Message queuing is a middleware technology that greatly simplifies

communication between the nodes of a system and between the nodes that

connect systems together.

• This technology builds on two basic concepts:

Message : String of Bytes/Information that needs to be communicated from one

node to another.

Has two parts : Header (message ID, message type, priority, expiration time )

and Data (can be XML, binary data etc)

Queues : Data Structures used to store messages.

• They enable applications and services to communicate asynchronously,

without each having to be available at the same time.

• When both nodes are available, messages arrive at destination in near real

time.

• If either of the network or node is unavailable/busy, the queue holds the

messages until they can be received and processed, thus assuring

delivery.

Message Persistence:

• Persistent messages are logged (data synchronously written to disk at the

same time as sending) thus enabling recovery of message on failure.

• Semi Persistent Messaging : Tradeoff between robustness of delivery and

throughput- logging occurs asynchronously after message dispatch- faster but

riskier than persistent mode.

• Non persistent Messaging : No logging thus no recovery on failure !!

• Recovery , persistence and assured delivery of messages : handled by the

Queue Managers which are themselves backed up in high availability

environments.

Types of Queues:

Remote Queues: References/handles to queues residing on other machines.

Transmission Queues: Special Queues used internally to move messages across

a network.

Dynamic Queues: Local Queues created on demand.

Dead Letter Queues: Queues that store messages that can not be delivered

(destination queue full scenario).

Repository Queues: Queues holding clustering information.

Messaging Topology:

Point to Point : Useful for simple

scenarios and NOT for larger

deployments because of the large

number of connections required.

{n(n-1)/2}.

Hub and Spoke: Requires more

initial planning but much lesser

number of connections for the

same number of nodes. Single

point of failure exists !

Bus: Same number of connections

as Hub and Spoke with distributed

connectivity logic across the

backbone. Thus, no single point of

failure.

Messaging Styles:

• Fire & Forget: Sender does not need a confirmation from receiver.

• Request - Response: Replies need to be sent from receiver.

• Publish & Subscribe: Event driven style of messaging, backbone

dynamically determines where the messages have to be delivered, relives

applications and services of information burden.

Benefits of Message Queuing Technology in Banking:

• Letting the applications talk to the messaging backbone through a

simple common API saves a lot of complex and application specific

connectivity logic to be defined on each application.

• Assured delivery: Critical business information like a banking

transaction needs assured and only-once delivery.

• Time-independent (asynchronous) processing.

• Avoids redundancy and duplication of data.

• Can easily scale and adapt to business requirements change.

Message Queuing Implementations:

Proprietary software:• Microsoft Message Queuing (tied to specific operating system).

• IBM's WebSphere MQ.

Message Queuing as a Service: • StormMQ

• IronMQ

• Amazon SQS.

Open Source Software: • JBoss Messaging

• JORAM

• Apache ActiveMQ

• Sun Open Message Queue

• Apache Qpid

• RabbitMQ,

• Beanstalkd,

• Tarantool

• HTTPSQS

.

Queuing Solutions on the Cloud:

Amazon Simple Queue Service (Amazon SQS):

A distributed queue messaging service introduced by Amazon.com in April

2006.

Storm MQ: available as hosted, On-Premise or Cloud solution for

Machine-to-Machine Message Queuing using the standard Advanced

Message Queuing Protocol (AMQP).

Rabbit MQ : open source software that implements the AMQP & is written

in the Erlang programming language.

In Focus : IBM MQ WebSphere

WebSphere MQ moves more than 10 billion messages every day,

supporting more than US$1 quadrillion (US $1 000 000 000 000 000)

worth of business transactions !!

• launched by IBM in March 1992

• available on a large number of platforms (both IBM and non-IBM),

including z/OS (mainframe), OS/400 (IBM System i or

AS/400), Transaction Processing Facility, UNIX (AIX, HP-UX, Solaris),

HP NonStop, OpenVMS, Linux, OS 2200, and Microsoft Windows.

Message Handling in Websphere MQ:

• Each WebSphere MQ message can be up to 100 MB in size.

• Larger messages and files can be transported by segmenting them into

smaller chunks.

• Messaging traffic can be reduced when many very small

messages are being sent to the same destination by grouping these

together into larger WebSphere MQ messages.

• By default, WebSphere MQ uses persistent messaging.

• WebSphere MQ can preserve the order of messages, delivering them to

applications in the same order as they were dispatched. First-In-First-Out

(FIFO) is the default.

Clustering

• Enables to improve the performance of a messaging backbone, more

messaging engines to be added as needed.

Scalability

• WebSphere MQ takes advantage of multiprocessor and multicore

machines so that you can scale your messaging backbone by using

parallel processing to accelerate messaging.

Security

• Supports the industry standard Secure Sockets Layer (SSL) for strong

authentication of message channels before messages are exchanged to

help prevent malicious attacks to the backbone.

Message Queuing is Ubiquitous !

Twitter : Open Source Message Queue Service: Kestralhttps://blog.twitter.com/2009/building-open-source

Whatsapp and Facebook : Messaging Cluster based on XMPP

(Extensible Messaging and Presence Protocol) and Erlang Programming

Language.http://www.erlang-factory.com/upload/presentations/558/efsf2012-whatsapp-scaling.pdf

References:

• http://www.redbooks.ibm.com/redbooks/pdfs/sg247128.pdf

• ftp://ftp.software.ibm.com/software/integration/wmq/WS_MQ_Messaging_Backbone_for_SOA.pdf

• http://blog.codepath.com/2013/01/06/asynchronous-processing-in-web-applications-part-2-

developers-need-to-understand-message-queues/

• http://www.infoq.com/articles/AMQP-RabbitMQ