Integrating Systems in the Cloud using Message Queuing

Post on 21-Nov-2014

2,893 views 1 download

description

Integrating Systems in the Cloud using Message QueuingSlides for talk given at EuroCloud Ireland 2010.

transcript

Integrating Systems in the Cloudraphael.cohn@stormmq.com

1

The Integration Tag Soup

WS – File Transfer

!"#$%&'(#)#*#+%+

Message QueuingSOAP

EMail

SMTP

FTP

RSYNC

NTFS

!','-

!).$%&'/$.0%&1$%+

234%5)6!%$7%$

SOAMessage

OrientatedMiddleware

JMS

MQ

MQSeries

AMQP

Amazon SQS

Tibco

StormMQ

RabbitMQ

HTTP

TCP/IPDCE / RPC

CORBA

DCOM

REST

RMIXML-RPC

.NET Remoting

Remote Procedure Call

XML-RPC

2

The Integration Tag Soup

WS – File Transfer

!"#$%&'(#)#*#+%+

Message QueuingSOAP

EMail

SMTP

FTP

RSYNC

NTFS

!','-

!).$%&'/$.0%&1$%+

234%5)6!%$7%$

SOAMessage

OrientatedMiddleware

JMS

MQ

MQSeries

AMQP

Amazon SQS

Tibco

StormMQ

RabbitMQ

HTTP

TCP/IPDCE / RPC

CORBA

DCOM

REST

RMIXML-RPC

.NET Remoting

Remote Procedure Call

XML-RPC

2

Under Examination, though

File Transfer

!"#$%&'(#)#*#+%+

Remote Procedure Call

Don’t Work in the Cloud

3

Under Examination, though

Deployment

Intimate System Knowledge

Configuration

Admin

File Transfer

!"#$%&'(#)#*#+%+

Remote Procedure Call

Don’t Work in the Cloud

Scaling

3

Ideal: Message Queuing

Systems are “loosely-coupled”

MessageQueuing

Systems don’t know each other

4

Why Use it: Loose Coupling

BillingCatalogue

Shipping S-a-a-S Inventory

How do we connect them, without one outage or system change taking everything down like a pack of cards?

5

Why Use it: Loose Coupling

BillingCatalogue

Shipping S-a-a-S Inventory

How do we connect them, without one outage or system change taking everything down like a pack of cards?

Message Queuing lets Systems and Components exchangedata, events, commands and actions with one another

with no explicit knowledge or need for them to be online

5

It should be ideal for the cloud

6

It should be ideal for the cloud

6

!!!Until today, messaging is …

(ZeroMQ)

(SQS uses HTTP)

(MSMQ uses COM)

(memcached)

(Java JMSuses

Source)

(MQSeries / WebsphereMQ)

(Talarian, Rendezvous, etc)

Platform Restricted BespokeProprietary

7

!!!Until today, messaging is …

(ZeroMQ)

(SQS uses HTTP)

(MSMQ uses COM)

(memcached)

(Java JMSuses

Source)

(MQSeries / WebsphereMQ)

(Talarian, Rendezvous, etc)

7

However, there’s a solution

8

A-MQPAdvanced Message Queue Protocol

However, there’s a solution

8

AMQP is Open and Neutral

A common wire-level binary format and protocol

An explicit definition of a server (aka broker)’s

semantics

Open Means

Interoperable

“AMQP will be to Messaging what HTTP was to the Web”

9

That is good …

“And it works perfectly in the cloud”

MRG

Clients run on any Platform Vendors are Interoperable

10

That is good …

“And it works perfectly in the cloud”

MRG

Clients run on any Platform Vendors are Interoperable

64K

10

From Anywhere to Anywhere

11

From Anywhere to Anywhere

11

From Anywhere to Anywhere

11

From Anywhere to Anywhere

11

From Anywhere to Anywhere

11

Instant Cloud Architecture

12

Instant Cloud Architecture

Cloud Service

Managed Service

On-Site Service

No-Lock™ Option

Locate-It™ Option

Real SLA

12

Instant Cloud Architecture

Self-Configure: REST API

Cloud Service

Managed Service

On-Site Service

No-Lock™ Option

Locate-It™ Option

Real SLA

Any Programming Language

Secure by Default

Isolated Environments

12

Store and Forward

Billing

“Could you send me a duplicate of my bill please”

REST

14

Store and Forward

Billing

“Could you send me a duplicate of my bill please”

REST !14

Store and Forward

Billing

“Could you send me a duplicate of my bill please”

REST

StormMQMessaging

Billing

!14

Store and Forward

Billing

“Could you send me a duplicate of my bill please”

REST

StormMQMessaging

Billing

!!14

Store and Forward

Billing

“Could you send me a duplicate of my bill please”

REST

StormMQMessaging

Billing

!14

Fire and Forget

Shipping

“Too many orders. The website is running like a dog.”

SQL Push SQL Pull

15

Fire and Forget

Shipping

“Too many orders. The website is running like a dog.”

SQL Push SQL Pull

StormMQMessaging

Shipping

15

One-To-Many

“There’s a new widget in the Catalogue: Tell all the systems.”

CatalogueShipping

S-a-a-S Inventory

File Transfer

ETL

16

One-To-Many

“There’s a new widget in the Catalogue: Tell all the systems.”

CatalogueShipping

S-a-a-S InventoryStormMQMessaging

16

Publish-Subscribe (“Topics”)

“Shipments Sent, Delivered and Returned”

Shipping

S-a-a-S InventoryBilling

AnySentSent or Returned

17

StormMQMessaging

Publish-Subscribe (“Topics”)

“Shipments Sent, Delivered and Returned”

Shipping

S-a-a-S InventoryBilling

AnySentSent or Returned

17

Round-Robin

Billing

“How do we easily scale a massive batch job like Billing?”

EMail BillGenerate

18

StormMQMessaging

Round-Robin

Billing

“How do we easily scale a massive batch job like Billing?”

EMail BillGenerate

18

Integrating Systems in the Cloudraphael.cohn@stormmq.com

19

A History of Messaging

Manual Telegraphy Machine Assisted Telegraphy

1920s1911 - 192019th Century 1900s

Telegrams sent using“Store and Forward”

1900

1930s

22

A History of Messaging

Electronic Telegraphy

1950s1940s1950s1940s

Electronic Telegram Machines, eg Plan 55-A

1948

IBM System/360 with BTAM & QTAM

Message Switching

Machine Assisted Telegraphy

1920s1911 - 1920 1930s

22

A History of Messaging

Telcos UseElectronic Telegraphy

1950s1940s 1960s

IBM System/360 with BTAM & QTAM

Message Switching

1964

First Electronic Mail Solutions

1965

Banking Users

1970s

IBM TCAM which is the first true solution

Retired 1990!

1971

Machine Assisted Telegraphy

22

A History of Messaging

First Electronic Mail

Financial Trading UsersBanking Users FMCG & Utilities

1980s 1990s

Growth of SMTPOrigins of Tibco in

Stock Price Messaging

1980s

IBM Launch MQSeries(now WebsphereMQ)

1992

1970s

IBM TCAM which is the first true solution

Retired 1990!

1971

Machine Assisted Telegraphy

22

A History of Messaging

First Electronic Mail

FMCG & Utilities

1990s

IBM Launch MQSeries(now WebsphereMQ)

1992

Machine Assisted Telegraphy Corporates Large Websites YOU

Noughties Today

Sun Release Java JMS, Reinvigorating

Enterprise Messaging

2001

AMQP Working Group Formed by Investment Banks

2006

Cloud Enables and Drives StormMQ

Adoption

2009

22

Jargon Buster: Messaging

EMail(SMTP, POP3, IMAP)

VoIP(VoiceMail, XMPP)

Texting(SMS)

Instant Messaging(ICQ, MSN, Jabber)

Twitter

Enterprise Service Bus(ESB)

Dynamic OO Languages(eg Ruby)

Message Queuing(MQ)

Message Queuing(MQ)

What do wemean by

Messaging?

23

Essential Terms

Message Queue What’s a Message?

Envelope

Payload

Header

24