+ All Categories
Home > Documents > Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

Date post: 15-Jun-2015
Category:
Upload: josh-long
View: 4,017 times
Download: 3 times
Share this document with a friend
Description:
Internally, some projects at Shopzilla have recently started to leverage the HornetQ messaging system to meet performance and scalability requirements. In this talk, Mark Lui and Josh Long review the basic principles of messaging and distributed communication. They demonstrate how loosely coupled, asynchronous communication can improve performance, scalability and reliability and finally touch on Shopzilla-specific use cases for messaging.
Popular Tags:
32
-1- Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla Mark Lui ( Merchant Services) and Josh Long (Inventory)
Transcript
Page 1: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 1 -

Better Living Through Messaging -

Leveraging the HornetQ Message

Broker at Shopzilla

Mark Lui ( Merchant Services)

and

Josh Long (Inventory)

Page 2: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 2 -

What is Messaging

Page 3: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 3 -

What is Messaging?

Store and Forward

Enables asynchronous communication

• Reduces latency for information transfer

Page 4: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 4 -

What is Messaging?

Loosely coupled distributed communication (Data or Command)

Exchange of messages between software components

Two forms

• Point-to-Point (Queue)

• Publish/Subscribe (Topic)

Page 5: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 5 -

Enterprise Application Integration

Integration of data and services between applications

Communication between information silos

Approaches

• File Transfer

• Shared Database

• Remote Procedure Call

• Messaging

Page 6: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 6 -

Integration Patterns

Patterns

• Data Synchronization

• Workflow

• Portal

Page 7: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 7 -

Enterprise Application Integration

• Approaches integration using messaging

• Enables Event Driven Architecture

• Multiple users of information

Page 8: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 8 -

Message Queue Features

Reliable

Fast

Secure

Duplicate message detection

Flow control – Slow/Fast consumer handling

Wild card destinations

Durable subscription

Scheduled Delivery

Page 9: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 9 -

Message Queue Features

Redelivery handling

• DLQ/Discard

• Delay redelivery

Message Ordering on Topics

Client side load balancing options

• Random

• Round Robin

Message prioritization

Page 10: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 10 -

Introducing HornetQ

New ultra high performance enterprise grade messaging system

from JBoss

Apache V2 License

HornetQ 2.0.GA obtained scores up to 307% higher than

previously published SPECjms2007 benchmark results

“Throughput of 800K small non persistent messages per second

on a single 4 x 2.5GHz core server” – Tim Fox

Page 11: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 11 -

HornetQ – Secret Sauce

LibAIO

• Linux Kernel 2.6 Subsystem

• Able to submit I/O requests without waiting for completion

HornetQ uses an append-only journal using LibAIO

ActiveMQ 6 will support LibAIO

Page 12: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 12 -

ActiveMQ Issues

Difficult to configuration correctly

Prone to hang

Poor persistence options

• KahaDB – better scalability, shorter recovery time

• AMQ Message Store – faster

• JDBC database – too slow

Page 13: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 13 -

High Availability - About Resiliency

Active/Passive replication

HornetQ Example

Page 14: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 14 -

High Availability - About Resiliency

Live Server

Page 15: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 15 -

High Availability - About Resiliency

Backup Server

Page 16: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 16 -

Clustering - About Scalability

Client side load balancing

Page 17: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 17 -

About performance

Offer Bidding Service Performance Data

Page 18: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 18 -

Spring Framework

Connection Options

Page 19: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 19 -

Spring Framework

Alternative Connection Options

Page 20: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 20 -

Spring Framework

JMS Template

Page 21: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 21 -

Spring Framework

Message Driven POJOs : MessageListener

Page 22: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 22 -

Spring Framework

Message Driven POJOs : MessageListener

Page 23: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 23 -

Spring Framework

Message Driven POJOs : SessionAwareMessageListener

Page 24: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 24 -

Spring Framework

Message Driven POJOs : SessionAwareMessageListener

Page 25: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 25 -

Spring Framework

Spring Integration

Page 26: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 26 -

Spring Framework

Spring Integration

Page 27: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 27 -

Messaging at Shopzilla

Merchant Service

• Offer Bidding Service – Using to scale

Inventory

• Offer Service

Dampening request to Coherence

Dampening request to Oracle for write behind

Page 28: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 28 -

Merchant Service’s Order Bidding Service

Demo

Page 29: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 29 -

Inventory’s Offer Service

Demo

Page 30: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 30 -

HornetQ – Lessons Learned

Use the right Connection Factory

Use Spring 2.5.6 or 3.0

For High Availability disable Spring reconnection logic

Be aware of the High Availability restart sequence

Paging is turned off by default

JMX is your friend

Page 31: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 31 -

Possible Future Opportunities

SLA compliance

Unified messaging

Real-time, event driven architectures

Page 32: Better Living Through Messaging - Leveraging the HornetQ Message Broker at Shopzilla

- 32 -

Conclusion

Q and A?

Don't forget to tip your waiter!


Recommended