Date post: | 20-May-2015 |
Category: |
Technology |
Upload: | eduardo-pelegri-llopart |
View: | 2,529 times |
Download: | 1 times |
Linda SchneiderTechnical LeadSun Microsystems, Inc.
Using OpenMQ
Copyright Sun Microsystems Inc. All Rights Reserved. [2]
What will be covered ?• An introduction to OpenMQ.• A Customer example.• Basic customer requirements.• Building a piece of the infrastructure.
•Warnings:> No in depth coverage > Assumes basic JMS knowledge
Copyright Sun Microsystems Inc. All Rights Reserved. [3]
What is OpenMQ ?
Copyright Sun Microsystems Inc. All Rights Reserved. [4]
• Allows heterogenous applications to reliability and asynchronously pass data between each other.•Open Source Java Message Service (JMS)
implementation (+ additions)• Default Messaging Provider for Glassfish• Useful on its own for standalone JMS applications
Overview
Copyright Sun Microsystems Inc. All Rights Reserved. [5]
Overview (cont.)• Enterprise level quality (>8 years in development)•Open Source since JavaOne 2006• Available as a supported product: Sun Java System
Message Queue (SJSMQ)
Copyright Sun Microsystems Inc. All Rights Reserved. [6]
What this looks like ....
A set of loosely coupled applications talking through OpenMQ apis
Copyright Sun Microsystems Inc. All Rights Reserved. [7]
Ok – maybe a little about JMSSome basic concepts of JMS:• Producers and Consumers• Destinations• Consumer Types• Reliability
Copyright Sun Microsystems Inc. All Rights Reserved. [8]
What are Producers and Consumers?• Producers >Create messages which are given to the “provider” (e.g.
OpenMQ) to deliver.>Once the message has been handed off to the provider it
goes off and does other processing.
• Consumers>Are sent messages from the “provider” to process>Notify the provider when they are done with the message.
Copyright Sun Microsystems Inc. All Rights Reserved. [9]
So, what are destinations ?• Named buckets to hold groups of messages.•Messages are sent to “destinations”>Topics: each messages goes to all consumers>Queues: a message goes to only one consumer
Copyright Sun Microsystems Inc. All Rights Reserved. [10]
What are the types of Consumers ?• Receivers are delivered messages from a Queue• Subscribers are delivered messages from a Topic>Normal subscribers only receive messages sent while they
are running>Messages for durable subscribers are maintained when
the subscriber is not running
Copyright Sun Microsystems Inc. All Rights Reserved. [11]
And what does reliability mean ?•Messages can be set to be persistent or non-
persistent>Persistent messages are stored>Non-persistent messages are not and could be lost if a
server crashed•Messages can be transacted or non-transacted>Transactions guarantees that a set of messages will be
processed or not processed (not partially processed) if a server fails>XA transactions allow other resources (e.g. databases) to
be processed in a group with messages. [Requires an Application Server]
Copyright Sun Microsystems Inc. All Rights Reserved. [12]
Tell Me More• Developer and User discussion forums• Stable builds with product releases• Early access, promoted builds available>New features, and fixes
• Dual license support (GPL v2 and CDDL)•Open source version of Java MQ is available from
http://mq.dev.java.net
Copyright Sun Microsystems Inc. All Rights Reserved. [13]
Using OpenMQ
Copyright Sun Microsystems Inc. All Rights Reserved. [14]
An exampleTo mimic problems faced in designing applications, an example:• Represents a complex system with loosely
connected applications• Utilizes various types of messaging• Is easy to understand• Is at least minimally interesting
Copyright Sun Microsystems Inc. All Rights Reserved. [15]
Our Example: Santa Claus, IncWhy, you ask ?
• Even if you don't believe in Santa Claus, you must still understand that delivering all those presents would be a daunting task• And while its not Christmas which comes but once a
year, but requires year round planning and preparation.• Just because Santa Claus lives at the North Pole,
doesn't imply he can't use technology
Copyright Sun Microsystems Inc. All Rights Reserved. [16]
Overall System RequirementsSanta Claus, Inc. software applications need do the
following:• Handle gift selection and delivery• Manage resources e.g.> gifts> reindeer> Elves
• Track general status information> how many days before Christmas> etc.
Copyright Sun Microsystems Inc. All Rights Reserved. [17]
What are we doing:• Focusing on handling christmas gift processing• Steps to design it include:>Determining the high level operation>Coming up with the name and type of destinations>Determining models used for the messaging>Determining load characteristics>Looking at code for some components
Copyright Sun Microsystems Inc. All Rights Reserved. [18]
Defining the High Level Operation of the System
Copyright Sun Microsystems Inc. All Rights Reserved. [19]
What do we need to do ?
Copyright Sun Microsystems Inc. All Rights Reserved. [20]
What do we need to do ? (cont.)
Copyright Sun Microsystems Inc. All Rights Reserved. [21]
How can Santa Do it ?
Copyright Sun Microsystems Inc. All Rights Reserved. [22]
What are the destinations ?
Copyright Sun Microsystems Inc. All Rights Reserved. [23]
The Child Queue
Copyright Sun Microsystems Inc. All Rights Reserved. [24]
The Naughty/Nice Queues
Copyright Sun Microsystems Inc. All Rights Reserved. [25]
The Wrap Queue
Copyright Sun Microsystems Inc. All Rights Reserved. [26]
The Stuff to Pack Queue
Copyright Sun Microsystems Inc. All Rights Reserved. [27]
Select-a-gift Queues
Copyright Sun Microsystems Inc. All Rights Reserved. [28]
Topic LogChild
Copyright Sun Microsystems Inc. All Rights Reserved. [29]
A quick overview to design patterns
Copyright Sun Microsystems Inc. All Rights Reserved. [30]
Some basic design patterns:Pattern Description
Request/Reply
Step OperationsBroadcast One message goes to many consumers
ConduitBatch Messages are processed in a chunk
Time Critical/Sensitive
Message is sent to another application who sends back a responseMessages go through several iterations, the message is persisted at key points where processing it again would be expensive
Multiple consumers send messages to a single destination
Messages must be processed within a short period of time (e.g. under an hour) and can not be lost
Copyright Sun Microsystems Inc. All Rights Reserved. [31]
More things to think about:• Use persistent messages if it can not afford to be lost• Use non-persistent messages for:> Non-critical step messages (when it can be repeated> Request/Reply> Anytime a message can be lost on a server crash
• Use durables for Topics when it may need to be retrieved later• Use normal or XA transactions when multiple things
must process together:> XA if it includes other resources like databases
Copyright Sun Microsystems Inc. All Rights Reserved. [32]
Processing Queue Child• Conduit: many producers to one queue• Persistent: would be time consuming to lose message
Copyright Sun Microsystems Inc. All Rights Reserved. [33]
Processing Naughty and Nice• Step Pattern: one step of it• Naughty Queue: Non-
Persistent > its OK if a child who is bad
misses their coal• Nice Queue: Persistent. > They must get their present.
Copyright Sun Microsystems Inc. All Rights Reserved. [34]
Processing Nice• Step Pattern: more steps of it• Multiple resources so XA
Copyright Sun Microsystems Inc. All Rights Reserved. [35]
Processing Nice (Select a gift)• Request/Reply Pattern• Non-persistent• Action repeated on failure
Copyright Sun Microsystems Inc. All Rights Reserved. [36]
The Wrap Queue • Step operation• Persistent: end of an expensive set of steps that they
don't want to repeat
Copyright Sun Microsystems Inc. All Rights Reserved. [37]
The Log Child Topic• Broadcast Pattern• Persistent because santa wants his database
accurate
Copyright Sun Microsystems Inc. All Rights Reserved. [38]
In this example:• The batch pattern was not used> Santa does use It for processing HR updates for the
elves• The time sensitive/critical data pattern was not used:> Santa does use it during present delivery on christmas
eve to track where he is• Because he has no time sensitive/critical data,
reliability is important however data availability isn't for gift processing
Copyright Sun Microsystems Inc. All Rights Reserved. [39]
Performance Requirements• 22 billion kids• 364 days for preparation (since christmas is taken) > 31,526,000 seconds
• 70 children/second must be processed• Assume 60% are “nice”• Assume 40% downtime to cover outages and normal
processing (so goal is approx 100 kids/second)
Copyright Sun Microsystems Inc. All Rights Reserved. [40]
Performance Requirements (cont)• Naughty Kids use> 1 Persistent queue (child)> 1 Non-persistent queue (naughty)> 1 Persistent Topic (log child)
• Nice Kids Use:> Persistent Queue (child)> Persistent Queue (nice)> 2 Non-Persistent Queues (Inventory request and reply
queues)> Non-persistent queue (Wrap)> 1 Persistent Topic (log child)
Copyright Sun Microsystems Inc. All Rights Reserved. [41]
The cold hard requirements• Messages:> Child: 100 msgs/second (persistent)> Naughty: 40 msgs/second (non-persistent)> Log Child: 100msgs/second (persistent)> Nice: 60 msgs/second (persistent in XA transaction)> Inventory request/reply: 60 msgs/second *2 (non-
persistent)> Wrap: 60 msgs/second (persistent)
• TOTALS:> Persistent: 380 msgs/second> Non-persistent: 160 msgs/second
Copyright Sun Microsystems Inc. All Rights Reserved. [42]
Some Sample Code
Copyright Sun Microsystems Inc. All Rights Reserved. [43]
Sending the “Child” messagepublic void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
// retrieve initial context (ic) QueueConnectionFactory qcf = (QueueConnectionFactory) ic.lookup("MyConnectionFactory"); Queue destQueue = (Queue)ic.lookup("Child"); QueueConnection connection = qcf.createQueueConnection(); try { QueueSession session = connection.createQueueSession( False, Session.AUTO_ACKNOWLEDGE); QueueSender sender = session.createSender(destQueue); MapMessage msg = session.createMapMessage(); // Set each item msg.setString(“firstname”, request.getParameter(“firstname”)); // … retrieve other properties …; sender.send(msg); } finally { connection.close(); }}
Copyright Sun Microsystems Inc. All Rights Reserved. [44]
Processing the “nice” queuepublic void onMessage(Message inMessage) { TextMessage msg = null; try { //Message is of type text and has a unique child id msg = (TextMessage) inMessage; String id = msg.getText(); String[] list = db.getList(id); // makes SQL call String item = null; if (list == null) { // no list, send request String item = getListItem(); //next slide } else { item = list[0]; } //update inventory db.updateInventory(item, id);//makes SQL call // put on packing list pack(item, id); } catch (Exception e) { // things went wrong roll back mdc.setRollbackOnly(); }}
Copyright Sun Microsystems Inc. All Rights Reserved. [45]
Processing the “nice” queue (step 2)public String getListItem(String childid) throws Exception { QueueConnectionFactory factory = jndiContext.lookup(“MyQueueFactory”); QueueConnection qc = factory.createQueueConnection(); qc.start(); QueueSession session = qc.createSession(true, Session.AUTO_ACKNOWLEDGED); Queue q = session.createQueue(“RandomPresent”); Queue reply = session.createTemporaryQueue(); // get sender and receiver QueueSender sender = session.createSender(q); QueueReceiver receiver = session.createReceiver(q); //send message and wait TextMessage m = session.createTextMessage(childid); m.setJMSReplyTo(reply); //send the message sender.send(m); TextMessage back = (TextMessage) receiver.receive(60*1000); // wait a minute if (back == null) { didn't get anything throw new Exception(“Nothing”); return back.getText();}
Copyright Sun Microsystems Inc. All Rights Reserved. [46]
Processing the “nice” queue (step 3)
public String pack(String item, String child_id) throws JMSException{ QueueConnectionFactory factory = jndiContext.lookup(“MyQueueFactory”); QueueConnection qc = factory.createQueueConnection(); QueueSession session = qc.createSession(true, Session.AUTO_ACKNOWLEDGED);); Queue q = session.createQueue(“Pack”);
// get sender QueueSender sender = session.createSender(q);
//send message MapMessage m = session.createMapMessage(childid); m.setString(“child_id”, child_id); m.setString(“present”, item); sender.send(m);}
Copyright Sun Microsystems Inc. All Rights Reserved. [47]
You'll need to fill in the rest
Copyright Sun Microsystems Inc. All Rights Reserved. [48]
More Information
Copyright Sun Microsystems Inc. All Rights Reserved. [49]
OpenMQ -- More Information•Visit the product webpage>http://sun.com/software/products/message_queue•Join the Open Message Queue project>https://mq.dev.java.net•Browse the product documentation>http://docs.sun.com/app/docs/coll/1307.3•Take the free technical training>http://www.sun.com/training/catalog/courses/WMT-SMQ-1491.xml
Copyright Sun Microsystems Inc. All Rights Reserved. [50]
Related Information•Java Composite Application Platform Suite>http://sun.com/software/javaenterprisesystem/javacaps/•Java System Identity Manager>http://sun.com/software/products/identity•Project GlassFish >https://glassfish.dev.java.net/•The Aquarium, A community forum>http://blogs.sun.com/theaquarium/