Simplify, Isolate and Scale with RabbitMQ

Simplify, Isolate and Scale with RabbitMQ

John Blackmore - @johnblackmore

Message Queues• Simplify: Reduce Complexity

• Isolate: Decouple Systems

• Scale: Enable Scalability


What is a message queue?


Simple Queue

• Producer sends messages to a queue

• Consumer receives messages from a queue

• Queue buffers messages un8l they are processed


Why is this Useful?• "Fire and Forget" Commands

• Decoupling Systems

• Asynchronous Processing

• Parallel Processing

• Fault Tolerance


www.rabbitmq.com


RabbitMQ: Exchanges


Exchange Types

Most common Exchange types:

• Direct: Messages are routed to queues along bindings that exactly match the rou9ng key.

• Fanout: Messages are routed to all bound queues, rou9ng key ignored.

• Topic: Messages are routed to queues along bindings that pa@ern match the rou9ng key.


RabbitMQ Examplesgithub.com/johnblackmore/rabbitmq-examples-php


New User Emails• As part of new registra0on a welcome email is sent

• Sending the email takes 1-3 seconds on average

• Under high traffic, the web server can max-out

Underlying Problem:

• Sending email can take a long 0me


New User Emails - Exis/ng Code// register the new user, then send the welcome emailmail('[email protected]', 'Welcome to GroupBuyWidgets!', 'Thanks for signing up.');

• Pro: Simple, easy to read

• Con: Takes a long .me, Blocks un.l email is sent

Solu%on:

• Queue the emails for asynchronous processing


New User Emails - Diagram

• Producer: Registra1on Script on Web Server

• Consumer: Background Worker on Applica7on Server

• Direct Exchange with Single Queue


New User Emails - Produceruse PhpAmqpLib\Connection\AMQPStreamConnection;use PhpAmqpLib\Message\AMQPMessage;

// Connect to RabbitMQ and open a channel$connection = new AMQPStreamConnection('localhost', 5672, 'guest', 'guest');$channel = $connection->channel();

// We will push onto the "email" exchange$channel->exchange_declare('email', 'direct', false, false, false, false, false);

// Compose the message i.e the email.$payload = json_encode([ 'to' => '[email protected]', 'subject' => 'Welcome to GroupBuyWidgets!', 'body' => 'Thanks for signing up.',]);

// Wrap the payload and publish to the exchange$message = new AMQPMessage($payload, ['delivery_mode' => 2]);$channel->basic_publish($message, 'email', 'email.send');


New User Emails - Benchmarks

Apache Benchmark, 100 requests, 5 connec6ons:

Non Queued==========Time taken for tests: 41.232 secondsRequests per second: 2.43 [#/sec] (mean)Time per request: 2061.575 [ms] (mean)

Queued======Time taken for tests: 6.186 secondsRequests per second: 16.17 [#/sec] (mean)Time per request: 309.310 [ms] (mean)


Batch Processing Payments

Problem:

• Lots of data records to process.

• Processing is 3me cri3cal.

Solu%on:

• Parallel Processing

• Mul%ple Consumers


Batch Processing Payments

Number of Consumers | Time to Process-----------------------|---------------------- 1 | 4 - 7 Hours 10 | 25 - 40 Minutes 100 | < 4 Minutes


More RabbitMQ Examples• Publish/Subscribe - Sending the same message to

mul7ple queues.

• Rou7ng & Topic - Receiving messages selec7vely, or based on a paAern.

• Remote Procedure Calls - Receiving replies using anonymous exclusive callback queues.

h"p://www.rabbitmq.com/getstarted.html


Ques%ons?

Twi$er: @johnblackmoreWeb: johnblackmore.com

Email: [email protected]

github.com/johnblackmore/rabbitmq-examples-php


Sources• RabbitMQ Tutorials

• AMQP 0-9-1 Model Explained

• Ge=ng Started with RabbitMQ and CloudAMQP

• DigitalOcean - Install and Manage RabbitMQ


https://www.rabbitmq.com/getstarted.html

https://www.rabbitmq.com/tutorials/amqp-concepts.html

https://www.cloudamqp.com/rabbitmq_ebook.html

https://www.digitalocean.com/community/tutorials/how-to-install-and-manage-rabbitmq

Date post:	07-Jan-2017
Category:	Technology
Upload:	techexeter
View:	44 times
Download:	3 times

Simplify, Isolate and Scale with RabbitMQ

Technology