Microservice PlumbingWithRabbitMQASYNC MESSAGINGFORMICROSERVICES

@jsonrowjasonrowe@gmail.com

https://github.com/JasonRowe

AboutMe

@JSONROW

jasonrowe.com

PastAsync MessagingExperience• MSMQ• NServiceBus• MassTransit

CurrentTechStack• .NETCore• RabbitMQ• Docker• Vue.JS

RabbitMQ OverviewWhatisRabbitMQ?

@JSONROW

WhatisAMQP?AdvancedMessageQueueProtocol

Openinternetprotocolforbusinessmessaging

Binarywire-levelprotocol

AMQPwasdesignedwiththefollowingmaincharacteristicsasgoals:

Security

Reliability

Interoperability

Standard

Open

@JSONROW

AMQPModel

Exchange->Binding->[Queue]

Exchanges

Receivesandroutesmessages

Binding

Definestherelationshipbetweenexchangeandqueue

Queues

Storesmessagesuntiltheyareprocessed

@JSONROW

RabbitMQ Hello,world.

@JSONROW

RabbitMQ – changingamonolith

@JSONROW

P/C P/C P/C

P CX Q

P

C

C

BigChallenge- changingthecommunicationpattern.

MessageDurabilityPublisherconfirms

Durabilityexchangesandqueues

message acknowledgements

Deliveryandprocessingconfirmation

@JSONROW

MessagePublishing

@JSONROW

MessagePublishinginRabbitMQ,by GavinMRoy

ReceivingMessages

@JSONROW

MessagePublishinginRabbitMQ,by GavinMRoy

HighAvailability

EnablePublisherconfirms

UseDurablequeues,exchanges

Extensionstoconsider- consistentlyhashandsharding

DonotenableHiPE

Prefetch configuration

MultinodeswithHApolicy

@JSONROW

HighPerformanceEnableHiPE

DisableLazyQueue

Shortqueues

TransientMessages

DisableHA

@JSONROW

MyRecentRabbitMQ successstory.RefiningaservicesboundariesandswitchingtoRabbitMQ

@JSONROW

RecentRabbitMQ successstory.

0

10,000

20,000

30,000

40,000

50,000

60,000

70,000

Run1 Run2 Run3

AverageTime(m

s)

Baselinetestruns

Legacy RabbitMQ

@JSONROW

DirectExchange

Publisher DirectexchangeCreatedModifiedDeleted

RoutingKeys

Queue

Queue

Queue

Consumer

Consumer

Consumer

@JSONROW

Modified

Bindings

Fanoutexample

P

C

Fanoutexchange

QueueA

QueueB

QueueC

@JSONROW

C

C

TopicexchangeexampleWildcardroutingbyroutingkey*(star)cansubstituteforexactlyoneword.#(hash)cansubstituteforzeroormorewords.

@JSONROW

P Topicexchange

Q

Q

Q

C

C

C

*.*.mdc

Bindings

AnalysisRequestExample

@JSONROW

P X

Q

cnc.emea

C

AnalysisRequest

Q

Q

C

C 3DPAmericaAnalysis

CNCUKAnalysis

InjectionMoldingJapanAnalysis

Q C AnalysisArchive

CompetingConsumers

@JSONROW

Rabbitmq UseCasesGood

EventStream

Connectingappsnewandoldwrittenindifferentlanguages

WhenyouareuncertainRabbitMQ isacertainchoice

multiprotocolandpatterns

Federationbetweendatacenters

Bad

Largebinaryvideosandimages

Useasadatastorelargequeues

@JSONROW

LessonsLearnedMajorRabbitMQ upgradesandErlang upgradescomeswithsomedowntimeevenonmulti-nodeclusters.(bluegreen)

Connections- don'topennewconnectionswitheverymessage.Don'tletqueuesgrow10,000messagesistoomuchlimitwithTTLormaxsizeSendpersistentmessagesandusedurablequeuesandexchangeUsepublisherconfirmswhenitmakessenseTakealookatprefetch andadjustifneeded

@JSONROW

MonitoringQueueLength

MissingConsumers

Howlongmessageshavebeeninqueue

CPU,RAM,Disk

https://www.rabbitmq.com/production-checklist.html

@JSONROW

Summary• RabbitMQ isagreatsolutionforconnectingasync messagingforMicroservices

• RabbitMQ providesoptionsforhighperformanceandhighreliability

• Messagingdoesn’tneedtobehard,itcanbeboring,andboringisgood.

• It’sageneralpurposemessagingsystemthatmaybetherightchoicetohelpbreakupyourmonolithorconnectourmicroservices

@JSONROW

Questions?

@JSONROW