Building DistributedApplications Using CQRS

Principals

David Hoerster

About MeC# MVP (Since April 2011)Sr. Director of Web Solutions at RGPConference Director for Pittsburgh TechFestCo-Founder of BrainCredits (braincredits.com)Member of Advisory Board for Microsoft’s p&p CQRS GuidancePast President of Pittsburgh .NET Users Group and organizer of recent Pittsburgh Code Camps and other Tech EventsTwitter - @DavidHoersterBlog – http://geekswithblogs.net/DavidHoersterEmail – david@agileways.com

Application EvolutionI have an application that reads and writes informationIt has user authentication/authorization servicesIt has search services

Right now, it’s monolithic

I want to make it more distributed. Why?Performance / ScalabilityMaintenenaceExtensibility

What is a Distributed Application?

A system that uses multiple processes to run as a single unit

ServicesPhysical MachinesThe “Cloud”

Search is to be a distributed element of our applicationCreepers

DuplicationComplexity

Application Evolution

Client Black Box System

Data Store

SearchingBusiness

Logic

User Services

Application Evolution

Client Application System Data Store

SearchingBusiness

Logic

User Services

Application Evolution

Client Application System Data Store

Searching

Business Logic

User Services

Reposi

tory

Application Evolution

Client

Application System

Data Store

Searching

Business Logic

User Services

Reposi

tory

Repository

Application Evolution

ClientApplication

SystemData Store

Searching

Business Logic

User Services

Repository

Repository

Repository

Building a Distributed Application

Not separating pieces allows similar functionality to bleed through layers

Business logic duplicatedValidation logic duplicatedWET (anti-DRY) behavior

Clipboard Inheritance or Copy and Paste Development

Building a Distributed Application

DDD to the Rescue?Domain Driven Design (DDD)Provides foundation for building distributed applicationsClearly define roles and boundariesSRP is embracedConcepts involved:

Domains (Aggregate Roots)Entities and Value ObjectsBounded Contexts

Application Evolution (Reminder)

ClientApplication

SystemData Store

Searching

Business Logic

User Services

AppRepo

UserRepo

SearchRepo

Application Evolution

ClientApplication

SystemData Store

Searching

Business Logic

User Services

AppRepo

UserRepo

SearchRepo

Search Store

Bounded Context

Application StatusBreaking out parts into smaller componentsStill some duplicationComplexity

How is search store builtMoving data to search store (triggers? .NET functions?)

DependencyOverall flow of data

Repos probably have all CRUD operationsBusiness logic handles all CRUDTwo way traffic throughout

Application StatusTwo way traffic isn’t badBUT…

Potential Application Evolution

Client

Application System

Data Store

Search Svc

Business Logic

User Services

AppRepo

UserRepo

SearchRepo

Search Store

Search

QueueHandler

WritesReads

Bounded Context

Application BreakoutSearch is now practically standaloneEntry points for querying and writing

Writing to search repository is a messageQuerying is a service call or direct invocation

In theory, search is self-contained and can be distributedHow to best model this interaction?CQRS could be a solution

What is CQRS?CommandQueryResponsibilitySegregation

Is it a pattern or an architecture?

What is CQRS?

Command Query Responsibility SegregationBased upon Bertrand Meyer’s Command Query Separation (CQS) principle:

“every method should either be a command that performs an action, or a query that returns data to the caller, but not both. In other words, asking a question should not change the answer” (Meyer)

In essence, commands return acknowledgement; queries return data.

What is CQRS?Taking this further, CQRS describes a system where

Commands are messagesQueries go against a read model

Since commands are returning void, it allows for the message to be transformed into the read modelThis flips the conventional wisdom of fast writes and slow reads

What is CQRS?

CQRS essentially splits your application into two parts

The Read SideThe Command Side

Read Side = requests dataCommand Side = issues commands

What is CQRS?Common Architecture Issues

Business Logic Bleeds Through LayersValidation Logic Ends Up EverywhereCopy and Paste DevelopmentApps That Cater To Modification OperationsLeads To General Complexity Between Tiers

CQRS Simplifies This With Classes Performing Specific Duties (SRP)

The Why And When

Why?Your system is task-basedWant to cleanly separate commands and queriesDifferent teams? Different skill levels?

When?Not just a CRUD appDiscrete service/system that needs isolated, or can

be reused

The potential for eventual consistency is OK

Application Evolution

ClientApplication

SystemData Store

Search Req’s

Business Logic

User Services

AppRepo

UserRepo

SearchRepo

Search Store

Search

Queue

Cmd Handler

Writes

Reads

Bounded Context

Cmd

Search Domain

and Events

What is CQRS?Client

Read Store

Write Store

Repository / Read Model

Command

Command Handler

Domain

Event(s)Event

Handler(s)

Read Model

CQRS Pieces

CommandTells the system you want to do somethingImperative tense

public class CreatePlayer : CommandBase { public Int32 LahmanId { get; set; } public String PlayerId { get; set; } public String FirstName { get; set; } public String LastName { get; set; } public String NickName { get; set; } public String College { get; set; } public String Bats { get; set; } public String Throws { get; set; } public Int32 Height { get; set; } public Int32 Weight { get; set; } public DateTime? Debut { get; set; } public DateTime? FinalGame { get; set; } }

CQRS Pieces

Command HandlersHandles commandsRoutes to the appropriate DomainPrepares the DomainOne command maps to one domain

CQRS Pieces

DomainYour “business logic” componentBusiness rulesCan populate itselfPersists itselfTies in well with Event Sourcing with event replays

CQRS Pieces

EventsGenerally raised by domainSays that something happenedPast tenseMore than one event can be raised by a domain method

public class PlayerCreated : EventBase { public String FullName { get; set; } public String PlayerId { get; set; } }

CQRS Pieces

Event HandlersHandles events raised by the domainCan persist to a read model, send an email, submit other commands, etc.More than one event handler can be mapped to an event

public class PlayerHandler : IHandle<PlayerCreated>, IHandle<PlayerYearAdded> { public void Handle(PlayerCreated theEvent) { //do something }

public void Handle(PlayerYearAdded theEvent) { //do something else } }

BenefitsMove towards SRPComplexity Within Goes DownMaintenance Can Be EasierReduction in Hidden IntentIsolation of Application Parts (Resiliency)

PotentialsIncorporating Event Sourcing

Projecting to Denormalized Views

DrawbacksMore classesPotentially more projectsLearning curve when initially exposedGoes against majority think

MisconceptionsEverything is asyncEventual consistency is badEntire application must use this

Projecting to Denormalized View• Get all of the orders

for user “David” in last 30 days

Customer (SalesLT)CustomerID

NameStyle

Title

FirstName

MiddleName

LastName

Suffix

CompanyName

SalesPerson

EmailAddress

Product (SalesLT)ProductID

Name

ProductNumber

Color

StandardCost

ListPrice

Size

Weight

ProductCategoryID

ProductModelID

SellStartDate

SellEndDate

ProductCategory (SalesLT)ProductCategoryID

ParentProductCategoryID

Name

rowguid

ModifiedDate

ProductModel (SalesLT)ProductModelID

Name

CatalogDescription

rowguid

ModifiedDate

SalesOrderDetail (SalesLT)SalesOrderID

SalesOrderDetailID

OrderQty

ProductID

UnitPrice

UnitPriceDiscount

LineTotal

rowguid

ModifiedDate

SalesOrderHeader (SalesLT)SalesOrderID

RevisionNumber

OrderDate

DueDate

ShipDate

Status

OnlineOrderFlag

SalesOrderNumber

PurchaseOrderNumber

AccountNumber

CustomerID

ShipToAddressID

BillToAddressID

ShipMethod

CreditCardApprovalCode

SubTotal

TaxAmt

Freight

Projecting to Denormalized View

Get all the orders for user ‘David’ in last 30 days

SELECT c.FirstName, c.MiddleName, c.LastName, soh.SalesOrderID, soh.OrderDate, sod.UnitPrice, sod.OrderQty, sod.LineTotal, p.Name as 'ProductName', p.Color, p.ProductNumber, pm.Name as 'ProductModel', pc.Name as 'ProductCategory', pcParent.Name as 'ProductParentCategory'FROM SalesLT.Customer c INNER JOIN SalesLT.SalesOrderHeader soh

ON c.CustomerID = soh.CustomerIDINNER JOIN SalesLT.SalesOrderDetail sod ON soh.SalesOrderID = sod.SalesOrderIDINNER JOIN SalesLT.Product p ON sod.ProductID = p.ProductIDINNER JOIN SalesLT.ProductModel pm ON p.ProductModelID = pm.ProductModelIDINNER JOIN SalesLT.ProductCategory pc ON p.ProductCategoryID = pc.ProductCategoryIDINNER JOIN SalesLT.ProductCategory pcParent ON pc.ParentProductCategoryID = pcParent.ProductCategoryID

WHERE c.FirstName = 'David'AND soh.OrderDate > (GETDATE()-30)

Projecting to Denormalized View

Wouldn’t it be great if it were something like this?

SELECT FirstName, MiddleName, LastName, SalesOrderID, OrderDate, UnitPrice, OrderQty, LineTotal, ProductName, Color, ProductNumber, ProductModel, ProductCategory, ProductParentCategoryFROM CustomerSalesWHERE FirstName = 'David'

AND OrderDate > (GETDATE()-30)

How CQRS Can Help Traditional Architecutre

Many apps cater to modification operationsReads sufferWhat would make querying faster?

Business logic and domain objects bleed through layers as a result, too.

Validation logic may end being in multiple placesThe evil of cut-and-paste development

General complexity within tiers

Demo App Architecture

ClientWeb

Server (Nancy)

Solr Service

MongoDB

MSMQ

Cmd

Command

HandlerDomain

Event Handler

Event

Repository

(SolrNet)

Cmd

DemoLooking at CQRS with Search

ResourcesConference Reservation Example on GitHub https://github.com/DavidHoerster/Conference

CQRS Journey – https://github.com/mspnp/cqrs-journey-docCQRS Info Site – http://cqrs.wordpress.com/DDD/CQRS Google Group - http://groups.google.com/group/dddcqrs

Udi Dahan’s CQRS article “Clarified CQRS” – http://www.udidahan.com/2009/12/09/clarified-cqrs/

Rinat Abdullin’s CQRS information – http://abdullin.com/cqrsDistributed Podcast - http://distributedpodcast.com/

What is CQRS?Client

Read Model Store

Event Store

Repository Command Service

Domain (ARs)

Denormalizer

Event Handler

Command Issued – ack/nackData Requested/Returned

Command SideQuery Side