CQRS & Event Sourcing in the wild (ScotlandPHP 2016)

transcript

CQRS & EVENT SOURCING IN THE WILD

Michiel Rook - @michieltcs

➤ Java, PHP & Scala developer

➤ Consultant, trainer, speaker

➤ Dutch Web Alliance

➤ make.io

➤ Maintainer of Phing

➤ @michieltcs

RAISE YOUR HAND

IF YOU HAVE

heard about CQRS / Event Sourcing

RAISE YOUR HAND

IF YOU HAVE

followed a tutorial, built a hobby project

RAISE YOUR HAND

IF YOU HAVE

followed a tutorial, built a hobby project

used it in production

RAISE YOUR HAND

IF YOU HAVE

TOPICS

➤ Quick recap

➤ Replays and rebuilds

➤ Event versioning

➤ Concurrency

➤ Scale

QUICK RECAP

' Event Sourcing ensures that all changes to application state are stored as a sequence of events.

-Martin Fowler

ACTIVE RECORD VS. EVENT SOURCING

Account number

Balance12345678 �€ 50,00

... ...

Money WithdrawnAccount number 12345678

Amount �€ 50,00

Money DepositedAccount number 12345678

Amount �€ 100,00

Account CreatedAccount number 12345678

COMMANDS TO EVENTS

Deposit MoneyAccount number 12345678

Amount �€ 100,00

class DepositMoney { public $accountNumber; public $amount; }

COMMANDS TO EVENTS

Amount �€ 100,00

function depositMoney(DepositMoney $command) { $this->apply(new MoneyDeposited( $command->accountNumber, $command->amount, date())); }

command handler

COMMANDS TO EVENTS

Amount �€ 100,00

class MoneyDeposited { public $accountNumber; public $amount; public $timestamp; }

command handler

AGGREGATES

class BankAccount { public $accountNumber; public $balance; // ... public function applyMoneyDeposited( MoneyDeposited $event) { $this->balance += $event->amount; }

AGGREGATE STATE

Account number

Balance12345678 �€ 0,00

Money WithdrawnAccount number 12345678

Amount �€ 50,00

Amount �€ 100,00

Account CreatedAccount number 12345678

Account number

Balance12345678 �€ 100,00

Account number

Balance12345678 �€ 50,00

CQRS + EVENT SOURCING

Domain

Event Bus

Event Handlers

Command

Repository

Data Layer

Database Database

Event Store

commands

events

queries DTOs

Aggregates

PROS AND CONS

➤ Domain fit

➤ Testing

➤ Audit trail

➤ Scalability

➤ Complexity

➤ Library support / maturity

DISCLAIMER

REPLAYS AND REBUILDS

PROJECTIONS AND READ MODELS

User Registered

User Unregistered Number of active users?

User Registered

User Deactivated

User Reactivated

Number of active users?User Registered

User Unregistered

User Registered Event HandlerNumber of

active users +1

User Unregistered Event HandlerNumber of

active users -1

Events Event Handler(s) Storage

ELASTICSEARCH

class CompanyRegistered { public $companyId; public $name; public $street; public $city; }

ELASTICSEARCH

function handleCompanyRegistered( CompanyRegistered $event) { $this->elasticsearch->index([ 'index' => 'companies', 'type' => 'company', 'id' => $event->companyId, 'body' => [ 'name' => $event->name, 'address' => [ 'street' => $event->street, 'city' => $event->city ] ] ]); }

READ MODEL UPDATES

➤ New type

➤ New structure

➤ Based on existing events

➤ Generate from scratch?

REBUILDING

Stop application

Remove old read model

Loop over events

Apply to read model

Start application

ZERO DOWNTIME

Loop over existing events

Apply to new read

Apply queued events

Start using new read

New events Queue

CHALLENGE: LONG RUNNING REBUILDS

➤ Alternatives:

➤ In memory

➤ Distributed

➤ Partial

➤ Background

CHALLENGE: SIDE EFFECTS

User RegisteredUser Id 123abc

Email Address test@example.netEvent Handler

Exclude during replays!

CHALLENGE: TRANSACTIONS

Event Handler

Event Handler ?

CHALLENGE: EVENTUAL CONSISTENCY

➤ Asynchronous event handlers

➤ Reads eventually return the same value

➤ Compare with ACID

➤ UI?

EVENT VERSIONING

DILEMMA

➤ New business requirements

➤ Refactoring

➤ New view on events

NEW EVENTS / VERSIONS

➤ No longer relevant

➤ Renamed

➤ Additional or renamed field(s)

➤ Too coarse, too fine

SUPPORT YOUR LEGACY?

➤ Commands can be renamed

➤ Events are immutable

➤ Correct (incorrect) old events with new events

UPCASTING

Event Store

UserRegistered_V1

Upcaster

UserRegistered_V2

Event Handler

UPCASTING

class UserRegistered_V1 { public $userId; public $name; public $timestamp; }

UPCASTING

class UserRegistered_V2 { public $userId; public $name; public $date; }

UPCASTING

function upcast($event): array { if (!$event instanceof UserRegistered_V1) { return []; } return [ new UserRegistered_V2( $event->userId, $event->name, $event->timestamp->format("Y-m-d")) ]; }

REWRITING HISTORY

Load (subset of) events

Deserialize

Modify

Serialize

Save/replace

THINGS TO BE AWARE OF

Upcasting

➤ Performance

➤ Complexity

➤ Existing projections not automatically updated

Rewriting events

➤ Running code that depends on old structure

➤ Breaking serialization

➤ Changing wrong events

CONCURRENCY

CONCURRENT COMMANDS

Withdraw MoneyAccount number 12345678

Amount �€ 50,00

Amount �€ 100,00

PESSIMISTIC LOCKING

Amount �€ 50,00

Amount �€ 100,00

Account number

Balance12345678 �€ 100,00

Account number

Balance12345678 �€ 50,00

wait for lock

OPTIMISTIC LOCKING

Amount �€ 50,00version 1

Amount �€ 100,00version 1

Account number

Balance12345678 �€ 100,00

version 2

ConcurrencyException

PERFORMANCE

➤ Server

➤ Database

➤ Framework

➤ Language

➤ Serializer

STORAGE

➤ #events

➤ #aggregates

➤ #events_per_aggregate

➤ Serializer

➤ Event payloads

➤ Costs

SNAPSHOTS

Events1 Account Created2 Money Deposited3 Money Withdrawn4 Money Deposited

SNAPSHOT5 Money Withdrawn

Events1 Account Created2 Money Deposited3 Money Withdrawn4 Money Deposited5 Money Withdrawn

SHARDING

➤ Aggregate Id, Type, Event Timestamp, ...

➤ Rebalancing

➤ Distribution

ARCHIVING EVENTS

➤ Reduce working set

➤ Inactive / deleted aggregates

➤ Historic / irrelevant events

➤ Cheaper storage

FRAMEWORK COMPARISON

Framework Upcasting Snapshots Replaying

Broadway (PHP) No (PR) No (PR) Not in core

Prooph (PHP) MessageFactory Yes, triggers on

event countExample code,

off line

Axon (Java/Scala)

Upcaster / UpcasterChain

Yes, triggers on event count

Yes, ReplayingCluster

Akka Persistence (Java/Scala)

Event Adapter Yes, decided by actor Yes

QUESTIONS@michieltcs / michiel@make.io

www.touchdownconsulting.nl

https://joind.in/talk/584c0

THANK YOU!@michieltcs / michiel@make.io

www.touchdownconsulting.nl

https://joind.in/talk/584c0

CQRS & Event Sourcing in the wild (ScotlandPHP 2016)

Software