Event Sourcing in einer Microservice-Architektur
Ein Erfahrungsbericht aus der Werkhalle
Michael Omann Senacor Technologies AG
Event Sourcing in einer
Microservice-Architektur
Michael Omann
Michael Omann Architect bei Senacor
Softwareentwickler!
- Retail Banking,- Commercial Banking,- Investment Banking- Banking, Banking, Banking,…
Meine Branchen und Kunden:
ERP
No to NoSQL!
eCommerce Payment Solution
eCommerce Payment Solution?
Web Shop
Sockenkäufer
Server
checkout
Händler
Käufer
Banken
Vermittler
Email Notifications
Mobile TAN
SMS TAN
Photo TAN
Payments
Data Mining
Reporting
Fraud Prevention
Security
Authorization
Ticketsystem
Analysis
Dispute
Anforderungen
Audit
Business Intelligence
B2B
B2EOnline Shops
PSP
Big Data
Checkout Funnel
Mobile Commerce
Payment Gateway
Stammdatenverwaltung
Händler
Käufer
Banken
Vermittler
Email Notifications
Mobile TAN
SMS TAN
Photo TAN
Payments
Data Mining
Reporting
Fraud Prevention
Security
Authorization
Ticketsystem
Analysis
Dispute
Audit
Business Intelligence
B2B
B2EOnline Shops
PSP
Big Data
Checkout Funnel
Mobile Commerce
Payment Gateway
Zahlungssicherheit
Händler
Käufer
Banken
Vermittler
Email Notifications
Mobile TAN
SMS TAN
Photo TAN
Payments
Data Mining
Reporting
Fraud Prevention
Security
Authorization
Ticketsystem
Analysis
Dispute
Audit
Business Intelligence
B2B
B2EOnline Shops
PSP
Big Data
Checkout Funnel
Mobile Commerce
Payment Gateway
Notifications
Händler
Käufer
Banken
Vermittler
Email Notifications
Mobile TAN
SMS TAN
Photo TAN
Payments
Data Mining
Reporting
Fraud Prevention
Security
Authorization
Ticketsystem
Analysis
Dispute
Audit
Business Intelligence
B2B
B2EOnline Shops
PSP
Big Data
Checkout Funnel
Mobile Commerce
Payment Gateway
Business Intelligence
Händler
Käufer
Banken
Vermittler
Email Notifications
Mobile TAN
SMS TAN
Photo TAN
Payments
Data Mining
Reporting
Fraud Prevention
Security
Authorization
Ticketsystem
Analysis
Dispute
Audit
Business Intelligence
B2B
B2EOnline Shops
PSP
Big Data
Checkout Funnel
Mobile Commerce
Payment Gateway
Microservice-Architektur
Account
Bank
Merchant
Checkout SMS
Browser
iOS
android Payments Fraud
ServerClient
Spring Boot
Account
Stammdatenverwaltung
Bank
Merchant
Customer
BankPortal
MerchantPortal
Account
Stammdatenverwaltung
Bank
Merchant
HTTP 200 Ok
Customer
BankPortal
MerchantPortal
HTTP GET /accounts/12345
Account
Stammdatenverwaltung
Bank
Merchant
Customer
BankPortal
MerchantPortal
HTTP 200 Ok
HTTP PUT /banks/555/IBAN/DE98…
HTTP 200 Ok
HTTP GET /accounts/12345
Account
Stammdatenverwaltung
Bank
Merchant
HTTP GET /accounts/12345
HTTP 200 Ok
HTTP PUT /banks/555/IBAN/DE98…
Customer
BankPortal
MerchantPortal
HTTP 200 Ok
HTTP POST /merchants/232/shops
HTTP 201 Created
Account
Stammdatenverwaltung
Bank
Merchant
HTTP GET /accounts/12345
HTTP 200 Ok
HTTP PUT /banks/555/IBAN/DE98…
Customer
BankPortal
MerchantPortal
HTTP 503 Service Unavailable
HTTP POST /merchants/232/shops
HTTP 201 Created
ZahlungsinitierungWeb Shop
Sockenkäufer
Server
checkout
Service B
Service C
Service D
Service A
Account BankMerchant
WebShop
POST /checkouts Checkout
Zahlungsinitierung
Account BankMerchant
WebShop
POST /checkouts Checkout
Zahlungsinitierung
HTTP
Käuferdaten
Account BankMerchant
WebShop
POST /checkouts Checkout
Zahlungsinitierung
HTTP
Käuferdaten Händlerdaten
HTTP
Account BankMerchant
WebShop
POST /checkouts Checkout
Zahlungsinitierung
HTTP
Käuferdaten Händlerdaten
HTTP
Bankdaten
HTTP
Account BankMerchant
WebShop
POST /checkouts Checkout
Zahlungsinitierung
HTTP
Käuferdaten Händlerdaten
HTTP
Bankdaten
HTTP
Authorization
Fraud
External Services
Payments
Microservices
Account BankMerchant
WebShop
POST /checkouts Checkout
Zahlungsinitierung
HTTP
Käuferdaten Händlerdaten
HTTP
Bankdaten
HTTP
Authorization
Fraud
External Services
Payments
HTTP 200 Ok
Microservices
Account BankMerchant
WebShop
POST /checkouts Checkout
Zahlungsinitierung
HTTP
Käuferdaten Händlerdaten
HTTP
Bankdaten
HTTP
Authorization
Fraud
External Services
Payments
JVM JVM JVM
JVM
CheckoutHTTP 200 Ok
Microservices
Account BankMerchant
WebShop
POST /checkouts Checkout
Zahlungsinitierung
HTTP
Käuferdaten Händlerdaten
HTTP
Bankdaten
HTTP
Authorization
Fraud
External Services
Payments
HTTP 200 Ok
Microservices
Account Merchant Bank
WebShop
POST /checkouts Checkout
Zahlungsinitierung
HTTP
Käuferdaten Händlerdaten
HTTP
Bankdaten
HTTP
Authorization
Fraud
External Services
Payments
HTTP 200 Ok
Microservices
Service2Service Kommunikation
Merchant
Checkout
Service2Service Kommunikation
Merchant
Checkout
HTTP GET /merchants/{id}
Service2Service Kommunikation
Merchant
Checkout
HTTP GET /merchants/{id} HTTP GET /merchants/{id}/shops/{shop-id}
Service2Service Kommunikation
Merchant
Checkout
HTTP GET /merchants/{id} HTTP GET /merchants/{id}/shops/{shop-id} HTTP GET /merchants/{id}/account
Service2Service Kommunikation
Merchant
Checkout
HTTP GET /merchants/{id} HTTP GET /merchants/{id}/shops/{shop-id} HTTP GET /merchants/{id}/account
Latenz
Service2Service Kommunikation
Merchant
Checkout
HTTP GET /merchants/{id} HTTP GET /merchants/{id}/shops/{shop-id} HTTP GET /merchants/{id}/account
Latenz
Timeouts?Http 5xx?
Service2Service Kommunikation
Merchant
Checkout
HTTP GET /merchants/{id} HTTP GET /merchants/{id}/shops/{shop-id} HTTP GET /merchants/{id}/account
Latenz
Timeouts?Http 5xx?
Resiliency
Service2Service Kommunikation
Merchant
Checkout
HTTP GET /merchants/{id} HTTP GET /merchants/{id}/shops/{shop-id} HTTP GET /merchants/{id}/account
HTTP GET /merchants/{id}/checkout-information Views
Service2Service Kommunikation
Merchant
Checkout
HTTP GET /merchants/{id} HTTP GET /merchants/{id}/shops/{shop-id} HTTP GET /merchants/{id}/account
HTTP GET /merchants/{id}/checkout-information Views
Service-spezifischeURL-Endpoints
Account BankMerchant
POST /checkouts Checkout
Zahlungsinitierung
Käuferdaten Händlerdaten Bankdaten
Authorization
Fraud
External Services
Payments
HTTP 200 Ok
Microservices
Account BankMerchant
WebShop
POST /checkouts Checkout
Tests?
HTTP
Käuferdaten Händlerdaten
HTTP
Bankdaten
HTTP
Authorization
Fraud
External Services
Payments
HTTP 200 Ok
Microservices
Account BankMerchant
WebShop
POST /checkouts Checkout
HTTP
Käuferdaten Händlerdaten
HTTP
Bankdaten
HTTP
Authorization
Fraud
External Services
Payments
HTTP 5xx Internal Server Error
Tests?
Microservices
Fail!
service-e2e-test
Account BankMerchant
Checkout
low availabilitylow traffic
high availabilityhigh traffic
Verfügbarkeit
Account BankMerchant
Checkout
high availability high traffic
high availabilityhigh traffic
Verfügbarkeit
Account BankMerchant
WebShop
POST /checkouts Checkout
:-(
HTTP
Käuferdaten Händlerdaten
HTTP
Bankdaten
HTTP
Authorization
Fraud
External Services
Payments
HTTP 200 Ok
Event Sourcing
Event Streams
PhoneNumberAdded
AddressUpdated
AccountLimit
Updated
reduce
Account Events Event Store
PhoneNumberAdded
AddressUpdated
AccountLimit
Updated
reduce
Current State
“Account” : { “id: “0123456789” “name: “Mike” “lastName: “Magic”
“limit” : 2000 “address: {…} “phoneNumber: “+49 126 555 555 …
}
Account Events Event Store
PhoneNumberAdded
AddressUpdated
AccountLimit
Updated
reduce“Account” : { “id: “0123456789” “name: “Mike” “lastName: “Magic”
“limit” : 2000 “address: {…} “phoneNumber: “+49 126 555 555 …
}
Account Events Event Store
Current State
Event Store
Current State
Event Store
Current State
“Account” : {
“limit” : “high” “mobile: “+49 126 555 555”
}
Event Store
Current State
id limit age333 2000 25368 1500 34
Event Store
Current State
“Germany” : {
“totalAccounts” : 120.000 “combinedLimit: “302.340.500”
} “Austria” : {
… }
Account BankMerchant
WebShop
POST /checkouts Checkout
Event Sourcing?
HTTP
Käuferdaten Händlerdaten
HTTP
Bankdaten
HTTP
Authorization
Fraud
External Services
Payments
HTTP 200 Ok
Microservices
Account BankMerchant
WebShop
POST /checkouts Checkout
Event Sourcing
Authorization
Fraud
External Services
Payments
HTTP 200 Ok
Account Events Bank Events
Shop Events
optimierte Read ModelsMicroservices
Account BankMerchant
WebShop
POST /checkouts Checkout
Event Sourcing
Authorization
Fraud
External Services
Payments
HTTP 200 Ok
Account Events Bank Events
Shop Events
optimierte Read ModelsMicroservices
Implementierung?
Bus
Write ServicePUT /accounts/333/limit
Read Service A
AccountDBEvent Store
Read Service B Read Service C
UpdateLimit Id: 333 newLimit: 2000
command
“Standardansatz”
Bus
Write ServicePUT /accounts/333/limit
Read Service A
AccountDBEvent Store
Read Service B Read Service C
check/validate
CommandHandler
“Standardansatz”
Event handleCommand(UpdateLimit cmd, Account account) { if (account.isLocked) throw new IllegalStateException("Account locked"); if (cmd.getLimit() > MAX_LIMIT) throw new UnsupportedOperationException("Limit exceeded") ... ... ... return new LimitUpdated(cmd.getId(), cmd.getLimit(), account.getLimit()); }
Bus
Write ServicePUT /accounts/333/limit
Read Service A
AccountDBEvent Store
Read Service B Read Service C
LimitUpdated Id: 333 newLimit: 2000 oldLimit: 1000
Event
Account<<save>>
“Standardansatz”
Bus
Write ServicePUT /accounts/333/limit
Read Service A
AccountDBEvent Store
Read Service B Read Service C
EventHandler
“Standardansatz”
void handleEvent(LimitUpdated evt, AccountQueryModel account) { account.setLimit(evt.getNewLimit());}
Write ServiceWrite ServicePUT /accounts/333/limit
AccountDBEvent Store
Write ServicePUT /accounts/333/limit
AccountDBEvent Store
UpdateLimit Id: 333 newLimit: 2000
command
Write Service
Write ServiceWrite ServicePUT /accounts/333/limit
AccountDBEvent Storecheck/validate
1. Command ändert Account State + =>
Write Service
2. Ableitung eines Events
3. Speichern in AccountDB
4. Speichern in Event Store
Write ServicePUT /accounts/333/limit
AccountDBEvent StoreLimitUpdated
Id: 333 newLimit: 2000 oldLimit: 1000
Event
1. Command ändert Account State
diff( , ) =>
Write Service
AccountDBEvent Store
Event
Account
PUT /accounts/333/limit
symmetrisch?
Verteilte Transaktion!
Write Service
+ =>
Read Service
+ =>
Commands vs Events
UpdateLimit Id: 333 newLimit: 2000
LimitUpdated Id: 333 newLimit: 2000 oldLimit: 1000
Was möchte der Client?Wie interagiert der Client mit dem Server?
Welche Zustandsänderung(en) wurde(n) durch den Command ausgelöst?
Commands vs Events
LockAccount Id: 333
LimitChanged Id: 333 newLimit: 0 oldLimit: 1000
VIPStatusRevoked Id: 333
StatusChanged Id: 333 oldStatus: active newStatus: locked
+ =>
Write Service
AccountDBEvent Store
Event
Account
PUT /accounts/333/limit
symmetrisch?
Write Service
+ =>
Read Service
+ =>
+ => :-(
Commands dürfen keine Zustandsänderung bewirken!
+ => :-(
Events leiten sich aus Commands + Current State ab
:-)+ =>
+ => :-(
Events ändern den Zustand
+ => :-)+ => :-)
Write ServicePUT /accounts/333/limit
AccountDBEvent Store
UpdateLimit Id: 333 newLimit: 2000
command
Write Service
Write ServiceWrite ServicePUT /accounts/333/limit
AccountDBEvent Storecheck/validate
Write ServiceWrite ServicePUT /accounts/333/limit
AccountDBEvent StoreLimitUpdated
Id: 333 newLimit: 2000 oldLimit: 1000
Event
1. Event aus Command angeleitet + =>
2. Speichern in Event Store
3. Publishen der Events zur AccountDB
4. Events ändern Current State + =>
Write ServiceWrite Service
AccountDB
Event Store
Event
Account
PUT /accounts/12345/limit
AccountDB als ‘Projection’ der Events
Keine verteilten Transaktionen
Write ServiceWrite Service
AccountDB
Event Store
Event
Account
PUT /accounts/12345/limit
Race Condition!
AccountDB als ‘Projection’ der Events
Keine verteilten Transaktionen
Write ServicePOST /accounts/12345/transactions
AccountDBEvent Storecheck/validate
Race Condition
“Account” : { “id: “0123456789” “balance” : 700 “currency: EUR
}
2x
PurchaseItem Id: 333 amount: 500 currency: EUR
PurchaseItem Id: 333 amount: 700 currency: EUR
“Account” : { “id: “0123456789” “balance” : -500 “currency: EUR
}
“Account” : { “id: “0123456789” “balance” : 700 “currency: EUR
}
Write ServiceAccountDB
Event Store
ItemPurchased Id: 333 amount: 500 currency: EUR
POST /accounts/12345/transactions
ItemPurchased Id: 333 amount: 700 currency: EUR
Race Condition
2x
PurchaseItem Id: 333 amount: 500 currency: EUR
Write ServicePOST /accounts/12345/transactions
Event Storecheck/validate
Eventual Consistency
Race Condition
2x
Write ServicePOST /accounts/12345/transactions
Event Storecheck/validate
Validierung gegen out-of-date Account
Eventual Consistency
Race Condition
2x
Write ServicePOST /accounts/12345/transactions
Event Storecheck/validate
Current State nicht für Plausibilisierung verwenden
Race Condition
2x
AccountDB Optimierung für lesenden Zugriff
Write Service
Event Store
Read - Write
GET /*
PUT/POST/DELETE /*
für lesenden Zugriff optimiert
Write ServicePOST /accounts/12345/transactions
AccountDBEvent Store
Write Service 3
2x
Write ServicePOST /accounts/12345/transactions
AccountDBEvent Storecheck/validate
Write Service 3
2x
1. Erzeugung des current state aus Events =>
Write ServicePOST /accounts/12345/transactions
AccountDBEvent Storecheck/validate
Write Service 3
2x
1. Erzeugung des current state aus Events =>
Write ServicePOST /accounts/12345/transactions
AccountDBEvent Storecheck/validate
Write Service 3
2x
1. Erzeugung des current state aus Events =>
Write ServicePOST /accounts/12345/transactions
AccountDBEvent Storecheck/validate
Write Service 3
2x
1. Erzeugung des current state aus Events =>
Write ServicePOST /accounts/12345/transactions
AccountDBEvent Storecheck/validate
Write Service 3
2x
1. Erzeugung des current state aus Events =>
2. Sequenzielles Validieren/Plausibilisieren
Write ServicePOST /accounts/12345/transactions
AccountDBEvent Storecheck/validate
Write Service 3
2x
1. Erzeugung des current state aus Events =>
2. Sequenzielles Validieren/Plausibilisieren
400 Bad Request
Event Sourcing im Cluster?
Write ServicePOST /accounts/12345/transactions
Event Store
Cluster
AccountDB
Node 1
Node 22x
Write ServicePOST /accounts/12345/transactions
Event Store
Cluster
Erzeugung des current state aus Events =>
Validierung auf unterschiedlichen Knoten im Cluster
AccountDB
2xNode 1
Node 2
Write ServicePOST /accounts/12345/transactions
AccountDB
Event Store
Cluster
“Account” : { “id: “0123456789” “balance” : -500 “currency: EUR
}
2xNode 1
Node 2
Cluster Sharding
Write Service
Event Store
Cluster Sharding
AccountDB
004
001
003
006005
002
009
007
008
011010
312
997
995
995
999998
996
Shard 1 Shard 2 Shard n
……..
Write Service
Event Store
Cluster Sharding mit Akka
AccountDB
Write Service
Event Store
AccountDB
ShardRegion (Actor)
Cluster Sharding mit Akka
Write Service
Event Store
AccountDB
ShardRegion (Actor)
POST /accounts/12345/transactions
2x
ShardCoordinator (Actor, ClusterSingleton)
PersistentActor
Cluster Sharding mit Akka
Account BankMerchant
WebShop
POST /checkouts Checkout
:-(
HTTP
Käuferdaten Händlerdaten
HTTP
Bankdaten
HTTP
Authorization
Fraud
External Services
Payments
HTTP 200 Ok
Account BankMerchant
WebShop
POST /checkouts Checkout
Stammdaten Eventsourced
Authorization
Fraud
External Services
Payments
HTTP 200 Ok
Account BankMerchant
WebShop
POST /checkouts Checkout
Event Publishing?
Authorization
Fraud
External Services
Payments
HTTP 200 Ok
??? ???
Bus
Write Service
Read Service A
AccountDBEvent Store
Read Service B
Pub-Sub mit Bus
Bus
Write Service
Read Service A
AccountDBEvent Store
Read Service B Read Service C
Pub-Sub mit Bus
Bus
Write Service
Read Service A
AccountDBEvent Store
Read Service B Read Service C
Pub-Sub mit Bus
:-(
Bus
Write Service
Read Service A
AccountDBEvent Store
Read Service B Read Service C
:-(
Pub-Sub mit Bus
Enterprise DB2
MongoDB
Bus
Write Service
Read Service A
AccountDBEvent Store
Read Service B Read Service C
:-(
Pub-Sub mit Bus
Enterprise DB2
:-(
MongoDB
Bus
Write Service
Read Service A
AccountDBEvent Store
Read Service B Read Service C
:-( :-(
Pub-Sub mit Bus
Cluster?
Bus
Write Service
Read Service A
AccountDBEvent Store
Read Service B Read Service C
:-( :-(
Pub-Sub mit Bus
Tests?
Bus
Write Service
Read Service A
AccountDBEvent Store
Read Service B Read Service C
:-( :-(
Pub-Sub mit Bus
Debugging?
Queues!
Write Service
Read Service A
AccountDBEvent Store
Read Service B Read Service C
Point2Point mit Queues
Read Service A
Account Events
Read Service B Read Service C
Point2Point mit Queues
Händler Events
Bank Events
Read Service A
Account Events
Read Service B Read Service C
:-o
Händler Events
Bank Events
Read Service A
Account Events
Read Service B Read Service C
:-o
Händler Events
Bank Events
Operations?
Poll???
Polling mit Akka
Read Service A Read Service B Read Service C
Write Service
AccountDBEvent Store
Polling mit Akka
Read Service A Read Service B Read Service C
Write Service
AccountDBEvent Store
Cluster Singleton (Actor)
Polling mit Akka
Read Service A Read Service B Read Service C
Write Service
AccountDBEvent Store
Cluster Singleton (Actor)Poll
Poll
Poll
Account BankMerchant
WebShop
POST /checkouts CheckoutAuthorization
Fraud
External Services
Payments
HTTP 200 Ok
Account BankMerchant
WebShop
POST /checkouts CheckoutAuthorization
Fraud
External Services
Payments
HTTP 200 Ok
Poll Poll Poll
Poll for the win
Account BankMerchant
Auditlog
Audit
Account BankMerchant
Business Intelligence
Audit BI