Microservice Scars:Lessons we learnt

Abhaya Chauhan@AbhayaChauhan

www.abhayachauhan.comJoshua Toth@TothJoshuaJ

Question Policy

Let’s go on a Journey

Context

PageUp

Context

19 years

Context

The Monolith

Context

Fragility

The Monolith

Context

Agility

The Monolith

Context

Complexity

The Monolith

Context

Innovation

The Monolith

Context

#Scale

Context

If you can’t build a monolith, what makes you think

microservices are the answer?

Simon Brown

Context

First rule of Distributed Computing

Context

Don’t do it.

Context

Why Microservices?

Context

Reduce time to market

Context

Autonomous Services

Context

EnableEvolutionary Architecture

Context

Sprint Zero

Decisions

Sprint Zero

The stack

Sprint Zero

Sprint Zero

Sprint Zero

Frontend

Sprint Zero

Backend

Frontend

Sprint Zero

Backend

DB

Frontend

Sprint Zero

Elastic Beanstalk

S3

Self hosted

Sprint Zero

Pact

Sprint Zero

Infancy

Sprint Zero

Limitations

Sprint Zero

Investment

Sprint Zero

Deployment

Sprint Zero

Continuous

Sprint Zero

Phabricator

Sprint Zero

Sprint Zero

Pipeline

Sprint Zero

Sprint Zero

A first

Sprint Zero

F# Make (Fake)

Sprint Zero

Monolith Deployment

Scripts

Sprint Zero

Sharing != caring

Sprint Zero

Sprint Zero

Sprint Zero

Clear separation

Sprint Zero

Development Practices

Sprint Zero

TDD

Sprint Zero

BDD

Sprint Zero

Sprint Zero

Elastic Beanstalk

Full stack

Elastic Beanstalk

Visualization

Elastic Beanstalk

Monitoring

Elastic Beanstalk

Monitoring

Elastic Beanstalk

Monitoring

Elastic Beanstalk

Elastic Beanstalk

No Downtime

Elastic Beanstalk

50 shades of “fuck we can’t roll this back”

Grey

Elastic Beanstalk

Authentication

IdentityServer3(Locksmith)

OAuth2

Immediate return

OAuth2

Messaging

Messaging

Messaging

Messaging

Request Response Pain

Messaging

Resource intensive

Messaging

Fault Tolerance

Messaging

1 to 1 communication

Messaging

Event Driven Architecture

Messaging

Queuing (SQS)

Messaging

Not Resource Intensive

Fault Tolerant

1 - many Communication

Messaging

Not Resource Intensive

Fault Tolerant

1 - many Communication

Messaging

Not Resource Intensive

Fault Tolerant

1 - many Communication

Messaging

Not Resource Intensive

Fault Tolerant

1 - many Communication

Messaging

Simple Notification Service (SNS)

Messaging

Push messages

Messaging

Mobile Devices

Messaging

Mobile DevicesSMS

Messaging

Mobile DevicesSMS

Emails

Messaging

Mobile DevicesSMS

EmailsHTTP Endpoints

Messaging

Mobile DevicesSMS

EmailsHTTP Endpoints

Lambda

Messaging

Mobile DevicesSMS

EmailsHTTP Endpoints

LambdaQueues

Messaging

Messaging

Messaging

Messaging

Messaging

SNS + SQS = Amazing Service Bus

Not Resource Intensive

Fault Tolerant

1 - many Communication

Messaging

Circuit Breaker

Messaging

Messaging

CloudWatch

Messaging

Historical Events

Messaging

A store for events

Messaging

Client subscriptions

Messaging

DynamoDb

Fully Managed NoSQL DB

DynamoDb

Fully Managed NoSQL DB

DynamoDb

Fast and Predictable

DynamoDb

Distributed

DynamoDb

Key-ValueDocument

DynamoDb

Hash (Range)

Partition 1 Partition 2 Partition 3 Partition N

Read & Write capacity equally distributed

Max 400kb

DynamoDb

1 Read unit = 4KB/s1 Write unit = 1KB/s

DynamoDb

CQRS

DynamoDb

DynamoDb

Oh shit!

DynamoDb

Write is 20x more expensive

DynamoDb

DynamoDb

{"Key":"Mauritania","Value":"42803"},{"Key":"Mauritius","Value":"42804"},{"Key":"Mexico","Value":"42805"},{"Key":"Micronesia","Value":"42806"},{"Key":"Monaco","Value":"42807"},{"Key":"Mongolia","Value":"42808"},{"Key":"Montenegro","Value":"42809"},{"Key":"Montserrat","Value":"42810"},{"Key":"Morocco","Value":"42811"},{"Key":"Mozambique","Value":"42812"},{"Key":"Namibia","Value":"42813"},{"Key":"Nauru","Value":"42814"},{"Key":"Nepal","Value":"42815"},{"Key":"Netherlands","Value":"42816"},{"Key":"Netherlands Antilles","Value":"42817"},{"Key":"New Caledonia","Value":"42818"},{"Key":"New Zealand","Value":"42819"},{"Key":"Nicaragua","Value":"42820"},{"Key":"Niger","Value":"42821"},{"Key":"Nigeria","Value":"42822"},{"Key":"Niue","Value":"42823"},{"Key":"Northern Mariana Islands","Value":"42824"},{"Key":"Norway","Value":"42825"},{"Key":"Oman","Value":"42826"},{"Key":"Pakistan","Value":"42827"},{"Key":"Palau, Republic Of","Value":"42828"},{"Key":"Palestine","Value":"42829"},{"Key":"Panama","Value":"42830"},{"Key":"Papua New Guinea","Value":"42831"},{"Key":"Paraguay","Value":"42832"},{"Key":"Peru","Value":"42833"},{"Key":"Philippines","Value":"42834"},{"Key":"Poland","Value":"42835"},{"Key":"Portugal","Value":"42836"},{"Key":"Puerto Rico","Value":"42837"},{"Key":"Qatar","Value":"42838"},{"Key":"Republic Of Korea","Value":"42839"},{"Key":"Republic Of Moldova","Value":"42840"},{"Key":"Republic Of Yemen","Value":"42841"},{"Key":"Reunion","Value":"42842"},{"Key":"Romania","Value":"42843"},{"Key":"Russian Federation","Value":"42844"},{"Key":"Rwanda","Value":"42845"},{"Key":"Saint Kitts And Nevis","Value":"42846"},{"Key":"Saint Lucia","Value":"42847"},{"Key":"Sao Tome & Principe","Value":"42848"},{"Key":"Saudi Arabia","Value":"42849"},{"Key":"Senegal","Value":"42850"},{"Key":"Serbia","Value":"42851"},{"Key":"Serbia & Monten","Value":"42852"},{"Key":"Seychelles","Value":"42853"},{"Key":"Sierra Leone","Value":"42854"},{"Key":"Singapore","Value":"42855"},{"Key":"Slovakia","Value":"42856"},{"Key":"Slovenia","Value":"42857"},{"Key":"Solomon Islands","Value":"42858"},{"Key":"Somalia","Value":"42859"},{"Key":"South Africa","Value":"42860"},{"Key":"South Sudan","Value":"42861"},{"Key":"Spain","Value":"42862"},{"Key":"Sri Lanka","Value":"42863"},{"Key":"St. Vincent And The Grena","Value":"42864"},{"Key":"Sudan","Value":"42865"},{"Key":"Surinam","Value":"42866"},{"Key":"Swaziland","Value":"42867"},{"Key":"Sweden","Value":"42868"},{"Key":"Switzerland","Value":"42869"},{"Key":"Syrian Arab Republic","Value":"42870"},{"Key":"Tajikistan","Value":"42871"},{"Key":"Tanzania, United Rep. Of","Value":"42872"},{"Key":"Tfyr Of Macedonia","Value":"42873"},{"Key":"Thailand","Value":"42874"},{"Key":"Timor-Leste","Value":"42875"},{"Key":"Togo","Value":"42876"},{"Key":"Tokelau Islands","Value":"42877"},{"Key":"Tonga","Value":"42878"},{"Key":"Trinidad And Tobago","Value":"42879"},{"Key":"Tunisia","Value":"42880"},{"Key":"Turkey","Value":"42881"},{"Key":"Turkmenistan","Value":"42882"},{"Key":"Turks And Caicos Island","Value":"42883"},{"Key":"Mauritania","Value":"42803"},{"Key":"Mauritius","Value":"42804"},{"Key":"Mexico","Value":"42805"},{"Key":"Micronesia","Value":"42806"},{"Key":"Monaco","Value":"42807"},{"Key":"Mongolia","Value":"42808"},{"Key":"Montenegro","Value":"42809"},{"Key":"Montserrat","Value":"42810"},{"Key":"Morocco","Value":"42811"},{"Key":"Mozambique","Value":"42812"},{"Key":"Namibia","Value":"42813"},{"Key":"Nauru","Value":"42814"},{"Key":"Nepal","Value":"42815"},{"Key":"Netherlands","Value":"42816"},{"Key":"Netherlands Antilles","Value":"42817"},{"Key":"New Caledonia","Value":"42818"},{"Key":"New Zealand","Value":"42819"},{"Key":"Nicaragua","Value":"42820"},{"Key":"Niger","Value":"42821"},{"Key":"Nigeria","Value":"42822"},{"Key":"Niue","Value":"42823"},{"Key":"Northern Mariana Islands","Value":"42824"},{"Key":"Norway","Value":"42825"},{"Key":"Oman","Value":"42826"},{"Key":"Pakistan","Value":"42827"},{"Key":"Palau, Republic Of","Value":"42828"},{"Key":"Palestine","Value":"42829"},{"Key":"Panama","Value":"42830"},{"Key":"Papua New Guinea","Value":"42831"},{"Key":"Paraguay","Value":"42832"},{"Key":"Peru","Value":"42833"},{"Key":"Philippines","Value":"42834"},{"Key":"Poland","Value":"42835"},{"Key":"Portugal","Value":"42836"},{"Key":"Puerto Rico","Value":"42837"},{"Key":"Qatar","Value":"42838"},{"Key":"Republic Of Korea","Value":"42839"},{"Key":"Republic Of Moldova","Value":"42840"},{"Key":"Republic Of Yemen","Value":"42841"},{"Key":"Reunion","Value":"42842"},{"Key":"Romania","Value":"42843"},{"Key":"Russian Federation","Value":"42844"},{"Key":"Rwanda","Value":"42845"},{"Key":"Saint Kitts And Nevis","Value":"42846"},{"Key":"Saint Lucia","Value":"42847"},{"Key":"Sao Tome & Principe","Value":"42848"},{"Key":"Saudi Arabia","Value":"42849"},{"Key":"Senegal","Value":"42850"},{"Key":"Serbia","Value":"42851"},{"Key":"Serbia & Monten","Value":"42852"},{"Key":"Seychelles","Value":"42853"},{"Key":"Sierra Leone","Value":"42854"},{"Key":"Singapore","Value":"42855"},{"Key":"Slovakia","Value":"42856"},{"Key":"Slovenia","Value":"42857"}

DynamoDb

{"Key":"Mauritania","Value":"42803"},{"Key":"Mauritius","Value":"42804"},{"Key":"Mexico","Value":"42805"},{"Key":"Micronesia","Value":"42806"},{"Key":"Monaco","Value":"42807"},{"Key":"Mongolia","Value":"42808"},{"Key":"Montenegro","Value":"42809"},{"Key":"Montserrat","Value":"42810"},{"Key":"Morocco","Value":"42811"},{"Key":"Mozambique","Value":"42812"},{"Key":"Namibia","Value":"42813"},{"Key":"Nauru","Value":"42814"},{"Key":"Nepal","Value":"42815"},{"Key":"Netherlands","Value":"42816"},{"Key":"Netherlands Antilles","Value":"42817"},{"Key":"New Caledonia","Value":"42818"},{"Key":"Nicaragua","Value":"42820"},{"Key":"Niger","Value":"42821"},{"Key":"Nigeria","Value":"42822"},{"Key":"Niue","Value":"42823"},{"Key":"Northern Mariana Islands","Value":"42824"},{"Key":"Norway","Value":"42825"},{"Key":"Oman","Value":"42826"},{"Key":"Pakistan","Value":"42827"},{"Key":"Palau, Republic Of","Value":"42828"},{"Key":"Palestine","Value":"42829"},{"Key":"Panama","Value":"42830"},{"Key":"Papua New Guinea","Value":"42831"},{"Key":"Paraguay","Value":"42832"},{"Key":"Peru","Value":"42833"},{"Key":"Philippines","Value":"42834"},{"Key":"Poland","Value":"42835"},{"Key":"Portugal","Value":"42836"},{"Key":"Puerto Rico","Value":"42837"},{"Key":"Qatar","Value":"42838"},{"Key":"Republic Of Korea","Value":"42839"},{"Key":"Republic Of Moldova","Value":"42840"},{"Key":"Republic Of Yemen","Value":"42841"},{"Key":"Romania","Value":"42843"},{"Key":"Russian Federation","Value":"42844"},{"Key":"Rwanda","Value":"42845"},{"Key":"Saint Kitts And Nevis","Value":"42846"},{"Key":"Saint Lucia","Value":"42847"},{"Key":"Sao Tome & Principe","Value":"42848"},{"Key":"Saudi Arabia","Value":"42849"},{"Key":"Senegal","Value":"42850"},{"Key":"Serbia","Value":"42851"},{"Key":"Serbia & Monten","Value":"42852"},{"Key":"Seychelles","Value":"42853"},{"Key":"Sierra Leone","Value":"42854"},{"Key":"Singapore","Value":"42855"},{"Key":"Slovakia","Value":"42856"},{"Key":"Slovenia","Value":"42857"},{"Key":"Solomon Islands","Value":"42858"},{"Key":"Somalia","Value":"42859"},{"Key":"South Africa","Value":"42860"},{"Key":"South Sudan","Value":"42861"},{"Key":"Spain","Value":"42862"},{"Key":"St. Vincent And The Grena","Value":"42864"},{"Key":"Sudan","Value":"42865"},{"Key":"Surinam","Value":"42866"},{"Key":"Swaziland","Value":"42867"},{"Key":"Sweden","Value":"42868"},{"Key":"Switzerland","Value":"42869"},{"Key":"Syrian Arab Republic","Value":"42870"},{"Key":"Tajikistan","Value":"42871"},{"Key":"Tanzania, United Rep. Of","Value":"42872"},{"Key":"Tfyr Of Macedonia","Value":"42873"},{"Key":"Timor-Leste","Value":"42875"},{"Key":"Togo","Value":"42876"},{"Key":"Tokelau Islands","Value":"42877"},{"Key":"Tonga","Value":"42878"},{"Key":"Trinidad And Tobago","Value":"42879"},{"Key":"Tunisia","Value":"42880"},{"Key":"Turkey","Value":"42881"},{"Key":"Turkmenistan","Value":"42882"},{"Key":"Turks And Caicos Island","Value":"42883"},{"Key":"Mauritania","Value":"42803"},{"Key":"Mauritius","Value":"42804"},{"Key":"Mexico","Value":"42805"},{"Key":"Micronesia","Value":"42806"},{"Key":"Monaco","Value":"42807"},{"Key":"Mongolia","Value":"42808"},{"Key":"Montenegro","Value":"42809"},{"Key":"Montserrat","Value":"42810"},{"Key":"Morocco","Value":"42811"},{"Key":"Mozambique","Value":"42812"},{"Key":"Namibia","Value":"42813"},{"Key":"Nauru","Value":"42814"},{"Key":"Nepal","Value":"42815"},{"Key":"Netherlands","Value":"42816"},{"Key":"Netherlands Antilles","Value":"42817"},{"Key":"New Caledonia","Value":"42818"},{"Key":"New Zealand","Value":"42819"},{"Key":"Nicaragua","Value":"42820"},{"Key":"Niger","Value":"42821"},{"Key":"Nigeria","Value":"42822"},{"Key":"Niue","Value":"42823"},{"Key":"Northern Mariana Islands","Value":"42824"},{"Key":"Norway","Value":"42825"},{"Key":"Oman","Value":"42826"},{"Key":"Palau, Republic Of","Value":"42828"},{"Key":"Palestine","Value":"42829"},{"Key":"Panama","Value":"42830"},{"Key":"Papua New Guinea","Value":"42831"},{"Key":"Paraguay","Value":"42832"},{"Key":"Peru","Value":"42833"},{"Key":"Philippines","Value":"42834"},{"Key":"Poland","Value":"42835"},{"Key":"Portugal","Value":"42836"},{"Key":"Puerto Rico","Value":"42837"},{"Key":"Qatar","Value":"42838"},{"Key":"Republic Of Korea","Value":"42839"},{"Key":"Republic Of Moldova","Value":"42840"},{"Key":"Republic Of Yemen","Value":"42841"},{"Key":"Reunion","Value":"42842"},{"Key":"Romania","Value":"42843"},{"Key":"Russian Federation","Value":"42844"},{"Key":"Rwanda","Value":"42845"},{"Key":"Saint Kitts And Nevis","Value":"42846"},{"Key":"Saint Lucia","Value":"42847"},{"Key":"Sao Tome & Principe","Value":"42848"},{"Key":"Saudi Arabia","Value":"42849"},{"Key":"Senegal","Value":"42850"},{"Key":"Serbia","Value":"42851"},{"Key":"Serbia & Monten","Value":"42852"},{"Key":"Seychelles","Value":"42853"},{"Key":"Singapore","Value":"42855"},{"Key":"Slovakia","Value":"42856"},{"Key":"Slovenia","Value":"42857"}

{"Key":"New Zealand","Value":"42819"},{"Key":"Reunion","Value":"42842"},{"Key":"Sri Lanka","Value":"42863"},{"Key":"Thailand","Value":"42874"},{"Key":"Pakistan","Value":"42827"},{"Key":"Sierra Leone","Value":"42854"},

Static (Read Optimised) Volatile (Write Optimised)

Oh shit #2

DynamoDb

Recovery Point Objective = 30

minutes

DynamoDb

Recovery Time Objective = 1 day

DynamoDb

DynamoDb

DynamoDb

DynamoDb

DynamoDb

S3 Versioned Buckets

DynamoDb

DynamoDb

DynamoDb

DynamoDb

DynamoDb Version Control

DynamoDb

Retrospective

Manage your risks explicitly

Monolith Gateway

Synchronization

Stakeholders

More than you think

Stakeholders

Metrics

Stakeholders

Transparency

Stakeholders

Business value

Team goals

Support Balance

Bake some goods

Time to iterate

Don’t underestimate

users

Pair pair pair

Questions

Thank you

https://github.com/PageUpPeopleOrg

https://en.wikipedia.org/wiki/Test-driven_developmenthttps://en.wikipedia.org/wiki/Behavior-driven_developmenthttps://angularjs.org/http://fsharp.github.io/FAKE/http://owin.org/https://aws.amazon.com/documentation/https://twitter.com/simonbrownhttp://phabricator.org/https://github.com/realestate-com-au/pacthttps://github.com/IdentityServer/IdentityServer3http://martinfowler.com/bliki/CQRS.html