Proxy / Load Balancer
Web Server
Database Server
Application Server
Web Server Web Server
Application Server
~3 YEARS
Proxy / Load Balancer
Web Server
Database Server
Application Server
Web Server Web Server
Application Server
Proxy / Load Balancer
Web Server
Application Server
Web Server Web Server
Application Server
Hot
Cold
Database Server
~2 YEARS
Proxy / Load Balancer
Web Server
Database Server
Application Server
Web Server Web Server
Application Server
Proxy / Load Balancer
Web Server
Application Server
Web Server Web Server
Application Server
Hot
Cold
GLB / DNS
Reporting Database Server
~2 YEARS
Proxy / Load Balancer
Web Server
Database Server
Application Server
Web Server Web Server
Application Server
Proxy / Load Balancer
Web Server
Application Server
Web Server Web Server
Application Server
Reporting Database Server
Hot
Hot
Database Server
GLB / DNS
~2 YEARS
“Loosely coupled Service Oriented Architecture with Bounded Contexts”
-Adrian Cockcroft, Microservices Thoughtleader
IF EVERY SERVICE HAS TO BE
UPDATED AT THE SAME TIME
IT’S NOT LOOSELY COUPLED
IF YOU HAVE TO KNOW TOO
MUCH ABOUT SURROUNDING
SERVICES YOU DON’T HAVE A
BOUNDED CONTEXT. SEE THE
DOMAIN DRIVEN DESIGN BOOK
BY ERIC EVANS.
CONS
PROS
• Monitoring/Logging becomes more important than ever.
• You trade code simplicity for operational complexity.
• Easy to chase the latest shiny technology / mix new languages & tech in.
• Another dimension to scale the application.
• Easier to have different services usedata persistence most appropriate for the use case.
• Easier to reason about moving applications to the cloud.
• Easier to have a developer understandthe portion of the system and contribute
• Easier to test & deploy a single service.
• Cascading failures can cause problems.
• Orchestration can become complex
FIRST PRINCIPLES DevOps Culture,
Evolutionary Architecture, Scale Cube & CAP Theorem
MICROSERVICESPATTERNS & PRACTICES
PATTERNS & PRACTICESTechnical, Operational, Cultural
DEVOPS CULTURE IS A MUST
• Rapid provisioning
• Basic monitoring
• Rapid Application Deployment
http://martinfowler.com/bliki/MicroservicePrerequisites.html
“Signing up for Microservices is signing up for evolutionary architecture. There’s no point where you’re just done.” -Josh Evans, Director of Operations Engineering @Netflix
Proxy / Load Balancer
Web Server
Database Server
Application Server
Web Server Web Server
Application Server
Proxy / Load Balancer
Web Server
Application Server
Web Server Web Server
Application Server
Reporting Database Server
Hot
Hot
Database Server
GLB / DNS
https://www.infoq.com/presentations/netflix-failure-multiple-regions
SCALE CUBE
X axis - horizontal Scaling (Scale by Cloning)
Z ax
is - D
ata P
artiti
oning
(Sca
le by
Sha
rding
)Y axis - Functional Decomposition
(Scale by Splitting)
The CAP theorem is the idea that a distributed computing system is not able to provide partition tolerance, consistency and availability at
the same time.
MICROSERVICESPATTERNS & PRACTICES
AVAILABILITY/PARTITION TOLERANCE
CONSISTENCY/PARTITION TOLERANCE CP: Choose Consistency over Availability when your business
requirements dictate atomic reads and writes. Wait for a response from the partitioned node which could result in a timeout error. The
system can also choose to return an error, depending on the scenario you desire.
AP: Choose Availability over Consistency when your business requirements allow for some flexibility around when the data in the
system synchronizes. Return the most recent version of the data you have, which could be stale. This system state will also accept writes that can be processed later when the partition is resolved. Availability is also
a compelling option when the system needs to continue to function in spite of external errors (shopping carts, etc.)
Proxy / Load Balancer
Web Server
Database Server
Application Server
Web Server Web Server
Application Server
Proxy / Load Balancer
Web Server
Application Server
Web Server Web Server
Application Server
Reporting Database Server
Hot
Hot
Database Server
GLB / DNS
Evolu&onary Architecture
Scale Cube
CAP Theorem
Microservices is a architecture that conFnues to change
and adapt
Scale Cube consists of an X, Y and Z axes – each
addressing a different approach to scale a service.
The idea that a distributed compuFng system is not able
to provide parFFon tolerance, consistency and availability
at the same Fme.
FIRST PRINCIPLESThings we need to
understand about as we plan/scale systems
DevOps is a MustMicroservices is the first architecture post devops culture
shiP
TECHNICAL Tools, Event Sourcing, CQRS,
Circuit Breaker, Retrys
OPERATIONALObservability, Containers, Polyglot
CULTURALDevOps Culture, Conway’s Law
MICROSERVICESPATTERNS & PRACTICES
123
SPINE MODEL
• Needs: We are here to satisfy ________
• Values before PrinciplesWe optimize for ________
• Principles before Practices We leverage _____ to change the system
• Practices before tools We do _____ to create value
• We have always been Tool users and makers. Tools should answer the question: We use ____ to get our work done.
http://spinemodel.info/explanation/introduction/
Proxy / Load Balancer
Web Server
Database Server
Application Server
Web Server Web Server
Application Server
Proxy / Load Balancer
Web Server
Application Server
Web Server Web Server
Application Server
Reporting Database Server
Hot
Hot
Database Server
GLB / DNS
Hot
Hot
GLB / DNS
User Facing AvailabilityConcerns
Internal Consistency Concerns
User Facing AvailabilityConcerns
Internal Consistency Concerns
Student Enroll In Workshop Security Module Student Module Workshop Module
reserveSeat() isAuthorized()
isEligible()
isAvailable()
updateSchedule()
updateSchedule()
ALL SYNCHRONOUS
MUTIABLE STATE!
The fundamental idea of Event Sourcing is that of ensuring every change to the state of an application is captured in an event object, and that these event objects are themselves stored in the sequence they were applied for the same lifetime as the application
state itself.- M Fowler
Student Enroll In Workshop Security Module Student Module Workshop ModuleQueue/Topic
subscribe()subscribe()
subscribe()
reserveSeat() isAuthorized()
reserveSeat()
ASYNCHRONOUS
IMMUTIABLE STATE!
updateSchedule()
updateSchedule()
Par
Immutable Events
Recreate the exact state
Increase in complexity
Performance / Load tes&ng
Provides a model to think about scalability
Because you have all of the state changes, you have all of
the changes that affected state
No longer doing CRUD. You have calculate the result.
Leverage things like snapshots
You an reply the events across the Enterprise
EVENT SOURCINGThe fundamental idea of Event
Sourcing is that of ensuring every change to the state of an
application is captured in an event object, and that these
event objects are themselves stored in the sequence they
were applied for the same lifetime as the application state
itself.
Command and Query Responsibility Segregation (CQRS) is a pattern that segregates the operations
that read data (Queries) from the operations that update data (Commands) by using separate
interfaces.
Presentation
Commands
Data Persistance
Data Storage
Updates (Writes)Domain Logic
Queries (Reads)
Presentation
Presentation
Commands
Data Persistance
Data Storage
Updates (Writes)Domain Logic
Queries (Reads)
Presentation
Presentation
Presentation
Presentation
Commands
Data Persistance
Data Storage
Updates (Writes)
Domain Logic
Queries (Reads)
Presentation
Presentation
Presentation
Presentation
Presentation
Presentation
Read Data StoreWrite Data Store
EVENTUAL
CONSISTENCY?
Domain Specific Models
Scale
Tradeoff
Performance
Clean Domain models queried
Scale the reads above the writes
Increased the operaFonal complexity of the data stores
Tune different datastore for different performance needs
CQRSCommand-Query-Responsibility-
Separation (CQRS) separates query processing from command
processing, which updates entities using business logic that's
commonly implemented using event sourcing.
SecurityUpdates to the model controlled different from the reads
Service A called B.
TRIPPED
Service B is Sick, let’s flag is and
try back in 5min
Service C
Different Instance of Service B
Service B is sick. Don’t send traffic there for 5
minute (Tripped)
Service A called B.
OPEN
Service B is still sick
Service C
Different Instance of Service B
Service B is still sick.
Isolate it and remove from
ELB
Service responds normally.
Service C
After 5 minutes. Traffic back to service B.
Closed
set the flag to closed. Nothing
else to do.service B
responded correctly,
Prevents Doomed to Fail
Three states
Prevents Cascading Failures
purpose is to detect external service failures and prevent
the applicaFon from trying to perform an acFon that is
doomed to fail.
Closed (Normal), Open(Tripped), & Half-Open state
Helps prevent cascading failures and improves resiliency.
CIRCUIT BREAKER
Circuit breaker is used to detect failures and
encapsulates logic of preventing a failure to reoccur
constantly (during maintenance, temporary external system failure or
unexpected system difficulties). Part of Hysterix
Part of Ne\lix OSS library for latency & fault tolerance
RETRY
“Upstream timeout must always be longer than total downstream
timeout * retries delay”-Adrian Cockcroft
Bad Service
Good Service
Edge Service
3s
1s 1s
Fails fast @ 2s
“Upstream timeout must always be longer than total downstream timeout * retries delay” -Adrian Cockcroft
Budgeted timeout, one retry
Tools
Resiliency PaIerns
Architectural PaIerns to Consider
Don’t start with the tool.
Circuit Breaker, Retry
Event Sourcing & CQRS
TECHNICAL
These are some interesting technical considerations
with Microservices
TECHNICAL Getting started, Event Sourcing, CQRS
OPERATIONALObservability, Containers, Polyglot
CULTURALDevOps Culture, Conway’s Law,
MICROSERVICESPATTERNS & PRACTICES
123
OBSERVABILITY
In control theory, observability is a measure for how well internal states of a system can be inferred by knowledge of
its external outputs.
At a minimum setup log stash, look at going an extra step to setup
ELK.
OBSERVABILITY: THINGS TO THINK ABOUT
LOGS
• Need consistent, structured logging
• Logs are not for humans
• Logstash, Elasticsearch, Kibania works great
OBSERVABILITY: THINGS TO THINK ABOUT
PROTOCOLS/TRANSPORT
RPC/HTTP vs
JSON vs Binary
RPC vs HTTP
JSON vs Binary
• Speed/Performance vs Readability
• Measure serialization, transmission, deserialization costs
OBSERVABILITY: THINGS TO THINK ABOUT
How do you Recreate a failure in a distributed resilient
environment?
TRACING
Look into:
• Zipkin
• Prometheus
Best way to understand Fanout
CONTAINERS
“What do you feel is the most disruptive (or at least potentially disruptive technology)?”
Observability
Polyglot
Containers
Logging, Service CommunicaFon, Tracking
Containers makes it easier, but observability, logging,
tracking can be more challenging
Low fricFon deployment, Finer-grained execuFon
environments, speed, isolaFon
OPERATIONAL
Microservices simplifies code. It trades code
complexity for operational complexity
TECHNICAL Getting started, Event Sourcing, CQRS
OPERATIONALObservability, Containers, Polyglot
CULTURALDevOps Culture, Conway’s Law
MICROSERVICESPATTERNS & PRACTICES
123
CONWAY'S LAW
“Organizations which design systems ... are constrained to produce designs which are
copies of the communication structures of these organizations”
-Conway
Model
One size does not fit all
Tradeoff
Organiza&onal Lifecycle Stages
Provides a model to think about scalability
Microservices (funcFon decomposiFon) is AN approach
Each approach has different tradeoffs (For example: as
we’ll find with Y-axis scaling, you may trade code
complexity for organizaFonal complexity)
Different needs at different points of an org’s lifecycle
CONWAY’S LAW
Organizations which design systems ... are constrained to produce designs which
are copies of the communication structures of
these organizations
Proxy / Load Balancer
Web Server
Database Server
Application Server
Web Server Web Server
Application Server
Proxy / Load Balancer
Web Server
Application Server
Web Server Web Server
Application Server
Reporting Database Server
Hot
Hot
Database Server
GLB / DNS
Document Repo
Recruiting
Processing
Boards
Applicants
Recruiting
Processing
Document Repo
Applicants
Applicants
API Gateway
API Gateway
Applicants
Boards
https://www.infoq.com/presentations/netflix-failure-multiple-regions
FIRST PRINCIPLES
MICROSERVICESPATTERNS & PRACTICES
• Have capabilities in your org to do things like rapidly provision, monitor, and use rapid deployment. (DevOPs Culture is a must)
• Accept that the service going to evolve (Evolutionary Architecture)
• Handle their own data needs (Consistency/Availablity Tradeoffs)
• Be able to scale based on it’s needs (Scale Cube)
Processing
DevOps Culture, Evolutionary Architecture,
Scale Cube & CAP Theorem
TECHNICAL Spine Model
Event Sourcing/CQRS Circuit Breaker
Retry
OPERATIONALObservability
Containers Polyglot
CULTURALConway’s Law
MICROSERVICESPATTERNS & PRACTICES Processing
• Have a decision process(Spine Model)
• Consider using events where it makes sense and how you might scale commands & queries(Event Sourcing / CQRS)
• Protect against cascading failures(Circuit Breaker, Bulkhead)
• “Everything is a tradeoff, do it intentionally” Matt Ranney (RPC/HTTP, Containers, Polyglot)
• Bootstrap services with organizational capabilities (Logs, Tracing)
• Beware of Conway’s Law
CONS
PROS
• Monitoring/Logging becomes more important than ever.
• You trade code complexity for operational complexity.
• Easy to chase the latest shiny technology / mix new languages & tech in.
• Another dimension to scale the application.
• Easier to have different services usedata persistence most appropriate for the use case.
• Easier to reason about moving applications to the cloud.
• Easier to have a developer understandthe portion of the system and contribute
• Easier to test & deploy a single service.
• Cascading failures can cause problems.
• Orchestration can become complex
“If you don’t end up regretting your early technology decisions, you probably over engineered.” - Randy Shoup, Engineering Director @StitchFix
MICROSERVICESPATTERNS & PRACTICES