Pragmatic Microservices

Randy Shoup @randyshoup

linkedin.com/in/randyshoup

Background• VP Engineering at Stitch Fix

o Revolutionizing retail

• Consulting “CTO as a service”o Helping companies scale their organizations and technology

• Director of Engineering for Google App Engineo World’s largest Platform-as-a-Service

• Chief Engineer at eBayo Multiple generations of eBay’s infrastructure

“Tell us how you did things at Google and eBay.”

“Sure, I’ll tell you, but you have to promise not to do them! [… yet]”

ArchitectureEvolution

• eBay • 5th generation today• Monolithic Perl Monolithic C++ Java microservices

• Twitter• 3rd generation today• Monolithic Rails JS / Rails / Scala microservices

• Amazon• Nth generation today• Monolithic C++ Java / Scala microservices

Are microservices the right approach to every problem?

PragmaticEvolution

• Monolithic Architectures

• Microservice Architectures

• Deciding to Rearchitect

• Rearchitecture Tactics

PragmaticEvolution

• Monolithic Architectures

• Microservice Architectures

• Deciding to Rearchitect

• Rearchitecture Tactics

The Monolithic Application

Simple at first

In-process latencies

Single codebase, deploy unit

Resource-efficient at small scale

Pros

Coordination overhead as team grows

Poor enforcement of modularity

Poor scaling (vertical only)

Single point of failure

Cons

The Monolithic Database

Simple at first

Join queries are easy

Transactions

Resource-efficient at small scale

Pros

Coupling over time

Performance and scalability bottleneck

Difficult to tune for multiple workloads

Single point of failure

Cons

Improving theMonolith

• Exploit “Natural Partitions”o Many domains cleanly partition into independent applicationso E.g., Buyer / seller, rider / driver, etc.

• Modularity Disciplineo Internal componentization bounds cognitive loado Easier to modify or replace

• Continuous Deliveryo Rapid deployment allows rapid evolution

PragmaticEvolution

• Monolithic Architectures

• Microservice Architectures

• Deciding to Rearchitect

• Rearchitecture Tactics

Microservices

• Single-purpose• Simple, well-defined interface• Modular and independent• Isolated persistence (!)

A

C D E

B

“Microservices are nothing more than SOA done properly.”

-- me

Microservices

Each unit is simple

Independent scaling and performanceIndependent testing and deploymentCan optimally tune performance

Pros

Many cooperating units

Network latencies

No transactionsRequire more sophisticated tooling and dependency management

Cons

Prerequisitesfor Success

• Process Maturity

• Organizational Maturity

• Operational Maturity

Process Maturity:Continuous Delivery

• Repeatable Deployment Pipelineo Low-risk, push-button deploymento Rapid release cadenceo Rapid rollback and recovery

• Automated Testingo Developers write tests and code togethero Confidence to make risky changes

• Continuous Integrationo System components assembled and tested together on each checkin

Organizational Maturity:Conway’s Law

• Organization constrains architectureo Design of a system will be a reflection of the communication paths within the organization

• Modular system requires modular organizationo Small, independent teams lead to more flexible, composable systemso Larger, interdependent teams lead to larger, more monolithic systems

• To get the system we want, we need to engineer the organization

Organizational Maturity:Service Teams

• Teams Aligned to Domainso Clear, well-defined area of responsibilityo Single service or set of related serviceso A given service is managed within one team

• Cross-functional Teamso Team has inside it all skill sets needed to do the jobo Depends on other teams for supporting services, libraries, and tools

• End-to-end Ownershipo Team owns service from design to deployment to retiremento No separate maintenance or sustaining engineering teamo DevOps philosophy of “You build it, you run it”

Operational Maturity:Monitoring

• Strong practice of detailed, end-to-end monitoring of production systems

• Ability to detect and alert on issues anywhere in the system

• Sufficient monitoring to be able to do remote runtime diagnosis

“If you ever have to ssh into a machine, your monitoring has failed you.”

-- me

PragmaticEvolution

• Monolithic Architectures

• Microservice Architectures

• Deciding to Rearchitect

• Rearchitecture Tactics

WhyRearchitect?

• Productivity / Velocityo Time to market is constrained by coupling and lack of isolation in the monolitho Teams step on each others’ toes, and can no longer develop independentlyo Difficult for new engineers to be productive

• Scaling o Vertical scaling of the monolith no longer workso Parts of the system need to scale independently of others

WhyRearchitect?

• Deploymento Parts of the system need to deploy independently of otherso Monolithic release is too slow, too complicated, too risky

“Getting to rearchitect a system is a sign of success, not failure.”

-- me

“If you don’t end up regretting your early technology decisions, you probably over-engineered.”

-- me

PragmaticEvolution

• Monolithic Architectures

• Microservice Architectures

• Deciding to Rearchitect

• Rearchitecture Tactics

“The only thing a Big Bang migration guarantees is a big *Bang*.”

-- Martin Fowler

Incremental Migration

• Step 0: Pilot Implementationo Choose initial end-to-end vertical experience to migrate / createo (+) Opportunity to learn and adjusto (+) Demonstrate feasibility and gain confidenceo (+) Bound investment and risko (+) Provide real customer value

• Initial step is the hardesto Learning how to do things in the new wayo Building out basic supporting capabilities

Incremental Migration

• Steps 1-N: Incremental Migrationo Prioritize business value -- highest ROI areas firsto Focus on areas with greatest rate of changeo (+) Maximize near-term payoff from investmento (+) Confront and solve hard problems sooner rather than later

• New feature development in parallelo Typically cannot pause all feature work in all areas to migrate o Within a particular area, try to separate feature work from migration work in distinct steps

Incremental Migration

• Residual monolith may remain indefinitelyo Lowest business valueo Most stable and least changingo Can migrate – or not – opportunistically

Carving up theMonolith

• Look for (or create) a “seam” in the monolitho This is often the hardest part (!)

• Wall it off behind an interface

• Write automated tests around the interface

• Replace implementation with a remote service

• Rinse and Repeat

Maintaining Interface Stability

• Backward / forward compatibility of interfaceso Can *never* break your clients’ code o Often multiple interface versionso Sometimes multiple deployments

• Explicit deprecation policy o Strong incentive to wean customers off old versions (!)

ServiceAnti-Patterns

• The “Mega-Service”o Overbroad area of responsibility is difficult to reason about, changeo Leads to more upstream / downstream dependencies

• “Leaky Abstraction” Serviceo Interface reflects provider’s implementation, not the consumer’s modelo Consumer’s model is typically more aligned with the domain, simpler, more abstracto Leaking provider’s model in the interface constrains evolution of the implementation

ServiceAnti-Patterns

• Shared persistenceo Breaks encapsulation, encourages “backdoor” interface violationso Unhealthy and near-invisible coupling of serviceso (-) Initial eBay SOA efforts

• “Client-less” Serviceo Service built without a specific client in mindo Very difficult to design a minimal, complete interface without a tangible use-caseo Opportunity cost and wasted efforto Usage is the true metric of the value of a service

Microservices maynot be for you if …

• … you have a simple system

• … you have a small team

• … you are not able to invest in continuous delivery, monitoring, etc.

• … they don’t solve the problems you actually have

Thank You!• @randyshoup

• linkedin.com/in/randyshoup