Decompose That WAR! Architecting for Adaptability, Scalability, and Deployability (jmaghreb,...

transcript

@crichardson

Chris Richardson

Author of POJOs in ActionFounder of the original CloudFoundry.com @crichardson chris@chrisrichardson.net http://plainoldobjects.com

Decompose That WAR! Architecting for Adaptability, Scalability, and Deployability

@crichardson

Presentation goalHow decomposing applications

improves deployability and scalability

and simplifies the adoption of new

technologies

@crichardson

(About Chris)

@crichardson

About Chris()

@crichardson

About Chris

http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/

@crichardson

About Chris

@crichardson

Agenda

The (sometimes evil) monolith

Decomposing applications into services

API gateway design

Refactoring the monolith

@crichardson

Let’s imagine you are building an online store

@crichardson

Tomcat

Traditional application architecture

Browser

WAR/EAR

MySQL Database

Review Service

Product InfoService

Recommendation Service

StoreFrontUI

developtest

deploy

Simple to

Apache/Load

balancer

Spring MVC

SpringHibernate

Order Service

@crichardson

But there are problems with a monolithic architecture

@crichardson

Users expect a rich, dynamic and interactive

experience

@crichardson

Browser

StoreFrontUI

View Controller

Presentation layer evolution....

HTML / HTTP

Old style UI architecture isn’t good enough

@crichardson

Browser - desktop and mobile Web application

RESTfulEndpoints

View Controller

...Presentation layer evolution

JSON-REST

HTML 5 - JavaScript

Native mobile clientIoS or Android

Event publisher

Events

Static content

No elaborate, server-side web framework required

@crichardson

Intimidates developers

@crichardson

Obstacle to frequent deployments

Need to redeploy everything to change one component

Interrupts long running background (e.g. Quartz) jobs

Increases risk of failure

Fear of change

Updates will happen less often - really long QA cycles

e.g. Makes A/B testing UI really difficult

Eggs in one basket

@crichardson

Overloads your IDE and container

Slows down development

@crichardson

Shipping team

Accounting Engineering

Obstacle to scaling development

E-commerce application

@crichardson

Review service

Product Info service

Recommendation service

StoreFrontUI

Reviews team

Product Info team

Recommendations team

UI team

Order serviceOrders team

@crichardson

Lots of coordination and communication required

I want to update the UI

But the backend is not working

@crichardson

Requires long-term commitment to a technology stack

@crichardson

Agenda

API gateway design

@crichardson

The scale cube

X axis - horizontal duplication

Z axis

- data

partit

ioning

Y axis - functional

decomposition

Scale b

y split

ting s

imilar

Scale by splitting

different things

@crichardson

Y-axis scaling - application level

ReviewService

Product InfoService

RecommendationService

StoreFrontUI

OrderService

@crichardson

Y-axis scaling - application level

Store front application

reviews application

recommendations application

Apply X axis cloning and/or Z axis partitioning to each service

product info application

ReviewService

Product InfoService

StoreFrontUI

OrderService

orders application

@crichardson

Partitioning strategies...

Partition by noun, e.g. product info service

Partition by verb, e.g. shipping service

Single Responsibility Principle

Unix utilities - do one focussed thing well

@crichardson

Partitioning strategies

Too few

Drawbacks of the monolithic architecture

Too many - a.k.a. Nano-service anti-pattern

Runtime overhead

Potential risk of excessive network hops

Potentially difficult to understand system

Something of an art

@crichardson

Inter-service communication options

Synchronous HTTP ⇔ asynchronous AMQP

Formats: JSON, XML, Protocol Buffers, Thrift, ...

Asynchronous is preferred but REST is popular too

JSON is fashionable but binary format is more efficient

@crichardson

Example micro-serviceclass RegistrationSmsServlet extends RegistrationSmsScalatraStack {

post("/") { val phoneNumber = request.getParameter("From") val registrationUrl = System.getenv("REGISTRATION_URL") + "?phoneNumber=" + encodeForUrl(phoneNumber) <Response> <Sms>To complete registration please go to {registrationUrl} </Sms> </Response> } }

For more on micro-services see http://oredev.org/2012/sessions/micro-

service-architecture

@crichardson

Real world examples

http://highscalability.com/amazon-architecture

http://techblog.netflix.com/

http://www.addsimplicity.com/downloads/eBaySDForum2006-11-29.pdf

http://queue.acm.org/detail.cfm?id=1394128

@crichardson

There are drawbacks

@crichardson

Complexity

See Steve Yegge’s Google Platforms Rant re Amazon.com

@crichardson

Multiple databases &

Transaction management

@crichardson

Implementing features that span multiple services

@crichardson

When to use it?In the beginning: •You don’t need it •It will slow you down

Later on:•You need it•Refactoring is painful

@crichardson

But there are many benefits

@crichardson

Smaller, simpler apps

Easier to understand and develop

Reduced startup time - important for GAE

Less jar/classpath hell

Who needs OSGI?

@crichardson

Scales development: develop, deploy and scale each service independently

@crichardson

Improves fault isolation

@crichardson

Eliminates long-term commitment to a single technology stack

Modular, polyglot, multi-framework applications

@crichardson

Two levels of architectureSystem-level

ServicesInter-service glue: interfaces and communication mechanisms

Slow changing

Service-level

Internal architecture of each serviceEach service could use a different technology stack

Pick the best tool for the jobRapidly evolving

@crichardson

Easily try other technologies

... and fail safely

@crichardson

The human body as a system

@crichardson

50 to 70 billion of your cells die each day

@crichardson

Yet you (the system) remain you

@crichardson

Can we build software systems with these characteristics?

http://dreamsongs.com/Files/WhitherSoftware.pdf

http://dreamsongs.com/Files/DesignBeyondHumanAbilitiesSimp.pdf

@crichardson

Agenda

API gateway design

@crichardson

Directly connecting the front-end to the backend

View Controller Product Infoservice

Reviewservice

View Controller

Browser/Native App

Traditional web application

Chatty API

Web unfriendly protocols

@crichardson

Use an API gateway

Reviewservice

APIGateway

View Controller

Browser/Native App

Single entry point

Client specific APIs

Protocol translation

@crichardson

Optimized client-specific APIs

DesktopBrowser

MobileApp

NodeJS

APIGateway

RESTproxy

Event publishing

Product Infoservice

Reviewservice

getProductInfo()getRecomm...()getReviews()

getProductDetails()

@crichardson

Netflix API Gateway

http://techblog.netflix.com/2013/01/optimizing-netflix-api.html

Device specific end points

@crichardson

Developing an API gateway

Java EE web technologies

Netty-based technology stack

Non-Java options: e.g. Node.JS

@crichardson

API gateway design issues

@crichardson

The need for parallelism

ProductDetails

Product Info

Recommendations

Reviews

getProductDetails()

getRecomendations()

getReviews()

Call in parallel

get ProductDetails

@crichardson

Futures are a great concurrency abstraction

An object that will contain the result of a concurrent computationhttp://en.wikipedia.org/wiki/Futures_and_promises

Future<Integer> result = executorService.submit(new Callable<Integer>() {... });

Java has basic futures. We can do much better....

@crichardson

Better: Futures with callbacks

val f : Future[Int] = Future { ... } f onSuccess { case x : Int => println(x) } f onFailure { case e : Exception => println("exception thrown") }

Guava ListenableFutures, Java 8 CompletableFuture, Scala Futures

@crichardson

Even better: Composable Futuresval f1 = Future { ... ; 1 }val f2 = Future { ... ; 2 }

val f4 = f2.map(_ * 2)assertEquals(4, Await.result(f4, 1 second))

val fzip = f1 zip f2assertEquals((1, 2), Await.result(fzip, 1 second))

def asyncOp(x : Int) = Future { x * x}val f = Future.sequence((1 to 5).map { x => asyncOp(x) })assertEquals(List(1, 4, 9, 16, 25), Await.result(f, 1 second))

Scala Futures

Transforms Future

Combines two futures

Transforms list of futures to a future containing a list

@crichardson

Composing concurrent requests using Scala Futures

class ProductDetailsService @Autowired()(....) {

def getProductDetails(productId: Long) = { val productInfoFuture = productInfoService.getProductInfo(productId) val recommendationsFuture = recommendationService.getRecommendations(productId) val reviewsFuture = reviewService.getReviews(productId)

for (((productInfo, recommendations), reviews) <- productInfoFuture zip recommendationsFuture zip reviewsFuture) yield ProductDetails(productInfo, recommendations, reviews) }

Asynchronously creates a Future containing result

@crichardson

Handling partial failures

ProductDetails

Controller

Product Info

Recommendations

Reviews

getProductDetails()

getRecomendations()

getReviews()

get ProductDetails X

@crichardson

About Netflix> 1B API calls/day

1 API call ⇒ average 6 service calls

Fault tolerance is essential

http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html

@crichardson

How to run out of threads

Tomcat

Execute thread pool

HTTP Request

Thread 1

Thread 2

Thread 3

Thread n

API Gateway

If service is down then thread will

be blocked

Eventually all threads will be blocked

@crichardson

Their approachNetwork timeouts

Invoke remote services via a bounded thread pool

Use the Circuit Breaker pattern

On failure:

return default/cached data

return error to caller

https://github.com/Netflix/Hystrix

@crichardson

Agenda

API gateway design

@crichardson

How do you decompose your big, scary monolithic

application?

@crichardson

Strategy #1: Stop digging

@crichardson

New functionality = service

Monolith ServiceAnti-corruption layer

Glue code

Pristine

@crichardson

Sounds simple but...

Dependencies between monolith and service

e.g. Common entities

Building the anti-corruption layer can be challenging

Must replicate data between systems

@crichardson

Strategy #2: Split front-end & backend

@crichardson

Split front-end & backend

MonolithPresentation

Backend

Front-end

Backend

Independently deployable

@crichardson

Strategy #3: Decompose front-end

@crichardson

Decompose front-endFront-end

Catalog Checkout Account Orders

/catalog /checkout /account /orders

@crichardson

Issues and challenges

Server-side Session state:

Need to use a separate session state server

@crichardson

Strategy #4: extract services

@crichardson

Module ⇒ service ...

Module

@crichardson

... Module ⇒ service

WAR ServiceAnti-corruption layer

@crichardson

What to extract?

Have the ideal partitioned architecture in mind:

Partitioned by verb or noun

Start with troublesome modules:

Frequently updated

Stateful components that prevent clustering

Conflicting resource requirements

@crichardson

Service

Domain model 2

Monolith

Untangling dependencies

API A API B API C

Domain model 1

Trouble!

@crichardson

Useful idea: Bounded contextDifferent services have a different view of a domain object, e.g.

User Management = complex view of user

Rest of application: User = PK + ACL + Name

Different services can have a different domain model

Services exchange messages to synchronize data

@crichardson

Customer management

Untangling orders and customers

Order management

Order Service

placeOrder()

Customer Service

availableCredit()updateCustomer()

Customer

creditLimit...

has ordersbelongs toOrder

Invariant:sum(order.total) <= creditLimit

available credit= creditLimit - sum(order.total)Trouble!

@crichardson

Customer management

Replicating the credit limit

Order management

Order Service

placeOrder()

Customer

creditLimit...

Customer’

creditLimit

CreditLimitChangedEvent

sum(order.total) <= creditLimit

Customer Service

updateCustomer()

Simplified

@crichardson

Customer management

Maintaining the openOrderTotal

Order management

Order Service

placeOrder()

Customer Service

availableCredit()

Customer

creditLimit

customerIdtotal

= creditLimit - openOrderTotal

OpenOrderTotalUpdated

openOrderTotal

@crichardson

Refactoring a monolith is not easy

the alternative is far worse

@crichardson

Summary

@crichardson

Monolithic applications are simple to develop and deploy

BUT have significant drawbacks

@crichardson

Apply the scale cube

Modular, polyglot, and scalable applications

Services developed, deployed and scaled independently

@crichardson

Use a modular, polyglot architecture

Reviewservice

APIGateway

View Controller

@crichardson

Questions?

@crichardson chris@chrisrichardson.net

http://plainoldobjects.com

Decompose That WAR! Architecting for Adaptability, Scalability, and Deployability (jmaghreb,...

Technology