Date post: | 10-May-2015 |
Category: |
Technology |
Upload: | chris-richardson |
View: | 7,070 times |
Download: | 2 times |
Decomposing applications for deployability and scalability
Chris Richardson, Author of POJOs in Action, Founder of the original CloudFoundry.com @[email protected]://plainoldobjects.com/
Presentation goal
2
How decomposing applications improves deployability and
scalability and
How Cloud Foundry helps
About Chris
3
(About Chris)
4
About Chris()
5
About Chris
6
About Chris
http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/
7
vmc push About-Chris
8
Developer Advocate
Signup at http://cloudfoundry.com
9
AgendaThe (sometimes evil) monolithDecomposing applications into servicesHow do services communicate?Presentation layer designHow Cloud Foundry helps
Let’s imagine you are building an e-commerce application
10
Tomcat
Traditional web application architecture
11
Browser
WAR
MySQL Database
ShippingService
AccountingService
InventoryService
StoreFrontUI
developtestdeploy
Simple to
Apache
scale
But there are problems with a monolithic architecture
12
Users expect a rich, dynamic and interactive experience
13
Java Web ApplicationBrowser
HTTP Request
HTML/Javascript
Old style UI architecture isn’t good enough
Real-time web ≅ NodeJS
Intimidates developers
14
Obstacle to frequent deploymentsNeed to redeploy everything to change one componentInterrupts long running background (e.g. Quartz) jobsIncreases risk of failure
Fear of change
Updates will happen less oftene.g. Makes A/B testing UI really difficult
15
Overloads your IDE and container
16Slows down development
17
Shipping team
Accounting team
Engineering
Obstacle to scaling development
E-commerce application
18
WAR
Shipping
Accounting
InventoryService
StoreFrontUI
Shipping team
Accounting team
Inventory team
UI team
Obstacle to scaling development
19
Lots of coordination and communication required
Obstacle to scaling developmentUI team Accounting team
20
Requires long-term commitment to a technology stack
21
AgendaThe (sometimes evil) monolithDecomposing applications into servicesHow do services communicate?Presentation layer designHow Cloud Foundry helps
22
The scale cube
23
X axis - horizontal duplication
Z axi
s - da
ta pa
rtitio
ning
Y axis - functionaldecomposition
Scale
by sp
litting
simila
r
thing
s
Scale by splitting different things
Y-axis scaling - application level
24
WAR
ShippingService
AccountingService
InventoryService
StoreFrontUI
Y-axis scaling - application level
25
Store front web application
shipping web application
inventory web application
Apply X axis cloning and/or Z axis partitioning to each service
ShippingService
AccountingService
InventoryServiceStoreFrontUI
accounting web application
Partitioning strategiesPartition by verb, e.g. shipping servicePartition by noun, e.g. inventory serviceSingle Responsibility PrincipleUnix utilities - do one focussed thing well
26
Something of an art
Real world examples
27
http://highscalability.com/amazon-architecture
Between 100-150 services are accessed to build a page.
http://techblog.netflix.com/
http://www.addsimplicity.com/downloads/eBaySDForum2006-11-29.pdf
http://queue.acm.org/detail.cfm?id=1394128
There are drawbacks
28
Complexity
29
See Steve Yegge’s Google Platforms Rant re Amazon.com
Multiple databases =
Transaction management challenges
30
When to use it?
31
In the beginning: •You don’t need it •It will slow you down
Later on:•You need it•Refactoring is painful
But there are many benefitsScales development: develop, deploy and scale each service independentlyUpdate UI independentlyImproves fault isolationEliminates long-term commitment to a single technology stack
32
Modular, polyglot, multi-framework applications
Two levels of architecture
33
System-level
ServicesInter-service glue: interfaces and communication mechanismsSlow changing
Service-level
Internal architecture of each serviceEach service could use a different technology stackPick the best tool for the jobRapidly evolving
If services are small...Regularly rewrite using a better technology stackAdapt system to changing requirements and better technology without a total rewritePick the best developers rather than best <pick a language> developers ⇒ polyglot culture
34
Fred George “Developer Anarchy”
The human body as a system
35
50 to 70 billion of your cells die each day
36
Yet you (the system) remain you
37
Can we build software systems with these characteristics?
38
http://dreamsongs.com/Files/WhitherSoftware.pdf
http://dreamsongs.com/Files/DesignBeyondHumanAbilitiesSimp.pdf
39
AgendaThe (sometimes evil) monolithDecomposing applications into servicesHow do services communicate?Presentation layer designHow Cloud Foundry helps
Inter-service communication optionsSynchronous HTTP ⇔ asynchronous AMQP
Formats: JSON, XML, Protocol Buffers, Thrift, ...
Even via the database
40
Asynchronous is preferred
JSON is fashionable but binary format is more efficient
StoreFrontUI
wgrus-store.war
AccountingService
wgrus-billing.war
InventoryService
wgrus-inventory.war
ShippingService
wgrus-shipping.war
MySQL
41
RabbitMQ(Message Broker)
Asynchronous message-based communication
BenefitsDecouples caller from serverCaller unaware of server’s coordinates (URL)Message broker buffers message when server is down/slow
42
DrawbacksAdditional complexity of message broker RPC using messaging is more complex
43
Writing code that calls services
44
The need for parallelism
45
Service A
Service B
Service C
Service D
b = serviceB()
c = serviceC()
d = serviceD(b, c)
Call in parallel
Java Futures are a great concurrency abstraction
46
http://en.wikipedia.org/wiki/Futures_and_promises
Using Java Futures
47
public class Client {
private ExecutorService executorService; private RemoteServiceProxy remoteServiceProxy;
public void doSomething() throws ... { Future<Integer> result =
executorService.submit(new Callable<Integer>() { @Override public Integer call() throws Exception { return remoteServiceProxy.invokeRemoteService(); } });
/// Do other things
int r = result.get(500, TimeUnit.MILLISECONDS);
System.out.println(r);
}}
Eventually contains result
When needed wait for result
Akka’s composable futures are even better
48
Composable Futures
49
val f1 = Future { ... ; 1 }val f2 = Future { ... ; 2 }
val f4 = f2.map(_ * 2)assertEquals(4, Await.result(f4, 1 second))
val fzip = f1 zip f2assertEquals((1, 2), Await.result(fzip, 1 second))
http://doc.akka.io/docs/akka/2.0.1/scala/futures.html
Transforms Future
Combines two futures
Using Akka futures
50
def callB() : Future[...] = ...def callC() : Future[...] = ...def callD() : Future[...] = ...
val future = for { (b, c) <- callB() zip callC(); d <- callD(b, c) } yield d
val result = Await.result(future, 1 second)
http://doc.akka.io/docs/akka/2.0.1/scala/futures.html
Two calls execute in parallel
And then invokes D
Get the result of D
Spring IntegrationProvides the building blocks for a pipes and filters architectureEnables development of application components that are•loosely coupled•insulated from messaging infrastructureMessaging defined declaratively
51
Handling failure
52
Service A Service B
Errors happen in distributed systems
About Netflix
53
> 1B API calls/day1 API call ⇒ average 6 service calls
Fault tolerance is essential
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
How to run out of threads
54
Tomcat
Execute thread pool
HTTP Request
Thread 1
Thread 2
...
Thread N
Service A Service B
If service B is down then thread will be
blocked
XXXXX
Eventually all threads will be blocked
Use timeouts and retries
55
Never wait forever
Errors can be transient ⇒ retry
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
Service A
Service B
Use per-dependency bounded thread pool
56http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
Runnable 1
Runnable 2
Runnable ...
bounded queue
Task 1
Task 2
Task ...
bounded thread pool
Limits number of outstanding requests
Fails fast if service is slow or down
Use a circuit breakerHigh error rate ⇒ stop calling temporarily
Down ⇒ wait for it to come back up
Slow ⇒ gives it a chance to recover
57http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
On failureReturn cached data
Return default data
Fail fast
58http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
AvoidFailing
WorkerActor
Aspects + Actors
CallerDependency
InvokerAspect
DependencyStub
CircuitBreakerActor
WorkerActor
Implements circuit breaker state
machine
Equivalent of thread pool
59
@DependencyProxy annotation
@Service@DependencyProxy(circuitBreaker = "factualCircuitBreaker", timeoutInMilliseconds=750)class FactualRestaurantService extends RestaurantService { ...}
trait RestaurantService { def findNearbyRestaurants(location: Location) : Future[FindNearbyRestaurantResponse]}
60
Aspect-based Async Execution
@Aspectclass DependencyInvokingAspect {
@Pointcut("execution(* (@DependencyProxy *).*(..))") def dependencyProxyMethods {}
@Around("dependencyProxyMethods()") def invoke(jp: ProceedingJoinPoint) = { val a = AnnotationUtils.findAnnotation(jp.getTarget.getClass, classOf[DependencyProxy]) val actor = findActor(a) val timeout = Timeout(a.timeoutInMilliseconds milliseconds) actor.ask(InvokeDependency( () => jp.proceed())) (timeout) }
}
Ask actor to invoke jp.proceed() and return Future
61
See https://github.com/cer/votemeeteat
for the Actor code
62
63
AgendaThe (sometimes evil) monolithDecomposing applications into servicesHow do services communicate?Presentation layer designHow Cloud Foundry helps
Modular application
Choice of presentation layer technology
64
NodeJS is the fashionable technology
65
Why NodeJS?
66
Familiar Javascript
High-performance, scalable event-driven, non-blocking I/O model
Over 13,000 modules developed by the community
Many JavaScript client frameworks have a NodeJS counterpart, e.g. socket.io
NodeJS example
67
var http = require('http');var fs = require("fs");
http.createServer(function (req, res) { fs.readFile('somefile.txt', function (err, data) { if (err) throw err; res.writeHead(200, {'Content-Type': 'text/plain'}); res.end(data); });}).listen(1337, '127.0.0.1');
console.log('Server running at http://127.0.0.1:1337/');
Handle HTTP request
Handle file read
Socket.io - server-side
68
var express = require('express') , http = require('http') , app = express() , server = http.createServer(app) , io = require('socket.io').listen(server) ;
app.configure(function(){ app.use(express.static(__dirname + '/public'));});
server.listen(8081);
io.sockets.on('connection', function (socket) { var counter = 0; function tick() { counter = counter + 1; socket.emit('tick', counter); }; setInterval(tick, 1000);});
Serve static content from public dir
Publish event
Create a timer for socket.io client
Listen on port 8081
Socket.io - client-side
69
var socket = io.connect(location.hostname);
function ClockModel() { self.ticker = ko.observable(1); socket.on('tick', function (data) { self.ticker(data); });};
ko.applyBindings(new ClockModel());
<html><body>
The time is <span data-bind="text: ticker"></span>
<script src="/socket.io/socket.io.js"></script><script src="/knockout-2.0.0.js"></script><script src="/clock.js"></script>
</body></html>
clock.js
Connect to socket.io server
Subscribe to tick event
Bind to model
Update model
NodeJS isn’t the only game in town
70
JVM-based http://vertx.io/
A modern web application
71
Browser
Service 1
Service 2
...
HTML 5Application
Socket.ioclient
Events
RESTful WS
Server Application
Socket.ioserver
Node JS
NodeJS - using RESTful WS and AMQP
72
Node JS
Service
RabbitMQ Service
REST
AMQP AMQP
RESTRequests
Events
socket.io
73
AgendaThe (sometimes evil) monolithDecomposing applications into servicesHow do services communicate?Presentation layer designHow Cloud Foundry helps
Tomcat
Original architecture
74
Browser
WAR
MySQL Database
ShippingService
AccountingService
InventoryService
StoreFrontUI
Apache
Inventory Service Shipping Service
NodeJS
Modern architecture
75
RabbitMQ
NodeJS
Inventory Service Shipping Service
Billing Service Redis Inventory Database
Mongo Order Database
Standalone“headless” Spring/Java applications
Spring/Scala web application
MySQL Customer Database
Desktop Browser Native Mobile application HTML5 mobile application
StoreUI StoreUI StoreUI
StoreUIJavascriptAsynchronous, scalable
communication
Traditional tools: monolithic applications
76
Developing modular apps is more difficultMany more moving parts to manage•Platform services: SQL, NoSQL, RabbitMQ•Application services: your code
Who is going to setup the environments:•the developer sandbox?•...•QA environments?
77
But Cloud Foundry helps...
Applica'on Service Interface
Data Services
Other Services
Msg Services
Easy polyglot application deployment and service provisioning
vFabric Postgres
vFabric RabbitMQTM
Additional partners services …
OSS community
Private Clouds
PublicClouds
MicroClouds
Creating a platform service instance$ vmc create-service mysql --name mysql1Creating Service: OK
$ vmc services......
=========== Provisioned Services ============
+-------------+---------+| Name | Service |+-------------+---------+| mysql1 | mysql |+-------------+---------+
Multi-application manifest - part 1--- applications: inventory/target: name: inventory url: cer-inventory.chrisr.cloudfoundry.me framework: name: spring info: mem: 512M description: Java SpringSource Spring Application exec: mem: 512M instances: 1 services: si-rabbit: type: :rabbitmq si-mongo: type: :mongodb si-redis: type: :redis
80
Path to application
Required platform services
Multi-application manifest - part 2 store/target: name: store url: cer-store.chrisr.cloudfoundry.me framework: name: spring info: mem: 512M description: Java SpringSource Spring Application exec: mem: 512M instances: 1 services: si-mongo: type: :mongodb si-rabbit: type: :rabbitmq
81
Path to application
Required platform services
One command to create platform services and deploy application
$ vmc push Would you like to deploy from the current directory? [Yn]: Pushing application 'inventory'...Creating Application: OKCreating Service [si-rabbit]: OKBinding Service [si-rabbit]: OKCreating Service [si-mongo]: OKBinding Service [si-mongo]: OKCreating Service [si-redis]: OKBinding Service [si-redis]: OKUploading Application: Checking for available resources: OK Processing resources: OK Packing application: OK Uploading (12K): OK Push Status: OKStaging Application 'inventory': OK Starting Application 'inventory': OK Pushing application 'store'...Creating Application: OKBinding Service [si-mongo]: OKBinding Service [si-rabbit]: OKUploading Application: Checking for available resources: OK Processing resources: OK Packing application: OK 82
vmc push:•Reads the manifest file•Creates the required platform services•Deploys all the applications
Micro Cloud Foundry: new developer sandbox
83
Open source Platform as a Service project
App Instances Services
10.04
A PaaS packaged as a VMware Virtual Machine
Use as a developer sandbox• Use the services from Junit integration tests
• Deploy your application for functional testing
• Remote debugging from STS
Using Caldecott to tunnel into your services
84
Caldecott = TCP over HTTP
85
Your computer
Caldecott gem
Cloud Foundry
Caldecott application
Service
HTTPnative
protocol
Port NNN
Service client
nativeprotocol
Using Caldecott…$ vmc tunnel1: mysql-135e02: mysql1Which service to tunnel to?: 2Password: ********
Stopping Application: OKRedeploying tunnel application 'caldecott'.Uploading Application: Checking for available resources: OK Packing application: OK
Uploading (1K): OK Push Status: OKBinding Service [mysql1]: OKStaging Application: OK Starting Application: OK
Getting tunnel connection info: OK
Service connection info: username : uMe6Apgw00AhS password : pKcD76PcZR7GZ
name : d7cb8afb52f084f3d9bdc269e7d99ab50
Starting tunnel to mysql1 on port 10000.1: none2: mysql
Which client would you like to start?: 2
…Using CaldecottLaunching 'mysql --protocol=TCP --host=localhost --port=10000 --user=uMe6Apgw00AhS --
password=pKcD76PcZR7GZ d7cb8afb52f084f3d9bdc269e7d99ab50'
Welcome to the MySQL monitor. Commands end with ; or \g.Your MySQL connection id is 10944342Server version: 5.1.54-rel12.5 Percona Server with XtraDB (GPL), Release 12.5, Revision 188
Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>
Running JUnit test with Caldecott
88
Configure your test code to use port + connection info
Summary
89
Monolithic applications are simple to develop and deploy
BUT have significant drawbacks
90
Apply the scale cube
91
X axis - horizontal duplicationZ ax
is - d
ata pa
rtitio
ning
Y axis - functionaldecomposition
Modular, polyglot, and scalable applicationsServices developed, deployed and scaled independently
Cloud Provider Interface
Applica'on Service Interface
Private Clouds
PublicClouds
MicroClouds
Data Services
Other Services
Msg Services
.js
Cloud Foundry helps
Questions?
@crichardson [email protected]://slideshare.net/chris.e.richardson/
www.cloudfoundry.com @cloudfoundry