Chris Birchall (@cbirchall)M3, Inc

ScalaMatsuri 2014

m3.com

● Portal site for medical professionals○ Medical news, pharmaceutical info, forums, Q&A

site, jobs, shopping, blogging, …○ PC site, smartphone site, a few native apps

● Service-oriented architecture○ Dozens of frontends and backends○ Mostly in data center, a few in the cloud

Problems● Fault tolerance

○ 1 backend goes down → takes down 5 backends and 10 frontends with it

● Visibility○ debugging and recovering when things go wrong○ measuring performance of backend APIs

● Performance○ lots of serial API calls○ timeouts too long/non-existent○ caching of API responses inconsistent across clients

Solution!

Octo-what?Octoparts is an API aggregation middleware

Web

FrontendA

FrontendB

FrontendC

BackendW

BackendX

BackendY

BackendZ

Features

● API call parallelization● Hard timeouts● Fault tolerance● Powerful and flexible caching● Visibility● Admin UI● Alert mails● Stateless, async → scalable

Feature: parallelization

Frontend

BackendX

BackendY

BackendZ

x, y and z please!

x please!

y please!

z please!

Feature: hard timeouts

Frontend

BackendX

BackendY

BackendZ

x, y and zwithin 500ms

please!

x please!

y please!

z please!

Feature: hard timeouts

Frontend

BackendX

BackendY

BackendZ

Here’s x and y.z didn’t make it.

500ms later ...

here’s x

here’s y

Feature: fault tolerance

● Circuit breakers○ If an API appears to be down, don’t bother

contacting it for the next N seconds

Frontend

BackendX

BackendY

BackendZ

x, y and zplease!

Here’s x and y.z didn’t make it.

Feature: caching

● Supports both TTL and ETag/Last-Modified● Powerful cache invalidation API

○ Cache data indefinitely, invalidate it when necessary○ Cache grouping

■ 1 piece of user data changes → invalidate N different caches

Frontend

BackendX

BackendY

BackendZ

x, y and z please!

Memcached

Quick as a flash,straight from the cache!

Feature: visibility Real-time Hystrix dashboard

Kibana (performance, req counts, cache hit rate, ...)

Implementation● Scala (of course!)● Play Framework● Hystrix● Skinny ORM● Scaldi

● Memcached● Postgresql● Fluentd● Elasticsearch● Kibana

Moving parts

Elasticsearch

Kibana

Alert mailerscript

Alertmail

MemcachedPostgres Hystrix dashboard

octo1

Play.log

octo2

Play.log

octo3

Play

.log

Tomcat

Open source

しちゃいますよ。。。

GitHub https://github.com/m3dev/octopartsDocs http://m3dev.github.io/octopartsDemo http://octoparts.herokuapp.com/

We’re hiring!Come and help us hack on cool stuff with Scala

http://at.m3.com/job