GLUE CON 2015Architectural Patterns for Scaling Microservices and APIs

Scalability is not a Singleton

You have less than 5 seconds to

deliver content.

Maintain availability Maintain performance

Average cost of

critical app failure per

hour is $500,000

to $1 million.

Scale is achieved by optimally distributing load

TCP Connections Response time HTTP Requests

Least connections Fastest responseFastest appRound Robin

Thresholds Methods

The Scale Cube

X-axis Y-axis Z-axis

http://microservices.io/articles/scalecube.html

Scalability: Out, Up, and Across Creating

scalability domains

Balance performance and availability

X-Axis

Like clustering Leverages cloning Typically operates at

layer 4 (TCP)

Y-Axis

Like routing Uses identifiable variables

(usually from HTTP headers)

Operates at layer 7 (HTTP)

Z-Axis

Like sharding Uses one or more identifiable variables (usually from HTTP

headers) Operates at layer 7 (HTTP)

APIs and MicroservicesAPI as Façade API as Endpoint

SERVICE

API

API

SERVICE

API

SERVICE

API

SERVICE

API

API as Endpoint

API

Standard X-axis Scaling Pattern

API

API

API Façade: Interpolated in Network

Standard Y-axis Scaling Pattern

Microservice A

Microservice B

Microservice C

API

One pattern is rarely enough

Most often combined in 2-3 tiers

Y-axis routing pattern used to distribute API calls to X-axis scaled service clusters

2-Tier Interpolated API

Y-Axis Scaling X-Axis Scaling

API

Top Mistakes Regardless of Pattern

• Choosing the wrong thresholds • Scaling too late leads to time outs &

performance degradation• Not monitoring • Services fail, users get angrified by time

outs• Not scaling the scaler • Overwhelming the upstream scaler is a

Very Bad Thing™

@lmacvittie F5 Networks

THANK YOU!