(DEV309) From Asgard to Zuul: How Netflix’s Proven Open Source Tools Can Help Accelerate and Scale...

Post on 24-Jun-2015

916 views 4 download

Tags:

description

Learn how you can leverage the many Netflix Open Source tools to help grow your services to web-scale, and make them robust and resilient. We cover a variety of the OSS components-from operational tools like Asgard and Simian Army, to core services and libraries like Zuul, Eureka, Archaius, and Hystrix, plus a variety of security and big data tools. We walk through a sample application to illustrate how the many components fit together to build a cohesive solution.

transcript

Growth

Data Center

MySQL

Middle Tier

Web UI / Front End API

Regional Load Balancers Regional Load Balancers

API or Web Calls

memcached

Cassandra

Web service

S3 bucket

We’re here to help you get to global scale…

Apache Licensed Cloud Native OSS

http://netflix.github.com

Micro service Implementation

Web App

Front End

(REST

services)

Hystr

ix

Call “Auth Service”

Ribbon REST client

with Eureka

Fallback

Implementation Karyon

App Service

(auth-service)

Eureka

Server(s)

Karyon Eureka

Ribbon

Hystrix

Speed

Hardware Will FailSlowly Changing

Large Scale

Everything Is BrokenRapid Change

Large Scale

Everything WorksSlowly Changing

Small Scale

Hardware Will FailRapid Change

Small Scale

Scale

Regional Load Balancers Regional Load Balancers

Micro service Implementation

Web App

Front End

(REST

services)

Hystr

ix

Call “Auth Service”

Ribbon REST client

with Eureka

Fallback

Implementation Karyon

App Service

(auth-service)

Eureka

Server(s)

Client

A

p

a

c

h

e

Eureka (Service

Registry)

Server (Karyon)

Apache

Tomcat

Ribbon

Load

Balancing

Eureka

Integration

Metrics

(Servo)

Bootstrapping (Governator)

Metrics (Servo)

Admin ConsoleHTTP

Eureka Integration

Registration

Fetch Registry

H

T

T

P

C

L

I

E

N

T

H

y

s

t

r

i

x

E

V

C

a

c

h

e

Eureka (Service

Registry)

Client (Ribbon 2.0) Server (Karyon)

Ribbon Transport

Load

Balancing

Eureka

Integration

Metrics

(Servo)

Bootstrapping (Governator)

Metrics (Servo)

Admin Console

HTTP

Eureka Integration

Registration

Fetch Registry

Ribbon

Hystrix

EVCache

R

x

N

e

t

t

y

RxNetty

UDP

TCP

WebSockets

SSE

@ArchaiusBootstrap@KaryonBootstrap(name = "hello-netflix-oss")@Modules(include = {HelloNossApp.KaryonJerseyModuleImpl.class, KaryonWebAdminModule.class, KaryonEurekaModule.class})public final class HelloNossApp {

public static class KaryonJerseyModuleImpl extends KaryonJerseyModule {protected void configure() {

super.configure();bind(AuthenticationService.class).to(AuthenticationServiceImpl.class);

}public int serverPort() { return 8888; }public int shutdownPort() { return 8899; }public void configureInterceptors(GovernatorHttpInterceptorSupport<ByteBuf, ByteBuf> interceptorSupport) {

interceptorSupport.forUri("/hello").interceptIn(AuthInterceptor.class);}

}}

interface

recommendationsByUserId @Var "userId"

@Http(method = HttpMethod.GET, uriTemplate="/users/{userId}/recommendations")

@Hystrix(fallbackHandler = RecommendationFallbackHandler.class)

@EvCache(name = "movie-rec", appName = "movieService", cacheKeyTemplate = "{userId}")

MovieService movieService = Ribbon.from(MovieService.class);

Observable<Recommendations> result =

movieService.recommendationsByUserId(“user1”)

.toObservable();

createHttpResourceGroup "movieService"

newRequestTemplate "recommendationsByUserId"

"GET"

"/users/{userId}/recommendations"

requestBuilder

"userId" “user1”

AZ1 AZ2 AZ3

Regional Load Balancers

Zuul – Traffic Shaping/Routing

Data Data Data

Geo-located

Chaos Kong

Chaos Kong

AZ1 AZ2 AZ3

Regional Load Balancers

Zuul – Traffic Shaping/Routing

Data Data Data

Customer

Device

A/B Test

Engine

Personalization

EngineUser Info

Movie

Metadata

Movie

Ratings

Similar

Movies

API

Reviews

A/B Test

Engine

Personalization

EngineUser Info

Movie

Metadata

Movie

Ratings

Similar

MoviesReviews

User

Info

API

A/B Test

Engine

Personalization

Engine

Movie

Metadata

Movie

Ratings

Similar

MoviesReviews

User

Info

API

User

Info

API

A/B Test

Engine

Personalization

Engine

Movie

Metadata

Movie

Ratings

Similar

MoviesReviews

A/B Test

Engine

Personalization

Engine

Movie

Metadata

Movie

Ratings

Similar

MoviesReviews

User

Info

API

User Info

API

A/B Test

Engine

Personalization

Engine

Movie

Metadata

Movie

Ratings

Similar

MoviesReviews

A/B Test

Engine

Personalization

Engine

Movie

Metadata

Movie

Ratings

Similar

MoviesReviews

User

Info

API

Personaliza

tion Engine

User

Info

Movie

Metadata

Movie

Ratings

Similar

Movies

API

ReviewsA/B Test

Engine

Personaliza

tion Engine

User

Info

Movie

Metadata

Movie

Ratings

Similar

Movies

API

ReviewsA/B Test

Engine

Fallback

Personaliza

tion Engine

User

Info

Movie

Metadata

Movie

Ratings

Similar

Movies

API

ReviewsA/B Test

Engine

Fallback

SketchyScumblr

+

http://netflix.github.com

https://github.com/Answers4AWS/netflixoss-ansible/wiki/AMIs-for-NetflixOSS

http://answersforaws.com/resources/netflixoss/cloudformation/

https://github.com/Netflix-Skunkworks/zerotodocker

https://hub.docker.com/u/netflixoss/

Mac OS X

Virtual Box

Ubuntu 14.04

single kernel

Conta

iner

#1

File

syste

m +

pro

cess

Eure

ka

Conta

iner

Zu

ul

Conta

iner

Anoth

er

Conta

iner

...

Talk Time Title

http://netflix.github.com

@NetflixOSS

http://netflix.github.com

Please give us your feedback on this session.

Complete session evaluations and earn re:Invent swag.

http://bit.ly/awsevals