Vert.x for Microservices Architecture

Idan Fridman

Idan.frid@gmail.com

www.idanfridman.com

Agenda• The C10k problem

• Understanding the Reactor pattern

• The good (and the bad) of Vert.x and when we should use it.

• Vertx and Websockets

• RxJava And Vertx

• Vert.x into production

Traffic, Traffic Traffic...!Lots of machines

http://www.internetlivestats.com/one-second/

One machine scenario

Simple use case: Response Request

Datasource

Conventional Technology Stack

Current Setup Opening a new connection for every single request/response pair

Default HTTP connector is blocking and follows a one thread per connection model

to serve 1000 concurrent users, it requires 1000 active threads

C10K ProblemPrevented servers from handling more than 10,000 concurrent connections.

The Apache problem is the more connections the worse the performance.

Let’s assume we execute 1000 “quick” transactions

then you’ll only have about a 1000 concurrent connections to the server

Change the length of each transaction to 10 seconds

now at 1000 transactions/sec - you’ll have 10K connections open

“Why like this?” Webserver gets a connection

it starts a worker/thread

1. Memory limit

Each Thread takes min memory(default 1024k == 1mb)

2. Operating system limit

The operating system by default can't open 10k threads.

10 concurrent requests==10k threads==10k * 1MB==10GB

Reactor pattern

From wiki:

The reactor design pattern is an event handling pattern for handling service requests delivered concurrently to a service handler by one or more inputs.

The service handler then demultiplexes the incoming requests and dispatches them synchronously to the associated request handlers.

Reactor pattern(Revisited)

We already understand the The disadvantage of using a separate thread for each event listener

The reactor pattern is one implementation technique of event-driven architecture

it uses a single threaded event loop blocking on resource-emitting events and dispatches them to corresponding handlers and callbacks.

Reactor pattern modelReactor

A Reactor runs in a separate thread, and its job is to react to IO events by dispatching the work to the appropriate handler.

It’s like a telephone operator in a company who answers calls from clients and transfers the line to the appropriate contact.

Reactor pattern model

Handlers

A Handler performs the actual work to be done with an I/O event, similar to the actual officer in the company the client wants to speak to.

A reactor responds to I/O events by dispatching the appropriate handler. Handlers perform non-blocking actions.

Event-LoopStandard reactor implementation:

single event loop thread which runs around in a loop delivering all events to all handlers as they arrive.

Who implementing this?

Don’t call us, we’ll call you

Main thread should works very quickly

No long jobs in the loop

Schedule a call asynchronously

Operations in event loop should just schedule all asynchronous operations with callbacks and go to next request without awaiting any results.

We love the JVM - Let’s have vert.x1. Microservices toolkit for the JVM

2. Asynchronous

3. Scalable

4. Concurrent services development model

5. Polyglot language development with first class support for JavaScript, Ruby, Groovy, Scala, and of course Java.

Vertx is a toolkit not a framework or/container

(means you can use it within your existing application to give it the Vert.x super powers)

For example embedding it inside Spring

Vertx & Friends

Vertx main server types

Httpserver

Websockets server

TCP server

Easy as that

Building Vertx with VerticlesVerticle is the building blocks of Vert.X which reminds an Actor-like approach for concurrency model and avoiding mutable shared data

Each Verticle can communicate with another using EventBus(explain later)

Verticle types:

Standard Verticles

Worker Verticles

Verticle Types

Standard verticlesAssigned an event loop thread when they are created

All the code in your verticle instance is always executed on the same event loop

You can write all the code in your application as single threaded and let Vert.x worrying about the threading and scaling.

How to run blocking-code anyway?

Worker verticlesExecuted not using an event loop, but using a thread from the Vert.x worker thread pool.

Worker verticles are designed for calling blocking code, as they won’t block any event loops.

*Unlike Nodejs where you need to use promises with vertx you can handle blocking-code out of the box

The Event Bus

Eventbus is build-in tunnel which providing a messaging mechanism to all Vertx components.

Different parts of your application able communicate with each other irrespective of what language they are written in.

The event bus supports publish/subscribe, point to point, and request-response messaging.

Eventbus features:

registering handlers

unregistering handlers and

sending messages

publishing messages.

Eventbus - Event Driven built-In

Eventbus will be help you to scale up your Vert.x Cluster.

Eventbus and the Microservices way

ClusteringDiscovery and group membership of Vert.x nodes in a cluster

Maintaining cluster with topic subscriber lists (so we know which nodes are interested in which event bus addresses)

Distributed Map support

Distributed Locks

EventBus can be clustered:

When clustering different vert.x instances on the network they going to use the same single distributed

eventbus

Hazelcast

Vertx can use different cluster-managers. The default one is Hazelcast

Hazelcast is in-memory operational platform

Easy Discovery service

Adding JWT capabilities

Security is very important within microservices.

Vert.x Has Built-in JWT support.

You can issue JWT tokens and validate them within your Vert.x Endpoints

Websockets

Web technology that allows a full duplex socket-like connection between HTTP

servers and HTTP clients (typically browsers).

Once established:

“Push” messages

Data frames can be sent back and forth between the client and the server in full-

duplex mode

Native to Browsers

WebSocket connection remains open so there is no need to send another request to the server

Websockets and Vertx

Vertx enable out of the box support using Websockets.

Vert.x supports WebSockets on both the client and server-side.

Websockets Handler

server.websocketHandler(websocket -> { System.out.println("WebSocket is On!");});

Handler will be called when connection established:

SockJS

Client side JavaScript library and protocol which provides a simple WebSocket-like interface

Make connections to SockJS servers irrespective of whether the actual browser or network will allow real

WebSockets.

Supporting various different transports between browser and server, and choosing one at run-time according

to browser and network capabilities.

Transparent

Eventbus using sockJS bridgeClient side allows you to send and publish messages to the event bus and register handlers to receive messages

Router router = Router.router(vertx);

SockJSHandler sockJSHandler = SockJSHandler.create(vertx);BridgeOptions options = new BridgeOptions();sockJSHandler.bridge(options);

router.route("/eventbus/*").handler(sockJSHandler);

Reactive And VertxNatural couple

Open many workers in eventloop? bad

Open too many workers won't makes you any different

Work asynchronously

Who said callback (hell)?

Vertx Supports the popular RxJava library that allows you to write observers that react to sequences of events.

Scenario Example

1. Client calls to our app’s web service ->

2. our webservice need to request multiple micro-services->

3. uses a callbacks interface to pass the successful result to the next web service call

4. define another success callback- >

5. and then moves on to the next web service request.

Orchestrator

A B C F

Client Request

Looks like that->

Also known as:“The Callback Hell”

//The "Nested Callbacks" Way

public void fetchUserDetails() { //first, request the users...

mService.requestUsers(new Callback<GithubUsersResponse>() { @Override

public void success(final GithubUsersResponse githubUsersResponse, final Response response) {

Timber.i(TAG, "Request Users request completed"); final List<GithubUserDetail> githubUserDetails = new ArrayList<GithubUserDetail>();

//next, loop over each item in the response

for (GithubUserDetail githubUserDetail : githubUsersResponse) { //request a detail object for that user

mService.requestUserDetails(githubUserDetail.mLogin, new Callback<GithubUserDetail>() {

@Override public void success(GithubUserDetail githubUserDetail,

Response response) { Log.i("User Detail request completed for user : " + githubUserDetail.mLogin);

githubUserDetails.add(githubUserDetail); if (githubUserDetails.size() == githubUsersResponse.mGithubUsers.size()) {

//we've downloaded'em all - notify all who are interested!

mBus.post(new UserDetailsLoadedCompleteEvent(githubUserDetails)); }

}

@Override public void failure(RetrofitError error) {

Log.e(TAG, "Request User Detail Failed!!!!", error); } });

} }

Async our microservices call(Using Reactor)

A library for composing asynchronous and event-based programs by using observable sequences.

● Allow you to compose sequences together declaratively ● Abstracting away :

o low-level threadingo synchronizationo thread-safetyo concurrent data structureso non-blocking I/O.

RxJava for the rescue● RxJava is single jar lightweight library.

● Using the Observable abstraction and related higher-order functions. (Support Java6+)

The following external libraries can work with RxJava:

● Camel RX provides an easy way to reuse any of the

Apache Camel components, protocols, transports and data formats with the RxJava API

● rxjava-http-tail allows you to follow logs over HTTP, like tail -f

● mod-rxvertx - Extension for VertX that provides support for Reactive Extensions (RX) using the RxJava library

Zipping Observables (Without blocking)

public Observable<Boolean> registerRequest(..) { return Observable.zip( createNewRoomNode(), createNewWaveNode(), logRecordOnMysql(), sendWelcomeMessage() , (r1, r2, r3, r4) -> r1 && r2 && r3 && r4); }

private Observable<Boolean> createNewRoomNode(..) { ... return } private Observable<Boolean> createNewWaveNode(..) { ... return } private Observable<Boolean> logRecordOnMysql(..) { ... return }

Vertx going to production(Deployment)

Vertx can be deployed to AWS with Hazelcast

Discovery is taking placing using the same security groups

Vertx going to production

Enterprise ready solution which can be deployed on amazon

Combining powerful technology with complete production solution

Empower Spring configurations, env’s, defaults, libs with vertx technology

Spring Into Vertx(Benefits)

DI

Have a deployment ready container

Configurations setup

Endpoints and JMX ready

Spring Data and more

Practical implementation Spring into vert.x

● On starter Verticle initiate Spring context

● Pass Spring context to each verticle as param

● Gotchas: Make sure you always share the same spring context

Vertx Into Spring

Leverage Spring-MVC while expose controllers endpoints

Send Operational events via Eventbus(e.g send commands to connected endpoints, websockets, etc..

Deploy Verticles inside Spring

My Vert.X Case-Study

Thank you:)

Idan Fridman

Idan.frid@gmail.com

www.idanfridman.com