Rick Hightower - Mammatus Technology Inc.
WebSocket MicroService vs. REST Microservice
How does the performance of a REST microservice compare to a WebSocket MicorService?
Comparison of Speed
❖ WebSocket is faster and uses less resources for more calls
❖ REST is slower due to each connection needing to wait for a response before it can send another request
REST/HTTP vs WebSocket❖ With REST you have to wait for a reply before
you can send another request (over the same connection)❖ Unless you use pipelining which seems to
work better in benchmarks than the real world❖ With WebSocket you can stream requests and
stream responses and then marry them together❖ Still request/reply but WebSocket can send
responses while it is getting other requests
Example uses QBit❖ QBit supports REST and WebSocket RPC❖ QBit allows you to stream requests over the wire
and receive responses over the wire. It employs micro-batching to batch requests and responses into larger chunks to minimize IO and thread hand-off costs
❖ QBit allows async REST calls and async WebSocket calls. (As well as internal services based on queues)
wrk Testing tool for REST/HTTP
❖ To perf test REST code we use wrk ❖ wrk load tester capable of generating significant
load when run on a single multi-core CPU❖ wrk load tester uses multithreaded and event
notification systems ❖ epoll and kqueue
❖ wrk maximizes the number of HTTP requests per second
Trade Service
wrk LUA script
Caveats About the Test❖ We can tweak server a bit and reduce the flush rate or reduce the batch size to get
higher throughput with lower connections❖ We can also tweak the OS so we can have more ephemeral ports available and then
use a lot more connections❖ experience tells me that we can get close to 90K TPS or so on a MacBook Pro
❖ We could test from two machines ❖ one of those machines being a Linux server with a tweaked stack
❖ This test has disadvantage of all being run on same machine❖ but same disadvantage that the WebSocket version will have so it is somewhat
fair❖ We could also employ HTTP pipelining to increase the throughput,
❖ great trick for benchmarks rarely works in production environments with real clients
❖ On commodity Linux server, we can get close to 150K TPS to 200K TPS from experience
WebSocket Test❖ This will be a bit harder❖ No tool like wrk to test WebSocket RPC calls so
we will write our own❖ RPC calls are QBit centric and rely on JSON plus
some custom ASCII protocol ❖ Some overhead in the protocol because we have
to match responses to clients and requests need unique identifier (as well as the client needing a unique identifier)
To use WebSocket we will need Async interface to service
WebSocket PERF Test code
WebSocket Perf code continued
How does the WebSocket microservice do?
1 Million messages a second.
700K messages for a single WebSocket connection.
7.5x better than REST version.
Overhead❖ Running code through perf showed it was waiting for IO
a lot. Better IO. Better NIC card. More TCP/IP stack tuning. You can expect better throughput.
❖ QBit has to marry requests with responses/client id combinations. There is overhead in doing this. QBit also uses JSON/ASCII protocol which could be optimized to reduce IO (gzip it or use message pack).
❖ Custom more efficient streaming can be implemented if needed (from experience, this is another 2x to 4x)
❖ But in general, WebSocket will do better than HTTP
More Caveats❖ When using WebSocket, you also need service
discovery, and you need to implement your own failover ❖ You can’t just throw a bunch of microservices
behind a load balancer❖ QBit integrates with DNS and consul.io to
provide service discovery (also allows you to use push)
Follow up❖ This slide deck as a blog post:
❖ http://rick-hightower.blogspot.com/2015/12/websocket-microservice-vs-rest.html
❖ The QBit project: ❖ https://github.com/advantageous/qbit
❖ Mammatus Technologies ❖ http://www.mammatustech.com/
❖ Rick Hightower❖ https://twitter.com/RickHigh❖ https://www.linkedin.com/in/rickhigh
There is a lot more to QBit than this
QBit supports reactive stream over WebSocket
QBit implements Service Actors