Make it fast for everyonePerformance and Middleware design

S. Suhothayan

Technical Lead, WSO2 Inc.

Optimising Applications for

Performance

Latency vs Throughput

Latency measures the end-to-end time processing time. In a messaging environment, teams determine latency by measuring the time between sending a request and receiving the response. Latency is measured from the client machine and includes the network overhead as well.

Throughput measures the amount of messages that a server processes during a specific time interval (e.g. per second). Throughput is calculated by measuring the time taken to processes a set of messages.

Latency vs Throughput

http://wso2.com/library/articles/2012/03/importance-performance-wso2-esb-handles-nonobvious/

WSO2 CEP

Data structures

o Linked List vs Array List vs Map

CEP window processor :

o Average temperature of last 10 min. o Inmemory

o Average temperature of last 24 hour.o Disc Based implementation

Sactificing Accuracy for Performance (Bloom filters)

http://lkozma.net/blog/sketching-data-structures/

Initial configuration : 0000000000Insert("Sally") : 0100000001 # h1("Sally") = 2, h2("Sally") = 10Insert("Jane") : 1110000001 # h1("Jane") = 1, h2("Jane") = 3Insert("Mary") : 1110100001 # h1("Mary") = 5, h2("Mary") = 2 [collision]

Query("Sally") # bits 2 and 10 are set, # return HITQuery("John") # h1("John") = 10 set, but h2("John") = 4 not set # return MISSQuery("Bob") # h1("Bob") = 5 set, h2("Bob") = 1 set # return HIT (false positive)

Syncing Nodes (Merkle tree)

Working with untrusted peer to peer networks (Torrents, Cassandra ring)

hash 0 = hash( hash 0-0 + hash 0-1 )

http://en.wikipedia.org/wiki/Merkle_tree

Improve performance by Multi threading

Use correct number of threads too much or too little is not good.

Scaling Vertical : Improve Throughput reduce LatencyHorizontal : Improve both Throughput & Latency

Commonly used data structure : ArrayBlockingQueue

Removing contingency (disruptor)

https://lmax-exchange.github.io/disruptor/

Removing contingency (disruptor)

o Minimise thread contentiono Maximise the use of single threaded algorithms and data

structures.

https://lmax-exchange.github.io/disruptor/

Optimising Performance over Network

Blocking vs Non Blocking

o Handling of thousands of connections simultaneously (significant number of connections may be in idle state as well)

o Handling high latency connectionso Request/response handling needs to be

decoupledo Minimize latency, maximize throughput and

avoiding unnecessary CPU cycles

WSO2 ESB

Routing: Header based, content based, rule-based and priority-based routing

Mediation: EIPs (including scatter/gather, message filters, recipient list, dead-letter channels, guaranteed delivery and message enrichment), database integration, event publishing, logging & auditing, validation

Transformation: XSLT 1.0/2.0, XPath, XQuery, Smooks

Reactor Pattern

Uses Readiness selection: It is the ability to choose a socket that will not block when read or written. This is a very efficient way of handling thousands of concurrent clients and scales well.

http://wso2.com/library/articles/2013/12/demystifying-wso2-esb-pass-through-transport-part-i/

Message Flow

NHTTP

Message Relay

Why we need to read the whole message if the necessary info is only at the message headers?

Read only the headers and stream the message as bytes !

Passthrough Transport

Going beyond design patterns & conventions

Performance Comparison

XML vs JSON & SOAP vs REST

XML vs JSONo Serialization and Deserialization Performanceo Data Storageo Data Transfero Interoperability SOAP vs RESTo REST has minimal overhead on top of HTTPo REST easy to scale don't have any server side sessionso REST Provides easier interface for cliento SOAP is better when it comes to security o SOAP has better tools for schema, code generation and

validation

Binary Transports (Thrift, Protocol buffers)

http://ganges.usc.edu/pgroupW/images/a/a9/Serializarion_Framework.pdf

Binary Transports (Thrift, Protocol buffers)

http://ganges.usc.edu/pgroupW/images/a/a9/Serializarion_Framework.pdf

Monitor all times (Java Flight Recorder)

Monitor all times (Java Flight Recorder)

Monitor all times (Java Flight Recorder)

Contact us !