Hazelcast Deep Dive

Emrah Kocaman

Software Engineer @ Hazelcastemrahkocaman

emrah@hazelcast.com

Hazelcast ArchitectureHazelcast Configuration OptionsHazelcast IMap Configuration DetailsHazelcast ClientsHazelcast SerializationQ/A Session

Agenda

Hazelcast Architecture

Map Set Queue Lock/Sem. Atomics Topic User DefinedMultiMap Ring

Buffer

Web Sessions

Java

C++ .NET

Portable Serialization / Pluggable Serialization

Memcached REST Internal ClientNetwork Protocol

java.util.concurrent.*javax.cache.*Hibernate 2nd Level Cache

Executor ServiceSQL Query Map / Reduce Aggregation

Low-level Services API

Node Engine(Threads, Instances, Eventing, Wait/Notify, Invocation)

Partition Management(Master Partition, Data Affinity, Replicas, Migrations, Partition Groups)

Cluster Management(Multicast, IP List, AWS/OpenStack)

Networking(IPv4, IPv6)

On-heap Storage

Managem

ent Center

(AP

I, JMX

)

Security(C

onnection, Encryption, A

uthentication, Authorization)

WA

N(Topology A

ware P

artition Managem

ent, WA

N R

eplication)

Hazelcast Open Source Hazelcast Enterprise

Predicate Entry Processor

High-Density (HD) Memory Store

Continuous Query

HD Near Cache

Open Client Network Protocol

Hot Restart Store*

* Coming in 3.6

Hazelcast Configuration Options

XML ConfigurationProgrammatic configurationSpring Configuration

XML Configuration

XML Configuration

XML Configuration

XML Configuration`hazelcast.config` system property Working directory Classpath Default from Hazelcast.jar

XML Configuration ClasspathXmlConfig FileSystemXmlConfig InMemoryXmlConfig UrlXmlConfig

Programmatic Configuration

Distributed Map Configuration

Distributed Map Configuration

IT’S DEMO TIME

Hazelcast Clients Same intuitive API Similar configuration approach SSL Support Hazelcast Client ProtocolJava, C/C++, .NET, REST, memcache

IT’S DEMO TIME

Hazelcast Serialization

Map, Cache, Queue, Set, List Executor Service Entry Processors Lock Topic

Where it is used?serialization + deserialization

serialization

serialization

serialization

serialization

deserialization

Optimised TypesByteBooleanCharacterShortIntegerLongFloatDouble

byte[]char[]short[]int[]long[]float[]double[]

StringDateBigIntegerBigDecimalClassEnum

Hazelcast Serialization

Serializable DataSerializable IdentifiedDataSerializable Portable Pluggable

Shopping Cart Item

Shopping Cart

Benchmark – 100,000 times maxOrders = 100 * 1000

maxCartItem= 5

java.io.SerializablePros

StandardDoesn’t require any implementation

ConsTakes more time and cpuOccupies more space

java.io.Serializable - Results

Read Performance31 ops in ms

Write Performance46 ops in ms

Binary object size525 bytes

java.io.ExternalizablePros

Standard Efficient than Serializable in terms of CPU and

Memory

ConsRequires to implement the actual serialization

ShoppingCartItem - implementation

ShoppingCart- implementation

java.io.Externalizable - Results

Read Performance61 ops in ms

Write Performance60 ops in ms

Binary object size235 bytes

DataSerializablePros

Efficient than Serializable in terms of CPU and Memory

ConsHazelcast specificRequires to implement the actual serializationUses Reflection while de-serializing

ShoppingCartItem - implementation

ShoppingCart- implementation

DataSerializable- ResultsRead Performance

64 ops in ms

Write Performance59 ops in ms

Binary object size231 bytes

IdentifiedDataSerializablePros

Efficient than Serializable in terms of CPU and Memory

Doesn’t use Reflection while de-serializing

ConsHazelcast specificRequires to implement the actual serializationRequires to implement a Factory and

configuration

ShoppingCartItem - implementation

ShoppingCart- implementation

IdentifiedDataSerializable – Factory Implementation

IdentifiedDataSerializable- Results

Read Performance68 ops in ms

Write Performance60 ops in ms

Binary object size186 bytes

PortablePros

Efficient than Serializable in terms of CPU and MemoryDoesn’t use Reflection while de-serializingSupports versioning Supports partial de-serialization during Queries

ConsHazelcast specificRequires to implement the actual serializationRequires to implement a Factory and Class DefinitionClass definition is also sent together with Data but

stored only once per class

ShoppingCartItem - implementation

ShoppingCart- implementation

Portable – Factory Implementation

Portable- ResultsRead Performance

65 ops in ms

Write Performance54 ops in ms

Binary object size386 bytes

Pluggable – (ex: Kryo)

ProsDoesn’t require class to implement an interfaceVery convenient and flexible Can be stream based or byte array based

ConsRequires to implement the actual serializationRequires to plug and configure

Stream and ByteArray Serializers

ShoppingCartItem - implementation

ShoppingCart- implementation

ShoppingCart Kryo StreamSerializer

Pluggable Serialization Configuration

Kryo- ResultsRead Performance

60 ops in ms

Write Performance51 ops in ms

Binary object size210 bytes

CompressionCompresses the data.

Can be applied to Serializable and Externalizable only.

Very slow (~1000 times) and CPU consuming.

Can reduce 525 bytes to 26 bytes.

SummarySerializable

R:31 ops/ms, W: 46 ops/ms, Size: 525 bytesExternalizable

R:61 ops/ms, W: 60 ops/ms, Size: 235 bytesDataSerializable

R:64 ops/ms, W: 59 ops/ms, Size: 231 bytesIdentifiedDataSerializable

R:68 ops/ms, W: 60 ops/ms, Size: 186 bytesPortable

R:65 ops/ms, W: 54 ops/ms, Size: 386 bytesKryo

R:60 ops/ms, W: 51 ops/ms, Size: 210 bytes

Thank you ! :)

emrah@hazelcast.comany questions ?

http://www.zenika.com/formation-hazelcast-essentials.html

30Th November - Free Training