HAZELCAST 3IN-MEMORY DATAGRID

www.hazelcast.com

WHO AM IChristoph Engelbert (@noctarius2k)8+ years of professional Java development5+ years of backend developmentSpecialized to performance, GC, traffic topicsWas working for int. companies as Ubisoft and HRSSince November 2013 official Hazelcast HackerApache DirectMemory / Lightning Committer and PMCDeveloper CastMapR - MapReduce on Hazelcast 3

www.hazelcast.com

HAZELCAST IN KEYWORDSIn-Memory DataGridDistributed CacheDistributed Execution FrameworkNoSQLClusteringScalabilityPartitioningCloud ReadyOpenSource - Apache License 2.0

www.hazelcast.com

WHO USES HAZELCAST?

and many more ...

www.hazelcast.com

USECASESScale your applicationDistribute and share dataPartition your dataDistribute messagesProcess in parallel on multiple machinesLoad balancing

www.hazelcast.com

ALTERNATIVES?Oracle CoherenceIBM eXtreme ScaleVMware GemfireGigaspacesRedhat InfinispanGridgainTerracotta

www.hazelcast.com

DIFFERENCES?Easy to useLicense / CostLightweightFeatures

www.hazelcast.com

FEATURESJava Collection API

Map, Queue, Set, ListMultiMapTopic (PubSub)Java Concurrency API

Lock, Semaphore, CountDownLatch, ExecutorServiceTransactionsCustom SerializationOff-Heap supportNative client: C#, C++, Java, REST, memcached

www.hazelcast.com

EASY API// Creating a new Hazelcast nodeHazelcastInstance hz = Hazelcast.newHazelcastInstance();

// Getting a Map, List, Topic, ...Map map = hz.getMap("MapName");List list = hz.getList("ListName");ITopic topic = hz.getTopic("TopicName");

// Shutting down the nodehz.shutdown();

www.hazelcast.com

HOW DOES IT WORK?

www.hazelcast.com

DATA PARTITIONING (1/2)Multiple partitions per nodeConsistent Hashing: hash(key) % partitioncountOption to control partitioning: "key@partitionkey"Possibility to find key owner for every keySupport for Near-Caching and executions on key ownerAutomatic Fault-ToleranceSynchronous and Asynchronous backupsDefine sync / async backup counts

www.hazelcast.com

With 4 cluster nodes every server holds1/4 real data and 1/4 of backups

DATA PARTITIONING (2/2)

www.hazelcast.com

A HAZELCAST NETWORK

www.hazelcast.com

HAZELCAST IN NUMBERSDefault partition amount 271Any partition amount possibleBiggest cluster 100+ membersHandles 100k+/sec messages using a topicMax datasize depends on RAMOff-Heap for low GC overhead

www.hazelcast.com

COMMUNITY VS. ENTERPRISEFeature Community EnterpriseJava Collection API X XJava Concurrency API X XSSL Socket X XElastic Memory (Off-Heap) XJAAS Security / Authentication XManagement Center X

www.hazelcast.com

CODE SAMPLES

www.hazelcast.com

EASY TO UNITTESTpublic class SomeTestCase {

private HazelcastInstance[] instances;

@Before public void before() throws Exception { // Multiple instances on the same JVM instances = new HazelcastInstance[2]; instances[0] = Hazelcast.newHazelcastInstance(); instances[1] = Hazelcast.newHazelcastInstance(); } @After public void after() throws Exception { Hazelcast.shutdownAll(); }}

www.hazelcast.com

SERIALIZATION// java.io.Serializablepublic class User implements Serializable {}

// or java.io.Externalizablepublic class User implements Externalizable {}

// or (com.hazelcast.nio.serialization).DataSerializablepublic class User implements DataSerializable {}

// or new in Hazelcast 3 (multi version support) Portablepublic class User implements Portable {}

www.hazelcast.com

MAPinterface com.hazelcast.core.IMap<K, V> extends java.util.Map, java.util.ConcurrentMap

HazelcastInstance hz = getHazelcastInstance();

IMap<String, User> hzMap = hz.getMap("users");hzMap.put("Peter", new User("Peter", "Veentjer"));

Map<String, User> map = hz.getMap("users");map.put("Peter", new User("Peter", "Veentjer"));

ConcurrentMap<String, User> concurrentMap = hz.getMap("users");concurrentMap.putIfAbsent("Peter", new User("Peter", "Veentjer"));

User peter = map.get("Peter");

www.hazelcast.com

LISTinterface com.hazelcast.core.IList<E> extends java.util.List

HazelcastInstance hz = getHazelcastInstance();

List<User> list = hz.getList("users");list.add(new User("Peter", "Veentjer"));

User peter = list.get(0);

www.hazelcast.com

QUEUEinterface com.hazelcast.core.IList<E> extends java.util.List

HazelcastInstance hz = getHazelcastInstance();

BlockingQueue<Task> queue = hz.getQueue("tasks");

queue.offer(newTask());queue.offer(newTask(), 500, TimeUnit.MILLISECONDS);

Task task = queue.poll();Task task = queue.poll(100, TimeUnit.MILLISECONDS);Task task = queue.take();

www.hazelcast.com

LOCK (1/3)interface com.hazelcast.core.ILock extends java.util.concurrent.locks.Lock

www.hazelcast.com

LOCK (2/3)HazelcastInstance hz = getHazelcastInstance();

// Distributed ReentrantLock lock = hz.getLock("myLock");lock.lock();try { // Do something} finally { lock.unlock();}

www.hazelcast.com

LOCK (3/3)HazelcastInstance hz = getHazelcastInstance();

// Map (Row-)locksIMap<String, User> map = hz.getMap("users");map.lock("Peter");try { // Do something with Peter} finally { map.unlock("Peter");}

www.hazelcast.com

TOPIC / PUBSUBpublic class Example implements MessageListener<String> { public void sendMessage { HazelcastInstance hz = getHazelcastInstance(); ITopic<String> topic = hz.getTopic("topic"); topic.addMessageListener(this); topic.publish("Hello World"); } @Override public void onMessage(Message<String> message) { System.out.println("Got message: " + message.getMessageObject()); }}

www.hazelcast.com

EXECUTORSERVICEpublic interface com.hazelcast.core.IExecutorService extends java.util.concurrent.ExecutorService

HazelcastInstance hz = getHazelcastInstance();

IExecutorService es = hz.getExecutorService("name");es.executeOnAllMembers(buildRunnable());es.executeOnKeyOwner(buildRunnable(), "Peter");es.execute(buildRunnable());

Map<..> futures = es.submitToAllMembers(buildCallable());Future<..> future = es.submitToKeyOwner(buildCallable(), "Peter");

es.submitToAllMembers(buildCallable(), buildCallback());es.submitToKeyOwner(buildCallable(), "Peter", buildCallback());

www.hazelcast.com

ADVANCED TECHNIQUES

www.hazelcast.com

ADVANCED TECHNIQUESIndexing keys, values and value propertiesDistributed SQL-like queryWrite-Behind / Write-Through persistenceRead-Through (if key not loaded use MapLoader)TransactionsEntryListeners / EntryProcessorsAutomatic evictionControl partitioning (Version 3.1)and many more ...

www.hazelcast.com

CODE SAMPLES

www.hazelcast.com

INDEXINGIMap<String, User> map = Hazelcast.getMap("users");map.addIndex("age", true); // orderedmap.addIndex("active", false); // not ordered

<map name="employees"> ... <indexes> <index ordered="true">age</index> <index ordered="false">name</index> </indexes></map>

www.hazelcast.com

DISTRIBUTED SQL-LIKE QUERIESIMap<String, User> map = Hazelcast.getMap("users");Predicate predicate = new SqlPredicate("active AND age <= 30");

Set<User> users = map.values(predicate);

Set<Entry<String, User>> entries = map.entrySet(predicate);

www.hazelcast.com

MAPLOADER / MAPSTOREpublic class MapStorage implements MapStore<String, User>, MapLoader<String, User> { // Some methods missing ... @Override public User load(String key) { return loadValueDB(key); } @Override public Set<String> loadAllKeys() { return loadKeysDB(); } @Override public void delete(String key) { deleteDB(key); } @Override public void store(String key, User value) { storeToDatabase(key, value); }}

<map name="users"> <map-store enabled="true"> <class-name>com.hazelcast.example.MapStorage</class-name> <write-delay-seconds>0</write-delay-seconds> </map-store></map>

www.hazelcast.com

TRANSACTION (1/2)HazelcastInstance hz = getHazelcastInstance();

final Map map = hz.getMap("default");final Queue queue = hz.getQueue("default"); hz.executeTransaction(new TransactionalTask<Void>() { @Override public Void execute(TransactionalTaskContext context) { Tweet tweet = (Tweet) queue.poll(); processTweet(tweet); map.put(buildKey(tweet), tweet); return null; }});

www.hazelcast.com

TRANSACTION (2/2)HazelcastInstance hz = getHazelcastInstance();

TransactionContext context = hz.newTransactionContext();context.beginTransaction();

TransactionalMap map = context.getMap("default");TransactionalQueue queue = context.getQueue("default");

try { Tweet tweet = (Tweet) queue.poll(); processTweet(tweet); map.put(buildKey(tweet), tweet); context.commitTransaction();

} catch (Exception e) { context.rollbackTransaction();}

www.hazelcast.com

Force location of corresponding data in the same partitionby providing a special partition key

CONTROL PARTITIONING

HazelcastInstance hz = getHazelcastInstance();

Map users = hz.getMap("users");users.put("Peter@Peter", new User("Peter", "Veentjer"));

Map friends = hz.getMap("friends");friends.put("Peter-Chris@Peter", new User("Christoph", "Engelbert"));friends.put("Peter-Fuad@Peter", new User("Fuad", "Malikov"));

www.hazelcast.com

WHAT ELSE?

www.hazelcast.com

SPI (NEW IN HAZELCAST 3)Possibility to build own distributed datastructuresHook into datastructure eventsImplement your own services (like RemoteInvocation,MapReduce)React on membership eventsManipulate migrations on your purposeHandle splitbrain eventsand many more ...

www.hazelcast.com

EXTERNAL ADDONShazelgrails - Hazelcast integration for Grailshazelmr / castmapr - MapReduce for Hazelcast 2.x / 3.xhazelblast - Remote Invocationhazelcast-actors - Actor frameworkmaybe your's next?

www.hazelcast.com

@noctarius2k@hazelcast

hazelcast@googlegroups.comhttp://www.hazelcast.com

http://github.com/hazelcast/hazelcast

THANK YOU!ANY QUESTIONS?

Images: www.clipartist.info, Gnome Nebula Theme, KDE theme

www.hazelcast.com