In-Memory Big Data

Fuad Malikov

Thursday, June 13, 13

Big Data

90% of the data in the world has been created in the last two years alone.

Thursday, June 13, 13

Big Data Storage

Hadoop

Map/Reduce

NoSQL

Thursday, June 13, 13

Disk is the New Tape

Memory is the New Disk

Thursday, June 13, 13

Distributed System

We should be able to partition the data across many servers

Thursday, June 13, 13

Big Data In-Memory

Partitioned Scalable

RedundantDynamic

Thursday, June 13, 13

Used by

Every 1.5 sec one Hazelcast instance is starting around the globeHazelcast Elastic Memory on 100 node, storing 4 TB data in-memory and doing more than 1.3M distributed ops on AWS.

Thursday, June 13, 13

Keywords

In-Memory Data GridDistributed (Elastic) CacheNoSQL Key-Value StoreClustering, Scalability, Partitioning, Big DataCloud Computing

Thursday, June 13, 13

Map

import java.util.Map; import java.util.HashMap;

Map map = new HashMap();map.put(“1”, “value”); map.get(“1”);

Thursday, June 13, 13

Thread-safe Map

import java.util.Map; import java.util.concurrent.*;

Map map = new ConcurrentHashMap();map.put(“1”, “value”); map.get(“1”);

Thursday, June 13, 13

Distributed Map

import java.util.Map; import com.hazelcast.core.Hazelcast;

Map map = Hazelcast.getMap(“mymap”);map.put(“1”, “value”); map.get(“1”);

Thursday, June 13, 13

Demo

Thursday, June 13, 13

We Scale the Application Layer

Distributed Data Structures from JavaMap, Queue, List, Lock, Executor Service, Topic, Events and etc.

NoSQL

Thursday, June 13, 13

Use Cases

Big Data Sharing/CachingSession ReplicationHibernate 2nd Level CacheSpring CacheParallel ProcessingSend/Receive Messages

Ex: Mozilla uses Hazelcast + Hadoop to process Firefox Crash Reports.

Thursday, June 13, 13

Thank You!

www.hazelcast.comTwitter: @hazelcastSource: github.com/hazelcasthazelcast@googlegroups.com

Bay Area Hazelcast Hackers Meetup Group

Thursday, June 13, 13