Home of RedisWhat’s New with Enterprise Redis

Leena Joshi

2

Agenda

• What is Enterprise Redis?

• Redis on Flash

• Redis and Spark

• Redis Module Hub

3

Enterprise Conjures Up..

OR

4

But What It Really Means Is:

5

Which In Turn Means:

Infinite Seamless Scaling

True High Availability

Top notch expert support

6

Redis Labs Enhances OSS RedisRedis Labs Node

Open Source

Zero latency proxy Cluster Manager

REST API

Odd number of nodes needed to handle network splits- not three copies of data

Redis Labs Cluster• Shared nothing cluster

architecture• Fully compatible with open

source commands & data structures

Proprietary

7

The Same Technology Runs Redis Cloud

Cluster Management Path

ProxiesNode WatchdogCluster Watchdog

Node 1 Node 2 Node N (uneven number)…

Redis Shards

Unique multi-tenant “Docker” like architecture enables running hundreds of databases over a single, average cloud instance without performance degradation and with maximum security provisions

Data Path

Distributed ProxiesSingle or Multiple Endpoints

50,000+ Customers

8

Tremendous Customer TractionRedis Cloud

Available since mid-2013

6000+ enterprise customers

Redis Labs Enterprise Cluster (RLEC)Available since early-2015

100+ enterprise customers

9

Always On - Highly Available & Persistent

Simple, SeamlessClustering. Linear

Scalability.

Enterprise-Class Management and

Support

Enterprise-Class Redis – The Benefits

Stable & Predictable Top Performance

Operational Cost Savings

10

Simple, SeamlessScaling and Clustering

Auto- scaling/re-sharding/re-balancing

No downtime while scaling

Supports cross-shard operations

Simple, Seamless Clustering. Linear Scalability.

Linear Scalability

11

Always On - Highly Available & Persistent

Seamless cross datacenter/region/cloud

replication

Instant auto-failover

Persistence, backups and DR

Always On - Highly-Available & Persistent

12

Stable & Predictable Top Performance

Consistent high performance achieved under any load or

cluster size

Database processed by multiple cores

Built-in performance enhancement techniques

Stable & Predictable Top Performance

13

Operational Cost Savings

OSS Redis Redis Labs

More efficient hardware utilization: fewer servers, lower power & cooling and operational costs

Reduced manual labor through automation - reduced time writing scripts, scaling,

configuration, monitoring, re-balancing and more

Run Redis on flash memory as RAM extender – up to 10 times cheaper

Reduced downtime incidents

Shorten time to deploy Redis by over 50%

14

Enterprise Management and Support

UI, CLI, REST API -based management & alerting

Proven technology supporting thousands of customers

24x7 enterprise support, top notch Redis expertise

Enterprise-Class Management & Support

15

Redis Labs: Fastest Recovery, No Data Loss

% o

f tim

es d

ata

was

lost

Aver

age

time

to re

cove

r in

seco

nds

Redis Labs recovers in 5 seconds and does not lose data.All other vendors lose data and take many minutes to recover

Vendors evaluated include(not in order)• Heroku Redis• AWS ElastiCache• Microsoft AzureCache• Compose.io

16

Redis Labs: The Only True HA Redis16

Failure Event In-memory Replication

Multi-DC/Zone replication

Auto-failover AOF Data Persistence

Backup (using snapshots)

Multi-region/Cloud replication

Process failure Instant recovery* Slow recoveryNode failure Instant recovery* Slow recoveryMulti-node failure Instant recovery*Network split Instant recovery*Zone/Rack failure Instant recovery* Slow recovery Fast recoveryRegion/Cloud failure Slow recovery Fast recovery

Type

of o

utag

e

Essential features for high availability

*Auto-failover should run on same nodes as Redis deployment

Redis Labs provides all the essential HA features that protect against every type of outage

Redis on Flash

18

Why Analyze Data In-Memory?“Information is the oil of 21st century, and analytics is the combustion engine”- Peter Sondergaard, Gartner Analyst

19

Decision Speeds Are Accelerating..

“Big Data” gains popularity as tools become available to harness it

Batch insights start to drive business

Real time insights automate decision-

making

2005 2012 - 2015 2016…

THE DATA REVOLUTION IS MATURING..

20

The Race Is On..INSIGHTS FROM YOUR DATA NEED TO BE

INSTANTANEOUS

COST EFFECTIVE

21

Price/Performance of Memory Technology

22

Redis on Flash Flash used as a RAM extender and NOT as persistent storage

23

Redis on Flash Concepts

• Flash used as a RAM extender (NOT as a persistent storage)• Global key list in RAM; ‘hot’ values in RAM; ‘cold’ values on Flash. • Multi-threaded & async Redis when accessing objects on Flash.

Utilizes multi-core and Flash concurrency architecture• 100% compatibility with Redis

24

How to Achieve Optimal Price/PerformanceBy dynamically setting RAM/Flash ratio

25

Single Server Results with Dell & Samsung NVMe

readwrite

readwrite

Avg: 2.04M ops/sec

Max: 2.14M ops/sec

Avg: 0.91msec

Max: 0.98 msec

% below 1msec: 100%

Avg: 313RMB / 9.4WMB

Max: 1.71RGB / 96WMB

Avg: 1.45Gbps (Tx) / 0.97Gbps (Rx)

Max: 1.6Gbps (Tx) / 1.2Gbps (Rx)

Test setup:• Redis Labs Enterprise

Cluster v3.2• Dell Xeon CPU E5-

2670 v3 @ 2.50GHz• 4x Samsung NVMe

PM1725• Memtier benchmark-

open source tool• 100B object size• 80% read• 20% write

Throughput – ops/sec

Latency – msec

Disk Bandwidth – MB/sec

NW Bandwidth – Gb/sec

>2M Ops/sec, <1 ms latency, > 1GB disk bandwidth

26

A Real Life Example With Redis On Flash

Customer Scenario:• Genome dataset • Key sizes: 32B, value sizes : 5-12B• No of keys: 250 x109… 250 x1012

Key1: AAAAAAAAAAAAAAAAAAAAAAAACCCCAAA Value1 = Freq=4 IE=A OE=A

Key2: AAAAAAAAAAAAAAAAAAAAAAAAACAACCC Value2 = Freq=7 IE=A,C,T,G OE=A,C,T,G

*IE – Inside End sequences

*OE – Outside End sequences

27

Optimizing Redis UsageRAW

• # of keys 250x10^9

• Key size = 32B

• Value size = 8B (average)

• Overhead per object (key+value) = ±61B (key) + 9 (value) = 70

• Internal fragmentation per object = 14B

• RAM size = ±31TB

28

Further OptimizationsEncoding keys and values to compress sizes

• # of keys 1.25x10^9

• Key size = 4B

• Value size = 3612B

• RAM overhead per object (key+value) = ±40B

• RAM size = ±55GB // for optimal performance we used 500GB to keep 10/90 RAM/Flash ratio

• Flash size = ±4.5TB

Using Redis Hashes to store compressed keys/values (200 keys and values per hash)

1 2

29

Memory Usage and Cost Comparison

Redis on RAMStrings

Redis on FlashHashes

RAM size 31TB 0.5 TB

Flash size - 4.5TB

EC2 instances 155 x r3.8xlarge 2 x i2.8xlarge

1yr costs (reserved instances)

$2,017,325 $49,862

Savings with Flash & Hashes %

97.6%

Home of Redis

Redis in Analytics

31

Spark & Redis - Connector & Service Layer

  

Data Source Serving Layer

Spark SQL &Data Frame

RDD, Data Source,Data Set

RDD, Data Source,Data Set

Analytics & BI

32

Spark & Redis – Internal Accelerator

  

Data Source

RDD, Data Source,Data Set

RDD, Data Source,Data Set

Spark SQL &Data Frame

Analytics & BI

RDD, Data Source,

Data Set,Redis API

Data Sink

33

Accelerate Spark Time-Series with Redis

Redis sorted sets accelerate time series data processing by 100 times compared to other in-

memory K/V stores

Example time series data: Stock prices for 1024 stocks over 32 years

34

Spark-Redis Package : The Results

Redis faster by upto 100 times compared to HDFS and over 45 times compared to Tachyon or Spark

Home of Redis

Redis Modules

36

36

Modules Extend Redis’ Use Case CoverageMongoDB

Cassandra/ Datastax

Couchbase Redis (original) Redis + Modules

Single View Coming Soon

Personalization

Catalog Coming Soon

IoT

Real-Time Analytics

Content Management Coming Soon

Messaging

Fraud Detection Coming Soon

Graph Coming Soon

Time Series

Caching

Text Search Coming Soon

Image Processing Coming Soon

Machine Learning Coming Soon

Linear Algebra Coming Soon

Probabilistic data structures for processing continuous,

Coming Soon

More

37

Modules Turn Redis into a Multi-Model Database37

MongoDBCassandra/

DatastaxCouchbase Redis (original) Redis + Modules

Document-Based Coming Soon

Column-Based Coming Soon

Key-Value Data Structures Data Structures

Graph Coming Soon

38

Redis Module Hub

• A Redis Module Marketplace – for everyone

• Every problem a developer solves with Redis – now extended to the Enterprise

• Will help developers monetize their work and reach enterprise Redis users

• Will give enterprise Redis users the confidence and peace of mind to easily deploy modules

www.redismodules.com

39

4040

Full text search Prefix search0.00

5.00

10.00

15.00

20.00

25.00

30.00

3.15 2.40

21.00

8.70

24.57

10.61

Average Latency (msec)

RLEC Elasticsearch Solr

Full text search Prefix search0

5,000

10,000

15,000

20,000

25,000

20,045

6,831

690

3,686

621

3,133

Ops/sec

RLEC Elasticsearch Solr

85% higher

32x higher

7.8x faster 4.1x faster

redisearch

The world fastest text search engine

41

What Can Modules Do41

• All modules are certified by Redis Labs for full compliance with OSS Redis, Redis Cloud and Redis Labs Enterprise Cluster (RLEC)

Full Text Search Enhanced JSON Graph Operations Secondary Indexes

Linear Algebra SQL Support Image Processing N-Dimension Queries …

Thank You!