Oracle NoSQL Database -- Big Data Bellevue Meetup - 02-18-15

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

NoSQL in the Real World

Dave Segleau Dir. Product Management Feb. 18, 2015


Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

2


Agenda

Why NoSQL?

Oracle NoSQL Database

Use cases

What are the NoSQL Challenges?

3


Why NoSQL?

1. The V’s of Big Data

– More Data, More Queries, Low Latency Operations (SLAs) Driving Business Innovation

2. Cost of Operations

– Cost effective horizontally scalable simple operations on simple data, Requires less HW, Lower license costs than traditional RDBMS databases

3. Time to Market

– Developer-centric Data Management, Data Variety, Rapid Evolution Shorter Time to Market

4


Where is NoSQL used?

5

Simple Data Management

Globally Distributed,

“Always On” data

Competitive Advantages of

“Fast Data”

Lower TCO, commodity

HW scale-out

ERP

EAM

Inventory Control

Accting & Payroll

Process Mgmt

Business Analytics

CRM

…

Driver

Application Real Time Event

Processing

Web-Scale Personalization &

Xaction Processing

Aggregation & Consolidation

…

Mobile Data Management

Time Series & Sensor Data Mgmt


The NoSQL Landscape

NoSQL

Columnar & Key/Value

• Keyspaces, Tables & Records

• Key-based access

• Limited Transactions

• Broad set of use cases

Document

• Collections

• Document-based access

• JSON & XML

• “Objects as documents” use cases

Graph

• Interconnected graphs

• Relatedness-based access

• Properties and Graphs, RDF

• Specific use cases

• Developer- centric APIs • Flexible schemas • Partitioned/sharded data

• Horizontally scalable • High Availability via Replication • Integrated with Hadoop

6

What we hear from our customers: • There is no “one size fits all NoSQL solution” • Customers use multiple NoSQL products

• NoSQL and SQL (RDBMS) must work together • Application version 2.0 is the one that really works


Agenda

Why NoSQL?


Use cases


7


What is Oracle NoSQL Database

8

101100101001001001101010101011100101010100100101

Reliable Flexible Fast Simple

advanced Key-Value database designed as cost effective, high performance solution for simple operations on collections of data with built in high availability and elastic

scale-out.

less is more


Oracle NoSQL Database NoSQL for Developers and IT

9

Setup, Admin, API & Integration

Built-in HA, Predictable Performance

Parallel Access & Scale-out

Simple:

Fast:

Flexible:

Reliable:

Flexible schema & Agile development


• Features

– Elastic

– BASE Operations

– Tables / JSON / Binary

– Online management

• Differentiators

– ACID transactions

– Online rolling upgrades

– Streaming large object support

– Oracle technology integrated

– Engineered Systems and Commodity HW

Enterprise Ready

10

Application

Storage Nodes Datacenter B

Storage Nodes Datacenter A

Application

NoSQL DB Driver

Application

NoSQL DB Driver

Application – Data Center Support

– Secondary Indexes

– Secure Access

– Flexible schema


Architecture (logical) Linear scaling and replication

11

• Elastic Auto Sharding

(split, add, contract)

Store

Shard

M

Shard

R

R

R R

Application

NoSQL Driver

M

• Writes to elected

node with flexible

durability

• Reads from any

node in system

Shard

R

R

M

Expand and Rebalance

Shard

M

R

R • Auto re-balance of

data on expansion

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 12

Oracle Big Data Management System

Data Warehouse Data Reservoir +

Oracle Big Data Connectors

Oracle Big Data SQL

Oracle Advanced Analytics

Oracle

Database

Oracle Spatial & Graph

Cloudera Hadoop

Oracle R Distribution


Oracle Industry Models

Oracle GoldenGate

Oracle Data Integrator

Oracle Event Processing

Oracle Event Processing

Apache Flume

Oracle GoldenGate

Oracle Advanced Analytics

Oracle Database

Oracle Spatial & Graph

Oracle Industry Models


• Query NoSQL data from Oracle Database and Hive

• Access NoSQL data from Hadoop and Spark for DW and analytics

• Share data with Oracle Coherence for extensible in-memory cache grid

• Persist history & event streams for processing with Oracle Event Processing

• Store & query RDF data using Oracle RDF for NoSQL

• Replicate changes in Oracle Database to NoSQL DB using Oracle Golden Gate

• Monitor your NoSQL cluster using Oracle Enterprise Manager

13

Enterprise ready -- Integrated out of the box


Agenda

Why NoSQL?


Use cases


14


Web-Scale Personalization & Xaction Processing • Direct-to-Consumer web services

• Real-time transaction capture

• High volume, Low latency

• Retail, Financial Svcs, Insurance

Time Series & Sensor Data Mgmt • Efficient, scalable data persistence

• Requires analytics and visualization

• Scalability, Cost management

• Utilities, Oil & Gas, Manufacturing, Financial Svcs

15

Real Time Event Processing • Scalable container for rules and history

• Provides context to interpret events

• Low latency, Cost trade-offs

• Credit Card Svcs, Content Recommend Engines

Aggregation & Consolidation • Disparate data sources, integrated data

• Customer Service, 360 View

• Business Innovation from legacy systems

• Insurance, Banking, Healthcare, Entertainment

Common NoSQL Use Cases


Finance

• Large credit card network provider

• Fraud rule and history management

• 6,500 ops/sec, 85% update on read

Manufacturing

• Large airline manufacturer

• Developing next generation planes

• 750K key inserts/sec, 6TB/hour

16

Telecoms

• Large mobile telecom operator

• Mobile personalized advertizing

• 24,000 ops/sec, 100% update on read

Customer Loyalty

• Operationalize analytics to NoSQL Customer Portal

• Addressed processing conflicts

• 20M+ customers in multiple countries

Oracle NoSQL Database customer success…

……many more


Agenda

Why NoSQL?


Use cases


17


Getting Started -- What’s Easy?

• Initial Installation and Deployment Simple out of the box experience

• Developer-centric APIs Flexible documents and key/value pairs

• Data Ingestion and Query Fast and simple

18


Getting Real – What are the First Challenges?

• Too Many Choices Dozens of NoSQL products

• Data Modeling Model affects behavior, specially performance

• Functional Integration How do I do X? Analytics is a hot topic

19


Deploying to Production -- What’s Hard?

• Scalability, Reliability & Support Buyer beware

• Manageability at Scale Common “hidden” cost

• Continuous Integration Integration with RDBMS and other systems

20


Enterprise NoSQL

Text

Core Database

Functionality

Application Developer

Friendly

Enterprise-grade

Software

• Predictable, low latency

• Highly available

• Highly scalable

• Administrator friendly (IT)

• Strong Integration

• Enterprise-grade support

• Multiple APIs

• Simple, flexible schemas

• Flexible data model

2/18/2015 21


Q&A

22



Appendix


Example Use Cases

26


Benefits

Scalable multi-channel customer portal

Predictable performance for all operations

Reduced time to market

Easy application evolution

Coupon redemption, Vendor recommendation

NoSQL for Customer Loyalty

Objectives

Scalable customer loyalty portal

New multi-channel consumer model

Improve operational efficiency

Solution

Personalized multi-channel coupon generation and redemption

Cross-promote affiliated vendors

Scale system with customers and participating retailers

NoSQL DB Driver Application

Retail Partners

Customer Profiles

End Customers

Available Coupons Market Segmentation

27

http://www.payback.net/


Financial Services coordinated theft prevention

NoSQL for Fraud Scoring

Objectives

Solution

Benefits

Application Data Ingestion

Tran

sact

ion

Au

tho

riza

tio

n

Pro

cess

or

Combine data sources for complex scoring

Detect, alert analyst with low latency

Handle burst seasonal transaction volumes

Oracle Coherence cluster for real time transaction object management

Oracle NoSQL Database for fraud model and customer profile management

Oracle Database for statistics and fraud modeling-related data

Simple data model, ACID transactions

Scalability, Reliability, Low Latency

Elasticity of sharded data repositories

Easy configuration and administration

NoSQL DB Driver

28


Objectives

Solution

Benefits

• Increase scalability of data storage •Deliver higher concurrency analytic data access •Scale data loading independently from analysis •Commercial support for mission critical system

•Oracle NoSQL database for high speed storage and range based extraction of time series data.

•Oracle NoSQL database for agile schema, replaced HDF5 storage format, kept analysis client program

•Oracle Big Data Appliance for efficient manageability and lowest TCO

•Hadoop post processing and RDBMS connectivity to Enterprise systems

• Improve scale of storage for flight test sensor data • Increase concurrency of access to data for analysis • Improve system availability for analysts by allowing

simultaneous data ingestion and analysis

Big Data Appliance

NoSQL DB Driver

Event Ingestion and Extraction

NoSQLDB/ Oracle RDBMS

Hadoop/ Oracle RDBMS

Oracle or Any third parties

SQL/Data Analytics Tools

NoSQL for Sensor Event Storage & Processing Large scale sensor data capture and analysis

29


Product Recommendation Platform

30

Elastic Data Store

Recommendation Rule Engine

Source Systems

Admin

Customers

Corporate Web Page Recommendation Platform

Rules

Customer Info

Recommend

User Info, Billing Info, etc

Request Respond Login

User ID


Oracle NoSQL Database Technical Features

31


Smart Topology Management

• Automated Resource Planning

• Storage nodes have indication of “capacity”

• Allocates replicas per storage node

• Intelligent Master/Replica load balance

• Ensures distribution of replicas

• Efficient use of system resources

• Reduces operator-caused configuration errors

32

Application

Smart Topology Driver


Data Center Support

• Availability Zones

• Flexible configuration

• Primary Zones

– Durability guarantees

– Low latency writes, HA

• 2nd’ary Read-Only Zones

– Asynchronous replication

– Analytic workloads

– Report generation

• Topology Aware Client Driver

• Provides business continuity and distributed workload management

33

DC1 DC2 DC3

PrimaryZones

Reports

Batch Analytics


Configurable CAP

• Greater Flexibility

– Configurable • Durability per operation

• Consistency per operation

– ACID by default

– Transaction scope is single API call

– Records share same shard key

– Multiple operations supported

34


Large Object Support

• Parallel Streaming Interfaces

– Efficient storage and retrieval of large objects

– Client side streaming interface for low memory consumption

– Server side splitting and distribution of object chunks across nodes for better read/write latency

– Automatic partial LOB detection

35

Large

Object

No

SQL

DB

Dri

ver

Ap

plic

atio

n

Shard 2

Shard N

Shard 1


Flexible Data Model


advanced key-value database

1. KV API, Application specific opaque values

Primary key & Shard key Automatic Data sharding & local indexing BASE & ACID Transactions

1. KV API, Application specific opaque values 2. JSON API, JSON Structures 1. Application specific opaque values 2. JSON API, JSON Structures 3. Table API, Tables


Secondary Indexes

37

Pkey A Value A

F1 F2 F3 F4 F5 F6

Pkey B Value B

F1 F2 F3 F4 F5 F6 Skey A Pkey A

F5 F6 F1 F2 F3

Skey B Pkey B

F5 F6 F1 F2 F3

Primary Records

Secondary Indexes

Shard 1

Writes

Reads

Shard 2

Shard N

Index on (F4)

Skey A Pkey A

F4 F1 F2 F3

Skey B Pkey B

F4 F1 F2 F3

Index on (F5, F6)

• Data local b-tree

• Broader low latency use cases

• Low cardinality indexing

• Consistent indexes

• Simple & Compound indexes

• Simple & Complex data types

• Range and equality searches

• Store wide sorted —Parallel scans

—Ordered results

—Array indexing


Security

• Data Access Protection

– Configurable enforcement

– Authentication • User/Password

• Configurable client time-outs

• Oracle Wallet integration

• Internal components self-authenticate

– Encryption over the wire • All channels SSL encrypted

– Authorization • Read/Write Access

• Administration

– Future integration with Oracle Products

38

Store

Shard

M

Shard

R

R

Shard

R

R R

NoSQL Driver

M R

M

Username Password

SSL

SSL SSL

Application

Read/Write or Admin


Online Rolling Upgrades

• Linear time impact for upgrade – Admin commands available to describe

safe upgrade order – Scripted available hands-free upgrade

experience – Read/Write availability throughout the

upgrade process

200 Node version upgrade in 15 minutes

39

0

2.5

5

7.5

10

12.5

15

17.5

72 (24x3) 144 (48x3) 216 (72x3)

Tim

e t

o U

pgr

ade

(m

in)

Total Nodes (Shards x Rep. Factor)

Online Rolling Upgrade


Developer and Admin Tools

• Standards based tooling

– SNMP / JMX metrics

– Oracle Enterprise Manager

– Cloud ready HTML5 browser admin

– Command line interface • Scripting

• Query prototyping

• Data load

– Easy to use developer API • Java, C, REST

• R, JRuby, Jython community drivers

• Coming: Javascript, Python

40


Developer Centric APIs • Java API for Key-Value, JSON and Table data models

• C API for Key-Value & JSON

• C API for Table data model

– Implementing using a new Thrift, JNI-less protocol

– Additional Thrift-based APIs to follow

• REST API available through Oracle Rest Data Services

– Deployments using Oracle Web Logic Server, Glassfish, Tomcat

– Directs Web Service calls to NoSQL DB

– Marshals data returned into JSON format

– Full CRUD operations across NoSQL Cluster

• Client-side DDL API

– Create and Alter tables and schemas inside an application

41

Date post:	17-Jul-2015
Category:	Technology
Upload:	dave-segleau
View:	728 times
Download:	2 times

Oracle NoSQL Database -- Big Data Bellevue Meetup - 02-18-15

Technology