Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
In-Memory Analytics & Big Data Discovery
Impulsvortrag & Diskussion Björn Ständer Business Analytics - Director Business Development Oracle Deutschland B.V. & Co KG, München October 21st , 2014
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Enterprise Computing Trends
GLOBALIZATION
9 Billion Internet Devices
in 2012
50 Billion by 2020
DATA EXPLOSION
90% Created within Last Two Years
50X Growth by 2020
RISE OF MOBILITY
6 Billion Mobile Subscribers
87%
of World‘s Population
Mobile Data
Growing 78% CAGR
CLOUD MODERNIZE TO SURVIVE
Lots of
20 year-old Legacy
Applications
90% of new software
delivery for public cloud
Public IT cloud
services will grow
to $107B in 2017
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
“Engage me everywhere.”
“Know me. Wow me.”
“Meet my expectations.”
“Understand and reward me.”
Customers Demand Great Experiences
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Actionable Events
Data Reservoir
Data Factory Data Warehouse In-Memory
BI and Reporting
Discovery Lab
Actionable Information
Actionable Insights
Data Streams
Execution
Innovation
Discovery Output
Events & Data
Next Generation Data Management Architecture
6
Enterprise Data
Web & Social Data
Event Engine & Real-Time Decision
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Use Data
12%
Executives who feel they understand the impact data
will have on their organizations
Produce Data
Big Problem With Big Data
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Big Data Discovery & Oracle In-Memory
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle BI – Key Investment Areas
• Business User Experience
• Mobile BI
• Analytics in the Cloud – Platform and Applications
• Big Data Discovery
• Innovations in Engineered Systems and In-Memory Analytics
9
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Hadoop Data Reservoir Concept Gaining Momentum
10
Data Warehouse Data Reservoir
Emerging Sources Existing Sources
Source: wikibon.org/wiki/v/Big_Data_Vendor_Revenue_and_Market_Forecast_2013-2017 Source: 451 Research – Total Data Warehousing: 2013-2018
Source: The Forrester WaveTM: Big Data Hadoop Solutions, Q1 2014
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle Big Data Analytics Strategy
Data Lake (BDA or Object Storage Service)
Data Factory In-Memory Data Warehouse
Data Discovery Analytic Models
Event Data
Social Data
Enterprise Data
Oracle Visual Analytics (OBIEE)
Oracle Big Data Discovery
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle Big Data Discovery In Preview
12
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Not Easy to Get Analytic Value from Hadoop Data Reservoir
13
• Existing analytic tools fall short – Too much effort on upfront data preparation
– Manual exploration for understanding new data sets
– Depend on ETL to cleanse data and make it ready
– Assume questions known in advance
• Only point solutions emerging – Separate data wrangling, visualization
– Leads to constant context switching
• Need end-to-end capabilities
• Native Hadoop tools are complex – Pig, Oozie, Sqoop, Hive, Spark
• Specialized skills are scarce – Programming languages
(e.g. Map Reduce, Python, Scala)
– Statistics and machine learning
– Command line interfaces
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Big Data Analytics. Requires a Fundamentally New Approach
14
Explore
Transform Discover
Find
An intuitive, interactive and visual user interface
for anyone to quickly find, explore, transform and analyze
data in Hadoop
then share results for enterprise leverage
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
• Navigate a rich catalog of all data in the Hadoop cluster
• Familiar search and guided navigation for ease of use
• Access data set summaries, annotation and recommendations
• Provision your own data through self-service upload
• Data is automatically enriched with extracted locations, terms, sentiment
• Browse personal big data projects and those shared by the community
15
Easily Find Relevant Data Sets
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
• Understand shape of the data. Visualize attributes by type
• Machine learning algorithms sort attributes by importance
• View attribute statistics, data quality and outliers
• See statistical correlations between attribute combinations
• Evaluate whether a data set is worthy of further investment
16
Explore the Data and Understand Potential
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
• Interactive, intuitive user-driven data wrangling
• Library of data transformations to replace values, convert types, collapse, reshape, pivot, group, custom tag, merge and much more
• Data enrichments for inferring location and language. Theme, entity and sentiment enrichments for text
• Preview results, undo, commit and replay transforms
• Run on sample data in memory or full data set in Hadoop
17
Transform and Enrich Data to Make it Ready
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
• Mash up different data sets for deeper perspectives
• Filter through data with powerful search and intuitive guided navigation
• Drag and drop from a rich library of interactive visualizations to compose discovery dashboards
• Publish blended data sets back to Hadoop
• Share projects, bookmarks and snapshots with team members for collaboration
18
Analyze the Data to Discover New Insights
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 19
Oracle Big Data Discovery. The Visual Face of Hadoop
Explore
Transform Discover
Find
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Share Results and Publish for Enterprise Leverage
20
• Share and collaborate with the team
– Share projects, bookmarks and snapshots then collaborate and iterate
• Publish back to Hadoop
– Transforms and enrichments may be applied to original data sets in Hadoop
– Publish blended data sets back to HDFS
• Leverage results in other tools
– Publish data to Hadoop in format optimized for advanced analytic tools (e.g. ORAAH)
– Hadoop compliant BI tools (e.g. OBIFS) can burst out to the masses
– Leverage any native Hadoop tooling (e.g. Pig, Hive, Impala, Python, etc)
– Integrate BDD data sets with DWH to secure, govern and optimize for query performance (e.g. Oracle Big Data SQL)
Oracle Big Data Discovery plays well with the Big Data ecosystem
Explore
Transform Discover
Find
Share & Collaborate
raw data
transformed data
data reservoir
(HDFS)
Publish
data warehouse
business intelligence
advanced analytics
other hadoop tools
Leverage
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle Big Data SQL
21
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Barriers to Big Data Adoption Complexity
• Skills
– Lack tools and training to exploit Big Data
– IT Operations ability administer and manage Big Data
• Integration – Adding Big Data to existing architecture is complex
– Too much effort required in data preparation
• Security
– No clear route to governance or enforcement
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle Big Data Analytics
Prediction & Collaboration
Exploration & Discovery
Data Transformation
Data Profiling & Preparation
Hadoop
Big Data Discovery Visual Face to Hadoop
Big Data SQL Fast, Secure Oracle SQL on All your Data
SQL
Query Offload to Data Nodes
Query Offload to Storage Servers
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle Big Data SQL. One Fast Query Across all Data, in Place
24
Tables in Hadoop
Tables in DB
SQL join
PRODUCT DIMENSION
SALES FACT
ENRICHED WEBLOGS
CUSTOMER DIMENSION
ENRICHED WEBLOG
{"logid":"L23588999", "customerid":"[email protected]", "platform":"Desktop", "eventdate":"11/6/12", "httpcode":"200", "brandname":"Coca-Cola”, "ipaddress":"167.134.198.24", "qty":1}
• What does it do?
– Query tables in Hadoop via SQL as if they lived natively in Oracle DB
• Avoid data movement + data latency
• Use the SQL skills you already have
– Apply security and redaction features of Oracle DB to Hadoop
• How do we make it fast?
– Smart Scan for HDFS. Apply filters and project columns before streaming results back to Oracle DB
– Storage Indexing. Instead of executing full scans – only read the data relevant to a given query
– Complex evaluation. Performing expensive parsing of JSON/XML where the data lives
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
• OBIEE
– Query & Report on Hadoop, NoSQL & Relational
– Certified with 12c & Big Data SQL
• Endeca – Use and extend your existing
investment
Oracle Business Analytics on Big Data
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle 12c In-Memory DB
26
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Accelerate Mixed Workload OLTP
Real Time
Analytics No Changes to
Applications
Trivial to
Implement
Oracle Database In-Memory Goals
100x
27
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Row Format Databases vs. Column Format Databases
Rows Stored Contiguously
Transactions run faster on row format
– Example: Query or Insert a sales order – Fast processing few rows, many columns
Columns Stored
Contiguously
Analytics run faster on column format
– Example : Report on sales totals by region – Fast accessing few columns, many rows
SALES
SALES
28
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle Big Data Analytics
Column Format
Memory
Row Format
Memory
Analytics OLTP Sales Sales
Data Mashups
In-Memory ROLAP
In-Memory M-OLAP
Predictive Analytics Sales
In-Memory Database 100X Faster: OLTP & Analytics
Oracle BI: Visual Analytics Interactive Visual Data Analysis
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle In-Memory Requires Zero Application Changes
Full Functionality - ZERO restrictions on SQL
Easy to Implement - No migration of data
Fully Compatible - All existing applications run unchanged
Fully Multitenant - Oracle In-Memory is Cloud Ready
Uniquely Achieves All In-Memory Benefits With No Application Changes
30
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle Database In-Memory is Trivial to Deploy No changes required to existing applications
Step1: Configure Memory Capacity
– inmemory_area = XXXX GB
Step2: Configure tables or partitions to be in memory
– alter table | partition … inmemory;
Step3: Drop analytic indexes to speed up OLTP
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle Database In-Memory Customer Experience Schneider Electric analysis of up to 2 billion General Ledger entries
0
20
40
60
80
100
2B 300K 30K
S
e
c
o
n
d
s
Analytics Query Results
Row Format Column Format
• Analytic queries 7-128x faster
• OLTP transactions 5-9x faster
• 76% storage savings
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle Applications In-Memory Examples
Oracle Application Module Improvement Elapsed Time
In-Memory Cost Management 1003x Faster 58 hours to 3.5 mins
In-Memory - Financial Analyzer 1,354x Faster 4.3 hours to 11 seconds
In-Memory Sales Order Analysis 1,762x Faster 22.5 minutes to < 1 sec
Subledger Period Close Exceptions 200x Faster 600 seconds to 3 secs
Call Center Ad-hoc query pattern 1247x Faster 129 seconds to < 1 secs
33
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Getting The Most From In-Memory
• Fast cars speed up travel, not meetings
• In-Memory speeds up analytic data access, not:
– Network round trips, logon/logoff
– Parsing, PL/SQL, complex functions
– Data processing (as opposed to access) • Complex joins or aggregations where not much data is filtered before processing
– Load and select once – Staging tables, ETL, temp tables
Understand Where it Helps
34
Know your bottleneck!
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Become a Real-Time Enterprise Using Oracle Database In-Memory
• Data-Driven
• Get immediate answers to any question with real-time analytics
• Agile
• Eliminate latency with analytics directly on OLTP data
• Efficient
• Easily and Non-disruptive deployment accelerates all applications
35
AGILE
EFFICIENT
DATA-
DRIVEN
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
“Now we can run time-sensitive analytical queries directly against our OLTP database. This is something we wouldn't have dreamt of earlier.”
– Arup Nanda, Enterprise Architect Starwood Hotels and Resorts
36
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
“Full support for RAC scale-out means Oracle Database In-Memory can be used on our largest Data Warehouse, enabling more near real-time analytics.”
– Sudhi Vijayakumar, Senior Oracle DBA Yahoo Inc.
37
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
“Downtime is extremely costly for our business. Oracle’s In-Memory architecture takes the right approach to balancing real-time speed with continuous availability.”
– Jens-Christian Pokolm Analyst IT-DB Architecture & Engineering Postbank Systems AG
38
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle Exalytics In-Memory Machine
Oracle Confidential – Internal/Restricted/Highly Restricted 39
Best Platform for…
Optimum Performance
Best End-to-end Experience
Best Total Cost of Ownership
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle Exalytics In-Memory Machine
Oracle Confidential – Internal/Restricted/Highly Restricted 40
Best Platform for…
Oracle’s strategic platform for Analytic applications
Business Intelligence
Enterprise Performance Management
Endeca Information Discovery
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Introducing Exalytics X4-4
• Brand new Software
– Oracle Database In-Memory Option: 100% Data In-Memory
– Oracle Essbase In-Memory option: Maximum concurrency; 2X faster on X4-4
• Brand new Hardware
– Intel Xeon E7-V2; 50% faster clock speed; 50% more processing cores; co-engineered with Oracle
– 50% more memory; 100% more flash; 33% more storage; 10X more networking
Continued exploitation of D-RAM & Flash to make Analytics 100X faster
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Exalytics evolution
X2-4 X3-4
X4-4
Upgrade Kit
1 TB RAM
4 Intel® Xeon® E7-4870
40 physical cores total
3.6 TBs of raw disk capacity
40 Gbps InfiniBand – 2 ports
10 Gbps Ethernet – 2 ports
1 Gbps Ethernet – 4 ports
2 TB RAM
4 Intel® Xeon® E7-4870
40 physical cores total
2.4TB PCI Flash
5.4 TBs of raw disk capacity
40 Gbps InfiniBand – 2 ports
10 Gbps Ethernet – 2 ports
1 Gbps Ethernet – 4 ports
2 TB DRAM
4 Intel® Xeon® E7-8895 v2
60 physical cores total
2.4TB PCI Flash
7.2 TBs of raw disk capacity
40 Gbps InfiniBand – 2 ports
10 Gbps Ethernet- 4 ports
Flash Upgrade kit
Memory and Flash
Upgrade kit
T5-8
+ 1TB RAM + 2.4 TB PCI Flash + bigger disks
+ Super-SKU CPU + even bigger disks
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
• Rapid global growth since GA
• 100+ certified analytic applications and tools, including full suite of OBI and Hyperion applications
• Proven ROI based on superfast analytics and TCO advantages
Oracle Exalytics Solid Customer Momentum
43
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Oracle Big Data Stack
z BY INDUSTRY & LINE OF BUSINESS B
IG D
ATA
A
PP
LIC
ATI
ON
S
DISCOVERY
BU
SIN
ESS
AN
ALY
TIC
S
BUSINESS ANALYTICS
DATA RESERVOIR
BIG
DA
TA
MA
NA
GEM
ENT
DATA WAREHOUSE
SOU
RC
ES
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Questions & Answers
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |