Application-Tier In-Memory Analyticspuredba.co.uk/wordpress/wp-content/uploads/2015/01/CON...•...

Application-Tier In-Memory AnalyticsBest Practices and Use Cases

Susan CheungVice President Product Management

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Vice President Product ManagementOracle, Server TechnologiesOct 01, 2014

Guest Speaker:Kiran TailorSenior Oracle DBA and ArchitectChartered Institute of Management Accountants

Safe Harbor Statement

The following is intended to outline our general product direction. It is intended forinformation purposes only, and may not be incorporated into any contract. It is not acommitment to deliver any material, code, or functionality, and should not be relied upon


commitment to deliver any material, code, or functionality, and should not be relied uponin making purchasing decisions. The development, release, and timing of any features orfunctionality described for Oracle’s products remains at the sole discretion of Oracle.

Program Agenda

TimesTen In-Memory Database Overview

TimesTen for In-Memory Analytics

Common Use Cases and Best Practices

1

2

3


Common Use Cases and Best Practices

Real-life Customer Use Case at CIMA

Product roadmap

Q&A

3

4

5

6

Oracle TimesTen In-Memory DatabaseMicroseconds Response Time in Application Tier

• Persistent and Recoverable– Database and Transaction logs

persisted on disk and flash storage

• Relational Database– Pure in-memory– ACID compliant– Standard SQL

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 5

• Extremely Fast– Microseconds response time

– Very high throughput

Standard SQL

• Compatible with Oracle Database– Data types, PL/SQL, OCI, ODP.NET, PHP, R

– Integrated with RAC, Data Guard,Enterprise Manager, SQL Developer, etc.

TimesTen In-Memory DatabasePersistence and Recovery

• TimesTen database persistence can beconfigured on– Flash, SSD, hard disk storage (HDD)

• All transactions are logged and persisted– Redo, undo, and recovery


– Redo, undo, and recovery

• Dual database Checkpoint files– Database restart

– Database recovery after failures

• Separate storage for Transaction Logs andCheckpoint files if using HDD

6

TimesTenCheckpoint

Files

TimesTenTransaction

Log Files

Application Development with TimesTen

SQL

PL/SQL

Languages C/C++/C#

ODP.NETODBCOCI, Pro*CttClasses

J2EE App Servers OR MappingJava/JDBC

Oracle R


Languages C/C++/C# J2EE App Servers OR MappingJava/JDBC

TimesTen In-Memory DatabaseLow Latency - Microseconds Response Time

Millionthsof a

Second9.07

6.00

8.00

10.00M

icro

seco

nd

sResponse Time


Second

Millionthsof a

SecondTPTBM Read and Update

E7-4890 v2 @ 2.80GHz1 socket, 15

cores/socket, 2threads/core

TimesTen 11.2.2.7.5(100M rows, 17GB)

3.58

0.00

2.00

4.00

6.00

SELECT Query UPDATE Transaction

Mic

rose

con

ds

25,192,607

15,000,000

20,000,000

25,000,000

30,000,000

Qu

eri

esP

er

Seco

nd

TimesTen In-Memory DatabaseRead Scalability – 25 Million Queries per Second 25 Million

Queries perSecond


279,2620

5,000,000

10,000,000

15,000,000

1 10 20 30 60 90 120

Qu

eri

esP

er

Seco

nd

Concurrent Processes

9

TPTBM 100% ReadE7-4890 v2 @ 2.80GHz

4 sockets, 15cores/socket, 2threads/core


800,000

1,000,000

1,200,000

1,400,000U

pd

ate

Tran

sact

ion

sP

er

Seco

nd

Update Throughput Per Processor

TimesTen In-Memory Database1.3 Million Update Transactions Per Second Per Processor

1.3 MillionUpdate TPS


0

200,000

400,000

600,000

800,000

1 5 10 15 20 25

Up

dat

eTr

ansa

ctio

ns

Pe

rSe

con

d

Concurrent Update Processes

10

TPTBM 100% UpdateE7-4890 v2 @ 2.80GHz

1 socket, 15cores/socket, 2threads/core


TimesTen In-Memory Database3.4 Million Transactions Per Second Per Processor

2,000,000

2,500,000

3,000,000

3,500,000Tr

ansa

ctio

ns

Pe

rSe

con

dMixed Workload Throughput Per Processor

3.4 MillionTransactionsPer Second

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 11

TPTBM 100% MixedWorkload (80-10-5-5)

E7-4890 v2 @ 2.80GHz1 socket, 15 cores/socket,

2 threads/coreTimesTen 11.2.2.7.5(100M rows, 17GB)

0

500,000

1,000,000

1,500,000

2,000,000

1 5 10 15 20 25

Tran

sact

ion

sP

er

Seco

nd

Concurrent Processes

80-10-5-5 Workload = 80% select, 10% updates, 5% inserts, 5% deletes

TimesTen 6 TimesTen 11g TimesTen 11g 11.2.2.x

Oracle TimesTen – Pure In-Memory Relational DatabaseVery Fast Response Time for Very High Throughput in Application Tier

1996|

2005

2006|

2008

2009|

2011

2012|

20132014


TimesTen 6TimesTen 7

Pre-Oracleacquisition

TimesTen 11g11.2.1

TimesTen 11g11.2.2

• 1998 First commercialIn-Memory RDBMS

• HA Replication• Online Upgrades• Application-tier Cache

for Oracle Database

• PL/SQL and OCI Support• Oracle Clusterware Integration• Cache Grid for Scale Out• ODP .NET Support• BLOB, CLOB, NCLOB data types

• Parallel Replication• In-Memory Analytics• Columnar Compression• Index Advisor• Oracle R Support• In-Memory Star Join• Oracle Golden Gate Integration

11.2.2.xEnhancements

• Parallel data import fromOracle Database

• Parallel database restart• Highly concurrent range indexes

• Oracle RAC integration• National Language Support• Oracle Data Types support• SQL Developer Integration• Enterprise Manager integration

Analytics Use Cases forOracle TimesTen


In-Memory Analytics - Summary Aggregates Scenarios

• Aggregations are suitable for analysis of higher-level grains of FACT data

• Knowledge of query patterns required

• Aggregated tables and indexes typically much smaller than detail-tables

• Reports using summary aggregates typically provide exceptional response time (sub-second)

Reports Using Aggregate Set 1




Summary AggregateTables and Indexesstored in TimesTen

12

OLTP

Operational Data Store

Data Warehouse

In-Memory Analytics - Data Mart Use Cases

• Data mart and operational data stores–For analysis where summary aggregates are not sufficient

–Require access to detail source tables (fact and dimension tables)

• Consider “hot” set of data*

Subset of Data from a Data Warehouse


–Data volume constrained by availability of RAM in the system

–May use compression to include more data

• Common use cases have both aggregations and detail tables

• If the entire data warehouse is too big to fit into memory,only load portion of the data to TimesTen

In-Memory AnalyticsODS and Data Mart Detail Tables Scenarios

Operational Reports

Operational Data Store


Reports with AggregatesAND Detail Tables

Aggregates + Detail tables +indexes

OLTP

Data Warehouse

Best PracticesTips and Tricks


TimesTen In-Memory DatabaseStorage Speed Matters to Database Restart Time

• Storage speed matters

• Faster read operations using Flash andSolid State Disks


• Higher throughput with parallel threads

• Significant time reduction for databaseload to memory

18

TimesTenCheckpoint

Files

Dual Parallelism – Oracle TimesTen and Intel NVMe P3700Speed up Database Load Time to Memory

630500

600

700

100GB Database Load to MemoryTime in Seconds

Database Load Time


212

490

100

200

300

400

HDD 15K RPM Sata SSD Intel NVMe P3700

19

TimesTen 11.2.2.7 on Oracle Linux, Intel Xeon E5-2690 3.0 Ghz, 2 processors 20 core server

Dual Parallelism – Oracle TimesTen and Intel NVMe P3700Multiple Flash Cards Lower Load Time and Higher Throughput

4240

50

60

100GB Database Load to MemoryTime in Seconds

2.2 GB/sSingle NVMe P3700

100GB Database Load to MemoryRead Rate


30

0

10

20

30

40

Single NVMe P3700 Striped Two P3700

20

TimesTen 11.2.2.7 on Oracle Linux, Intel Xeon E7-4890 v2 2.8 Ghz, Two 15-core Processors

3.3 GB/s

0 1 2 3 4

Striped Two P3700Cards

TimesTen Cost-Based SQL Optimizer

• Hash indexes– Best performance for equality matches

– Must be properly sized for good performance

– Undersized hash indexes can result in severe performance penalties

• Range Indexes

Proper Indexes and Up-to-date Statistics Optimized Query Execution Plans


• Range Indexes– Range indexes best for range search

– Can ALTER Unique PK range indexes Default index type for primary key indexes

– to hash indexes

• Up-to-date table and column statistics– Important for a cost-based SQL optimizer

Summary

• TimesTen is an excellent choice for in-memory analytics in the applicationtier

• Correct setup and usage is vital to achieve good performance and easymanagement

– OS, configuration, operations, persistence storage, tuning, etc.


– OS, configuration, operations, persistence storage, tuning, etc.

• Use the tools available to make your life easier and get the best results

• Many more best practices available

• Visit TimesTen product center on Oracle Technology Network

• Refer to TimesTen documentation for more information (available online)

Most Widely Used Relational In-Memory DatabaseDeployed by Thousands of Companies


Charter Institute of Management Accountants (CIMA)


Charter Institute of Management Accountants (CIMA)Real-life Use Case of In-Memory Analytics

Kiran TailorSenior Oracle DBA and ArchitectChartered Institute of Management Accountants

CIMA, Exalytics and TimesTen

Kiran TailorSenior Oracle DBA and ArchitectThe Chartered Institute of Management Accountants

Date: 01/10/2014

About Me

• Kiran Tailor, Senior Oracle DBA/Architect

• Sixteen years working with Oracle

• Started working with Oracle Forms, moved to databases

• Oracle Databases for 15 years

01/10/2014© CIMA CIMA,Exalytics,TimesTen 2

• OBIEE for more than five years

• Our mission is to help people and businesses to succeed in the public andprivate sectors

• We have more than 218,000 members and students in 177 countries

• They work at the heart of business in industry, commerce and not for profitorganisations

CIMA

• We have 28 offices around the world, employing 450 people

• We constantly update our qualification. This ensures that our membersand students are first choice for employers who are recruiting financiallytrained business leaders

• In 2012, together with the AICPA, we launched the CGMA designation


Our Challenges

•Meaningful information from the systems, that we could action upon instantly


Our Challenges

• As systems were slow. Staff/Business managers constantly phoning forinformation


Our Challenges

• Marketing Campaign Reports – 8 minutes

• Exam Reports – 2 minutes individually, thousands to do

• Anything with Activities – 15 minutes+


CIMA Before Exalytics


OLTP Siebel CRM

DW

Exalytics/TimesTen common Setup


OLTP Siebel CRM

DW

TimesTen CIMA Set Up

Siebel CRM

Siebel DW


TimesTen DAC CIMA Set Up

How DAC identifiesrecords in TimesTen

© CIMA CIMA,Exalytics,TimesTen 10


How DAC identifiesrecords that need to be

copied to TimesTen

01/10/2014© CIMA CIMA,Exalytics,TimesTen


How DAC identifiesrecords in TimesTen at


records in TimesTen attask level


How DAC identifies which records have

Siebel CRM01/10/2014© CIMA CIMA,Exalytics,TimesTen 13

How DAC identifies which records havechanged in the warehouse that need to

be copied to TimesTen

Outcome

• Full warehouse approx 300gbloaded into memory

• Full load took approx 3 hours

• Incremental nightly copyapproximately 6 minutes

• Achieved huge performance upliftwith zero application code changes

• Enabled previously unachievableanalysis, leading to greaterknowledge of customers andchallenges across the company

• 100x + performance improvement,in some cases much higher

• iPad apps roll-out ongoing

INTELLIGENCE AND BUSINESS RESULTS2014© CIMA CIMA,Exalytics,TimesTen 14

100%In-memory

Daily load time

6 mins

ZeroApplication

codechanges Performance Improvement

Feedback

“It’s all a lot better. It’s so quick now it’s difficultto measure without an atomic stopwatch.” UKUser

“I didn’t used to use the old dashboards to the extentthat I do now- maybe that’s partly because I used toget so fed up with the speed that I gave up whereas

2014© CIMA CIMA,Exalytics,TimesTen 15

get so fed up with the speed that I gave up whereasnow it’s there at the press of a button” UK User

Report

UK - WAN connection on standard Analytics 228.6 secondsOutside of UK - Wifi connection on Analytics not possible dueto errorOutside of UK - Wifi connection on Times Ten In Memory DW1.1 second via webex

Deployment Tips

• In RPD make sure you use correct TimesTen ODBC settings

• We used DAC to create the table and index scripts

• INITIAL load: run the table scripts on the TimesTen Server, load the data usingthe DAC, run the create index scripts on the TimesTen server, update the stats onTimesTen

• OBIEE Caching: Linux Ram Disk – cache into memory!


• OBIEE Caching: Linux Ram Disk – cache into memory!

• If using Exalytics and LDAP for OBIEE, follow the weblogic admin guide forconfiguration

• Keep checking support for Patch updates

• If any columns are being used to do updates make sure they are indexed.

• Update the stats constantly especially if you are updating and inserting rows

Спасибо!

Dziękujemy!

Thank you!

謝謝！Спасибо!

!شكرا لك

Kiran [email protected]

Product Roadmap


Product Roadmap

TimesTen 12c Roadmap

• In-memory columnar storage– Row-based and Columnar co-exist in the same TimesTen database

– Entire database is in memory (just like today)

• TimesTen Grid scale-out architecture– Single image Distributed database, transactional and persistent

In-Memory Database Technology in Application Tier


– Single image Distributed database, transactional and persistent

– Transparent data distribution and parallel query processing

– Elastic scalability and availability

TimesTen In-Memory Columnar

• Process Billion rows per second

• Columnar and row-based tables co-exist– User choose preferred format at table level

• Store more data in memory– No Indexes required

Everything In Memory – Row-based Storage plus Columnar Storage

Row-basedtables

plus


– No Indexes required

– Lower administration cost

• Offload analytics processing• Data and Operational isolations from production

data warehouse

• Complementary to Oracle Database In-Memoryoption

tables

Columnar

Co-exist in same TimesTenin-memory database

TimesTen In-Memory Grid Database Architecture

• Distributed In-Memory RDBMS

• Single database image across all hosts

• Application transparent datadistribution

• Connect to any instance and access

Distributed – Scale Out – Always Available – Fault Tolerant

Instance 1

App App App App

Single Image IMDB


• Connect to any instance and accessto ALL data on the entire Grid

• Parallel query processing• High availability (k-safety, k >= 2)• Fault-tolerant

• Scale-out to hundreds of machines

TimesTen GridTimesTen Grid

Instance 2

Instance 3

InstanceN

Instance 5

Instance 4

Q & A


Q & A


Date post:	22-Apr-2020
Category:	Documents
Upload:	others
View:	1 times
Download:	0 times

Application-Tier In-Memory Analyticspuredba.co.uk/wordpress/wp-content/uploads/2015/01/CON...•...

Documents