BI Mega Trends

Where we are today!

Thomas Zimmer

HP Business Intelligence

BI Forum 2009

2 27 November 2009

Over the past 20 years, BI has evolved

Y2K bug

fails to bite

Query and

reporting

technologies

converge

Advent of

e-business

Oracle SQL

RDBMS

SAP BW

1.0

Structured and

unstructured data

converge

Operational BI

embedded in

business process

Data privacy

and security

Managing

information

as an asset

Packaged BI

applications

emerge

Business

Drivers

Information

as a strategic

differentiator

Data

integration

technologies

converge

Business

process

reengineering

Balanced

scorecard

introduced

Rise of the

technology-enabled

knowledge worker

ETL

emergesTDWI is

founded

The Health

Insurance

Portability and

Accountability

Act becomes

law

Dr. E.F. Codd

defines the

principles of

OLAP

Howard

Dresner defines

“business

Intelligence”

Bill Inmon

defines “data

warehousing”

Cutter Consortium

survey finds that

20% of DW

projects fail

Technology

Drivers

Sarbanes-

Oxley

Patriot Act

becomes law

META Group

survey finds

that more than

50% of DW

projects fail

1985 2000 2005 20101990 1995

3 27 November 2009

What Businesses are Saying…• “Data is growing at a compounding rate and we

have multiple non-integrated source systems. Our IT environment is too expensive to operate, manage, and maintain.”

• “We have multiple versions of the „truth‟ and I can‟t reconcile these reports.”

• “We‟re having problems complying with regulations because our metrics are inconsistent.”

• “Our sources often supply data of dubious qualityand I don‟t have control over the sources.”

• “We spend too much time fixing data and not enough time analyzing data. “

• “Our users are asking for analysis reports and intelligence in near-real time and we can‟t keep up with the demands.”

• “While our users continually scream for wider access we need to protect our data assets from unauthorized access and threats.”

4 27 November 2009

Pro

duct P

erf

orm

ance

Time or Engineering Effort

Executive Information System

Independent Data Mart

Enterprise Data Warehouse

Data Provisioning

Platform

BI technology S-Curve

5 27 November 2009

Executive Information System (EIS)P

roduct P

erf

orm

ance

Time or Engineering Effort

• Very few users, typically management

• Standard reports, usually monthly

• Often run on production system

• High IT requirement

• Full centralized control, minimal user flexibility

• No persistent analytic data store

• Minimal data integration

Dimensional

Proprietary

PC based (if you‟re lucky)

6 27 November 2009

Data Warehousing: Independent DMsP

roduct P

erf

orm

ance

Time or Engineering Effort

• Rise of the power user

• Query capability and analysis added

• Dimensional systems; typically departmental

• Introduction of BI tools and normalization for more user flexibility

• Rise of data redundancy and synchronization problems

• Heavy management overhead

• Data integrated from multiple sources via ETL

Dimensional (star schema)

OLAP

SMP based

Oracle

7 27 November 2009

Data Warehousing: Enterprise DWP

roduct P

erf

orm

ance

Time or Engineering Effort

• Eliminates redundant data; single version of

the truth• Ad hoc query and cross-functional analysis

• Reduces ETL costs; continuous data loading

possible• Re-centralizes control

• Integrates data from multiple sources, but requires agreement on a single data model

• Complex to manage, not adaptable to change

Normalised

ROLAP, MOLAP

MPP

Teradata, DB2

Blue Chip

Out Of Business

(fast)

Intelligence

Speed

Out Of Business

(slowly)

The Winners !

The Critical Balancing Act

9 27 November 2009

Data Warehouse Evolution Model

Strategic, predictive

Strategic + tactical, post-predictive

What Why did it What will What is What do I

happened? happen? happen? happening? want to happen?

Stage 1

REPORTING

Stage 2

ANALYTICAL

Stage 3

PREDICTIVE

Stage 4

OPERATIONAL

Stage 5

ADAPTIVE

Inactive Reactive Proactive /Autonomous

Custom scripts ETL + scripts EAI + ETL + scripts TDM or maxing out EAI

BATCH RIGHT-TIME NEAR REAL REAL TIME

Weeks Days Hours Minutes Seconds Sub-seconds

CH

AS

M

10 27 November 2009

Data Provisioning?? Operational BI??P

roduct P

erf

orm

ance

Time or Engineering Effort

Appliances?

Columnar?

Exadata?

Teradata?

??

??

Neoview

HP Confidential 11

• Strategic

• Reporting

• Standalone

• Weekly batch updates

• Simple ETL

• Single-function departmental data marts

• Few users doing strategic analysis

• Data volume < 1 TB

• Response time and availability not critical

• Summarized data

• Operational

• Automating action

• Mission-critical component

• Continuous online updates

• Sophisticated data integration

• EDW supporting “single version of the truth” for multiple applications

• Thousands of users performing many types of tasks

• Data volumes at more than 100 TB

• Near-real-time response and 24x7 online everything

• Detail plus years of history

Business intelligence is evolving to become an integral part of business operations

Why business intelligence matters

Capitalize on regulatory change

Respond to market opportunities

Increase profitability

Improve efficiency and productivity

Optimize growth opportunities

Reduce the cost of running the business

Maximize up sell and cross sell

Manage operational and financial risk

Grow and retain customer base

Connect your business to new opportunities

New World Requirements

Workload

MixQuery

Complexity

Active Data Warehousing

3-5 Way

Joins

Normalized

TB’s

MB’s

GB’s

Query Data

Volumes

10 TB

Old World

New World

15 TB

20 TB

Multiple, Integrated

Stars and Normalized

15+ way Joins +

OLAP operations +

Aggregation +

Complex “Where”

constraints +

Views

Parallelism

Batch Reporting,

Repetitive Queries

“Iterative”, Ad Hoc Queries

Data Analysis/Mining

Near Real Time Data Feeds

Simple

Star

Multiple,

Integrated

Stars

Data Storage

Schema

Sophistication

5-10 Way

Joins

5 TB

# of

Concurrent

Queries

1,000

15 27 November 2009

Questions?

“Even if you‟re on the right track, you‟ll get

run over if you just sit there.”Will Rogers

Copyright © 2007 Hewlett Packard. All rights reserved.