Business Analytics:

Collapsing The Stack

Timo Elliott September 2011

2

Business Analytics Has Struggled to Keep Up

“Where are you going? Ah -- If I were you, I wouldn’t start from here”

3

Reporting

“Typical” Business Intelligence Today

Slow

Painful

Expensive

Operational Data Store

Data Warehouse

Indexes

Aggregates

DataBusiness Applications

Copy

ETLCalculation EngineBusiness Intelligence

Query Results

Query

Slow

Painful

Expensive

Operational Data Store

Data Warehouse

Indexes

Aggregates

DataBusiness Applications

Copy

ETL

Calculation EngineBusiness Intelligence

Query Results

Query

Data

Marts

4

It’s like an Onion…

the more layers there

are, the more it makes

you cry

5

What’s the Problem?

Slow Disks & CPUs

I/O Bottleneck

Expensive Memory

Optimized for Transactions

BI is an Afterthought

30 Year-Old Database Design Principles

6

A Revolution…

Credit Suisse, “The Need for Speed”

7

Today’s Disks Can’t Keep Up With Processing Power

8

In-Memory Computing Costs Have Plummeted

BT Tower

152m

Cost of 1 Mb of

memory in 2000: ≈£1

9

In-Memory Computing Costs have Plummeted

Cost of 1 Mb of memory

today: ≈ ½ p

My daughter:

1.30m

And shrinking, and

shrinking, and shrinking….

Price/performance of

in-memory has

DOUBLED in last 9

months

10

In-Memory Computing

Operational Data Store

Data Warehouse

Indexes

Aggregates

DataBusiness Applications

Copy

ETL

Calculation EngineBusiness Intelligence

Query Results

Query

Up to 1,000x faster

No optimizations requiredData

Marts

11

Row vs. Column Databases

My Filing System

My Wife’s Filing System

Row-based Column-based

12

Row-Based Data

Wasted space,

and a full scan to

aggregate any

particular field

13

Column Data

More efficient data storage, better compression, faster queries

14

Data WarehouseData Warehouse

Column Databases

Operational Data Store

Data Warehouse

DataBusiness Applications

Copy

ETL

Calculation EngineBusiness Intelligence

Query Results

Query

Up to 1,000x faster

More data in less space

15

Data Warehouse

Massively Parallel Hardware

Operational Data Store

DataBusiness Applications

Copy

ETL

Business IntelligenceQuery Results

Query

Up to 1,000x faster

Optimized for hardware – especially good for column stores

Calculation Engine

16

In-Database Processing

user changes

a plan value

52 weeks x 500 branches = 26000 values

26000 database writes 1 database write

17

A Database Designed for Business

Volume Driver

Cycles

Driver

Forecast Driver

Forecast Agents

Grow

Seasonal Complex

Assortment Planning

Cumulate

Days

Days Outstanding

Discounted Cash Flow

De-cumulate

Delay

Delay Debt

Delay Stock

Annual Depreciation

Annual Depreciation

Diminishing Balance

Depreciation

Sum of Year Depreciation

Year To Date Statistical

YOY/ YOY Difference

Forecast Dual Driver

Forecast Sensitivity

Feed

Feed Overflow

Forecast

Funds

Future Value

Inflated Cash Flow

Internal Rate of Return

Moving Median

Number of Periods

Net Present Value

Outlook

Payment

Present Value

Lag

Last

Lease

Lease Variable

Linear Average

Forecast Mix

Moving Average/Sum

Proportion

Rate

Repeat

Seasonal Simple

Seasonal Simulation

Stock Flow

Stock Flow Reverse

Stock Flow Batch

Time

Time Sum

Max Value

Minimum Value

Transform

Rounding

Up until now, there’s been a false separation between application logic and

database functionality

18

In-Database Analytics

Forecasting ClusteringAnomalies

Influencers Trends Meaningful or Random?

19

Data Warehouse

In-Database Analytics

Operational Data Store

DataBusiness Applications

Copy

ETL

Business IntelligenceQuery Results

Query

Up to 1,000x faster

Push processing down to dedicated hardware, less traffic

Analytic Appliance

Calculation Engine

20

Integrating Flows of Data

Incremental loads, replication

21

Integrating Flows of Data

22

Streaming Data

23

Real-Time Data

Operational Data Store

Copy

ETL

Real-time replication — why have a separate operational data store?

DataBusiness Applications

Analytic ApplianceBusiness Intelligence

25

The Basis For Applications of The Future

Copy

Business Applications

Analytic ApplianceBusiness Intelligence

Use a single appliance for both analytics and applications

Data

26

Applications of the Future

27

Virtuous Circle of Technology

In-Memory

Columnar Databases

Hardware Acceleration

Calculation Engine

Columnar storage

increases the

amount of data that

can be stored in

limited memory

(compared to disk)

Column databases

enable easier

parallelization of

queries

In-memory processing

gives more time for

relatively slow updates

to column data

In-memory allows

sophisticated calculations

in real-time

Hardware acceleration

makes sophisticated

calculations like

allocations possible

Each technology works well

on its own, but combining

them all is the real

opportunity — provides all of

the upside benefits while

mitigating the downsides

28

Extended Architecture

Business Applications

Analytic ApplianceBusiness Intelligence

Cloud computing

Unstructured and personal data

Mobile revolution

Collaboration

29

In-Memory Computing is Like Digital Photography

A transformative

technology that slowly

but surely upturns the

whole industry

Faster, Easier, More

Convenient

Evolved Faster Than

The Alternatives

30

It’s All About Flexibility and Evolution

“It's not the strongest that

survive, nor the most

intelligent, but the ones

most responsive to

change.”

Charles Darwin

31

Reality Is, and Always Will be, Messy

Different information

sources

Different

levels of

expertise

Different access

devices

Different time

horizons

Different levels of

analytic need

Different

project phases

Risk

Politics

But new architectures mean simplification and new opportunities

32

What About Flash Disk / SSDs?

15X

9000X

16X

Cost-effective, but not a revolution

33

What About Big Data / NoSQL / Hadoop?

Thanks!

Email:

timo.elliott@sap.com

BI Blog:

timoelliott.com

You Should Follow Me on Twitter: @timoelliott