Business Analytics: The Big Leap Forward

Timo Elliott September 2011

2

Top Business Issues 2011

3

Business Analytics Provides Great Value

Data is extremely important for competitiveadvantage

Data makes an important contribution to customer relations efforts

Business information has helped manage costs or improve operations

Executives believe companies can benefit greatly from using data, especially information generated within the company

Agree: 69% Agree: 77% Agree: 70%

4

Surging Growth in Business Analytics

2009 2010

+3.8%

+13.4%

Gartner: worldwide BI, analytics and performance management software revenue

BI Growth more than tripled between 2009 and 2010!

5

Analytics is an Ever-Increasing Share of IT Budget

2009 2010 2011

3.9%

+4.1%

+4.3%

Gartner: worldwide BI, analytics and performance management software revenue

“BI spending has far surpassed IT budget growth overall for several years”

Dan Sommer, Gartner

6

Business Analytics Around the World

Business Analytics MarketGrowth 2010

3.0%

3.7%

6.7%

17.8%

18.3%

19.5%

22.9%

Eastern Europe

Japan

Western Europe

North America

Middle East and Africa

Latin America

Asia/Pacific

13.2%

11.6%

Gartner Market Share Analysis: Business Intelligence, Analytics and Performance Management Software, Worldwide, published March 2011

77

Market Consolidation Continues

8

Business Analytics Market (BI, EPM, Analytic Applications)Share of Market, 2010

Business Analytics Market Shares

SAP

Oracle

SAS Institute

IBM

15.6%

13.2%

11.6%

23%

Gartner Market Share Analysis: Business Intelligence, Analytics and Performance Management Software, Worldwide, published May 2011

The 4 “Megavendors” continued to increase their market share– smaller vendors took from each other

9

Business Analytics is Nothing New

The “What” doesn’t fundamentally change — but the “How” does

10

Business Analytics Has Struggled to Keep Up

11

Reporting

“Typical” Business Intelligence Today

Slow

Painful

Expensive

Operational Data Store

Data Warehouse

Indexes

Aggregates

DataBusiness Applications

Copy

ETLCalculation EngineBusiness Intelligence

Query ResultsQuery

Slow

Painful

Expensive

Operational Data Store

Data Warehouse

Indexes

Aggregates

DataBusiness Applications

Copy

ETL

Calculation EngineBusiness Intelligence

Query ResultsQuery

DataMarts

12

What’s the Problem?

Slow Disks & CPUs

I/O Bottleneck

Expensive Memory

Optimized for Transactions

BI is an Afterthought

30 Year-Old Database Design Principles

13

A Revolution…

Credit Suisse, “The Need for Speed”

14

Today’s Disks Can’t Keep Up With Processing Power

15

In-Memory Computing Costs have Plummeted

Oslo Plaza117m

Cost of 1 Mb of memory in 2000: ≈$1

16

In-Memory Computing Costs have Plummeted

Cost of 1 Mb of memory today: ≈ ½ cent

My daughter:1m

And shrinking, and shrinking, and shrinking….

Price/performance of in-memory has

DOUBLED in last 9 months

17

In-Memory Computing

Operational Data Store

Data Warehouse

Indexes

Aggregates

DataBusiness Applications

Copy

ETL

Calculation EngineBusiness Intelligence

Query ResultsQuery

Up to 1,000x fasterNo optimizations required

DataMarts

18

Row vs. Column Databases

My Filing System

My Wife’s Filing System

Row-based Column-based

19

Row-Based Data

Wasted space, and a full scan to aggregate any particular field

20

Column Data

More efficient data storage, better compression, faster queries

21

Data WarehouseData Warehouse

Column Databases

Operational Data Store

Data Warehouse

DataBusiness Applications

Copy

ETL

Calculation EngineBusiness Intelligence

Query ResultsQuery

Up to 1,000x fasterMore data in less space

22

Data Warehouse

Massively Parallel Hardware

Operational Data Store

DataBusiness Applications

Copy

ETL

Business IntelligenceQuery Results

Query

Up to 1,000x fasterOptimized for hardware

Calculation Engine

23

In-Database Analytics

Forecasting ClusteringAnomalies

Influencers Trends Meaningful or Random?

24

In-Database Analytics

25

A Database Designed for Business

Volume DriverCyclesDriverForecast DriverForecast AgentsGrowSeasonal ComplexAssortment PlanningCumulateDaysDays OutstandingDiscounted Cash FlowDe-cumulateDelayDelay Debt

Delay StockAnnual DepreciationAnnual DepreciationDiminishing Balance

DepreciationSum of Year DepreciationYear To Date StatisticalYOY/ YOY DifferenceForecast Dual DriverForecast SensitivityFeedFeed OverflowForecastFundsFuture Value

Inflated Cash FlowInternal Rate of ReturnMoving MedianNumber of PeriodsNet Present ValueOutlookPaymentPresent ValueLagLastLeaseLease VariableLinear AverageForecast MixMoving Average/Sum

ProportionRateRepeatSeasonal SimpleSeasonal SimulationStock FlowStock Flow ReverseStock Flow BatchTimeTime SumMax ValueMinimum ValueTransformRounding

Up until now, there’s been a false separation between application logic and database functionality

26

Data Warehouse

In-Database Analytics

Operational Data Store

DataBusiness Applications

Copy

ETL

Business IntelligenceQuery Results

Query

Up to 1,000x fasterPush processing down to dedicated hardware, less traffic

Analytic Appliance

Calculation Engine

27

Integrating Flows of Data

Incremental loads, replication

28

Integrating Flows of Data

29

Streaming Data

30

Real-Time Data

Operational Data Store

Copy

ETL

Real-time replication — why have a separate operational data store?

DataBusiness Applications

Analytic ApplianceBusiness Intelligence

31

Real-Time Analytics on Big Data

32

The Basis For Applications of The Future

Copy

Business Applications

Analytic ApplianceBusiness Intelligence

Use a single appliance for both analytics and applications

Data

33

Virtuous Circle of Technology

In-Memory

Columnar Databases

Hardware Acceleration

Calculation Engine

Columnar storage increases the amount of data that can be stored in limited memory (compared to disk)

Column databases enable easier parallelization of queries

In-memory processing gives more time for

relatively slow updates to column data

In-memory allows sophisticated calculations

in real-time

Hardware acceleration makes sophisticated

calculations like allocations possible

Each technology works well on its own, but combining them all is the real opportunity — provides all of the upside benefits while mitigating the downsides

“By 2012, 70% of Global 1000 organizations will load detailed data into memory as the primary method to optimize BI application performance.”

- Gartner

35

6.6

41.9

HANA

Traditional DB

Data Compression with HANA (GB)

3.2

5.1

5.1

1050

1320

2660

Query 1

Query 2

Query 3

Query Run-Time (Seconds)

6.3x Data Compression

369x Average Query Speed-Up

No Schema Changes

Same Data

Same SQL

Immediate Benefits

Large Bank – 1 Month of Customer Information

36

37

38

Extended Architecture

Business ApplicationsAnalytic Appliance

Business Intelligence

Cloud computingUnstructured and personal dataMobile revolutionCollaboration

39

Do More, Faster

“The time between ‘event’ and ‘action’ is rapidly closing.”

“In the past, managers could take weeks or days to make important decisions, however to effectively compete globally, some companies are making critical decisions in hours, minutes or even seconds”

Paul Barsch, 2009

“If things seem under control, you’re just not going fast enough.”

-Mario Andretti

40

In-Memory Computing is Like Digital Photography

A transformative technology that slowly but surely upturns the whole industry

Faster, Easier, More Convenient

Evolved Faster Than The Alternatives

41

It’s All About Flexibility and Evolution

“It's not the strongest that survive, nor the most intelligent, but the one most responsive to change.”

Charles Darwin

42

Reality Is, and Always Will be, Messy

Different information sources

Different levels of expertise

Different access devices

Different time horizons

Different levels of analytic need

Differentproject phases

RiskPolitics

But new architectures mean simplification and new opportunities

43

What About Big Data / NoSQL / Hadoop?

44

What About Big Data / NoSQL / Hadoop?

Complementary technology

Very real value, but immature

Primarily used today for preprocessing unstructured data

Velocity

Volume Variety

New analytic

platforms

HADOOP

46

What About Flash Disk / SSDs?

15X

9000X

16X

47

“Poor-quality customer data costs U.S. businesses $611 billion a year. Yet nearly half of the companies surveyed admit they have no plans to improve data quality”

The Data Warehousing Institute study

What About Data Qwality?

48

Real-Time Data Quality

If everything’s incremental, when do we do data cleansing?

Levels of quality

In-db cleansing

49

Applications for Data Stewards

50

What About Social Data?

51

What About Unstructured Data?

Column stores are good at storing text data.

Can push the text analytics algorithms into the appliance, more flexibility

53

Text Data Processing for Unstructured Data

http://experience.sap.com/twitterta/sapsummit.jsp

http://experience.sap.com/twitterta/sapsummit.jsp�

54

What About End-User Enablement?

Data Warehouse

ApplicationData DepartmentData

Personal Data

From “self-service BI” to“self-service Data Warehousing”

What About Ease of Use?

Top Roadblocks to BI Success

Challenge Rank

Complexity of BI tools and interfaces 1Cost of BI software and per-user licenses 2

Difficulty accessing relevant, timely, or reliable data 3

Insufficient IT staffing or excessive software requirements for IT support 4

Difficulty identifying applications or decisions that can be supported by BI 5

Lack of appropriate BI technical expertise within IT 6

Lack of support from executives or business management 7

Poor planning or management of BI programs 8

Lack of BI technology standards and best practices 9

Lack of training for end users 10

1. Doug Henschen, InformationWeek, “BI Efforts Take Flight”, Oct 13, 2008

Donkey Kong

Grand Theft Auto

58

Progressive Expertise

View Reports Strategic Analysis

59

Use the Power to Improve Ease of Use

No longer query –wait – analyze –format …

Iterative feedback loop allows instant feedback and learning

60

WYN-WYN-WYNMobile Opportunities

More People, More Often, More Context

61

Mobile Isn’t Only About “Mobile”

64

10k m

De NHM kijker

Eerste Romeinsenederzetting: “OppidumBatavorum”Jaartal: 12 voor Chr.Afstand: 300 meter

0.3

Augmented Reality

65

Filter by: Branch

HighstreetOperations +23%

NE 0.1km

Augmented Corporate Reality

66

Augmented Corporate Reality

67

Filter by: Maintenance History

Tower Pipe 3Last Maintenance: 2 Weeks

E 0.1km

Photo by Thomas Hawk, Flickr

http://www.flickr.com/photos/thomashawk/�

68

Store 23Current sales: $15k

SE 0.1km

Filter by: Store Performance

69

“Computers are useless.

- Pablo Picasso

They can only give youanswers.”

70

What About Decision Making?

70

We rely onpeople! Source: IDC

71

Collaboration Around Data

Supermarine Spitfire

Jay Wright Forrester,Inventor of RAM Memory

72

Social Intelligence Needs The New Architectures

Expertise location — Relationship Mining — Social Network Analysis

73

Putting Social Into Business Processes

“The big failure of social business is a lack of integration of social tools into the collaborative workflow.”

74

Did You Know…

75

76

The REAL Big Leap Forward

© SAP 2008 / Page 76

Breadth and Sophistication of Possible Analytical Tasks

Perc

enta

ge o

f Use

rs D

oing

or

Thin

king

abo

ut th

ese

task

s

Quantitative Thinking Gap

Huge opportunity to make business people more productive and efficient, increase their satisfaction, save money for the company, and drive more revenue.

77

Conclusion

In-memory industry revolutionEvery company in the industry heading the same directionDon’t be the last one shooting on film

Beyond data warehousingBecomes part of the operational systemsPlatform for business applications of the future

Start experimentingThese systems are real, and can provide benefits today

Thanks!

Email:telliott@timoelliott.com

BI Blog:timoelliott.com

timoelliott.com/blog/docs/affecto_keynote.pdf

You Should Follow Me on Twitter: @timoelliott

http://timoelliott.com/blog�

Slide Number 1Top Business Issues 2011Business Analytics Provides Great ValueSurging Growth in Business AnalyticsAnalytics is an Ever-Increasing Share of IT BudgetBusiness Analytics Around the WorldMarket Consolidation ContinuesBusiness Analytics Market SharesBusiness Analytics is Nothing NewBusiness Analytics Has Struggled to Keep Up“Typical” Business Intelligence TodayWhat’s the Problem?A Revolution…Today’s Disks Can’t Keep Up With Processing PowerIn-Memory Computing Costs have PlummetedIn-Memory Computing Costs have PlummetedIn-Memory ComputingRow vs. Column DatabasesRow-Based DataColumn DataColumn DatabasesMassively Parallel HardwareIn-Database AnalyticsIn-Database AnalyticsA Database Designed for BusinessIn-Database AnalyticsIntegrating Flows of DataIntegrating Flows of DataStreaming DataReal-Time DataReal-Time Analytics on Big DataThe Basis For Applications of The FutureVirtuous Circle of TechnologySlide Number 34Large Bank – 1 Month of Customer InformationSlide Number 36Slide Number 37Extended ArchitectureDo More, FasterIn-Memory Computing is Like Digital PhotographyIt’s All About Flexibility and EvolutionReality Is, and Always Will be, MessyWhat About Big Data / NoSQL / Hadoop?What About Big Data / NoSQL / Hadoop?What About Flash Disk / SSDs?What About Data Qwality?Real-Time Data QualityApplications for Data StewardsWhat About Social Data?�What About Unstructured Data?Text Data Processing for Unstructured DataWhat About End-User Enablement?What About Ease of Use?Donkey KongGrand Theft AutoProgressive Expertise Use the Power to Improve Ease of UseMobile OpportunitiesMobile Isn’t Only About “Mobile”Slide Number 62Slide Number 63Augmented RealityAugmented Corporate RealityAugmented Corporate RealitySlide Number 67Slide Number 68Slide Number 69What About Decision Making?Collaboration Around DataSocial Intelligence Needs The New ArchitecturesPutting Social Into Business ProcessesDid You Know…Slide Number 75The REAL Big Leap ForwardConclusionSlide Number 78