2

Big Data,

Big Challenges,

Big Opportunities

Joe McKendrick

Lead Analyst

3

Survey on Big Data

• 298 data management and IT

managers/professionals

• Members of Independent Oracle Users Group

(IOUG); 98% run Oracle Databases

• Large organizations (>10,000 employees) 22%;

small firms (1-500 employees) 16%

• Major industries represented: manufacturing;

government/education/non-profit;

utility/telecommunications/transportation;

retail/wholesale; high-tech

4

Observations

• Big Data is here now, and is flowing through all

organizations.

• With all this Big Data now on the scene, more

needs to be done to educate the business about

the potential of Big Data.

• Capitalizing on Big Data doesn’t mean making

huge financial investments or tearing down your

current infrastructure; rather, it can be

integrated and incorporated into your existing

assets.

5

1.

Big Data is here now,

and is flowing through

all organizations.

6

Total Amount of Data Managed Today

11% of organizations now

manage more than a petabyte of

data ...

...another 20% have data in the

hundreds of terabytes.

7

It’s Mainly Larger Organizations, But...

28% of the largest

organizations have >1PB

8% of medium-size businesses

have >1PB

... Soon, most businesses of

all sizes will have data stores in

the PBs.

8

Many types of data: transactional,

user-generated, machine generated

9

Where It’s Coming From, Right Now:

10

The Problem...

SILOS,

SILOS,

SILOS

11

Growing Amounts of Unstructured Data

12

Industries With the Most Unstructured Data

13

2. With Big Data now

on the scene, more needs

to be done to educate the business

about its potential.

14

Barriers: Business Doesn’t Understand the Value

Yet—Thus, Budgets are Falling Short ...

15

Most Data Executives Do Not Feel Their

Data Infrastructure Is or Will Be Capable

72% of survey respondents are not completely

confident in their IT infrastructure and their

database systems for managing Big Data now ...

81% are not completely confident in their IT

infrastructure and their database systems for

managing Big Data in three years.

16

Where Confidence in Data Infrastructure

is Lowest

17

3.

Capitalizing on Big Data

doesn’t mean making huge

financial investments or

tearing down your current

infrastructure—it can be

integrated into your

existing assets.

18

Big data is a

“natural resource”—

and it’s unlimited!

19

There is Business Value in Big Data

55% of survey respondents

acknowledge that Big Data is

either “extremely” or “very”

important to their

business.

20

Industries Where Big Data Really Matters

“Speed and accuracy are of the essence in

winning new business and maintaining current

customers.”

21

Survey Respondents

Say Big Data Helps Them:

Just a few

other areas of

Big Data value:

customer

profitability,

text analytics,

e-commerce,

risk

management!

22

Different Industries, Different Motivations

23

“As the big data applications begin to

come on line, priorities within the

security/compliance group will become

more risk oriented. This focus will

allow the business to focus resources

toward those items that pose the

highest degree of risk to data.

Additionally, conditions that could be

seen as possible threat vectors or the

beginnings of events can be found

easier.”

24

Technology That Will Get Us There

25

Big Data Foundation Being Built on

Existing, Proven Environments—

Relational Databases

26

How Data is Integrated With BI Applications

32% pre-process

Big Data then load

into data warehouse for

integrated analysis, but

...

... 46% are still unsure

how this

will play out.

27

Are Data Warehouses a Big Company Thing?

36% of large organizations (>10,000

employees) pre-process Big Data

then load into data warehouse for

integrated analysis.

26% of small firms (<100 employees) use

data warehouses to manage

Big Data.

28

Hadoop— especially for ad-hoc queries

and data mining

29

Who Uses Hadoop?

(now/planned for this year)

30

How Hadoop Is and Will Be Used

31

Strive for a

“co-existence” strategy

between data systems—

not either/or.

32

Managing and Staffing

Big Data Environments

33

“We already have as close a

relationship with management as is

possible. We intend to keep it that

way by doing a great job on Big

Data, but we have no idea what the

percentage of the data flying past

[is]us good enough to capture.

Understanding the potential benefits

and liabilities of capturing a wide

range of data beyond traditional

transactions is an open-ended

subject.”

34

Where Do Big Data Projects Originate?

35

Even when business

takes the lead,

IT responsible for

implementation

In larger organizations, others also help oversee

implementations:

54% of respondents in large organizations say

BI/analytics team oversees Big Data projects.

62% of large organizations also charge Big Data

implementations to DBAs.

36

Who Makes It Happen?

37

Business-Side Driver of Big Data Initiatives

38

Financial Decisions

Recommendations

• Develop a business case.

• Get business buy-in and support.

• Develop an integration strategy between

unstructured and “traditional” enterprise

data.

• Strive for a “co-existence” strategy between

data systems—not either/or.

• Develop an integrated information

management lifecycle strategy.