BIG DATA AND OSFAOmer Trajman, Cloudera

END OF OSFA

0

7.5

15

22.5

30

2006 2007 2008 2010 2011 2012

CompaniesProducts

"...the commercial world will fracture into a collection of independent database engines..."

Number of

M. Stonebraker and U. Cetintemel. “One Size Fits All: An Idea Whose Time has Come and Gone”. In Proceedings of ICDE, 2005.

Gartner, Inc., Magic Quadrant for Data Warehouse Database Management Systems, 2006-2012

RESULTING IN...

EDW

Data Marts

ETL Grid

Enterprise Storage

OLTP Logs

Sensors

CEP and ESB

DATA ON OSFA

Kept for limited periods of time

Used for limited applications

Difficult to integrate with other data

Archived (deleted)

Photo credit/Flickr/bcostin

BIG DATA IS UNFILTERED

Different processing engines for different stages.

Lots of data makes it expensive to move.

Dynamic businesses requires flexibility.

Photo credit/Flickr/dervish

BIG DATA MOVES LESS

HDFS

Unified Scale Out Storage and Compute

EDW

CEP and ESB

OLTP Logs

Sensors

Flume Sqoop

Sqoop Flume

MR

MR = ETL?

HDFS

MR

IT'S NOT (JUST) ABOUT MR

HDFS

Avro

Crun

ch

Flum

e

Gira

ph

Ham

a

HBa

se

Hive

Mah

out

MaR

educ

e

Pig

Sqoo

p

OR (JUST) ABOUT OSSIn

form

atica

Micr

oStra

tegy

Table

au

RainS

tor

Had

apt

Dat

amee

r

Karm

asph

ere

NG

data

Plat

fora

Wib

iDat

a

HDFS

Avro

Crun

ch

Flum

e

Gira

ph

Ham

a

HBa

se

Hive

Mah

out

MaR

educ

e

Pig

Sqoo

p

OSFA AND BIG DATA

Flume Sqoop

HDFS

Pig HiveHBase

MicroStrategyInformatica Tableau

GOOGLE FIGHT?