State of Big Data Markets

Kyle Redinger@kyleredinger

Co-Founder, VividCortex

Key Takeaways

Everything is Digital

Data Growth

Technology

Data will grow 50x by 2020; budgets will only grow 1.5x

Data stores are behind everythingMore and more data is being produced

Key enablers have driven data growth, but new opportunities exist

Human Limits Humans need tools to help them manage performance; systems are inherently complex

The World is Changing

The Internet of Everything

Your Travel Remembered

Kyle’s December Travels Visualization from Google

Your Body Recorded

Your World Measured

Voronoi Diagram of 2.5 Million Weather Stations

Your Communications Tapped

Why All this Data?

Advertising, lead generation, services, recommendations, business development, data brokering, cohort analysis, sales funnels, customer insight, new products, new markets, more value etc.

THE MONEY

THE TECHNOLOGY

With legacy licensing and technologies, Big Data is too expensive (i.e. Oracle Exadata clusters $1 to $5 million)…

But, with cheaper technologies, Big Data is the reality and many new opportunities are created (i.e. Hadoop + basic machines, $100k to $500k)

Big Data What is it?

Hype

Anything That Doesn’t Fit in Excel

+Volume, Variety,

Velocity

+

Internet of Everything

BS Reality

The Problem

With data growing quickly, and IT budgets not growing, large gaps between humans and systems will cause problems for companies

IT Budgets Headcount Data & Servers

Data 50x Growth

IT Staff & Budgets 1.5x Growth

In 5 Years

The Enablers

Growth of Data

Virtualization

Cloud Computing

Data Storage

Smart People & Process

Moore’s Law

Management Tools

Open Source

Open Source

2/3rds of IT Leaders Will Purchase 50% of Software as Open Source

High Quality, Growth & Culture

2 Million Open Source Projects in 2014

Reduced Operating Cost, Feature Direction, Community Value

Open Source is Eating the World ($500+ Million invested in 2013)

Moore’s Law Exponential increase in performance per dollar, but reaching limits

1996 1998 2000 2002 2004 2006 2008 2010 20120

50

100

150

200

250

300

CostPerformance

Hardware continues to be less expensive with better performance

Virtualization One piece of hardware can run many different things.

Hardware

Operating System

Application

Hardware

Virtualization

OS OS OS

APP APP APP

Virtualization enabled sharing of underutilized resources

Cloud Moving computing power from capex to utility pricing

Hardware

Virtualization

OS OS OS

APP APP APP

$110 Billion Market, 18% CAGR through 2016

VividCortex15%

Startup 215%

Fortune 500 Client80%

Tech Model Simplified.

Data Storage

APP 3

APP 1 APP 2 APP 4

Network

Application(s)

Performance ManagementTools

Application, Network & Database

Moore’s Law Limits

Market Size

Big Money

$2.5 billion today; growing to $3.6 billion within 4 years

Hardware can’t solve all the problems; data is growing faster than the cost reduction in hardware performance

Downtime, slowness, lost data, bad responsiveness, missed SLAs, etc all destroy value

Human Limits Humans need tools to help them manage performance; systems are inherently complex

Database & Data Storage

When we talk about databases, what are they?

Definition

Technologies

Challenge

Oracle, SQL Server, MySQL, DB2, PostgreSQL, MongoDB, Hadoop … and 100s more

Organized collection of data and “dumb.”

Database technology isn’t innately scalable; i.e. humans need to build systems around the database to make them work

Market Size $25 + Billion (Technology, Services, Tools, Consulting) and Growing Fast

State of Leading Data Storage Markets

Closed source model can’t compete with open source/core model

ORACLE

LICENSE ADOPTION

$$$$$$

TOOLS SERVICES

SQL Server

ENTERPRISE MATURE

$$$$ ENTERPRISE MATURE MATURE

DB2 $$$$$$ ENTERPRISE MATURE

MySQL FREE EVERYONE MEDIUM MEDIUM

MongoDB FREE EVERYONE BASIC BASIC

MATURE

MATURE

OPEN SOURCE DATABASES SCALE FINANCIALLY WITH BIG DATA

SHARE

LEADER; DECLINE

LEADER;DECLINE

DECLINE

LEADER

HYPE

Hadoop FREE EVERYONE BASIC BASICHYPE

Data Storage Market Share of Leading Vendors

$25 Billion Market & Growing 16% Y/Y

Oracle

IBM (DB2)

Microsoft (SQL Server)

Others

$11.8 B

$4.9 B

$4.0 B

$3.3 B Open Source Has Massive Install Base

TOOL GAP

TOOL GAP

Why Are Tools Important

Total Cost of Ownership of a Small Enterprise Project

$0

OPEN SOURCE

LICENCES

UP FRONT

SETUP & HARDWARE

SUPPORT FEES

RECURRING

ADMIN & DEVELOPMENT

3 YEARS

$423,000

ORACLE

$166,000 $397,000

$15,600 $106,760

$90,000 $180,000

$482,200 $1,680,280

Current Tools

DISCOVERING PROBLEMS• False/Missed Alarms• Threshold-based

DIAGNOSIS• Low resolution charts• Application Focus

• Adaptive Fault Detection• Zero-config, no-threshold, real-time, efficient

• High res workload analysis• Problem DNA• Database Focus

VividCortex Solution

TIME, COST & STAFF• Long & expert troubleshooting• Open source command-line• Not scalable

• Faster humans• SaaS, Modern GUI• Cloud-ready and dynamic

The Future of Data Management

Gaps in Human Data Management Capacity

Open Source Solutions

New Vendors & Disruptors

Decline of Legacy Players That Don’t Adjust License Models

Software-as-a-Service for Database Performance Management

VividCortex

Creditshttp://cdn.historycommons.org/images/events/a213_nsa_octopus_2050081722-17000.jpg

http://www.waratuman.com/resources/images/posts/2010-11-04-weather-station-map.png

https://www.gartner.com/doc/2398815

http://blog.infochimps.com/2011/10/28/where-does-the-weather-data-come-from-visualizations-of-worldwide-weather-stations/

http://www.forbes.com/sites/louiscolumbus/2013/02/19/gartner-predicts-infrastructure-services-will-accelerate-cloud-computing-growth/

http://www.fstsummitus.com/media/whitepapers/2013/a_total_-10gen.pdf

http://www.slideshare.net/blackducksoftware/the-2013-future-of-open-source-survey-results