1 © Copyright 2011 EMC Corporation. All rights reserved.
How to Successfully Exploit Big Data for Business Advantage Bill Schmarzo, CTO EMC Consulting Julie Lockner, Senior Analyst and VP of Data Management Enterprise Strategy Group
2 © Copyright 2011 EMC Corporation. All rights reserved.
Agenda
• EMC Consulting – It’s a Whole New Big Data World – The Big Data Opportunity
• Enterprise Strategy Group – 2011 Data Analytics Research – Priorities – Challenges – Drivers
• Take Action
• Q&A
3 © Copyright 2011 EMC Corporation. All rights reserved.
It’s a Whole New Big Data World …
EMC Consulting
4 © Copyright 2011 EMC Corporation. All rights reserved.
Big data analytics provides potential for more timely, more complete, more actionable business insights
Less than 10% of available enterprise data
Vast majority of available data, including external sources
“Rearview mirror” reports, dashboards, and analysis
“Forward looking” predictions with recommendations
Weeks, months, or even quarters old Real-time or near real-time
Incomplete, inaccurate, and disjointed data
Correlated, high confidence, governed data
Architectures and methods that take 6 to 18 months to exploit
Vastly accelerated time to market
Today’s Situation Big Data Analytics Ramifications
5 © Copyright 2011 EMC Corporation. All rights reserved.
Deliver to any device at any time
Gain new insights on customers, products, and
operations
More than just data volume, smart big data strategies also consider the velocity, variety, and complexity of information
Smart Grid
Images
Audio Video Text
Documents
Transactional: ERP, CRM, SCM,
POS
Social media
Sensor/ location-based
Industry-specific
Web traffic
6 © Copyright 2011 EMC Corporation. All rights reserved.
“Through 2015, organizations integrating high value, diverse new information sources and types into a coherent information management infrastructure will outperform industry peers financially by more than 20%”
The impacts to both the Business and IT are significant, and early adopters will fundamentally change their industries
• More agile, more real-time, more accurate decision-making
• Predict and spot changes in dynamic and volatile markets
• Deeper understanding of customer preferences and behavior
• Greater fidelity in risk assessment and compliance enforcement
Source: "The New Value Integrator," Insights from the Global Chief Financial Officers Study
Business Impacts IT Impacts
• Enhanced user experience that delivers insights to any device
• Operationalization of data scientists and analytic insights
• Tools and processes for data quality, governance, and security
• Cloud for self-service, collaboration, agility, and cost reduction
7 © Copyright 2011 EMC Corporation. All rights reserved.
The Big Data Opportunity
8 © Copyright 2011 EMC Corporation. All rights reserved.
With big data, leading companies are making forward-looking decisions about customers, products, and operations using all available data in real time with complete data confidence
Social Media Site Telco Insurance Healthcare Payer
• Capture and analyze PBs of unstructured and structured data
• Time to market new features reduced from weekly to daily
• A/B testing changed UEX which increased time spent on site 50% and increased game revenue 3x
• Analysis across entire customer set generated a social network graph based on calling patterns
• Within 2 weeks identified customers who were 7x more likely to change providers
• Integrate and analyze patient demographics and treatment data to consolidate data silos and detect potential fraud in real-time vs. 2-3 weeks after claim authorization
• Enabled faster decisions in fraud detection for evidence-based specialty care
• Calculate catastrophic risk at household vs. zip code level
• Optimize hurricane policy pricing and renewal decisions for specific coastal area households at the policy level to reduce book of business risk
©2011 Enterprise Strategy Group
Enterprise Strategy Group | Getting to the bigger truth. T M
ESG Research 2011 Data Analytics Research Data
October 2011
Julie Lockner, Senior Analyst and VP of Data Management
Enterprise Strategy Group
Data Growth Is the Top Database Challenge
© 2011 Enterprise Strategy Group 10
1%
14%
27%
21%
27%
34%
36%
31%
28%
31%
51%
52%
51%
1%
1%
3%
4%
4%
4%
6%
7%
9%
11%
15%
15%
19%
0% 10% 20% 30% 40% 50% 60%
Other
Provisioning servers
Provisioning storage
Application data model knowledge transfer
Creating test/development environments
Keeping up with current version of supported database release
Patch & maintenance processes
Supporting databases in virtualized environments
Lack of skilled staff
Deploying new database technology platforms
Maintaining security/compliance
Keeping up with database performance requirements
Managing data growth and database size
In general, which of the following challenges does your organization have with its current database environment and supporting infrastructure? Which would you characterize as the
primary challenge for your organization? (Percent of respondents, N=270)
Primary database challenge
All database challenges
Which Industries Have the Most Database Data?
© 2011 Enterprise Strategy Group
20%
33%
40%
43%
46%
52%
44%
30%
33%
25%
28%
23%
30%
24%
29%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Health Care (N=25)
Manufacturing (N=61)
Communications & Media (N=20)
Financial (N=46)
Retail/Wholesale (N=24)
Total amount of database data, by industry. (Percent of respondents)
100 TB or more 10 TB to 99 TB Less than 10 TB
11
Data Analytics Is a Top IT Priority for Many Organizations
© 2011 Enterprise Strategy Group 12
Our most important IT priority, 6%
One of our top 5 IT priorities, 45%
One of our top 10 IT priorities, 27%
One of our top 20 IT priorities, 15%
Not among our top 20 IT priorities, 5%
Don’t know, 2%
Relative to all of your organization’s IT priorities over the next 12-18 months, how would you rate the importance of enhancing data analytics activities? (Percent of
respondents, N=270)
Similar Challenges Plague Data Analytics Projects
© 2011 Enterprise Strategy Group 13
21%
21%
25%
28%
29%
34%
47%
0% 10% 20% 30% 40% 50%
Storage requirements are too expensive
Current data analytics license costs are too expensive
Current database license costs are too expensive
Unable to complete analytics in a reasonable period of time
Data set sizes limit our ability to perform analytics
Lack of skills necessary to properly manage large data sets and derive value from them
Data integration is complex
Which of the following data analytics challenges has your organization experienced? (Percent of respondents, N=270, multiple responses accepted)
What about Data Integration Challenges?
© 2011 Enterprise Strategy Group
4%
18%
20%
21%
27%
28%
29%
35%
39%
0% 5% 10% 15% 20% 25% 30% 35% 40% 45%
We do not have data integration issues
Lack of adequate technology
Lack of adequate skills
Integrating data from cloud-based (i.e., SaaS) applications
Poor data quality inhibits proper data integration
Lack of process ownership due to disparate applications
It is difficult to add or integrate new data sources
Data volumes are too large
Integration processes take too long
Which of the following data integration challenges are currently facing your organization? (Percent of respondents, N=240, multiple responses accepted)
14
While Key Skills Are Sorely Lacking
© 2011 Enterprise Strategy Group
13%
15%
17%
17%
21%
24%
28%
31%
45%
45%
0% 10% 20% 30% 40% 50%
Data analyst
Server virtualization administrator
Application administrator
Data scientist
Business analyst
Enterprise architect
IT infrastructure (i.e., servers, storage, etc.) administrator
Application developer
Data architect
Database administrator
In which of the following areas of your database/data management environment does your organization have a shortage of skills? (Percent of respondents, N=75,
multiple responses accepted)
15
23%
17% 17% 17%
7% 7%
3%
1%
10%
0%
5%
10%
15%
20%
25%
Less than 250 GB
250 GB to 499 GB
500 GB to 999 GB
1 TB to 5 TB 6 TB to 10 TB
11 TB to 25 TB
26 TB to 50 TB
More than 50 TB
Don’t know
On average, approximately how much data is processed as part of a typical data analytics exercise? (Percent of respondents, N=270)
More than Half Are Processing at least ½ TB at a Time!
© 2011 Enterprise Strategy Group
Big Data Volume Zone
16
12%
38%
20% 17%
8% 5%
0%
5%
10%
15%
20%
25%
30%
35%
40%
We typically integrate from 2
unique data sources
We typically integrate from 3
unique data sources
We typically integrate from 4
unique data sources
We typically integrate from 5
unique data sources
We typically integrate from more than 5 unique data
sources
Don’t know
On average, how many data sources does your organization need to integrate in order to support data analytics activities (i.e., feeds to a data warehouse, business
intelligence system, etc.)? (Percent of respondents, N=252)
Nearly Half Are Integrating Data from at least 4 Sources
© 2011 Enterprise Strategy Group
Big Data Variety Zone
17
15%
38% 36%
6%
3% 2%
0%
5%
10%
15%
20%
25%
30%
35%
40%
Real-time Near real-time (within a day)
Batch – daily Batch – weekly Batch – monthly Batch – intermittently
How frequently is data typically added and/or updated during the integration process? (Percent of respondents, N=240)
More than Half Update Data in Real-time or Near Real-time
© 2011 Enterprise Strategy Group
Big Data Velocity Zone
18
Top Five Drivers for Evaluating New Data Analytics Solutions
© 2011 Enterprise Strategy Group 19
23%
25%
25%
26%
27%
21% 22% 23% 24% 25% 26% 27% 28%
New business processes have generated new data that needs to be analyzed
New application deployments and/or upgrades placed a new strain on current data management solutions
Organization is moving towards more real-time analytics
Current data analytics solution(s) does not meet requirements/needs
Cost reduction of existing platforms
What requirements are driving your organization to evaluate new data analytics solutions? (Percent of respondents, N=102, three responses accepted)
Expected Benefits of Big Data Analytics
© 2011 Enterprise Strategy Group
22%
26%
26%
30%
34%
34%
43%
44%
55%
0% 10% 20% 30% 40% 50% 60%
Ability to accommodate new data types
Simplified data integration
Reduced infrastructure costs
Ability to leverage existing resources (i.e., staff)
Reduced deployment time and cost
Ability to complete analytics on larger data sets
Easier to manage
Ability to complete analytics in a shorter period of time
Improved business agility
Which of the following benefits does your organization expect to derive from deploying a new data analytics solution? (Percent of respondents, N=102, multiple
responses accepted)
20
ESG Recommends Learning from Experienced Experts
© 2011 Enterprise Strategy Group 21
• Data analytics is a top IT priority
Challenges are only compounded by more data volumes and demand for faster results, as well as more data variety and complex integration challenges
• Leverage hired experts to augment your team
Big Data requires new techniques and technologies to harness value
Incorporate a train-the-trainer regiment in the methodology
• Build a plan that can accommodate future change
Leverage experience to look beyond today’s challenges
22 © Copyright 2011 EMC Corporation. All rights reserved.
Take Purposeful Action on the Big Data Imperative
© Copyright 2011 EMC Corporation. All rights reserved. 23
Pick the best-fit, priority business opportunity
Big Data Advisory Service: find the right big data business opportunity and build a comprehensive plan/roadmap
1
© Copyright 2011 EMC Corporation. All rights reserved. 24
Pick the best-fit, priority business opportunity
Build use cases that drive next generation BI and analytics
Develop a vision for applying cloud capabilities
Create a conceptual architecture for a more agile data platform
Assess readiness of data quality, governance, and security
Big Data Advisory Service: find the right big data business opportunity and build a comprehensive plan/roadmap
3
4
2
5
1
© Copyright 2011 EMC Corporation. All rights reserved. 25
Pick the best-fit, priority business opportunity
Build use cases that drive next generation BI and analytics
Develop a vision for applying cloud capabilities
Create a conceptual architecture for a more agile data platform
Assess readiness of data quality, governance, and security
Big Data Advisory Service: find the right big data business opportunity and build a comprehensive plan/roadmap
Integrate findings into a phased roadmap
3
4
2
5
6 1
© Copyright 2011 EMC Corporation. All rights reserved. 26
Q&A
© Copyright 2011 EMC Corporation. All rights reserved. 27
Next Steps • Contact us today for a follow up call or visit
– Bill Schmarzo [email protected] – Julie Lockner 508.377.3410 [email protected]
• Read our blogs – http://www.enterprisestrategygroup.com/bigger-data/ – http://infocus.emc.com
• Download our white papers – Big Data Analytics: Gain Competitive Advantage from the Combination of
Big Data and Advanced Analytics http://www.emc.com/collateral/emc-perspective/h8668-ep-cloud-big-data-analytics.pdf
– ESG Big Data Advisory http://www.emc.com/collateral/analyst-reports/esg-emc-consulting-big-data-advisory.pdf
© Copyright 2011 EMC Corporation. All rights reserved. 28
THANK YOU