Cox Business Intelligence
& Oracle BI/DW Stack
John Landis, Sr. Director BI & Data Architecture
April, 2012
2
Cox Communications Company Overview
• Is the third-largest cable
entertainment and broadband
services provider in the
country
• Has over 6 million customers
• Has over 22,000 employees
3
Cox Communications Services
• Residential TV, Internet,
Phone, Tech Solutions, Home
Security
• Business TV, Internet, Phone,
Security, Backups, Industry
Services for Real Estate,
Residential Communities,
Education, Government,
Healthcare, and Hospitality
• Media On Air, Online and On-
the-Go
4
CEBI 2008 Problem
• Business customers not
satisfied with multiple
platforms. Not sure where
to get data the right data.
• Business intelligence
platform has multiple
versions of the truth
• Data Integration is fractured
• Data Warehouse has not
had investment in 3 years
• Proliferation of tools has
become expensive and hard
to maintain
• Data needs of the company
are growing, offline
databases at all sites
• Development taking place in
multiple organizations
• No standards exist in the
enterprise
5
Cox Enterprise Business Intelligence (CEBI) 2008
As well
as……
6
CEBI 2008 Transition
Ad-Hoc Query
& Reporting
Standardized
Reporting
Advanced
Analytics
Modeling
Future Oriented
Operational Static
Strategic Dynamic
Past Oriented
Death by 1000 paper cuts
• Adopt a enterprise reporting
application to encourage
collaborative enterprise
development of reporting
across the organization and
lower the cost of reporting
throughout Cox.
• Reuse and optimization of
resources:
• People and Processes
• Application, Data, and
Services
• Time and Cost
7
CEBI 2008 Solution
• Oracle Business Intelligence
Enterprise Edition 10G chosen
as the enterprise BI platform
• Oracle Database chosen as
the Enterprise data platform
and Infomatica chosen as
integration platform
• Business Intelligence
Competency Center Deployed
• Data Warehouse Clean-Up
Begins
8
CEBI 2008 OBIEE 10G
• Total Cost of Ownership
• Common Semantic Layer
• Prebuilt Analytical Options
• Oracle’s Strategic BI Roadmap
• Single Sign On
• Embedded Metadata
• Self Service Reporting
9
CEBI 2008 OBIEE 10G
• Interactive Charts and Graphs
• Personal Dashboards
• One Suite of Tools
• Open Source not ready for
enterprise deployment
• Hyperion Integration
• Personalization
10
CEBI 2008 Solution Cont.
Cox lived happily ever after and I got to retire to my dream location…..
Not Exactly
11
CEBI 2008 Lesson’s Learned
• Data governance is required
• IT can only facilitate data
governance, business needs
to lead
• Training is critical
• Self Service Reporting
requires supervision
• Start small
• Not everyone likes change
• Garbage in, garbage out
• Most people don’t understand
data, therefore carefully create
your RPD
• OBIEE resources are hard to
find
• The business wants you to
challenge them on requirements
• Reports are only as fast as the
database
12
CEBI 2010 Exadata
• With the growing data and reporting needs within the
organization, the platform needed to expand to handle the
projected growth.
• Business data needs went from daily updates to near real
time updates.
• Existing hardware reached it’s capacity and new
technology was needed in order to meet the current and
upcoming demands.
• Without a platform and technology upgrade, data and
reporting would not be made available to the organization.
13
BASELINEEXADATA AS
ISEXADATA NO
INDEXES
EXADATACOMPRESSI
ON/ AGGREMOVAL
EXADATAMIXED LOAD
TESTING
Total Time in Seconds 22,077 9,774 3,998 2,316 2,399
-
5,000
10,000
15,000
20,000
25,000
EXADATA POC -OBIEE Queries Total Run Time (Total of 130 queries executed) BASELINE
EXADATA ASIS
EXADATA NOINDEXES
EXADATACOMPRESSION/ AGGREMOVAL
EXADATAMIXED LOADTESTING
• In April 2010, the EBI team partnered with Oracle
to perform a Proof of Concept (POC).
• Based on the results of the POC, an executive
decision was made to implement the full solution.
• In July 2010 the EBI team began the planning and
rollout of Exadata.
• With the help of the Operations Support group, all
EBI databases were implemented on Exadata in
Production.
CEBI 2010 Exadata
14
CEBI 2010 Exadata Pre-Launch Concerns
• People
- Support structure is different
- Adoption
- Learning curve for support and
development
• Process
- Compression can mask the lack of a
data lifecycle management
- It is not the way we have always
done it
- Performance increases can mask
architectural issues Note: Degradations in
performance caused by development code that should have
been avoided. Nearly 600K IOPS.
• Technology
- Oracle was new to the hardware market
- Technology had limited instances in
production.
- Switching from commodity based storage
to appliance; risk of stranding assets.
- Backup strategy changes and recovery
model changes
- Vendor “lock in” moving away from Oracle
becomes more expensive.
15
CEBI 2010 Exadata
• Reporting
• Nearly 6X improvement out of the box
• Up to 200X query performance
improvement. 9X on average
• Nearly 6X performance increase on the
work orders load (non Exadata source).
2X on average for non Exadata sources
and 10X on average for Exadata to
Exadata loads.
• Some reports showed worse
performance
16
CEBI 2010 Exadata Results
• 5-10x Compression saves Cox money over traditional storage
• Lowers backup time and tapes needed
• Estimated savings in space through 2012 range approx. $2.4M – $4.8M
Compression
• Less tuning reduces project timelines
• Enables near real time processing
• Able to process data previously not possible
• Estimated savings in time in 2012 approx. 5% or $400K
Performance
• Highly available, has uncovered issues in other Cox Oracle systems helping to improve reliability
• Reduces complexity of environment because Oracle has tested the integration points, all hardware is tested to work together unlike commodity solution
Enterprise Availability
• Oracle is our standard database today, no conversion costs were incurred.
• Cox employees already had a skill set in this technology
• Development best practices were enhanced
Leverage Existing Technologies
17
CEBI 2012
18
CEBI 2012 OBIEE 11G
• Score carding
• Mobility
• Improved Visualizations
• Spatial Intelligence via Map-
based Visualizations
• Business Process Invocation
• Packaged Apps
• Exalytics
19
CEBI 2012 Architecture
Financial Data
Enterprise Metadata Layer
Financial
Consolidation
Planning, Budgeting
& ForecastingPROJECT
EMPLOYEE
CUSTOMER
ORGANIZATION
ERP -
FInancials
ScorecardsInteractive
Dashboards
Reporting and
Publishing
Adhoc
Analysis
Office
Integration
Mobile and
Embedded
Enterprise Business Intelligence Platform
Governance and Monitoring
Master Data
Detect and
Alerts
Collaborate &
Seach
OLTP Data
ERP - HR
CRM Logistics
Time &
AttendanceBilling
Data Warehouse
Human
ResourcesFinance
Sales &
MarketingField
Business
Operations
Reporting and Analytics
Data Sources
Pro
ce
ss
Sta
nd
ard
s
20
CEBI 2012 Exadata
• Primary Database Areas:
- Reporting
- Applications
- Web Services
• Standby Database Areas:
- Analysis
- What If
- Predictions
- Data Mining
- Ad-Hoc
21
Process Time 2008 – 10.5 Hours/Night 2009 – 12.5 Hours/Night 2010 – 4 Hours/Night 2011 – 3.5 Hours/Night
Data Volume 2008 – < 2 Billion/Night 2009 – 36+ Billion/Night 2010 – 43+ Billion/Night 2011 – 50+ Billion/Night
Users 2008 – < 1000 2009 – 2500+ 2010 – 5000+ 2011 – 9000+
BICC Migrations/Reviews 2008 – 100 2009 – 1300 2010 – 2218 2011 – 3000+
Errors Per 1M Sessions 2008 – 500 2009 – 150 2010– 125 2011– 100
User Generated Reports 2008 – <700 Usr Rpts 2009 – >6000 Usr Rpts 2010 – 10000+ Usr Rpts 2011 – 15000+ Usr Rpts
Complexity 2008 – Single Billing, Weekly/Nightly Numbers 2011 – Multi Billing, Near Real Time
CEBI 2012 History
22
CEBI Customer Goals
Internal Cox Users want…
Product Planning and
Optimization Data Analysis and
Research
Real-time Operations
Monitoring
Data-driven Sales and
Marketing
Customers want…
Personal
Recommendations Product Personalization Customized Interfaces Personalized Services
Growing Analytical
Needs
Cross Product Usage Cross Team Efforts
23
CEBI Future
Growth is taking place in areas not well served by traditional databases
According to Gartner, Enterprise Data will grow 650% by 2014. 80% of this data will
be Unstructured Data, with a CAGR of 62% per year, far larger than transactional
data.
The 2011 IDC Digital Universe Study Sponsored by EMC
This chart shows the growth over data over
the next couple of years. It is projected that
a large portion of this growth will be
unstructured data (web logs, emails, social
interactions, etc.).
Unstructured data does not work well with
traditional databases. To achieve the low
response times, traditional databases rely
on strict data structures. These strict data
structures work well for certain types of
data. However, the growth of unstructured
data in the enterprise and the proposed
uses of it create the need for a new type of
data processing to be introduced to the
technology stack.
Unstructured data is driving
an explosive growth in data
Structured
data
Unstructured Data in Web Pages
24
CEBI Goals
Create additional value from customer data
Make Cox a more data-driven company
• Increase the perceived value of products by enabling a high degree of individual personalization.
• Give a highly tailored customer experience every time a customer interacts with Cox.
Democratize access to data
• Improve the efficiency and security of Cox operational processes.
• Allow the company to make decisions, spot trends, and react to competitive challenges more quickly.
• Allow the company can make quick, innovative use of the data that is already being generated every day.
• Improve cross-team and company wide insight into how customers are using Cox’s services.
25
CEBI Goals
Cox Data
Structured Data
Characterized by well-known use cases, requiring only
repeatable, static data cubes and ETL’s. Highly
productized results.
Example Use Cases
• Ad-hoc Queries
• Financial and Operational Dashboards
• Ad Impression Analyzer
• Marketing Analyzer
Unstructured/Semi-structured Data
Characterized by lack of established use cases and on-
the-fly analysis in a “Sandbox” manner. Useful in
developing new insights, products.
Example Use Cases • Ad-hoc Queries
• Data Mining/Discovery
• Large Datasets, Fast Response Times
• Predictive Analysis
Traditional Data Architecture Big Data Architecture
Challenges
• Sizing to support new reporting dimensions is not
always economically feasible.
• Analysis against new datasets can slow Time to Market
for new products.
Challenges
•Latency is greater than with traditional databases.
• Large unstructured datasets will need to be monitored
and managed at scale.
Data Decision Framework
26
CEBI Sample Decision Framework
• Used to evaluate analysis use
cases
• Can determine which system to
use:
• Traditional database
• Non-traditional data store
• Can standardize reporting and
analysis use cases across the
enterprise
Latency Complex Simple
High Exadata Any
Low Exadata Exadata
Latency Complex Simple
High Big Data Big Data
Low Big Data Exadata
High Data Volumes
Low Data Volumes
27
Future Solution Design
Data Sources (Raw Data)
Stagie
Replication Virtualization Mediation Ingestion
Transfo
rm
Store
Presen
t Presentation Applications Analytics Services
Pro
cess
ODS Federated NOSQL Hadoop
MDM Virtualization ETL/ELT Map Reduce
Billing Mediation Anonymization Data Cleansing
EDW Virtual Master Store NOSQL
28
POC in Progress
• Exalytics
• Oracle Big Data Appliance
• Endeca Information Discovery
• Packaged Apps