28
© 2007 IBM Corporation IBM Software Group Dynamic Data Warehousing Vince Leat ASEAN SW Group August 2007
© 2007 IBM Corporation
IBM Software Group
Dynamic Data Warehousing
Vince LeatASEAN SW Group
August 2007
IBM Software Group
© 2007 IBM Corporation1
Discussion
� Introduction
� Dynamic Data Warehousing
– Data Partioning
– Workload Control
– Compression
– Data Mining
– Industry Models
� Balanced Warehouse
IBM Software Group
© 2007 IBM Corporation2
� Competitors offer similar products
� Competitors use similar technologyand proprietary technologies are quickly copied
� Breakthrough innovation in products or services is increasingly difficult
� The World is Flat – losing geographic & regulatory advantages
Top of Mind Issues for the World’s Leading CEOs
CEO Challenges
� Execute with maximum efficiency and effectiveness
� Make the smartest business decisions possible
What’s Left?
IBM Software Group
© 2007 IBM Corporation3
“In God we trust; all others bring data”- W. Edwards Deming
An Information Based Strategy
IBM Software Group
© 2007 IBM Corporation4
The Next Generation of Business IntelligenceInsightful, Relevant Information When and Where it’s Needed
Help Solve Crimes by
Delivering Suspect List to Detectives Arriving
at the Crime Scene
Optimizing Police Force Deployments
Crime Rate Reports
OLAP & Data Mining
to Understand Why and Recommend Future Action
Query & Reporting
to Understand What Happened
Information On Demand
to Optimize Real-Time Processes
IBM Software Group
© 2007 IBM Corporation5
Using Information to CompeteDriving Real Business Results in Financial Services
Adjust CD rates & rollover incentives to attract and retain best customers at lowest price
Identify most profitable customers and minimize risk to maximize risk-value reward
Using Analytics to Compete
Drove 25% increase in revenue per customer account over first 3 years
Lowered customer acquisition costs by 83%
Increased customer retention for savings accounts by 87%
Business Results
IBM Software Group
© 2007 IBM Corporation6
Using Information to CompeteDriving Real Business Results in Retail & Distribution
Identify most loyal customers and maximize cross-selling
Maintaining lowest possible inventory while insuring availability
Using Analytics to Compete
Drove same store sales gains in 23 of 24 quarters over 6 year period
Increased market share from 36% to 43%
Saved $1.2 billion in inventory costs over 5 years
Business Results
IBM Software Group
© 2007 IBM Corporation7
Dynamic WarehousingA New Approach to Leveraging Information
Dynamic
Warehousing
Traditional Data
Warehousing
OLAP & Data Mining
to Understand Why and Recommend Future Action
Query & Reporting
to Understand What Happened
Information On Demand
to Optimize Real-Time Processes
Dynamic Warehousing Requires:
1. Real-time access – in context
2. Analytics – as part of a business process
3. Unstructured information – extracted knowledge
4. Extended infrastructure – tightly integrated
IBM Software Group
© 2007 IBM Corporation8
Dynamic warehousing Extending beyond the warehouse to enable information on demand
Process management
Enterprisedata modeling
Informationintegration
Search and text analytics
Master data management
Industryperspective
Data Warehouse
IBM Software Group
© 2007 IBM Corporation9
Dynamic warehousing Traditional
warehousing
More Examples of Dynamic Warehousing in ActionEnabling Information On Demand for Business Advantage
Insurance fraudanalysis and reporting
Identifying potentially fraudulent claims prior to approval and payment
Transforms healthcare
Reporting oncustomer issues
Identifying possible related issues, churn risk and cross-sell opportunities while engaged with the customer
Transforms customer service
Historical sales analysis and reporting
Discovering relevant customer information to identify cross sell opportunities and improve negotiating position at the point of sale
Transforms sales effectiveness
IBM Software Group
© 2007 IBM Corporation10
What are the challenges for traditional warehouses?
Leverage ALL information, including unstructured
Not just for traditional query and reporting
Address expanding needs for analytics
Increasing types of applications lead to varying service level demands
Increasingly mixed workload environmentsand the constantly changing needs of different business constituents
require more dynamic warehousing capabilities
IBM Software Group
© 2007 IBM Corporation11
Traditional warehouse
IBM provides more than just a warehouse DB2 Warehouse provides extended capabilities and value
Embedded analytics (Inline and as a Service)
Multidimensional analysisData mining and visualization
Beyond traditional structured data
Generate and leverage knowledge from
unstructured information
OLTPBenefits of a transactional data server foundation
Optimized for real-time access,High availability and reliability
Scalable, secure and auditable
DWDBMS
Dedicated warehousing
Shared-nothing architecture Advanced data partitioning
Workload management
Deep compression
Reduced storage costsBetter disk utilization
Query speed improvement
Best of Both Worlds Architecture
IBM DB2 Warehouse
Da
ta V
olu
me
s
Unstructured
Structured
IBM Software Group
© 2007 IBM Corporation12
Master data management
Industryperspective
Process management
Informationintegration
Search and text analytics
Enterprisedata modeling
IBM Global Services and Business Partners
SOA Infrastructure
How IBM Enables Dynamic WarehousingIntegrated offerings to enable information on demand
Dynamic Warehouse
IBMDB2 Warehouse
Process MgmtFileNet BPM
WebSphere BPM
Information IntegrationInformation Server
MDMWS Customer CenterWS Product Center
Industry PerspectiveIBM industrydata models
EnterpriseData Modeling
Rational®Data Architect
Search & Text Analytics
OmniFind™Analytics Edition
IBM Software Group
© 2007 IBM Corporation13
Warehousing strategic pillarsGuiding principles for innovation
Actionable InsightBeyond traditional capabilitiesFurther leverage informationExtended business insight
Support broader usage
SimplicityEasy to deploy and integrate
Easy to useEasy to manage
Easy to start and grow as needed
Reliability & PerformanceReliable access to information
Highly availableReal-time performance
Maximized resource efficiency
IBM Software Group
© 2007 IBM Corporation14
IBM Warehouse software A complete, integrated platform
Mo
de
lin
g a
nd
des
ign
Ad
min
istr
ati
on
an
d c
on
tro
l
Data movement and transformation
database management
Performance optimization
Workloadcontrol
Data partitioning
Deepcompression
Embedded analytics
Data mining, modeling and scoring
Visualization andin-line analytics
DB2 Warehouse
IBM Software Group
© 2007 IBM Corporation15
Warehousing Made SimpleCommon Eclipse Based Design Studio for All Administration
SQL Generated From Data
Flow
Data Flow
Control Flow
Enterprise Schema
Data Warehouse
Project
Simplicity
IBM Software Group
© 2007 IBM Corporation16
� Partition a database within a single server or across a cluster of servers
– Scale to support very large data sets
– Minimize impact of complex workloads
– Provide increased parallelism for administration tasks
Data Partitioning
Reliability & Performance
IBM Software Group
© 2007 IBM Corporation17
Streaming Updates andBatch ETL
Short TacticalQueries
Complex Strategic Queries
Workload Control
Reliability & Performance
“Query Traffic Corp”• Prioritize queries across applications• Ensure operational processes are
serviced first• Minimize impact of complex queries or
batch processes
“Accountant”• Keep a lid on “cost”
• Real-time & historical query-execution statistics
• Reports on:• data accessed most frequently• data NOT being accessed• users or groups generating most workload
IBM Software Group
© 2007 IBM Corporation18
Deep CompressionReduced storage costs – better disk utilization – faster queries
� NULL and Default Value Compression
– No disk storage consumed for NULL column values, zero length data in variable length columns and system default values
� Multidimensional Clustering
– Significant index compression can be achieved through block indexes
– One key per thousands of records (vs one key per record with traditional indexes)
� Database Backup Compression
– Smaller backup images; compress index and long tablespaces
� Data Row Compression
� Dictionary based - symbol table for compressing/decompressing data records
Compression of 100GB TPC-H Tables
0%
10%
20%
30%
40%
50%
60%
70%
LineItem Orders Entire DB
DB2
Other DB
Compression of 100GB TPC-H Tables
0%
10%
20%
30%
40%
50%
60%
70%
LineItem Orders Entire DB
DB2
Other DB
3x better
2x better
Reliability & Performance
Complex Queries Comparison( Lower is bet t er )
0
500
1000
1500
2000
2500
3000
V9 Baseline
V9 Compression
Response Times(milliseconds)
Compression Leads to Faster Queries!
IBM Software Group
© 2007 IBM Corporation19
� Associations– Which item affinities (“rules”) are in my data?
– [Beer => Diapers] …single transaction
� Sequences– Which sequential patterns are in my data?
– [Love] => [Marriage] => [Baby Products] …sequential transactions
� Clustering– Which interesting groups are in my data?
– …customer profiles, store profiles
� Classification– How to predict categorical values in my data?
– …will the patient be cured, harmed, or unaffected by this treatment?
� Prediction– How to predict numerical values in my data?
– …how likely a customer will respond to the promotion
– …how much will each customer spend this year?
DB2 Warehouse
Actionable Insight
Data MiningEnhancing Business Insight with Predictive Analytics
� “Easy Mining” algorithms
� Score data directly in DB2, scalable and real time
IBM Software Group
© 2007 IBM Corporation20
Embedded mining with integrated tools Seamless integration of analytics capabilities
Actionable Insight
Simplicity
Filter required data directly in the warehouse
Get subset of data for analysis
Drag-and-drop interface
Seamlessly add specific analytics and mining operations
IBM Software Group
© 2007 IBM Corporation21
Out-of-the-box visualization tools
Can be embedded directly into applications and Web pages
Deliver inline visualization and analyticsEmbedded analytics capabilities
Actionable Insight
Simplicity
IBM Software Group
© 2007 IBM Corporation22
Banking(Banking Data Warehouse)
Financial Markets(Financial Markets Data Warehouse)
� Claims
� Medical management
� Provider and network
� Sales, marketing and membership
� Financials
� Profitability
� Relationship marketing
� Risk management
� Asset and liability management
� Compliance
� Risk management
� Asset and liability management
� Compliance
Health Plan(Health Plan Data Warehouse)
� Customer centricity
� Claims
� Intermediary performance
� Compliance
� Risk management
Retail (Retail Data Warehouse)
� Customer centricity
� Merchandising management
� Store operations and product management
� Supply chain management
� Compliance
Telco(Telecommunications Data Warehouse)
� Churn management
� Relationship management and segmentation
� Sales and marketing
� Service quality and product lifecycle
� Usage profile
Insurance(Insurance Information Warehouse)
Industry data models Leverage industry best practices for faster time to market
New Offering!
EnhancedCapabilities!
Over 400 Customers!
IBM Software Group
© 2007 IBM Corporation23
Introducing IBM Balanced WarehouseTM
A fast track to warehousing
Simplicity� Predefined configurations for reduced
complexity� One number to contact for complete
solution support
Flexibility for growth
� Add BCUs to address increasing demands
� Multiple on-ramps for different needs
� Reliable, nonproprietary hardware for reusability
Optimized performance
� Preconfigured and certified for guaranteed
performance
� Based on best practices for reduced risk
Balanced Configuration Modules
Preconfigured, pretested allocation of software, storage and hardware to support a specified combination of function and scale
Better than an appliance
Balanced WarehouseSIMPLEFLEXIBLE
OPTIMIZED
IBM Software Group
© 2007 IBM Corporation24
IBM Balanced Warehouse Offerings
Starting at $15K
C-Class
Out of the Box Warehousing Solution for SMB Customers
(includes out-of-the-box BI tools)
Scales to 1 TB
Up to 200 Users
Affordable Hardware & Storage
E-Class
Starting at $850KStarting at $275K
D-Class
Departmental Data Marts and Small to Mid-Size Data Warehouses
Large Enterprise Data Warehouses
4TB and Up
Unlimited, Modular Scalability
High End Hardware & Storage
1TB to 5TB (target)
Modular Scalability
Mid-Range Hardware & Storage
Linux on System x Linux on System x AIX on System p
IBM Software Group
© 2007 IBM Corporation25
IBM is the Leading Provider of WarehousingThe Industry Leaders Use DB2 for Warehousing
11 of the top 12 banks
7 of the top 8 auto manufacturers
5 of the top 6 insurance companies
4 of the top 6 general merchandisers
4 of the top 5 specialty retailers
3 of the top 4 food & drug stores
IBM Software Group
© 2007 IBM Corporation26
IBM enables dynamic warehousingDelivering greater value from information
� More dynamic and balanced approach to warehousing is key
� Broad set of capabilities beyond the warehouse required
� IBM provides the most comprehensive platform to address these needs
IBM Software Group
© 2007 IBM Corporation27
Copyright information© Copyright IBM Corporation 2007
IBM CorporationSoftware GroupRoute 100Somers, NY 10589 U.S.A.
Produced in the United States of America03-07All Rights Reserved.
DB2, IBM, the IBM logo, OmniFind, Rational and System z are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries or both.
Other company, product and service names may be trademarks or registered trademarks or service marks of others.The information contained in this documentation is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this documentation, it is provided “as is”without warranty of any kind, express or implied. In addition, this information is based on IBM’s current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this documentation or any other documentation. Nothing contained in this documentation is intended to, nor shall have the effect of, creating any warranties or representations from IBM (or its suppliers or licensors), or altering the terms and conditions of the applicable license agreement governing the use of IBM software.
