Post on 10-Apr-2018
transcript
8/8/2019 TomPeters_Business Intelligence Overview
1/17
Business Intelligence:
What Does it Mean to Me?
Tom Peters
B.I. Practice Lead
Z Y Solutions Corporation
2007 Z Y Solutions Corporation
What is Business Intelligence
The Challenge
Getting to the required information quickly and accuratelyto make decisions
The Problem
Too much information that is not in a consistent usableformat readily available to those who need it
The Solution
Business Intelligence
2007 Z Y Solutions Corporation
Components of Business Intelligence
Data
Data store collection by individual applications
Production systems
Gathering of all data stores Data warehousing
Presentation
Scorecards & Dashboards
Enterprise Reporting
OLAP Analysis
Advanced & Predictive Analysis
Alerts & Proactive Notification
2007 Z Y Solutions Corporation
8/8/2019 TomPeters_Business Intelligence Overview
2/17
What is a Data Warehouse
A generic term for a system of storing, retrieving
and managing large amounts of data
Software often includes sophisticated compression
and hashing techniques for fast searching and
filtering
Typically remote database containing recent
snapshots of data
Planners and researchers can freely use this
information without worrying about slowing down
operation of the production database
2007 Z Y Solutions Corporation
What is a Data Mart
Type of data warehouse designed mainly to address
a specific function or departments needs
Often uses aggregation or summarization of the
data to enhance query performance
Important, however, to maintain the ability to
access the underlying base data to enable drill-
down analysis as necessary
2007 Z Y Solutions Corporation
Do I Data Warehouse?
Is the production database designed to be carved up
by the desired dimensions?
Designed for Online Transaction Processing (OLTP)
Is database index for all dimensions Can I create additional indexes
What affect will vendor updates have on the solution
2007 Z Y Solutions Corporation
8/8/2019 TomPeters_Business Intelligence Overview
3/17
8/8/2019 TomPeters_Business Intelligence Overview
4/17
Do I Data Warehouse?
Will I be replacing my production application
Design around your business NOT your source applications
Insulate from source applications allow for replacement
2007 Z Y Solutions Corporation
Do I Data Warehouse?
Do I need to aggregate data due to very high
transactions counts
Improve performance
Limit return row count
2007 Z Y Solutions Corporation
Do I Data Warehouse?
Always a good idea
Start simple
Grow as demands increase
2007 Z Y Solutions Corporation
8/8/2019 TomPeters_Business Intelligence Overview
5/17
B.I. Terms
Relational Database (RDB)
A database that conforms to the relational model
Relational Database Management System (RDBMS)
Refers to the software used to create a RDB
Informix
Microsoft SQL Server
Oracle
2007 Z Y Solutions Corporation
B.I. Terms
Online Transaction Processing (OLTP)
Operational systems
High volume data collection
2007 Z Y Solutions Corporation
B.I. Terms
Online Analytical Processing (OLAP)
Defined as providing fast access to shared multi-
dimensional data
Used to generically refer to software and applications thatprovide users with the ability to store and access data in
multi-dimensionally
2007 Z Y Solutions Corporation
8/8/2019 TomPeters_Business Intelligence Overview
6/17
Types of OLAP
Multidimensional Online Analytical Processing
(MOLAP) MOLAP cubes are built for fast data retrieval and are
optimal for slicing and dicing operations
Advantages
Excellent performance
Can return complex calculations
Disadvantages
Limited in scope as definition of cube creates boundaries
Limited in volume of data
2007 Z Y Solutions Corporation
Types of OLAP
Relational Online Analytical Processing (ROLAP)
Manipulates the data stored in the relational database togive the appearance of MOLAP's slicing and dicing
functionality.
Advantages
Does not constrain data
No data size limitation
Can leverage functions of RDB
Disadvantages
Each request must query the RDB
ROLAP itself is limited to RDB functionality
2007 Z Y Solutions Corporation
MOLAP vs. ROLAP
Total Revenue and Costs
In Jan 2004 and Jan 2003
At Top 10 Revenue Stores
MOLAP Manipulations Allow People To Slice-
and-Dice a Subset of Data To View It from Many
Different Perspectives
ROLAP Architecture Allows People To Drill Anywhere in The
Entire Relational Database Across All Dimensions and From
Summaries To Transactional-level Detail
ROLAP AnalysisMOLAP Analysis
Geography
Products
Tim
e
Revenue for Laptop Computers
In 2007
At All Stores
Revenue for All Electronics
In 2003 and Q1 2004
At Stores in the NE Region
2007 Z Y Solutions Corporation
8/8/2019 TomPeters_Business Intelligence Overview
7/17
Types of OLAP
Hybrid Online Analytical Process (HOLAP)
HOLAP combines the advantages of both MOLAP and ROLAP
Advantages
Combines both MOLAP and ROLAP
Disadvantages
Not available from most B.I. vendors or vendor has limited
experience in deploying
2007 Z Y Solutions Corporation
Metadata
Data about data
Dimension A perspective that can be used to analyze the data Dimensions become more useful when there are many descriptive
attributes that can be used for analyzing the data. The termAttribute is often used to describe the extended Dimension
Examples Customer Item Date
Fact The raw enumerable piece of information about the transaction always
a numeric value (usually aggregatable)
Examples Quantity Unit Price Count
2007 Z Y Solutions Corporation
Metadata
Measure
The product of one or more Facts
Can be the result of a formula derived from the RDB or the B.I.
Tool analytical engine
Examples
Quantity
Unit Price
Count
Quantity * Unit Price
Average(Unit Price)
Minimum(Quantity)
2007 Z Y Solutions Corporation
8/8/2019 TomPeters_Business Intelligence Overview
8/17
Metadata
Report Templates
Filters Prompts
Transformations
Consolidations
Drill Maps
Hierarchy
2007 Z Y Solutions Corporation
Metadata
Customer
PK Custom erNumber
CustomerNameCustomerAddress
InvoiceLine
PK,FK2 InvoiceNum berP K L in e Nu mb er
F K1 I t em N um be r QuantityUnitPrice
InvoiceHeader
P K I n vo i ce N u mb e r
InvoiceDateFK1 CustomerNumber
ItemMaster
P K I t em N u mb e r
ItemDescription
ItemSKU
PK,FK1 Item NumberP K ,F K 2 S K U
SKUMaster
P K S KU
LocationMaster
P K L o ca t io n
SKULocation
P K ,F K 2 S K UPK,FK1 Location
Quantity
OrderHeader
P K O r de r N um b er
OrderDateFK1 CustomerNumber
OrderLine
PK,FK1 OrderNum berP K L in e Nu mb er
F K2 I t em N um be r
2007 Z Y Solutions Corporation
Metadata
Customer
PK Custom erNumber
CustomerNameCustomerAddress
InvoiceLine
PK,FK2 InvoiceNum berP K L in e Nu mb er
F K1 I t em N um be r QuantityUnitPrice
InvoiceHeader
P K I n vo i ce N u mb e r
InvoiceDateFK1 CustomerNumber
ItemMaster
P K I t em N u mb e r
ItemDescription
ItemSKU
PK,FK1 Item NumberP K ,F K 2 S K U
SKUMaster
P K S KU
LocationMaster
P K L o ca t io n
SKULocation
P K ,F K 2 S K UPK,FK1 Location
Quantity
OrderHeader
P K O r de r N um b er
OrderDateFK1 CustomerNumber
OrderLine
PK,FK1 OrderNum berP K L in e Nu mb er
F K2 I t em N um be r
Item
2007 Z Y Solutions Corporation
8/8/2019 TomPeters_Business Intelligence Overview
9/17
Metadata
Customer
PK Custom erNumber
CustomerNameCustomerAddress
InvoiceLine
PK,FK2 InvoiceNum berP K L in e Nu mb er
F K1 I t em N um be r QuantityUnitPrice
InvoiceHeader
P K I n vo i ce N u mb e r
InvoiceDateFK1 CustomerNumber
ItemMaster
P K I t em N u mb e r
ItemDescription
ItemSKU
PK,FK1 Item NumberP K ,F K 2 S K U
SKUMaster
P K S KU
LocationMaster
P K L o ca t io n
SKULocation
P K ,F K 2 S K UPK,FK1 Location
Quantity
OrderHeader
P K O r de r N um b er
OrderDateFK1 CustomerNumber
OrderLine
PK,FK1 OrderNum ber
P K L in e Nu mb er
F K2 I t em N um be r
Item
2007 Z Y Solutions Corporation
Metadata
Customer
PK Custom erNumber
CustomerNameCustomerAddress
InvoiceLine
PK,FK2 InvoiceNum berP K L in e Nu mb er
F K1 I t em N um be r QuantityUnitPrice
InvoiceHeader
P K I n vo i ce N u mb e r
InvoiceDateFK1 CustomerNumber
ItemMaster
P K I t em N u mb e r
ItemDescription
ItemSKU
PK,FK1 Item NumberP K ,F K 2 S K U
SKUMaster
P K S KU
LocationMaster
P K L o ca t io n
SKULocation
P K ,F K 2 S K UPK,FK1 Location
Quantity
OrderHeader
P K O r de r N um b er
OrderDateFK1 CustomerNumber
OrderLine
PK,FK1 OrderNum berP K L in e Nu mb er
F K2 I t em N um be r
Item
Customer
InventoryLocation
StockKeeping
Unit
2007 Z Y Solutions Corporation
Metadata
Customer
PK Custom erNumber
CustomerNameCustomerAddress
InvoiceLine
PK,FK2 InvoiceNum berP K L in e Nu mb er
F K1 I t em N um be r QuantityUnitPrice
InvoiceHeader
P K I n vo i ce N u mb e r
InvoiceDateFK1 CustomerNumber
ItemMaster
P K I t em N u mb e r
ItemDescription
ItemSKU
PK,FK1 Item NumberP K ,F K 2 S K U
SKUMaster
P K S KU
LocationMaster
P K L o ca t io n
SKULocation
P K ,F K 2 S K UPK,FK1 Location
Quantity
OrderHeader
P K O r de r N um b er
OrderDateFK1 CustomerNumber
OrderLine
PK,FK1 OrderNum berP K L in e Nu mb er
F K2 I t em N um be r
Item
Customer
InventoryLocation
StockKeeping
Unit
Dimensions
2007 Z Y Solutions Corporation
8/8/2019 TomPeters_Business Intelligence Overview
10/17
Metadata
Customer
PK Custom erNumber
CustomerNameCustomerAddress
InvoiceLine
PK,FK2 InvoiceNum berP K L in e Nu mb er
F K1 I t em N um be r QuantityUnitPrice
InvoiceHeader
P K I n vo i ce N u mb e r
InvoiceDateFK1 CustomerNumber
ItemMaster
P K I t em N u mb e r
ItemDescription
ItemSKU
PK,FK1 Item NumberP K ,F K 2 S K U
SKUMaster
P K S KU
LocationMaster
P K L o ca t io n
SKULocation
P K ,F K 2 S K UPK,FK1 Location
Quantity
OrderHeader
P K O r de r N um b er
OrderDateFK1 CustomerNumber
OrderLine
PK,FK1 OrderNum ber
P K L in e Nu mb er
F K2 I t em N um be r
Item
Customer
InventoryLocation
StockKeeping
Unit
InventoryQuantity
InvoiceQuantity
InvoiceUnit Price
Dimensions
2007 Z Y Solutions Corporation
Metadata
Customer
PK Custom erNumber
CustomerNameCustomerAddress
InvoiceLine
PK,FK2 InvoiceNum berP K L in e Nu mb er
F K1 I t em N um be r QuantityUnitPrice
InvoiceHeader
P K I n vo i ce N u mb e r
InvoiceDateFK1 CustomerNumber
ItemMaster
P K I t em N u mb e r
ItemDescription
ItemSKU
PK,FK1 Item NumberP K ,F K 2 S K U
SKUMaster
P K S KU
LocationMaster
P K L o ca t io n
SKULocation
P K ,F K 2 S K UPK,FK1 Location
Quantity
OrderHeader
P K O r de r N um b er
OrderDateFK1 CustomerNumber
OrderLine
PK,FK1 OrderNum berP K L in e Nu mb er
F K2 I t em N um be r
Item
Customer
InventoryLocation
StockKeeping
Unit
InventoryQuantity
InvoiceQuantity
InvoiceUnit Price
Dimensions
Facts
2007 Z Y Solutions Corporation
Metadata
Customer
PK Custom erNumber
CustomerNameCustomerAddress
InvoiceLine
PK,FK2 InvoiceNum berP K L in e Nu mb er
F K1 I t em N um be r QuantityUnitPrice
InvoiceHeader
P K I n vo i ce N u mb e r
InvoiceDateFK1 CustomerNumber
ItemMaster
P K I t em N u mb e r
ItemDescription
ItemSKU
PK,FK1 Item NumberP K ,F K 2 S K U
SKUMaster
P K S KU
LocationMaster
P K L o ca t io n
SKULocation
P K ,F K 2 S K UPK,FK1 Location
Quantity
OrderHeader
P K O r de r N um b er
OrderDateFK1 CustomerNumber
OrderLine
PK,FK1 OrderNum berP K L in e Nu mb er
F K2 I t em N um be r
Item
Customer
InventoryLocation
StockKeeping
Unit
InventoryQuantity
InvoiceQuantity
InvoiceUnit Price
InventoryQuantity
InvoiceQuantity
InvoiceUnit Price
Dimensions
Facts
2007 Z Y Solutions Corporation
8/8/2019 TomPeters_Business Intelligence Overview
11/17
Metadata
Customer
PK Custom erNumber
CustomerNameCustomerAddress
InvoiceLine
PK,FK2 InvoiceNum berP K L in e Nu mb er
F K1 I t em N um be r QuantityUnitPrice
InvoiceHeader
P K I n vo i ce N u mb e r
InvoiceDateFK1 CustomerNumber
ItemMaster
P K I t em N u mb e r
ItemDescription
ItemSKU
PK,FK1 Item NumberP K ,F K 2 S K U
SKUMaster
P K S KU
LocationMaster
P K L o ca t io n
SKULocation
P K ,F K 2 S K UPK,FK1 Location
Quantity
OrderHeader
P K O r de r N um b er
OrderDateFK1 CustomerNumber
OrderLine
PK,FK1 OrderNum ber
P K L in e Nu mb er
F K2 I t em N um be r
Item
Customer
InventoryLocation
StockKeeping
Unit
InventoryQuantity
InvoiceQuantity
InvoiceUnit Price
InventoryQuantity
InvoiceQuantity
InvoiceUnit Price
InvoiceRevenue
Dimensions
Facts
2007 Z Y Solutions Corporation
Metadata
Customer
PK Custom erNumber
CustomerNameCustomerAddress
InvoiceLine
PK,FK2 InvoiceNum berP K L in e Nu mb er
F K1 I t em N um be r QuantityUnitPrice
InvoiceHeader
P K I n vo i ce N u mb e r
InvoiceDateFK1 CustomerNumber
ItemMaster
P K I t em N u mb e r
ItemDescription
ItemSKU
PK,FK1 Item NumberP K ,F K 2 S K U
SKUMaster
P K S KU
LocationMaster
P K L o ca t io n
SKULocation
P K ,F K 2 S K UPK,FK1 Location
Quantity
OrderHeader
P K O r de r N um b er
OrderDateFK1 CustomerNumber
OrderLine
PK,FK1 OrderNum berP K L in e Nu mb er
F K2 I t em N um be r
Item
Customer
InventoryLocation
StockKeeping
Unit
InventoryQuantity
InvoiceQuantity
InvoiceUnit Price
InventoryQuantity
InvoiceQuantity
InvoiceUnit Price
InvoiceRevenue
Dimensions
Facts
Measures
2007 Z Y Solutions Corporation
Designing your Data Warehouse
What information do I want to derive
Design to business needs, NOT just a replicate of the data
source
Consider performance, de-normalize where appropriate
2007 Z Y Solutions Corporation
8/8/2019 TomPeters_Business Intelligence Overview
12/17
Data Warehouse Schema Format
Third Normal Form (3NF)
Classical RDBMSmodeling technique
Minimizes data
redundancy
through
normalization
2007 Z Y Solutions Corporation
Data Warehouse Schema Format
Star Schema
Consists of a
single Fact table
Compound
primary key
One segment for
each Dimension
2007 Z Y Solutions Corporation
Customer
P K C ustomer Number
Customer Name
Customer Address
Item
P K I te m Nu m be r
ItemDescription
Sales
P K ,FK 1 C ustomer N umber P K ,FK 2 I tem N umber P K ,FK 3 Terr i to r y N umber
P K ,FK 4 WarehouseN umber
QuantityUnit Price
Territory
P K Terr i to r yN umber
Warehouse
P K Warehouse N umber
Data Warehouse Schema Format
Snowflake Schema
Variation on the
star schema
Very large
dimension tablesare normalized
into multiple
tables
2007 Z Y Solutions Corporation
8/8/2019 TomPeters_Business Intelligence Overview
13/17
Extract, Transform and Load (ETL)
Extract
Data extraction and staging
Minimize impact on production data sources
Transform
Convert to format required by data warehouse
Cleanse data to ensure accuracy
Validate primary keys against defined owner
Convert to different numbering schema
Load
Load data to data warehouse
Follow guidelines as outlined by data warehouse
2007 Z Y Solutions Corporation
ETL Tools
Vendor tools
Informatica
DataMirror
SAS ETL Solutions
Microsoft
SQL Server 2000
Data Transformation Services (DTS)
SQL Server 2005
SQL Server Integration Services (SSIS)
2007 Z Y Solutions Corporation
BI Tool vs. Query Tool
Whats the Difference?
Why Should I Care?
2007 Z Y Solutions Corporation
8/8/2019 TomPeters_Business Intelligence Overview
14/17
B.I. Tool vs. Query Tool
Knowledge of Database Structure Not Required
Knowledge of Table Names & Relationships Not Needed
Ability to Predefine Formulas
Define once for everyone
Multi-Pass SQL
Produces Results Not Otherwise Possible
Improve performance for complex queries
2007 Z Y Solutions Corporation
B.I. Tool vs. Query Tool
Single Version of the Truth
Empower end users to create their own reports withoutfear of varying results
Control who sees what
Row level security
Object level security (including columns)
Write one report to facilitate multiple user groups
Manage and monitor usage
2007 Z Y Solutions Corporation
Deployment Objectives
Ease of administration
One metadata layer definition shared by all 5 styles of BI
One administrative interface for all 5 styles of BI
Ease of deployment Zero foot print web interface
Ease of use
User friendly / intuitive interface. Web, drag and drop,context sensitive menus, user definable help text
Business terms familiar to the organization deploying
solution
2007 Z Y Solutions Corporation
8/8/2019 TomPeters_Business Intelligence Overview
15/17
Deployment Objectives
Reduce I.T. involvement
Empower end-users to maintain existing and create newreports
Empower end users to schedule execution of reports and
select delivery method
Web
External Device
Give end-users immediate access to data
Allow end-users to mine for new data
2007 Z Y Solutions Corporation
Full Feature BI Tool Example
MicroStrategy
2007 Z Y Solutions Corporation
5 Styles of B.I. within 1 Unified Architecture
SAPBW
DataMart
DataWarehouse
OperationalDatabase
(CRM,RFID)
OperationalDatabase
(ERP)
O pe ra ti on al D a ta ba se s D ec is io n Su pp or t Da ta ba se s
Integrated Backplane
Services Oriented Architecture
WebBrowsers EnterprisePortals
MicrosoftOffice
Scorecard
s
&
Dashboards
Reporting
OLAP
Advanced
Analysis
Alerts&Proactive
Notificatio
n
Unified Web Interface
2007 Z Y Solutions Corporation
8/8/2019 TomPeters_Business Intelligence Overview
16/17
Reusable Metadata
MicroStrategy
Report-Specific
Design
ReusableComponents
REPORT DESIGN
Layout
Format
Calculations
REPORT COMPONENTS
Parameterization
Templates
Filters
Autostyles
BUSINESS ABSTRACTION
Metrics
Hierarchies
Custom Groupings
Transformations
DATA ABSTRACTION
Attributes
Facts
Tables
Aliases
Range of Metadata
Other BI
Technologies
Report-Specific
Design
Reusable
Components
2007 Z Y Solutions Corporation
Development Efficiency
2007 Z Y Solutions Corporation
MicroStrategy Architecture
ODBCODBC
Metadata Warehouse
TCP/IPTCP/IP
MicroStrategy Intelligence ServerMicroStrategy Intelligence Server
MicroStrategy OLAP ServicesMicroStrategy OLAP Services
MicroStrategy Report ServicesMicroStrategy Report Services
MicroStrategy Narrowcast ServerMicroStrategy Narrowcast Server
MicroStrategy DesktopMicroStrategy Desktop MMicroStrategyicroStrategy DesktopDesktop
MicroStrategy ArchitectMicroStrategy Architect
MicroStrategy AdministratorMicroStrategy Administrator
MicroStrategyMicroStrategy
WebWeb
BrowsersBrowsers
HTTPHTTP
Web ServicesWeb Services
MicroStrategy OfficeMicroStrategy Office
Email, Wireless,Email, Wireless,
PortalPortal
Print & FaxPrint & Fax
SMTP/SMSSMTP/SMS/Portal/Portal
2007 Z Y Solutions Corporation
8/8/2019 TomPeters_Business Intelligence Overview
17/17
MicroStrategy Architecture
MicroStrategy Architecture
WebBrowser
HTTP
Web Browser
Web
ServerMicroStrategyDesktop
TCP/IP
HTTP
WebBrowser
HTTP
MicroStrategy
Intelligence
Server
TCP/IP
ODBC
Data
Wareh
ouse
Metad
ata
ODBC
MicroStrategy Desktop
MicroStrategy Architect
MicroStrategy Administrator
MicroStrategyIntelligenceServer
MicroStrategyOLAPServices
MicroStrategyReport Services
MicroStrategyNarrowcast Server
Web
Services
WebServices
Web
Services
SMTP/
SMS/Portal
SMTP/SMS/Portal
Mobile
FaxPrinter
2007 Z Y Solutions Corporation
Interactive Demonstration
2007 Z Y Solutions Corporation
Questions & Answers
Tom Peters
tpeters@zysolutions.com
(847) 487-2300, x204
2007 Z Y Solutions Corporation