Defining Data Warehouse Concepts and Terminology

Chapter 3

Definition of a Data Warehouse

“ An enterprise structured repository of subject-oriented, time-variant, historical data used for information retrieval and decision support. The data warehouse stores atomic and summary data.”

Oracle Data Warehouse Method

Data Warehouse Properties

DataWarehouse

Integrated

Time VariantNon Volatile

SubjectOriented

Subject-OrientedData is categorized and stored by business subjectrather than by application

EquityPlans Shares Customer

financialinformation

Savings

Insurance

Loans

OLTP Applications Data Warehouse Subject

Integrated

OLTP Applications

Savings

Currentaccounts

Loans

Data Warehouse

Data on a given subject is defined and stored once.

Customer

Time-VariantData is stored as a series of snapshots, each representing a period of time

Time DataJan-97 JanuaryFeb-97 FebruaryMar-97 March

NonvolatileTypically data in the data warehouse is not updated or delelted.

Insert UpdateDelete

Read Read

Operational Warehouse

Load

Changing Data

Warehouse Database

First time load

Refresh

Refresh

Refresh

Operational Database

Data Warehouse Versus OLTP

PropertyResponseTimeOperationsNature of Data

Data OrganizationSize

Data Source

Activities

OperationalSub seconds to secondsDML30-60 days

ApplicationsSmall to large

Operational, Internal

Processes

Data Warehouse

Seconds to hours

Snapshots over timeSubject, time

Large to very largeOperational, Internal,External

Analysis

Primarily read only

Usage CurvesOperational system is predictableData warehouse - Variable - Random

User ExpectationsControl expectationsSet achievable targets for query

responseSet SLAsEducateGrowth and use is exponential

Enterprisewide WarehouseLarge scale implementationScope the entire businessData from all subject areasDeveloped incrementallySingle source of enterprisewide dataSingle distribution point to

dependent data marts

Data Warehouses Versus Data Marts

Property Data Warehouse Data MartScope Enterprise DepartmentSubject Multiple Single-subject, LOBData Source Many FewSize(typical) 100 GB to>1 TB <100 GBImplementation time Months to years Months

DataWarehouse

DataMart

Dependent Data Mart

MarketingSales

FinanceHuman Resources

Marketing

Marketing

Marketing

External Data

DataWarehouse

OperationalSystems

Flat Files

Data Marts

Independent Data Mart

OperationalSystems

External Data

Sale or Marketing

Flat Files

Data Warehouse TerminologyOperational data store (ODS) Stores tactical data from production

systems that are subject-oriented and integrated to address operational needs

MetadataMetadata

Data Warehouse Terminology

DataIntegration

Enterprise data warehouse

Business areawarehouse

Source data

Architecture

MethodolgyEnsures a successful data warehouseEncourages incremental developmentProvides a staged approach to an

enterprisewide warehouse - Safe - Manageable - Proven - Recommended

ModelingWarehouses differ from operational structures: - Analytical requirements - Subject orientationData must map to subject oriented information: - Identify business subjects - Define relationships between subjects - Name the attributes of each subjectModeling is iterativeModeling tools are available

Extraction, Transformation, and Transportation

Purchase specialist tools, or develop programsExtraction-- select data using different methodsTransformation--validate, clean, integrate, and

time stamp dataTransportation--move data into the warehouse

OLTP Databases Staging File Warehouse Database

Data ManagementEfficient database server and management

tools for all aspects of data managementImperatives - Productive - Flexible - Robust - EfficientHardware, operating system and network

management

Data Access and Reporting

Tools that retrieve data for business analysis Imperatives - Ease of use - Intuitive - Metadata - Training More than one tool may be required

WarehouseDatabase

Simple Queries

Forecasting

Drill-down

Oracle Warehouse Components

Relational / Multidimensional

Text, image Spatial

Web Audio video

Externaldata

Operational data

Relational tools

OLAP tools

Applications/Web

Any DataAny Source Any Access

Oracle Data Mart Suite

Data ModelingOracle Data Mart Designer

OLTPEngines

OLTPDatabases

DataExtraction

Oracle Data MartBuilder

Ware-housingEngines

Data MartDatabase

SQL*Plus

DataManagement

Oracle EnterpriseManager

Data Access& Analysis

Discoverer &Oracle Reports

Data Mart Implementation with the Oracle Data Mart Suite

Oracle Enterprise ServerOracle Enterprise ManagerOracle Data Mart BuilderOracle Data Mart DesignerOracle DiscovererOracle Web Application ServerOracle Reports

Oracle Warehouse Builder Architecture

Sources

ExtractionFacilities• Loader• Remotes SQL• Gateways - OLE-DB/ODBC - Mainframe - Specialized• ERP Data - SAP - Peoplesoft - Oracle

PL/SQL, JavaTransforms

TransformDriver

PL/SQL, JavaWrapper

ExternalFunctions

TargetTablesFilter

Transform

Oracle 8i

Oracle Business Intelligence Tools

Current Tactical Strategic

IS developsuser’s Views Business users Analysis

Oracle Reports Oracle Discover Oracle Express

The Tool for Each TaskTool

OracleReports

OracleDiscover

OracleExpress

Productionreporting

Ad hocquery and analysis

Advancedanalysis

Question

What were sales byregion last quarter?

What is driving the increase in NorthAmerican sales?

Given the rapid increasein Web sales, what willtotal sales be for the restof the year?

Task

Oracle Warehouse Services

OracleEducation

OracleConsulting

Oracle Support Services

Customers

SummaryThis lesson covered the following topics:Identifying a common, broadly accepted definition

of the data warehouseDistinguishing the differences between OLTP

systems and analytical systemsDefining some of the common data warehouse

terminologyIdentifying some of the elements and processes in a

data warehouseIdentifying and positioning the Oracle Warehouse

vision, products, and services