Planning & Project Planning & Project ManagementManagement

Fahri Firdausillah [M031010012]

Joke First, Serious LaterJoke First, Serious Later Consultant:Consultant: So, your company is into data So, your company is into data

warehousing? How many data marts do you have?warehousing? How many data marts do you have?

Project Manager:Project Manager: Eleven.Eleven.

Consultant:Consultant: That’s great. But why so many?That’s great. But why so many?

Project Manager:Project Manager: Ten mistakes.Ten mistakes.

Defining the Business Defining the Business RequirementsRequirements

Chapter 5Chapter 5

PreamblePreamble OLTP and DW planning is different in term of OLTP and DW planning is different in term of

requirements clarityrequirements clarity

Planning DW is about solving users’ problems and Planning DW is about solving users’ problems and providing strategic information to the user.providing strategic information to the user.

OLTP systems are primarily data capture systems. On OLTP systems are primarily data capture systems. On the other hand, data warehouse systems are the other hand, data warehouse systems are information delivery systems.information delivery systems.

Unlike an OLTP system, which is needed to run the day-Unlike an OLTP system, which is needed to run the day-to-day business, no immediate payout is seen in a to-day business, no immediate payout is seen in a decision support system.decision support system.

Dimensional AnalysisDimensional Analysis

the users are generally unable to define the users are generally unable to define their requirements clearly.their requirements clearly.

For most of the users, this could be the For most of the users, this could be the very first data warehouse.very first data warehouse.

How can you build something the users are How can you build something the users are unable to define clearly and precisely?unable to define clearly and precisely?

Need different approach of requirements Need different approach of requirements gathering.gathering.

Dimensional Analysis (cont'd)Dimensional Analysis (cont'd) They can tell you what measurement units are They can tell you what measurement units are

important for them, how they combine the important for them, how they combine the various pieces of information for strategic various pieces of information for strategic decision making.decision making.

Although the actual proposed usage of a data Although the actual proposed usage of a data warehouse could be unclear, the business warehouse could be unclear, the business dimensions used by the managers for decision dimensions used by the managers for decision making are not nebulous at allmaking are not nebulous at all

Dimensional Analysis “in Action”Dimensional Analysis “in Action”

More Complex Dimensional ModelMore Complex Dimensional Model

Information PackagesInformation Packages The business dimensions and their hierarchical levels form The business dimensions and their hierarchical levels form

the basis for all further development phases.the basis for all further development phases.

The dimension hierarchies are the paths for drilling down or The dimension hierarchies are the paths for drilling down or rolling up in our analysisrolling up in our analysis

Requirements Gathering MethodsRequirements Gathering Methods

Questionn

aires Group Session

InterviewTypes of Questions Open Ended QuestionThese open up options for interviewees to respond

Closed QuestionThese allow limited responses to interviewees

ProbesThese are really follow-up questions. Probes may be used after open-ended or closed questions

Sample Expectation from Sample Expectation from InterviewsInterviews

Senior Executives Dept. Managers IT Dept. Professionals

Organization objectives

Criteria for measuring success

Key business issues, current

& future

Problem identification

Vision and direction for the

organization

Anticipated usage of the DW

Departmental objectives

Success metrics

Factors limiting success

Key business issues

Products & Services

Useful business dimensions

for analysis

Anticipated usage of the DW

Key operational source

systems

Current information delivery

processes

Types of routine analysis

Known quality issues

Current IT support for

information requests

Concerns about proposed DW

Adapting JADAdapting JAD

1. P

roje

ct

Defi

nitio

n 2. Research3. Pre

para

tion

4. JAD Sessions

5. F

inal

Docu

ment

1.1. Identify project objectives and Identify project objectives and limitationslimitations

2.2. Identify critical success factorsIdentify critical success factors

3.3. Define project deliverables Define project deliverables

4.4. Define the schedule of workshop Define the schedule of workshop activitiesactivities

5.5. Select the participantsSelect the participants

6.6. Prepare the workshop materialPrepare the workshop material

7.7. Organize workshop activities and Organize workshop activities and exercisesexercises

8.8. Prepare, inform, educate the Prepare, inform, educate the workshop participantsworkshop participants

9.9. Coordinate workshop logisticsCoordinate workshop logistics

Requirement Definition: Scope & Requirement Definition: Scope & ContentContent

Requirements definition document is the basis for the Requirements definition document is the basis for the next phases. Formal documentation will also validate next phases. Formal documentation will also validate your findings when reviewed with the usersyour findings when reviewed with the users Data SourcesData Sources

Data TransformationData Transformation

Data StorageData Storage

Information DeliveryInformation Delivery

Information Package DiagramsInformation Package Diagrams

Requirements Definition Document Requirements Definition Document OutlineOutline

1.1.IntroductionIntroduction

2.2.General Requirements DescriptionsGeneral Requirements Descriptions

3.3.Specific RequirementsSpecific Requirements

4.4.Information PackagesInformation Packages

5.5.Other RequirementsOther Requirements

6.6.User ExpectationsUser Expectations

7.7.User Participation and Sign-OffUser Participation and Sign-Off

8.8.General Implementation PlanGeneral Implementation Plan

Requirements as the Requirements as the Driving Force for Data Driving Force for Data

WarehousingWarehousing

Chapter 6Chapter 6

PreamblePreamble If accurate requirements definition is important for any If accurate requirements definition is important for any

operational system, it is many times more important for operational system, it is many times more important for a data warehousea data warehouse

extremely important that your datawarehouse contains extremely important that your datawarehouse contains the right elements of information in the most optimal the right elements of information in the most optimal formatsformats

Every task that is performed in every phase in the Every task that is performed in every phase in the development of the data warehouse is determined by development of the data warehouse is determined by the requirementsthe requirements

Every decision made during the design phase is totally Every decision made during the design phase is totally influenced by the requirements.influenced by the requirements.

Data DesignData Design

Data Design (cont'd)Data Design (cont'd) Structure for Business DimensionsStructure for Business Dimensions

Importance of having the appropriate dimensions and the right Importance of having the appropriate dimensions and the right contents in the contents in the information package diagramsinformation package diagrams..

Structure for Key MeasurementsStructure for Key Measurements

Users measure performance by using and comparing key Users measure performance by using and comparing key measurementsmeasurements

In order to review using proper key measurements, DW has to In order to review using proper key measurements, DW has to guarantee the information package diagrams contain all the relevant guarantee the information package diagrams contain all the relevant keys.keys.

Levels of DetailLevels of Detail

DW needs to provide drill-down and roll-up facilities for analysisDW needs to provide drill-down and roll-up facilities for analysis How deep detail of data is needed in DWHow deep detail of data is needed in DW

Data Design “in Action”Data Design “in Action”

Structure for Business Dimensions

Structure for Key Measurements

Levels of Detail

The Architectural PlanThe Architectural Plan

Source DataSource Data Production Data: Data get from operational system. Production Data: Data get from operational system.

Normally include financial system, customer Normally include financial system, customer relationship system, manufacturing system, etc.relationship system, manufacturing system, etc.

Internal Data: Private data keep by internal Internal Data: Private data keep by internal organization. Could be spreadsheets, documents, even organization. Could be spreadsheets, documents, even departmental databasedepartmental database

Archived Data: Old data that is already not to be used Archived Data: Old data that is already not to be used in operational system.in operational system.

External Data: Data from outside systems, it can also External Data: Data from outside systems, it can also from outside company. This type of data usually do not from outside company. This type of data usually do not conform internal formatconform internal format

Data StagingData StagingBad data lead to bad decision, Bad data lead to bad decision,

data quality in data warehouse is sacrosanctdata quality in data warehouse is sacrosanct ETL process ensure data to be ready stored and processed ETL process ensure data to be ready stored and processed

in DW.in DW.

In many cases data need to be extracted from sources in In many cases data need to be extracted from sources in different scheme, different vendor, even in different format different scheme, different vendor, even in different format of flat files.of flat files.

If data extraction for a DW poses great challenges, data If data extraction for a DW poses great challenges, data transformation presents even greater challenges.transformation presents even greater challenges.

Data need to be cleaned from misspelling, resolution Data need to be cleaned from misspelling, resolution conflict, duplication, setting default missing values, etc.conflict, duplication, setting default missing values, etc.

Initial load moves very large volumes of data. After that data Initial load moves very large volumes of data. After that data staging will continuously extract the changes from sources.staging will continuously extract the changes from sources.

Extract Transform Load

Sample ArchitectureSample Architecture

Data Storage SpecificationsData Storage Specifications DBMS SelectionDBMS Selection

User requirements affect the selection of the proper DBMS.User requirements affect the selection of the proper DBMS.

Choice of the DBMS may be conditioned by its tool kit Choice of the DBMS may be conditioned by its tool kit component.component.

Features to be considered: Level of User Experience, Types of Features to be considered: Level of User Experience, Types of Queries, Need for Openness, Data Loads, Metadata Management, Queries, Need for Openness, Data Loads, Metadata Management, Data Repository Locations, Data Warehouse Growth.Data Repository Locations, Data Warehouse Growth.

Storage SizingStorage Sizing Determined by how many data source and how much the Determined by how many data source and how much the

data will grows continuously.data will grows continuously. If DW is expected to support Online Analytical Processing If DW is expected to support Online Analytical Processing

OLAP, then how much OLAP is necessary.OLAP, then how much OLAP is necessary.

Information Delivery StrategyInformation Delivery Strategy

MetadataMetadata Operational Metadata:Operational Metadata:

When deliver information to the end-users, you must be able When deliver information to the end-users, you must be able to tie that back to the original source data sets. Operational to tie that back to the original source data sets. Operational metadata contain all of this information about theoperational metadata contain all of this information about theoperational data sources.data sources.

Extraction and Transformation Metadata:Extraction and Transformation Metadata:Storing information of extraction frequencies, extraction Storing information of extraction frequencies, extraction methods, and business rules for the data extraction.methods, and business rules for the data extraction.

End-User Metadata:End-User Metadata:Navigational map of the data warehouse, allows the end-Navigational map of the data warehouse, allows the end-users to use their own business terminology and look for users to use their own business terminology and look for information in those ways.information in those ways.

Management & ControlManagement & Control

Sits on top of all the other components.Sits on top of all the other components. Controls the data transformation and the Controls the data transformation and the

data transfer into the data warehouse data transfer into the data warehouse storage.storage.

Interacts with the metadata component to Interacts with the metadata component to perform the management and control perform the management and control functions.functions.

Metadata is the source of information for Metadata is the source of information for the management module.the management module.

End of PresentationEnd of Presentation&&

Thank You Very MuchThank You Very Much