Impossible Data Impossible Data Warehouse SituationsWarehouse Situations

Sid AdelmanSid Adelmansid@sidadelman.com

(818) 783 9634(818) 783 9634

IntroductionIntroduction

Data Warehouse problems with no solutions?Data Warehouse problems with no solutions? Taken from real situationsTaken from real situations

DMReview Ask the ExpertsDMReview Ask the Experts Data Warehouse Project Management ClassesData Warehouse Project Management Classes

No names givenNo names given Problems are rarely uniqueProblems are rarely unique Problems are rarely industry specificProblems are rarely industry specific You will experience many of themYou will experience many of them There is hope, there are solutionsThere is hope, there are solutions

22

Your RoleYour Role

You can identify the problem before You can identify the problem before it hurts the projectit hurts the project

You can weigh the optionsYou can weigh the options You can take action to keep the You can take action to keep the

impossible situation from causing a impossible situation from causing a disasterdisaster

33

Management SituationsManagement Situations

Management IssuesManagement Issues Changing Requirements & ObjectivesChanging Requirements & Objectives Justification & BudgetJustification & Budget Organization & StaffingOrganization & Staffing User IssuesUser Issues Team IssuesTeam Issues Project Planning & SchedulingProject Planning & Scheduling

44

Technical SituationsTechnical Situations

Data Warehouse StandardsData Warehouse Standards Data Warehouse ToolsData Warehouse Tools Data QualityData Quality IntegrationIntegration PerformancePerformance

55

Case 1.1 DW has a record of Case 1.1 DW has a record of failure failure

This is the third attempt at a data This is the third attempt at a data warehouse. The first two failed, and warehouse. The first two failed, and the general feeling is that this one the general feeling is that this one will also fail. What can the project will also fail. What can the project manager do to dispel the negative manager do to dispel the negative conventional wisdom about the data conventional wisdom about the data warehouse?warehouse?

66

Solution 1.1 – DW has a record Solution 1.1 – DW has a record of failureof failure

Read post-implementation reviews – was Read post-implementation reviews – was there a post-implementation review?there a post-implementation review?

Understand the causes of failureUnderstand the causes of failure Address the failures and show how they will Address the failures and show how they will

not cause a failure this timenot cause a failure this time Quick deliveryQuick delivery Deliver in phasesDeliver in phases Show valueShow value Communicate often with managementCommunicate often with management Strong sponsorStrong sponsor

77

Case 1.2 IT is UnresponsiveCase 1.2 IT is Unresponsive

The IT people on the operational/transaction side The IT people on the operational/transaction side are not cooperating with those on the data are not cooperating with those on the data warehouse side. The IT people on the operational warehouse side. The IT people on the operational side were unresponsive to requests for resources side were unresponsive to requests for resources and information about the operational systems, and information about the operational systems, and now they’re not responding to requests for and now they’re not responding to requests for source data that will go into the source data that will go into the extract/transform/load process. What should the extract/transform/load process. What should the project manager do?project manager do?

88

Solution 1.2 IT is Solution 1.2 IT is UnresponsiveUnresponsive

CIO must be convinced of the value of DWCIO must be convinced of the value of DW Form a DW advisory committee with CIO Form a DW advisory committee with CIO

as its chairmanas its chairman Get the business on your side and have a Get the business on your side and have a

high-level business manager take the high-level business manager take the problem to the CIOproblem to the CIO

Have lunch with the IT managers who are Have lunch with the IT managers who are not cooperating (one at a time) and not cooperating (one at a time) and determine why and try to find something determine why and try to find something in the DW that will benefit them.in the DW that will benefit them.

99

Case 1.3 Management Case 1.3 Management Constantly ChangesConstantly Changes

A very dynamic organization is A very dynamic organization is constantly changing management. It constantly changing management. It is unlikely that the corporate sponsor is unlikely that the corporate sponsor will remain for the duration of the will remain for the duration of the project. What should the project project. What should the project manager do?manager do?

1010

Solution 1.3 Management Solution 1.3 Management Constantly ChangingConstantly Changing

Get backup business sponsorsGet backup business sponsors Prepare a project agreement and Prepare a project agreement and

communicate the agreements to communicate the agreements to multiple managers.multiple managers.

Deliver incremental solutionsDeliver incremental solutions Measure resultsMeasure results Communicate regularly with many in Communicate regularly with many in

the business community.the business community.1111

Case 1.4 The Pilot Must be Case 1.4 The Pilot Must be PerfectPerfect

Management thinks the data warehouse Management thinks the data warehouse implementation must be perfect; as a implementation must be perfect; as a result, the scheduled implementation date result, the scheduled implementation date has already passed. The managers expect has already passed. The managers expect the quality assurance (QA) on the pilot to the quality assurance (QA) on the pilot to be at the same level as that of a production be at the same level as that of a production system. What can the project management system. What can the project management team do to convince management that a team do to convince management that a pilot system and a production system pilot system and a production system should have different levels of QA?should have different levels of QA?

1212

Solution 1.4 The Pilot Must be Solution 1.4 The Pilot Must be PerfectPerfect

Set expectations about the role of a Set expectations about the role of a pilot and how it differs from a pilot and how it differs from a production application.production application.

Promote the role of a pilot to uncover Promote the role of a pilot to uncover deficiencies. deficiencies.

Uncover concerns that the pilot will Uncover concerns that the pilot will be the final productbe the final product

1313

Case 1.5 Users Don’t Want to Case 1.5 Users Don’t Want to Share DataShare Data

The feeling seems to be—“By giving another The feeling seems to be—“By giving another department access to our data, we will be giving department access to our data, we will be giving them the ability to criticize us or even take over them the ability to criticize us or even take over our jobs.” Department heads sometimes pretend our jobs.” Department heads sometimes pretend they support the idea of sharing data, but in the they support the idea of sharing data, but in the case of the data in their department, they case of the data in their department, they maintain that no other group has the expertise to maintain that no other group has the expertise to accurately analyze their data. They want to be in accurately analyze their data. They want to be in control of who sees what data and when. How can control of who sees what data and when. How can the data warehouse team get department heads the data warehouse team get department heads to share access to their data?to share access to their data?

1414

Solution 1.5 User Department Solution 1.5 User Department Doesn’t Want to Share DataDoesn’t Want to Share Data

Satisfy the users that they will still Satisfy the users that they will still have a job.have a job.

Give the user department first rights Give the user department first rights to see the information.to see the information.

Attempt to get a corporate policy for Attempt to get a corporate policy for data sharing – who owns the data?data sharing – who owns the data?

Sell the idea that data is a corporate Sell the idea that data is a corporate asset along with other assets.asset along with other assets.

1515

Case 1.6 Senior management Case 1.6 Senior management does not know what the DW does not know what the DW

Team doesTeam does A newly built data warehouse meets A newly built data warehouse meets

almost all the measures of success. almost all the measures of success. The response from upper The response from upper management has not been management has not been supportive. It seems management supportive. It seems management does not recognize the success of does not recognize the success of the project. What should the data the project. What should the data warehouse team do?warehouse team do?

1616

Solution 1.6 Senior Management Solution 1.6 Senior Management Doesn’t Know What the DW Team Doesn’t Know What the DW Team

DoesDoes Cost/justify the DWCost/justify the DW Identify business benefits, tangible and Identify business benefits, tangible and

intangible intangible Calculate ROICalculate ROI Measure user satisfactionMeasure user satisfaction Ask business manager for a testimonialAsk business manager for a testimonial Give senior management some new Give senior management some new

information capability information capability

1717

2. Changing Requirements & 2. Changing Requirements & ObjectivesObjectives

““If one does not know to which port If one does not know to which port one is sailing, no wind is favorable.”one is sailing, no wind is favorable.”

SenecaSeneca

1818

Case 2.1 Operational System is Case 2.1 Operational System is ChangingChanging

The data warehouse project has been The data warehouse project has been in development for six months. Right in development for six months. Right in the middle of the project the team in the middle of the project the team discovers that the operational source discovers that the operational source systems are being rewritten, with the systems are being rewritten, with the new systems expected to be up and new systems expected to be up and running in eight months. What should running in eight months. What should the data warehouse team do?the data warehouse team do?

1919

Solution 2.1 Operational Solution 2.1 Operational System is ChangingSystem is Changing

Focus on what might not change Focus on what might not change Wait on ETL processesWait on ETL processes Use the logical model for impact analysis in Use the logical model for impact analysis in

the changing operational systemthe changing operational system Meet frequently with the operational teamMeet frequently with the operational team Explain the problem to managementExplain the problem to management Renegotiate the schedule and budgetRenegotiate the schedule and budget Keep the DW team together and motivatedKeep the DW team together and motivated Deliver a proof of concept (POC)Deliver a proof of concept (POC)

2020

Case 2.2 Source Systems Case 2.2 Source Systems Constantly ChangeConstantly Change

The source system is being built at the The source system is being built at the same time as the data warehouse. same time as the data warehouse. Although the design of the source system Although the design of the source system is supposed to be “frozen,” it is constantly is supposed to be “frozen,” it is constantly changing. The source system development changing. The source system development team doesn’t communicate the changes—team doesn’t communicate the changes—the data warehouse team discovers the data warehouse team discovers changes during testing when the system changes during testing when the system fails. What should the data warehouse fails. What should the data warehouse team do?team do?

2121

Solution 2.2 Source Systems Solution 2.2 Source Systems Constantly ChangeConstantly Change

Document the problems associated with Document the problems associated with the lack of communicationthe lack of communication

Make management, the business sponsor Make management, the business sponsor and IT, aware of the problem and tell and IT, aware of the problem and tell them how the problem should be fixedthem how the problem should be fixed

Develop relationship with those Develop relationship with those responsible for the source systemsresponsible for the source systems

Provide incentives for better Provide incentives for better communicationcommunication

2222

Case 2.3 DW Vision is Case 2.3 DW Vision is BlurredBlurred

When the data warehouse project began, the When the data warehouse project began, the organization searched out the best practices organization searched out the best practices in the industry and developed a strong set of in the industry and developed a strong set of data warehouse objectives. However, over data warehouse objectives. However, over the months and years, decisions were made the months and years, decisions were made and actions were taken that went counter to and actions were taken that went counter to the initial objectives. Most of these decisions the initial objectives. Most of these decisions and actions were considered practical and and actions were considered practical and sometimes recognized as temporary, with sometimes recognized as temporary, with the goal to eventually return to the initial the goal to eventually return to the initial objectives. What should this organization do?objectives. What should this organization do?

2323

Solution 2.3 DW Vision is Solution 2.3 DW Vision is BlurredBlurred

Revisit the original objectives and best Revisit the original objectives and best practices, validate and attempt to reinstate practices, validate and attempt to reinstate themthem

Document how the standards and best Document how the standards and best practices were overridden and whypractices were overridden and why

Recruit a high-level IT person (perhaps the Recruit a high-level IT person (perhaps the CTO) to sponsor the best practices.CTO) to sponsor the best practices.

Establish an advisory group to oversee DW Establish an advisory group to oversee DW practices and procedurespractices and procedures

Review the results of the DW implementations Review the results of the DW implementations and identify how the lack of best practices hurt and identify how the lack of best practices hurt

Develop a plan to replace “temporary” fixesDevelop a plan to replace “temporary” fixes

2424

Case 2.4 Objectives are Case 2.4 Objectives are MisunderstoodMisunderstood

The objectives for the data The objectives for the data warehouse were never properly warehouse were never properly identified. The project is well under identified. The project is well under way but there is no method for way but there is no method for judging whether the project will be judging whether the project will be successful. What should the successful. What should the organization do?organization do?

2525

Solution 2.4 Objectives are Solution 2.4 Objectives are MisunderstoodMisunderstood

Steering committee should establish realistic Steering committee should establish realistic and measurable objectives that support the and measurable objectives that support the objectives of the organizationobjectives of the organization

Document the objectives and provide these to Document the objectives and provide these to all the DW Team members and to stakeholdersall the DW Team members and to stakeholders

Steering committee should establish measures Steering committee should establish measures of successof success

As applications are implemented, determine if As applications are implemented, determine if objectives were met and whyobjectives were met and why

2626

Case 2.5 Prototype Becomes Case 2.5 Prototype Becomes ProductionProduction

A company developed a prototype so that the A company developed a prototype so that the users and management could get a sense of users and management could get a sense of what they would be getting. The prototype was what they would be getting. The prototype was never intended to become a production system, never intended to become a production system, so the team gave little effort to cleaning the so the team gave little effort to cleaning the data, producing a workable database design, data, producing a workable database design, testing the prototype, or performing all that is testing the prototype, or performing all that is necessary to deliver a high-quality product. necessary to deliver a high-quality product. Management said, “Great, let’s give it to all the Management said, “Great, let’s give it to all the users now.” How should the developers users now.” How should the developers convince management that a prototype is not convince management that a prototype is not ready for production?ready for production?

2727

Solution 2.5 Prototype Solution 2.5 Prototype Becomes ProductionBecomes Production

Educate the business on the role and limitations Educate the business on the role and limitations of a prototype before the prototype is builtof a prototype before the prototype is built Data quality problemsData quality problems Doesn’t support user requirementsDoesn’t support user requirements Poor performancePoor performance Incomplete dataIncomplete data Lack of documentationLack of documentation Incomplete testingIncomplete testing Poor database designsPoor database designs Lack of scalabilityLack of scalability Inability to support service level agreements (SLAs)Inability to support service level agreements (SLAs)

Manage expectationsManage expectations

2828

Case 2.6 Management Does Case 2.6 Management Does Not Recognize the Success of Not Recognize the Success of

the DW Projectthe DW Project A data warehouse manager has now A data warehouse manager has now

implemented three data marts but implemented three data marts but has been unable to convince has been unable to convince management of the success of these management of the success of these efforts. What should the data efforts. What should the data warehouse manager do to show how warehouse manager do to show how successful these data marts are?successful these data marts are?

2929

Solution 2.6 Management Does Solution 2.6 Management Does Not Recognize Success of the DW Not Recognize Success of the DW

ProjectProject Be sure that the DW is for the business, not Be sure that the DW is for the business, not

just ITjust IT Determine how managers are measuredDetermine how managers are measured Provide metrics from the DW based on Provide metrics from the DW based on

manager measurementsmanager measurements Provide monthly metrics on usage and valueProvide monthly metrics on usage and value Compare results to baselines of what it was Compare results to baselines of what it was

like before the DWlike before the DW Measure tangible and intangible benefitsMeasure tangible and intangible benefits Ask for testimonials from the business sponsorAsk for testimonials from the business sponsor

3030

3. Justification and Budget3. Justification and Budget

““Sometimes the accounting people act Sometimes the accounting people act as if they think the organization as if they think the organization exists so they can keep books on it.”exists so they can keep books on it.”

Karl Albrecht and Ron Karl Albrecht and Ron ZemkeZemke

3131

Case 3.1 User Productivity Case 3.1 User Productivity Justification not AllowedJustification not Allowed

An organization will only fund and support a An organization will only fund and support a data warehouse if it can be justified by data warehouse if it can be justified by increases in revenue or decreases in cost. increases in revenue or decreases in cost. Management will not sign off on productivity Management will not sign off on productivity improvement cost savings since the improvement cost savings since the company has a “no layoff policy” and any company has a “no layoff policy” and any personnel will have to be retained even with personnel will have to be retained even with no work to do. How do you justify a project no work to do. How do you justify a project along with the tools and other costs along with the tools and other costs associated with it?associated with it?

3232

Solution 3.1 User Productivity Solution 3.1 User Productivity Justification not AllowedJustification not Allowed

Consider benefits of not having to Consider benefits of not having to hire more staff when people quithire more staff when people quit

Measure user benefits other than Measure user benefits other than productivity (speed, quality, productivity (speed, quality, understandability of results)understandability of results)

Find other tangible benefitsFind other tangible benefits

3333

Case 3.2 Identify Infrastructure Case 3.2 Identify Infrastructure BenefitsBenefits

A committee was formed to make A committee was formed to make recommendations for the infrastructure recommendations for the infrastructure that will be needed for the data that will be needed for the data warehouse. They were asked to justify warehouse. They were asked to justify any expenditures for data warehouse any expenditures for data warehouse tools (ETL, BI, etc.). All the benefits tools (ETL, BI, etc.). All the benefits they were able to identify have been they were able to identify have been intangible. What should they do? intangible. What should they do?

3434

Solution 3.2 Identify Solution 3.2 Identify Infrastructure BenefitsInfrastructure Benefits

Without the infrastructure, there is Without the infrastructure, there is no DWno DW

Start small, show benefits, growStart small, show benefits, grow Work with existing productsWork with existing products Evaluate tool buy versus build costEvaluate tool buy versus build cost

3535

Case 3.4 How Can Costs be Case 3.4 How Can Costs be Fairly Allocated?Fairly Allocated?

The sponsor of the first data warehouse The sponsor of the first data warehouse project absorbed the entire cost of the project absorbed the entire cost of the data warehouse infrastructure. Now data warehouse infrastructure. Now there are additional divisions in the there are additional divisions in the company that will be using the data company that will be using the data warehouse – these divisions do not warehouse – these divisions do not want to pay for any part of the want to pay for any part of the infrastructure. How should the costs be infrastructure. How should the costs be allocated and what should the sponsor allocated and what should the sponsor do?do?

3636

Solution 3.4 How Can Costs be Solution 3.4 How Can Costs be Fairly Allocated?Fairly Allocated?

Do you have charge-backs? Charge-backs Do you have charge-backs? Charge-backs can reduce usagecan reduce usage

Separate budget for infrastructure – Separate budget for infrastructure – allocate across enterpriseallocate across enterprise

Data mart allocated to owning departmentData mart allocated to owning department Volume of dataVolume of data Number of usersNumber of users Activity of usersActivity of users

3737

Case 3.5 Historical Data Must Case 3.5 Historical Data Must be Justifiedbe Justified

Users are asking for ten years of Users are asking for ten years of historical data. Not only will this historical data. Not only will this require much more hardware, but we require much more hardware, but we now have to reconcile all code now have to reconcile all code changes for the last ten years. How changes for the last ten years. How can all this historical data be can all this historical data be justified?justified?

3838

Solution 3.5 Historical Data Solution 3.5 Historical Data Must be JustifiedMust be Justified

Historical data often needs work to clean, Historical data often needs work to clean, integrate, and reconcile with current dataintegrate, and reconcile with current data

Is detail data needed?Is detail data needed? What is the cost justification of historical data?What is the cost justification of historical data? Ask the people who want the historical data to Ask the people who want the historical data to

pay for it. They may choose not topay for it. They may choose not to Monitor historical usage and provide feedbackMonitor historical usage and provide feedback Historical data may be mandated by law or it Historical data may be mandated by law or it

may be a regulatory requirementmay be a regulatory requirement Consider lower cost storage for older dataConsider lower cost storage for older data

3939

4. Organization and Staffing4. Organization and Staffing

““If you’re riding ahead of the herd, If you’re riding ahead of the herd, take a look every now and then to take a look every now and then to make sure it’s still there.”make sure it’s still there.”

Will RogersWill Rogers

4040

Case 4.1 To Whom Should the Case 4.1 To Whom Should the DW Team Report?DW Team Report?

A data warehouse manager has responsibility for A data warehouse manager has responsibility for the three data warehouse projects that are under the three data warehouse projects that are under development and the two that have already been development and the two that have already been rolled out to the users. She is fighting to keep the rolled out to the users. She is fighting to keep the function as a separate entity reporting to the CIO. function as a separate entity reporting to the CIO. However, a powerful application development However, a powerful application development manager feels that the data warehouse should manager feels that the data warehouse should report to him and he is making a case for him to report to him and he is making a case for him to take control of the DW. Where do you feel the take control of the DW. Where do you feel the data warehouse manager should report and what data warehouse manager should report and what recommendations would you give the data recommendations would you give the data warehouse manager?warehouse manager?

4141

Solution 4.1 To Whom Should Solution 4.1 To Whom Should the DW Team Report?the DW Team Report?

As high as possible – in IT, the CIOAs high as possible – in IT, the CIO Depends on CIO perspective but should Depends on CIO perspective but should

not report to Application Developmentnot report to Application Development Should the Team report to the Business Should the Team report to the Business

and where in the Business hierarchy?and where in the Business hierarchy? Chief Operating Officer (COO)Chief Operating Officer (COO) Chief Financial Officer (CFO)Chief Financial Officer (CFO) Chief Knowledge Officer (CKO)Chief Knowledge Officer (CKO)

4242

Case 4.2 Organization Uses Case 4.2 Organization Uses Matrix ManagementMatrix Management

A company has created multiple core competency A company has created multiple core competency groups and loans out these skills as needed. One groups and loans out these skills as needed. One such group is the database administration group. The such group is the database administration group. The data warehouse project manager is not allowed to data warehouse project manager is not allowed to hire a DBA for the project but must rely on the hire a DBA for the project but must rely on the goodwill of the DBA Manager to provide a skilled DBA goodwill of the DBA Manager to provide a skilled DBA when needed. This has not worked in the past as when needed. This has not worked in the past as multiple DBAs were assigned during the course of multiple DBAs were assigned during the course of development and the project lost continuity. In development and the project lost continuity. In addition, the DBAs were not always available when addition, the DBAs were not always available when they were needed. The data warehouse project they were needed. The data warehouse project manager is about to begin another project. What manager is about to begin another project. What should she do about this matrix organization?should she do about this matrix organization?

4343

Solution 4.2 Organization Uses Solution 4.2 Organization Uses Matrix ManagementMatrix Management

Core team is required with full time Core team is required with full time commitmentscommitments

Matrix management does not work Matrix management does not work for the DWfor the DW Extended schedulesExtended schedules Productivity lossProductivity loss Loss of continuityLoss of continuity Learning curve problemsLearning curve problems

4444

Case 4.3 Project Has no Case 4.3 Project Has no Consistent Business SponsorConsistent Business Sponsor

The organization has a policy of constantly The organization has a policy of constantly rotating managers. The business sponsor rotating managers. The business sponsor who started with the project has been who started with the project has been reassigned and now has no interest in the reassigned and now has no interest in the project. The new business sponsor is not project. The new business sponsor is not familiar with the data warehouse or the familiar with the data warehouse or the project. She also has different views about project. She also has different views about what is important. This will probably lead to what is important. This will probably lead to changes in the scope of the project. What changes in the scope of the project. What should the project manager do?should the project manager do?

4545

Solution 4.3 Project has no Solution 4.3 Project has no Consistent Business SponsorConsistent Business Sponsor

Project agreement is criticalProject agreement is critical Project justification is criticalProject justification is critical Steering committee should provide Steering committee should provide

some consistencysome consistency Sell the project throughout the Sell the project throughout the

sponsoring divisionsponsoring division Deliver in smaller phases and measure Deliver in smaller phases and measure

the benefitsthe benefits4646

Case 4.4 Should a LOB Own its Case 4.4 Should a LOB Own its Own Data Mart?Own Data Mart?

A sales unit needs the capability of a A sales unit needs the capability of a data mart. They have asked IT to build data mart. They have asked IT to build it for them but the request is low on it for them but the request is low on IT’s priority list. The Sales line-of-IT’s priority list. The Sales line-of-business has neither the expertise nor business has neither the expertise nor the inclination to build a data mart on the inclination to build a data mart on its own, but it does have the budget. its own, but it does have the budget. What should the Sales organization do?What should the Sales organization do?

4747

Solution 4.4 Should a Line of Solution 4.4 Should a Line of Business (LOB) Own its Own Business (LOB) Own its Own

Data Mart?Data Mart? Depends on its power and budget – they Depends on its power and budget – they

may not ask your permissionmay not ask your permission IT should be involved in sourcing the IT should be involved in sourcing the

data, enterprise standards, metadata, data, enterprise standards, metadata, tools, training, support, and looking at tools, training, support, and looking at sharing, integration, and MDM sharing, integration, and MDM opportunitiesopportunities

If the LOB engages a consultant, IT If the LOB engages a consultant, IT should monitor plans and activitiesshould monitor plans and activities

4848

Case 4.5 Project has no Case 4.5 Project has no Dedicated StaffDedicated Staff

A data warehouse project manager has been tasked A data warehouse project manager has been tasked to manage, develop and support an enterprise data to manage, develop and support an enterprise data warehouse that crosses multiple divisions. This warehouse that crosses multiple divisions. This manager has almost no dedicated staff, but must manager has almost no dedicated staff, but must rely on pulling business and IT people from each rely on pulling business and IT people from each line of business as they are needed on the project. line of business as they are needed on the project. The people he needs are often not available The people he needs are often not available especially when they are needed. Important especially when they are needed. Important meetings are unproductive when key personnel are meetings are unproductive when key personnel are not in attendance. This has caused major delays not in attendance. This has caused major delays and wasted time for those who did attend the and wasted time for those who did attend the meetings. Sign-offs have not taken place on time meetings. Sign-offs have not taken place on time and many decisions have had to be delayed. What and many decisions have had to be delayed. What should this data warehouse project manager do?should this data warehouse project manager do?

4949

Solution 4.5 Project has no Solution 4.5 Project has no Dedicated StaffDedicated Staff

Is the project worth doing? Is the project worth doing? Is there any justification?Is there any justification? Has the project been sold?Has the project been sold? Is there any support in the business Is there any support in the business

or in IT?or in IT?

If the answer is “No” abort the projectIf the answer is “No” abort the project

5050

Case 4.6 Project Manager has a Case 4.6 Project Manager has a Bad ReputationBad Reputation

The data warehouse project manager The data warehouse project manager reports to the data warehouse reports to the data warehouse manager. The project manager comes manager. The project manager comes from the IT side of the organization from the IT side of the organization and is not well liked or respected by and is not well liked or respected by the business. The business people do the business. The business people do not answer his phone calls or respond not answer his phone calls or respond to his email messages. What should to his email messages. What should the data warehouse manager do?the data warehouse manager do?

5151

Solution 4.6 Project Manager Solution 4.6 Project Manager has a Bad Reputationhas a Bad Reputation

Project Manager with a bad reputation will:Project Manager with a bad reputation will: Will not get cooperationWill not get cooperation Will not be able to recruit and retain good Will not be able to recruit and retain good

peoplepeople Will not be respected by his or her teamWill not be respected by his or her team Will not be trusted to deliverWill not be trusted to deliver

Replace the project managerReplace the project manager Project manager should be highly respected by Project manager should be highly respected by

business and ITbusiness and IT

5252

Case 4.8 Organization is Not Case 4.8 Organization is Not Ready for a DWReady for a DW

The organization is not ready for the data The organization is not ready for the data warehouse. This lack of readiness extends warehouse. This lack of readiness extends from technical skills, availability of staff, lack from technical skills, availability of staff, lack of motivation, political infighting, assassins, a of motivation, political infighting, assassins, a CIO ready to retire – who doesn’t want to take CIO ready to retire – who doesn’t want to take any risks, the business that neither wants the any risks, the business that neither wants the data warehouse nor has the money or the data warehouse nor has the money or the inclination to participate in any data inclination to participate in any data warehouse endeavor. What should the newly warehouse endeavor. What should the newly appointed data warehouse manager do?appointed data warehouse manager do?

5353

Solution 4.8 Organization is not Solution 4.8 Organization is not Ready for a DWReady for a DW

Find a powerful business sponsorFind a powerful business sponsor Implement small reporting improvementsImplement small reporting improvements Improve data quality Improve data quality Measure and report on the small Measure and report on the small

successessuccesses

DW Manager should probably find DW Manager should probably find another jobanother job

5454

5. User Issues5. User Issues

““With the customer as the reference With the customer as the reference point, priorities become easier to point, priorities become easier to set.”set.”

Mary WaltonMary Walton

5555

Case 5.1 All the Users Want it Case 5.1 All the Users Want it All NowAll Now

An organization has a corporate culture An organization has a corporate culture that fosters “siloized” business units that that fosters “siloized” business units that have IT implementations that are not have IT implementations that are not integrated with each other. However the integrated with each other. However the multiple business unit sponsors want their multiple business unit sponsors want their data in the data warehouse and they all data in the data warehouse and they all want it now. It’s obvious that not all the want it now. It’s obvious that not all the sponsors can be satisfied at once. What sponsors can be satisfied at once. What should be done so that none of the should be done so that none of the business unit sponsors is angered or business unit sponsors is angered or develops his or her own data warehouse?develops his or her own data warehouse?

5656

Solution 5.1 All the Users Solution 5.1 All the Users Want it All NowWant it All Now

Establish an advisory board/steering Establish an advisory board/steering committee to prioritize projectscommittee to prioritize projects

Use cost justification to aid in prioritizationUse cost justification to aid in prioritization Build a project time line showing when each Build a project time line showing when each

user’s project should begin and completeuser’s project should begin and complete Sell the idea of an enterprise DW and Sell the idea of an enterprise DW and

discourage departments from building their discourage departments from building their own showing the costs and problems of own showing the costs and problems of doing sodoing so

5757

Case 5.2 Business Does Not Case 5.2 Business Does Not Support the ProjectSupport the Project

A consulting organization is hired by A consulting organization is hired by IT to build a data warehouse. The IT to build a data warehouse. The business is not supportive of the business is not supportive of the project but IT tells the consultant to project but IT tells the consultant to keep working even though the keep working even though the business side is making plans to business side is making plans to terminate them. What should the terminate them. What should the consulting company do?consulting company do?

5858

Solution 5.2 Business Does Not Solution 5.2 Business Does Not Support the ProjectSupport the Project

Determine why the business is not Determine why the business is not supportivesupportive

Determine if there is a business case Determine if there is a business case for the DWfor the DW

Try to sell the business on the value of Try to sell the business on the value of the DWthe DW

If none of this works, the project If none of this works, the project should be killedshould be killed

5959

Case 5.4 Users Have High Data Case 5.4 Users Have High Data Quality ExpectationsQuality Expectations

Somehow the business users have been Somehow the business users have been led to believe that the data they will be led to believe that the data they will be seeing will be complete, accurate and seeing will be complete, accurate and very timely. They came to that very timely. They came to that conclusion since no one at the time conclusion since no one at the time indicated anything to the contrary. indicated anything to the contrary. What should be done to reset their What should be done to reset their expectations to the reality of what they expectations to the reality of what they will be getting?will be getting?

6060

Solution 5.4 Users Have High Solution 5.4 Users Have High Data Quality ExpectationsData Quality Expectations

Educate users on the various dimensions of Educate users on the various dimensions of data qualitydata quality

Understand users’ requirements for data Understand users’ requirements for data quality – not just wishesquality – not just wishes

Set user expectations about data quality early Set user expectations about data quality early and oftenand often

Profile the data and report findings to the usersProfile the data and report findings to the users Capture quality metrics in the ETL processCapture quality metrics in the ETL process Keep data quality indicators in metadataKeep data quality indicators in metadata Involve the users in your data quality effortInvolve the users in your data quality effort

6161

Case 5.5 Users Don’t Know Case 5.5 Users Don’t Know What They WantWhat They Want

An organization with unforgiving An organization with unforgiving users is attempting a data users is attempting a data warehouse. It’s become very difficult warehouse. It’s become very difficult to get the users to articulate what to get the users to articulate what they want or even why they would they want or even why they would want a data warehouse. What should want a data warehouse. What should the data warehouse team do?the data warehouse team do?

6262

Solution 5.5 Users Don’t Know Solution 5.5 Users Don’t Know What They WantWhat They Want

Does the DW Team have a solution looking Does the DW Team have a solution looking for a problem?for a problem?

Develop a proof of concept (POC) or a Develop a proof of concept (POC) or a prototype and use it to help the users prototype and use it to help the users understand what they can receiveunderstand what they can receive

Provide the users with stories of the DW in Provide the users with stories of the DW in their industrytheir industry

Identify opportunities and present them to Identify opportunities and present them to the usersthe users

Be sure you are using business terminology Be sure you are using business terminology in your discussions with the usersin your discussions with the users

6363

6. Team Issues6. Team Issues

““The greater the loyalty of the The greater the loyalty of the members of a group toward the members of a group toward the group, the greater is the motivation group, the greater is the motivation among the members to achieve the among the members to achieve the goals of the group, and the greater goals of the group, and the greater the probability that the group will the probability that the group will achieve its goals.”achieve its goals.”

Rensis LikertRensis Likert

6464

Previous ExperiencePrevious Experience

The previous experience each member The previous experience each member has had with the other team has had with the other team members is probably more important members is probably more important than any other single factor in than any other single factor in predicting how well the people on the predicting how well the people on the team will interact.team will interact.

6565

Case 6.2 Management has Case 6.2 Management has Assigned a Dysfunctional Team Assigned a Dysfunctional Team

Member to the Project Member to the Project A project manager has been given a A project manager has been given a

team that is unskilled, unmotivated team that is unskilled, unmotivated and generally the worst of what and generally the worst of what other managers did not want on their other managers did not want on their team. The project manager has been team. The project manager has been asked to do the best she can. What asked to do the best she can. What should she do?should she do?

6666

Solution 6.2 Management has Solution 6.2 Management has Assigned a Dysfunctional Team Assigned a Dysfunctional Team

Member to the ProjectMember to the Project Management needs to understand that with Management needs to understand that with

this unskilled and unmotivated team, the this unskilled and unmotivated team, the project will failproject will fail

Pair the team members with experienced Pair the team members with experienced consultantsconsultants

Bonuses should motivate the team but if they Bonuses should motivate the team but if they don’t, remove those who are unmotivateddon’t, remove those who are unmotivated

Provide time and training to learn the Provide time and training to learn the necessary skills necessary skills

6767

Case 6.7 Consultants are in Case 6.7 Consultants are in ChargeCharge

The new CIO came from one of the big consulting The new CIO came from one of the big consulting organizations and brought in three of his lieutenants organizations and brought in three of his lieutenants with him. These lieutenants now hold the important with him. These lieutenants now hold the important positions in the IT organization. The data warehouse positions in the IT organization. The data warehouse reports to one of these managers. This manager has reports to one of these managers. This manager has contracted with his old organization for consulting contracted with his old organization for consulting help for the new data warehouse project. The project help for the new data warehouse project. The project manager has been asked to work with these manager has been asked to work with these consultants who seem to have great power and consultants who seem to have great power and influence over the data warehouse project. In fact, influence over the data warehouse project. In fact, the views and positions of the project manager have the views and positions of the project manager have been seriously undermined. The charges are large been seriously undermined. The charges are large and the project looks like it will be way over budget. and the project looks like it will be way over budget. What should the project manager do?What should the project manager do?

6868

Solution 6.7 The Consultants Solution 6.7 The Consultants are in Chargeare in Charge

Project manager should insist on Project manager should insist on having authority for the projecthaving authority for the project

Find an ally on the business sideFind an ally on the business side Document all decisions made or Document all decisions made or

overturned by the consultantsoverturned by the consultants Failing to get authority, the project Failing to get authority, the project

manager should quit the projectmanager should quit the project

6969

Case 6.8 The Consultants Have Case 6.8 The Consultants Have FledFled

A data warehouse was built three years ago. A data warehouse was built three years ago. None of the contractors who developed the None of the contractors who developed the system have remained and the system have remained and the documentation is poor and out of date. The documentation is poor and out of date. The data is dirty and there are no controls for data is dirty and there are no controls for data integrity. The users are unhappy with data integrity. The users are unhappy with the existing data warehouse. A new manager the existing data warehouse. A new manager has been given responsibility for all the data has been given responsibility for all the data warehouse activity in the organization. What warehouse activity in the organization. What should this manager do?should this manager do?

7070

Solution 6.8 The Solution 6.8 The Consultants have FledConsultants have Fled

Asses the existing DW. Can it be salvaged? If it Asses the existing DW. Can it be salvaged? If it can, create a plan, estimate budget, resources, can, create a plan, estimate budget, resources, cost justification, and schedule. If it can’t, give cost justification, and schedule. If it can’t, give management the bad news and develop a new management the bad news and develop a new DW and give it a new name.DW and give it a new name. Are original requirements being satisfied?Are original requirements being satisfied? What pieces of the design are appropriate and What pieces of the design are appropriate and

effective?effective? What’s the status of the documentation?What’s the status of the documentation? What’s the state of the quality of the data?What’s the state of the quality of the data? What are the skills and experience of the DW Team?What are the skills and experience of the DW Team?

7171

Case 6.9 Knowledge Transfer is Case 6.9 Knowledge Transfer is Not HappeningNot Happening

An organization planned to bring in a data An organization planned to bring in a data warehouse consultant to help with the first warehouse consultant to help with the first implementation. They then planned to continue implementation. They then planned to continue developing using their own staff who should have developing using their own staff who should have been trained by the consultant. The consultant been trained by the consultant. The consultant assured the client that knowledge transfer was part assured the client that knowledge transfer was part of their work but as the schedule became tight, the of their work but as the schedule became tight, the consultant did not have the time to transfer their consultant did not have the time to transfer their knowledge to the organization’s employees. The knowledge to the organization’s employees. The employees were only performing simple tasks and employees were only performing simple tasks and they learned very little from the first implementation. they learned very little from the first implementation. How can an organization assure itself that knowledge How can an organization assure itself that knowledge transfer does take place?transfer does take place?

7272

Solution 6.9 Knowledge Solution 6.9 Knowledge Transfer is not HappeningTransfer is not Happening

The contract with the consultant should have The contract with the consultant should have included measurable knowledge transferincluded measurable knowledge transfer

Payment to the consultant should include metrics for Payment to the consultant should include metrics for the deliverables of knowledge transferthe deliverables of knowledge transfer

The project plan should have included knowledge The project plan should have included knowledge transfer activities including seminars, classes, and transfer activities including seminars, classes, and mentoring sessionsmentoring sessions

Pair up consultants with employeesPair up consultants with employees Phase 1 – consultant does most of the major work, employee Phase 1 – consultant does most of the major work, employee

watches and learnswatches and learns Phase 2 – consultant and employee trade major tasksPhase 2 – consultant and employee trade major tasks Phase 3 – employee performs most of the major work, Phase 3 – employee performs most of the major work,

consultant reviews and advisesconsultant reviews and advises

7373

Case 6.10 How to Best Use Case 6.10 How to Best Use ConsultantsConsultants

A manager has been given the assignment A manager has been given the assignment to design and build a data warehouse to design and build a data warehouse infrastructure complete with standards, infrastructure complete with standards, methodology and tools. He was given the methodology and tools. He was given the budget and the mandate to bring in new budget and the mandate to bring in new tools along with consultants and tools along with consultants and contractors as needed. He does not have an contractors as needed. He does not have an unlimited budget. How should he bring in unlimited budget. How should he bring in consultants, for which jobs and for how consultants, for which jobs and for how long? How should he most effectively use long? How should he most effectively use consultants and contractors?consultants and contractors?

7474

Solution 6.10 How to Best Use Solution 6.10 How to Best Use ConsultantsConsultants

Differentiate between contractors and consultantsDifferentiate between contractors and consultants When specific skills and expertise are lacking – use When specific skills and expertise are lacking – use

contractorscontractors When high-level advice is required – use consultantWhen high-level advice is required – use consultant To assess the organization’s plans, progress, To assess the organization’s plans, progress,

organization, use of tools, …- use consultantorganization, use of tools, …- use consultant When the organization is unwilling to fill full time When the organization is unwilling to fill full time

positions – use contractorpositions – use contractor As a mentor to the DW Team – use contractor and As a mentor to the DW Team – use contractor and

consultant consultant One-time-only tasks – use contractorOne-time-only tasks – use contractor To convince management – use consultantTo convince management – use consultant

7575

Case 6.11 Management Wants Case 6.11 Management Wants to Outsourceto Outsource

A company is making some major changes. They A company is making some major changes. They will be outsourcing their operational systems to an will be outsourcing their operational systems to an application service provider (ASP) and they are application service provider (ASP) and they are considering outsourcing some or all of their data considering outsourcing some or all of their data warehouse activities. The new focus is on the warehouse activities. The new focus is on the customer and they are planning significant customer and they are planning significant customer relationship management (CRM) customer relationship management (CRM) capability. They have some minor data warehouse capability. They have some minor data warehouse capability today, but with this major change, capability today, but with this major change, should they even use any of the existing DW? How should they even use any of the existing DW? How can the organization be sure the outsourcing can the organization be sure the outsourcing organization will deliver the functions and the organization will deliver the functions and the capabilities needed? What recommendations do capabilities needed? What recommendations do you have for how they should proceed?you have for how they should proceed?

7676

Solution 6.11 Management Solution 6.11 Management Wants to OutsourceWants to Outsource

Understand the reasons for outsourcingUnderstand the reasons for outsourcing Understand the costs, risks, delays, and effort Understand the costs, risks, delays, and effort

to outsourceto outsource Understand how much intellectual capital is Understand how much intellectual capital is

lost with outsourcinglost with outsourcing Determine which activities and roles could be Determine which activities and roles could be

outsourced and which should remain in houseoutsourced and which should remain in house Understand the costs and problems if the Understand the costs and problems if the

work has to be brought back in (insourced)work has to be brought back in (insourced)

7777

7. Project Planning and 7. Project Planning and SchedulingScheduling

““The reason we don’t have the time to The reason we don’t have the time to fix it today is that we didn’t take the fix it today is that we didn’t take the time to do it right yesterday.”time to do it right yesterday.”

H. James H. James HarringtonHarrington

7878

Case 7.2 IT Management Sets Case 7.2 IT Management Sets Unrealistic DeadlinesUnrealistic Deadlines

IT has missed deadline after deadline and IT has missed deadline after deadline and has a reputation for never bringing in a has a reputation for never bringing in a project on time. This time they really don’t project on time. This time they really don’t want to miss. IT management has already want to miss. IT management has already made commitments to their bosses for a made commitments to their bosses for a schedule that is unrealistic but they are schedule that is unrealistic but they are counting on the data warehouse project counting on the data warehouse project manager to come through and deliver on manager to come through and deliver on time. Management has made it clear to the time. Management has made it clear to the project manager that her reputation and project manager that her reputation and career within the company depend on career within the company depend on meeting the schedule. What should she do?meeting the schedule. What should she do?

7979

Solution 7.2 IT Management Solution 7.2 IT Management Sets Unrealistic DeadlinesSets Unrealistic Deadlines

Make management aware of what is realistic and what is Make management aware of what is realistic and what is not and give management a best estimate of when the not and give management a best estimate of when the project will be completedproject will be completed

Ask management if they want the bad new now or later Ask management if they want the bad new now or later when the schedule slipswhen the schedule slips

Negotiate to phase the deliverables implementing a few Negotiate to phase the deliverables implementing a few that meet the schedule and deferring others to a later that meet the schedule and deferring others to a later phasephase

Share the project plan (at a high level) with managementShare the project plan (at a high level) with management Do not sacrifice qualityDo not sacrifice quality Adding more people will probably slip the schedule even Adding more people will probably slip the schedule even

moremore Do not ask your team to work 12 hour days and work Do not ask your team to work 12 hour days and work

weekendsweekends

8080

Case 7.3 Sponsor Changes Case 7.3 Sponsor Changes Scopes but Doesn’t Want to Scopes but Doesn’t Want to

Change ScheduleChange Schedule The project manager had a project The project manager had a project

agreement, developed a project plan and agreement, developed a project plan and allowed an additional 20% time, effort and allowed an additional 20% time, effort and budget for unanticipated contingencies. budget for unanticipated contingencies. The sponsor and a few others in the The sponsor and a few others in the sponsor’s department had been asking for sponsor’s department had been asking for some minor additional function that the some minor additional function that the project manager accepted. However, just project manager accepted. However, just recently, the sponsor made major requests recently, the sponsor made major requests for new data. Additionally, he did not want for new data. Additionally, he did not want the schedule to change. What should the the schedule to change. What should the project manager do?project manager do?

8181

Solution 7.3 Sponsor Changes Solution 7.3 Sponsor Changes Scope but does not want to Scope but does not want to

Change ScheduleChange Schedule Go back to your project agreementGo back to your project agreement Use your organization’s change control Use your organization’s change control

processesprocesses How important is the original schedule?How important is the original schedule? Do not agree to the changes without other Do not agree to the changes without other

concessionsconcessions Do not say “No”Do not say “No” Negotiate using the project agreement and Negotiate using the project agreement and

your project planyour project plan Defer some functions/deliverables to future Defer some functions/deliverables to future

phasesphases

8282

Case 7.4 Users Want First DW Case 7.4 Users Want First DW Delivered to Include EverythingDelivered to Include Everything The users are afraid that if they don’t ask The users are afraid that if they don’t ask

for everything in the first release, they may for everything in the first release, they may never see it at all so they are asking for all never see it at all so they are asking for all the functions they will eventually need and the functions they will eventually need and much more. This nullifies one of the benefits much more. This nullifies one of the benefits of a data warehouse implementation – the of a data warehouse implementation – the ability to phase deliveries. What approach ability to phase deliveries. What approach should the data warehouse project manager should the data warehouse project manager take to convince management that trying to take to convince management that trying to put it all in the initial release is a sure put it all in the initial release is a sure formula for either failure or for a very long formula for either failure or for a very long delivery schedule?delivery schedule?

8383

Solution 7.4 Users Want First Solution 7.4 Users Want First DW Delivered to Include DW Delivered to Include

EverythingEverything Sell the ability of the DW to nicely phase Sell the ability of the DW to nicely phase

projectsprojects Explain the value of learning from each phaseExplain the value of learning from each phase Highlight the risk with a very large projectHighlight the risk with a very large project Give a choice of everything in four years or a Give a choice of everything in four years or a

set of phased deliverables – they always select set of phased deliverables – they always select the phased approachthe phased approach

Show a plan that does include everything but Show a plan that does include everything but delivers in phases – three to five months is a delivers in phases – three to five months is a reasonable interlude for phasesreasonable interlude for phases

8484

Case 7.5 Project Manager Case 7.5 Project Manager Severely Underestimates the Severely Underestimates the

ScheduleSchedule A project manager has decades of A project manager has decades of

experience and is very competent in a experience and is very competent in a number of computer languages and number of computer languages and operating systems. He has extensive operating systems. He has extensive experience in his industry and knows who experience in his industry and knows who to call and where to find whatever he is to call and where to find whatever he is looking for. On top of this he is an eternal looking for. On top of this he is an eternal optimist, believing everything will progress optimist, believing everything will progress perfectly with no delays or false starts. He perfectly with no delays or false starts. He bases his team’s work estimates on how bases his team’s work estimates on how long it would take him to perform the task. long it would take him to perform the task. What should his manager do?What should his manager do?

8585

Solution 7.5 Project Manager Solution 7.5 Project Manager Severely Underestimates the Severely Underestimates the

ScheduleSchedule The estimates should be calculated by those The estimates should be calculated by those

who will be doing the work – with reviewswho will be doing the work – with reviews Ask the team members for three estimates for Ask the team members for three estimates for

each of their tasks, the best, the worst, and each of their tasks, the best, the worst, and their best guess. Take the best guess and add their best guess. Take the best guess and add 20% - 30%20% - 30%

Ask the team members for predecessors and Ask the team members for predecessors and dependenciesdependencies

Ask the team members for risks and Ask the team members for risks and assumptions in their estimates and factor assumptions in their estimates and factor those inthose in

8686

Technical SituationsTechnical Situations

Data Warehouse ToolsData Warehouse Tools Data QualityData Quality IntegrationIntegration Data Warehouse ArchitectureData Warehouse Architecture PerformancePerformance

8787

9. Tools and Vendors9. Tools and Vendors

““Praise from a salesman, in my humble Praise from a salesman, in my humble opinion, is one of life’s less opinion, is one of life’s less convincing complements.”convincing complements.”

Peter MaylePeter Mayle

8888

Case 9.1 Best Practices for Case 9.1 Best Practices for Writing a Request for ProposalWriting a Request for Proposal

A non-profit organization is considering A non-profit organization is considering a data warehouse to keep track of their a data warehouse to keep track of their membership and solicitation activity. membership and solicitation activity. Any project this large requires a request Any project this large requires a request for proposal/price (RFP). Of course they for proposal/price (RFP). Of course they do not want the RFP process to do not want the RFP process to significantly slow them down. What significantly slow them down. What recommendations would you give recommendations would you give them?them?

8989

Solution 9.1 Best Practices for Solution 9.1 Best Practices for Writing a Request for ProposalWriting a Request for Proposal

As simple and short as possibleAs simple and short as possible Include only characteristics used for Include only characteristics used for

comparisoncomparison Include only mandatory and highly Include only mandatory and highly

desirable requirementsdesirable requirements Include a glossaryInclude a glossary Give vendors your criteria for judgingGive vendors your criteria for judging Give vendors rules for respondingGive vendors rules for responding

9090

Case 9.2 Users Don’t Like the Case 9.2 Users Don’t Like the Query and Reporting ToolQuery and Reporting Tool

A company was sorry to learn that only A company was sorry to learn that only 5% of the users who went through 5% of the users who went through training used the data warehouse training used the data warehouse regularly. On further investigation, they regularly. On further investigation, they learned that those who did not use the learned that those who did not use the data warehouse were uncomfortable data warehouse were uncomfortable with the query/reporting tool and with the query/reporting tool and reluctant to use it. What should be reluctant to use it. What should be done?done?

9191

Solution 9.2 Users Don’t Like Solution 9.2 Users Don’t Like the Query and Reporting Toolthe Query and Reporting Tool

Investigate why they don’t like the toolInvestigate why they don’t like the tool If the tool has a bad reputation, get a new oneIf the tool has a bad reputation, get a new one The tool and facilities of the tool should match the The tool and facilities of the tool should match the

capabilities, interests, and work activities of the userscapabilities, interests, and work activities of the users Training should be tailored to users’ capabilities and interestsTraining should be tailored to users’ capabilities and interests Give the casual users access to a query and report library Give the casual users access to a query and report library

and only train them on accessing the libraryand only train them on accessing the library Provide mentors during the training workshopsProvide mentors during the training workshops Teach the users about the dataTeach the users about the data Evaluate the effectiveness of the trainingEvaluate the effectiveness of the training Users should only have access to the tool when they have Users should only have access to the tool when they have

completed the trainingcompleted the training User support should be sensitive to problems with the toolUser support should be sensitive to problems with the tool Measure user satisfactionMeasure user satisfaction

9292

Case 9.3 IT has Already Chosen Case 9.3 IT has Already Chosen the Toolthe Tool

Even though a tool selection committee was Even though a tool selection committee was formed and supposedly has the authority to formed and supposedly has the authority to chose data warehouse tools, some powerful chose data warehouse tools, some powerful people in the IT department already know people in the IT department already know what they want and will resist any what they want and will resist any recommendations that do not correspond to recommendations that do not correspond to their choice. Some members of the tool their choice. Some members of the tool selection committee are concerned that the selection committee are concerned that the chosen tools won’t perform. What should the chosen tools won’t perform. What should the selection committee do?selection committee do?

9393

Solution 9.3 IT Has Already Solution 9.3 IT Has Already Chosen the ToolChosen the Tool

Determine if the selection committee has or does Determine if the selection committee has or does not have the authority to make the decisions. If not have the authority to make the decisions. If they don’t have the authority, the committee they don’t have the authority, the committee should be disbanded.should be disbanded.

Determine if the tool will performDetermine if the tool will perform Determine if the tool has the functions and Determine if the tool has the functions and

capabilities neededcapabilities needed Document the business requirements and match Document the business requirements and match

each product against the requirements along with each product against the requirements along with the metrics and weighting for comparisonsthe metrics and weighting for comparisons

Document the evaluations including the reasons Document the evaluations including the reasons for applying the gradesfor applying the grades

9494

Case 9.4 Will the Tools Perform Case 9.4 Will the Tools Perform Well?Well?

A company expects to have a data warehouse A company expects to have a data warehouse with over 50 million records within two years. with over 50 million records within two years. They would like to use an They would like to use an extract/transform/load (ETL) tool but they are extract/transform/load (ETL) tool but they are concerned about the ability of the tool to concerned about the ability of the tool to perform. They know these tools are expensive perform. They know these tools are expensive and they do not want to get started with a and they do not want to get started with a product that will have to be abandoned as the product that will have to be abandoned as the volumes increase. How can they be sure that volumes increase. How can they be sure that the tools they are considering will perform up the tools they are considering will perform up to their expectations?to their expectations?

9595

Solution 9.4 Will the Tools Solution 9.4 Will the Tools Perform Well?Perform Well?

Pay no attention to theoretical volumesPay no attention to theoretical volumes Talk to references with volumes as large as Talk to references with volumes as large as

those you are expecting and with similar those you are expecting and with similar levels of ETL complexity and with as many levels of ETL complexity and with as many queries – with the same or greater level of queries – with the same or greater level of complexity – as you are expectingcomplexity – as you are expecting

The references should be on the same The references should be on the same platform platform

Ask the references if they had to take Ask the references if they had to take extraordinary actions to make the tool extraordinary actions to make the tool perform or if special skills were neededperform or if special skills were needed

Ask the vendors to run benchmarksAsk the vendors to run benchmarks

9696

Case 9.5 The Vendor is Trying Case 9.5 The Vendor is Trying to Sell His Complete Suite of to Sell His Complete Suite of

ProductsProducts A data warehouse vendor has a suite of A data warehouse vendor has a suite of

data warehouse products, some of data warehouse products, some of which are excellent but others are far which are excellent but others are far from best-of-breed. The vendor is from best-of-breed. The vendor is recommending the entire suite making recommending the entire suite making it clear that the only way to get top-it clear that the only way to get top-level support is to buy the complete level support is to buy the complete package. What should the package. What should the organization’s response be?organization’s response be?

9797

Solution 9.5 The Vendor is Solution 9.5 The Vendor is Trying to Sell His Complete Trying to Sell His Complete

SuiteSuite Do not be influenced by threats of reduced Do not be influenced by threats of reduced

support if the entire suite is not purchased. support if the entire suite is not purchased. Push back. Tell the vendor that if he can’t Push back. Tell the vendor that if he can’t guarantee good support for the products you guarantee good support for the products you do purchase, his products will not be chosendo purchase, his products will not be chosen

There are benefits to getting everything There are benefits to getting everything from the same vendorfrom the same vendor

If portions of the suite are substandard, If portions of the suite are substandard, purchase a better tool and integrate it with purchase a better tool and integrate it with those of the vendorsthose of the vendors

9898

Case 9.7 Vendors Acquiring Case 9.7 Vendors Acquiring Company Provides Poor Company Provides Poor

SupportSupport An organization was happy with the An organization was happy with the

query tool it was using but the vendor query tool it was using but the vendor was in financial trouble and sold out. was in financial trouble and sold out. The acquiring company fired most of The acquiring company fired most of the developers and the support staff. the developers and the support staff. Needless to say, support is terrible and Needless to say, support is terrible and based on the reputation of the based on the reputation of the acquiring vendor, support is not acquiring vendor, support is not expected to improve. What should the expected to improve. What should the company do?company do?

9999

Solution 9.7 Vendor’s Acquiring Solution 9.7 Vendor’s Acquiring Company Provides Poor Company Provides Poor

SupportSupport Try to determine if support will be Try to determine if support will be

improved and how long that will takeimproved and how long that will take All new development should be with a All new development should be with a

tool that does have good supporttool that does have good support Consider switching to a vendor with Consider switching to a vendor with

good supportgood support Research the possibility that a third Research the possibility that a third

party might provide supportparty might provide support

100100

11. Data Quality11. Data Quality

““There is no manual that deals with There is no manual that deals with the real business of motorcycle the real business of motorcycle maintenance, the most important maintenance, the most important aspect of all. Caring about what aspect of all. Caring about what you’re doing is considered you’re doing is considered unimportant or taken for granted.”unimportant or taken for granted.”

Robert PirsigRobert Pirsig

101101

Case 11.2 Redundant Data Case 11.2 Redundant Data Needs to be EliminatedNeeds to be Eliminated

A telecommunication company has a A telecommunication company has a data warehouse containing 14 terabytes data warehouse containing 14 terabytes of data. It has been estimated that more of data. It has been estimated that more than ten terabytes is probably than ten terabytes is probably redundant. The company has no naming redundant. The company has no naming conventions and only 20% of the data conventions and only 20% of the data have associated meta data. How can have associated meta data. How can they identify and eliminate unneeded they identify and eliminate unneeded redundant data?redundant data?

102102

Solution 11.2 Redundant Data Solution 11.2 Redundant Data Needs to be EliminatedNeeds to be Eliminated

Research the sources of the redundant data – how is it Research the sources of the redundant data – how is it getting here? Determine which tables are based on getting here? Determine which tables are based on the same sourcesthe same sources

Are there business reasons for the redundancy?Are there business reasons for the redundancy? Research identical data elements with different namesResearch identical data elements with different names Metadata should play a role in this processMetadata should play a role in this process What is the cost justification for eliminating the What is the cost justification for eliminating the

redundant data?redundant data? Develop a standard naming convention for the Develop a standard naming convention for the

business terms and the physical data namesbusiness terms and the physical data names Identify tables with redundant data and track their Identify tables with redundant data and track their

sourcessources

103103

Case 11.3 Management Case 11.3 Management Underestimated the Amount of Underestimated the Amount of

Dirty DataDirty Data Management has never recognized just how dirty Management has never recognized just how dirty

the data is in the operational systems. They are the data is in the operational systems. They are unaware of the degree of redundancy, how unaware of the degree of redundancy, how incomplete many of the records are, the use of incomplete many of the records are, the use of inappropriate defaults, the data that does not inappropriate defaults, the data that does not conform to the valid values, the lack of referential conform to the valid values, the lack of referential integrity and the data that is just inaccurate. As integrity and the data that is just inaccurate. As the data warehouse has been piloted, feedback the data warehouse has been piloted, feedback from the project team and from the users has from the project team and from the users has made it clear that the quality of the data is not made it clear that the quality of the data is not acceptable to allow the project to proceed. acceptable to allow the project to proceed. Cleaning up the data will take significant time, and Cleaning up the data will take significant time, and that time has not been allocated in the project that time has not been allocated in the project schedule. What should the project manager do?schedule. What should the project manager do?

104104

Solution 11.3 Management Solution 11.3 Management Underestimated the Amount of Underestimated the Amount of

Dirty DataDirty Data Provide a monthly report card to IT management Provide a monthly report card to IT management

and to the data owners based on profiling the dataand to the data owners based on profiling the data Give a presentation on data quality that Give a presentation on data quality that

incorporates the result of the profiling and incorporates the result of the profiling and estimates of the cost of dirty data and what steps estimates of the cost of dirty data and what steps should be taken to improve data qualityshould be taken to improve data quality

Include the time and effort to clean up the data in Include the time and effort to clean up the data in the project planthe project plan

Do not deliver dirty data to the usersDo not deliver dirty data to the users Determine which data elements must be clean and Determine which data elements must be clean and

which are not as importantwhich are not as important

105105

Case 11.4 Managements does Case 11.4 Managements does not Recognize the Value of not Recognize the Value of

Data QualityData Quality No one is sure just how dirty the data is but it’s No one is sure just how dirty the data is but it’s

pretty clear that the level of quality will not be pretty clear that the level of quality will not be acceptable for the data warehouse. It is also clear acceptable for the data warehouse. It is also clear that the cleansing process will be costly and will that the cleansing process will be costly and will take dedicated staff. Management is not even take dedicated staff. Management is not even aware of the data quality problems – the data aware of the data quality problems – the data seems to be working just fine for the operational seems to be working just fine for the operational systems. Furthermore, management is not systems. Furthermore, management is not inclined to spend money or resources fixing the inclined to spend money or resources fixing the very dirty data. What should be done to convince very dirty data. What should be done to convince management of the need to clean the data?management of the need to clean the data?

106106

Solution 11.4 Management Does Solution 11.4 Management Does Not Recognize the Value of Data Not Recognize the Value of Data

QualityQuality Capture internal stories of problems resulting Capture internal stories of problems resulting

from poor quality data and try to quantify the from poor quality data and try to quantify the costs or problems associated with bad datacosts or problems associated with bad data

Research stories from other companies, Research stories from other companies, especially those in your industryespecially those in your industry

Data quality vendors have metrics on data Data quality vendors have metrics on data quality justificationquality justification

Identify potential regulatory fines, problems, Identify potential regulatory fines, problems, and embarrassments from poor data qualityand embarrassments from poor data quality

Identify potential bad business decisions that Identify potential bad business decisions that could resultcould result

107107

12. Integration12. Integration

““Few things are less productive than Few things are less productive than duplication of effort and the resulting duplication of effort and the resulting need for reconciliation of inconsistent need for reconciliation of inconsistent data.”data.”

- Repository Data Model Strategy - Repository Data Model Strategy PaperPaper

108108

Case 12.4 Reports From the Case 12.4 Reports From the DW and the Operational DW and the Operational

System Don’t MatchSystem Don’t Match The data warehouse manager is responsible for all The data warehouse manager is responsible for all

the data warehouse initiatives in the company. He the data warehouse initiatives in the company. He recognizes that he will have credibility problems if recognizes that he will have credibility problems if the reports and queries that come from the data the reports and queries that come from the data warehouse do not correspond to those of the warehouse do not correspond to those of the operational systems. He also knows that much of operational systems. He also knows that much of the operational data is dirty and must be cleansed the operational data is dirty and must be cleansed to satisfy the needs of the analysts who will be the to satisfy the needs of the analysts who will be the primary users of the data warehouse. He knows primary users of the data warehouse. He knows that if he transforms and cleans up the data as he that if he transforms and cleans up the data as he brings it into the data warehouse, the report brings it into the data warehouse, the report results will not correspond to those of the results will not correspond to those of the operational systems and the validity of the data operational systems and the validity of the data warehouse will be questioned. What should he do?warehouse will be questioned. What should he do?

109109

Solution 12.4 Reports From the Solution 12.4 Reports From the DW and the Operational DW and the Operational

System Don’t MatchSystem Don’t Match Educate the users on the reasons the DW is Educate the users on the reasons the DW is

different than the operational reports. different than the operational reports. Different timingDifferent timing Different controlsDifferent controls Different editsDifferent edits Data quality improvementsData quality improvements

Demonstrate the DW improvementsDemonstrate the DW improvements Convince users to rely on the DW reportsConvince users to rely on the DW reports Convince users that “matching to the penny” is Convince users that “matching to the penny” is

not a strategic requirementnot a strategic requirement Involve the users early on and in testing so they Involve the users early on and in testing so they

will anticipate and understand the differenceswill anticipate and understand the differences

110110

Case 12.5 Should the DW Case 12.5 Should the DW Team Fix a Problem Team Fix a Problem Operational System?Operational System?

It has become clear that the It has become clear that the operational system that feeds the operational system that feeds the data warehouse is inadequate and data warehouse is inadequate and management believes that a part of management believes that a part of the job of the data warehouse project the job of the data warehouse project is to fix the operational system. is to fix the operational system. Should this be attempted?Should this be attempted?

111111

Solution 12.5 Should the DW Solution 12.5 Should the DW Team Fix a Problem Team Fix a Problem Operational System?Operational System?

The DW Team cannot and should not have responsibility The DW Team cannot and should not have responsibility for fixing a broken operational systemfor fixing a broken operational system

The DW Team should provide feedback and information The DW Team should provide feedback and information on what needs fixing to those responsible for the on what needs fixing to those responsible for the operational system including the business ownersoperational system including the business owners

Continued contact with the operational team should Continued contact with the operational team should minimize problems when the operational system is fixedminimize problems when the operational system is fixed

The DW Team must identify operational problems that The DW Team must identify operational problems that would cause the DW project to failwould cause the DW project to fail

However, the rule always is that it is cheaper and easier However, the rule always is that it is cheaper and easier to fix the data as close to its source as possibleto fix the data as close to its source as possible

112112

Case 13.3 Case 13.3 Management Wants to Management Wants to Develop a DW Simultaneously with a Develop a DW Simultaneously with a

New Operational SystemNew Operational System A manufacturing company is in the midst A manufacturing company is in the midst

of implementing a new operational of implementing a new operational system with the normal problems of any system with the normal problems of any new system. Management is pushing to new system. Management is pushing to install a data warehouse with this new install a data warehouse with this new operational system’s data as the source. operational system’s data as the source. Is there anything they can do now or Is there anything they can do now or should they wait until all the bugs are should they wait until all the bugs are out before they start their work?out before they start their work?

113113

Solution 13.3 Management Wants to Solution 13.3 Management Wants to Develop a DW Simultaneously with a Develop a DW Simultaneously with a

New Operational SystemNew Operational System Alert management to the extra time, effort and money that Alert management to the extra time, effort and money that

will be requiredwill be required Start with user information requirementsStart with user information requirements Traditional project management activities can begin before Traditional project management activities can begin before

the operational system is in placethe operational system is in place Hardware and DW software can be installed and testedHardware and DW software can be installed and tested IT personnel can be trained and given DW tasks to performIT personnel can be trained and given DW tasks to perform Data models can be developed recognizing that changes will be Data models can be developed recognizing that changes will be

requiredrequired Stable data sources can be profiled for data quality and ETL Stable data sources can be profiled for data quality and ETL

processes against these sources can be developed and testedprocesses against these sources can be developed and tested Avoid physical DW and ETL design until the operational Avoid physical DW and ETL design until the operational

system is stabilizedsystem is stabilized DW Team can build prototypes and show them to the usersDW Team can build prototypes and show them to the users But make the DW schedule dependent on the But make the DW schedule dependent on the

operational system completion operational system completion

114114

Case 13.4 The DW Gets Case 13.4 The DW Gets Assigned the Role of a Assigned the Role of a

Reporting SystemReporting System The team for a large bank knew early in the The team for a large bank knew early in the

project that they could deliver a web based project that they could deliver a web based reporting system on time if it was done in reporting system on time if it was done in moderation. Additional reports were to be added moderation. Additional reports were to be added iteratively after the initial implementation as iteratively after the initial implementation as enhancements. This strategy was completely enhancements. This strategy was completely abandoned and the reporting requirements started abandoned and the reporting requirements started to increase as the more realistic team members to increase as the more realistic team members were no longer able to make decisions. The were no longer able to make decisions. The Advisory Group that made the decisions to include Advisory Group that made the decisions to include the total reporting capability failed to understand the total reporting capability failed to understand the meaning of a data warehouse - to them it the meaning of a data warehouse - to them it means only a reporting system. What can be done means only a reporting system. What can be done to set the Advisory Group on the right path?to set the Advisory Group on the right path?

115115

Solution 13.4 The DW Gets Solution 13.4 The DW Gets Assigned the Role of a Assigned the Role of a

Reporting SystemReporting System Educate management on the Educate management on the

opportunities that go far beyond opportunities that go far beyond reportingreporting

Demonstrate queries, analysis, data Demonstrate queries, analysis, data mining, predictive analyticsmining, predictive analytics

Give the users new ways of receiving Give the users new ways of receiving their reports (web, PDAs, email)their reports (web, PDAs, email)

116116

14. Performance14. Performance

““The ultimate test of management is The ultimate test of management is performance.”performance.”

Peter DruckerPeter Drucker

117117

Case 14.1 Software Does Not Case 14.1 Software Does Not Perform ProperlyPerform Properly

An organization made a major An organization made a major financial, training and financial, training and implementation commitment to a implementation commitment to a software product. It now appears software product. It now appears that the software will not perform. that the software will not perform. What should the company do?What should the company do?

118118

Solution 14.1 Software Does Solution 14.1 Software Does Not Perform ProperlyNot Perform Properly

Software conversions are expensive – Software conversions are expensive – are they necessary?are they necessary?

Are you properly using the software?Are you properly using the software? Does the contract with the vendor Does the contract with the vendor

have any performance guarantees?have any performance guarantees? Does the vendor have plans for Does the vendor have plans for

improvement?improvement? Be sure any new product will perform – Be sure any new product will perform –

how can you be sure?how can you be sure?119119

Case14.2 DW Grows Faster Case14.2 DW Grows Faster Than the Source DataThan the Source Data

The data warehouse is growing much faster The data warehouse is growing much faster than the source data that feeds it. The costs than the source data that feeds it. The costs for the hardware are already over budget for the hardware are already over budget and there appears to be no end in sight. and there appears to be no end in sight. Management is concerned and is asking Management is concerned and is asking some embarrassing questions. Should the some embarrassing questions. Should the data warehouse grow disproportionately to data warehouse grow disproportionately to the source data? If not, what can be done to the source data? If not, what can be done to stem the growth?stem the growth?

120120

Solution 14.2 DW Grows Faster Solution 14.2 DW Grows Faster Than the Source DataThan the Source Data

Are new applications cost justified?Are new applications cost justified? Are new requests for more data cost justified?Are new requests for more data cost justified? Improve requirements gathering processImprove requirements gathering process Measure usage so you know what data is being accessedMeasure usage so you know what data is being accessed Evaluate the need to maintain detail data versus summarized Evaluate the need to maintain detail data versus summarized

datadata Understand the need for how frequently the data is stored in Understand the need for how frequently the data is stored in

the DWthe DW Develop an archive strategy to place rarely used and older Develop an archive strategy to place rarely used and older

data on less expensive mediumdata on less expensive medium Plan to minimize redundant data making use of metadata Plan to minimize redundant data making use of metadata

and the means to share dataand the means to share data Consider a chargeback scheme where the user pays for the Consider a chargeback scheme where the user pays for the

data they want storeddata they want stored Perform a design reviewPerform a design review

121121

SummarySummary

So many problemsSo many problems There are solutions to most of those There are solutions to most of those

problems and impossible situationsproblems and impossible situations Don’t kid yourselfDon’t kid yourself Identify the problems earlyIdentify the problems early Take the steps necessary to be Take the steps necessary to be

successfulsuccessful

122122