7/29/2019 2008 07 the Road to Conversion

1/5TORONTO USERS GROUP for Power Systems July 20088

Picture yoursel in the ollowing situation:You have a new job, in a dierent city, onthe opposite coast. Your move will be billedby the pound and you will pay or hal oit. Would you want to take everything youhave in your current house? You may ask

yoursel: Is everything I have accumulatedstill useul? and most importantly: howmuch will it cost to move items that are notnecessary useul and that will take space?ypically, this would be a good opportunityor tossing, selling (garage sale), or evengiving away as many obsolete or unused

items, as possible beore the actual move.

What makes easy sense in the exampleabove does not always jump to the top-o-mind awareness as the rst step in a systemsconversion project. Te perception, oen, isthat i the data is good enough to work with,it is probably clean enough. Te reality isthat most likely, a signicant percentage othe data no longer has any signicant value.

Worse, it will cost more to convert and willprobably become a liability. o avoid this,

the ollowing steps can be taken:

1) Obsolete data can be identied inadvanceand thus can be completelyignoredand not subsequently converted.

2) Valid data destined to be converted canbe cleaned. oward this end one must:

Ensure that the data is complete.Examine each column/row thatwill be exported to the newsystem. Ensure that this data does

not have blanks or NULLs.Ensure that the data is accurate. Makesure that it is valid and current.

Ensure that the data is consistentlypositioned within ree-orm elds.For example, i going rom a systemwhere the address elds are ree-ormto one where each part o the addresshas a discrete eld or column e.g.apt #, street #, street name, city,county, state or province, zip or postal

By Tibault Dambrine

Anyone who has been througha soware package conversionknows that, even in the best ocases, these types o assignments

can be stressul. Tere are a number oreasons or this.

Te new package implementationis essentially a bet by managementthat a new, bigger, better, typicallymore complex, more sophisticated

packaged application will workbetter than the one currently beingused. Te new package will typicallycost more than the previous one, theconversion will entail a one-time

project budget, there is a lot of moneyat stake, and failure is not usuallyconsidered an option.

The Road to ConversionThe critical value of clean legacy system data

It is alwaysgood toremember thatsoware isreally about thebusiness rstand the people

who run it.Converting toa new soware

package alsomeans a changein process, in

the way thebusiness will goabout their day-to-day activities.Change isstressul, or theusers as muchas or the Ista.

From atechnical,purely soware

conversionpoint o view,there are

typically a number o unknowns atthe beginning o the project. Itemssuch as the state o the legacy systemdata, how the mapping will be done,and the conversion eort estimationare just a ew. All these will have tobe determined at the beginning othe project, typically with educatedguesses, more than hard data.

In this article, I will expose techniques andprinciples designed to minimize the stressinherent to most conversion projects. Teaim of these pre-conversion steps is toensure the conversion stage would start

with the best possible chances of success.While aiming to start a conversion projecton strong foundations sounds like commonsense, the tasks necessary to achieve thisstate of readiness are not necessarilyobvious or trivial.

7/29/2019 2008 07 the Road to Conversion

2/5

TORONTO USERS GROUP for Power Systems July 2008 9

code and country, you may want to ensure that the datais consistently placed in the same position within theree-orm elds. Tis way, you will be able to pick upinormation at predictable positions and save time.Ensure that there are as ew exceptions as possible in thedata i a rule should apply. For example, i in your legacytables, plant X item descriptions should start with X,let there not be any Y item numbers made available orthis plant. Simple exceptions such as these are enough tocause more work at conversion time.Ensure that the data is ree o duplicates. For example,look or duplicate descriptions, addresses, manuacturers

part numbers, names or telephone numbers. Tere aremany ways to look or duplicates and every one that isremoved will be a uture saving in the new system. Oneless piece o lint to worry about.Ensure that the data matches reality, by involving thebusiness is involved early on. Tey work with the data everyday. Tey will be the best people to be able to say whatmakes sense. I you tell them I have cleaned up the Bills oMaterials and only 300 out o 3000 will be converted, ask

the business: does this make sense? I your business analystthinks your fgure does not pass the smell test, you may

want to have a second look at your selection criteria.

First Tings First Create a Data Cleanup Project beforethe conversion can start. o achieve the expected benets andsavings anticipated at conversion time, a pre-conversion datacleanup project cannot be casual. Best practice would be to

write a ormal Statement o Work (SOW) identiying the goalsand rules that will guide the project activities. Management

will want to make a conscious eort to review, understand andapprove this SOW. Tey will also allocate a budget to nance

this eort and i possible, engage a project manager to runand monitor this task. Here are some things to look or in aStatement o Work...

Divide and Conquer:Identify your Data Objectso understand what data to clean and what changes to make,one must know something about both the origin and thedestination o the data. Te assumption here is that both oldand new soware systems will have similar unctions; e.g. anolder ERP system to a newer ERP system. Since the purposeo the new system is similar to the old one, but likely more

powerul. It may have similar data, but not necessarily arrangedor organized exactly the same way. Expect also that while thegeneral concepts may remain, the specic terminology in thenew system may be dierent rom the old one. Mapping theold le names, eld names, and sometimes even module namesearly on will ease the understanding.

A simple and eective way to match the data between the oldand the new systems in view o the cleanup and subsequentconversion is to use the concept o data objects.

Data Objects are logical groupings o related data sets.

An Example o these could be Item Master,which can actually be a set o tables, rather thana single table, covering generic items used acrossall plants, items used at specic plants only, itemsrepresenting labour as opposed to physical items.An other example could be Customer Care, aset o tables covering inormation on customerssuch as name and addresses, previous purchases,discount structures, memberships etc. Identiyingdata objects has some immediate benets:

It allows a division o labour. Te cleanupexercise can be divided along data objectboundaries. Each data cleanup sub-teamcan tackle a separate data object or data set.Responsibilities are clearly delimited.Data objects in the legacy system can bematched to data objects in the new system.For example the vendor master data objecton ERP Package A should be comparableor the one in ERP Package B, even i theirrespective physical implementations (data

structures) are dierent.Understanding the mapping o old to newdata system at the data structure level willenable the data cleaners to do more than justcleanup duplicates or obsolete data. Withthis understanding, the legacy data cleanupcan be used as an opportunity to optimizethe legacy data to acilitate the conversion.

The Big Rules and ScopeNow that the data objects are identied, the seconditem to attack is the Big Rules. Big Rules can be

likened to general guidelines that will help both thebusiness and the stakeholders to make decisions asto how the data will be either deemed obsolete (noneed to clean!) or how the cleanup needs to bedone. Here are some examples:

Te conversion scope will span data createdwithin the last 24 months.Any purchase order that can be closed shouldbe closed prior to the conversion. No closedPOs will be converted.Any vendor with no activity within the last12 months will not be converted.Any cleanup activity should be done by thebusiness, in the (legacy system) productionsystems wherever possible.

Cleanup Methods:the How and the WhereTe data cleanup, when done in the legacy system,can be done manually, with some measure oautomation (read programs) or a mixture o both,depending on the data set:

How: Manually. Clean up should be done

9

7/29/2019 2008 07 the Road to Conversion

3/5

TORONTO USERS GROUP for Power Systems July 200810

manually i the data volume is verysmall and all o the data to be cleanedup can be accessed by the businessusing their access screens, or i thedata is highly unstructured and has tobe eye-balled to make sense beorecleansingHow: Using sofware. Clean up withprograms i the data volume is largerthan what can be economically done byhand and simple rules can be applied.

Where: In Production. I the datais cleaned up in production, chancesare it will stay clean and will not haveto be cleaned again. Each new dataextraction rom production should becleaner than the previous oneandideally, data should not have to becleaned more than once.

Where: Using a staging area. In this situ-ation, the data is extracted rom the legacy

system into a staging database, cleansedand reormatted to be ready or the conver-sion outside o the legacy system. I thereare multiple practice conversions beorethe real deal, every practice will require are-run o the cleanup routine in staging.

Putting it all together, you will likely usea mixture of all of these methods, butkeep the following guidelines in mind:As much as possible, ensure manual or

labour-intensive cleanups are done inproduction. All repeatable, program-run type of cleanups can be on eitherproduction or staging , but being able toclean in production will (as a rule) savetime. Yes, I have written this once beforeit is worth repeating. Te business will notnecessarily like having extra work, but itremains worthwhile. Extract from thelegacy system, clean the data in a stagingdatabase, transform and export to the new

system OR clean the data in the legacysystem, then extract, transform and exportto the new system. It could also be acombination of both solutions, dependingon the data objects being converted.

A good rule of thumb is to decide earlywhich method would suit your conversionsituation best for each individual dataobject. o decide, a good test is to askis, Does this data have dependencies?

When changing or removing any existingdata within a complex system, one has tobe careful not to orphan any dataa

purchase order without vendor forexample. ypically, this type of situationis easier to monitor in the legacy systemthan when doing the cleanup outside ina staging database, which will not likelyhave all the same triggers and referentialintegrity structures as the legacy system.

A note on resource planning: Physicalresources: planning a staging area to dothe cleanup is no dierent than any othersystem. It will require disk space, processing

power and people to do the job. Te businesspeople already have jobs and you will hearthis more than once i you ask them to startcleaning and standardizing data. Te onlymitigating actor here is to plan early. Datacleanup does not come naturally and theramp-up is typically long.

Defne your CleanupApproachTe ollowing is a list o the dierentcomponents to consider in the approach tothe data cleanup:

1. Determine dataextraction criteria

Minimize as much as possiblethescope o data to be converted. Tis willaect every subsequent cleanup activity,as in make them less painul. Figure outcriteria that will help minimize the eort.Do not cleanup data rom divisions thatare not operational. Do not cleanup datathat is past a certain age. Any identiablecriteria you may nd and can validate withthe business to reduce your scope will help.

Standardize the dataor example, the

same description should be used to describethe same material even i it was in twodierent locations.

Ensure consistency. For ree-orm elds tobe converted to discrete columns, one othe biggest benets o a pre-conversion datacleanup is to make sure the data consistentlyollows strict rules. For example, i thelegacy system has our ree-orm linesor the address and the target system hasdiscrete elds or each part o the address,the cleanup opportunity is to ensure thateven in the ree-orm legacy system, each

part o the address is stored in a consistentposition. Tis will ease the conversionprocess by reducing the eort to take thedata rom the old system to the new. YourSQL conversion code can then pick upeach data element in a specic spot withouthaving to sh it out with a LOCAEoperation. For dependencies that may notbe enorced with database level constraints,ensure (or example) that there would be

The business must be involved early in the Big Rules decisions

7/29/2019 2008 07 the Road to Conversion

4/5

TORONTO USERS GROUP for Power Systems July 2008 11

no children without a parent (e.g. orderdetails without order headers). One o theside benets o good consistency is that itenables the spotting o duplicates, whichare undesirable.

Determine i the cleanup scope needs tospan the data coming via interaces. Inother words, should data provided by other,(external or internal) systems via interacesbe part o the cleanup scope? Note that itmay or may not be possible, especially ithat data comes rom outside the company.

Identiy and remove duplicate records.None o them should be le beore theactual conversion starts. Note that spottingduplicates can only be done well aer allthe previous steps are completed.

2. Close out completed

transactions

Ensure that housekeeping procedures (ithey exist) have been adhered to and areup-to-date. Examples are:

Veriy current min/max levels orstocked items.Close out POs linked to Work Orders

with status complete.Set all completed work ordersto status complete. Tis is anopportunity to easily identiy datathat should not be ported over in theconversion.

3. Reconcile fnancial /inventory balances:

For any item with a nancial balance, ensurethe balances reconcile between dierentledgers. For example:

Non-reconciled open items (GL, AR)Bank reconciliationReconciliation between accountingsystems and legacy systems

For any item with an inventorybalance, ensure that physicalinventory has been taken within anacceptable time period. Financialsare a particularly sensitive area.

A conversion will be deemedquestionable or even ailed ithe nancial picture is aectedsignicantly or reasons that aredifcult to explain. Tis part has to

work. Te company depends on it.

4. Complete missinginformation:

Identiy any data that have been createdwith incomplete, incorrect, or outdatedsettings or content. Te business will notstop running while the I department iscleaning and preparing the data or theconversion. For that reason, it is alsoimportant to ensure that as little as possible

new bad data is created.

o ensure this is the case, you may wish toupdate actual procedures to set up data in

the source system, and create vericationprocedures to monitor that the new dataentered conorms to the updated procedures,i.e. to ensure that the data remain consistent,complete, and correct.

7/29/2019 2008 07 the Road to Conversion

5/5

T

he5thWave,www

.the5thwave.com

TORONTO USERS GROUP for Power Systems July 200812

5. Set measurable effort andresult goals:In his original observation (early 1900s), theItalian economist Vilredo Pareto noticed that80% o his countrys land was owned by 20% othe population. Te Pareto principle, as it is nownamed, also known as the 80/20 rule, will applyin data cleanup assignments.

While working through data cleansing andmigration projects, there is a cost benetto achieve a balance between eort andaccomplishment. rue to the 80-20 rule,a high percentage, say, 80% o recordsto be migrated will likely require little or no modications.Te remaining 20 percent on the other hand may requirea combination o committee decision making, meetings,exception processing, extra system time and ultimatelysolutions requiring creative data manipulation.

When analyzing the 20% o the data which will requiremore eort, do not orget to understand what areas are

critical to cleanup. Tese should be done rst. Te benet omaximizing the return on eort in this area in particular iscritical. ypically, there is a nite conversion budget and it isimportant to use it wisely.

Engage the BusinessTe implementation o a new data processingpackaged system is above and beore all a BUSINESSDECISION not just an I decision. Te businessmust be involved early in the Big Rules decisions, inrecognizing what cleanup methods will be used, indeciding what is the scope o the data cleanup willbe, and in deciding who will be involved, rom theBusiness point o view as well as I. Implementing aRACI chart (Responsible, Accountable, Consulted,Inormed) early on will help dene who will need todo what during the project. For each cleanup activity,gure out who needs to be involved, what methodo cleanup will be used, and who will signo on the

cleaned-up data in eect, who will be the data owner.

Initiate a Data Cleaning Project Planand Stick to it!Cleaning data prior to a conversion is typically not a simple task.Tere are a number o tasks and a number o stakeholders andpriorities in the mix. It should be planned as a proper project. It is

desirable to assign a project manager to put the tasks in order andmonitor their progress to complete the task on time. Start Early. You

will never do enough to be completely ready. Unless your businessis bankrupt, you will constantly have the challenge o dealing withnew data coming in while cleaning up what you can see is worth

cleaning. Starting early will help.

In ConclusionIdle data is not good to keep in the best o circumstances. Itliterally uses disk space, backup time and resources, requiresattention, all the while adding little or no value. In the case oa conversion project, it actually adds negative value by costing

more to convert and oen orcing more exception processingto ensure the data goes over.

Soware conversions are typically complicated projects.Stephen Covey, in his rst two habits (out o 7) o highlysuccessul people, said it best: Habit Number 1: Be Proactive.Tis is what a pre-conversion data cleanup is all about! HabitNumber 2: Begin with the End in Mind. Never lose sight othe nal goal, which is a smooth, successul conversion. Withthat in mind, BEFORE the conversion: 1. Make every eortto reduce the data to convert (identiy old data, scope it out).2. Make every eort to ensure that the data to convert is clean,consistent, and complete. Te net eect o a pre-conversion datacleanup should be a much smoother, less stressul conversioneortone that would not give any surprises and that can usea minimum amount o rules. TG

Thibault Dambrine works forShell Canada Limited as a senior

systems analyst. He holds theITIL Foundations as well as the

Release and Control PractitionersCertifcates. His past articles can

be found at www.tylogix.com.