FME data migration and Q&A strategies

Ulf MånssonFME Certified Professional/Trainer

Sept 2012

2Ulf Månsson

!!!NERD WARNING!!!

FME-code have been excavated from 1997-1998

FME Innovator Award 2006 Vancouver

FME Certified Professional 2007

FME Idol Final 2008 Vancouver

FME Certfied Trainer 2009

Ulf Månsson

INSPECTION AT EVERY STAGE

DESTINATIONMAGICSOURCE

132

Example 1E.ON Map Base

DESTINATION

MAGIC

SOURCE

ORACLE & DGN

FORMATSCHEMA

REPROJECTIONSYMBOLS

QA

DWG/DXFDGN

2004-04-01/PEALUlf Månsson

Example 1 - E.ON Map BaseMap data / map information for more than 100 municipalities has been

• handled,• analysed,• controlled,• reworked and• archived.

Data has normally been• metadata,• map data, • transformation parameters, • end products in DGN-format and Oracle Spatial• overviews, reports and other data

Ulf Månsson

DESTINATION SOURCE Inspection

Tables for schemasDESTINATION SOURCE

MAGIC

Schema mapping QA

Restructuring

Example 1E.ON Map Base

DESTINATION

MAGIC

SOURCE

ORACLE & DGN

FORMATSCHEMA

REPROJECTIONSYMBOLS

QA

DWG/DXFDGN

Example 2City of Malmö

TILED DWG

FORMATSCHEMA

REPROJECTIONSYMBOLS

QA/CLIPPING

ArcSDE

MAGICSchema mapping

QARestructuring

Example 2 – City of Malmö

MAGIC of batching

So many ways:

-Wildcard reading ”fan in out”-Database transformers-Workspacerunner driven by ”Directory and file path reader”-FME Server Job submitters-Core FME Server functionality

-Commandline? .BAT files-FME Objects? .NET etc

Example 1E.ON Map Base

DESTINATION

MAGIC

SOURCE

ORACLE & DGN

FORMATSCHEMA

REPROJECTIONSYMBOLS

QA

DWG/DXFDGN

Example 2City of Malmö

TILED DWG

FORMATSCHEMA

REPROJECTIONSYMBOLS

QA/CLIPPING

ArcSDE

Example 3QA in Utility

ORACLEEXCEL

QALOGGING

CANDIDATES

ORACLE

The mercyless ConvexHullAccumulator exposes bad quality.

Defining municipality by zip-code from CIS…

Example 3 – QA in utilityMAGIC

Validering

Ulf Månsson

ConvexHullAccumulator - Good Data

Example 1E.ON Map Base

DESTINATION

MAGIC

SOURCE

ORACLE & DGN

FORMATSCHEMA

REPROJECTIONSYMBOLS

QA

DWG/DXFDGN

Example 2City of Malmö

TILED DWG

FORMATSCHEMA

REPROJECTIONSYMBOLS

QA/CLIPPING

ArcSDE

Example 3QA in Utility

ORACLEEXCEL

QALOGGING

CANDIDATES

ORACLE

Example 4Södra forestry

SHP

QALOGGING

GENERALIZING

ArcSDE

2004-04-01/PEALUlf Månsson

•We are handling appr. 8-12 Gb vector data, in ESRI shape format. Map data are stored in an ArcSDE database which provides our clients business systems with map data through web services.

•FME Workbench is used to optimize map data and thus enhance map performance dramatically. The workbench files are also used as documentation of our working process and to keep track on which data have been used, which data have been created and for what purpose!

Maintaining and optimising data for a nation-wide ArcSDE database

2004-04-01/PEALUlf Månsson

 •Optimisation of map data is performed considering which information is relevant and map performance at pre-defined zoom scale levels. 

Maintaining and optimising data for a nation-wide ArcSDE database

21

SWECO EnvironmentB. Almström, O. Persson

The FME approach is the same !

►ulf.mansson@sweco.se

22

Nya lager i ArcSDE SQLServer=> Nya lager i Oracle Spatial

Why do so many live in Origo???

”You should at least have used the

2DPointReplacer before deployment!”

Thank You!

Questions?

For more information: Ulf.Mansson@sweco.se