ARNOLD Challenges & FME

Bo Guo, PhD, PE, Gistic ResearchDave Campanas, Safe Software

April 5, 2016GIS-T 2016, Raleigh, NC

Workshop HostsBo Guo, PE, PhDGistic Research, Inc.- Developers of LinearBench products- Safe Software Partner

Dave CampanasSafe Software, Inc.

Attendee Introduction OrganizationUnitPositionGIS, IT or otherYears of FME experienceEver used FME for LRS?

Workshop OrganizationI. ARNOLD Challenges

Group discussion on technical challenges

II. FME ToolboxGroup discussion on priorities and use cases

III. Use Case Walk Through

Section I

ARNOLD & its Technical Challenges

FHWA-sponsored ARNOLD Study1. Project Schedule, Work Plan, Risk Assessment and

TFTN Crosswalk2. Local Road Collection Systematic Approach Report3. LRS Components and Best Practice Report 4. LRS Temporal Maintenance Plan Report 5. LRS Technical Instructions, Rules and Diagrams

Report 6. Reference Manual

Technical Challenges“Linear referencing systems are among the most important and complex datasets within a DOT.”

“Development and maintenance of a statewide, all roads network containing LRS is an involved and complex process.”

“ARNOLD amplifies the challenges because of its scope.”

ARNOLD Study Recommendations Shared, enterprise-wide LRS

Adaptable (requirements will change)

Change management

Flexible and scalable

ARNOLD Deliverables

Column NotesRoute_ID Unique road ID number

Road NameFunctional Classification (7 classifications)

Ownership (27 ownership types)Facility Type (7 types)State Code

Year_RecordSource Entity providing the data

GeometryWKB using (x,y,m), w/ measures in miles to 1/1000 of a mile(Missing CS)

All-roads with LRS in the ARNOLD schema:

ARNOLD Technical Process

Group Discussion I Topics Identify common technical challenges

Describe current technical solution, if any

Rank the technical challenges by priority

Rank the technical challenges by difficulty

Group Discussion I Setting 3 groups, each with a Lead

Group discussion document

24 minutes of group discussion

2 minutes for group presentation

Section II FME & its LRS Toolbox

An Introduction to

FME Desktop

FME Server- and -

Safe Software3 Core Products

FME Desktop

FME Server

FME Cloud

FME EngineAll Safe products share a common platform

Powered by FME Engine……driven by FME Workspaces

• Choice of operating platform:• Windows• Linux• MAC OS

• 32 or 64 bit FME

CONNECT HUNDREDS OF FORMATS WITH FME

INTEROPERABILITY

TRANSFORMATION- is all about -

Connect to your format

Transform the data model 400+ data

transformation tools in FME

- and -

DATA MODEL TRANSFORMATION

TRANSFORMER

WORKBENCH

• Everything FME starts here

• A graphical authoring environment used to create repeatable workflows

• Author FME workflows with a library of 400+ powerful data transformers

Inspect data structure - before, during, and after transformation.

DATA INSPECTOR

FME Workbench

The FME Engine enabler…

…exposes the power of the FME Engine

Author and run FME Workspaces using FME Workbench……part of FME Desktop

FME ServerFME Engine workhorse and runs FME Workspaces

Automate Deliver Data Download Data

Love

FME!

ScaleReal Time Data

FME Cloud

FME Server- is -

S A F E . C O M / F M E C L O U D

… with more FME Engines driven by FME Workspaces

Introduction: Linear Referencing Systems

The fence is referenced to Hwy 6, from 2.0 km to 2.8 km

Linear referencing:Associates attributes (events) to locations (station points) or portions of a linear feature (Network Linear Element or Route)

Pipelines

Trains

Automobiles

Knut JetlundStatens vegvesen

Common LRS Tasks▪ Cardinal Orientation▪ Creating Network Linear Elements (NLE) or

Routes▪ Adding Measures to NLE▪ Creating Event Tables from Geometry▪ Asset Management – Stationing Points▪ NLE / Events ➔ Segments

▪ Dynamic Segmentation▪ Segments ➔ NLE / Events▪ Back / Ahead Measures

FME as an LRS Toolkit▪ LRS Transformer Category

▪ Snipper▪ LengthToPointCalculator▪ MeasureGenerator▪ MeasureExtractor▪ MeasureSetter▪ MeasureRemover

▪ Ad hoc Transformers▪ LineJoiner

creates Paths▪ NeighborFinder▪ LineOnLineOverlayer▪ PathBuilder▪ PathSplitter

▪ Data Joins▪ FeatureMerger▪ InlineQuerier▪ SQLExecutor▪ Joiner

Prerequisites for LRS Success

▪ Clean Data▪ Even Cleaner Data

▪ Segments are noded and connect▪ Complex edges are split at intersections

▪ Handy Transformers for Clean-up▪ Snapper▪ Intersector▪ TopologyBuilder▪ NetworkTopologyCalculator

What FME Is and Isn’tData Functions Capability Notes

ETL Yes Supports 400+ formats

Projection Yes 2000+ predefined CS

QC Yes

Batch CRUD Yes Can write SQL as well

Interactive Editing No

Analysis Yes

Services Yes Through FME Server

Presentation No Inspection viewer

FME ...Geospatial Data ETL Tool

ModelBuilder on SteroidsCrosses different geospatial ecosystemsVersatile, Efficient, Elegant!

Knows LRS & ARNOLD!

Software RequirementsTask Categories Task FME

Centerline Maintenance

Interactive editing No/YesData import/export YesETL YesConflation Yes/No

LRS MaintenanceCalibrating the LRM YesApplying an LRM YesEvents handling Yes

LRS Data Sharing

Ability to publish web services Yes

Programmatic access to LRS via APIs Yes

Download of LRS information Yes

Centerline MaintenanceOperation Type Task Description Mode

Edit Adding new line segment InteractiveEdit Modifying line segment InteractiveEdit Populating line attributes InteractiveETL Loading jurisdictional data BatchETL Connecting demarcation points BatchETL Standardizing schema BatchETL Conflating BatchQC Topological BatchQC Attribute BatchQC Geometry Batch

LRS MaintenanceOperation Type Task Description Mode

Edit Building routes interactively Interactive

ETL Bulk-building routes Batch

ETL Calibrating routes Batch

ETL Scaling and shifting routes Batch

ETL Supporting MP equation Batch

QC Route measure consistencies Batch

QC Other QC rules Batch

QC Route direction consistencies Batch

Event MaintenanceOperation Type Task Description Mode

Edit Event editing InteractiveETL Propagating measure changes BatchETL Geocoding BatchETL Reverse geocoding BatchETL Event join BatchETL Event merge BatchQC Domain rule BatchQC Gap rule BatchQC Overlap rule Batch

ARNOLD Reporting

Operation Type Task Description Mode

ETL Schema mapping Batch

ETL Converting to WKB Batch

Group Discussion II Topics

Identify use cases for FME walk through

Group Discussion II Setting 3 groups, each with a Lead

Group discussion document

15 minutes of group discussion

Section III Demo and Walk-through of Selected Use Cases