ARNOLD Challenges & FME
Bo Guo, PhD, PE, Gistic ResearchDave Campanas, Safe Software
April 5, 2016GIS-T 2016, Raleigh, NC
Workshop HostsBo Guo, PE, PhDGistic Research, Inc.- Developers of LinearBench products- Safe Software Partner
Dave CampanasSafe Software, Inc.
Attendee Introduction OrganizationUnitPositionGIS, IT or otherYears of FME experienceEver used FME for LRS?
Workshop OrganizationI. ARNOLD Challenges
Group discussion on technical challenges
II. FME ToolboxGroup discussion on priorities and use cases
III. Use Case Walk Through
FHWA-sponsored ARNOLD Study1. Project Schedule, Work Plan, Risk Assessment and
TFTN Crosswalk2. Local Road Collection Systematic Approach Report3. LRS Components and Best Practice Report 4. LRS Temporal Maintenance Plan Report 5. LRS Technical Instructions, Rules and Diagrams
Report 6. Reference Manual
Technical Challenges“Linear referencing systems are among the most important and complex datasets within a DOT.”
“Development and maintenance of a statewide, all roads network containing LRS is an involved and complex process.”
“ARNOLD amplifies the challenges because of its scope.”
ARNOLD Study Recommendations Shared, enterprise-wide LRS
Adaptable (requirements will change)
Change management
Flexible and scalable
ARNOLD Deliverables
Column NotesRoute_ID Unique road ID number
Road NameFunctional Classification (7 classifications)
Ownership (27 ownership types)Facility Type (7 types)State Code
Year_RecordSource Entity providing the data
GeometryWKB using (x,y,m), w/ measures in miles to 1/1000 of a mile(Missing CS)
All-roads with LRS in the ARNOLD schema:
Group Discussion I Topics Identify common technical challenges
Describe current technical solution, if any
Rank the technical challenges by priority
Rank the technical challenges by difficulty
Group Discussion I Setting 3 groups, each with a Lead
Group discussion document
24 minutes of group discussion
2 minutes for group presentation
FME EngineAll Safe products share a common platform
Powered by FME Engine……driven by FME Workspaces
• Choice of operating platform:• Windows• Linux• MAC OS
• 32 or 64 bit FME
INTEROPERABILITY
TRANSFORMATION- is all about -
Connect to your format
Transform the data model 400+ data
transformation tools in FME
- and -
DATA MODEL TRANSFORMATION
TRANSFORMER
WORKBENCH
• Everything FME starts here
• A graphical authoring environment used to create repeatable workflows
• Author FME workflows with a library of 400+ powerful data transformers
FME Workbench
The FME Engine enabler…
…exposes the power of the FME Engine
Author and run FME Workspaces using FME Workbench……part of FME Desktop
FME ServerFME Engine workhorse and runs FME Workspaces
Automate Deliver Data Download Data
Love
FME!
ScaleReal Time Data
FME Cloud
FME Server- is -
S A F E . C O M / F M E C L O U D
… with more FME Engines driven by FME Workspaces
Introduction: Linear Referencing Systems
The fence is referenced to Hwy 6, from 2.0 km to 2.8 km
Linear referencing:Associates attributes (events) to locations (station points) or portions of a linear feature (Network Linear Element or Route)
Pipelines
Trains
Automobiles
Knut JetlundStatens vegvesen
Common LRS Tasks▪ Cardinal Orientation▪ Creating Network Linear Elements (NLE) or
Routes▪ Adding Measures to NLE▪ Creating Event Tables from Geometry▪ Asset Management – Stationing Points▪ NLE / Events ➔ Segments
▪ Dynamic Segmentation▪ Segments ➔ NLE / Events▪ Back / Ahead Measures
FME as an LRS Toolkit▪ LRS Transformer Category
▪ Snipper▪ LengthToPointCalculator▪ MeasureGenerator▪ MeasureExtractor▪ MeasureSetter▪ MeasureRemover
▪ Ad hoc Transformers▪ LineJoiner
creates Paths▪ NeighborFinder▪ LineOnLineOverlayer▪ PathBuilder▪ PathSplitter
▪ Data Joins▪ FeatureMerger▪ InlineQuerier▪ SQLExecutor▪ Joiner
Prerequisites for LRS Success
▪ Clean Data▪ Even Cleaner Data
▪ Segments are noded and connect▪ Complex edges are split at intersections
▪ Handy Transformers for Clean-up▪ Snapper▪ Intersector▪ TopologyBuilder▪ NetworkTopologyCalculator
What FME Is and Isn’tData Functions Capability Notes
ETL Yes Supports 400+ formats
Projection Yes 2000+ predefined CS
QC Yes
Batch CRUD Yes Can write SQL as well
Interactive Editing No
Analysis Yes
Services Yes Through FME Server
Presentation No Inspection viewer
FME ...Geospatial Data ETL Tool
ModelBuilder on SteroidsCrosses different geospatial ecosystemsVersatile, Efficient, Elegant!
Knows LRS & ARNOLD!
Software RequirementsTask Categories Task FME
Centerline Maintenance
Interactive editing No/YesData import/export YesETL YesConflation Yes/No
LRS MaintenanceCalibrating the LRM YesApplying an LRM YesEvents handling Yes
LRS Data Sharing
Ability to publish web services Yes
Programmatic access to LRS via APIs Yes
Download of LRS information Yes
Centerline MaintenanceOperation Type Task Description Mode
Edit Adding new line segment InteractiveEdit Modifying line segment InteractiveEdit Populating line attributes InteractiveETL Loading jurisdictional data BatchETL Connecting demarcation points BatchETL Standardizing schema BatchETL Conflating BatchQC Topological BatchQC Attribute BatchQC Geometry Batch
LRS MaintenanceOperation Type Task Description Mode
Edit Building routes interactively Interactive
ETL Bulk-building routes Batch
ETL Calibrating routes Batch
ETL Scaling and shifting routes Batch
ETL Supporting MP equation Batch
QC Route measure consistencies Batch
QC Other QC rules Batch
QC Route direction consistencies Batch
Event MaintenanceOperation Type Task Description Mode
Edit Event editing InteractiveETL Propagating measure changes BatchETL Geocoding BatchETL Reverse geocoding BatchETL Event join BatchETL Event merge BatchQC Domain rule BatchQC Gap rule BatchQC Overlap rule Batch
ARNOLD Reporting
Operation Type Task Description Mode
ETL Schema mapping Batch
ETL Converting to WKB Batch
Group Discussion II Setting 3 groups, each with a Lead
Group discussion document
15 minutes of group discussion