Esri UC 2014 | Technical Workshop |
Data Interoperability: An IntroductionBruce Harold [email protected]
Dale Lutz Safe Software
Esri UC 2014 | Technical Workshop |Esri UC 2014 | Technical Workshop |
Why Data Interoperability?
Background
ArcGIS + FME
Esri UC 2014 | Technical Workshop |
What is the Data Interoperability Extension?
• Connect to Formats- 200+ added file formats
- Feeds & Web sources
- Custom formats
• Quick Translation- Any format to File GDB
- Any format to another Format
• Transformation- Schema transformation
- Geometry transformation
Connects your World of Apps
File, Photo, Parcel, BIM, MS Office, DBMS, CSV, Network, Military, Survey, Sensor, GPS…
Esri UC 2014 | Technical Workshop |
ArcGIS Data Interop Common Tasks
• Integration
• Migration
• Cleanup
• Validation
• Distribution
• Merging Changes
Connected workflows
Spatial ETL
Esri UC 2014 | Technical Workshop |
Data Interoperability DifferentiatorsFor Data Triathletes
• Feature by Feature Processing
• Inline SQL across all your data
• Data Inspection
• Deep format & standards support
• XML Documents as Features or Attributes
• JSON Documents as Features or Attributes
• Protocol Support (FTP, HTTP, RSS…)
• URL and Compressed Inputs
• Flow Control & Parallel Processing
• No Coding
Esri UC 2014 | Technical Workshop |Esri UC 2014 | Technical Workshop |
What you get just by installing
Basic OGC Support
Esri UC 2014 | Technical Workshop |
WFS and GML
• GML- Simple Features Profile
• WFS in Data Interoperability- Cache lives 24 hours by default
- Numeric Identifier forces live access
• You must install the extension
Supported without Licensing
OGC GML & WFS suitable for Geoprocessing
Type Presentation Title Here
Esri UC 2014 | Technical Workshop |Esri UC 2014 | Technical Workshop |
Data Interoperability as an OGC client
OGC GML & WFS
Demo
Data Interoperability - An Introduction
Esri UC 2014 | Technical Workshop |
Other OGC Stuff
• KML- Schema object preserved
- KML Properties
- KML Styling
• CityGML- CityEngine Tools coming
- To & From File GDB
But wait there’s more
OGC Formats in Data Interoperability
Type Presentation Title Here
Esri UC 2014 | Technical Workshop |Esri UC 2014 | Technical Workshop |
Ways to read any format data
Using Formats
Esri UC 2014 | Technical Workshop |
Direct Read/Write From Disk
• Catalog browse tree- No need to add file type
- Feature Dataset icon
- Available feature types shown
- Format defaults used
• As geoprocessing input- Tools
- Models
- Scripts (enable the extension!)
File Based – recognized by extension
Recognized formats ‘just work’
Type Presentation Title Here
Esri UC 2014 | Technical Workshop |
Interoperability Connections
• When default format connection is:- Infeasible, e.g. authentication required
- Undesirable, e.g. optional parameter wanted
• Authored in Catalog tree
• Off-disk feeds & web sources
• May view source data prior to connecting
• Not cached, read when used
• Coordinate system may be selected
• May be shared (.fdl file)
Manually set up and persistent
Yes, Excel can be read as Point data!
Type Presentation Title Here
Esri UC 2014 | Technical Workshop |
Selecting Formats
• Browse
• Search
• Customize
Or, ‘What was that “FME Reader Gallery” thing in the previous slide?’
Don’t see what you’re looking for? Ask us!
Type Presentation Title Here
Esri UC 2014 | Technical Workshop |Esri UC 2014 | Technical Workshop |
Using any format in ArcGIS for Desktop
Direct Read & Connections
Demo
Data Interoperability - An Introduction
Esri UC 2014 | Technical Workshop |Esri UC 2014 | Technical Workshop |
Quick Import & Quick Export GP Tools
Quick Translation
Esri UC 2014 | Technical Workshop |
Quick Import & Quick Export
• Quick Export- To Any Format
• Quick Import- To File Geodatabase
Geoprocessing tools in Data Interoperability
Rapid format conversion using format defaults
Type Presentation Title Here
Esri UC 2014 | Technical Workshop |
Automate Quick Translations
• Easy way to ship multiple formats
• Retain your GIS schema
Use the Quick tools in ModelBuilder
Supply popular formats with GIS-friendly properties
Type Presentation Title Here
Esri UC 2014 | Technical Workshop |Esri UC 2014 | Technical Workshop |
Automated Format Supply
Quick Tools
Demo
Data Interoperability - An Introduction
Esri UC 2014 | Technical Workshop |Esri UC 2014 | Technical Workshop |
Custom Formats
Leveraging Formats
Esri UC 2014 | Technical Workshop |
What if you need your own format?
• Based on a known format
• Build format definition workflow once
• Remains available in the gallery
Save time by “canning” a custom format workflow
You can share a custom format with other Data Interoperability users
Based on JSONBased on JSON
Esri UC 2014 | Technical Workshop |Esri UC 2014 | Technical Workshop |
JSON read from the web
Custom Formats
Demo
Data Interoperability - An Introduction
Esri UC 2014 | Technical Workshop |Esri UC 2014 | Technical Workshop |
Spatial ETL
The Power of Transformation
Esri UC 2014 | Technical Workshop |
The Full Power of Spatial ETL
• Format reading and writing is only a beginning- Information has no ‘format’
- Formats are only languages; many have dialects
- Formats merely facilitate information (or wreck it)
• Information is often hidden or requires business logic
• Transformation reveals and derives information- Calculate stuff, mess with words, ignore junk
- Use relational, dictionary, list & network structures
- Way, way more…
• Stop. Writing. Code. (Unless you find it fun)
Connect. Transform. Automate.
Transformation is the real power of Data Interoperability
Esri UC 2014 | Technical Workshop |
Workbench Application
• Create via context menu
• Follow the wizard
Graphically Design Data Flows
The Workbench application authors Spatial ETL tools
Esri UC 2014 | Technical Workshop |
Workbench Authoring
• Read
• Transform- Attributes
- Geometry
- Style
• Write
• Run standalone
• Run as tool
Design & Document
Workbench has a canvas and supporting windows
Type Presentation Title Here
Esri UC 2014 | Technical Workshop |
Transformers
• Eponymous- Named after their function
• Categorized
• Browse
• Search
• Popup
Hundreds of functions at your fingertips
Drag or click transformers onto the canvas or use the popup functionality
Type Presentation Title Here
Esri UC 2014 | Technical Workshop |Esri UC 2014 | Technical Workshop |
Connect. Transform. Automate.
ETL
Demo
Data Interoperability - An Introduction
Esri UC 2014 | Technical Workshop |Esri UC 2014 | Technical Workshop |
A few more observations…
Wrapping Up
Esri UC 2014 | Technical Workshop |
Data Interoperability in ArcGIS for Server
• Map services can use supported formats- 64 bit only
• Spatial ETL tools must be in Models- Publish service as usual from Result
• Good Zip & Ship solution- Generic Writer into ZIP file
- Output format made parametric
Considerations
Data Interoperability is available in ArcGIS for Server
Type Presentation Title Here
Esri UC 2014 | Technical Workshop |
Coming in 10.3
• FME 2014 SP 2 Engine
• Write images & hyperlinks to Excel (maps!)
• Data Interoperability for ArcGIS Pro- 64 bit
• More…
And it’s service packs…
Data Interoperability will be released for ArcGIS Pro
Type Presentation Title Here
Esri UC 2014 | Technical Workshop |
Learn More
• At the User Conference- Esri Showcase Tuesday 9am until Thursday 1:30pm
- After this presentation
• 60 Day Software Evaluation- http://www.esri.com/software/arcgis/extensions/datainteroperability/evaluate
• Training- http://training.esri.com/gateway/index.cfm?
fa=search.results&searchterm=data+interoperability
Data Interoperability
Talk to us now!
Type Presentation Title Here
Esri UC 2014 | Technical Workshop |
Thank You! Now its Question Time!
• Please fill out the session survey:
First Offering ID: 1221
Second Offering ID: 1402
Online – www.esri.com/ucsessionsurveys
Paper – pick up and put in drop box
Data Interoperability – An Introduction
Have a great conference!
Type Presentation Title Here
Esri UC 2014 | Technical Workshop | Type Presentation Title Here