Post on 08-Jan-2017
transcript
AUTOMATING REGIONAL DATA INTEGRATION WITH PYTHON & ARCPY
Heather WidlundGIS Coordinator, San Miguel County
West Region Data Integration Group
Integrated dataStandardized schemasoStreet centerlines
oAddress/structure points
oAddressed places/landmarks
Other: ESNs, PSAP boundaries, Parcels
Emergency preparedness & response focus oVerify emergency notification addresses (CodeRED)
oVisual mapping in multi-agency dispatch
o911 call address verification (Spillman CAD)
4/29/2016GIS COLORADO | SPRING 2016 MEETING | BRECKENRIDGE, COLORADO
4/29/2016GIS COLORADO | SPRING 2016 MEETING | BRECKENRIDGE, COLORADO
Data updates are arduous.What is the problem?
Why is this a problem?
Data updates are infrequent.
What are the consequences?
What is the solution?
4/29/2016GIS COLORADO | SPRING 2016 MEETING | BRECKENRIDGE, COLORADO
>>Make a machine do the work:
. . .
Automation ProjectPhases
1) Source data with valid schema2) Ability to integrate easily & automatically3) Data updates pushed to Spillman CAD
4/29/2016GIS COLORADO | SPRING 2016 MEETING | BRECKENRIDGE, COLORADO
Script tools, Python & arcpy libraryoFamiliar, available
oNo extra installs/libraries
Ground-up: hosted at each agencyoPros: data ownership & responsibility, specific needs
oCons: keeping scripts in sync, specific needs
ImplementationSet up environment
Run script tools to create configuration filesoEsri’s Community Addresses Solution
Configure batch file
Run batch fileoManually
oWindows Task Scheduler
4/29/2016GIS COLORADO | SPRING 2016 MEETING | BRECKENRIDGE, COLORADO
4/29/2016GIS COLORADO | SPRING 2016 MEETING | BRECKENRIDGE, COLORADO
replacedata.pybatch.bat runupdate.py
Validate fieldsMap fieldsExtract dataDrop old dataAppend new dataUnique IDsCalc Lat/LongStrip metadata
Schematic
Lessons learned (so far)Test, test, test: real data & real environments
Customized agency scripts
FunctionsoGenerate a result (e.g. T/F)
oKeep them short & re-useable
ToolsoIDE: PyScripter
oLogging module
oThe internet – document your sources
4/29/2016GIS COLORADO | SPRING 2016 MEETING | BRECKENRIDGE, COLORADO
ProgressInstallations to date
Next stepsoScript integration
oScript data push to Spillman CAD
GIS in the Rockies
4/29/2016GIS COLORADO | SPRING 2016 MEETING | BRECKENRIDGE, COLORADO