Post on 31-Jan-2020
transcript
Esri UC 2014 | Technical Workshop |
ArcGIS for Server: Administrative Scripting and Automation Shreyas Shinde Ranjit Iyer
Esri UC 2014 | Technical Workshop |
• Introduction to server administration • Command line tools • ArcGIS Server Manager • Overview of the administrative API • Automation • Popular scripting workflows
- Publishing of services - Setting up permissions - Extracting popular extents - Monitoring services and databases - Scaling and statistics
Agenda
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop |
Introduction to server administration
ArcGIS for Server: Administrative Scripting and Automation
ArcGIS Server Installation Media Provision hardware Select a deployment pattern
ArcGIS Server Setup Software Authorization
Keep it running…
Esri UC 2014 | Technical Workshop |
The administrator’s toolbox
ArcGIS for Server: Administrative Scripting and Automation
ArcGIS Server Manager
ArcGIS Server Administrator Directory
Console tools
Esri UC 2014 | Technical Workshop |
• Located under /arcgis/server/tools • A collection of most popular tools • Categories
- Administrative credentials - For resetting administrative accounts
- SSL/certificate management - Backup and restore - Managing cache
• Lock the tools down using OS file permissions!
Command line tools
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop | Esri UC 2014 | Technical Workshop |
Command line tools
Demo
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop |
• Browser based interactive application • Located at: http://server.domain.com:6080/arcgis/manager • Simple, easy-to-use • Ability to manage your entire server site
- Even though you are accessing Manager on a specific server node
• Great for common workflows!
ArcGIS Server Manager
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop | Esri UC 2014 | Technical Workshop |
le Tour de Manager
Demo
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop |
ArcGIS Server Manager – under the hood
ArcGIS for Server: Administrative Scripting and Automation
Simple HTTP requests to the administrative API
Esri UC 2014 | Technical Workshop |
• Entire server administration is an API - If ArcGIS Server Manager can use it, so can you!
• RESTful - GET and POST of JSON data
• Well organized and well documented • Lots of samples • Scriptable • Designed for automation!
Overview of the administrative API
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop |
The administrative API
ArcGIS for Server: Administrative Scripting and Automation
Admin API
Services
Create and manage your services. Start or stop them as wanted
Machines +
Clusters
Add new nodes to grow capacity. Remove them to shrink. Organize machines into clusters.
Security
Configure GIS tier or web tier authentication. Manage accounts, roles and permissions on services. Configure SSL.
Statistics (new at 10.3)
Get fine grained statistics on your GIS services.
Logs
Harvest logs for auditing, understanding usage patterns and troubleshooting.
Databases Manage all your database connections through a single interface.
System environment
Manage system properties, web adaptors, server directories.
Esri UC 2014 | Technical Workshop |
• Common repeated workflows - Plot popular extents for the data
• Scheduled actions - Restarting services when data is updated - Build map caches during low demand by moving machines to caching cluster
• Reactive actions - Adding capacity (server nodes) to a cluster when demand increases - Send email notifications when errors are logged
• Works with all programming/scripting languages that support HTTP!
Why automate?
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop |
Where should you begin?
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop | Esri UC 2014 | Technical Workshop |
Publish Services
Demo
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop |
• Publishing many services • Moving services between Dev, Staging, Production environments
- Backup/restore is an option but sometimes you just want to republish services
• Can publish from Desktop and Manager • Tedious if you have lots of services • Automation is your friend!
Motivation
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop |
• Help system has a script that uses ArcPy • ArcPy is an python library that complements the Admin REST API • Provides many convenience functions
Task : Publish a list of services listed in a file
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop |
• Works great, just need a few lines of code • However, you need ArcPy on the machine • Script publishes service definitions sequentially
- For every SD - Upload - Publish
• With many SDs to publish this can be a very slow process • Publishing ~100 services took hours
Caveats
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop |
• Run the script from any machine - No ArcPy dependency
• Exploit concurrency
- Parallel uploads - Parallel publishing
• If your script is amenable to concurrency, exploit it by all means
We can do better
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop |
• Two step process
- Upload the SD to the Server - QUICK
- Invoke the publishing tool - INTENSIVE
Publishing a service with an SD
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop |
Script design
ArcGIS for Server: Administrative Scripting and Automation
Upload
Upload
Upload
Publish
Publish
SD SD SD UpldID UpldID UpldID SD
Esri UC 2014 | Technical Workshop |
Comparison results
ArcGIS for Server: Administrative Scripting and Automation
Speed up of ~3
ArcPy publishing script DIY multi-threaded publishing script
Esri UC 2014 | Technical Workshop | Esri UC 2014 | Technical Workshop |
Starting a service
Demo
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop |
• Manager
• Admin
• Script
Options
ArcGIS for Server: Administrative Scripting and Automation
Start Geometry Service
Esri UC 2014 | Technical Workshop |
• Main function - Drives the logic
• Internal utility functions
- Get Token - Make HTTP requests - Parse JSON responses
Anatomy of an Admin script
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop | Esri UC 2014 | Technical Workshop |
Apply Permissions
Demo
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop |
• Permissions are the way to secure your service • Do it in Manager
• Tedious for lots of services. Must automate!
Task : Apply permissions to services
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop |
• Show them previously published services in Manager • Show them roles • Run the script • Discuss the script
Demo
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop | Esri UC 2014 | Technical Workshop |
Health check for services
Demo
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop | ArcGIS for Server: Administrative Scripting and Automation
Get all the folders Using the report API, get all the service
instances.
Using Geo-services REST API to test
availability.
Print report
Esri UC 2014 | Technical Workshop | Esri UC 2014 | Technical Workshop |
Health check for databases
Demo
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop | ArcGIS for Server: Administrative Scripting and Automation
Use findItems to search for all
enterprise geodatabases Use validateDataItem
to test database connectivity.
Print output
Esri UC 2014 | Technical Workshop | Esri UC 2014 | Technical Workshop |
Popular extents
Demo
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop | ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop | Esri UC 2014 | Technical Workshop |
Elastic scaling
Demo
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop |
• Add machines to reduce latency and increase throughput • Dynamically add them when a latency threshold is breached • De-allocate machines when traffic is less
Task : Dynamically add machines to your site
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop |
• Start with a single node site • Make service requests and watch latency (we’ll use JMeter) • Run a script that adds a machine to a cluster • Watch the latency drop
Demo
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop |
Statistics in 10.3 (Sneak peak)
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop |
• Server is always collecting statistics - Avg response - Number of requests - Number of timeouts - . . .
• Report defines the resource, metric and interval - Previous day, last 7 days, last 30 days - Custom time window
• Server chooses intelligent data point intervals in the reports • REST API to harvest all this information
Statistics at 10.3
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop |
Python is great!!
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop |
• Most modern languages (like Python) support Threads • Some languages provide concurrency as a language feature
- Go - Erlang - Clojure - Scala
• Suited for concurrent programming
Short detour…
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop |
• Invented at Google • About concurrency • Lightweight threads (goroutines) • Typed thread-safe communication and synchronization (channels) • Readable concurrent code
Go
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop |
• Invented at Ericsson • Functional language • Light weight ‘processes’ • Can spawn 100s of thousands of them • Concurrency via message passing • Actors model
Erlang
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop |
Python is great!!
ArcGIS for Server: Administrative Scripting and Automation
Don’t get us in trouble, we don’t want to start a rebellion J
Esri UC 2014 | Technical Workshop |
Thank you…
• Please fill out the session survey:
First Offering ID: 1146 Second Offering ID: 1344
Online – www.esri.com/ucsessionsurveys Paper – pick up and put in drop box
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop |
Questions?
ArcGIS for Server: Administrative Scripting and Automation
Esri UC 2014 | Technical Workshop |
Resources
ArcGIS for Server: Administrative Scripting and Automation
GIS Tutorial for Python Scripting Esri Press, 2014
Paperback and e-book Just released! Offers several hands-on tutorial exercises.
Python Scripting for ArcGIS Esri Press, 2013
Paperback and e-book Good reference text
Esri UC 2014 | Technical Workshop | ArcGIS for Server: Administrative Scripting and Automation