Raster Serving with ArcGIS
Katja Krivoruchko
Agenda
• Introduction to server technology• Workflow considerations• ESRI raster serving technology
– ArcGIS Server Basic (ArcSDE)– ArcGIS Server Standard/Advanced Web Services– ArcGIS Server Image Extension– File Server– ArcGIS Online
• Software demonstration• Implementation scenarios• Steps to success• Resources
Server Technology
• Enables storage and serving of data from central servers• Supports fast and efficient visualization and analytics
– increases network efficiency– enables simultaneous multiuser access
• Enables sharing of data, maps, models throughout organizations
ArcGIS Server
What Does “Raster Serving” Mean To Me?
• Making rasters available to GIS analysts in ArcMap– As background imagery– For analytical purposes (i.e., feature extraction)
• Serving elevation data to replace default layers in ArcGIS Explorer• Making rasters available to many different client applications
– AutoCAD, third-party GIS software• Serving rasters as background imagery on intranet/internet sites
Wide range of User Requirements for Imagery
Use Manage
Displayperformance
Query
Analyze
Storage
CatalogProcess
Measure
Most organizations have mixture of users
Accessibility
TimelinessMetadata
Serviceproperties
Export
Geometricaccuracy
Radiometricaccuracy
Workflows
Performance
StandardsInteroperability ScalabilitySca
Maintenance
de a ge o UseIn nearly all industries
4 Typical Imagery User Types
• Simple Use– “I want imagery as a background
to my application”
• Advanced Use– “I want to perform image analysis”
• Simple Management– “How do I serve all our orthoimages?”
• Advanced Management– “How do I processes and serve
the imagery that we collect?”
Most organizations have mixture of types
Simple Adv.
Mgmt.
Use
4 Typical Imagery User Types
• Simple Use– “I want imagery as a background
to my application”
• Advanced Use– “I want to perform image analysis”
• Simple Management– “How do I serve all our orthoimages?”
• Advanced Management– “How do I processes and serve
the imagery that we collect?”
Most organizations have mixture of types
Simple Adv.
Mgmt.
Use
Si l
Web API,Explorer etc
4 Typical Imagery User Types
• Simple Use– “I want imagery as a background
to my application”
• Advanced Use– “I want to perform image analysis”
• Simple Management– “How do I serve all our orthoimages?”
• Advanced Management– “How do I processes and serve
the imagery that we collect?”
Most organizations have mixture of types
Simple Adv.
Mgmt.
Use
Si l
Web API,Explorer etc
ArcGISDesktop
4 Typical Imagery User Types
• Simple Use– “I want imagery as a background
to my application”
• Advanced Use– “I want to perform image analysis”
• Simple Management– “How do I serve all our orthoimages?”
• Advanced Management– “How do I processes and serve
the imagery that we collect?”
Most organizations have mixture of types
Simple Adv.
Mgmt.
Use
le Adv.
Mosaic DatasetImage ServicesImage Extension
Choosing Appropriate SoftwareConsiderations
• Based on business workflows…– Do I need to serve just raster data? Maps? Tools and models?– What client applications should the solution support?– How often do my rasters get updated?– Is good quality raster data available as a service?
• …and IT capabilities:– What is my network bandwidth?– Does staff know IIS (Tomcat, Apache, etc.)? Do I have a DBA?– Will I need any custom development? Do I have programmers? – Will I do implementation in-house or will I hire a consultant?
Choosing Appropriate SoftwareArcGIS Server licensing
• Three editions of ArcGIS Server functionality– Basic, standard, advanced
• Scaled by capacity– Workgroup or enterprise
• ArcSDE technology bundled with all editions
Advanced(extensions)
Standard(extensions)
Basic
Spatial analysisand editing
Data management
Visualizationand mapping
WorkgroupEnterprise
Functionality scaled by capacity
ArcGIS Server Basic Edition (ArcSDE)
Network
Client Application ArcSDE
Some Characteristics
• LAN/WAN RDBMS Gateway• Data server (vector and raster)• Mosaics, Catalogs, Raster
Attributes• Clients are Desktop and Internet
Servers
5x5 pixelblock
ArcGIS Server Basic (ArcSDE)Considerations
• Loading rasters into database takes time• Duplicating data (original tiles and database)• Using ArcSDE just for raster serving not recommended
– Significant investment into infrastructure, software, and staff training• Provides exceptional security/scalability options
– Options depend on specific RDBMS
ArcGIS Server Standard/Advanced Edition
• Serve raster and vector data
• Out of the box applications• Ability to cache data
ArcGIS Desktop
Browsers
ArcGIS Mobile
ArcGISExplorer
Enterprise RDBMS
ArcGIS Server
Some Characteristics
• Internet server• Maps & GIS functionality• Dynamic & Cached content• Integrate rasters and vectors• Desktop, browser, AutoCAD, and
mobile clients
1:500,000
1:250,000
Cached Versus Dynamic Services
• Maps can include both dynamic and static layers
• Dynamic Layers contain rapidly changing data– Roads symbolized by current snow depth– Electrical network showing the latest posted work order
• Cached Layers contain more slowly changing data– Landuse / Landcover– Road Network– Basemap data
• The classification is subjective / application specific
ArcGIS Server Standard/AdvancedConsiderations
• Caching highly recommended for raster data– Caching takes time and disk space– In a web application, zoom limited to predetermined scale levels– In desktop applications, resampling occurs
• Can be used to overlay vector and raster data• Web services can be consumed by variety of applications
– Out-of-the-box web application– Flex/JavaScript API allows integration with other web services– ArcMap
• Image processing and distribution• Variety of LAN clients• Ability to expose service as WMS
ArcMap
Image Server
ServiceEditor
ServiceDefinition
ServiceDefinition
ServiceDefinition
Raster Data
ArcGIS Server Image Extension
• LAN/WAN raster server– Enabling WMS capabilities
makes it an Internet server • Serve and process raster data• Serve unprocessed rasters, or
create multiple “virtual” processed raster products
• Desktop (ESRI & 3rd party) and Internet Servers
Service
Process(es)
Derived
Images
Output
Rasters
ArcGIS Server Image Extension
ArcGIS Server Image ExtensionConsiderations
• No loading or preprocessing required– Reduces delay from acquisition to dissemination– On-the-fly processing puts additional load on the server
• Ability to create “virtual products” on the fly– Visualize elevation– NDVI– Pan-sharpening– Many others
• Support for many client applications• Ability to expose service as WMS for web applications
– Viewable at any scale level• REST API support added at 9.3 SP3 and JavaScript API v1.2
File Server
• Highly compressed data (.sid, .ecw, etc.)
• Commonly used in raster catalogs
MyImage
.sid
MyImage
MyImage
MyImage
MyImage
Network
Some Characteristics
• LAN/WAN server• Serve tiled or highly compressed data• Desktop (ESRI & 3rd party)• Wide range of format choices• Limited small scale viewing options
File ServerConsiderations
• No additional server side software required• Limited small scale viewing options• No web access
– LAN/WAN solution only
ArcGIS Online
• Ready-to-use Maps and Globes– Standard and Premium content
• Use in Desktop and Server• Online capabilities
– Tasks in ArcGIS Explorer– Tools in Desktop and Server
• Content also available offline– Data Appliance
Some Characteristics
• Raster imagery is 1-meter or better for United States– Eliminates the need for some organizations to store data in-house
• ArcGIS Online Content Sharing Program– ArcGIS users contribute data content to be hosted and published by
ESRI for access by them and other users for free
Existing ESRI World Imagery New User Provided Imagery
ArcGIS OnlineConsiderations
• No need to maintain imagery in-house• Services include maps, globes, and tasks• Available to Desktop, Server, and JavaScript clients
– Licensing costs associated with commercial/internet use• Available as web service or offline via Data Appliance
– For facilities without internet access• May not have most up-to-date imagery
– Unless you provide it to us!
Network TrafficSample Dataset
• Compare amount of data transferred over network for:– Raw rasters, ArcSDE, ArcGIS Server Image and cached ArcGIS Server
map service
Size of rasters on disk: 15.40 GBRaw raster: 58.50 MBArcSDE: 892.50 KBArcGIS Server Image: 30.10 KBArcGIS Server cached map service: 94.60 KB
Demonstration
Data: Set of 144 raster tiles
• Create an unmanaged raster catalog• Create Image Service definition• View data as:
– ArcSDE raster– Unmanaged raster catalog– Image Service– ArcGIS Online service
gg
Training
• Training is Mission-Critical• By investing in GIS staff training, you speed up the time it takes to
implement a GIS solution…saving time means saving money• People ultimately bring life to GIS and insure a successful
implementation• Well trained staff can overcome obstacles• The untrained or under-trained desktop user will cost five times
more to support than a well-trained worker (Gartner, March 2006)
ESRI Training
• ESRI offers a variety of training options– Free or low cost Virtual Campus Training– Instructor-led (classroom) training– Client-site training– Custom training
• Multiple options for developing a training plan– Work with your Education Sales Consultant (Kerri Manorek) and
account manager
Example: Instructor-Led Classes
• Introduction to ArcGIS Server (2 days)• Introduction to ArcGIS Server Image Extension (2 days)• Introduction to Multiuser Geodatabase (2 days)
• Building Web Maps Using ArcGIS API for JavaScript (2 days)• Developing Applications with ArcGIS Server, MS .NET or for Java (3
days each)• ArcGIS Server Web Administration for Microsoft .NET (3 days)
• Data Management in the Multiuser Geodatabase (3 days)• ArcGIS Server Enterprise Configuration and Tuning, for SQL Server
or for Oracle (2 days each)
Implementation Scenario
• Organization:– Small Port Authority– About 10 square miles of area (800 MB of data)
• Business Needs: – Enable non-GIS users to make a map of work area
• Data Needs: – One raster layer (does not change)– Several background vector layers ( change quarterly)– One layer with dynamic data (permitting)
• IT Capabilities:– GIS staff of one; no DBA– One server
Implementation Suggestion
• Web application for non-GIS users with minimal functionality• Cached raster data (background)• Dynamic vector data (permitting layer)• Vector data stored and edited in ArcSDE
• Software Solution: ArcGIS Server Standard Workgroup– Does not require DBA for data management– Best performance over the network with cached rasters– Easy-to-use web application interface for non-GIS professionals
• Training: ArcGIS Server Web AdministrationData Management in Multiuser Geodatabases
Implementation Scenario
• Organization:– Engineering firm– Many projects over a large area
• Business Needs: – Create derivatives from DEM data (aspect, slope, hillshade, etc.)
• Data Needs: – DEM data from several sources– Constantly looking for new, better data
• Capabilities:– Robust knowledge of servers– Existing user of ESRI server technology
Implementation Suggestion
• Need to create multiple elevation derivatives• Potentially large number of outputs is hard to store and manage
• Software Solution: ArcGIS Server Image Extension– Virtual products created “on-the-fly”– No duplication of data
• Training: Introduction to ArcGIS Server Image Extension
Implementation Scenario
• Organization: – Conservation Organization– Large service area, small project areas
• Business Needs:– Deploy raster data fast with minimal administration
• Data:– Image tiles of service area
• IT Capabilities:– Limited IT support
Implementation Suggestion
• Implementation Suggestion 1: ArcGIS Online– Cached web services for USA include imagery, street, political maps– Very good data available for free– Prime data available with paid subscription
• Implementation Suggestion 2: File Server– Raster catalog requires almost no processing time– Raster tiles can be “turned off” until zoomed to small area– Use highly compressed tiles (like MrSID) for best performance– ESRI & 3rd party clients can access data
Implementation Scenario
• Organization:– Large Utility in “Hurricane Alley”
• Business Needs:– Large amounts of imagery need to be deployed enterprise-wide
monthly and more often during emergencies– Imagery needs to be accessible to variety of client applications,
including web applications, CAD, and GIS clients• IT Capabilities:
– Robust network infrastructure
Implementation Suggestion
• Avoid loading imagery into database or caching– Too much data (takes too long to load)– Changes too fast (short lifespan once loaded)
• Expose imagery to non-ESRI clients
• Implementation Suggestion: ArcGIS Server Image Extension– No data loading (still need to build service overviews)– No data pre-processing– Accessible by multiple client applications
• ArcIMS, ArcGIS Server, AutoCAD, Microstation, Desktop
• Training: Introduction to ArcGIS Server Image Extension
Implementation Scenario
• Organization– County Government
• Business Needs– Access to “better” (more detailed) elevation data via ArcGIS Explorer
• IT Capabilities– Existing ArcGIS Server Standard Enterprise implementation– ArcGIS Server web services currently used in internal and external
web applications– Knowledgeable DBA, web administrator
Implementation Suggestion
• Leverage existing technology• Elevation data does not change often
• Implementation Suggestion: ArcGIS Server Globe Service– Leverage existing software– Improve performance of elevation service by caching
• Training: ArcGIS Server Web Administration for Microsoft .NET frameworkLearning ArcGIS 3D Analyst (virtual campus class)
Steps to SuccessTest, Test, Test
• Begin with needs assessment– ESRI offers Professional Services to help get started
• Whatever solution you choose, test it• Testing includes
– Functionality testing– Performance testing
• Functionality testing– Is web browser app really going to meet my business needs?
• Performance testing– Shapefiles were slow, but is ArcSDE really any faster for me?
• ESRI offers special (=cheaper) licenses for development and testing environments
– Talk to your account manager
System Architecture Design
• Many Resources Available–System Design Strategies white paper–System Architecture Design
instructor-led class–Building a GIS book with On-line
Resource center
Steps to SuccessPlan for Success
• Have a plan for urgent situations– Identify support channels for new workflows
• Internal Help Desk? External tech support?• Services Available from ESRI
– Help Desk Setup Assistance– Premium Support Services (PSS)– Professional Services– EEAP (ESRI Enterprise Advantage Program)
• Remember, Failing to Plan is Planning to Fail
ESRI Enterprise Advantage Program
• Subscription program designed to provide ongoing advisory– Great for ongoing project and software support– Pre-paid subscription to be used throughout the year as needs arise
• Includes – Technical Advisor– GIS environment review– Credits to be used for Professional Services or Training– Annual account and GIS strategy review– Exclusive enterprise GIS webcasts
Summary
• Many Server Technologies to Serve Many Needs• Choose one based on business needs and staff skills• Ensure implementation success through testing• Ensure continued success through staff training• Many resources available from ESRI to assist in planning and
implementation
Resources
• Resource Centers http://resources.esri.com• ESRI Professional Services
http://www.esri.com/consulting/index.html• Online Help http://webhelp.esri.com
Thank YouKatja Krivoruchko
ESRI380 New York Street Redlands, California92373-8100 USA Phone: 909-793-2853Fax: 909-793-5953E-mail: [email protected]