Post on 11-Dec-2015
transcript
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 11
Managing Indiana’s Managing Indiana’s
2005 2005 Orthophotography Orthophotography in a Geodatabase in a Geodatabase
Anna RadueAnna RadueUniversity Information Technology Services University Information Technology Services
(UITS)(UITS)
Indiana UniversityIndiana University
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 22
Serving Raster Data in a RDBMSServing Raster Data in a RDBMS
Why use ArcSDE?Why use ArcSDE? Overview of system hardwareOverview of system hardware Basic raster ArcSDE ConceptsBasic raster ArcSDE Concepts
• Geodatabase rastersGeodatabase rasters• Pyramid layersPyramid layers• Raster compressionRaster compression
Loading tips and challengesLoading tips and challenges
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 33
Isn’t MrSID enough?Isn’t MrSID enough?
Good for small regionsGood for small regions Not appropriate for massively large Not appropriate for massively large
datasetsdatasets Does not support multi-useDoes not support multi-use No-Data areas may be a problem for No-Data areas may be a problem for
seamless viewingseamless viewing
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 44
Why Put Images in a Database?Why Put Images in a Database?
Multiuser AccessMultiuser Access Common Data ManagementCommon Data Management Access to very large images Access to very large images Data SecurityData Security PerformancePerformance
Database
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 55
ArcSDEArcSDE
ESRI’s Advanced Spatial Data ServerESRI’s Advanced Spatial Data Server Serves Spatial Data toServes Spatial Data to
• ArcGIS Desktop applicationsArcGIS Desktop applications• Internet ClientsInternet Clients• Applications developed with ArcGIS Applications developed with ArcGIS
Engine and ArcGIS ServerEngine and ArcGIS Server Supports terabytes of imagerySupports terabytes of imagery
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 66
Research Database Complex (RDC)Research Database Complex (RDC)
Supercomputing system dedicated to Supercomputing system dedicated to Oracle database use Oracle database use
Managed by RAC Unit within UITSManaged by RAC Unit within UITS 3 Solaris Servers3 Solaris Servers Located in Bloomington machine roomLocated in Bloomington machine room Attached EMC CLARiiON system –Attached EMC CLARiiON system –
networked storage system with FC and networked storage system with FC and ATA disksATA disks
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 77
Solaris ServersSolaris Servers Oracle & ArcIMS Spatial Oracle & ArcIMS Spatial
ServerServer• Sun Fire V1280 ServerSun Fire V1280 Server
12 UltraSPARC III 12 UltraSPARC III ProcessorsProcessors
96 GB of memory96 GB of memory Gb switched ethernetGb switched ethernet
Apache/Tomcat/ArcIMS Apache/Tomcat/ArcIMS Web Server Web Server • Sun Fire V880 ServerSun Fire V880 Server
4 UltraSPARC III 4 UltraSPARC III ProcessorsProcessors
8 GB of memory8 GB of memory
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 88
Fiber-Channel DisksFiber-Channel Disks
Servers shareServers share Provides the highest level of disk Provides the highest level of disk
access performance and capacity access performance and capacity available in the storage marketplaceavailable in the storage marketplace
Technology supports peak Technology supports peak concurrent client loadsconcurrent client loads
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 99
Parallel ATA DisksParallel ATA Disks
Advanced Technology Attachment Advanced Technology Attachment (ATA) – std first introduced in 1986(ATA) – std first introduced in 1986
Generally relegated to non-server Generally relegated to non-server scenarios such as laptops and scenarios such as laptops and desktops desktops
About 50% the cost of fibre channel About 50% the cost of fibre channel disksdisks
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 1010
Disk Access PerformanceDisk Access PerformanceESRI Systems Integration Technical Brief: Image Storage ArchitecturePrepared by: Dave Peters, ESRI
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 1111
EMC Storage SystemEMC Storage SystemTwo-Tier ArchitectureTwo-Tier Architecture
Combination of Combination of Fibre channel and Fibre channel and parallel ATA disks parallel ATA disks in RAID systemin RAID system
Oracle system Oracle system tables, vector data, tables, vector data, database index database index and log files on FCand log files on FC
Raster imagery on Raster imagery on ATAATA
FC
ATA
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 1212
Geodatabase RastersGeodatabase Rasters
1.1. Raster CatalogRaster Catalog
2.2. Raster DatasetRaster Dataset
3.3. Hybrid Raster CatalogHybrid Raster Catalog
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 1313
Raster CatalogRaster Catalog
• Individual images are displayed as Individual images are displayed as seamless imageseamless image
• Useful for users having frequent image Useful for users having frequent image updatesupdates
• Maintains areas of overlapMaintains areas of overlap• Image cells do not need to alignImage cells do not need to align• Database can store the images or Database can store the images or
referencereference
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 1414
Raster DatasetRaster Dataset
A raster composed of multiple input A raster composed of multiple input images – 2813 for DOQQ 1998images – 2813 for DOQQ 1998
Image overlap not keptImage overlap not kept Cells must alignCells must align Mosaiced rasters perform better than a Mosaiced rasters perform better than a
raster catalograster catalog
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 1515
Hybrid Raster CatalogHybrid Raster CatalogArcCatalog ArcMa
p
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 1616
What are pyramid layers?What are pyramid layers?
Reduced resolution Reduced resolution representations of representations of the data setthe data set
Improve display Improve display performanceperformance
Increase size of Increase size of imageimage
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 1717
Image and 2 PyramidsImage and 2 Pyramids
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 1818
Pyramid StatsPyramid Stats
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 1919
Pyramid Resampling MethodsPyramid Resampling Methods
Nearest NeighborNearest Neighbor
BilinearBilinear
BiCubicBiCubic
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 2020
CompressionCompression LZ77 LZ77
• LosslessLossless• Requires the most disk spaceRequires the most disk space
JPEG JPEG • Lossy – different qualitiesLossy – different qualities• 75% ESRI default75% ESRI default
JPEG 2000JPEG 2000• lossy and lossless depending on qualitylossy and lossless depending on quality• Loading and pyramid building SLOWLoading and pyramid building SLOW
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 2121
JPG 75%, bilinear pyramidJPG 75%, bilinear pyramid
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 2222
ArcSDE Tips and Tricks #1ArcSDE Tips and Tricks #1
Create the Raster Catalog using Create the Raster Catalog using ArcToolbox UtilitiesArcToolbox Utilities
Unless loading a small number of Unless loading a small number of images, load imagery using sderaster images, load imagery using sderaster command – trying to load 200+ command – trying to load 200+ images in ArcCatalog 9.1 crashes images in ArcCatalog 9.1 crashes the application even on robust the application even on robust systemssystems
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 2323
ArcSDE Tips and Tricks #2ArcSDE Tips and Tricks #2Spatial DomainSpatial Domain
Change Spatial Change Spatial Domain default Domain default valuesvalues
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 2424
ArcSDE Tips and Tricks #3 ArcSDE Tips and Tricks #3
Output Spatial Grid Output Spatial Grid Set to 250,000Set to 250,000
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 2525
ArcSDE Tips and Tricks #4ArcSDE Tips and Tricks #4ArcSDE ViewsArcSDE Views
Views filter information stored in ArcSDE Views filter information stored in ArcSDE layers and database tableslayers and database tables
Simplifies or limits data access for the end Simplifies or limits data access for the end useruser
Access to information stored in database Access to information stored in database tables is controlled with tables is controlled with wherewhere clauses – clauses – only completely loaded counties are only completely loaded counties are viewable by end users – partially loaded viewable by end users – partially loaded counties without pyramids will always counties without pyramids will always hang ArcMaphang ArcMap
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 2626
ArcSDE Raster DatasetsArcSDE Raster Datasets
Serve imagery quickly to clients even Serve imagery quickly to clients even using lower performance disksusing lower performance disks
Can store terabytes of imageryCan store terabytes of imagery http://gis.iu.edu http://gis.iu.edu
• Indiana Spatial Data ServiceIndiana Spatial Data Service• Indiana Spatial Data PortalIndiana Spatial Data Portal
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 2727
AcknowledgementsAcknowledgements ArcSDE/Oracle ConsultantArcSDE/Oracle Consultant
Mannion Geosystems, LLCMannion Geosystems, LLC Data Management Support, UITSData Management Support, UITS
• Michael Halla, ManagerMichael Halla, Manager• Nathan EatonNathan Eaton• Stephanie SniderStephanie Snider
Research and Technical Services, UITSResearch and Technical Services, UITS• Stephanie Burks, Oracle AdministratorStephanie Burks, Oracle Administrator• Teresa Hunter, UNIX AdministratorTeresa Hunter, UNIX Administrator
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 2828
Financial and Data SupportFinancial and Data Support IGIC FGDC CAP Grant purchased ATA IGIC FGDC CAP Grant purchased ATA
disks Jill Saligoe-Simmeldisks Jill Saligoe-Simmel Department of Homeland Security Department of Homeland Security
purchased FC disks and approved purchased FC disks and approved service Roger Koelpinservice Roger Koelpin
Pinnacle providing IU with imageryPinnacle providing IU with imageryPhil WorrallPhil Worrall
3/8/20063/8/2006 http://gis.iu.eduhttp://gis.iu.edu 2929
Questions?Questions?
aradue@indiana.edu