Post on 16-Jul-2018
transcript
ArcGIS Server Solutions Architecture Considerations
Andrew SakowiczESRI Professional Services
11
AgendaAgenda
•• What is System Architecture?What is System Architecture?•• Design Design -- System ArchitectureSystem Architecture
–– BusinessBusiness–– DataData–– ApplicationApplication–– TechnologyTechnology
•• Deploying Deploying -- System ArchitectureSystem Architecture–– Scalability & ReliabilityScalability & Reliability–– PerformancePerformance–– SecuritySecurity–– Capacity PlanningCapacity Planning–– Virtualization and CloudVirtualization and Cloud
•• Architecture SnapshotsArchitecture Snapshots•• SummarySummary
22
What is System Architecture?The Simple DefinitionThe Simple Definition
• Provide optimal solutions specifying implementation information for 4 basic components– Users– Applications– Data– Network
Users
Applications Data1
23
Network4
Network4
33
What is System Architecture?What is System Architecture?Sounds Simple EnoughSounds Simple Enough
•• Only 4 basic componentsOnly 4 basic components–– Why are we in this session then?Why are we in this session then?
•• The difficultyThe difficulty–– Addressing two sets of extremely complex Enterprise RequirementsAddressing two sets of extremely complex Enterprise Requirements
Organizations turn to Enterprise Architecture Frameworks as a roadmap
Quality Attribute RequirementsQuality Attribute RequirementsInfrastructureInfrastructure
Performance & ScalabilityPerformance & ScalabilitySecurity Security ……
Functional RequirementsFunctional RequirementsVisualizationVisualization
AnalysisAnalysisWorkflow Integration...Workflow Integration...
44
Why is System Architecture important? Why is System Architecture important? Helps address challenges faced by Energy CompaniesHelps address challenges faced by Energy Companies
•• Global companiesGlobal companies•• Integration with other vendorsIntegration with other vendors
–– standards play a key rolestandards play a key role•• Multiple data sources, projection and stewards Multiple data sources, projection and stewards •• ““Real TimeReal Time”” information requirement from remote nodesinformation requirement from remote nodes
–– What is an appropriate replication mechanism?What is an appropriate replication mechanism?–– Should deploy ArcGIS Server locally on each node?Should deploy ArcGIS Server locally on each node?–– Is Could a solution?Is Could a solution?
•• Insufficient network bandwidth between nodesInsufficient network bandwidth between nodes•• Costly software upgradesCostly software upgrades•• Integrating GIS fit within the Enterprise Integrating GIS fit within the Enterprise –– role of ITrole of IT
55
What is System Architecture?What is System Architecture?Key Enterprise Architecture AreasKey Enterprise Architecture Areas
•• Your Enterprise GIS Architecture is based on four* key areas:Your Enterprise GIS Architecture is based on four* key areas:
–– Business ArchitectureBusiness Architecture -- Business needsBusiness needs–– Data ArchitectureData Architecture -- Data and workflowData and workflow–– Application ArchitectureApplication Architecture -- Applications utilizedApplications utilized–– Technology ArchitectureTechnology Architecture -- Supporting infrastructureSupporting infrastructure
•• Each area provides key information for System ArchitectureEach area provides key information for System Architecture
* Based on The Open Group Architecture Framework (Version 9), a mainstream IT framework
66
What is System Architecture?What is System Architecture?Scope in an Enterprise ProjectScope in an Enterprise Project
Project PhasesProject Phases
System ArchitectureSystem Architecture
Design and Deployment are the next two sections of this presentationDesign – Focuses on Architecture Inputs and Decisions
Deployment – Focuses on Implementation Quality Attributes77
Design Design -- System ArchitectureSystem Architecture
88
Design Design -- System ArchitectureSystem ArchitectureBusiness ArchitectureBusiness Architecture
99
DesignDesignBusiness ArchitectureBusiness Architecture
•• IdentifyIdentify–– Business strategyBusiness strategy–– GovernanceGovernance–– Organization e.g. global vs. regionalOrganization e.g. global vs. regional–– Key business processesKey business processes
•• DecisionsDecisions–– Project ScopeProject Scope
•• One site, multiple sites, your whole org, interaction with otherOne site, multiple sites, your whole org, interaction with other orgsorgs……
–– Is the organization structured as centralized or deIs the organization structured as centralized or de--centralized?centralized?
–– What are the key scenarios?What are the key scenarios?–– MigrationMigration
1010
Design Design -- System ArchitectureSystem ArchitectureData ArchitectureData Architecture
1111
DesignDesignData ArchitectureData Architecture
•• IdentifyIdentify–– Logical data assetsLogical data assets–– Physical data assetsPhysical data assets–– Data management resourcesData management resources
•• DecisionsDecisions–– How to address multiple data sources, projection and stewards How to address multiple data sources, projection and stewards –– Is data going to be physically centralized or deIs data going to be physically centralized or de--centralized?centralized?
•• If deIf de--centralized, is replication necessary?centralized, is replication necessary?–– If replication is necessary, determine appropriate replication mIf replication is necessary, determine appropriate replication mechanismsechanisms–– Is spatial versioning planned or nonIs spatial versioning planned or non--simple spatial features?simple spatial features?
»» If yes, then If yes, then ArcGIS Server Replication ArcGIS Server Replication might be a best fitmight be a best fit»» If no, then Relational database or disk hardware level replicatiIf no, then Relational database or disk hardware level replication might be beston might be best
1212
Design Design -- System ArchitectureSystem ArchitectureApplication ArchitectureApplication Architecture
1313
DesignDesignArcGIS Server Conceptual ArchitectureArcGIS Server Conceptual Architecture
•• TiersTiers–– ClientClient–– WebWeb–– GISGIS–– DataData
1414
Web Service ClientsArcMap, ArcGIS Explorer, Gaia, etc.
SOAP REST WMS WCS WFS KML
Map / Image Services
ArcGIS ServerArcGIS Server
FGDB
Data
DesignDesignApplication ArchitectureApplication Architecture
•• IdentifyIdentify–– Individual application needsIndividual application needs–– Application interactionsApplication interactions–– Application relationships to the core business processes of the Application relationships to the core business processes of the organization organization
•• DecisionsDecisions–– First: Choose application typesFirst: Choose application types–– Second: Choose corresponding ESRI applicationsSecond: Choose corresponding ESRI applications
Use the Enterprise GIS Resource Center to help identify what is appropriate for your project1515
DesignDesignApplication ArchitectureApplication Architecture
•• Choose application type(s)Choose application type(s)
–– Application patterns are Application patterns are applicable cross platform applicable cross platform and vendor solutionsand vendor solutions
–– MS Open Source MS Open Source Application PatternsApplication Patterns
SMS,
Flex
1616
DesignDesignChoose Application ArchitectureChoose Application Architecture
Example: ArcGIS API for FlexExample: ArcGIS API for Flex•• TiersTiers
–– ClientClient–– WebWeb–– GISGIS
•• ArcGIS SOM and ArcGIS SOCArcGIS SOM and ArcGIS SOC
•• Management InterfacesManagement Interfaces–– ArcCatalogArcCatalog–– ArcGIS Services DirectoryArcGIS Services Directory–– ArcGIS Server ManagerArcGIS Server Manager
•• Processing LoadsProcessing Loads–– Notice Presentation Layer offloaded Notice Presentation Layer offloaded
to clientto client–– Cache balance on client and serverCache balance on client and server
1717
Design Design -- System ArchitectureSystem ArchitectureTechnology ArchitectureTechnology Architecture
1818
DesignDesignTechnology Architecture Technology Architecture –– key decisionskey decisions
•• IdentifyIdentify–– IT infrastructureIT infrastructure–– Middleware needsMiddleware needs–– NetworksNetworks–– Standards Standards
•• DecideDecide–– Choose an Architectural Style and standardsChoose an Architectural Style and standards–– Where Virtualization should be utilizedWhere Virtualization should be utilized–– How to best minimize Network loadsHow to best minimize Network loads–– What storage mechanisms to utilizeWhat storage mechanisms to utilize
1919
DesignDesignTechnology ArchitectureTechnology Architecture
•• Storage MechanismsStorage Mechanisms
–– SANSAN•• Provides optimal Provides optimal
performance, availability and performance, availability and scalabilityscalability
•• Common for RDBMS and Common for RDBMS and Vector dataVector data
–– NASNAS•• SAN typically does not allow SAN typically does not allow
concurrent access of multiple concurrent access of multiple servers to file based storage servers to file based storage on one LUNon one LUN
•• Common for File Based Common for File Based imageryimagery
–– DASDAS•• Utilize local disks on web Utilize local disks on web
application servers with readapplication servers with read--only File Geodatabasesonly File Geodatabases
•• Fast performanceFast performance
File SystemRaster only
Network Attached Storage (NAS)
Web Service ClientsArcMap, ArcGIS Explorer, Gaia, etc.
SOAP REST WMS WCS WFS KML
Map / Image Services
ArcGIS ServerArcGIS Server ImageServerImageServer
Direct ClientsArcMap, ERDAS, Server Manager, etc.
GeoDatabaseRaster, Vector,
Footprints, & MetadataStorage Area Network (SAN)
FGDB
2020
DesignDesignTechnology Architecture Technology Architecture –– use virtualization when appropriateuse virtualization when appropriate
•• Primary virtualization technologies utilized by ESRI customersPrimary virtualization technologies utilized by ESRI customers
–– Application Virtualization Application Virtualization •• Useful for scenario of:Useful for scenario of:
–– Centralized DataCentralized Data–– Distributed UsersDistributed Users–– User functional requirements demand Rich Client ApplicationsUser functional requirements demand Rich Client Applications
•• Common supporting products includeCommon supporting products include–– Windows Terminal ServicesWindows Terminal Services–– Citrix Citrix –– Provides additional Enterprise management capabilitiesProvides additional Enterprise management capabilities
–– Hardware Virtualization Hardware Virtualization •• Useful for scenario of:Useful for scenario of:
–– Multiple servers are not heavily utilizedMultiple servers are not heavily utilized–– Facilitates server consolidationFacilitates server consolidation
•• ArcGIS Server now fully supportedArcGIS Server now fully supported–– Testing indicates 20Testing indicates 20--30% performance/throughput degradation due to high I/O 30% performance/throughput degradation due to high I/O
demands of GIS imagery and functionsdemands of GIS imagery and functions–– 60% degradation has been observed so testing is key to identifyi60% degradation has been observed so testing is key to identifying your needsng your needs
2121
Deployment Deployment -- System ArchitectureSystem Architecture
2222
DeploymentDeploymentEnvironmentsEnvironments
Staging Virtual Machines(Reflect Production)
EDN System Production Systems
Reverse ProxyReverse Proxy Reverse ProxyReverse Proxy
Network LoadbalancerNetwork Loadbalancer Network LoadbalancerNetwork Loadbalancer
2323
DeploymentDeploymentEnsure Performance, Scalability and Reliability Service Level AgEnsure Performance, Scalability and Reliability Service Level Agreement (SLA)reement (SLA)
•• PerformancePerformance
•• Scale up vs. Scale outScale up vs. Scale out
•• Distributed vs. NonDistributed vs. Non--DistributedDistributed
•• Load balancing vs. ClusteringLoad balancing vs. Clustering
2424
Performance,Performance,Scalability & ReliabilityScalability & ReliabilityDistributed vs. NonDistributed vs. Non--DistributedDistributed
ArcSDEArcSDESOCSOC
ManageManageWeb TierWeb Tier
Single Machine ScenarioSingle Machine Scenario Multiple Machine ScenarioMultiple Machine Scenario
DBMSDBMS
DBMSDBMS
SOMSOMWeb TierWeb Tier
ArcSDEArcSDESOCSOC
ArcSDEArcSDESOCSOC
SOMSOMWeb TierWeb Tier
DBMSDBMS
•• Install ArcGIS Server close to data sourcesInstall ArcGIS Server close to data sources•• Only distribute components where necessaryOnly distribute components where necessary
•• This provides best performanceThis provides best performance•• Distribution may be necessary forDistribution may be necessary for
•• Security policies, physical constraints, shared business logic, Security policies, physical constraints, shared business logic, and scalabilityand scalability•• Design architecture to leverage Elasticity of the Cloud ComputinDesign architecture to leverage Elasticity of the Cloud Computingg 2525
Scalability & ReliabilityScalability & ReliabilityScale Up vs. Scale OutScale Up vs. Scale Out
•• Scale Up Scale Up –– Get a Bigger Server (Get a Bigger Server (AFTER TUNINGAFTER TUNING))–– Common recommendation for Relational Database ServersCommon recommendation for Relational Database Servers–– Look at scaleLook at scale--up options first if availability requirements not highup options first if availability requirements not high–– ProsPros
•• Simple option with no additional maintenance costsSimple option with no additional maintenance costs–– Cons: Cons:
•• Single point of failureSingle point of failure•• Windows limit is roughly 100 ArcSOC processesWindows limit is roughly 100 ArcSOC processes
•• Scale Out Scale Out –– Get more ServersGet more Servers–– Common recommendation for Web and ArcGIS Application ServersCommon recommendation for Web and ArcGIS Application Servers–– ProsPros
•• Increases load capacity, protects against hardware failures, andIncreases load capacity, protects against hardware failures, and ArcGIS ArcGIS products designed to scale out on multiple tiers.products designed to scale out on multiple tiers.
–– ConsCons•• Additional administrative and equipment costsAdditional administrative and equipment costs
2626
Scalability & ReliabilityScalability & ReliabilityLoad Balancing vs. ClusteringLoad Balancing vs. Clustering
•• Requests distributed across all Requests distributed across all healthy servers healthy servers
•• Enhances performance, availability, Enhances performance, availability, and scalability of services such as and scalability of services such as ArcGIS Web App servers, and readArcGIS Web App servers, and read--only storesonly stores
Load Balancing Failover Cluster
•• Upon failure, standby server takes overUpon failure, standby server takes over•• Provides high availability and scalability Provides high availability and scalability
for read/write stores such as databasesfor read/write stores such as databases•• ArcGIS SOM cluster can provide HA ArcGIS SOM cluster can provide HA
without burden of 2 SOMs to maintainwithout burden of 2 SOMs to maintain2727
DeploymentDeploymentPerformancePerformance
•• Selecting ServicesSelecting Services
•• CachingCaching
2828
PerformancePerformanceSelecting ServicesSelecting Services
•• Different geospatial Different geospatial services have services have different loads on different loads on your systemsyour systems
–– This table This table summarizes the summarizes the relative relative performance of performance of servicesservices
–– If performance is If performance is high, then a fast high, then a fast result is returned result is returned with less system with less system overheadoverhead
2929
DesignDesignTechnology Architecture Technology Architecture –– Optimize network loadsOptimize network loads
•• How to best minimize Network loadsHow to best minimize Network loads–– ArcGIS Server should be installed closed to the data sourcesArcGIS Server should be installed closed to the data sources–– Visited tiles are cached in a browserVisited tiles are cached in a browser–– Map Service vs. Query tasksMap Service vs. Query tasks–– JPEG vs. PNGJPEG vs. PNG–– Bandwidth acceleratorsBandwidth accelerators–– Optimize content delivery, e.g. AKAMIOptimize content delivery, e.g. AKAMI
3030
DesignDesignTechnology Architecture Technology Architecture –– optimize network loadsoptimize network loads
•• Estimate Network loadsEstimate Network loads–– Identity available infrastructure bandwidthsIdentity available infrastructure bandwidths–– Plan for infrastructure upgrades as necessary and balance with aPlan for infrastructure upgrades as necessary and balance with adjusting djusting
workflows and services utilizedworkflows and services utilized
3131
SecuritySecurityArcGIS Component LocationsArcGIS Component Locations
Data
Web Server
GIS ServerInternetInternet(HTTP)(HTTP)
Rich Client Rich Client ApplicationsApplications
ArcGIS Explorer, ArcGIS Explorer, ArcGIS Desktop, ArcGIS Desktop, ArcGIS EngineArcGIS Engine
Web Web ApplicationApplication
Mobile Mobile DeviceDevice ManagerManager
ArcCatalogArcCatalog
ArcGIS Desktop ArcGIS Desktop Content AuthorContent Author
Internet Internet Connection Connection ––
Web UsersWeb Users
Local Local Connection Connection ––Local UsersLocal Users
ArcGIS Server
ReverseProxy
Internal Local Area NetworkDMZ
SOM & SOC
3232
DeploymentDeploymentCapacity PlanningCapacity Planning
•• Initial Capacity Planning GuidanceInitial Capacity Planning Guidance
•• Single Workflow ModelingSingle Workflow Modeling
•• Enterprise Workflow ModelingEnterprise Workflow Modeling
•• Load TestingLoad Testing
3333
DeploymentDeploymentInitial Capacity Planning GuidanceInitial Capacity Planning Guidance
High-level capacity guidance
Useful when user and business needs not clearly delineated
3434
DeploymentDeploymentModel Driven Capacity PlanningModel Driven Capacity Planning
•• Single Workflow ModelingSingle Workflow Modeling–– Simple Capacity CalculatorSimple Capacity Calculator–– NonNon--Distributed server modelDistributed server model–– Benchmark comparisonsBenchmark comparisons–– See Enterprise Resource CenterSee Enterprise Resource Center
•• Enterprise Workflow ModelingEnterprise Workflow Modeling–– Capacity Planning Tool (CPT)Capacity Planning Tool (CPT)–– Modeling includesModeling includes
•• Multiple workflowsMultiple workflows•• Multiple Servers and tiersMultiple Servers and tiers•• Network loadsNetwork loads
•• LoadLoad--TestingTesting–– Collect your own statistics and modelCollect your own statistics and model
•• See Enterprise GIS Resource Center for guidanceSee Enterprise GIS Resource Center for guidance–– Enterprise Implementation Services TeamEnterprise Implementation Services Team
•• Collect statistics for your environment and provide recommendatiCollect statistics for your environment and provide recommendationsons
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
0
5,000
10,000
15,000
20,000
25,000
0 10 20 30 40 50 60
Respon
se Tim
e
Throughp
ut
User Load
Benchmark and Target Response Time and Throughput
Throughput, B Throughput, T Response Time, B Response Time, T
Workflow Name: Export Map Dynamic Rest Service
RTt 3.00 sec
Userst 58.00 users
Think 6.00 sec
ST ArcSOC Zoom b 0.60 sec
Mbits/tr b 1.81 Mbits/tr
SpecRatebPerCPU 13.425
SpecRatetPerCPU 13.425
%CPU 95 %
THt 23,200 tr/hr
#CPUt 4.04 CPU cores
Mbps 11.69 Mbps
input valuescalculated values
3535
Architecture SnapshotsArchitecture Snapshots
3636
Architecture SnapshotsArchitecture SnapshotsGeocentric Geocentric -- Shell OilShell Oil
Find &Retrieve
UserApplicationinterfaces
CorporateMemory
Documents Library Records
A B
CFederated DBs Federated DBMSsImagery
Business Catalog(Central Asset Register)
GIS Catalog(Metadata per layer)
Search (spatial, text,
natural language)
Geoparsing&
Geocoding
Browse (folders,
categories)
Specialist Apps(Business/GeoScience)
Corporate Portal(User-customisable)
GIS(ArcGIS Desktop)
Internet (search, trade journals,
hosted services)
www
3737
Architecture SnapshotArchitecture SnapshotCounty ImplementationCounty Implementation
3838
Architecture SnapshotsArchitecture SnapshotsGeospatiallyGeospatially--Enabled Enabled -- SearsSears
Customer RequestHome Delivery OR
Home Product Repair Mainframe CIS
GIS Based Service Area PlanningSears Enhanced Home Sears Enhanced Home
Delivery System (EHDS)Delivery System (EHDS) Routing forkliftsWarehouse OptimizationWarehouse Optimization
Computer Aided Routing Computer Aided Routing System (CARS)System (CARS)
Customer ServiceDelivery Vehicle departs to your
home to Deliver MerchandiseService Technician Departs to your
home for Product Repair
Mobile MappingMobile Mapping
Assign Technicians to work areasAssign Technicians to work areas
Capacity Area Management Capacity Area Management System (CAMS)System (CAMS)
Nationwide Geocoding and Route Optimization
Geocoding and Route OptimizationGeocoding and Route Optimization
Forecasted Demand Available Fleet
ArcObjects
GPS
Mobile Workstation
In-Vehicle Navigation
NationwideGeocoding and Routing
Turn Right on EastWashington Street
Route Management
InIn--Vehicle NavigationVehicle Navigation
3939
Architecture SnapshotArchitecture SnapshotAcGIS Server on Amazon CloudAcGIS Server on Amazon Cloud
4040
Architecture SnapshotArchitecture SnapshotAcGIS Server on Amazon Cloud AcGIS Server on Amazon Cloud –– building map cachebuilding map cache
4141
Architecture SnapshotArchitecture SnapshotAcGIS Server on Amazon Cloud AcGIS Server on Amazon Cloud –– building map cachebuilding map cache
4242
Architecture SnapshotsArchitecture SnapshotsReference ImplementationsReference Implementations
•• ESRI will be posting ESRI will be posting Reference Implementations to Reference Implementations to the new Enterprise GIS the new Enterprise GIS Resource CenterResource Center
•• Located in the Located in the Implementation GalleryImplementation Gallery
•• Reference Implementations Reference Implementations are a mechanism to are a mechanism to communicate deployment communicate deployment best practicesbest practices
–– Goal is to also provide Goal is to also provide performance and security performance and security validation for Reference validation for Reference ImplementationsImplementations
4343
SummarySummary
4444
SummarySummary
•• Remember the 4 basic components of System ArchitectureRemember the 4 basic components of System Architecture–– User, Application, Data, and NetworkUser, Application, Data, and Network
•• Architecture is an iterative processArchitecture is an iterative process–– Use your key scenarios to validate proposed alternativesUse your key scenarios to validate proposed alternatives
•• Enterprise solutions fail if Quality Attributes not addressedEnterprise solutions fail if Quality Attributes not addressed–– System Architecture addresses Quality Attribute RequirementsSystem Architecture addresses Quality Attribute Requirements
•• Patterns and Best Practice References are your friendPatterns and Best Practice References are your friend–– Application Architecture PatternsApplication Architecture Patterns–– Architecture StylesArchitecture Styles–– Reference ImplementationsReference Implementations–– Capacity Planning GuidesCapacity Planning Guides
Check out the Enterprise GIS Resource Center for the technical answers you need to provide optimal solutions
4545
ReferencesReferences
•• ESRI Enterprise GIS Resource Center WebsiteESRI Enterprise GIS Resource Center Website–– Focused Enterprise GIS Technical Solutions Focused Enterprise GIS Technical Solutions –– http://resources.esri.com/enterprisegis/http://resources.esri.com/enterprisegis/
•• MS Open Source Application Architecture Pattern GuidanceMS Open Source Application Architecture Pattern Guidance–– http://www.codeplex.com/AppArch http://www.codeplex.com/AppArch
•• TOGAF TOGAF –– An Enterprise Architecture FrameworkAn Enterprise Architecture Framework–– http://www.opengroup.org/togaf/http://www.opengroup.org/togaf/
•• ESRI Project Center WebsiteESRI Project Center Website–– Guidance for managing an Enterprise GIS projectGuidance for managing an Enterprise GIS project–– http://support.esri.com/index.cfm?fa=projectCenter.gatewayhttp://support.esri.com/index.cfm?fa=projectCenter.gateway
4646