Date post: | 01-Jan-2016 |
Category: |
Documents |
Upload: | traugott-nigel |
View: | 22 times |
Download: | 0 times |
August 25 2005 Briefing [email protected] [email protected] [email protected] 1
ANABAS
Use of Grids in DoD Applications
Geoffrey Fox, Alex Ho, Marlon PierceSBIR Briefing August 25, 2005
August 25 2005 Briefing [email protected] [email protected] [email protected] 2
What do Web Services Prescribe?• The specify interfaces for system services (and generally useful
services like database) • They specify an interface language (WSDL) for all services• They develop containers and frameworks to use to host services• They specify a message format (SOAP) for ALL messages that
defines both application and system actions precisely• They imply a process be started to define domain specific
services• There are multiple competing activities from Microsoft and IBM
to Apache, IU and Anabas (for example) developing system and application services
• Unlike for RTI and CORBA, services from different vendors should interoperate
H1 H4H3H2 Body F1 F2 F3 F4 Service
Container Handlers
Container System Processing
August 25 2005 Briefing [email protected] [email protected] [email protected] 3
What do Grids Add?• Grids use all of the Web Services • They address management and deployment of large
distributed systems of services • They address security and management issues of virtual
organizations crossing multiple administrative domains• GGF is developing specific services of relevance including
job management, many aspects of data and scheduling• GGF has a good process for developing new higher level
specifications– For example GGF15 will address Cross enterprise security,
Campus Grids, Enterprise Grids, High Performance messaging, GIS Grids, Portals and continue OGSA work integrating data, compute and management
August 25 2005 Briefing [email protected] [email protected] [email protected] 4
The Grid and Web Service Institutional Hierarchy
1: Container and Run Time (Hosting) Environment
2: System Services and FeaturesHandlers like WS-RM, Security, Programming Models like BPEL
or Registries like UDDI
3: Generally Useful Services and FeaturesSuch as “Access a Database” or “Submit a Job” or “ManageCluster” or “Support a Portal” or “Collaborative Visualization”
4: Application or Community of InterestSpecific Services
such as “Run BLAST” or “Look at Houses for sale”
OGSAand otherGGF/W3C/ ………
WS-* fromOASIS/W3C/Industry
Apache Axis.NET etc.
August 25 2005 Briefing [email protected] [email protected] [email protected] 5
The Grid and Web Service Functional Hierarchy
B: Resources
C: Electronic Proxy Services for Resources
G: User Interface
F: Portal: Aggregation, Profiles
E: Manipulating and Linking Services
A: Pervasive System Services: Security, Collaboration, Messaging, Metadata
D: Brokering Monitoring and Managing Resources and Services
August 25 2005 Briefing [email protected] [email protected] [email protected] 6
Composing Functionality and Resourcesin the Grid of Grids
Overlayand ComposeGrids of Grids
Methods Services Functional Grids
CPUs Clusters ComputeResource Grids
MPPs
DatabasesFederatedDatabases
Sensor Sensor Nets
DataResource Grids
August 25 2005 Briefing [email protected] [email protected] [email protected] 7
Critical Infrastructure (CI) Grids built in composite fashion and linked to an NCOW (GiG) Grid
DoD Servicesand Filters
Physical Network
Registry Metadata
Flood Servicesand Filters
Flood CIGrid NCOW Grid… Electricity CIGrid …
Data Access/Storage
Security WorkflowNotification Messaging
Portals Visualization GridCollaboration Grid
Sensor Grid Compute GridGIS Grid
August 25 2005 Briefing [email protected] [email protected] [email protected] 8
Mediation and Transformation in a Grid of Grids and Simple Services
Po
rtP
ort
Port PortInternal
Interfaces
Subgrid or service
Po
rtP
ort
Port PortInternal
Interfaces
Subgrid or service
Po
rtP
ort
Port PortInternal
Interfaces
Subgrid or service
Messaging
Mediation andTransformationServices
External facingInterfaces
August 25 2005 Briefing [email protected] [email protected] [email protected] 9
DoD Services as Part of Grid of Grids
PortalServicesJSR168 Portlets
GridService1
XMSF RTIBridge
GridService2HPCMOGateway
GridService4Sensor(Net)
HLA Federation
e.g. SAFIRE
GridService5
XMSF RTIBridge
HLA Federate
GridService3
OGC MapService
GridService6
Discovery
GridService7
Visualization
Aggregation/Customization Portal
Resource Level
Grid Messaging Infrastructure
ANABAS
August 25 2005 Briefing [email protected] [email protected] [email protected] 10
The Global Information Grid Core Enterprise Services
Core Enterprise Services Service Functionality
CES1: Enterprise Services Management (ESM)
including life-cycle management
CES2: Information Assurance (IA)/Security
Supports confidentiality, integrity and availability. Implies reliability and autonomic features
CES3: Messaging Synchronous or asynchronous cases
CES4: Discovery Searching data and services
CES5: Mediation Includes translation, aggregation, integration, correlation, fusion, brokering publication, and other transformations for services and data. Possibly agents
CES6: Collaboration Provision and control of sharing with emphasis on synchronous real-time services
CES7: User Assistance Includes automated and manual methods of optimizing the user GiG experience (user agent)
CES8: Storage Retention, organization and disposition of all forms of data
CES9: Application Provisioning, operations and maintenance of applications.
August 25 2005 Briefing [email protected] [email protected] [email protected] 11
The Ten areas covered by the core WS-* Specifications
WS-* Specification Area Examples
1: Core Service Model XML, WSDL, SOAP
2: Service Internet WS-Addressing, WS-MessageDelivery; Reliable Messaging WSRM; Efficient Messaging MOTM
3: Notification WS-Notification, WS-Eventing (Publish-Subscribe)
4: Workflow and Transactions BPEL, WS-Choreography, WS-Coordination
5: Security WS-Security, WS-Trust, WS-Federation, SAML, WS-SecureConversation
6: Service Discovery UDDI, WS-Discovery
7: System Metadata and State WSRF, WS-MetadataExchange, WS-Context
8: Management WSDM, WS-Management, WS-Transfer
9: Policy and Agreements WS-Policy, WS-Agreement
10: Portals and User Interfaces WSRP (Remote Portlets)
August 25 2005 Briefing [email protected] [email protected] [email protected] 12
Activities in Global Grid Forum Working Groups
GGF Area Standards Activities
1: Architecture High Level Resource/Service Naming (level 2 of fig. 1),Integrated Grid Architecture
2: Applications Software Interfaces to Grid, Grid Remote Procedure Call, Checkpointing and Recovery, Interoperability to Job Submittal services, Information Retrieval,
3: Compute Job Submission, Basic Execution Services, Service Level Agreements for Resource use and reservation, Distributed Scheduling
4: Data Database and File Grid access, Grid FTP, Storage Management, Data replication, Binary data specification and interface, High-level publish/subscribe, Transaction management
5: Infrastructure Network measurements, Role of IPv6 and high performance networking, Data transport
6: Management Resource/Service configuration, deployment and lifetime, Usage records and access, Grid economy model
7: Security Authorization, P2P and Firewall Issues, Trusted Computing
August 25 2005 Briefing [email protected] [email protected] [email protected] 13
Core Services in Action INCOW Service or Feature WS-* Service area GGF and Others Demo Status
A: General Principles
Use Service Oriented Architecture Core Service Model (#1)
Build Grids on Web Services
YES
Grid of Grids Composition DTA7
B: NCOW Core Services (to be continued)
CES 1: Enterprise Services Management
WS-* #8 Management GGF #6: ManagementCIM
Phase II (DTA9)
CES 2: Information Assurance(IA)/Security
WS-* #5WS-Security
GGF #7, Grid-Shib, Permis Liberty Alliance etc.
Phase II (Not in DTA)
CES 3: Messaging WS-* #2, #3 JMS, MQSeries,Streaming /Sensor Technologies
SBIR (DTA1)
CES 4: Discovery WS-* #6 SBIR (DTA8)
CES 5: Mediation WS-* #4 workflow Treatment of Legacy systems. Data Transformations
SBIR (DTA6 DTA7)
CES 6: Collaboration VO GGF VO. XGSP, Shared Web Service ports
SBIR (DTA2 DTA3)
CES 7: User assistance WS- * #10 Portlets, JSR168NCOW Capability Interfaces
SAB (DTA10)
August 25 2005 Briefing [email protected] [email protected] [email protected] 14
Core Services in Action IINCOW Service or Feature WS-* Service area GGF and Others Demo Status
B: NCOW Core Services Continued
CES 8: Storage (not real-time streams)
GGF #4 DataNCOW Data Strategy
Phase II (Not in DTA)
CES 9: Application GGF #2 ; Best Practice in building Grid/Web services
SAB (DTA6 DTA7)
Environmental Control Services ECS
WS-*, #9 Phase II (Not in DTA)
Resource Infrastructure GGF #5; Ad-hoc networks important
NO
C: Key NCOW Capabilities not directly in CES
Meta-data WS-* #7 SAB (DTA8)
Resource/Service Matching/Scheduling
Distributed Scheduling and SLA’s (GGF # 3)
GGF scheduling work extended to networks
Phase II with scheduled messaging (DTA1)
Sensors (real-time data) OGC Sensor standards SBIR (DTA6)
GIS OGC GIS standards SBIR (DTA4 DTA5)
August 25 2005 Briefing [email protected] [email protected] [email protected] 15
Analysis of Grid Technology for DoD• We developed four significant documents• DoD Grid Opportunities for the GiG and NCOW
– Grids for the GiG and Real Time Simulations (Proceedings of Ninth IEEE International Symposium DS-RT 2005 on Distributed Simulation and Real Time Applications' Montreal October 10-12 2005) is subset
• Appendix on Grid and Web Services covering all basic Grid and Web services
• Grid Technology Overview and Status reviews Grids• Grid Application Areas within DoD is a general study of
applications of Grids in DoD• Complete list of References for Reports on Grids for GiG and
NCOW contains all references in above reports• Implementing some Grid Application Areas within NCOW 1.1 of
DoD is not started but is intended to write up specific applications such as demos
August 25 2005 Briefing [email protected] [email protected] [email protected] 16
Major Conclusions I• One can map “broadly” NCOW and GiG core capabilities
into Web Service (WS-*) and Grid (GGF) architecture and core services– Analysis of Grids in NCOW document inaccurate (confuse Grids
and Globus and only consider early activities)• Important “mismatches” on both NCOW and Grid sides• Grid/WS-* do not have messaging and collaboration• NCOW does not have system metadata and
resource/service scheduling and matching• Higher level services of importance include GIS
(Geographical Information Systems), Sensors and data-mining
• Criticisms of Web services in a recent paper by Birman seem to be addressed by Grids or reflect immaturity of initial technology implementations
August 25 2005 Briefing [email protected] [email protected] [email protected] 17
Major Conclusions II• NCOW does not seem to have any analysis of how
to build their systems on WS-*/Grid technologies in a layered fashion; they do have a layered service architecture so this can be done– In particular do not clearly endorse critical header/body
structure of SOAP messages
• Grid of Grids allows modular architectures and natural treatment of legacy systems
August 25 2005 Briefing [email protected] [email protected] [email protected] 18
Areas in Birman Paper• Information Architecture and Service Description
• Document-Centric Nature
• Time-critical Events
• Life Cycle Support for Services
• Reliable Messages
• Security
• Scalability
August 25 2005 Briefing [email protected] [email protected] [email protected] 19
Grids and HLA/RTI I• HLA through IEEE1516 has specified the interfaces for its key services
that are supported by RTI (Run Time Infrastructure)• HLA does not specify each message semantics or core system services
– RTI implementations are NOT interoperable although each one should support any HLA federation
– RTI implementations become a full distributed system environment as need metadata, reliable messaging etc. with simulation support only a small part
• Grids can be used in HLA with– Dynamic assignment of compute resources to support federates– Building web service interfaces to federates (XMSF)– Infrastructure to build a new generation of RTI that will use Web
system services and just add simulation support
August 25 2005 Briefing [email protected] [email protected] [email protected] 20
Grids and HLA/RTI II• HLA specifies
– Declaration management – achieved through use of publish/subscribe Grid Messaging (NaradaBrokering)
– Data Distribution management – corresponds to geometry sensitive publish and subscribe model (add to NaradaBrokering)
– Time management – corresponds to simulation framework (use best event driven and time stepped models – as infrastructure generic, one can support broad range of simulations including classic parallel computing and agent-based simulations)
– Object management - Very specific to HLA and should be built as per IEEE1516
– Ownership management - could use Grid virtualization and use metadata catalog catalogs to handle properties – might be generalizable
– Federation management - Could generalize to support of general simulation models (federates and federations are a general concept)
August 25 2005 Briefing [email protected] [email protected] [email protected] 21
Demo Technology Areas DTA1) NaradaBrokering: Grid Messaging2) Anabas Collaborative Applications3) GlobalMMCS/XGSP Collaborative Stream Management
Framework and A/V Conferencing4) Geographical Information Systems (GIS) Web Feature
Services (WFS)5) GIS Web Map Services (WMS) and Clients 6) GIS Sensor Enablement Services7) Wrapping of non Grid Applications as Grids of grids or
Grids of Services8) Information and meta-data Services 9) HPSearch Workflow and Management10) Portals and Portlets
August 25 2005 Briefing [email protected] [email protected] [email protected] 22
IN1 Data Mining and GIS Grid
WMS handlingClient requests
WMS Client
UDDI
WFS2
Databases withNASA, USGS features
SERVOGrid Faults
WFS1 NASA WMS
HTTP
SOAP
WFS3
Data Mining Grid
WMS Client
August 25 2005 Briefing [email protected] [email protected] [email protected] 23
California fault data from Quake Tables fault database via Web Feature Service.
Standard Open Geospatial Consortium WMS Clients
August 25 2005 Briefing [email protected] [email protected] [email protected] 24
Get Feature Info allows users to get map information. This can also be used to read feature info off the map when creating input data for applications
Standard Open Geospatial Consortium WMS Clients
August 25 2005 Briefing [email protected] [email protected] [email protected] 25
IN1 Data Mining Grid
HPSearchWorkflow
UDDI
Databases withNASA,USGS features
SERVOGrid FaultsWFS4
SOAP
WS-Context
WFS3
PI Data Mining
Filter
GIS Grid
Filter
NaradaBrokering
Pipeline
System Services
August 25 2005 Briefing [email protected] [email protected] [email protected] 26
PI demo combines WFS, WMS, and HPSearch for service orchestration.
Tool bar items allows you to adjust maps
Users set up problems by adding filtered seismic archives from WFS as map layers.
August 25 2005 Briefing [email protected] [email protected] [email protected] 27
Hot spots calculations--areas of increased earthquake probability in the forecast time-- calculations are re-plotted on the map as features.
August 25 2005 Briefing [email protected] [email protected] [email protected] 28
Typical use of Grid Messaging in NASA
Datamining Grid
Sensor Grid
Grid Eventing GIS Grid
August 25 2005 Briefing [email protected] [email protected] [email protected] 29
August 25 2005 Briefing [email protected] [email protected] [email protected] 30
Typical use of Grid Messaging
HPSearchManages
NaradaBrokering
Sensor Grid
WS-ContextStores dynamic data
Filter orDatamining
WFS
Post beforeProcessing
Post afterProcessing
Notify
SubscribeDatabaseArchivess
August 25 2005 Briefing [email protected] [email protected] [email protected] 31
IN2 Google Map Client
Google Central
Google Map Client
UDDI
WFS2
Databases withSERVOGrid Faults
WFS1
SOAP
Sensor Grid
HTTP
Helper Services
Archived Real Time
August 25 2005 Briefing [email protected] [email protected] [email protected] 32
IN2: Real Time GPS and Google Maps
Subscribe to live GPS station. Position data from SOPAC is combined with Google map clients.
Select and zoom to GPS station location, click icons for more information.
August 25 2005 Briefing [email protected] [email protected] [email protected] 33
Google maps can be integrated with Web Feature Service Archives to filter and browse seismic records.
IN2: Integrating Archived Web
Feature Services and Google Maps
August 25 2005 Briefing [email protected] [email protected] [email protected] 34
IN2: Integrating Archived Web
Feature Services and Google Maps
Google maps can be integrated with Web Feature Service Archives to browse earthquake fault records.
Faults are typically stored by segment number, so map interfaces are convenient for both verifying continuity and setting up input files for computing problems.
August 25 2005 Briefing [email protected] [email protected] [email protected] 35
IN2: Same Screens with Hybrid Maps
August 25 2005 Briefing [email protected] [email protected] [email protected] 36
IN3 Google Maps as a WMS Web service
Google Central
WMS handlingClient requests
WMS Client
UDDI
WFS2 …..
Databases withNASA,USGS features
SERVOGrid Faults
WFS1Google Maps
WMS (Gateway)
HTTP
SOAPOther
Servicesusing results
August 25 2005 Briefing [email protected] [email protected] [email protected] 37
Google Maps as Service
accessed from our WMS
Client
August 25 2005 Briefing [email protected] [email protected] [email protected] 38
CollaborativeStreaming
Grid
IN4 Server-side Streaming from a WMS
WMS handlingClient requests
UDDI
SOAP
XGSP MediaService
NaradaBrokering
WFS2
Databases withNASA, USGS features
SERVOGrid Faults
WFS1 NASA WMSWFS3
Data Mining Grid
August 25 2005 Briefing [email protected] [email protected] [email protected] 39
CollaborativeStreamingGrid
IN4 Shared Display Streaming from a WMS
WMS handlingClient requests
UDDI
SOAP
XGSP MediaService
NaradaBrokering
WFS2
Databases withNASA, USGS features
SERVOGrid Faults
WFS1 NASA WMSWFS3
Data Mining Grid
WMS Client
GMC
GMC = GlobalMMCS Client
August 25 2005 Briefing [email protected] [email protected] [email protected] 40
Web Service Collaboration
Web Service NaradaBrokering
WS1
WS2
WS3
NaradaBrokering
Shared Input Port with replicated services
Shared Output port with replicated recipients
August 25 2005 Briefing [email protected] [email protected] [email protected] 41
Pipelined Web Service Collaboration
• In a workflow, one can invoke collaborative streams on any flow and this splitting is between output port of one and input of next Web Service in chain
WS1
WS2
WS3
NaradaBrokering
WS4
WS5
WS6
WS-BWS-A
Shared Input Port
Shared Output Port
August 25 2005 Briefing [email protected] [email protected] [email protected] 42
Collaboration Grid
UDDI NaradaBroker
HPSearch
WS-Context
Gateway
WS-Security
NaradaBroker
NaradaBroker
Gateway
Gateway
Gateway
XGSP MediaService
Video Mixer
Transcoder
Audio Mixer
Replay
Record
Annotate
Thumbnail
WhiteBoard
SharedDisplay
SharedWS
August 25 2005 Briefing [email protected] [email protected] [email protected] 43
Annotation of GIS Maps from WMS
converted into H261 Video Stream
August 25 2005 Briefing [email protected] [email protected] [email protected] 44New GlobalMMCS Client
Chat
TV
WebcamVideo Mixer
GIS
August 25 2005 Briefing [email protected] [email protected] [email protected] 45
Collaborative Map Pages• Shared Input Port
Anabas Endpoint
XML Sharedlet interfacesimilar to WSDL Narada
Brokering
Anabas Endpoint
Anabas Endpoint
Google Central
Input to Googleweb page
August 25 2005 Briefing [email protected] [email protected] [email protected] 46Impromptu Google Interface
August 25 2005 Briefing [email protected] [email protected] [email protected] 47
Impromptu Google Map on Whiteboard
August 25 2005 Briefing [email protected] [email protected] [email protected] 48
Clients and Services• Thick clients can be Grid (WSDL) endpoints and
act as services
• Can break thick client up as MVC with Model becoming a service – Message-based MVC
ClientGrid ServiceSOAP
Model asService
Grid Service
SOAP
Portal
HTTP
Portlet View
August 25 2005 Briefing [email protected] [email protected] [email protected] 49
Collaborative “Legacy” Object• Shared Output Port with Anabas Endpoint providing
Service wrapping of RMI Object
Anabas Endpoint
XML Sharedlet interfacesimilar to WSDL Narada
Brokering
Anabas Endpoint
Anabas Endpoint
Java Distributed Object
RMI
Output from Java Object
August 25 2005 Briefing [email protected] [email protected] [email protected] 50
Collaborative Annotation of Map Pages• Shared Input Port with replicated white board
XML Sharedlet interfacesimilar to WSDL Narada
Brokering
Anabas WB Service
Google Central
White Boardchanges
Anabas Google
Endpoint
WB Service
Anabas WB Service
August 25 2005 Briefing [email protected] [email protected] [email protected] 51
Collaborative Search• Shared Input Port for annotation with replicated search
display (shared output port for search service)
NaradaBrokering
AnabasAnnotatable Text Service
Search Resultsand annotation
Anabas Search
Endpoint
Annotatable Text Service
AnabasAnnotatable Text Service
Google SearchYahoo Search MetaSearch
August 25 2005 Briefing [email protected] [email protected] [email protected] 52
Impromptu Yahoo Search
August 25 2005 Briefing [email protected] [email protected] [email protected] 53
Impromptu GoogleSearch
August 25 2005 Briefing [email protected] [email protected] [email protected] 54
Impromptu MetaSearch