Date post: | 18-Dec-2015 |
Category: |
Documents |
Upload: | milton-washington |
View: | 217 times |
Download: | 0 times |
Federated Service Oriented Federated Service Oriented Information ManagementInformation Management
Ahmet Sayar
Introduction
Aim is utilizing distributed heterogeneous information and knowledge provided by different repositories and vendors in an efficient and robust manner.
No agreed upon –useful- architecture framework for Federating Obtaining Analyzing Interpreting
the heterogeneous distributed data/information for decision makers in scientific application domains
Motivation SOA based on Web Services Information Sources are “Filters”:
A service inputs DIKW (Data-Information-Knowledge-Wisdom Hierarchy) from Grid and outputs DIKW
Web Services, easy to extend and federate. Easy to publish, located and bind. predictable input and output interfaces and defined by metadata
Information management through ASIS (Application Specific Information System) framework in Science Domains such as GIS.
Data and metadata concepts and formats A repository or sensor has or gets DIKW from
"outside Grid"; it outputs DIKW
Problem Recognition
Vector data
Bitmapdata
netCDF
Bar graphs
Coveragedata
Imagejpeg
XMLdata
Statisticsdata
Plots images
Binarydata
Interactive Tools
DB
DB
DB
DB
DB
DB
DB
DB
Raw Data
Data
Information
Knowledge
Wisdom Decisions
SS
SS
SS
SS
SS
HDF5
Problem Recognition Services like discovery and notification do not need to be made
application specific. BUT If the domain changes then :
choices, Database requirements, data format, core service requirements, attributes, and metadata context
CHANGES ! What are the common concepts and characteristics for
Data, Metadata, Query Language, Services, and Communication language,
in order to drive information/knowledge from the heterogeneous data/information sources in Application Domains ?
Overall Structure Solution ASL : Application Specific Language. XML based
hierarchical data representation format. Cross language, platform and operating system
ASVS : Application Specific Visualization System Last filter before the decision maker. Provides information/knowledge in human readable formats
ASFS : Application Specific Feature Service. Stores and provides common data model (ASL)
Treat binary and common data (in ASL) differently.
ASFS
AS“Sensor”
AS Tool(generic)
AS Service(user defined)
AS Tool(generic)
ASVSDisplay
Message Using ASL
ASRepository
Overall Structure Solution -cont Common data (in ASL) is kept in ASFS. Enables
interactive querying through GUI. Tentative architecture. In the DIKW world, everything is mixed as data
and filters In a given domain every filter speaks in ASL ASVS both visualize information and provide a
way of navigating ASFS and their underlying DB. ASVS can itself be federated and present output
interface. GIS and Astronomy have some standards but not
many others have
Example (1):GIS Domain (OGC)
FS-2MD
Vector data
FS-1
Capabilities
Meta-data
FilteringModule
-Core Service-
FS
Raster data
FS-3
FS-4
Interactive Decision Support
Data
capability
FS-1 : Master Filter (WMS) Providing the available data list and
capabilities to the end user clients - Interactive tools
FS-2 : Web Feature Server Provides vector data such as rivers, state
and city boundaries in GML FS-3 : Web Map Server
Provides image data in the form of jpeg, svg, png etc. Defined in its capabilities file
FS-4 : Web Coverage Server Provides coverage (raster) data. Grided
data, pixel info Query : No Standard – Filter specification –
SQL Data Encodings : GML, images Metadata : capability doc. No event notification – we use WSContext for
asynchronous run Registry : WRS – MD Queryable Data in : WFS
(Nasa)
(CGL)
(CGL)
(Minnesota)
Data:a
Data:b
Data:b Data:c
Data:a Data:b Data:c
PORTAL
Data:a Data:b Data:c
Example (2):Astronomy Domain (IVOA)
FS-2
DB
FS-1
DB
FS-3
FS-1 : VOPlot Integrating, Interacting
visualization tools FS-2 : SkyNode
ADQL based SOAP interface returning VOTable based results
FS-3 : SIA 2D sky projection, logically a grid of
pixels encoded as a FITS image FS-4 : SSA
URL-based returning a dataset "document" (VOTable)
Query : ADQL –extension of SQL Data Encoding: VOTable, FITS Metadata : UCD, VOResource Event notification : VOEvent Registry : VORegistry QueryableData in : SSAP and SIAP,
VOStore
DB
FS-4
MD
Interactive Decision Support
Data
capability
PORTAL
Interactive Decision Support Tools- Interactive query,- Interactive display, movie and animation- Integration to Application Science Simulations
http://virtualsky.org (R. Williams et al.)
Issues To Be Discussed (1) Requirements for the domain metadata in
capability What does capabilities do and need to have to
federate filters?
Requirements for the ASL (such as CML, GML) What does ASL need to have to federate the filters?
Concept of data (such as feature, coverage) Common representation? Possible? To what extend?
A common information management framework which can be applied to any domain. some instructions- any field, what needs to be done
Issues To Be Discussed (2) Application level data/information federation Integrating the system with application science
simulations. Creating interactive decision support tools
utilizing integrated filter services. Tools for map animation, map movies, images Interactive query support to get further information on
the image and/or animation. Enabling binding of services into pipelines with
or without human intervention through metadata. Caching and load balancing to handle large
scientific data in an efficient and robust manner (application based)
Summary of SRB & Ogsa-DAI SRB
Storage Resource Broker Uniform access to dist. heterogeneous data resources by attributes Catalog service is MCAT (Metadata Catalog Service) Resource and data location transparency Remote authentication authorization – user groups Not just for access, transferring and replicating Sample projects using SRB: BIRN and NASA IPG
Ogsa-DAI Open Grid Service Architecture - Data Access and Integration Access to heterogeneous data via common interfaces on the grid. Catalog service is MCS (Metadata Catalog Service) OGSI-compliant Grid Components are Grid services. Resources should be registered. Sample projects using Ogsa-DAI : LEAD, MyGrid
Discussions on SRB & Ogsa-DAI
SRB Monolithic – does too much MCAT dependent MCAT has limited support for application-level metadata
Need diff metadata for diff domain, and extensions for applications Not standard based – Not open source Not handling data based on DIKW hierarchy
Ogsa-DAI At the data and Database level MCS dependent MCS has limited support for application-level metadata
Need diff metadata for diff domain, and extensions for applications For Grid applications - GGF standards Data only in relational and XML database or ordinary files Not handling data based on DIKW hierarchy
Our Work Compared to SRB & Ogsa-DAI
MasterSRBOgsa-GDSF
R R R
FS
R R R
FS FS
FS FS
FS
Wisdom decisions, knowledge and information extraction by the user
-Reusable components Filter Services with specific ports and interfaces
-Distributed DIKW abstraction
-Metadata in capability document
-Metadata aggregators
-New metadata for different domains
-User uses just getData interface to query
Ready to use information and knowledge
-Central data access abstraction. Uniform access to heterogeneous data sources
-Metadata : SRB/MCAT, Ogsa-DAI/MCS
-Both provides extensible metadata arch for diff domains
-SRB has “zone” concept address similar issues but different
Wisdom decisions
Information/knowledge
Data access and query
SRB Agents Ogsa/GDS
Why are we different ? SOA (Service Oriented Architecture)
Easy to extend Reusable components Cross platform and language. XML based hierarchical data representation
Easy data integration Easy querying Human readable information
Easy to access data – no command line Interactive tools On the fly query creation.
Not only accessing data but also transforming through its path to end users.
Ports to integrate application simulation to application specific information system (ASIS)
Contributions Instructions how to build ASL and metadata
in capability for the application sciences. Instructions how to build application specific
information system (ASIS) federating multiple filters speaking ASL.
Information grid (ASIS) formalization through capabilities metadata, defining all the data/information sources as interacting Web Service filters with standard metadata service ports.
Optimize and enhance the distributed heterogeneous information management.