20
Mike Jackson EPCC michaelj@ epcc .ed.ac. uk OGSA-DAI Architecture + Extensibility OGSA-DAI Tutorial GGF17, Tokyo
Mike JacksonEPCC
OGSA-DAI Architecture +
ExtensibilityOGSA-DAI Tutorial
GGF17, Tokyo
11 May 2006 http://www.ogsadai.org.uk/ 2
O G S A -D A I
Outline
• “Let us out”– Exposing data to clients – the server’s perspective
• “Let us in”– Getting to the data – the client’s perspective
• “More, more more…”– Extending OGSA-DAI
11 May 2006 http://www.ogsadai.org.uk/ 3
O G S A -D A I
Data resources
• Relational– MySQL– Microsoft SQL Server– Oracle– IBM DB2– PostGres– HSQL
• XML– eXist– Xindice
• File system– SwissPROT– OMIM– Text– Binary
Relational
XMLDB
Files
11 May 2006 http://www.ogsadai.org.uk/ 4
O G S A -D A I
Data resource accessors
• Interfaces between data resources and OGSA-DAI
• Relational– JDBC drivers– java.sql.*
• XML– XMLDB API and compliant drivers– org.xmldb.api.*
• File system– Java file and directory utilities– java.io.*
• OGSA-DAI extensibility point
Data Resource Accessor
Relational
XMLDBData
Resource Accessor
Data Resource Accessor
Files
11 May 2006 http://www.ogsadai.org.uk/ 5
O G S A -D A I
Data service resources
Data Resource Accessor
Relational
XMLDBData
Resource Accessor
Data Resource Accessor
Data Service
Resource
Files
Data Service
Resource
Data Service
Resource
SQLOne
XMLOne
FilesOne
11 May 2006 http://www.ogsadai.org.uk/ 6
O G S A -D A I
Data service resources
• OGSA-DAI’s core functionality
• Manages– Access to a data resource via a data resource accessor– Execution of data-related activities– Data caching and streaming of data to and from clients– Creation, access and termination of sessions
• Exposes data service resource properties– Information about a data resource– Information about supported activities– Information about current requests
11 May 2006 http://www.ogsadai.org.uk/ 7
O G S A -D A I
Requests and responses
Data Resource Accessor
Data Service
Resource
SQLOne
Relational
SQL Query SQL Query
Results
Perform Document
Response Document
ResultSet
11 May 2006 http://www.ogsadai.org.uk/ 8
O G S A -D A I
Requests and responses
• Request– A connected collection of activities that the data resource should
execute– XML perform document submitted by a client
• Activity– An individual data-related operation– 0 or more inputs and 0 or more outputs
• Response– Status of execution of a request possibly with result data– XML response document returned to a client
• OGSA-DAI engine– Parses requests, executes activities, builds responses
11 May 2006 http://www.ogsadai.org.uk/ 9
O G S A -D A I
Activities
• Relational– SQL query, update, stored procedure, bulk load, extract logical and physical
schema
– Convert ResultSet to WebRowSet, ResultSet to CSV, ResultSet to bytes, relational database schema to XML
– Project ResultSet or WebRowSet onto a column
– Extract bytes from ResultSet
• XMLDB– Resource and collection management, XPath, XQuery, XUpdate, bulk load
• Files– List directory, create, read, write and update files
– Index files, search indexed files
11 May 2006 http://www.ogsadai.org.uk/ 10
O G S A -D A I
Activities
• Transformation and Compression– GZIP compression, ZIP archive– XSLT– Project CSV data onto a column– Distribute numerical data onto spaces– Create random sample of data
• Delivery– From and to URLs, files, GridFTP, remote data service resources, SOAP attachments– To servlets, SMTP, resource properties
• Factory– Create and destroy persistent and transient data service resources
• Relational multi-resources– Bag and resilient queries
• OGSA-DAI extensibility point
11 May 2006 http://www.ogsadai.org.uk/ 11
O G S A -D A I
Data services
Data Resource Accessor
Relational
XMLDBData
Resource Accessor
Data Resource Accessor
Data Service
Resource
Files
Data Service
Resource
Data Service
Resource
SQLOne
XMLOne
FilesOne
Data
Service
11 May 2006 http://www.ogsadai.org.uk/ 12
O G S A -D A I
Data services
• Web services
• Expose 0..N data service resources to the outside world
• Two flavours– OGSA-DAI WSRF services
– Compliant with the Web Services Resource Framework
– Implemented using Globus Toolkit (4.0+)
– OGSA-DAI WSI services
– Compliant with vanilla WSDL
– Implemented using Apache Axis (1.2.1 or 1.2RC3)
• Service-level operations– GetVersion – which version of OGSA-DAI is running
– ListResources – which data service resources does a service expose
– Deploy / Undeploy
– Expose / withdraw data service resources to / from the outside world
11 May 2006 http://www.ogsadai.org.uk/ 13
O G S A -D A I
Data services
• Data service resource-specific operations– Perform
– Receive requests for a data service resource and return responses
– GetFully / GetNBlocks / GetBlock – Stream data from a data service resource
– PutFully / PutBlock– Stream data to a data service resource
– GetResourceProperty / GetMultipleResourceProperties / QueryResourceProperties
– Expose information about the data service resource state
11 May 2006 http://www.ogsadai.org.uk/ 14
O G S A -D A I
Clients
Data Resource Accessor
Data
Service
Data Service
ResourceClient
Perform Document
SQLOne
Relational
Perform Document
SQL Query
ResultSet
SQL Query
ResultsResponse Document
Response Document
Authorization points
11 May 2006 http://www.ogsadai.org.uk/ 15
O G S A -D A I
Identifying a data service resource
Data
Service
Data Service
ResourceClient
RequestSQLOne
SQLOne
http://host:port/services/axis/DataService/DAISQLOne
http://host:port/services/wsrf/DataService
Request
11 May 2006 http://www.ogsadai.org.uk/ 16
O G S A -D A I
Clients and the client toolkit
• Clients interact with data services via SOAP over HTTP– Deduce service interface from service WSDL description– Construct SOAP request to invoke operation– Parse SOAP response from service– Resource identification scheme must be assumed from WSDL namespace
• OGSA-DAI client toolkit:– Construct and submit requests in Java not XML
– Toolkit handles SOAP request construction and response parsing– Renders OGSA-DAI service types transparent– Java abstractions of
– Data services– Data service resource IDs and session IDs– Requests and responses– Activities
11 May 2006 http://www.ogsadai.org.uk/ 17
O G S A -D A I
Relational multi-resources
Multiple Data Resource Accessor
Multi Resource
Data Service
Resource
MultiOne
Data
Service
Data
Service
Relational
SQL Query
Multi Results
SQL Query
Results x M
Data Service
Resource
Data
Service
Relational
SQL Query SQL QueryResults Results
SQL QuerySQL Query ResultsResults
SQL Query SQL QueryResults Results
11 May 2006 http://www.ogsadai.org.uk/ 18
O G S A -D A I
Factory resources
Factory Data Resource Accessor
Factory Resource
FactoryOne
Data
Service
Resource
Create Resource
InstanceOne
Instance Data Resource Accessor
Instance Resource
InstanceOne
InstanceOne
11 May 2006 http://www.ogsadai.org.uk/ 19
O G S A -D A I
Extending OGSA-DAI
• Data resource accessors– Use your own data resources within OGSA-DAI– Expose local or remote data resources– Expose virtual resources created by aggregation or integration
• Resource management– Creation and destruction of persistent and transient data service resources– Another type of data resource accessor
• Activities– Use your own activities that can be executed by a data service resource– Can be resource specific e.g query or update– Or generic e.g. transformation, compression, delivery, resource management, monitoring
• Authorization– Resource access – Activity execution– In application-specific ways
11 May 2006 http://www.ogsadai.org.uk/ 20
O G S A -D A I
Summary
• The OGSA-DAI layer cake– Data– Data resource accessors– Data service resources– Data services– Clients
• Extending OGSA-DAI– Data resource accessors– Activities– Authorization
