1

OGSA-DAI: Status and Future Plans

Neil Chue Hong

2

Status

Project started 1st October 2003– DAIT Funding for 2 years

– EPCC, NeSC, Newcastle, Manchester

Industry Collaboration– IBM engaged at start of 2004

– Oracle and Fujitsu on Programme Board

People:– 5 FTEs at EPCC, 5 FTEs at IBM

– 1 PDRA at each of NeSC, Newcastle, Manchester

Releases:– 6 monthly major releases, next is April

3

3. OGSA-DAI/T Implementation

2. DAIS

1. Exploitation

Project cycle

4

MilestonesJan '04 -

Feb '04 -

Mar '04 -

Apr '04 -

May '04 -

Jun '04 -

Jul '04 -

Aug '04 -

Sep '04 -

Oct '04 -

Nov '04 -

Dec '05 -

Jan '05 -

Feb '05 -

Mar '05 -

Apr '05 -

May '05 -

Jun '05 -

Jul '05 -

Aug '05 -

Sep '05 -

Oct '05 -

Nov '05 -

Dec '05 -

Jan '06 -

Feb '06 -

Mar '06 -

Stable Database Services Specification and Primer put forward as a Proposed Recommendation

R3.1

R7.0

R5.0

R6.0

WS-RFAnnounced

R4.0

R3.1: Technical preview of parts of R4

R5: Compliance with DAIS?, distributed query and transactions, improved dependability and security, coordinated contributor community

R6: Features depend on user priorities, context and research

R7: Maintainable release for the user community

Stable Database Services Specification and Primer put forward as a Proposed Recommendation

5

Release 4

Built on top of Globus Toolkit 3.2– No major change to perform doc schema

Supported Client Toolkit library– Easier development of applications

Updated ActivityEngine– Have addressed some memory and performance issues

Additional DBMSs supported – SQL Server, Postgres

New GUI data browser– contributed by FirstDIG Project

Bulk load supported– Allows distributed join scenarios using data browser

6

Release 4

Secure Grid Data Transport implementation Documentation now in XHTML format Metadata registered with DAISGR Updated Exception Hierarchy Stored Procedure support DBMS Specific management operations

– Not DBMS transparent operations More file access prototypes User support, courses, training material Updated performance report

Release date: end April / early May

7

DAIT Roadmap R5

R5 October 2004– Possible alignment with WS-RF and DAIS Specs

• Assuming they settle in time

– At least a tech preview of OGSA-DAI and GT4– Possible WS-I interface implementation?– Distributed Relational Query Processing

• Looking to integrate the OGSA-DQP code

– Improved dependability and security integration– Extended & integrated XML and relational facilities– Transaction participation– Coordinated OGSA-DAI contributor community

8

R6 R7

R6 April 2005– Integrated with GT4?– Functionality driven by user group (i.e. YOU)– New facilities depend on user priorities, context and research– OGSA-DAI components from contributor community– Increased data integration tools

R7 October 2005– Maintainable release for the user community

9

OGSA-DAI and WS-RF

OGSA-DAI R4 will still be OGSI OGSA-DAI R5 will probably have a WS-RF

interface Currently investigating options for supporting

different interfaces in the future– WS-RF, WS-I, legacy OGSI, …

OGSA-DAI codebase and architecture makes it easier to port– But still costly to support multiple versions

10

Issues

Important facets that need to be taken into account:– Metadata

• Data structure, access policies, data content, provenance, physical properties, etc.

– Registries required for resource discovery/matching• Allow dynamic binding to data sources based on provision of metadata

– Different modes of operation/delivery• JDBC, ODBC, bulk data transfer, third-party data transfer, incremental

data transfer, delayed data transfer– Security mechanisms

• Authentication, authorisation, accounting, audit, privacy/encryption– Data transformation processes

• Reformatting, compression – Facilitate transaction / workflow participation arrangements

11

Protecting Your Users

Very rapidly moving field– Technology changes

– Standard changes

– Middleware changes

Need to ensure:– Technology adopters investment in

OGSA-DAI is protected

– Shielded from future change

Positives:– Document interface helpful

– Client toolkit

12

DAIS Documents

OGSA Data Services Document

Grid Data Services Document

RelationalRealisation

XMLRealisation

FileRealisation

OtherSpecialisations

PerformDocument

14

Supported Data Resources

Relational XML Other

MySQL Xindice Files DB2 eXist ?Oracle PostgreSQL SQLServer

15

Client Toolkit

Why? Nobody wants to write XML! A programming API which makes writing

applications easier– Now: Java– Next: Perl, C, C#?

// Create a querySQLQuery query = new SQLQuery(SQLQueryString);ActivityRequest request = new ActivityRequest();request.addActivity(query);

// Perform the queryResponse response = gds.perform(request);

// Display the resultResultSet rs = query.getResultSet();displayResultSet(rs, 1);

16

We have delivered several releases of OGSA-DAI There is much to be learned this year from taking the application view

Tasks– User Group Meetings

• Input to DAIS and DAI/T implementation

– Support to projects • Skyserver, eDiaMoND, SIMDAT etc..

– Publications and presentations• Globus World, GGF Workshops, All Hands etc

– Surveys and questionnaires

Exploitation

18

Requirements

What do you consider important?– Data access– Data integration– Application support– Service Extensibility– Performance– Security– Useability– User Support

19

Versioning

Which is more important?– OGSI / WS-RF /WS-I– DAIS specifications– Globus Tookit– Java– Java libraries– Databases– RowSet– XML Schema– Client Toolkit API– OGSA-DAI interfaces

How many versions should we support?

20

Platforms

Which is more important?– Ease of installation and support– Ability to access underlying platforms

Black box versus full extensibility– How many people need to know whether its running on

Tomcat? On Axis?

21

Interfaces

Which is more important?– Continued support for OGSI– Port to WS-RF– Port to WS-I– Port to…

Web Services– WSDL/SOAP/OGSA– REST– …

22

Data Integration

Which is more important?– Distributed Query Processing– Distributed Queries– Distributed Data Management– Federated databases– Virtual federation– Data virtualisation– Data provenance– Transactions

23

Languages

Which is more important? Query languages

– SQL (92, 99, …), XPath, XQuery, RegExp, …

Programming languages– Java– C / C++– C#– Fortran– Perl / Python– …

24

Security

Which is more important?– Transport Level Security– Message Level Security

– Standards (GSI, GSS, Kerberos, WS-Security, …)

– Scalability– Robustness– Performance– Definition– Management– Usability

25

Useability

Which is more important?– Client libraries for usability– Exemplar client framework– Additional Client Toolkit Activity implementation– Support for Client Toolkit version changes– Graphical components

26

User Support

Which is more important?– Continue support for community

• via web site, mail list, announce, help, consultations

– Training courses – Talks, demonstrations and presentations– User documentation– "How To" guides– Installation and configuration wizards– Different Installers and Uninstallers (e.g. WAR, …)

27

Requirements (again)

Which is more important?– Data access

• SQL refactoring• Interoperability of results

– Data integration• Schema integration tools• Data description tools

– Application support– Service Extensibility– Performance– Security– Useability– User Support

28

Summary

Many paths which OGSA-DAI can take– Which one to take

Future releases – need to be driven by your requirements– You need to tell us

It is a trade off– No infinite development team– Port versus support– Functional and non-functional requirements