Experiments with Remote Access to Scientific Data using

HDF5

Kun YanRobert E. McGrath(mcgrath@ncsa.uiuc.edu)

NCSAMay 15, 2002

Today’s Talk

1. Background

2. Experiments with JSP, XML, HDF4, HDF5, etc.

3. Experiments with CORBA, Java, HDF5

4. Discussion

Acknowledgements

Kun Yan conducted these experiments as a Research Assistant at NCSA.

This report is based upon work supported in part by a Cooperative Agreement with NASA under NASA grants NAG 5-2040 and NCC5-599. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Aeronautics and Space Administration.

Other support provided by NCSA and other sponsors and agencies.

Java Native

Interface

HDF Libraryand File

H5view,etc

C

Java

Java

C

Applet

Java

CORBAServer

OtherApp. Other

App.

C++

Any

Any

Client/Remote Server/Local

Java Server

Platform

Webbrowser

HTML

XML

Java

Any

Java Native

Interface

Java C

Distributed Product

Demonstrated in Research

Should work, but not demonstrated

2. Experiments with JSP

• Experiments with JSP, XML, XSL, etc.– Web access to Data and services.– Demonstration of browsing HDF5 datasets

• Generate HTML or XML• Convert data from HDF4 to HDF5 on demand

http://arabica.ncsa.uiuc.edu:7070/hdf5/

Tomcat

• is a standalone web server that supports the Servlet and JSP specifications.

• developed under the Jakarta project at the Apache Software Foundation.

• integrated into the J2EE Reference Implementation.

• Tomcat 4.x (Catalina, Servlet 2.3, JSP 1.2)

• Tomcat 3.3

JavaServer Pages

• JavaServer PagesTM (JSPTM) technology

• rapidly develop and easily maintain, dynamic web pages

• uses XML-like tags and Java scriptlets

• application logic can reside in server-based resources

The Flow of a Request

beans

request

response

clientJSP

DB datalib

XSL

XMLbeansbeans

Xalan.jarXerces.jar

HTML

response

How JSP and Beans Work together

Application

3. Experiments with CORBA

• Created a demonstration CORBA server that reads HDF5– Written in C++, linked directly to HDF5 library.

• Created a pure-java client– No CORBA required on the client– Can be used from an applet within a standard

Web browser

Technology used

• Orbacus 4.1.0 (C++) (free, full source)

• Java 1.3 or above

• HDF5.1.4.3, HDF5 C++ interface

HDF5 CORBA Experiment -- Design DiagramCORBA Object Hierarchy

HDF5 CORBA Experiment – IDL File1. module HDF5{

2. struct ObjInfo{3. long oid;4. long fid;5. long type;6. string name;7. long numOfMembers;8. };9. 10. typedef sequence<long> Sizes;

11. struct DatasetInfo{12. string name;13. long numOfDim;14. Sizes dimSizes;15. Sizes dimMaxSizes;16. long dataTypeClass;17. long dataTypeSize;18. long long storageSize;19. };20. 21. struct DatatypeInfo{22. string name;23. long type;24. long size;25. };

26. interface H5Obj{27. void getObjInfo(out ObjInfo obji);

28. void getNumAttributes(out long attrNum);29. void openAttributes (out long attr);30. };31.

32. interface H5DatasetObj:H5Obj{33. void getDatasetInfo(out DatasetInfo dsinfo);34. long long getStorageSize();35. };36. 37. interface H5GroupObj:H5Obj{38. ObjInfo getGMember(in long fid, in string gName, in long index); 39. void openGroup(in string gName, out H5GroupObj h5gobj);40. void getDatasetInfo(in string dsName, out DatasetInfo dsinfo);41. void openDataset(in string dsName, out H5DatasetObj h5ds);42. void getDataTypeInfo(in string ttName, out DatatypeInfo dtinfo);43. };44. 45. interface H5FileObj{46. void getTOC(out ObjInfo obji);47. void openRootGroup(out H5GroupObj h5gobj); 48. };49. 50. interface H5FileAccess{51. void openH5File(in string fileName, out H5FileObj h5fobj);52. void numOfGMembers(in long fileId, in string gName, out long

gNum);53. string closeH5File(in long fileId);54. };55. 56. };

HDF5 CORBA Experiment -- Design Diagram C++ Servant Object Hierarchy

H5FileAccess H5FileObjH5Obj

H5GroupObjH5DatasetObj

H5FileAccess_impl

H5::H5Filepoa_

openFile:closeFile:

H5FileObj_impl

H5::H5FileObjInfopoa_

getTOCopenRootGroupsetFileName: H5GroupObj_impl

H5::H5Grouppoa_

openGroupgetGMembersopenDataset:getDatasetInfo:getDatatypeInfo:

H5Obj_impl

ObjInfopoa_

getObjInfosetObjName:

H5DatasetObj_impl

H5::H5DarasetDatasetInfo

getDatasetInfogetStorageSize

HDF5 CORBA Experiment -- How to Use the Client Applet

• Select file name from drop-down list (file resides corba server side)

• Select Flags and Access Property of the selected file

• Click on “open file” to open the file

• Click on “get file TOC” to get the file object info

• Click on “open root” to open the root group

• Click on “get group TOC” to get the root group object info

• Click on “iterate group” to get the group member’s info of the root and each subgroup

• Input dataset name including path, then click on “get dataset” to get the dataset object

– Click on “get dataset TOC” to get the dataset object info

– Click on “get storage size” to get the storage size of the opened dataset object

• Input dataset name including path, then click on “get dataset info” to get object info of the given dataset

• Input datatype name including path, then click on “get datatype info” to get object info of the given datatype

URLs

• JSP experiments: http://hdf.ncsa.uiuc.edu/HDF5/XML/JSPExperiments/

• http://hdf.ncsa.uiuc.edu

• http://hdf.ncsa.uiuc.edu/h4toh5/

• http://hdf.ncsa.uiuc.edu/HDF5/XML/

• http://hdf.ncsa.uiuc.edu/hdf-java-html

4. Discussion

Java Native

Interface

HDF Libraryand File

H5view,etc

C

Java

Java

C

Applet

Java

CORBAServer

OtherApp. Other

App.

C++

Any

Any

Client/Remote Server/Local

Java Server

Platform

Webbrowser

HTML

XML

Java

Any

Java Native

Interface

Java C

Distributed Product

Demonstrated in Research

Should work, but not demonstrated

HD

F5

CO

RB

A E

xperim

ent -- U

ser Inte

rface

HDF5 CORBA Experiment -- Examples of Object Info window

Other HDF ToolsOther HDF Tools

Viewer/Editor

GeneratorSource code

ASCII

HDF5 file

HDF5 file

XML

XML

XML

HDF5 file

Java

HDF4

h4toh5

HDF4

Dumper

HDF DDL

HDF5 file XML

Non-Java

(C)

h4toh5HDF4 file HDF5

h5toh4HDF5 file HDF4

gif2hdfGIF file HDF5

fp2hdfflat file HDF5

H4 to H5 LIbrary

Silly XMLTricks

Netcdf to HDF5, via XML using XSL

netcdf ncxdump nc.xmlnetcdf.dtd

h5.xml

xalan

nctoh5.xsl

h5genhdf5

URLs

• JSP experiments: http://hdf.ncsa.uiuc.edu/HDF5/XML/JSPExperiments/

• http://hdf.ncsa.uiuc.edu

• http://hdf.ncsa.uiuc.edu/h4toh5/

• http://hdf.ncsa.uiuc.edu/HDF5/XML/

• http://hdf.ncsa.uiuc.edu/hdf-java-html