CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

ESA return from experienceESA return from experience

&

f

Stéphane MbayeStephane.Mbaye@gael.fr

Mathias MouchaMathias.Moucha@gael.fr

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Context

Support to XFDU Specifications

Historical Archive Rationalization (HARM)

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Support to XFDU Specifications

(Among other requirements)• Edit/Review « XFDU Construction Rules » releases• Edit/Review « XFDU Green Book » releases• Critical analysis of the standard regarding ESA needs• Provide an early implementation of the standard• Produce sample packages• Perform interoperability testing with other agencies

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Historical Archive Rationalization (HARM)

• ESA Project: Historical Archive Rationalization and Management (HARM)

• Consortium: ACS, GAEL, Werum, DLR• (Among other requirements)

• Develop a Standard Archive Format for Europe (SAFE)• Support all ESA archived products (ESA and third party missions)• Conform to modern standards including OAIS RM and W3C.

• Develop an API for handling SAFE products• Java and C++ API• Read/Write SAFE Products• Convert native formats to SAFE formats

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Mutual Interests of the activities

• CCSDS/MOIMS/IPR XFDU• HARM may provide a testbed for improving « XFDU

Construction Rules »• SAFE I/O API or any of its component may provide an alternate

implementation of XFDU standard as required by the CCSDS• SAFE test data may provide IPR working group with meaningful

sample data

• HARM Project• SAFE may benefit from studies and international consensus

conducted by CCSDS agencies• SAFE may conform to XFDU and, therefore, tackle with the

CCSDS recommendation: « Each Agency is, however, strongly encouraged to direct planning for its new standards and implementations towards the later version of the Recommendation »

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Decision: Joint Activities

• SAFE shall follow the « XFDU Construction Rules »• SAFE sample and test data should be proposed to the

IPR as XFDU sample data• SAFE I/O API shall be split in two separate layers:

• An independent XFDU I/O API (only those features required by SAFE will, however, be implemented)

• A top level API implementing SAFE specific features and converters

• « SAFE Control Book Volume 2 – Recommendation for Specialisations » shall be written to help the editing of « XFDU Green Book »

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

SAFE Core Specifications

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Information Model

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Logical Model

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Physical Model

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Manifest - Internal Constraints

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Manifest - Identification of Product Types

<xfdu:XFDU xmlns:xfdu="http://www.ccsds.org/xfdu/2004"

xmlns:safe="http://www.esa.int/safe/1.0"

xmlns:xs="http://www.w3.org/2001/XMLSchema"

xmlns:envisat="http://www.esa.int/safe/1.0/envisat"

xmlns:xlink="http://www.w3.org/1999/xlink"

xmlns:gml="http://www.opengis.net/gml"

version="esa/safe/1.0/envisat/gomos/nl/level-0">

[...]

</xfdu:XFDU>

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Manifest - Reference Informations

« SAFE requires Representation Information of each Data Object »

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Manifest – Multiple Rep. Info’s

« Representation Information may be composed of several components »

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Manifest – Binary Metadata

« XFDU Recommends to refer to Binary Metadata Components through a Data Object »

« XFDU requires that all Data Objects should be referenced from the Information Package Map »

« The second version of XFDU may simplify the mechanism for describing object to object associations »

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

SAFE Specializations

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Specialization - Definition

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Abstract specializations

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Abstract specializations - Example

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Redefinitions and Imports

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Example of SAFE Product’s Manifest

• Refer to « SAFE Control Book Volume 2 – Recommentation for Specialisations », Appendix A

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

ESA/XFDU I/O Java/C++ API

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Overview

JNI Interface

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Use cases

• Create of an XFDU package

• Open an XFDU package

• Remove an XFDU package

• Move an XFDU package considering referenced objects

• Validate an XFDU package including the object contents

• Verify fixty i.e. Components checksums

• Add/Remove/Browse Content Units of the Information Package Map

• Add/Remove/Retrieve Metadata Objects

• Add/Remove/Retrieve Data Objects

• Apply Transformation Objects

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Java Packages

• esa.xfdu General XFDU packages management.

• esa.xfdu.map XFDU Information Package Map Section

• esa.xfdu.object Classes handling all XFDU object types.

• esa.xfdu.object.metadata XFDU meatadata object types.

• esa.xfdu.object.data Classes handling XFDU data object types.

• esa.xfdu.object.behavior Classes handling XFDU behavior object types.

• esa.xfdu.object.behavior.impl Behavior Implementations

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Main Java Interfaces

• esa.xfdu.object.Object• esa.xfdu.object.metadata.MetadataObject

• esa.xfdu.object.data.DataObject

• esa.xfdu.object.behavior.BehaviorObject

• esa.xfdu.object.behavior.Mechanism• esa.xfdu.object.behavior.Transformer

« The second version of XFDU may gather the means for referencing Metadata/Data/Behavior objects in a common object list. »

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

C++ Wrapper

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Current Version

xfdu-1-0-rc-10(April 28, 2006)

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Code Statistics

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Further information

http://www.gael.fr/xfdu

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Connected Application

• CNES has developped a GUI on top the XFDU I/O Java API: the « XFDU View »

• C.f. http://www.gael.fr/xfdu-view

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

ESA/SAFE I/O Java/C++ API

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Overview

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Use cases

• Create of a SAFE product

• Open a SAFE product

• Remove a SAFE product

• Move a SAFE product considering referenced objects

• Validate a SAFE product including the object contents

• Verify fixty i.e. Components checksums

• Add/Remove/Browse Content Units of the Information Package Map

• Add/Remove/Retrieve SAFE Specialized Metadata Objects

• Add/Remove/Retrieve SAFE Data Objects

• Browse XML and Binary Data Object Contents

• Accelerate Content access using SAFE Indexes

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Java Classes

esa.safe.object.metadata.AcquisitionPeriod

esa.safe.object.metadata.CloudVoteNotationSystem

esa.safe.object.metadata.Container

esa.safe.object.metadata.CorruptedCause

esa.safe.object.metadata.CorruptedElements

esa.safe.object.metadata.Facility

esa.safe.object.metadata.Frame

esa.safe.object.metadata.FrameSet

esa.safe.object.metadata.FrameTile

esa.safe.object.metadata.GridReference

esa.safe.object.metadata.Instrument

esa.safe.object.metadata.Line

esa.safe.object.metadata.Location

esa.safe.object.metadata.MissingElementsesa.safe.object.metadata.OrbitNumberesa.safe.object.metadata.OrbitReferenceesa.safe.object.metadata.Platform esa.safe.object.metadata.Point esa.safe.object.metadata.Processing esa.safe.object.metadata.QualityInformation esa.safe.object.metadata.Resource esa.safe.object.metadata.Softwareesa.safe.object.metadata.Time

esa.safe.Safeesa.safe.SchemaValidator

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

C++ Wrapper

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Current Version

safe-1-0-rc-13(June 10, 2006)

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Code Statistics

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Further information

ESA Official Web:

http://earth.esa.int/SAFE

GAEL’s Project Web:

http://www.gael.fr/safe

CCSDS MOIMS Springs Meeting 2006 – Rome - June 2006

XFDU & SAFE - ESA return from experience

Conclusions

• SAFE Specifications verified that XFDU is suitable for some amount of EO data sets (59)

• An alternate XFDU implementation is available (C++/Java) and interoperates with NASA API

• Up to 59 EO sample packages have been generated• SAFE Volume 2 is ready to be conveyed to the XFDU

Green Book• An XFDU v2 may improve the v1, at least, by:

• Simplifying the association mechanism between Objects• Generalizing Metadata/Data/Behavior Object references

• Future of XFDU and SAFE within ESA: c.f. ESA Agency Report