WDCs and GSDI

David M. ClarkWorld Data Center PanelGlobal Data Access and Integration WorkshopMay 8-9, 2000, Canberra, Australia

The World Data Center System

Guiding Principles

• Open to all scientists

• Non-discriminatory access to data

• Data exchanged free or for the cost of reproduction

• Host institutions support long-term operations

Where are the WDCs?

• 48 WDCs worldwide

• China, Russia, US, Japan, United Kingdom, The Netherlands, Denmark, Belgium, Czech Republic, France, India, and AUSTRALIA

• Disciplines cover entire range of Earth and Space Science

• New WDC applications are in progress

• Organized under and sanctioned by national ICSU adherent body

WHAT MAKES IT WORK?• Open access to data

• Data with scientists

• Scientists with data

• Coupling scientific and data management expertise

– Database compilations

– Database integration

– Product development

– Data publishing

• Long-term commitment

– Archive

– Infrastructure

WDC Global Infrastructure

• WDCs are a loose federation of Data Centers

• All operating under basic high level principles

• Disciplines are the foundation of the WDC system

• “Standards” are currently implemented between disciplines

• WDC disciplines developing a “network” infrastructure– SPIDR, Paleoclimatology, Geomagnetism

• Factors– Implement standards of National sponsoring agency

– Resources limit level of implementation

The Space Physics Interactive Data Resource II (SPIDR)

Promoting international scientific data exchange through enabling technology.

SPIDR II “Wolf” version

Data Sets DescriptionDMSP Polar Satellite Imagery

Visible/IR,Global coverage since 1992.

IONOSPHERIC F2, F1, E, Es, h’F, hpF2, hmF2, HcIonospheric stationsGlobal coverage since 1942

GEOMAGNETIC Geomagnetic stationsX,Y,D,H,ZGlobal coverage at 1.0 minand 1.0 hr since 1942

GOES X-Ray, Electrons, Magnetic Field, Ions.Multiple satellites.Data at 1.0 and 5.0 minutes since 1986.

IMF Magnetic Fields, Ion Density, Flow SpeedOMNI Data setHourly data since 1963.

SOLAR INDICES Kp, Ap, Cp, C9, Dst, SSn, 10.7 cm fluxLenhart Data setDaily and 3 hrs since 1942

SPIDR IIObjectives & Principles

• Provide a uniform interface to STP data.

• Provide an open extensible architecture.

• Provide intelligent services to handle ever expanding data archives.

• Provide integrated tools to the end user.

• Meet the needs of the STP data user.

• Develop using the “open source” paradigm.

• Provide visualization tools for all data sets.

• Employ “intelligent” computing techniques, such as neuro-fuzzy applications.

• Provide automatic reliable data mirroring.

• Respond to the needs of the STP user community.

WDC for Paleoclimatology Mirror Sites

WDC for PaleoclimatologyBoulder, Colorado, USA

Médias-France: Toulouse, France

University of Nairobi: Nairobi, Kenya

University of the Witwatersrand: Johannesburg, South Africa

Centro Regional de Investigaciones Científicas y Tecnológicas: Mendoza, Argentina (Dec 2000)

Complete mirror of www and ftp sites, updated weekly Improved regional access, redundant storage

What is needed?

• International Standard (e.g. ISO/TC211)• National commitment to GSDI• Discipline acceptance of new standard(s)• Heritage “systems” integration• International program and organization

implementation– IGOS, IGBP, UNEP, WDC, etc

What Does The Future Hold?

• New ISO standards will make an impact• Implementation of COTS using standards• New “network” and “systems” paradigms will

affect the mode of operation of WDCs• Ultimately user driven

All WDC Workshop to plan the future of the WDC System

• November 6-8, 2000, at the National Climatic Data Center, Asheville, North Carolina (USA)

• Possible themes:– Networks (human, WWW, and systems) are the future for the WDC system– WDC nodes or variations of a “standard” data center may be necessary– Regional WDCs may be considered– Data restrictions and commercial ownership of data will affect the future of

the WDCs

• The 21st century WDC will evolve to address these issues