DOCUMENTATION/ DCIO
Data Center InterOperability (DCIO)
Files
DCIO OAI-PMH metadata exchange structure files
Introduction
The Data Center InterOperability project is an initiative started by the European Space Agency (ESA) in December 2008. The DCIO was motivated by the GECA project (Generic Environment for Calibration/Validation Analysis) which supports existing and future ESA calibration and validation programs, and requirements to access correlative datasets from multiple Earth Observation (EO) domains and from multiple data centers.
The main objective of DCIO is to bring EO data centers closer together and foster future collaboration.
The specific objectives of DCIO are:
- Motivate collaboration of peer data centers
- Harmonize metadata standards for EO
- Provide data discovery and search for distributed resources
- Increase exposure of hosted data
- Develop joint data exchange agreements
- Enable uniform user authentication
- Allow systematic exchange of cal/val data
- Provide feedback on data use
DCIO has been developed by data centers in close contact with data providers. The group of data centers participating in DCIO is enabled to host data in a collaborative manner and to share metadata and data through agreed and harmonized channels. The DCIO will ensure the visibility of many data resources and enable access to databases across several EO domains. Principle investigators from networks can benefit from DCIO as their data will be available to a greater number of data users. This may enable a better collaboration between scientists, can result in an increased quality of data sources and will eventually lead to more publications.
As of October 2010 the DCIO consists of the following partners:
1 Username and password required. Please contact the data center.
The Generic Earth Observation Metadata Standard - GEOMS
Each data center maintains a database of any kind, where data can be discovered and found through information stored in a catalog. In order to enable a collaborative access to data center resources, DCIO agreed on catalog metadata and harmonized file metadata. DCIO partners developed catalog metadata based on Dublin CoreŽ, an international and widely accepted metadata standard which has been broadened by GEOMS elements in order to fit all DCIO needs.
To enhance the usability of the diverse correlative datasets collected for satellite validation activities, metadata definitions, covering a broad range of instrument types and geophysical parameters have been established. The Generic Earth Observation Metadata Standard (GEOMS) guidelines and templates describe the standard metadata definitions adopted for the correlative, experimental and model data archived for the Aura validation program, the Envisat calibration and validation campaign, and the GECA project, as well as for NDACC data. The definitions have been carefully chosen to allow applicability to other scientific endeavors. This development was initiated in 1998 through the European Commission (EC) project COSE, Compilation of atmospheric Observations in support of Satellite measurements over Europe, and extended in collaboration with ESA, NASA, principal investigators (PI) of the Envisat and Aura validation campaigns, and selected PIs from NDACC, for the implementation of a uniform data exchange standard. GEOMS compliant data can be implemented on any kind of hierarchical data format, though GEOMS compliant data centers currently only support the HDF4 and HDF5 data formats. There are discussions on a future support of netCDF.
Data centers participating in DCIO provide a variety of GEOMS compliant online and offline tools for data download, access, conversion, comparison, and collocation. Data conversion routines from various kinds of data formats and metadata (e.g., NASA-Ames, netCDF, WOUDC) into GEOMS are available. Some of these tools are published as open source code and/or compiled versions. Any data provider or data user can adapt these source codes and is encouraged to feed them back to the community.
Provide data discovery and search from distributed sources
DCIO chose the Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH) for interchanging catalog metadata. Some peer data centers are fully OAI-PMH compliant, and some have chosen to provide metadata through a custom-built mechanism. OAI-PMH provides structured access to the catalog metadata of peer data centers. OAI-PMH runs as an application on the data centers' web-servers. Metadata can be retrieved by any enabled data center or data user. The system also allows automated retrieval of metadata through regular harvesting.
When DCIO centers support commonly shared metadata, users can find data resources across the centers. All DCIO centers can also participate in broader shared-catalog 'libraries' such as the WMO Information System (WIS), the Global Change Master Directory (GCMD), and the Global Earth Observation System of Systems (GEOSS), among others.
How to join the DCIO group and become a peer data center
DCIO development and maintenance tasks are shared between ESA and NASA. DCIO meets in regular telecons (1 - 2 months) organized by ESA and infrequent meetings every 1 - 2 years. ESA and NASA with AVDC are currently responsible for the update of DCIO catalog metadata and further provide support for new data centers to become compliant. If you are a data center manager, who wants to pursue collaboration with DCIO please contact either the DCIO responsible at ESA, Rolf von Kuhlmann (rolf.von.kuhlmann [at] esa.int) and Yasjka Meijer (yasjka.meijer [at] esa.int) or at NASA, Christian Retscher (christian.retscher [at] nasa.gov).
|
