DOCUMENTATION/ GEOMS DATA REPORTING/ GEOMS FAQ
Frequently Asked Questions (FAQ) on GEOMS compliant file generation
- I want to submit a file to a GEOMS compliant data center. What should I do?
- Check the latest TAV and make sure that your AFFILIATION, DATA_LOCATION, DATA_SOURCE, and DATA_VARIABLES are available. Check as well all other TAV attributes. It is unlikely, but you may still want to report attributes which are currently not registered in GEOMS.
- Provide the data center with your PI/DO/DS information.
- Check your DATA_SOURCE and see, if your file needs to be reported compliant to a GEOMS template. If yes, check template requirements which extend general GEOMS requirements.
- From GEOMS document, or the AVDC GEOMS website check which attributes are mandatory or optional.
- The official TAV does not contain information on e.g. my AFFILIATION, DATA_LOCATION, DATA_SOURCE, and DATA_VARIABLES. What should I do?
Before you proceed, please provide the data center with your information. Requests for additional entries will be evaluated and will likely result in an updated TAV. For testing on your local system, you can download the latest TAV and add your information at the appropriate sections. Note, your new AFFILIATION and DATA_LOCATION are likely to be adapted by the data center, while attributes like DATA_SOURCE and DATA_VARIABLES are reviewed following more specific rules and restrictions.
- Why is DATA_SOURCE in the TAV not equal to DATA_SOURCE in the HDF file?
In the HDF file the first part (before '_') of DATA_SOURCE refers to DATA_SOURCE type (e.g. LIDAR.O3) and the second part (after '_') refers to the affiliation acronym followed by a three-digit number (e.g. NASA.GSFC001). In the TAV the DATA_SOURCE section only refers to DATA_SOURCE type, while the affiliation acronym is the second part (after ';') in the AFFILIATION entry (e.g. NASA Goddard Space Flight Center;NASA.GSFC). The three digit number is not shown in the TAV, but needs to be set by the data provider at the time of construction of a GEOMS compliant file.
- What is the TAV and how does it align with on-line information on GEOMS attributes and values?
The GEOMS on-line documents reflect the latest status of allowed attributes and values. They are listed in a user friendly layout. The Table Attribute Values (TAV) file goes inline with the GEOMS on-line documents, but is used as an off-line input for file generation, QA and harmonization.
- Is there any GEOMS HDF generation/reading software ready to be used by data providers and data users?
Yes, you can use the AVDC GEOMS idlcr8 suite.
- Is there any GEOMS netCDF generation/reading software ready to be used by data providers and data users?
Unfortunately we don't provide this yet. Please check A9) for further options.
- I have an HDF file ready. How can I make sure that it is GEOMS compliant?
You can check your file with the AVDC GEOMS QA software.
- I have old HDF files, which I want to harmonize into new GEOMS.
You can use the AVDC GEOMS harmonization software.
- My file passes on-line and/or off-line AVDC GEOMS QA checks, but is still rejected by a GEOMS compliant data center. What should I do?
Please read A1).
- I have an HDF4 file ready but want to report in HDF5 or vice versa. What should I do?
There are two options. First, you can use the AVDC GEOMS software idlcr8ascii to dump the old data into ASCII and then use idlcr8hdf to re-create the file into the new format. Second, you can use ESA GECA tools, which allow for conversion from and into HDF4/HDF5/netCDF.
GEOMS templates
In general, a GEOMS template states, which metadata attributes and data variables must be reported. The general guidelines in the GEOMS document are valid for all templates and all rules must be followed. Templates cannot be more loose on general rules, but only more restrictive in the usage of metadata attributes and data variables. Templates may e.g. rule the usage of generally optional, but mandatory variable attributes if used in the GEOMS-TE context.
How to construct a file according to a GEOMS template
- Check your DATA_SOURCE and see, if your file needs to be reported compliant to a GEOMS template.
- If your file needs to comply to a template, check the corresponding template for further instructions.
- Check the 'Req' column in the GEOMS attributes and templates for mandatory, optional, and conditional use of variables.
How to construct a GEOMS template
- Check with a GEOMS compliant data center if templates similar to your proposed template exist or are in discussion.
- Generate templates as well as template examples for global and variable attributes and provide them in GEOMS compliant csv format.
- Distinguish between mandatory 'x', optional 'o', and conditional 'x if' attributes and declare the use in the 'Req' column of the template tables.
- Determine the status of your attributes/variables and declare the use in the 'Status' column of the template tables.
- 0: expired variable
- 1: [reserved for data center's internal use]
- 2: regular variable
- 3: major change to variable or new variable
- 4: small change to variable (e.g. typo correction)
- 5: variable in review
- The following sections need to be addressed for any template:
- Authors list
- Overview
One - two paragraphs. Provide information on the nature of the measurements to be reported. Which institutions, measurements networks, and/or data centers are involved? Who is the target audience?
- Template description
- Instrument Naming
One - two paragraphs. How are your measurements identified? What is the name of your instrument and which primary (geo-)physical quantities do you want to report (DATA_SOURCE). How can data users recognize your files?
- Variable Reporting
One - two paragraphs. How are your (geo-)physical quantities measured/derived? Is data directly retrieved from your measurements, are models involved?
- Additional Notes
One - two paragraphs. How does your datetime and altitude information look like? Provide additional information if averaging is involved. Which methods are involved?
- Variable Fill Values
One paragraph. How do you plan to report fill values? Default sentence: 'The variable fill value is a number inserted as a substitute data element if a data element of a variable is default or missing.'
- File Granularity
One paragraph. What time granularity will your files have? Default sentence: 'The reporting granularity for the measurements is one file per measurement.'
- Metadata - Global Attributes (GA)
Every GEOMS file needs GA. What are the specifics of your GA? How does this information translate into the actual file.
- Metadata - Variable Attributes (VA)
Every reported variable in a GEOMS file needs VA. What are the specifics of your VA? How does this information translate into the actual file.
- Template and example tables (csv format)
As a minimum you need the cover the following tables:
- GEOMS-TE-[instrument name/sub-name]-DS
List all DATA_SOURCEs which shall be covered by your instrument, e.g. [instrument name/sub-name].O3, [instrument name/sub-name].NO2.
- GEOMS-TE-[instrument name/sub-name]-GA
In case you plan to report one general template for all (geo-)physical quantities, e.g. O3 and NO2, you need to provide only one GA template, otherwise report as many GA templates as needed to cover your measurements. Each GA template must be reported in a csv file with comma-separated-values in columns referring to:
Status,Variable name,Variable value,Comment,Req (find more details in sections 3 and 4) Note, online tables are displayed with color-coded status and an automated variable count is added.
- GEOMS-TE-[instrument name/sub-name]-VA
Each reported data variable needs to have VA. Each VA template must be reported in a csv file with comma-separated-values in columns referring to:
Status,Variables,Units,Data type,Comment,Req (find more details in sections 3 and 4) Note, online tables are displayed with color-coded status and an automated variable count is added.
- GEOMS-EX-[instrument name/sub-name]-GA
Give one example GA as it would appear in your files.
- GEOMS-EX-[instrument name/sub-name]-VA
Give one example VA as it would appear in your files.
|
