NASA Logo - Goddard Space Flight Center region to select the NASA home page region to select the NASA GSFC home page
 
AVDC Banner

DOCUMENTATION/ GEOMS DATA REPORTING/ FTIR

Fourier Transform Infrared Spectroscopy (FTIR) template description


Document history


Version Date Comments
002 December 01, 2021 GEOMS FTIR template
001 September 05, 2012 GEOMS FTIR template v001
v5 December 01, 2021 AVDC/NDACC FTIR reporting guidelines (revised version in preparation)
v4 September 14, 2009 AVDC/NDACC FTIR reporting guidelines
v3 February 21, 2008 AVDC/NDACC FTIR reporting guidelines
v2 September 21, 2006 AVDC/NDACC FTIR reporting guidelines

Authors


Martine De Mazière (BIRA-IASB, Martine.DeMaziere [at] biraiasb.oma.be), C. Retscher (UMBC/GEST and NASA/GSFC, christian.retscher [at] nasa.gov) and Ian Boyd (NIWA, i.boyd [at] niwa.co.nz)

Overview


This document outlines data reporting requirements for the Fourier Transform Infrared (FTIR) Spectrometers of the Network for the Detection of Atmospheric CompositionChange (NDACC, formerly called NDSC or Network for the Detection of Stratospheric Change). These guidelines were developed by the Infrared Working Group (IRWG) of NDACC and the Aura Validation Data Center (AVDC) to facilitate the submission of FTIR datasets compliant to the Generic Earth Observation Metdata Standard (GEOMS) to the Aura Validation Data Center (AVDC), the Envisat Validation Data Center (EVDC) and the NDACC Data Handling Facility (DHF).

Template description

  • Instrument Naming
  • Although FTIR spectrometer systems have the capability to measure multiple atmospheric entities simultaneously, we adopt the convention that each file reports data for a single target gas. Therefore, the different files are distinguished by the instrument name that appears in the filename and that is a concatenation of the instrument name (FTIR) + primary measured target gas [GAS] reported in the file, as shown in Table .

  • Variable Reporting
  • Each measured primary entity [GAS] requires a mandatory set of variables to be reported within a FTIR.[GAS] file. A description of the variables is given in Table . Notice that all variables are mandatory for the submission of vertical profile data. If only total column data are reported, then some variables need not be reported.

    Individual data will be reported, each with its own random and systematic error estimates and averaging kernels. As such the same data files that are archived in NDACC can be used for satellite validation purposes and any other applications.

  • Additional Notes
  • To accommodate variable altitude retrieval grids from one reported data point to another, however always with the same number of retrieval levels (PROFFIT retrieval code) or layers (SFIT2 retrieval code), the ALTITUDE.LEVEL.INDEX and ALTITUDE.LAYER.INDEX vectors have been introduced, respectively. The retrieval altitude grid (variable ALTITUDE) is then referenced to this variable. In the case of SFIT2, ALTITUDE contains the output altitude vector (i.e., the center altitudes of the retrieval layers, N in number). The target gas mixing ratio profile and the pressure and temperature profiles are referenced to this ALTITUDE vector. The ALTITUDE.BOUNDARIES vector provides the additional information about the layer boundary altitudes; it is a variable dependent on ALTITUDE (its first dimension must have size N; its second dimension has size 2). It provides the boundaries of the layers for which the vertical partial columns of the target gas are reported ([GAS].COLUMN.VERTICAL.PARTIAL_ABSORPTION.SOLAR (LUNAR)). In the case of PROFFIT, ALTITUDE contains the N altitude levels upon which the target gas mixing ratio profile is retrieved and reported; also the pressure and temperature profiles are reported on this altitude grid. The ALTITUDE.BOUNDARIES vector has again size (N,2); the boundary altitudes are equal to the successive level altitudes. As the number of partial columns between the levels is one less than the number of levels, the highest partial column is set to zero. To avoid end-user confusion, it is essential to explain the interpretation of the dependencies of the geophysical parameters (mixing ratio, pressure, temperature, and partial columns) on the altitudes in the VAR_DESCRIPTION variable.

  • Variable Fill Values
  • The variable fill value is a number inserted as a substitute data element if a data element of a variable is missing or erroneous.

  • File Granularity
  • The choice of the file granularity for archiving FTIR measurements in NDACC is left to the data provider, but it is forbidden that the file covers more than 1 year of data. It is possible now to store in the same file data retrieved with different a-priori profiles, and therefore different averaging kernels and uncertainty estimates, as all individual data are reported. The most common granularities will be monthly, seasonal (3 months) or yearly.

  • Metadata - Global Attributes
  • Each FTIR.[GAS] file requires one set of Global Attributes. These have been grouped into three categories describing the file contents, namely Originator Attributes, Dataset Attributes and File Attributes. An example of global attributes for an FTIR methane measurement at Reunion (PI and data submitter: M. De Mazière, BIRA.IASB; Data analysis using SFIT2 (layer based retrieval): Corinne Vigouroux, BIRA.IASB) is given in Table .

  • Metadata - Variable Attributes
  • Each variable reported in a FTIR.[GAS] file requires one set of Variable Attributes. These have been grouped into two categories describing the variable, namely the Variable Description Attributes and the Variable Visualization Attributes. An example of an attribute set is given in Table .

Templates and examples


1 GEOMS-EX-MWR-VA   MWR Variable Attributes Example
2 GEOMS-TE-FTIR-DS   FTIR Data Sources
3 GEOMS-TE-FTIR-VA   FTIR Variable Attributes Template
4 GEOMS-EX-FTIR-GA   FTIR Global Attributes Example

 GEOMS-EX-MWR-VA (MWR Variable Attributes Example)
Download in CSV format GEOMS-EX-MWR-VA.csv (last updated: December 01, 2021).
#Variable nameVariable valueComment
0 VAR_NAME O3.MIXING.RATIO.VOLUME_EMISSION_AVK Refer to standard
1 VAR_DESCRIPTION O3 Retrieval Averaging Kernels Free format
2 VAR_NOTES Averaging Kernels are associated with the given ozone mixing ratio values only. First three AVK values for the lowest altitude level are: 5.81e-002 9.55e-002 1.06e-001 Free format
3 VAR_SIZE 49;49 The number of elements in each dimension
4 VAR_DEPEND ALTITUDE; ALTITUDE INDEPENDENT, CONSTANT, self-referencing or a previously given one dimensional variable (in this case ALTITUDE)
5 VAR_DATA_TYPE REAL Allowable formats are BYTE, SHORT, INTEGER, REAL, DOUBLE, STRING
6 VAR_UNITS 1 Refer to standard for permissible units
7 VAR_SI_CONVERSION 0.0;1.0;1 Refer to standard
8 VAR_VALID_MIN -1.0
9 VAR_VALID_MAX 1.0
10 VAR_FILL_VALUE -90000.0
top

 GEOMS-TE-FTIR-DS (FTIR Data Sources)
ERROR_SQL=select show, datasource, description from datasource where datasource like '%FTIR%' and show != '0' ORDER BY datasource;top

 GEOMS-TE-FTIR-VA (FTIR Variable Attributes Template)
Download in CSV format GEOMS-TE-FTIR-VA.csv (last updated: January 21, 2023).
#Variable nameDependencies/DimensionsUnitsData typeCommentReq
0 DATETIME DATETIME MJD2K DOUBLE Mean of the zero path difference times of the individual scans that were averaged to produce the fitted spectrum x
1 INTEGRATION.TIME DATETIME s REAL Duration of the measurement corresponding to the retrieved datapoint x
2 LATITUDE.INSTRUMENT [CONSTANT|DATETIME] deg REAL Inst. geolocation (+ for north; - for south) x
3 LONGITUDE.INSTRUMENT [CONSTANT|DATETIME] deg REAL Inst. geolocation (+ for east; - for west) x
4 ALTITUDE.INSTRUMENT [CONSTANT|DATETIME] m REAL Inst. geolocation x
5 SURFACE.PRESSURE_INDEPENDENT DATETIME hPa REAL Surface/ground pressure x
6 SURFACE.PRESSURE_INDEPENDENT_SOURCE [CONSTANT|DATETIME] [empty] STRING Surface/ground pressure profile source (e.g. Mercury barometer etc.) x
7 SURFACE.TEMPERATURE_INDEPENDENT DATETIME K REAL Surface/ground temperature x
8 SURFACE.TEMPERATURE_INDEPENDENT_SOURCE [CONSTANT|DATETIME] [empty] STRING Surface temperature profile source (e.g. Meteorological thermometer etc.) x
9 HUMIDITY.RELATIVE.SURFACE_INDEPENDENT DATETIME % REAL Relative humidity at the station x
10 HUMIDITY.RELATIVE.SURFACE_INDEPENDENT_SOURCE [CONSTANT|DATETIME] [empty] STRING Relative humidity source (e.g. Anemometer; ECMWF etc.) x
11 WIND.DIRECTION.SURFACE_INDEPENDENT DATETIME deg REAL Wind direction at the station using WMO definition (wind from the north is 360; from the east is 90 and so on. No wind (calm) is 0) x
12 WIND.DIRECTION.SURFACE_INDEPENDENT_SOURCE [CONSTANT|DATETIME] [empty] STRING Wind direction source (e.g. Anemometer; ECMWF etc.) x
13 WIND.SPEED.SURFACE_INDEPENDENT DATETIME m s-1 REAL Wind speed at the station x
14 WIND.SPEED.SURFACE_INDEPENDENT_SOURCE [CONSTANT|DATETIME] [empty] STRING Wind speed source (e.g. Anemometer; ECMWF etc.) x
15 ALTITUDE [ALTITUDE|DATETIME;ALTITUDE] km REAL Retrieval effective altitude vector (if DATETIME independent) or matrix (if DATETIME dependent). Values are monotonically increasing x
16 ALTITUDE.BOUNDARIES [ALTITUDE;INDEPENDENT|DATETIME;ALTITUDE;INDEPENDENT] km REAL 2D (if DATETIME independent) or 3D (if DATETIME dependent) matrix with on each row the lower and upper boundaries of the layers for which the partial columns are reported. In layer-based retrieval these are equal to the lower and upper boundaries of the respective retrieval layers x
17 PRESSURE_INDEPENDENT DATETIME;ALTITUDE hPa REAL Effective air pressure at each altitude x
18 PRESSURE_INDEPENDENT_SOURCE [CONSTANT|DATETIME|DATETIME;ALTITUDE] [empty] STRING Pressure profile source (e.g. Lidar; NCEP; Sonde; ECMWF etc.) x
19 TEMPERATURE_INDEPENDENT DATETIME;ALTITUDE K REAL Effective air temperature at each altitude x
20 TEMPERATURE_INDEPENDENT_SOURCE [CONSTANT|DATETIME|DATETIME;ALTITUDE] [empty] STRING Temperature profile source (e.g. Lidar; NCEP; Sonde; ECMWF etc.) x
21 DRY.AIR.COLUMN.PARTIAL_INDEPENDENT DATETIME;ALTITUDE Zmolec cm-2 REAL Vertical profile of partial columns of air number densities (for conversion between VMR and partial column profile) x
22 DRY.AIR.COLUMN.PARTIAL_INDEPENDENT_SOURCE [CONSTANT|DATETIME|DATETIME;ALTITUDE] [empty] STRING Partial columns of air source (e.g. NCEP; Sonde; ECMWF etc.) x
23 [GAS].MIXING.RATIO.VOLUME.DRY_ABSORPTION.[SOLAR|LUNAR] DATETIME;ALTITUDE [ppmv|ppbv|pptv] REAL Retrieved target vertical profile in VMR units (with respect to dry air) o
24 [GAS].MIXING.RATIO.VOLUME.DRY_APRIORI DATETIME;ALTITUDE [ppmv|ppbv|pptv] REAL A-priori target vertical profile in VMR units (with respect to dry air) x
25 [GAS].MIXING.RATIO.VOLUME.DRY_APRIORI.SOURCE [CONSTANT|DATETIME|DATETIME;ALTITUDE] [empty] STRING Source of the vertical profile of a-priori per layer x
26 [GAS].MIXING.RATIO.VOLUME.DRY_ABSORPTION.[SOLAR|LUNAR]_AVK DATETIME;ALTITUDE;ALTITUDE 1 REAL A or Averaging kernel matrix for retrieved target profile (expressed in VMR/VMR units; the second ALTITUDE dimension (the fastest running) is the kernel dimension or the column dimension of the AVK matrix) x if [GAS].MIXING.RATIO.VOLUME.DRY_ABSORPTION.[SOLAR|LUNAR] is provided
27 [GAS].MIXING.RATIO.VOLUME.DRY_ABSORPTION.[SOLAR|LUNAR]_UNCERTAINTY.RANDOM.COVARIANCE DATETIME;ALTITUDE;ALTITUDE [ppmv2|ppbv2|pptv2] REAL Total random error covariance matrix associated with the retrieved vertical profile (expressed in same units as the profile) (without smoothing error!) x if [GAS].MIXING.RATIO.VOLUME.DRY_ABSORPTION.[SOLAR|LUNAR] is provided
28 [GAS].MIXING.RATIO.VOLUME.DRY_ABSORPTION.[SOLAR|LUNAR]_UNCERTAINTY.SYSTEMATIC.COVARIANCE DATETIME;ALTITUDE;ALTITUDE [ppmv2|ppbv2|pptv2] REAL Total systematic error covariance matrix associated with the retrieved vertical profile (expressed in same units as the profile) x if [GAS].MIXING.RATIO.VOLUME.DRY_ABSORPTION.[SOLAR|LUNAR] is provided
29 [GAS].COLUMN.PARTIAL_ABSORPTION.[SOLAR|LUNAR] DATETIME;ALTITUDE molec cm-2 (or a unit scaled by 1E3*n ex. Pmolec cm-2) REAL Vertical profile of partial columns per layer x if [GAS].MIXING.RATIO.VOLUME.DRY_ABSORPTION.[SOLAR|LUNAR] is provided
30 [GAS].COLUMN.PARTIAL_APRIORI DATETIME;ALTITUDE molec cm-2 (or a unit scaled by 1E3*n ex. Pmolec cm-2) REAL Vertical profile of a-priori partial columns per layer x
31 [GAS].COLUMN_ABSORPTION.[SOLAR|LUNAR] DATETIME molec cm-2 (or a unit scaled by 1E3*n ex. Pmolec cm-2) REAL Total vertical column of target gas x
32 [GAS].COLUMN_APRIORI DATETIME molec cm-2 (or a unit scaled by 1E3*n ex. Pmolec cm-2) REAL A-priori total vertical column of target gas x
33 [GAS].COLUMN_ABSORPTION.[SOLAR|LUNAR]_AVK DATETIME;ALTITUDE 1 REAL Averaging kernel matrix associated with the total vertical column of the target gas (in molec cm-2/molec cm-2 units) x
34 [GAS].COLUMN_ABSORPTION.[SOLAR|LUNAR]_UNCERTAINTY.RANDOM.STANDARD DATETIME molec cm-2 (or a unit scaled by 1E3*n ex. Pmolec cm-2) REAL Total random uncertainty on the retrieved total column (expressed in same units as the column) (without smoothing error) x
35 [GAS].COLUMN_ABSORPTION.[SOLAR|LUNAR]_UNCERTAINTY.SYSTEMATIC.STANDARD DATETIME molec cm-2 (or a unit scaled by 1E3*n ex. Pmolec cm-2) REAL Total systematic uncertainty on the retrieved partial columns (expressed in same units as the column) x
36 ANGLE.[SOLAR|LUNAR]_ZENITH.ASTRONOMICAL DATETIME deg REAL The solar (or lunar) astronomical zenith angle at which the measurement was taken x
37 ANGLE.[SOLAR|LUNAR]_AZIMUTH DATETIME deg REAL The azimuth viewing direction of the instrument using north as the reference plane and increasing clockwise (0 for north; 90 for east and so on) x
38 LATITUDE_LOS [DATETIME|DATETIME;ALTITUDE] deg REAL Meas. Geolocation; latitude (decimal degrees) along the line of sight of the measurement (+ for north; - for south) x
39 LONGITUDE_LOS [DATETIME|DATETIME;ALTITUDE] deg REAL Meas. Geolocation; longitude (decimal degrees) along the line of sight of the measurement (+ for east; - for west) x
40 H2O.MIXING.RATIO.VOLUME.DRY_APRIORI DATETIME;ALTITUDE ppmv REAL Vertical profile of H2O adopted in the target gas retrieval in VMR units x
41 H2O.MIXING.RATIO.VOLUME.DRY_APRIORI.SOURCE [CONSTANT|DATETIME|DATETIME;ALTITUDE] [empty] STRING Source of vertical profile of H2O adopted in the target gas (e.g. MERRA2; ECMWF; NCEP; or retrieved separately from dedicated microwindow ) x
42 H2O.COLUMN_APRIORI DATETIME Zmolec cm-2 REAL Total vertical column of H2O adopted in the target gas retrieval x
43 SOURCE.PRODUCT INDEPENDENT
[empty] STRING Information relevant to the source history of the Metadata and Data o
top

 GEOMS-EX-FTIR-GA (FTIR Global Attributes Example)
Download in CSV format GEOMS-EX-FTIR-GA.csv (last updated: December 01, 2021).
#Variable nameVariable valueComment
0 PI_NAME De Maziere;Martine
1 PI_AFFILIATION Belgian Institute for Space Aeronomy;BIRA.IASB
2 PI_ADDRESS Avenue Circulaire, 3;B-1180 Brussels;BELGIUM
3 PI_EMAIL martine@bira-iasb.oma.be
4 DO_NAME Vigouroux;Corinne
5 DO_AFFILIATION Belgian Institute for Space Aeronomy;BIRA.IASB
6 DO_ADDRESS Avenue Circulaire, 3;B-1180 Brussels;BELGIUM
7 DO_EMAIL corinnev@bira-iasb.oma.be
8 DS_NAME De Maziere;Martine
9 DS_AFFILIATION Belgian Institute for Space Aeronomy;BIRA.IASB
10 DS_ADDRESS Avenue Circulaire, 3;B-1180 Brussels;BELGIUM
11 DS_EMAIL martine@bira-iasb.oma.be
12 DATA_DESCRIPTION FTIR vmr vertical profile data of CH4 at Reunion Island (St Denis) Free format
13 DATA_DISCIPLINE ATMOSPHERIC.PHYSICS;REMOTE.SENSING;GROUNDBASED Refer to GEOMS document
14 DATA_GROUP EXPERIMENTAL;PROFILE.STATIONARY Refer to GEOMS document
15 DATA_LOCATION LA.REUNION Refer to GEOMS document
16 DATA_SOURCE FTIR.CH4_BIRA.IASB001 Refer to GEOMS document
17 DATA_VARIABLES DATETIME;
LATITUDE.INSTRUMENT;
LONGITUDE.INSTRUMENT;
ALTITUDE.INSTRUMENT;
SURFACE.PRESSURE_INDEPENDENT;
SURFACE.TEMPERATURE_INDEPENDENT;
ALTITUDE;
ALTITUDE.BOUNDARIES;
PRESSURE_INDEPENDENT;
TEMPERATURE_INDEPENDENT;
INTEGRATION.TIME;
CH4.MIXING.RATIO.VOLUME_ABSORPTION.SOLAR;
CH4.MIXING.RATIO.VOLUME_ABSORPTION.SOLAR_APRIORI;
CH4.MIXING.RATIO.VOLUME_ABSORPTION.SOLAR_AVK;
CH4.MIXING.RATIO.VOLUME_ABSORPTION.SOLAR_UNCERTAINTY.RANDOM.COVARIANCE;
CH4.MIXING.RATIO.VOLUME_ABSORPTION.SOLAR_UNCERTAINTY.SYSTEMATIC.COVARIANCE;
CH4.COLUMN.PARTIAL_ABSORPTION.SOLAR;
CH4.COLUMN.PARTIAL_ABSORPTION.SOLAR_APRIORI;
CH4.COLUMN_ABSORPTION.SOLAR;
CH4.COLUMN_ABSORPTION.SOLAR_APRIORI;
CH4.COLUMN_ABSORPTION.SOLAR_AVK;
CH4.COLUMN_ABSORPTION.SOLAR_UNCERTAINTY.RANDOM.STANDARD;
CH4.COLUMN_ABSORPTION.SOLAR_UNCERTAINTY.SYSTEMATIC.STANDARD;
ANGLE.SOLAR_ZENITH.ASTRONOMICAL;
ANGLE.SOLAR_AZIMUTH;
H2O.MIXING.RATIO.VOLUME_ABSORPTION.SOLAR;
H2O.COLUMN_ABSORPTION.SOLAR
18 DATA_START_DATE 20070525T040000Z ISO8601
19 DATA_STOP_DATE 20071020T150000Z ISO8601
20 DATA_FILE_VERSION 001 Refer to GEOMS document
21 DATA_MODIFICATIONS None Free format
22 DATA_QUALITY Reference paper Senten et al.
23 DATA_CAVEATS None Free format
24 DATA_RULES_OF_USE please contact M. De Mazière Free format
25 DATA_ACKNOWLEDGEMENT EU projects UFTIR and HYMN Free format
26 DATA_TEMPLATE GEOMS-TE-FTIR-002 Refer to GEOMS document
27 DATA_PROCESSOR SFIT2v3.94 Refer to GEOMS document
28 FILE_DOI Currently left empty. Definition yet to come.
29 FILE_NAME groundbased_ftir.ch4_bira.iasb001_la.reunion_02_20070525t040000z_001.hdf Naming convention
30 FILE_GENERATION_DATE 20080312T143444Z ISO8601
31 FILE_ACCESS HYMN;NDACC Project dependent
32 FILE_PROJECT_ID AOID9999 Project dependent
33 FILE_ASSOCIATION NDACC Project dependent
34 FILE_META_VERSION 04R007;IDLCR8HDF
Refer to GEOMS document
top

back | home