-- Chris Justice (justice@kratmos.gsfc.nasa.gov), University of Virginia
-- Dave Starr (starr@climate.gsfc.nasa.gov), NASA Goddard Space Flight Center
-- Diane Wickland (diane.e.wickland@hq.nasa.gov), NASA Headquarters
-- Jeff Privette (privette@chaco.gsfc.nasa.gov), NASA Goddard Space Flight Center
-- Tim Suttles ( tim.suttles@gsfc.nasa.gov), Raytheon STX Corporation
Validation of EOS data products is an integral part of the EOS Program (http://eospso.gsfc.nasa.gov/validation/valpage.html). The responsibility for product validation will be shared between the EOS Instrument Team Investigators and EOS Validation Investigators selected through the 1997 NASA Research Announcement. Summaries of the individual instrument team validation plans can be found at http://eospso.gsfc.nasa.gov/validation/valcharts.html, and abstracts of the Validation Investigators projects can be found at http://eospso.gsfc.nasa.gov/validation/frame.html. In the longer-term, contributions to validation will also be made by the broader user-community, through application of the data products to science questions. A distinction is made here between quality assurance for products and product validation, the latter generally relying on comparisons to independent correlative measurements and the former focused on the operational algorithmic implementation.
The EOS Validation Program sets a precedent in terms of a funded 'community' validation of land satellite products. As a relatively new endeavor shared between different research groups and spanning several disciplines, global data set validation necessitates community coordination. To initiate coordination, a preliminary meeting was held in March 1996 (The Earth Observer, Vol. 8, No. 2) to assess various approaches to land product validation and a range of long-term monitoring activities related to EOS validation. Following the selection of the EOS Validation Investigators in the fall of 1997 (http://eospso.gsfc.nasa.gov/directory/validation/validation.html), a Science Working group for the AM Platform (SWAMP) Land Validation Coordination meeting was held in December 1997 and attended by 60 scientists. The objectives of this meeting were: 1) to present individual investigator validation plans around themes of surface radiation and temperature, vegetation properties, and snow and ice product validation, 2) to assess aircraft validation missions in terms of site over-flight opportunities, 3) to agree on a number of Core Test Sites for EOS land validation, 4) to discuss standards for field/air measurement protocols, and 5) to develop a validation data policy and management plans both for satellite data products and correlative measurements. Breakout group sessions were held on biophysical/vegetation measurements, albedo measurements, snow and ice measurements, instrument calibration, test sites, aircraft coordination and data management.
The biophysical/vegetation breakout group was chaired by Alfredo Huete (University of Arizona) and Betty Walter-Shea (University of Nebraska) and discussed the design and protocols for the validation of the leaf area index (LAI), fraction of photosynthetically active radiation absorbed (fAPAR), net primary productivity (NPP), and vegetation index products. The objective was to develop a set of standardized procedures for the validation of vegetation-related satellite products across a global range of biomes, over the phenologic cycle, and within expected sun-target-sensor conditions. A standardized methodology is considered essential for cross-site comparability. Crucial to the measures discussed is the need for ground-to-air-to-satellite registration with the Global Positioning System (GPS).
Discussions focused on the need for field measures of both fraction of photosynthetically active radiation intercepted (fIPAR) and fAPAR. The group noted the need for daily measures of fAPAR with a minimum requirement being that instantaneous measures should be made close to time of overpass and over a range of sun angles bracketing the time of the satellite overpass. Since the interception and absorption of incoming photosynthetically active radiation (PAR) varies throughout the day, fAPAR measurements at various sun angles enable: (1) better integration into daily fAPARand photosynthesis/carbon uptake; and (2) standardization of fAPAR measures to constant sun angles, when spatial/temporal comparisons of instantaneous fAPAR are desired. The primary instruments required are ceptometers and line quantum sensors. A quantum sensor is also needed in an open area. Both instruments need calibration and cross-calibration to a standard reference and caution is needed with their sun-angle dependency on calibration.
The group defined LAI as one-half the total leaf area (i.e., one-sided) per unit area of ground surface. The group further distinguished between the LAI of broadleaf canopies (one-sided leaf area) and the LAI of needleleaf canopies (the projected area). The group recommended that measures of LAI be separated into green LAI (active canopy component) and non-green LAI, and noted that the non-green component may have large effects on the remotely-sensed interpretation of green LAI. Direct measures of LAI involve destructive harvesting and associated allometric equations. More indirect measures include use of ceptometers and the Li-Cor LAI-2000 instrument. Indirect methods measure light transmittance. The relationships of light transmittance with LAI are land-cover dependent, and corrections are generally needed. Initial discussions were held on sampling design, with each biome and land-cover class having an existing and recognized procedure for "point-based" LAI measures. Coupling the Li-Cor 2000 to such a reference will allow for the most accurate LAI measures to be extended from a meter-scale footprint of in situ sensors to the kilometer-scale footprint of satellite sensors.
Validation of the Vegetation Index (VI) products, will be coupled to biophysical vegetation measurements, as well as radiometric measurements of canopy reflectance properties. Since it is the output of the VI that must be validated, it is necessary to ensure that the variance or invariance in VI values corresponds to real, surface-related canopy behavior. Thus, measurements required are similar to those of the fAPAR and LAI products. As with the fAPAR product, VIs exhibit pronounced diurnal variations and require measurement over a range of sun angles. Preliminary discussions on sampling design focused on site heterogeneity issues, sampling statistics, and scaling issues.
The surface albedo breakout group was chaired by Alan Strahler (Boston University) and examined the issues involved in validation of the surface reflectance, the bidirectional reflectance distribution function (BRDF), and albedo land products from the MODIS and MISR Instrument Teams. The group stressed the conclusion that the best validation strategy would acquire data that could be used to reproduce and validate the entire chain of surface radiometric products, beginning with aerosol retrieval, through surface reflectance, to BRDF and albedo. This requires measurements of downwelling irradiance, both for characterization of illumination and for aerosol-sun photometry, using instruments such as the CIMEL and the Multi-Filter Rotating Shadowband Radiometer (MFRSR); upwelling radiance measurements, preferably directional and spectral in at least PAR wavelengths; and broadband hemispherical albedo measurements, preferably in the visible (<0.7 micrometers) and in the near-infrared and infrared regions (>0.7 micrometers). The CERES Team has developed a helicopter-based, multi-spectral radiometer system to measure the BRDF for various surface types. The CERES system will be used to measure BRDF at the ARM CART site in Oklahoma during an August 3-28, 1998 campaign. The group agreed to develop a validation protocol document for the entire measurement chain to include science principles, MISR and MODIS product descriptions, instrument measurement principles and procedures, and related information. The first draft of this protocol has been prepared and is now circulating for comment.
The snow and ice validation group was co-chaired by Anne Nolin (University of Colorado-Boulder) and Dorothy Hall (NASA/Goddard). EOS AM-1 cryospheric products to be validated include snow-covered area, snow albedo, sea ice surface temperature, and sea ice extent and concentration. During the working group session, product-specific test sites were chosen and prioritized (see section on Test Sites and Table 2a). These include both U.S. and international sites in polar, mid-latitude, and mountain regions. The group also discussed the planning of two upcoming field and ER-2 aircraft missions involving the MODIS Airborne Simulator (MAS) (http://ltpwww.gsfc. nasa.gov/MAS/). The first of these snow validation experiments was held following the validation meeting between March 10-20, 1998, in the vicinity of Mono Lake, California. Anne Nolin and Julienne Stroeve worked in coordination with members of the MISR validation team (Jim Conel and Mark Helmlinger). The experiment was headed by Zhengming Wan (University of California-Santa Barbara) whose group was on hand to measure surface temperature as part of their MODIS validation activities. Both AirMISR and MAS were flown on an ER-2, but due to technical and weather-related problems only MAS data were acquired. This experiment focused on validation of snow albedo. Ground support from the Jet Propulsion Laboratory (JPL) was supplied by the MISR team, and their measurements were critical to the success of the experiment. A second snow validation experiment is planned for winter, 1999 in the upper Midwest, with ER-2 flights out of Madison, Wisconsin. This project will serve as a follow-on to last year's Winter Cloud Experiment (WINCE) (http://cimss.ssec.wisc.edu/wince/wince.html). Again MAS and AirMISR will be flown in tandem on the ER-2, with ground measurements of albedo and snow-covered area to be made at various locations below the four flight lines. For sea ice validation, in situ measurements will be made in the Ross Sea by Li and colleagues from the University of Alaska.
The instrument calibration validation group was chaired by Kurt Thome (University of Arizona). Several topics need to be addressed concerning validation instrumentation. The first of these is the obvious need to determine the precision and accuracy of correlative measurements. To obtain better precision, it was decided to develop sets of protocols for the correlative validation measurements and to ensure traceability to a set of standards. To validate the accuracy of the measurements requires comparisons of results to sets of independent measurements with well-known precision. Two pathfinding activities known as the Prototype Validation Exercise (PROVE) campaigns, were undertaken in 1997 to establish such inter-instrument validation protocols (http://pratmos.gsfc.nasa.gov/~justice/ modland/valid).
Surface reflectance retrieval is one measurement that is common to many investigations. Methods for ensuring data consistency for this parameter were discussed.
Several groups expressed an interest in comparing reflectance retrievals to evaluate differences caused by instrumentation, reference field standards, and general methodologies and sampling strategies. It was decided that rather than create a new experiment to examine this problem, the groups would attempt to participate in an upcoming experiment in the Fall of 1998 at the Maricopa Agricultural Center, near Phoenix, Arizona (http://gaea.fcr.arizona.edu/MAC.html). Work in a similar fashion related to other parameters, such as albedo, fAPAR, LAI, NPP, and surface temperature and emissivity will also be done, but no specific plans have yet been made.
The EOS Core Test Site Group was co-chaired by Jeff Privette (NASA/Goddard) and Ken McGwire (Desert Research Institute). Final agreement was reached at the meeting on a set of EOS Core Test Sites for land validation (Table 1).
[Table 1. ACRONYMS: ARM/CART (Atmospheric Radiation
Measurement/Cloud and Radiation Testbed), LTER (Long-Term
Ecological Research), BOREAS (Boreal Ecosystem-Atmosphere Study),
NSA (Northern Study Area), SSA (Southern Study Area), USDA-ARS (U.S.
Department of Agriculture-Agricultural Research Service), SALSA
(Semi-Arid Land-Surface-Atmosphere Program).]
The sites are intended as a focus for land product validation over a range of biome types. However, they will also be useful for validation of some atmospheric products. The site list represents a consensus among the instrument teams and validation investigators, developed through a number of meetings and discussions. Most of the sites build on an existing program of long-term measurements and have an infrastructure to support in situ measurement. Each site has a point of contact responsible for overall validation coordination at the site (http://www-eosdis.ornl.gov/eos_land_val /list_sites.html). Although these sites are not intended to meet all EOS test site needs, they will, however, provide a focus for satellite, aircraft, and ground data collection for land product validation, and will provide sites for which scientists can readily access in situ and EOS instrument data sets. It is important to note that the EOS instrument vicarious calibration test-site needs will be coordinated by the EOS Calibration Working Group (Chair: Jim Butler, EOS Calibration Scientist - http://eospso.gsfc. nasa.gov/calibration/calpage.html) and are not included as part of the EOS Validation Core Test Sites.
Some products require special surface characteristics for validation. For example, the snow and ice products and the land surface temperature products will be validated at a number of product-specific test sites (Table 2a and 2b).
Other products such as MODIS Land Cover Change will require validation at sites where rapid change is taking place and will include targets of opportunity selected during the EOS mission. Although there will be a large number of such land-cover sites, validation will rely primarily on the acquisition of high-resolution satellite data. An early example of global land-cover product validation is being supported by the NASA Land-Cover Land-Use Program (J. Estes University of California Santa Barbara P.I. - http://rsrunt.geog.uscb. edu/igbp.html). Similarly, a set of high-resolution satellite training and testing sites used for AVHRR land cover classification has also been developed by John Townshend and Ruth Defries (University of Maryland) (http://www.inform. umd.edu/landcover/mss-training-areas.html).
The validation program will include the collection of aircraft data. MISR and ASTER will be relying heavily on airborne simulator data from AirMISR (http://www-misr.jpl.nasa.gov/armain.html) and the MASTER (http://asterweb.jpl.nasa.gov/master/) instruments. The large cost of aircraft flights makes coordination between flight mission, instrument, and validation investigators highly desirable. The tentative program of planned flights for 1999 provides opportunities for test site over-flights and piggy-back missions. Those interested in more information on flight itineraries and schedules should contact the individual flight mission principal investigators listed at http://eospso.gsfc. nasa.gov/eos_homepage/airborne.html.
Bill Emanuel (University of Virginia) and Dick Olson of the Oak Ridge National Laboratory (ORNL) Distributed Active Archive Center (DAAC) co-chaired the validation data management group. A validation data policy has been clearly specified by the EOS Validation Office (http://eospso.gsfc.nasa.gov/validation/policy.html). Efficient data management will be critical for validation of land products combining satellite and in situ data. Timely feedback on algorithm performance will be needed for early algorithm refinement and prior to product reprocessing. Those undertaking product validation will require rapid access to coincident in situ and EOS
instrument data. It was agreed that validation investigators would make their validation data available through the World Wide Web as soon as possible following data collection and at least within 6 months after acquisition. The ORNL DAAC has taken the charge to coordinate in situ data management. A central Web Site for EOS in situ data validation coordination will be established at the ORNL DAAC, providing metadata of the various investigator validation data holdings and pointers to investigator Web sites. Most of the higher order land products from the AM-1 platform will be available from EOSDIS via the Land DAAC at the EROS Data Center (EDC). The usefulness of a single Web page, with links to relevant in situ, aircraft and satellite sensor data sets, for each Core Validation Site was noted.
Since the December meeting there have been a number of developments with respect to validation. Efforts have been made through SWAMP to ensure that data acquisition from MISR, ASTER, MODIS and Landsat 7 will be made for the EOS Core Test Sites, as soon as possible following launch. ASTER and Landsat 7 have included the Core Test Sites in their acquisition strategies.
The individual validation investigators and the instrument teams are developing their own Web sites for validation data distribution. The ORNL DAAC has developed an EOS Validation Data Management Web site (http://www-eosdis.ornl.gov/eos_land_val/valid.html) and is testing a 'search tool' called 'Mercury', which will search each of the validation investigators and instrument team member Web sites for validation metadata and provide a catalogue of validation data. The ORNL DAAC is also developing a Web site for the FLUXNET program (http://cdiac.esd.ornl.gov/programs/NIGEC/fluxnet/). FLUXNET is a network of ground stations measuring carbon and energy fluxes and will be used to validate various primary production data sets. Carbon dioxide and energy flux data sets from two AMERIFLUX sites are currently being transferred to ORNL to prototype coordination of that network with EOS validation.
EDC DAAC representatives took the charge from the December meeting to develop plans to facilitate satellite data access in support of EOS validation. The EDC DAAC is currently exploring possibilities for packaging satellite data for the EOS Core Validation Sites, to facilitate data access and analysis by validation investigators. This data packaging will hopefully be a value-added activity by the DAAC which will speed up the validation process. As a precursor to EOS data collection and management, the NASA Landsat
Pathfinder Global Land Cover Test Sites (GLCTS) program being implemented at EDC (P.I.: Ken McGwire, Desert Research Institute) is compiling land cover, Landsat, AVHRR and Digital Elevation Model (DEM) data sets for the EOS Core Validation Sites (http://edcwww.cr.usgs.gov/landdaac/pathfinder/pathpage.html).
Cooperation between EOS Validation and the NASA regional intensive field campaigns has always been part of the MODIS land validation strategy. The NASA component of the Large Scale Biosphere-Atmosphere Experiment in Amazonia (LBA) is now underway and the LBA tower sites have been selected and will provide an opportunity for EOS validation. The Southern African Fires and Atmosphere Research Initiative (SAFARI 2000) will provide a continuation of the SCAR (Smoke/Sulfate Clouds and Radiation) class of ground and airborne campaigns during the early years of the AM-1 mission and an opportunity for EOS data product validation and early science results. The NASA contribution to the GEWEX Continental-scale International Project (GCIP) will provide a regional framework activity for the U.S., within which to validate and exploit EOS land products.
The MODIS land group (Lead: Alfredo Huete) is developing plans for a light-aircraft-based validation package, which will be used to extend point and ground-based reflectance measurements to larger footprints (1-5 km) at the Core Validation Sites. Simple and stable instrumentation with transfer radiometers coupled to MODIS Level 1b calibration activities would provide a consistent and mobile radiometric package. This versatile, low cost, rapid deployment would be made in conjunction with, and coupled to, ground-based fAPAR and LAI validation to characterize canopy reflectance over 1-km footprints at various view and sun angles. An open meeting to develop protocols for LAI/fAPAR validation measurement is planned for early this summer and is being organized by Jeff Privette (NASA/Goddard).
The NASA EOS Validation Program provides an important foundation for the development of international collaboration with respect to instrument calibration and product validation. The Committee on Earth Observation Satellites (CEOS) Working Group on Calibration and Validation is an obvious forum to further develop the international data requirements and cooperation for product validation. Similarly EOS land validation can both benefit from and contribute to the emerging Global Observing System Network (http://wwww.geog.umd.edu/landcover/bvs), which is creating a network of existing networks and sites making land measurements relevant to the Global Climate Observing System (GCOS) and the Global Terrestrial Observing System (GTOS). International cooperation and coordination of land satellite product validation holds much promise, but is something that has yet to be fully developed.