Having good data quality assessment (QA) is essential if the MISR data are to be scientifically meaningful to our users. Recognizing this, the EOS Project has asked each instrument team to create a detailed plan for reporting the quality of its data. We are currently participating in the effort to figure out how effective QA plans can be developed and implemented at a reasonable cost. In a previous article, we described the method we are using to identify QA indicators (Kahn et al., "How Will We Choose Which Quality Flags and Constraints to Report for MISR Level 2 Data?," The Earth Observer 7, p.32-33, May/June 1995). We present here an overview of how QA for MISR Level 2 data can be accomplished.

Following recommended EOS QA procedures (Internal EOS Communication, Bob Lutz, Hughes STX, 1996), we anticipate the need for parts of the MISR QA activity to occur: (1) in the Product Generation System (PGS) Software, (2) with the Distributed Active Archive Center (DAAC) operator, and (3) at the JPL-based Science Computing Facility (SCF).

We plan to automate the routine QA processing. Human involvement will be limited to: (1) spot checking of the data stream, and (2) investigating "anomalies." This puts most of the QA burden on the PGS Software, which will create "indicators" of key aspects of the data quality and algorithm performance. These indicators will be stored with other outputs from the Level 2 data stream. Here is a top-level diagram of this activity:

A general description of these indicators is given in the Appendix. The MISR Level 2 QA Parameter Table lists all the indicators and their allowed values. Some examples of the entries in the QA Parameter Table are shown in Table 1. We assume that external inputs to the PGS Software, such as atmospheric surface pressure and wind speed from a data assimilation model, will be delivered with their own quality indicators, generated under the guidance of the cognizant science teams. The PGS contains tables of climatological values for all the external parameters needed by the MISR algorithms; these will be used as default values if the external input data are unavailable, or are flagged as being of low quality. Such cases will be reported in a processing path indicator.

Product Generation System (PGS) Software	Outputs
instrument performance indicators processing path indicators algorithm performance indicators physical constraint indicators climatological likelihood indicators statistical summary data at granule or higher level	indicators (region or sub-region) statistics (granule or large spatial scale) anomaly list (QA Log entires)

We are planning to keep all QA Log information from the entire Level 2 data stream in a format which is easily searchable using at least: (1) the date of the entry, (2) the processing step in the data stream which produced the entry, (3) the physical location on the Earth, and (4) the error or warning code associated with the entry. This will make it easy to compare entries from different parts of the PGS Software when investigating anomalies.

Some of the indicators will be designated as "alarms." These will be used for near-real-time QA of the MISR data stream. QA operations at the DAAC will involve monitoring alarms, and possibly examining displays of data created by the real-time data stream. The operator will respond by recording anomalies in the QA Log, and contacting the SCF about the anomaly in a timely manner, for further action. The algorithm is being designed so that the DAAC operator, with the concurrence of the SCF, can switch off certain alarms to avoid excessive output. This may be particularly useful at the beginning of the MISR mission, before thresholds in the algorithm have been optimized and other characteristics of the data stream have been studied under routine operating conditions.

DAAC Operator	Responses
monitor alarms possibly examine image or plotted data in near-real-time	enter anomalies in QA Log contact SCF about anomalies

At the SCF, QA amounts to performing those tasks that require the attention of the MISR Instrument Team, and completing any processing steps that can not be automated at the DAAC. We anticipate the following QA activities at the SCF:

• Examining summaries of QA data produced routinely by the PGS.

• Performing in-depth analysis of algorithm and software results on samples taken from the MISR Level 2 data products.

• Performing any special investigations indicated by routine examination of QA data, by anomalies reported at the DAAC, or by data users at the SCF or elsewhere.

• Evaluating instrument and algorithm results using data from the MISR field validation program.

• Documenting SCF QA activities in the QA Log.

• Producing those routine statistical summaries that require the accumulation of time series of QA data, which might not be feasible to stage at the DAAC.

Among the issues that remain to be worked out for the pre-launch MISR Level 2 QA effort are: refining the QA Parameter Table, designing the DAAC interface and Level 2 QA Log, and establishing procedures for investigating anomalies and for field validation analysis at the SCF.

Of course, the final assessment of our QA plan itself must wait until bits begin to flow through the data stream. But we welcome further discussion of all aspects of this subject with our EOS colleagues.

Appendix. General Description of MISR Level 2 QA Indicator Types

1. Instrument Performance -- Instrument performance indicators that affect spectral, radiometric, and geometric performance are monitored for engineering purposes, and to effect updates to the instrument calibration parameters. The Level 1 data stream will produce summaries of instrument performance in three areas: (1) radiometric quality, (2) geometric quality, and (3) missing data. These metrics will be compared with sets of limits, and the relevant performance implications will be encoded into data quality indicators.

2. Processing Path Indicators -- Decisions made along the data processing stream, such as which retrieval path to follow, are retained as part of the processing record. For example, choices will be made as to whether an ocean, a Dense Dark Vegetated surface, or a heterogeneous land aerosol retrieval is attempted, whether near-real-time inputs or climatology are used for column ozone abundance, and whether cloud phase is set by observations or by model inputs.

3. Physical Constraints -- There are many physical constraints that can be applied to the retrieved results, some of which may be used as indicators of data quality. Some examples are: the requirement of non-negative radiances, albedo within the range of zero to one, and an upper bound on the total aerosol optical depth based on the darkest pixel in the scene.

4. Algorithmic Constraints -- Since keeping track of the assumptions and numerical behavior of the algorithm is part of the algorithm development effort, these constraints are relatively easy for us to identify. They include such items as: (1) convergence characteristics of numerical methods (residuals and number of iterations); (2) the limits of intrinsic assumptions made in the parameterizations used, such as an ocean surface roughness model that is meaningful only within a certain range of wind speeds; and (3) case limitations, such as treating pixels that may cross radically different terrain types, e.g., coasts, if the algorithm is designed to assume an "average" terrain type, and rejecting pixels that are too cloudy or with terrain too rough for the algorithm to work.

5. Climatological Constraints -- These are "statistical" constraints, which may be interpreted as warnings, but do not necessarily represent errors. An "unlikely" result may mean a misinterpretation of the data, or a discovery. Indicators based upon such constraints will be very helpful for the first-order analysis of the MISR Level 2 results. For example, the MISR Aerosol "Climlikely" Product, which is the retrieval algorithm's predicted aerosol climatology, may indicate that it is more likely to find biomass burning particles than mineral dust particles over a tropical rain forest. We are hoping to develop climatologies for as many of the MISR-retrieved physical parameters as possible (surface albedo and view-dependent reflectances, cloud cover, etc.), so comparisons with expectation for these quantities can also be made routinely.

Table 1: Examples of Entries in the MISR Level 2 Retrieval QA parmeter Table

[Table of Contents] [Previous] [Next]