--David M. Glover (david@plaid.whoi.edu), EOSDIS Panel Chair
A meeting of the IWG EOSDIS Panel was held at the Laboratory for Atmospheric and Space Physics (LASP) facility, University of Colorado, on February 12-14, 1997 to address some of the alternative suggestions being raised as solutions to the "EOSDIS problem." Since the announcement by Hughes of a slip in their delivery schedule for release A and the consequent cancellation of the TRMM portion of that release, a great deal of attention has been focused on the potential for downstream effects on release B. This meeting of the EOSDIS Panel was convened to better acquaint ourselves with the options now on the table prior to joining the rest of the IWG in San Diego.
This report covers a briefing from H.K. Ramapriyan on EOSDIS Status and Plans including the ESDIS backup plans for releases B.0 and B.1, an update on the current user/usage patterns at the DAACs with V.0, a discussion of the DAAC certification mechanism led by Betsy Edwards (GSFC, Code 170), a briefing on the biennial review from Skip Reber and Dolly Perkins (who chairs the EOSDIS Study Team for the biennial review), an update of the status of the Federation and in particular the status of the Working Prototype-Earth Science Information Partner (WP-ESIP) CAN, and a frank discussion of many of the suggested ways of dealing with the EOSDIS problem.
Rama briefed us on how ESDIS has recovered from the initial slip and release A TRMM termination to bring ECS back in line to support AM-1/SAGE-III missions. ECS's Release A activities in support of TRMM have been assigned to the Goddard and Langley DAACs. The remainder of Release A (needed as a precursor to Release B) has been merged with Release B. By freeing the release A personnel to work on release B, converting a configured release A system (already undergoing integration) into a pre-release B test bed, and instigating a new system of metrics to oversee Hughes progress, ESDIS believes that the influence of the release A slip on release B can be minimized. Rama showed examples of metrics that are now being used to track progress weekly and to identify problem areas. The metric now being employed by ESDIS is a point system based on the total amount of work that could ever be done, the amount of work expected to be finished by a certain date, and the actual amount of work accomplished by that date for each task. In addition, a number of alternative time-buying options are being explored. In particular two backup-plan studies were conducted for supporting Landsat-7 and AM-1/SAGE-III missions. ECS remains the primary path for mission support, but these backup plans were to buy insurance to reduce the risk. The backup plans are limited, interim (up to six months) solutions. In early February a decision was made to go forward with the contingency data system for Landsat-7 with a system requirements walkthrough planned for early March. A decision on whether or not to continue was to be made at that time. In mid-February (as these minutes were in preparation) a decision was made not to continue the AM-1/SAGE-III and Landsat-7 backup activities. This decision was based on the lack of funds in the FY97 and FY98 overall EOSDIS budget and no real hope of augmentation of those budgets. Extracting the necessary funds from ECS was deemed to substantially increase the risk of a release B.0 delivery delay (Since then, substantially scaled-back "emergency" back-up plans have been initiated for AM-1/SAGE-III in accordance with the recommendation from the Science Working Group for the AM Project (SWAMP) and the EOSDIS Panel.)
Bruce Barkstrom provided us with an appraisal of the three key drivers of EOSDIS cost. EOSDIS must produce data, help users find data, and distribute that data. In order to produce the data EOSDIS must accept code from the data producers, compile and link it, accept directions from the data producers, and store the data when the run(s) is/are complete. Users may want three kinds of information from EOSDIS: the data themselves, subsets of the data, and information about the data (metadata). Distribution of data seems to fall into two categories: large data orders from a small fraction of the user community and moderate data orders from the rest of the users. The large data users can and should be handled by media transfers, the remaining data requests can be handled with network deliveries. Bruce presented the distribution figures from the DAACs; it was surprising to see that they are already distributing 2.5 TB/month! Breakdown charts of the users that access the DAACs to obtain data reveal U.S. Education, U.S. Commerce, and foreign users lead the pack of data consumers. When it comes to finding the data there appear to be widely different points of view on the value of the metadata to do so. A discussion ensued regarding the merits of the current metadata model being employed.
Betsy Edwards from Code 170 presented a review of the current plans for certifying the current DAACs. At the request of Code 170, the NRC's Committee on Geophysical and Environmental Data (CGED) has agreed to review the current DAACs. This committee has been around for 35 years and acts to review the World (and National) Data Centers in this country. However, the CGED recognizes that the DAAC situation is unique and has begun the review process with "scout" visits to the various DAACs to help the NRC better understand what DAACs are and what they do. The review criteria will be established in February 1997 (with input and agreement from DAACs and NASA). Site visits from the review panels will be held in March and April of 1997 and the CGED will generate a report deliverable to NASA by October 1, 1997. Codes 170 and Y will go over this report and make the final decision about certification. The NRC will only operate in a fact-finding/advisory role. Edwards stated very clearly that this was not a DAAC hunt: the NRC will review; NASA will certify.
Skip Reber began the presentation of what the biennial review is all about. The idea for such a review grew out of Charlie Kennel's idea that NASA could gain a great deal of control over the MTPE-review process if it set up such a mechanism, rather than wait for Congress (or someone else) to instigate a review. There was no implicit or explicit agreement from an organization such as the NRC to replace its reviews with this internally generated review. Nevertheless, the concept of a biennial review has grown since Kennel's days, and it is now being presented to the world as the reinventing of MTPE. Reber pointed out that some requirements of EOSDIS are now in jeopardy because of this. Another discussion about the EOSDIS requirements ensued and we started down a road this panel has been down before. A draft list of high-level requirements for EOSDIS was floated before the panel until it was pointed out such lists are always disconnected from their associated costs. No matter how good our intentions, a list of high-level requirements (that everyone could buy into) never gives us the satisfaction we are looking for. As soon as someone suggests cutting one of these high-level requirements we immediately see that their real value is impossible to calculate because the price of EOSDIS goes down only a little bit, or not at all. This is because of the intertangled nature of the functionalities and requirements of EOSDIS. Cutting a requirement frequently does little to bring down the cost of the DIS because the functionalities that support the deleted requirement also support other requirements that have not been deleted and little or no cost savings is had. Of course this is not unique to EOSDIS--any large project has this sort of problem. However, this line of reasoning is something of a red herring itself. The revisit of requirements is NOT to revisit ESDIS' current implementation of those requirements. The revisit should be the background for FUTURE implementations, beyond the PM-1 mission. This is the sort of thing that the EOSDIS Study Team, led by Dolly Perkins (Code 510), is charged with doing as part of the biennial review.
Skip Reber presented a list of ten simple goals that the EOSDIS Panel endorses as a reasonable long-range outlook for EOSDIS. They are, in no particular order:
Dolly Perkins (Code 510) followed Reber to talk to us about the EOSDIS Study Team she chairs as part of the biennial review process. Her charge is to specifically look at EOSDIS in the post-release-B time frame. This roughly translates into DIS for MTPE in the year 2000 and beyond. What sort of DIS should we have? Should we continue with the evolutionary development of ECS until a Federation is in place? This study will provide input into the biennial review process to suggest new strategies for MTPE. In this context a discussion of the WP-ESIPs came up. Are the WP-ESIPs still being considered an experiment? Or, in other words, will the WP-ESIPs be allowed to fail? It was restated that the position of this panel has always been that unless allowed to fail the experiment of creating WP-ESIPs will be a waste of time and resources.
The EOSDIS biennial review study team discussion was followed by an update on the status of the EOSDIS Federation. After giving a quick review of the NRC report from La Jolla (1995), Betsy Edwards gave a review of the MTPE concept of a data federation with the goal of evolving a process by which Earth science data will be easy to locate, access, and use. Another goal of the Federation is to increase the involvement of the scientific community in the EOSDIS enterprise as a whole. NASA is considering three types of ESIPs: type 1 will handle the standard data products with an emphasis on reliability; type 2 will produce scientific research products with an emphasis on creativity; and type 3 will produce data products that extend beyond global change research with an emphasis on commercial applications. A Cooperative Agreement Notice (CAN) for types 2 and 3 will be issued in early 1997, and the DAACs will be certified during FY97 forming two federations until approximately the year 2000. One issue that still remains to be addressed in detail is how will the Federation be governed? Working out this issue will be one of the first tasks of the working prototype federation of types 2 and 3 ESIPs. Selection of these ESIPs is still planned for July 1997, but the writing of the type 2 CAN is breaking new ground for NASA and may affect the type 3 CAN. It is highly desirable to have both CANs released at the same time so that the Federation startup activities can be synchronous.
Those present at the meeting voted that, in our view, the Version 0 DAACs' user access statistics (presented by Bruce Barkstrom) are very good news and show strong support for MTPE science and data. The IWG should take note that these statistics point to a use of, and dependence on, EOSDIS that seems to get little fanfare.
It was noted, by the chair, that a fair amount of functionality is being delayed until release B.1. Although there may be little other alternative, given the current funding envelope, I am concerned that those functions are at risk of being dropped completely from the EOSDIS program. This is particularly true for functions that are considered to be outside the limited requirements of defined scientific and application users. If carried to a logical conclusion, such restrictions would amount to a giant step backwards for NASA when it comes to handling data collected at the taxpayer's expense.
We think that the backup plans produced by the ESDIS project are a good and prudent thing. We suggest that the instrument teams (along with their DAAC partners of choice) begin exploring backup options that go beyond ESDIS's interim six-month period, just in case B.0 is not ready at launch. This is because the critical and non-negotiable requirements on EOSDIS and the networks stem from the fact that validated scientific data products require a substantial amount of careful examination and rework. This fact requires access to data by the data producers, either in the form of adequate network bandwidth or media deliveries that are likely to be in excess of the 2X expectations that the Project has used. Because of the concern over production and distribution capacity limitations of the early B.0 release, it was the consensus of the data producers present at the meeting that they would be uncomfortable proceeding without implementing the backup capability they have recommended to the Project. Indeed, these data producers expect a period of six months or more of "configuration tuning" of the delivered early releases of the ECS system. In other words, the science software may run satisfactorily at the SCFs, but the teams are concerned that as the complexity of the ECS system becomes apparent, they cannot support production of good data and debug the system at the same time. To these teams, it appears there is a large risk that enough of the bugs will not have been worked out of a system that is delivered shortly before launch to make production feasible. The EOSDIS Panel is aware of the recent decision not to support the AM-1/SAGE-III backup study any farther and the explicit statement by the ESDIS project to, essentially, "bet the farm" on release B.0. A prudent instrument team PI would be exploring other options, regardless of the commitment of funds from ESDIS.
Since the best bandwidth ever mentioned for ECS is the canonical 2X, it is generally felt that most instrument teams are likely to want to get at least one copy of the Level-1 (calibrated) data. Where does this leave us in terms of network access to the rest of the Earth science community?
There is tension between at least two communities in the EOSDIS world: the instrument teams and the database engineers. In order to develop a system that is efficient in its searches the database engineers have created a large, all-encompassing list of metadata so that any request from any users can be handled quickly. However, the instrument teams feel burdened to extract even 30% of this total list from their data at a time when they are very busy finalizing production code, etc. The implication is that there are a number of metadata items requested for all data product granules that don't always make sense for a particular data product.
We recommend that a metadata workshop be held or better yet a series of metadata workshops. The users and the implementers of the metadata need to be brought together for an open exchange of ideas. Currently the "read DID-311" advice to instrument teams by the ECS folks is not helping. We suggest that at these workshops the required metadata for each data product be examined to remove items that make no sense. The various working groups should also be included. The Data Science Working Group (DSWG), Data Management Working Group (DMWG), and Client Design Working Group (CDWG) are too fractured; perhaps they should be pulled together into one umbrella organization for coordination.
Given that we cannot seem to put a price tag on specific requirements (regardless of the level) of EOSDIS, how can we attach a sense of worth to aspects of EOSDIS so that we can be comfortable with the expense of building our data system?
After serving the EOSDIS efforts handsomely, the Ad Hoc Working Group for Production (AHWGP) and Ad Hoc Working Group for Consumers (AHWGC) have run their course and probably should be disbanded. Kudos to the chairs, Barkstrom, Emery, Emmett, and Ramapriyan for their hard work pulling together difficult-to-obtain data for the ECS effort and ESDIS project office.
It is recommended that either the EOSDIS Panel elect members to join the nascent Federation after the CANs have been awarded to the type 2 and 3 ESIPs or the type 2 and 3 ESIP PIs join the EOSDIS Panel--to ensure the passing along of the corporate memory of where we've been during the last 8 years.