A Few Comments on “End-User Forecast Products” as input to the USWRP Warm Season Workshop of 5-7 March 2002
Roger A. Pielke, Jr.
University of Colorado
CIRES Center for Science and Technology Policy Research
1 February 2002
These comments begin with the assertion that if a goal of a Warm –Season QPF Project is to contribute to the practical use of the results of scientific research (i.e., through the eventual development of operational products and services), then such contributions will be more effectively made with consideration of issues associated with use and value as an integrated component of the research program. This short essay suggests a few issues to consider in the structure and content of a “Warm-Season QPF Project.” Links and references are provided for those wishing to explore further.
Products and services related to research results are solutions. Understanding the needs of end users is tantamount to defining the problem. With a better understanding of problems, it is logical that solutions might be more effectively tailored to meet the needs of end users. It should be pointed out that “end users” spans a wide range of groups and individuals, and often the path from research to end user is mediated by entities in the public and/or private sectors. Clarity in defining who, exactly, the end users or mediators actually are is essential to effective consideration of use in research design.
Working from problem to solution, there are at least four areas of research that would contribute a great deal to making more effective the connections of research and application:
For further reading:
Pielke, Jr., R. A., 1997: Asking the Right Questions: Meteorological Research and Societal Needs. Bulletin of the American Meteorological Society, 78(2), 255-264.
Pielke Jr., R. A., 1997: Reframing the U.S. Hurricane Problem. Society and Natural Resources, 10, 485-499. http://sciencepolicy.colorado.edu/pielke/hp_roger/pdf/1997.12.pdf
For further reading:
Pielke, Jr., R. A., D. Sarewitz, R. Byerly, and D. Jamieson, 1999: Prediction in the earth sciences: Use and misuse in policy making. EOS: Transactions of the American Geophysical Society, 80, #309 ff.
See also Katz and Murphy, Economic Value of Weather and Climate Forecasts (Cambridge 1997), http://www.cup.org/ObjectBuilder/ObjectBuilder.iwx?processName=productPage&product_id=0521434203&origin=redirect
Stewart, T. R., R. Nath, and R. A. Pielke, Jr. 2001: Societal Value of Improved Precipitation Forecasts: A Case Study in Surface Transportation, Final report submitted to the Forecast Systems Laboratory, National Oceanic and Atmospheric Administration.
And see Rick Katz’s homepage at:
And the searchable online bibliography on the use and value of weather and climate forecasts: http://sciencepolicy.colorado.edu/biblio/index.html
For further reading:
Pielke, Jr., R. A.,. 2001: Weather Research Needs of the Private Sector Workshop Report, U.S. Weather Research Program, Palm Springs. CA, December 2000.
See also, NRC 2001. Crossing the Valley of Death, http://www.nap.edu/books/0309069416/html/
Finally, Section 2.4 of the THORpex proposal to the WWRP begins to introduce some aspects of technology assessment. It can be found at:
See Chapter 1 in Katz and Murphy, Economic Value of Weather and Climate Forecasts (Cambridge 1997), http://www.cup.org/ObjectBuilder/ObjectBuilder.iwx?processName=productPage&product_id=0521434203&origin=redirect
Pielke, Jr., R.A., 1999: Who Decides? Forecasts and Responsibilities in the 1997 Red River Flood. American Behavioral Science Review 7, 2, 83-101.
Stewart, T. R., R. Nath, and R. Pielke, Jr. 2001. Understanding user decision making and value of improved precipitation forecasts: Lessons from a case study, Paper presented at the WWRP QPF Verification Workshop, Prague, Czech Republic 14-16 May 2001.
And excerpt from that paper’s conclusions follows (emphasis added):
The data (including meteorological data) necessary to support credible and defensible quantitative estimates of the value of improved forecasts are lacking, returning us to where we had begun with forecast value conclusions dependent almost entirely upon basic assumptions and not the empirical data of the case. Responsible estimates would have confidence bounds so wide as to be useless. A major lesson learned from this research is the critical importance of forecast verification data, data on actual decisions, and cost data for supporting studies of the value of weather information.
To realize the potential of improved forecasts would require careful attention to the forecast products themselves (and their production and dissemination through the public and private sectors) as well as the role of forecast products in the specific context of actual decision making processes. The major implication of this study is that a more holistic approach to understanding and realizing forecast value is needed, i.e., one in which information (both of forecast skill and usage) centered on the decision process is collected in a much more intensive manner than is presently the case. Just as meteorologists need detailed knowledge of atmospheric conditions to project atmospheric conditions, so too do students of forecast value need detailed observational knowledge in order to realistically project forecast value. Absent such detail, studies of forecast value will simply be extensions of the assumptions brought to bear upon the study.
And finally, see Harold Brooks online bibliography on verification and evaluation:
The four areas summarized in the bullets above each imply an important set of researchable questions. To date very little research attention has been paid to answering such questions, and the research that does exist is rarely well integrated with ongoing meteorology efforts. One suggestion would be to design and implement an integrated element of a “Warm-Season QPF Project” that focuses on these questions which has equal stature to other research elements. And it must be emphasized that to answer these questions will require resources, just as required by other elements of the program.
More broadly a research agenda for the “societal aspects” of weather has been developed and refined over the past decade under the U.S. Weather Research Program. Various aspects of this research agenda are described in the following USWRP reports: PDT-1, PDT-6, PDT-7, PDT-10, the Hurricane Landfall Implementation Plan, the Quantitative Precipitation and Data Assimilation Implementation Plan, the USWRP Vision Document for 2000-2006, and the Report of the USWRP Workshop on the Societal Aspects of Weather.
Elements of these research plans could easily be incorporated into a Warm-Season QPF Project with little need for additional deliberation/scoping of research questions, etc.