Lessons from the Blizzard of 2015 Concerning Uncertain Science/Weather Forecasts

The aftermath from the recent blizzard striking the northeast provides a fascinating example of decision making under uncertainty, without an available estimate of uncertainty. Meteorologists knew that there was uncertainty in their forecasts of the expect path of the blizzard, but most did not provide an estimate of that uncertainty. In contrast, meteorologists who provide forecasts of hurricane trajectories routinely provide a graphical assessment of landfall location probabilities, and local weather forecasters almost always provide a probability of rain for upcoming days.

Of course, as I have noted in previous blog posts, water quality modelers generally do not provide an estimate of the uncertainty in their forecasts. Without attempting to understand why different approaches to scientific uncertainty have emerged in these fields, it is quite clear that the deterministic forecasts of the blizzard trajectory (particularly around New York City) were believed by many to be precisely what would happen. Undoubtedly, the same people who expect daily probability of rain forecasts apparently were willing to accept no uncertainty in the blizzard forecasts.

 

After the blizzard, the “Monday morning quarterbacks” criticized meteorologists for their “faulty” forecasts, and they criticized decision makers for mandating extreme measures in preparation for the expected blizzard. In retrospect, meteorologists should have provided a “cone of uncertainty” in their forecasts of the trajectory of the blizzard.  In general, this would be useful for individual and societal blizzard preparation decision making. Further, it would have been helpful for other fields (such as water quality modeling) by acknowledging scientific uncertainty to the public.

Yet, if the blizzard forecasters had provided a visual estimate of uncertainty in the blizzard trajectory, similar to the forecasts provided by their hurricane-forecasting brethren, what might have changed? Probably very little, other than silencing most of the Monday morning quarterbacks. I conclude this because the blizzard was a high-consequence event, and people tend to be risk-averse.

Should blizzard preparation decisions have been made by meteorologists who knew of the forecast uncertainty, as some people have suggested? No. In my blog post “The Role of Scientists in Decision Making” (http://kreckhow.blogspot.com/2013/10/the-role-of-scientists-in-decision.html), I stress the point that to inform public sector decisions, scientists provide scientific assessment, but not the values necessary for decision making. These are public values, and they are provided by elected or appointed public officials as representatives of the public.

As I concluded in a previous blog post (“Scientific Uncertainty and Risk Assessment,” http://kreckhow.blogspot.com/2013/04/scientific-uncertaintyand-risk.html), every day, we make decisions based on an interplay, or mix, of uncertainty in an event (e.g., rain) and value (enjoyment) of an activity. We are used to weighing these considerations in our minds and deciding. These same considerations--getting new information on the weather (which is analogous to supporting new scientific research, as in adaptive management), and deciding how valuable the activity is to us (which is what we determine through cost/benefit analysis)--are key features of risk assessment. So let us move from our informal, everyday risk assessment to formal, scientific risk assessment, and identify the lesson and the opportunity as they relate to environmental management.

To me, the lesson in risk assessment is to recognize that the science in support of environmental management is usually uncertain, and sometimes highly uncertain. But the opportunity that is provided by risk assessment should result in improved decision making. To accomplish this, we must first require scientists to quantify or estimate the scientific uncertainty. Then we must require our decision makers to use the estimate of uncertainty to properly weigh the scientific information (not unlike what we do in our informal, everyday risk assessment). In the long run, this should improve environmental management decisions by making better use of the available information.

Lessons from the Blizzard of ’15 Concerning Uncertain Science/Weather Forecasts

The aftermath from the recent blizzard striking the northeast provides a fascinating example of decision making under uncertainty, without an available estimate of uncertainty. Meteorologists knew that there was uncertainty in their forecasts of the expect path of the blizzard, but most did not provide an estimate of that uncertainty. In contrast, meteorologists who provide forecasts of hurricane trajectories routinely provide a graphical assessment of landfall location probabilities, and local weather forecasters almost always provide a probability of rain for upcoming days.

Of course, as I have noted in previous blog posts, water quality modelers generally do not provide an estimate of the uncertainty in their forecasts. Without attempting to understand why different approaches to scientific uncertainty have emerged in these fields, it is quite clear that the deterministic forecasts of the blizzard trajectory (particularly around New York City) were believed by many to be precisely what would happen. Undoubtedly, the same people who expect daily probability of rain forecasts apparently were willing to accept no uncertainty in the blizzard forecasts.

After the blizzard, the “Monday morning quarterbacks” criticized meteorologists for their “faulty” forecasts, and they criticized decision makers for mandating extreme measures in preparation for the expected blizzard. In retrospect, meteorologists should have provided a “cone of uncertainty” in their forecasts of the trajectory of the blizzard.  In general, this would be useful for individual and societal blizzard preparation decision making. Further, it would have been helpful for other fields (such as water quality modeling) by acknowledging scientific uncertainty to the public.

Yet, if the blizzard forecasters had provided a visual estimate of uncertainty in the blizzard trajectory, similar to the forecasts provided by their hurricane-forecasting brethren, what might have changed? Probably very little, other than silencing most of the Monday morning quarterbacks. I conclude this because the blizzard was a high-consequence event, and people tend to be risk-averse.

Should blizzard preparation decisions have been made by meteorologists who knew of the forecast uncertainty, as some people have suggested? No. In my blog post “The Role of Scientists in Decision Making” (October 15, 2013), I stress the point that to inform public sector decisions, scientists provide scientific assessment, but not the values necessary for decision making. These are public values, and they are provided by elected or appointed public officials as representatives of the public.

As I concluded in a previous blog post (“Scientific Uncertainty and Risk Assessment,” April 26, 2013), every day, we make decisions based on an interplay, or mix, of uncertainty in an event (e.g., rain) and value (enjoyment) of an activity. We are used to weighing these considerations in our minds and deciding. These same considerations--getting new information on the weather (which is analogous to supporting new scientific research, as in adaptive management), and deciding how valuable the activity is to us (which is what we determine through cost/benefit analysis)--are key features of risk assessment. So let us move from our informal, everyday risk assessment to formal, scientific risk assessment, and identify the lesson and the opportunity as they relate to environmental management.

To me, the lesson in risk assessment is to recognize that the science in support of environmental management is usually uncertain, and sometimes highly uncertain. But the opportunity that is provided by risk assessment should result in improved decision making. To accomplish this, we must first require scientists to quantify or estimate the scientific uncertainty. Then we must require our decision makers to use the estimate of uncertainty to properly weigh the scientific information (not unlike what we do in our informal, everyday risk assessment). In the long run, this should improve environmental management decisions by making better use of the available information.