Multiattribute Decision Analysis for Water Quality Management

What can we learn from everyday decisions that can be helpful for critical thinking about complex decision making? To gain some insight concerning an answer to this question, let’s examine the decision that a family makes when considering a hike in the mountains for the weekend. The hike is an enjoyable family activity; it is an experience that has value, or in the terminology of decision analysis, it has utility for the family. In general terms, we can consider the utility, or “value,” of any item or experience as one of two essential components of a decision. For outdoor activities like hiking, the weather during the hike is also a factor; a family is apt to cancel or postpone a hike if the weather forecast calls for heavy rain. The weather in this situation is the state of nature, and the family’s knowledge of the weekend weather is uncertain, which is a common state of affairs for most decisions we confront that require us to estimate conditions in the future. In decision analysis, we characterize this uncertainty in the state of nature (e.g., the weekend weather forecast) as a probability. Probability is the second essential component of a decision.

Most interesting decisions involve multiple objectives or attributes. That is, real decisions usually require consideration of multiple endpoints or multiple outcomes of interest, such as overall costs, distribution of costs, environmental impacts, human health impacts,…To address these decisions, we can use the same approach involving probability and utility as described above. However, we first need to identify all objectives relevant to the decision, and the measures of effectiveness (or “attributes”) that indicate the degree to which each objective is achieved by a proposed action. Stated another way, all problem-specific objectives, and the attributes or features of an outcome that are valued by a decision maker and are affected by the decision, should first be determined.

While identification of each important objective may seem so obvious that it does not need to be stated, observation of current practice in environmental management indicates otherwise. In too many instances, relatively little time appears to be allocated to identifying and agreeing upon program objectives. Instead it seems that a few obvious objectives are quickly identified, and most of the effort is then devoted to data gathering, scientific research, modeling, and analysis.

For example, in lake eutrophication management studies, scientific research, monitoring, and assessment are often focused on quantifying the relationship between nutrient loading and in-lake nutrient (phosphorus and nitrogen) concentration. In some cases, this emphasis may be appropriate. However, in other cases this assessment focus may simply be following familiar, well-studied paths, with little forethought.  In these cases, a thoughtful consideration of objectives and attributes might have identified major fishkills as the most uncertain factor in need of scientific clarification. In this situation, the result of inadequate attention to the objectives may be an incomplete analysis or an analysis of the wrong problem.

   

The objectives of a problem under study may be clarified through the process of constructing an objectives hierarchy or value tree. For the management of eutrophication in Lake Sunapee, a recreational lake in New Hampshire, an objectives hierarchy has been constructed and is presented in Figure 1. This hierarchy begins with an all-encompassing objective at the top; a comprehensive set of issue-specific objectives is then derived with objectives that are consistent with the overall objective. Finally, attributes (identified by the arrowheads in the figure) that are meaningful, measurable, and predictable are derived for each specific objective.

Attributes provide the essential link between the program objectives or policy and the information needs. If decisions are to be made based on attribute levels, then the attributes must be meaningful to the decision maker. For example, even though Lake Sunapee is currently managed based on total phosphorus concentration, Figure 1 indicates that total phosphorus is not a meaningful attribute to decision makers. Meaningful attributes for eutrophication concern the areal extent of aquatic weed growth, fish quantity and quality, and other measures of direct concern to the public presented in Figure 1. While attributes like these are more difficult to scientifically understand and predict, they do reflect public values or utility, and thus they will be a measure by which the public assesses the success of a management program. The decision maker should translate all objectives into meaningful attributes like those above and then present these attributes to scientists/engineers as indicative of the specific information needs for the problem under study.

 

Figure 1

It is possible, of course, that the scientist/engineer may be unable to quantify or model an important attribute. Another necessary condition for attributes is that they should be measurable or that they can be predicted reasonably well with a mathematical model or with expert scientific judgment. In order for the scientist to provide information on an attribute, it must be possible to measure or observe the attribute. Alternatively, if prediction of a future, unrealized level of the attribute is needed, then consideration must be given to specifying, calibrating, and testing a model (mathematical, judgmental, or both) that can be used to provide the prediction.

Attribute determination may be an iterative process involving the scientist and the decision maker. Some attributes may not be both meaningful and measurable; as a result, compromises may be required to identify measurable attributes that have meaning to the decision maker. The final choice of the attributes should be the responsibility of the decision maker, not of the scientist/engineer, since the decision maker must interpret and use the information for management purposes.

Once there is general agreement on the management objectives and attributes, the analysis can begin; the purpose of the analysis at this stage is to estimate or predict the levels of the attributes associated with implementation of each of the management options. For the management of eutrophication in Lake Sunapee, the first column of Figure 2 provides a list of some of the options that have been proposed. Across the top of the table are the attributes identified through the development of the objectives hierarchy.

 

Figure 2

The next step sounds straightforward but is extremely difficult to do thoroughly and well - fill in Figure 2. The entries in the body of the table should represent what each management option achieves for each attribute. Thus, for example, the table cell for the intersection of "restrict shoreland fertilizer application" (management action) with "water quality standards" (attribute) should contain a prediction (with uncertainty estimated) of the level of the attribute expected if that particular management strategy is implemented. In Figure 3, a miniature “boxes and arrows” diagram is presented in this table cell to represent a probabilistic Bayes network model that would be used to predict the effect of that management action (restrict shoreland fertilizer application) on that attribute (water quality standards), with uncertainty analysis. This Bayes network is shown in Figure 4. The objectives-attributes table is presented again in Figure 5, with the prediction from the Bayes network model shown as a probability density function for the water quality standards attribute. In principle, models like the Bayes network would be applied to completely fill-in the objectives-attributes table; in reality, the most cost-effective management actions will be analyzed for the most important attributes.

 

Figure 3

Figure 4

As a final thought, several points should be made concerning this assessment for Lake Sunapee:

(1) Some attributes still need to be more specific (e.g., What are appropriate units of measurement for "fish quantity and quality"?).

(2) The management options need more explanation (e.g., What are the viable limits on impervious area, shoreland lawn, and marina activity?) so that predictions can be made.

(3) An overall strategy may involve a combination of management options.

(4) Prediction of attribute level is likely to involve a combination of statistical relationships, mechanistic simulation models, uncertainty analysis, and expert judgment.

How Protective of Designated Use are Nutrient Criteria?

The U.S. Environmental Protection Agency has recommended an ecoregion-based national strategy for establishing nutrient criteria. The importance of nutrient criteria is evident from the Clean Water Act’s required listing of impaired waters under Section 303(d); state water quality standard violations due to nutrient overenrichment are a leading cause of surface water impairment. Clearly, a sound scientific basis is needed for the many costly total maximum daily loads (TMDLs) that will be required.

Eutrophication-related water quality standards and criteria already widely exist. For example, most states have dissolved oxygen criteria intended to be protective of designated uses that are impacted by oxygen depletion, resulting from nutrient-enhanced algal production. Additionally, some states have adopted nutrient or chlorophyll criteria; for example, North Carolina has a chlorophyll a criterion of 40 ug/l. However, criteria like the North Carolina chlorophyll criterion were set years ago using informal judgment-based determinations; the EPA’s new strategy reflects a recognition that more analytic rigor is needed given the consequences of TMDL decisions.

State water quality standards are established in accordance with Section 303(c) of the Clean Water Act and must include a designated-use statement and one or more water quality criteria. The criteria serve as measurable surrogates for the narrative designated use; in other words, measurement of the criteria provides an indication of attainment of the designated use. Additionally, violation of the criteria is a basis for regulatory enforcement, which typically requires establishment of a TMDL. Thus, good criteria should be easily measurable and good predictors of the attainment of designated use.

This latter basis for criteria selection – that they must be good predictors of the attainment of designated uses, is the motivation for the analysis described in Reckhow (2005). I believe that the best criterion for eutrophication-related designated use is a measurable water quality characteristic that is also the best designated use predictor. In addition, I believe that there are alternative and arguably better ways to define the criterion level than through reference to least impacted waterbodies expected to be in attainment of designated use. Rather, because it is an enforceable surrogate for designated-use attainment, the level of the criterion should be chosen on the basis of societal values, which should reflect the realities of society’s tradeoffs between environmental protection and cost. Beyond that, selection of the level of the criterion should realistically take into consideration natural variability and uncertainty in predicting water quality outcomes, both of which imply that 100% attainment in space/time is not a realistic basis for a water quality standard.

Designated uses evolved from the goals of the Clean Water Act. As part of the water quality standard for a regulated water body, they are typically expressed as brief narrative statements listing the uses that the waterbody is intended to support, such as drinking water, contact recreation, and aquatic life. Water quality criteria must then be chosen as measurable quantities that provide an indication of attainment of the designated use. Finally a criterion level (and possibly the frequency and duration) must be selected as the cutoff point for nonattainment.

Traditionally, the task of setting criteria has involved judgments by government and university scientists concerning the selection of specific water quality characteristics and the levels of those characteristics that are associated with the designated use. For example, consider the North Carolina chlorophyll a criterion of 40 ug/l, which was established in 1979. This criterion applies to Class C waters, which are freshwaters with use designations of secondary recreation, fishing, and aquatic life support. To establish this criterion, the NC Division of Environmental Management examined the scientific literature on eutrophication and then recommended a chlorophyll criterion level of 50 ug/l to a panel of scientists for consideration. After reviewing a study of nutrient enrichment in 69 North Carolina lakes, the panel responded that 40 ug/l reflected a transition to algal, macrophyte, and DO problems and thus represented a better choice. Following public hearings, 40 ug/l was adopted as the chlorophyll water quality criterion. The 40 ug/l criterion developed from an ad hoc process of science-based expert judgment. In my view, we should avoid selecting a criterion level simply because it represents a change/transition point in waterbody response (e.g., transition to algal, macrophyte, and DO problems). The criterion level should also reflect public values on designated use; good water quality criteria selection is not strictly a scientific endeavor.

The current U.S. EPA approach for nutrient criteria development is a similar mix of science and expert-judgment. In 1998, the President’s Clean Water Action Plan directed the EPA to develop a national strategy for establishing nutrient criteria. The resultant multiyear study produced a set of documents and recommended criteria based on ecoregions and waterbody type. Specific modeling methodologies were proposed to aid in the extrapolation of reference conditions and to assist managers in setting loading allowances once nutrient criteria have been established. In addition, enforcement levels for the proposed criteria were based on “reference waterbodies” perceived to reflect essentially unimpacted or minimally-impacted conditions.

In principle, standard setting should be viewed from the perspective of decision making under uncertainty, involving interplay between science and public opinion. The determination of designated uses reflects public values, both in the statements in the Clean Water Act and in the waterbody-specific statement of designated use. The selection of the criterion is a choice based largely on science. Selection of a good criterion, one that is easily and reliably measured and is a good indicator of designated use, is largely a scientific determination.

However, determination of the level of the criterion associated with the attainment-nonattainment transition ideally requires the integration of science and values. Natural variability and scientific uncertainty in the relationship between the criterion and the designated use imply that selection of a criterion level with 100% assurance of use attainment is generally unrealistic. Accordingly, scientific uncertainty and attitude toward risk of nonattainment should be part of the criterion level decision. Therefore, the decision on a criterion level might be addressed by answering the following question: Acknowledging that 100% attainment is impractical for most criteria, what probability (or, perhaps, what percentage of space-time) of nonattainment is acceptable? EPA guidance addresses this question by suggesting that 10% of samples may violate a criterion before a waterbody is listed as not fully supporting the designated use. Analytically, this question may be answered by integrating the probability of use attainment (for a given criterion level) and a utility function reflecting water quality costs and benefits. The criterion level associated with the highest expected utility might then be chosen. Realistically, this decision analytic framework is prescriptive; it guides us toward what ought to be done, but it almost certainly exceeds what actually will be done.

An additional consideration that was discussed in NRC (2001) is where in the causal chain from pollutant source to designated use should a water quality criterion be placed? Referring to the figure (taken from NRC 2001), the NRC panel recommended that the preferred “location” should be in the “human health and biological condition” box. If instead, the pollutant loading or waterbody pollutant concentration box was selected, there would be additional hidden uncertainty in the causal chain (in the figure) to designated use. This hidden uncertainty can be reduced by selection of a criterion as close as possible to designated use.

In Reckhow et al. (2005), we addressed the process of numeric water quality criteria setting from the prescriptive basis that criteria should be predictive of designated use and from the pragmatic basis that risk of nonattainment should be acknowledged and therefore considered when setting a level or concentration. Thus, from a prescriptive standpoint, a good criterion should be an easily measurable surrogate for the narrative designated use and should serve as an accurate predictor of attainment. Correspondingly, from a pragmatic perspective, natural variability and criterion-use prediction uncertainty will likely result in some risk of nonattainment; thus the selection of a criterion level for the attainment-nonattainment transition realistically should be based on an acceptable probability of nonattainment. Furthermore, the selection of the acceptable probability is a value judgment best left to policy makers informed by scientists. To illustrate how this could be accomplished, Reckhow et al. (2005) used structural equation modeling to quantify the relationship between designated use and possible water quality criteria. This identified the best predictor of designated use, which would become the water quality criterion. This result can then be presented to decision makers for selection of the criterion level associated with the acceptable risk of nonattainment. Given the estimated number of nutrient-related TMDLs required, and the costs/benefits of addressing these ambient water quality standard violations, it is clear that the choice of water quality criteria for eutrophication management and nutrient TMDLs has significant consequences. Thus a rigorous procedure, like that described in Reckhow et al. (2005), should be considered for establishment of nutrient criteria.

NRC. 2001. Assessing the TMDL Approach to Water Quality Management; National AcademyPress: Washington, D.C.

Reckhow, K.H. G.B. Arhonditsis, M.A. Kenney, L. Hauser, J. Tribo, C. Wu, K.J. Elcock, L.J. Steinberg, C.A. Stow, S.J. McBride. 2005. A Predictive Approach to Nutrient Criteria. Environmental Science and Technology. 39:2913-2919. (https://www.researchgate.net/publication/7814574_A_predictive_approach_to_nutrient_criteria?ev=prf_pub)

How “Mechanistic” are Mechanistic Water Quality Models?

Mechanistic surface water quality models have been developed by scientists and engineers as mathematical descriptions of hydrologic and ecologic processes. Mechanistic modelers have tended to concentrate on the mathematical expression of theory, probably as a consequence of: (1) scientific interest and challenge, (2) a belief that the theory was reasonably well-understood and that this understanding could be expressed mathematically, (3) limited available data to fit and evaluate models, and (4) limited resources to collect additional data. For these reasons, model coefficients and reaction rates in mechanistic models generally are intended to characterize actual processes and are not (prior to model “tuning”) intended to be empirically-fitted constants (which might be considered an “effective” value for a model parameter).

Since the parameters of mechanistic models are intended to describe real processes, it may be assumed that an experimental study of a particular process can yield a parameter estimate that can be directly inserted into the model. In some cases, it is acknowledged that a reaction rate or coefficient in a model is affected by certain conditions in a waterbody (e.g., turbulence), and thus adjustments must be made to the experimentally-based value. However, if the model truly is a complete mechanistic description of the system of interest, then adjustment should be unnecessary; this is the underlying belief of modelers who advocate development of “physically-based” models.

However, given the relative simplicity of all simulation models in comparison to the complexity of nature, it seems reasonable to question the legitimacy of any "mechanistic" mathematical description of surface water quality. Further, given data limitations and scientific knowledge limitations, it seems reasonable to question even the goal to strive for a model that need not be calibrated. The correctness of model structure, the knowledge of the model user, and the availability of experimental and observational evidence all influence parameter choice for mechanistic models. Unfortunately, too often knowledge and data are extremely limited, making choice of parameters and choice of important model processes guesswork to a distressingly large degree. The example presented below is not re-assuring with respect to these two issues: (1) scientific support for the selection of model parameters, and (2) scientific support for the specification of appropriate model functional relationships.

One of the basic functions in an aquatic ecosystem model is phytoplankton settling. An early example of its use is in the model proposed by Chen and Orlob (1972):

   

  where:

         V = segment volume (m3)

         C1 = phytoplankton concentration (g/m3)

         Q = flow volume (m3/t)

         E = diffusion coefficient (m2/t)

         A = segment surface/bottom area (m2)

         µ1 = phytoplankton growth rate (t-1)

        R1 = phytoplankton respiration rate (t-1)

         s1 = phytoplankton settling rate (t-1)

         M1 = phytoplankton mortality rate (t-1)

         µ2 = zooplankton growth rate (t-1)

         C2 = zooplankton concentration (g/m3)

         F2,1 = fractional feeding preference

Other examples are quite similar; a common alternative approach is that phytoplankton settling is sometimes treated as a velocity term with an areal loss:

     phytoplankton settling (mass/time) = v1AC1                                                       

To understand some of the problems with the current approach for parameter determination in mechanistic surface water quality models, it is useful to examine this process further. For that purpose, "phytoplankton settling velocity" provides a good example. Phytoplankton, or algae, are important in aquatic ecosystems, and thus one or more phytoplankton compartments are found in most mechanistic surface water quality models concerned with nutrient enrichment. Phytoplankton settling is one of the key mechanisms for removal of phytoplankton from the water column.

Stoke's law provides the starting point for the mathematical characterization of phytoplankton settling. Few models, however, employ Stoke's law; instead a simple constant settling velocity (in units of length/time) expression is commonly used. To apply a model with this settling velocity term, a modeler must either measure phytoplankton settling directly, or select a representative value from another study. Since field measurement of phytoplankton settling is a difficult task, use of literature-tabulated values is standard practice.

Probably the most thorough listing of suggested values for phytoplankton settling velocity continues to be Bowie et al. (1985), which presents a thorough table of reported values, by algal type (see the table below). Bowie et al. note that under quiescent conditions in the laboratory, phytoplankton settling is a function of algal cell radius, shape, density, and special cell features such as gas vacuoles and gelatinous sheaths. For natural water bodies, water turbulence can be quite important. In two- or three-dimensional models with hydrodynamic simulation, turbulence is accounted for in the model equations; in zero- or one-dimensional models, the effect of turbulence on phytoplankton settling must usually be incorporated into the choice of settling velocity.

That information is typically the extent of technical guidance considered by modelers when selecting this parameter using a reference like the table from Bowie et al. The range of options in the table is substantial, even within a single category (e.g., diatoms) for algal type. The algal cell size, shape, and other features mentioned in the previous paragraph can vary from species to species within a single type category, so this may be responsible for some of the variability in the table. However, even if the modeler who must choose a point estimate has data that identify dominant species in a water body at a particular time and location, dominance is apt to change with time and location. Further, models contain at most only a few distinct phytoplankton compartments, so a choice must still be made concerning species to be modeled and their characteristics.

Examination of the original references from which the table was created does little to enlighten the parameter selection process. Most of the references summarized in the table do not present observational studies of phytoplankton; rather, they are simulation model studies, and the value for phytoplankton settling velocity listed in the table is the value chosen for the model. In some of the references checked, little or no basis was provided for the choice. When a rationale for choice was given, it was usually to adopt or adjust the few values presented in the literature from experimental studies, or to adopt a value from another modeling study. In one way or another, it appears that virtually all of the values presented in the table have some dependency on the early experimental work of Smayda and Boleyn (1965) and other work by Smayda.

Unfortunately, evaluation studies of simulation models have provided little insight on good point estimates for this parameter. Observational data on surface water quality are almost always inadequate for testing functional relationships and assessing parameter choices. Typical observational data sets are noisy, with few measurements of each of only a few variables. In the case of phytoplankton settling velocity, observational data are apt to consist of phytoplankton cell densities at various dates, times, and areal locations, but probably not depths. Since phytoplankton are also removed from the water column through consumption by higher food chain organisms, the observational data do not permit separate identification of the removal mechanisms.

Given this situation, modelers have relied almost exclusively on the few experimental studies in the laboratory and their judgment concerning adjustments to these values. For one-dimensional models without explicit modeling of hydrodynamics, the chosen value may be as much as an order of magnitude higher than the laboratory values. Two- or three-dimensional models with hydrodynamics may incorporate the unadjusted laboratory value. After early modeling studies presented chosen values, these values were sometimes adopted in subsequent studies without comment (in effect, "default" values were identified). Thus, there is probably much less information in the columns of the table than implied by the number of values reported.

In summary, the choices for phytoplankton settling velocity appear to be based on ad hoc adjustments to a few values measured under controlled conditions. There is virtually no field confirmation of choices made for parameters individually (as opposed to collectively). This situation is fairly typical of the state-of-the-art in mechanistic surface water quality simulation modeling.

Bowie, G.L., Mills, W.B., Porcella, D.B., Campbell, C.L., Pagenkopf, J.R., Rupp, G.L., Johnson, K.M., Chan, P.W.H., Gherini, S.A., and Chamberlin, C.E., 1985. Rates, Constants, and Kinetics Formulations in Surface Water Quality Modeling. U.S. Environmental Protection Agency, EPA/600/3-85/040.

Chen, C.W and Orlob, G.T., 1972. Ecologic Simulation for Aquatic Environments. Office of Water Resources Research, US Dept. of Interior. Washington, DC.

Smayda, T.I. and Boleyn, B.J., 1965. Experimental observations on the floatation of marine diatoms. Part I: Thalassiosira naria, T. rotula and Nitzschia seriata. Limnol. and Oceanogr., 10:499-510.

Dealing Effectively with Uncertainty

Are we better off knowing about the uncertainty in outcomes from proposed actions? That is, will our decisions generally be better if we have some idea of the range of possible outcomes that might result? I have always thought so, and yet current practice in water quality modeling and assessment suggests that others feel differently or perhaps believe that uncertainty is small enough so that it can be safely ignored.

Consider my experience from many years ago. While in graduate school, I became involved in a proposed consulting venture in New Hampshire. As a young scientist, I was eager to “shake up the world” with my new scientific knowledge, so I suggested to my consulting colleagues that we add uncertainty analysis to our proposed 208 (remember the Section 208 program?) study. Everyone agreed; thus we proposed that uncertainty analysis be a key component of the water quality modeling task for the 208 planning process. Well, after we made our presentation to the client, the client’s first question was essentially, “The previous consultants didn’t acknowledge any uncertainty in their proposed modeling study, what’s wrong with your model?” This experience made me realize that I had much to learn about the role of science in decision making and about effective presentations!

While this story may give the impression that I’m being critical of the client for not recognizing the ubiquitous uncertainty in environmental forecasts, in fact I believe that the fault primarily lies with the scientists and engineers who fail to fully inform clients of the uncertainty in their assessments. Partially in their defense, water quality modelers may fail to see why decision makers are better off knowing the forecast uncertainty, and perhaps modelers may not want to be forced to answer the embarrassing question like that posed to me years ago in New Hampshire.

For this situation to change, that is, for decision makers to demand estimates of forecast error, decision makers first need: (1) motivation - that is, they must become aware of the substantial magnitude of forecast error in many water quality assessments, and (2) guidance – they must have simple heuristics that will allow them to use this knowledge of forecast error to improve decision making in the long run. Once this happens, and decision makers demand that water quality forecasts be accompanied with error estimates, water quality modelers can support this need through distinct short-term and long-term strategies.

Short-term approaches are needed due to the fact that most existing water quality models are incompatible with complete error analysis as a result of overparameterization; thus short-term strategies should be proposed for: (1) conducting an informative, but incomplete error analysis, and (2) using that incomplete error analysis to improve decision making. In the long-term, recommendations can be made to: (1) restructure the models so that a relatively complete error analysis is feasible, and/or (2) employ Bayesian approaches that are compatible with adaptive management techniques that provide the best approach for improving forecasts over time.

In the short-term, if knowledge, data, and/or model structure prevents uncertainty analysis from being complete, is there any value in conducting an incomplete uncertainty analysis? Stated another way, is it reasonable that decision making will be improved with even partial information on uncertainties, in comparison to current practice with no reporting of prediction uncertainties? Often, but not always, the answer is “yes,” although the usefulness of incomplete uncertainty characterization, like the analysis itself, is limited.

Using decision analysis as a prescriptive model, we know that uncertainty analysis can improve decision making when prediction uncertainty is integrated with the utility (or loss, damage, net benefits) function to allow decision makers to maximize expected utility (or maximize net benefits). When uncertainty analysis is incomplete (and perhaps more likely, when the utility function is poorly characterized) the concepts of decision analysis may still provide a useful guide.

For example, triangular distributions could be assessed for uncertain model terms, and assuming that parameter covariance is negligible (which unfortunately may not be the case), then limited systematic sampling (e.g., Latin hypercube) could be used to simulate the prediction error. The result of this computation could be either over/under estimation of error, but it does provide some indication of error magnitude. However, this information alone, while perhaps helpful for research and monitoring needs, is not sufficient for informed decision making. The approximate estimates of prediction uncertainty need to be considered in conjunction with decision maker attitudes toward risk for key decision variables.

Implicit in this attitude toward risk is an expression of preferences concerning tradeoffs. For example, are decision makers (or stakeholders, or other affected individuals/groups) risk averse with respect to ecological damage, such that they are willing to increase project costs in order to avoid species loss? If a reasonable quantification of prediction uncertainty were available for the decision attribute - loss of an endangered species, then the prediction might be expressed as “there’s a 40% chance of loss of this species with plan A, but only a 5% chance of loss with plan B.” When costs of the plans are also considered, the tradeoff between species loss and cost is augmented by awareness of risk that comes from the prediction uncertainty characterization. Risk is not evident from deterministic (point) predictions of the decision attributes, so the decision is likely to be better informed with the risk assessment that is made possible with prediction uncertainty.

In the long run, a better strategy is to restructure the models, emphasizing the development of models that are compatible with the need for error propagation and adaptive assessment/management. Bayesian (probability) networks are particularly suitable for this task (see http://kreckhow.blogspot.com/2013/07/bayesian-probability-network-models.html), as are simulation techniques that address the problem of equifinality resulting from overparameterized models (see: http://kreckhow.blogspot.com/2013/06/an-assessment-of-techniques-for-error.html).

No one can claim that scientific uncertainty is desirable; yet, no one should claim that scientific uncertainty is best hidden or ignored. Estimates of uncertainty in predictions are not unlike the point estimates of predicted response. Like the point predictions, the uncertainty estimates contain information that can improve risk assessment and decision making. The approaches proposed above will not eliminate this uncertainty nor will it change the fact that, due to uncertainty, some decisions will yield consequences other than those anticipated. They will, however, allow risk assessors and decision makers to use the uncertainty to structure the analysis and present the scientific inferences in an appropriate way. In the long run, that should improve environmental management and decision making.

Are Periodic Beach and Shellfish Bed Closures a Cost of Development?

In the past ten years, I have assessed beach and shellfish bed closures due to exceedances of indicator organism criteria in both fresh and salt water. In each case, the problem appears to be caused by factors/sources that are unlikely to be effectively controlled to eliminate the exceedances. In some cases, the proliferation of birds, such as Canada Geese and gulls, which have expanded their numbers due to their successful adaptation to our urban/suburban environment, is a key factor. In other cases, subsurface drainage to lower water tables in estuarine environments to allow new residential development leads to a first flush following rainfall that causes a short-term exceedance of bacterial indicator organisms. In still other situations, many older developed areas around fresh and salt waterbodies are served by storm drains that result in bacterial exceedances following storms. The nature of each of these sources, and perhaps the lack of public will to incur the substantial cost to reduce these exceedances, may mean that these problems are part of modern life.

A pragmatic strategy has emerged to allow us to “live in harmony” with these realities. State agencies are using local rain gauges to estimate when individual rainstorms are of sufficient magnitude to result in exceedances of indicator organism concentrations in surface water bodies. Once “exceedance rainfall” is observed, temporary closures are posted. After a few days, the state agency will sample the affected surface waters, and if indicator organism concentrations are below the water quality criterion (standard), then the area is re-opened for the designated uses.

Unfortunately, permanent solutions to this problem are likely to have highly uncertain effectiveness and may be quite costly. So, while we may not like the current approach, as it may suddenly disrupt plans for recreation, we may be unwilling to assume the cost of change. Ultimately, this may be another example where the goals of the Clean Water Act (to eliminate the discharge of pollutants into the nation’s waters, and to achieve water quality levels that are fishable and swimmable) are unlikely to be attained.

The Role of Scientists in Decision Making

I have been working recently on a project to assess the “adequacy” of data to inform water quality management and decision making. Is this an appropriate task for a scientist? That is, should a scientist assess the adequacy of data/information to make water quality management decisions? Should a scientist recommend water quality standards? My answer to both questions is no, these are not appropriate tasks for scientists. What does my answer imply about the role of scientists in water quality management and decision making?

Scientists have an important responsibility in the interpretation of science to inform environmental management and policy development. For example, as a scientist, I know the strengths and weaknesses in scientific knowledge in my area of study, and thus I can readily identify gaps in that knowledge. Therefore I and my scientific colleagues should be consulted in an evaluation of general scientific research needs in our areas of expertise. That point seems obvious.

But, I am also a private citizen with personal beliefs, values, and preferences. How do I express those? Should those personal preferences affect my scientific input in support of management and policy? While it has been my experience that people often expect scientists to provide policy recommendations, I think that, upon reflection, most people would prefer a scientific assessment untainted by personal values. In effect, people would prefer a scientific expert to assess the impact of various courses of action, but not to recommend a course of action. In principle, assessment involves scientific expertise but not personal values or preferences. For example, assessments may be summarized by statements such as "if you do A, then my scientific analysis indicates that X is expected to happen" or "if you do B, then my scientific analysis indicates that Y is expected to happen," and so on.

If, however, I were to say "I recommend that you do C," then behind that recommendation, besides science, is my preference for the tradeoffs implicit in recommended action C. What are these tradeoffs? They involve all things affected by the decision, such as: Who pays? How much? What water quality conditions are achieved? As a scientist, I do not have the right to make decisions on those tradeoffs; that right is granted to elected or appointed public officials in most cases. I may have an opinion on the tradeoffs, but as a scientist/citizen, I express those opinions in others ways, such as in the voting booth.

Consider a specific example. At the local and regional level, many scientists volunteer their services on community environmental affairs boards. I have done this in the past in Durham, NC. If, as a member of the Durham Environmental Affairs Board (EAB), I am asked to assist the Durham County Commissioners with their decision concerning allowable land use to protect a water supply reservoir, how should my scientific assessment be expressed? Remember, as a member of the Durham EAB, I am serving as a technical expert. In my role as a technical expert, I am in a good position to use analysis and scientific assessment to make "if-then" statements. Examples of this are: "If Durham adopts land use strategy A, then based on my modeling and scientific analysis, X is predicted to happen to water quality" and "If Durham adopts strategy B, then Y is predicted to happen" and so on. The elected County Commissioners then take the scientific assessment from the EAB, along with technical assessments concerning other relevant attributes, and decide. Decisions by the County Commissioners should reflect community values, tradeoffs, and preferences. If they do not, then the voters have the opportunity to express their dissatisfaction with the Commissioners in the next election.

In summary, the community preferences and values are expressed in the decision by the Commissioners. Scientists provide technical assessments that may require interpretation and explanation. However, the scientific input should not be expressed as a management recommendation, and thus take decision making authority from those who have decision making responsibility.

What does this discussion mean, or imply, about standard setting (e.g., establishing nutrient standards)? Very simply, standard setting is decision making; it should be based on the same principles as outlined above. Thus, if scientists recommend specific water quality water quality standards, then they are making decisions, and in doing so they are expressing their values and preferences. Here, too, the proper role of the scientist is one of assessment, not recommendation and not decision making. For phosphorus water quality standards, scientific assessment might be expressed as "if the growing season total phosphorus standard is set at 0.040mg/l, then algal bloom conditions are predicted for 10 percent of the waterbodies in the state and recreational conditions are expected to be ...; if instead it is set at 0.030mg/l, then ..."  Similarly, an economic assessment might be provided to indicate the expected costs to achieve the standard. All important assessments like these are then provided to the appropriate decision maker(s) who must employ citizen values, preferences, and regulatory mandates to make the necessary tradeoffs and establish the standard.

With respect to my current project on the “adequacy” of water quality data to support decision making, stating that data are “adequate” for decision making is a value judgment that should not be made by scientists. In this situation, the scientist should assess the uncertainty in the data in terms that are understandable to stakeholders and decision makers. This allows decision makers to determine when data are adequate for their needs.

As scientists, we may truly believe that we know the best actions for water quality management. But, are all scientists going to be as enlightened as we are? Further, we may encounter decision makers who want us to recommend an action, and in effect make the decision for them. In those situations, will all scientists have the personal integrity that we have? We should resist these tempting opportunities, and instead work toward better-informed decision makers and citizens. In the long run, democracy and government accountability will benefit and good decisions will follow.