Last month on my blog,
I posted “Is Conventional Water Quality Model
Verification A Charade?” and I concluded that model verification is indeed a
charade as currently practiced. I received several comments on my posting of
this blog piece to LinkedIn, and a number of those who commented agreed with my
conclusions. So, what is to be done?
In
that blog post, I proposed a probabilistic method for assessing the rigor of a
model verification assessment. While my proposal is helpful for quantifying verification
rigor in an application, it neglects the “track record” of a particular model;
this is unfair to the model and to model users since my verification rigor test
depends on the availability of calibration-independent data. Thus, it is
conceivable that a good model will receive a poor verification rigor score
purely due to limited verification data.
To
address this conundrum, I propose that we employ a Bayesian approach that
includes an informative prior, and this prior should be based on an ex post
facto analysis of model performance. The National Academy of Sciences TMDL panel
that I chaired in 2001 discussed the need for this type of analysis for models
used for TMDL development; to my knowledge, this was never undertaken. So, what might be done to address this issue using
Bayes Theorem? In brief, it should not
be difficult to apply Bayes Theorem for a particular water quality model to
quantitatively combine:
1.
a statistical assessment of the prediction-observation
difference for any model, based on data collected after implementation
(and after water quality response) of a pollutant load management action.
2.
the results from the model verification
rigor assessment that I proposed, for the particular model application of
interest.
Let us consider making this a requirement for water quality models that
are used to inform multi-million dollar decisions!
No comments:
Post a Comment