7/29/2019 Alberto Montanari p 1

1/1

Assessing the uncertainty of rainfall-runoff simulations:

a review and comparison of different techniquesAlberto Montanari

Faculty of Engineering, University of Bologna(alberto.montanari@unibo.it)http://www.costruzioni-idrauliche.ing.unibo.it/people/alberto/indexeng.html

Uncertainty in the model output (total uncertainty) may be

due to:

Sources of uncertainty in rainfall-runoff modelling

Input uncertainty

Parameter uncertainty

Model uncertainty

A relevant source of uncertainty in rainfall-runoff modelling

may be non stationarity, which may originate observable

uncertainty, parameter uncertainty and model uncertainty

Non stationarity

We know very well that the performances of rainfall-runoff

models are often non-stationary in short time windows.

An interesting discussion in a general context is given by

Chatfield (1993). We may note that:

a) The prediction model may change in time.

b) The climatic regime is subjected to short- and long-term

variations (irreversible?) which can affect input

uncertainty.

c) Monitoring techniques may change.

d) Land-use may change.

Can we reliably assess the global uncertainty in rainfall-runoff modelling?

A main question

My personal opinion: NO

We can do very little for estimating the uncertainty inducedby non-stationarity.

Typically, any techniques for uncertainty assessment inrainfall-runoff modelling is calibrated by looking at themodel errors in the simulation of historical data. If thestatistical properties of the rainfall-runoff model error

change in time, past errors may not be representative of

future errors.In this case we cannot reliably estimate model uncertainty,unless we make some assumptions whose reliability cannot

be verified.

Wallis (1974): it is a common experience for models tohave worse error variances than they should when used in

forecasting outside the period of fit.See also Gardner (1988, empirical study)

Why??

What can we do then?We can try to estimate model uncertainty under some

assumptions (for instance by assuming stationarity, or by

assuming that we CAN quantify the effects of non-

stationarity)

We can estimate the effects induced by a single source of

uncertainty (for instance parameter uncertainty, or input

uncertainty)

It necessary that we clearly

understand which kind of uncertaintywe are estimating, and the underlyingassumptions.

The output of hydrological models is often treated as

deterministic.

In reality it is never exact; hydrological models provide us

only with a best estimate.

Therefore the hydrological model output should be treated

as stochastic.

A complete ed accurate definition of a stochastic variable

requires the estimation of its probability distribution (or at

least of some statistical properties); not only its expected

value.

Estimating the probability distribution of the hydrological

model output is equivalent to specifying the simulation

uncertainty.

Why should be care about uncertainty?

A possible classification of methods

Uncertainty assessment inhydrological modelling

(a) Structure the rainfall-runoff model as a probability model

(b) Use simulation and re-sampling methods(Monte Carlo simulation)

(c) Analyse the statistical properties of the series of therainfall-runoff model errors

(d) Sensitivity of objective functions to model parameters(parameter uncertainty)

(e) Mixed solutions

A preliminary choice

Uncertainty assessment inhydrological modelling

Freezee and Harlan blueprint (1969)Classical approach in an optimality context

Bevens alternative blueprint (Beven, 2001; 2002).Based on equifinality; we do not have to select anoptimal model and optimal parameter set. The modeldoes not need to be calibrated. The choice amongcompeting models and parameters is automaticallydone, in an equifinality context, accordingly to alikelihood measure. Uncertainty is quantifiedautomatically as well.

Different techniques for different applications

Uncertainty assessment techniquesClassical blueprint

Forecasting (global uncertainty):Bayesian Forecasting System (Krzysztofowicz, 2002

Generation of synthetic data (global uncertainty):GLUE

Meta-Gaussian approach (Montanari and Brath, WRR 2004)

Parameter uncertainty:GLUESCEM-UA (Vrugt et al., 2003)PEST (Brockwell and Davis, 1991)BaRE (Thiemann et al., 2002)NLFIT (Kuczera, 1994)

What kind of approach?

Generation of synthetic data

GLUE (can be applied to ungauged basins)

Meta-Gaussian approach (needs historical data)

Comparison between GLUE

(alternative blueprint) and meta-

Gaussian approach (classical

blueprint).

From Montanari and Brath (2004)0

40

80

120

160

200

0 10 20 30 40 50

Hours (from4 p.m. of 06/23/1995)

Riverdischarge(m3/s)

ObservedSimulatedDist ributed95%CI DistributedSimulatedADM95%CI ADM

Glue Parameter uncertainty is accounted for by performing many modelsimulations with different parameter sets

Input uncertainty is accounted for by performing model simulationwith different input data

Model uncertainty is accounted for by considering many candidate

models (alternative blueprint)

Meta-Gaussian approach Global uncertainty is estimated in an aggregated form, byinferring the probability distribution of the model errorconditioned to the value of the simulated river flow. It is notpossible to separate the contribution of each individual source ofuncertainty

Secchia River Basin (Italy) Generation of 100 years of rainfall in 5 raingauges (Neyman-Scott) Generation of 100 years of hourly river flow data using a spatially-distributed, physically-based rainfall-runoff model. These aretreated as true data

Scatterplotofobserved versus

simulateddischarges

50

100

150

200

250

300

350

400

450

500

550

600

50 100 150200250300350400450500550600

Simulated flows (m3/s)

Observedflows(m3/s)

Efficiencyof theNashmodel: 0.66

In thiscase onlyparameter

uncertaintyandmodel uncertainty

are present.Stationarity!

100.000 data used for calibrating the Nash model (with SCE-UAmethod), and the uncertainty assessment techniques

40.000 data used for validating the uncertainty assessmenttechniques

True data are fitted with a Nash model: a cascade of 4 linearreservoirs.

Uncertainty assessment: meta-Gaussian approachGlobal uncertainty is estimated

Scatterplotof observed versussimulated discharges

Validation carried out by using 40.000 hourly data. Only the riverflows greater than 50 m3/s are used

Black points: data which fallinside the 95% confidence

bands

Red points: data whichfalloutside the 95% confidence

bands

If the uncertaintyassessmenttechnique was reliable, red

pointsshould beabout5% ofthe total points

50

100

150

200

250

300

350

400

450

500

550

600

50 100 150 200 250 300 350 400 450 500 550 600

Simulated flows (m3/s)

Observedflows(m3/s)

Red pointsare 5.1% of thetotal

This very good resultis notsurprising: allthe underlying

assumtpions of the methodcan betested, butthe

hypothesis of stationarity.Whichin this case is verified

Uncertainty assessment: Glue approach

Validation carried out by using 40.000 hourly data. Only the riverflows greater than 50 m3/s are used

By lowering the number ofsimulation (uniform sampling) we

widen the confidence bands

Generation of 3.000 parametersets, in the range 50% of theoptimal value.

Only the data sets that leadto asimulationefficiencygreaterthan0.60 are retained(1000 behaviouralsets)

Red pointsare 71 % of the total

50

100

150

200

250

300

350

400

450

500

550

600

50 100 150 200 250 300 350 400 450 500 550 600

Simulated flows (m3/s)

Observ

edflows(m3/s)

Uncertainty assessment: Glue approach

Validation carried out by using 40.000 hourly data. Only the riverflows greater than 50 m3/s are used

This resultshouldnot be surprisingaswell:

Onlyparameteruncertainty isaccounted for by GLUE applied inthisway. Since the Nashmodel isrobust, parameteruncertainty is

not relevant.

Generation of 10.000 parametersets, in the range 50% of theoptimal value.

Only the data sets that lead toasimulationefficiencygreaterthan0.64 are retained(1000 behaviouralsets)

Red pointsare 85% of the total

50

100

150

200

250

300

350

400

450

500

550

600

50 100 150 200 250 300 350 400 450 500 550 600

Simulated flows (m3/s)

Observedflows(m3/s)

REMARK: by performingmodelsimulationswith different

parametersets onlyparameteruncertaintyis accountedfor. Thiscan be onlya smallfractionof the

global uncertainty.

Uncertainty assessment: Glue approachIncluding different Nash models(different number of reservoirs)

Generation of 3.000 differentcombinationsof models structuresand parametersets.

Nashmodelswithnumber of

reservoirs ranging from1 to 5Parameterswere changed in therange 50% of the optimal value.

Only the data sets that leadto asimulationefficiencygreaterthan0.60 are retained(1000 behaviouralsets)

REMARK: model uncertainty isaccounted for onlyby including

manydifferentmodels.Howmany?

Red pointsare 33 % of the total

50

100

150

200

250

300

350

400

450

500

550

600

50 100 150 200 250 300 350 400 450 500 550 600

Simulated flows (m3/s)

Observedflows(m3/s)

A possible suggestion for ungauged sites

Remark: the choice among different approaches should not be only a question ofpersonal preference; different applications may require different techniques.

Calibration of spatially-distributed rainfall-runoff model on the gauged site.

Simulation in the site of interest.

Uncertainty in the gauged site estimatedwith GLUE and statistical approaches.

Possible calibration of GLUE.

May be it works ????

Uncertainty in the ungauged site estimatedwith GLUE.

Final remarks

It is necessary that the hypothesis underlying uncertainty estimation

are clearly stated.

I do not think it exists a best technique.As modellers, we should be aware of all possible methods, their

peculiarities, suggested ambits of application, underlyinghypotheses.

It is not always possible to estimate global uncertainty.

If this is the case, we just have to be aware of which kind of

uncertainty we are estimating and the possible

uncertiainties in the uncertainty estimation.