A statistical approach to PVA validation: an example using diffusion approximation models
Eli HolmesREUT/NOAA/[email protected]
Updated with a few comments after talk
Overviewυ Different validation approachesυ Example: a cross-validation study of diffusion
approximation PVAsυ Presenting uncertainty.
FOR MORE INFO...
faculty.washington.edu/eeholmes
Methods for testing PVAsfrom McCarthy et al. 2002. “Testing the Accuracy of PVA” Cons. Bio.
Subjective, ignores variability, single trajectories unlikely to be similar to mean
Compare mean or median predictions with observations
Only assesses average number or frequency of occurrences within a group, ignores variability
Compare observed vs predicted frequency of events
Assesses both the mean and variability, generally requires transformation of data to a standard variate, lots of data
Compare probability distributions of population size or parameters
Diffusion approximation PVA
eggs 1 2 3 4
Nt=N0 exp(µt+εt) where ε ~ Normal(0,σ)
reproduction
Survival to next age class
Let reproduction and survival vary yearly
Basic Idea of DA PVA
0
20
40
60
80
100
120
1952
1955
1958
1961
1964
1967
1970
1973
1976
1979
1982
1985
1988
1991
1994
1997
redd
s p
er m
ile
0.5 1 1.5 2 2.5 3 3.5 4
ln(N) at 1985
prob
abili
ty d
ensi
ty
Parameters of a DA model
Nt=N0*exp(µt+εt) where ε ~ N(0,σ)
Parameter that governs the median rate of decline.
“Process error”: parameter that describes the long-term variability of the process.
µ̂ σ̂
Cross-validationυ 147 chinook and 42 steelhead 30-70 year time series from ESUs in
WA, OR, and CA
01020304050607080
1952
1955
1958
1961
1964
1967
1970
1973
1976
1979
1982
1985
1988
1991
1994
1997
red
ds
per
mil
e
parameterization evaluation
Does the DA model predict the frequency of actual declines?
0
0.2
0.4
0.6
0.8
1
10 8 6 4 2 1 1/2 1/4 1/6 1/8 1/10
Observed indata
threshold (rel. to size at the start of the eval. period)
fraction of stocks that
dip below a threshold
Do the projected population sizes follow the expected theoretical distribution?
0
20
40
60
80
100
120
1952
1955
1958
1961
1964
1967
1970
1973
1976
1979
1982
1985
1988
1991
1994
1997
redd
s p
er m
ile
0.5 1 1.5 2 2.5 3 3.5 4
ln(N) at 1985
prob
abili
ty d
ensi
ty
Do the projected population sizes follow the expected theoretical distribution?
1 5 9 13 171 5 9 13 17
1 5 9 13 17 1 5 9 13 17
µ̂
µ̂ of dist.
Do my estimates of σ follow the expected theoretical distribution?
0
5
1
5
2
σ̂
σ̂ of dist.1 5 9 13 17 1 5 9 13 17
1 5 9 13 17 1 5 9 13 17
Transforming data to a common currency
For other examples see McCarthy et al. 2002. Conservation Biology
Problem: don’t view the same population process over and over Actual data: many different processes with different underlying parameters (growth rates and variability)Solution: transform data to a standardized metric that has the same statistical distribution for all processes
slpdf is )ˆ var(offreedom of degrees
µ
Standardized ln(Nt+15/Nt)= ψdistribution
slpdf
epslp
eslppslp
ep t
LnLndfdfdf
222
1~11
2ˆˆ
)(γσσ
ψψ
⎟⎟⎠
⎞⎜⎜⎝
⎛
−+
−+
−
like~)ˆvar(likeNormal~ˆ
2 −
−
χµ
µ
-0
no trend
)ˆvar(of estimatein bias no
µ
Results for population distribution
Predicted t distribution
Histogram of actual t statistics
Trend in the rate of decline?υ Fluctuating or declining stocks
υ No significant trend
υ Rapidly increasing stocksυ Significant negative trendυ Estimate of µ lower for bigger population size
Standardized σ distribution
slpp dfσ is ˆ offreedom of degrees
2
x 1
no trend
( ) ),(~ˆˆ 22slpslppe dfdfFσσ
like~ˆ 22 −χσ
Results for σ Predicted F distribution
Histogram of actual F statistics
Trend in σ?
υ Estimate of σ was higher when counts were really smallυ Demographic stochasticity?υ Sampling effect?
Estimate of σ sensitive to percent of sampling error in the observation
Percent error tends to be larger when counts are smalle.g. Dunham and Rieman. 2001. Sources and magnitudes of
sampling error in redd counts for Bull Trout. North American Journal of Fisheries Management 21:343–352
Can the model predict actual declines?
υ Simple model makes many simplifying assumptionsυ density-independenceυ no environmental correlationυ no trendsυ diffusion approximation of age-structured
population
Actual frequency of declines
Estimate using Dennis-Holmes
Estimate using Dennis
Maximum yield relationship for modeling
Belief
Yie
ld
True believerNon-believer
25 yr 100 yr
support for different probs of 90% decline
support for different λ’s
25 yr 100 yr
pdf of prob of 90% decline
pdf of λsupport for different λ’s
support for different probs of 90% decline
25 yr 100 yr
support for different probs of 90% decline
support for different λ’s
FOR MORE INFO...
A variety of matlab and Splus code for DA PVAs is at faculty.washington.edu/eeholmes
Holmes, E. E. 2001. Estimating risks in declining populations with poor data. Proceedings of the National Academy of Science 98: 5072-5077.
Holmes and Fagan. 2002. Validating population viability analysis for corrupted data sets. Ecology in press.
Holmes, E. E. Beyond theory to application and evaluation: diffusion approximations for population viability analysis.
Reprints or pre-prints available at faculty.washington.edu/eeholmes