traffic counts and travel model performance a presentation to the tpb technical committee april 1,...
DESCRIPTION
Presentation to the TPB Technical Committee 4/1/05 3 Model Performance is Relative Performance is an Estimated-to-Observed Ratio Regional, Subregional, Jurisdiction, Screenline, Link, … Performance Depends on Accuracy of Both Estimated and Observed Figures A Balanced Understanding of Both Estimated and Observed Figures is Critical to ValidationTRANSCRIPT
Traffic Counts and
Travel Model Performance
A Presentationto the
TPB Technical Committee April 1, 2005
Item 9
Presentation to the TPB Technical Committee 4/1/05
2
Traffic Counts and Model Evaluation
The ‘Yardstick’ of Model Performance Regional/Jurisdictional VMT Screenline/Cutline Crossings
Air Quality Requirements VMT Tracking Emission Budgets
Increasing Specificity Sought AADT, AAWDT, Seasonal, Time of day, Classification, etc.
Presentation to the TPB Technical Committee 4/1/05
3
Model Performance is Relative Performance is an Estimated-to-Observed Ratio
Regional, Subregional, Jurisdiction, Screenline, Link, … Performance Depends on Accuracy of Both Estimated
and Observed Figures A Balanced Understanding of Both Estimated and
Observed Figures is Critical to Validation
Presentation to the TPB Technical Committee 4/1/05
4
Traffic Counts in the Region
Highway Performance Monitoring System (HPMS) is Primary Source of Count Data DDOT, MDSHA, VDOT, Local Jurisdictions
AADT (MD) / AAWDT (DC and VA) Coding Counts in the Highway Network
Historically a Manual Process Using Count Books/Maps More Recently Electronic Transfer Using Georeferenced Count
Data Bases and COG’s Transportation Data Clearinghouse Data Used ‘as is’; Maximum Link Coverage Sought
Presentation to the TPB Technical Committee 4/1/05
5
Samples Designed for Statewide Traffic Estimates Permanent Counting Stations, Continuous Operation Program Count Locations; Short Count Duration, 3-year
Collection Cycle Adjustments Applied to Counts
Current-year Program Counts Annualized Non-Current Year Program Counts Adjusted to Current Year Partially Operational Permanent Counts Annualized “Manual Adjustments” made to compensate for Equipment
Failure, Construction, Safety Issues, etc. Adjustments based on Perm.Counts Statewide
HPMS Overview
Presentation to the TPB Technical Committee 4/1/05
6
HPMS is not a count of traffic, per se-- HPMS is an annualized traffic volume estimate based on a statewide sample of a limited number of locations.
A Case Can be Made: Model Performance expressed as an ‘Estimated-to-Observed’ Ratio Should be Considered as an ‘Estimated-to-Estimated’ Ratio
When HPMS data is used for a specific metropolitan area, data noise is a potential issue.
Observations on the HPMS
Presentation to the TPB Technical Committee 4/1/05
7
Issues for Transportation Modeling
Highway Network is an Approximation of the Physical Roadway System
Resolving AADT Figures to AAWDT is Approximate
Directionality of Counts is Suppressed With HPMS Data
Ground Count Highway Network Coding Practices Can Also Introduce Noise/Bias Into the Performance Statistics
Presentation to the TPB Technical Committee 4/1/05
8
Daily VMT Over Timefor the Washington Region By State
0
10,000
20,000
30,000
40,000
50,000
60,000
70,000
1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003
Year
VM
T (i
n th
ousa
nds)
MD
VA
DC
Presentation to the TPB Technical Committee 4/1/05
9
Time Series VMT Changefor the Washington Region By State
-10.0%
-5.0%
0.0%
5.0%
10.0%
15.0%
20.0%
'93-'94 '94-'95 '95-'96 '96-'97 '97-'98 '98-'99 '99-'00 '00-'01 '01-'02 '02-'03
Year
Ann
ual G
row
th R
ate
DCMDVAAll
Presentation to the TPB Technical Committee 4/1/05
10
Yr 2000 Validation Freeways RMSE Summary
Links Ave Obs Ave Est Diff. Pct PctVolume Range Count Volume Volume (Obs-Est) Diff. RMSE RMSE
1.00-9.99K 23 8.04 18.65 -10.61 -131.89 14.00 174.0510.00-19.99K 94 14.65 27.80 -13.15 -89.76 17.38 118.6620.00-29.99K 63 24.81 40.00 -15.19 -61.23 18.26 73.6230.00-39.99K 159 35.33 41.85 -6.52 -18.44 12.57 35.5740.00-49.99K 136 43.79 52.43 -8.64 -19.73 17.24 39.3750.00-59.99K 90 54.20 65.33 -11.13 -20.54 15.57 28.7460.00-69.00K 113 64.68 65.88 -1.19 -1.85 14.96 23.1370.00-79.00K 84 73.62 76.55 -2.93 -3.98 17.10 23.2380.00-89.99K 78 84.62 81.81 2.81 3.32 15.41 18.2190.00-99.99K 90 94.89 90.64 4.24 4.47 15.23 16.05
100.00-109.99K 67 104.63 101.93 2.70 2.58 13.30 12.71110.00-119.99K 35 115.66 110.71 4.94 4.27 18.58 16.06120.00-129.99K 20 125.70 111.05 14.65 11.65 21.60 17.19130.00-139.99K 20 139.90 104.05 35.85 25.63 39.51 28.24Subtotal: 1,072 60.24 64.03 -3.79 -6.30 16.59 27.54
Presentation to the TPB Technical Committee 4/1/05
11
Updated Traffic Volume Estimates for V2.1.D Model Performance Tests
Review Traffic Volume Data/ Estimates for all jurisdictions in the metropolitan Washington region (MSA).
Link AAWDT Traffic Volume Estimate only to the network link where the Program or Permanent Counting Station was located (i.e. no carrying forward or averaging volume estimates for adjacent links).
Identify all Program and Permanent Counting station locations where actual traffic count data was collected in Year 2000.
Identify all Permanent Counting Stations that were operational in Year 2000.
Presentation to the TPB Technical Committee 4/1/05
12
All Program and Permanent Counting Locationswith Actual or Factored Daily Traffic Volume Estimates for Year 2000
Presentation to the TPB Technical Committee 4/1/05
13
All Program and Permanent Counting Locationswith Actual Traffic Count Data Collected in Year 2000
Presentation to the TPB Technical Committee 4/1/05
14
All Operational Permanent Counting Station Locations in Year 2000
Presentation to the TPB Technical Committee 4/1/05
15
Performance Test Results
Sample Description Obs. RMSE %RMSE R2
V2.1D#50 Model Validation 11,004 8.01 47.21 0.84Permanent Station and Program Counts - 2,953 7.12 44.89 0.89Permanent and Actual Program Counts 1,194 5.8 40.33 0.92Permanent Counting Stations 68 6.65 18.04 0.96
Presentation to the TPB Technical Committee 4/1/05
16
Validation E/O Scatterplots11,004 Observations
Plot of Estimated and Observed Year 2000 Link Volumes V2.1D#50 Validation Network
(Entire Modeled Area / 11,004 Obs. )
Linear Model:y = 0.93x + 2.34
R2 = 0.84
0
20
40
60
80
100
120
140
160
0 20 40 60 80 100 120 140 160
Observed Volume (in 000s)
Estim
ated
Vol
ume
(in 0
00s)
Est vs Obs Line of perfect agreement Linear (Est vs Obs)
Presentation to the TPB Technical Committee 4/1/05
17
Permanent /Actual & Factored Program Counts 2,953 Observations
Scatterplot of Estimated and Observed Year 2000 Link Volumes(All Program/Permanent Counts - 2,953 Obs
MSA Only/1.05 Ftr for ADT-to-AAWDT Conversion
Linear Model:y = 0.97x + 1.53
R2 = 0.89
0
20
40
60
80
100
120
140
160
0 20 40 60 80 100 120 140 160
Observed Volume (000s)
Estim
ated
Vol
ume
(000
s)
Est vs Obs Line of perfect agreement Linear (Est vs Obs)
Presentation to the TPB Technical Committee 4/1/05
18
Permanent /Actual Program Counts 1,194 Observations
Scatterplot of Estimated and Observed Year 2000 Link Volumes(Permanent & Program Counts Collected in 2000 - 1,194 Obs)
MSA Only/1.05 Ftr for ADT-to-AAWDT Conversion
Linear Model:y = 1.00x + 1.24
R2 = 0.92
0
20
40
60
80
100
120
140
160
0 20 40 60 80 100 120 140 160
Observed Volume (000s)
Estim
ated
Vol
ume
(000
s)
Est vs Obs Line of perfect agreement Linear (Est vs Obs)
Presentation to the TPB Technical Committee 4/1/05
19
Permanent Count Stations68 Observations
Scatterplot of Estimated and Observed Year 2000 Link Volumes(All Permanent Count Stations - 68 Obs.)
MSA Only/1.05 Ftr for ADT-to-AAWDT Conversion
Linear Model: y = 0.97x + 2.18
R2 = 0.96
0
20
40
60
80
100
120
140
160
0 20 40 60 80 100 120 140 160
Observed Volume (000s)
Estim
ated
Vol
ume
(000
s)
Est vs Obs Line of perfect agreement Linear (Est vs Obs)
Presentation to the TPB Technical Committee 4/1/05
20
Conclusions Model Performance Improves with Higher Quality Counts
Metropolitan areas using statewide counting program data should expect count accuracy limitations-- especially multi-state areas. Consider metropolitan samples?
Expect scatterplot outliers, seek explanations
Performance problems does not equate to model problems.