ACTIVITY-BASED TRAVEL MODEL CALIBRATION AND VALIDATION
FOR BASE YEAR 2012
November 2016
San Diego Association of Governments (SANDAG)
ABSTRACT
TITLE: Activity-Based Travel Model Calibration and Validation for Base Year 2012
AUTHOR: San Diego Association of Governments
DATE: November 2016
SOURCE OF COPIES: San Diego Association of Governments
401 B Street, Suite 800
San Diego, CA 92101
(619) 699-1900
NUMBER OF PAGES:
ABSTRACT: This document describes calibration and validation efforts of the San
Diego Association of Governments (SANDAG) Activity-Based Model (ABM)
for base year 2012. Prior to this effort, there were three rounds of
calibration and validation efforts for base years 2008, 2010, and 2012.
Since then, there are many changes to the ABM that have substantial
impact on model results, including input, model parameter, software and
network changes. The calibration and validation described in this
document is necessary to incorporate these changes in the SANDAG ABM.
i
TABLE OF CONTENTS
CHAPTER 1: INTRODUCTION ............................................................................................................................... 1
CHAPTER 2: MODEL AND INPUT CHANGES ........................................................................................................ 3
2.1: Population Input Change: from Population Synthesizer II to III ............................................................ 3
2.2: Land Use Input Changes ........................................................................................................................... 7
2.3: Traffic Count Update ................................................................................................................................ 8
2.3.1: PeMS Counts ...................................................................................................................................... 9
2.3.2: Caltrans District 11 State Highway Traffic Census Counts .............................................................. 9
2.3.3: Local Jurisdiction Counts ................................................................................................................... 9
2.3.4: Other Counts...................................................................................................................................... 9
2.3.5: Summary of Counts ......................................................................................................................... 10
2.4: Calibration Target Changes ................................................................................................................... 12
2.4.1: Auto Ownership Model Targets ..................................................................................................... 12
2.4.2: Work Trip Mode Choice Model Targets ......................................................................................... 14
2.5: Software Changes ................................................................................................................................... 15
2.6: Network Changes ................................................................................................................................... 16
CHAPTER 3: MODEL CALIBRATION ................................................................................................................... 17
3.1: Auto Ownership Model .......................................................................................................................... 17
3.2: Coordinated Daily Activity Pattern Model ............................................................................................ 19
3.3: Mode Choice Models .............................................................................................................................. 22
3.3.1: Work Trip Mode Choice .................................................................................................................. 23
3.3.2: Managed Lane (ML) Volume Adjustments .................................................................................... 24
3.3.3: Transit Mode Adjustments .............................................................................................................. 26
3.4: Crossborder Model ................................................................................................................................. 27
CHAPTER 4: MODEL VALIDATION ..................................................................................................................... 28
4.1: Roadway Validation ............................................................................................................................... 28
4.1.1: Roadway Validation by MSA .......................................................................................................... 28
4.1.2: Roadway Validation by Road Type................................................................................................. 30
4.1.3: Roadway Validation by Volume ..................................................................................................... 34
4.1.4: Roadway Validation by Count Source ............................................................................................ 36
4.1.5: Roadway Validation by Key Count Locations ................................................................................ 37
4.1.6: Roadway Validation by Corridor .................................................................................................... 38
4.1.7: Roadway Validation by Corridor Direction .................................................................................... 64
ii
4.2: Transit Validation ................................................................................................................................... 69
4.3: Regional VMT Validation ....................................................................................................................... 71
4.4: Comparisons of SANDAG Validation Results with FHWA Guidelines ................................................. 72
CHAPTER 5: MILITARY TRAVEL CALIBRATION AND VALIDATION .................................................................. 75
5.1: Military Base Traffic Count8 ................................................................................................................... 75
5.2: Roadway Network Adjustments ............................................................................................................ 76
5.2.1: Gate Location Validation ................................................................................................................ 76
5.2.2: Gate Connections to the Civilian Roadway Network .................................................................... 78
5.2.3: Internal Base Networks ................................................................................................................... 79
5.2.4: Zone Connector Configuration ....................................................................................................... 80
5.2.5: Speed Adjustments on Base Driveways or Networks .................................................................... 82
5.2.6: Summary Matrix for Roadway Network Adjustments .................................................................. 83
5.3: Military Travel Calibration and Validation Results ............................................................................... 86
5.3.1: Validation by Military Base ............................................................................................................. 86
5.3.2: Validation by Gate........................................................................................................................... 88
CHAPTER 6: CONCLUSIONS AND FUTURE WORK ............................................................................................ 92
iii
LIST OF TABLES
Table 2.1 ABM Milestone Scenarios ................................................................................................................. 3
Table 2.2 2012 Population Distributions by Person Type ............................................................................... 4
Table 2.3 Household Distributions by Number of Workers ............................................................................ 5
Table 2.4 Households by Size ........................................................................................................................... 6
Table 2.5 Summary of Land Use Inputs to ABM .............................................................................................. 8
Table 2.6 Roadway Traffic Counts by Data Source ......................................................................................... 8
Table 2.7 Roadway Traffic Counts by Source ................................................................................................ 10
Table 2.8 Count Distributions by Road Type ................................................................................................. 10
Table 2.9 Household Size by Vehicles Available 2010-2014 ACS 5-Year Estimates (San Diego) ................ 13
Table 2.10 Auto Ownership Model Calibration Targets ................................................................................. 14
Table 2.11 Commuting (Journey to Work) Mode Splits from 2010-2014 ACS
5-Year Release (San Diego) ........................................................................................................... 14
Table 2.12 Work Purpose Trip Mode Choice Targets ...................................................................................... 15
Table 3.1 Auto Ownership Model ASC Adjustments .................................................................................... 17
Table 3.2 Auto Ownership Model Calibration Results .................................................................................. 18
Table 3.3 CDAP Model ASC Adjustments ...................................................................................................... 19
Table 3.4 Coordinated Daily Activity Pattern Model Calibration Results .................................................... 20
Table 3.5 Adjusted Work Trip Mode Choice ASCs ........................................................................................ 23
Table 3.6 Work Trip Mode Choice Calibration Results ................................................................................. 23
Table 3.7 Adjusted ASCs to Reduce ML Volumes .......................................................................................... 25
Table 3.8 I-5 and I-15 ML Validation Results ................................................................................................. 26
Table 3.9 Transit Ridership Target Adjustments ........................................................................................... 26
Table 3.10 Cross Border Trips with Destinations in Military Base .................................................................. 27
Table 4.1 Roadway Validation Results by MSA Base Year 2012 ................................................................... 29
Table 4.2 Roadway Validation Results by MSA-Base Year 2008 .................................................................. 29
Table 4.3 Correspondence between IFC and Road Class .............................................................................. 31
Table 4.4 Roadway Validation Results by Road Type ................................................................................... 33
Table 4.5 Roadway Validation Results by Volume ........................................................................................ 35
Table 4.6 Validation Results by Key Count Location .................................................................................... 37
Table 4.7 Validation Results with Key Count Locations 12 and 13 Combined ............................................ 38
Table 4.8 Roadway Validation Results by Corridor Direction ...................................................................... 65
Table 4.9 Roadway Validation Results: Northbound Corridors .................................................................... 66
Table 4.10 Road Validation Results: Southbound Corridors .......................................................................... 67
Table 4.11 Roadway Validation Results: Westbound Corridors ..................................................................... 68
Table 4.12 Roadway Validation Results: Eastbound Corridors ....................................................................... 69
Table 4.13 Transit Validation Results by Line Haul Mode .............................................................................. 70
Table 4.14 Volume-Over-Count Ratios and Percent Error (Florida Sample) .................................................. 72
Table 4.15 Volume-Over-Count Ratios and Percent Error (SANDAG 2012 Base Year Model) ...................... 73
Table 5.1 Military Bases Participated in Traffic Count .................................................................................. 75
Table 5.2 Validation Results by Military Installation .................................................................................... 86
Table 5.3 Summary of Validation Results-by Installation ............................................................................. 86
Table 5.4 Validation Results by Gate ............................................................................................................. 89
iv
LIST OF FIGURES
Figure 2.1 PopSyn II vs. PopSyn III-Regional Households and Population .................................................... 4
Figure 2.2 Population Distributions by Person Type ..................................................................................... 5
Figure 2.3 Household Distributions by Number of Workers ......................................................................... 6
Figure 2.4 Household Distributions by Household Size ................................................................................ 7
Figure 2.5 Count Coverage by Source .......................................................................................................... 11
Figure 2.6 Count Coverage by Road Type .................................................................................................... 12
Figure 3.1 Auto Ownership Model Calibration Results ............................................................................... 18
Figure 3.2 CDAP Model Calibration Results ................................................................................................. 21
Figure 3.3 SANDAG Trip Mode Choice Structure ......................................................................................... 22
Figure 3.4 Work Trip Mode Choice Calibration Results .............................................................................. 24
Figure 3.5 Cross Border Trips with Destinations in Military Bases .............................................................. 27
Figure 4.1 Validation Results by MSA-Latest 2012 Version vs. 2008 Base Year ......................................... 30
Figure 4.2 Roadway Validation Results-All Road Classes ............................................................................ 31
Figure 4.3 Roadway Validation Results by Road Class ................................................................................ 32
Figure 4.4 Roadway Validation Results by Count Data Source ................................................................... 36
Figure 4.5 Validation Results: I-5 Northbound ............................................................................................ 39
Figure 4.6 Validation Results: I-5 Southbound ............................................................................................ 40
Figure 4.7 Validation Results: I-15 Northbound .......................................................................................... 41
Figure 4.8 Validation Results: I-15 Southbound .......................................................................................... 42
Figure 4.9 Validation Results: I-805 Northbound ........................................................................................ 43
Figure 4.10 Validation Results: I-805 Southbound ........................................................................................ 44
Figure 4.11 Validation Results: SR-125 Northbound ..................................................................................... 45
Figure 4.12 Validation Results: SR-125 Southbound ..................................................................................... 46
Figure 4.13 Validation Results: SR-163 Northbound ..................................................................................... 47
Figure 4.14 Validation Results: SR-163 Southbound ..................................................................................... 48
Figure 4.15 Validation Results: SR-67 Northbound ....................................................................................... 49
Figure 4.16 Validation Results: SR-67 Southbound ....................................................................................... 50
Figure 4.17 Validation Results: I-8 Westbound .............................................................................................. 51
Figure 4.18 Validation Results: I-8 Eastbound ............................................................................................... 52
Figure 4.19 Validation Results: SR-52 Westbound ......................................................................................... 53
Figure 4.20 Validation Results: SR-52 Eastbound .......................................................................................... 54
Figure 4.21 Validation Results: SR-54 Westbound ......................................................................................... 55
Figure 4.22 Validation Results: SR-54 Eastbound .......................................................................................... 56
Figure 4.23 Validation Results: SR-56 Westbound ......................................................................................... 57
Figure 4.24 Validation Results: SR-56 Eastbound .......................................................................................... 58
Figure 4.25 Validation Results: SR-78 Westbound ......................................................................................... 59
Figure 4.26 Validation Results: SR-78 Eastbound .......................................................................................... 60
Figure 4.27 Validation Results: SR-94 Westbound ......................................................................................... 61
Figure 4.28 Validation Results: SR-94 Eastbound .......................................................................................... 62
Figure 4.29 Validation Results: SR-905 Westbound ....................................................................................... 63
Figure 4.30 Validation Results: SR-905 Eastbound ........................................................................................ 64
Figure 4.31 Transit Validation Results: Regional Transit Ridership .............................................................. 70
Figure 4.32 Transit Validation Results by Line Haul Mode ........................................................................... 71
Figure 4.33 Regional VMT Validation Results ................................................................................................ 72
Figure 4.34 %RMSE by Volume Examples in FHWA Validation Guideline6 ................................................ 73
Figure 4.35 %RMSE by Volume-SANDAG 2012 Base Year Model ................................................................ 74
v
Figure 5.1 Count & Driveway Configuration at Gate 37 ............................................................................. 77
Figure 5.2 Ariel Image of 32nd Street Naval Base ....................................................................................... 78
Figure 5.3 USMC Camp Pendleton Internal Base Network Used in Previous Modeling Efforts ............... 79
Figure 5.4 USMC Camp Pendleton After Roadway & Zone Centroid Revisions ......................................... 80
Figure 5.5 USMC Recruit Depot Zone Centroid Coding Used in Previous Modeling Efforts .................... 81
Figure 5.6 USMC Recruit Depot Zone Centroid Coding After Editing ....................................................... 82
Figure 5.7 Speed Adjustment at NAS North Island South Gate .................................................................. 83
Figure 5.8 Roadway Network Adjustments Matrix ..................................................................................... 84
Figure 5.9 % Validation Results-By Installation ........................................................................................... 87
Figure 5.10 Validation Results by Gate .......................................................................................................... 90
1
CHAPTER 1: INTRODUCTION
This document describes calibration and validation effort of the San Diego Association of
Governments (SANDAG) Activity-Based Model (ABM) for base year 2012. Prior to this effort, there
were three rounds of calibration and validation efforts for base years 2008,1 2010,2 and 2012.3 Since
then, there are many changes to the ABM that have substantial impact on model results, including
input, model parameter, software and network changes. The calibration and validation described in
this document is necessary to incorporate these changes in the SANDAG ABM.
The decision to recalibrate the model was based on the following observations: First, as demographic
and socioeconomic conditions change in the San Diego region, land use and population inputs to
ABM also changed. Secondly, recent releases of American Community Survey (ACS)4 make it possible
to calibrate some ABM components to updated 2012 targets. Previous calibration efforts relied on
2006 San Diego household travel behavior (HHTS) and older Census data. Thirdly, gate counts
collected at ten military bases allows calibrating military travel modeling to match observed gate
counts. Lastly, staff added, validated, and cleaned 2012 traffic counts; resulted in improved counts
for calibration and validation.
Based on the analysis of what model components were affected significantly by the changes, the
calibration and validation effort was determined to focus on the following sub-models:
Auto ownership model
Coordinated Daily Activity Pattern (CDAP) model
Tour and trip mode choice models
Crossborder model
Military travel modeling
The following general conclusions can be made (more information can be found throughout the
document):
Impact of synthetic population, land use, network, and software changes on model results were
not negligible. The transition of synthetic population from version 2 (PopSyn II) to version 3
(PopSyn III) had the most significant impact on model results.
Network coding and quality of traffic counts are important to model calibration and validation,
especially for localized areas.
Results of the calibrated model match observed counts and targets better than those of the
uncalibrated model, therefore this effort improved model precision for regional planning
applications.
Although ACS releases provide some updated calibration targets, most calibration targets are
only available in the 2006 HHTS. A key limitation of this calibration and validation effort is the
lack of an up-to-date household travel behavior survey. Fortunately, the San Diego 2016 HHTS
which will facilitate complete ABM update is currently underway.
The objective of this effort is to calibrate and validate the ABM with the best available data;
create an intermediate version that serves as the quantitative analysis tool for regional planning
before the ABM is updated with the 2016 HHTS.
2
This documentation proceeds as follows:
Chapter 2 summarizes input, software, traffic count, and calibration target changes since last
calibration and validation.
Chapter 3 describes model calibration.
Chapter 4 describes roadway and transit validation that compares model estimated against
observed conditions.
Chapter 5 describes military travel modeling adjustments and the effort of matching estimated trips
to/from bases with observed gate counts.
Chapter 6 draws conclusions and identifies future work.
3
CHAPTER 2: MODEL AND INPUT CHANGES
This chapter describes population, land use, traffic count, calibration target, software, and network
changes. Table 2.1 lists three milestone scenarios; (1) scenario 123 is a calibrated 2012 model with
software version 13.2.3 and PopSyn II; (2) scenario 227 is an uncalibrated 2012 model with software
version 13.2.5 and PopSyn III; (3) scenario 540 is a calibrated 2012 model with software version 13.3.0
and PopSyn III. Throughout this document, these three scenarios are referred as ’calibrated with
PopSyn II,’ ‘uncalibrated with PopSyn III,’ and ‘calibrated with PopSyn III.’
Table 2.1 ABM Milestone Scenarios
Scenario PopSyn Software Version Year Date Name
123 II 13.2.3 2012 08/15 calibrated with PopSyn II
227 III 13.2.5 2012 11/15 uncalibrated with PopSyn III
540 III 13.3.0 2012 10/16 calibrated with PopSyn III
2.1: Population Input Change: from Population Synthesizer II to III
Population synthesis (PopSyn) is at the top of the ABM hierarchy; the impact of the transition from
PopSyn II to PopSyn III cascaded down to all ABM components. PopSyn III has several innovative
features5 that help improve the quality of the synthesized population. The first one relates to a
general formulation of convergence of the balancing procedure with imperfect (i.e., not fully
consistent) controls. The second one relates to the optimized discretizing of the fractional outcomes
of the balancing procedure to form a list of discrete households. The proposed method employs a
Linear Programming (LP) approach in order to optimize the discretized weights and preserve the best
possible match to the controls. The third one relates to multiple levels of geography where the
controls can be set. The geographic flexibility is essential for two reasons. First, some important
demographic, socio-economic, and land-use development trends affecting population synthesis can
only be translated into more aggregate controls than a TAZ-level control. Secondly, ABMs operate
with an enhanced level of spatial resolution where all location choices are modeled at the level of
Micro-Analysis Zones nested within the TAZs.
In PopSyn II it was difficult to match regional population and household targets simultaneously,
therefore priority was given to matching the household target. PopSyn III matches control targets
(provided by SANDAG land use modelers in April 2015) better than PopSyn II; regional population
and household targets are simultaneously produced to meet targets almost spot-on.
4
Figure 2.1 PopSyn II v s . PopSyn III -Regional Households and Population
The total regional households are almost identical in PopSyn II and III while regional population in
PopSyn III is 4 percent more than that of PopSyn II. Without proper validation and calibration, the
additional 4 percent population could have skewed the calibrated model with PopSyn II and shifted
model results away from the observed conditions in base year 2012.
Household and population characteristics, such as household size, income, number of workers, person
type, gender and age, also have impact on the model results. Table 2.2 and Figure 2.2 present
population distributions by person type summarized from 2012 PopSyn II and PopSyn III.
Table 2.2 2012 Population Distributions by Person Type
Person Type PopSyn II PopSyn III
Full-time Worker 38.0% 34.6%
Part-time Worker 8.3% 7.2%
College Student 8.7% 7.2%
Non-working Adult 13.0% 17.1%
Non-working Senior 9.7% 10.4%
Driving Age Student 3.4% 3.4%
Non-driving Student 11.9% 12.5%
Pre-school 6.9% 7.6%
5
Figure 2.2 Population Distributions by Person Type
Compared with PopSyn II, there are fewer full-time and part-time workers in PopSyn III. The
percentage of full-time workers dropped from 38 percent in PopSyn II to 34.6 percent in PopSyn III, a
significant 3.4 percent decrease. Conversely, there are more non-working adults and non-working
seniors in PopSyn III. Particularly, the percentage of non-working adults increased from 13 percent to
17.1 percent, a significant 4.1 percent increase. Workers are more likely to travel during peak hours
and less likely to participate in joint trips because of their work schedules. Without proper calibration
and validation, the 3.4 percent fewer workers in PopSyn III could have resulted in less congested peak
hour traffic and smaller shares of drive alone modes. To another degree, the 4.1 percent more
non-working adults in PopSyn III could have resulted in more off-peak traffic and larger shares of
shared ride modes.
Table 2.3 and Figure 2.3 present household distributions by number of workers in households.
Table 2.3 Household Distributions by Number of Workers
Households by Number of
Workers PopSyn II PopSyn III
0 22.6% 25.4%
1 39.8% 42.4%
2 29.6% 25.9%
3+ 8.0% 6.3%
0.0% 5.0% 10.0% 15.0% 20.0% 25.0% 30.0% 35.0% 40.0%
Full-time Worker
Part-time Worker
College Student
Non-working Adult
Non-working Senior
Driving Age Student
Non-driving Student
Pre-school
2012 Population by Person Type
PopSyn III PopSyn II
6
Figure 2.3 Household Distributions by Number of Workers
Compared with PopSyn II, there are more households with no worker, or one worker, and fewer
households with two or three-plus workers in PopSyn III. Particularly, no worker household
percentage increased by 2.8 percent from 22.6 percent to 25.4 percent. This is consistent with the
findings in the population by person type analysis; there are fewer workers in PopSyn III.
Table 2.4 and Figure 2.4 present household distributions by household size.
Table 2.4 Households by S ize
Household Size
PopSyn II PopSyn III
1 30.7% 28.8%
2 28.9% 30.5%
3 14.7% 15.3%
4+ 25.7% 25.5%
0.0% 5.0% 10.0% 15.0% 20.0% 25.0% 30.0% 35.0% 40.0% 45.0%
0
1
2
3+
2012 Households by Number of Workers
PopSyn III PopSyn II
7
Figure 2.4 Household Distributions by Household S ize
Compared with PopSyn II, there are fewer one-person households and more two-person and
three-person households in PopSyn III. Particularly, one-person household percentage dropped from
30.7 percent to 28.8 percent, a 1.9 percent decrease likely results in fewer joint trips. Without proper
calibration and validation, the change of household size distributions could have significant impact
on Managed Lanes (ML) traffic, many of which are from joint travelers, and shared ride modes.
2.2: Land Use Input Changes
Land use data such as employment and enrollment are among the key drivers behind trip/tour
attractions, particularly for mandatory work, university and school trips/tours. Land use data are
generated from the SANDAG land use models and then fed into ABM as inputs. Similar to
transportation models, land use models also evolve, and consequentially cause land use input
changes.
Table 2.5 summarizes key land use inputs to scenarios 123 (calibrated model with PopSyn II) and 540
(calibrated model with PopSyn III). Compared with scenario 123, in scenario 540 total employment
increased by 1.3 percent while military employment decreased by 6.3 percent. The changes, although
seemingly small, can’t be ignored as they affect model results, such as work location choice results.
Due to reclassification of land use categories, open space and active beach increased significantly as
shown in Table 2.5; as a result, could have resulted in more recreational trips if the model was not
calibrated.
0.0% 5.0% 10.0% 15.0% 20.0% 25.0% 30.0% 35.0%
1
2
3
4+
2012 Households by Size
PopSyn III PopSyn II
8
Table 2.5 Summary of Land Use Inputs to ABM
Scenario 123 540
emp_total 1,432,951 1,450,966
emp_mil 110,917 103,944
enroll_k_8 361,456 360,564
enroll_9_12 170,185 169,782
enroll_college 77,074 77,105
parkactive 6,231 6,231
openspace 666,809 1,351,368
beachactive 1,005 1,381
hotelrooms 56,469 56,646
2.3: Traffic Count Update
Traffic count data are the primary data source used for validating traffic assignment results. Counts
are often from various agencies such as, state Departments of Transportation, local jurisdictions, and
private contractors; each uses different counting techniques.
In this effort, SANDAG staff assembled, updated, and validated both roadway traffic counts and
transit ridership. The roadway traffic counts are from PeMS (Performance Measurement System)
counts, Caltrans District 11 State Highway Traffic Census counts, arterial counts from local
jurisdictions, and some special counts collected by SANDAG.
A traffic count for a facility is, in effect, a single sample of the set of daily traffic activity that occur
on the link over a period of time. Thus, a single traffic count or a set of traffic counts for a single
facility represent a sample for the link subject to sampling error. This suggests that traffic count data
based on one or two-day counts may be substantially different than the “true” average daily traffic
for a link, even when the traffic count data are adjusted for day of week and seasonal variation.6
Fortunately, the PeMS data are daily counts over the entire year of 2012, allowing deriving average
weekday traffic (AWDT) from a large data set. Other counts, particularly those from local jurisdictions,
normally do not cover the entire year and therefore are subject to larger error than the PeMS counts.
SANDAG staff tried to use as many reliable counts as possible. Table 2.6 shows a roadway count data
comparison, by data source, between prior and current validation efforts.
Table 2.6 Roadway Traffic Counts by Data Source
Data Source Previous Counts Current Counts
PeMS 167 918
Caltrans District 11 238 256
Local Jurisdiction 1,602 1,181
Other 54 60
Total 2,061 2,415
9
Overall, the total number of counts increased from 2061 to 2415; PeMS counts used in validation
increased from 167 to 918, attributed to including freeway ramps counts and matching more counts
to highway links; local jurisdiction counts used in validation decreased from 1602 to 1181; PeMS
counts tend to have better quality compared to arterial counts from local jurisdictions. In previous
validations, many local jurisdiction counts were matched to multiple network links, thus making the
comparison between estimated link traffic and counts a bit ambiguous. In this effort, each count was
matched to a link with a count closest to estimated volume among all links.
2.3.1: PeMS Counts
Caltrans PeMS provides both real-time and historical performance data including traffic flow and
vehicle occupancy in many formats. SANDAG staff downloaded the 2012 weekdays monthly average
daily traffic (MADT) of 1270 Vehicle Detector Stations (VDS) actively used in 2012, and calculated the
annual Weekday average. Then, each VDS is matched with a highway link in the model’s highway
network. On links where the Caltrans District 11 Traffic Census counts exist, the PeMS data was
removed to avoid duplicates. A small amount of counts was not used where the VDS locations can’t
be matched to the highway links precisely. Additionally, SANDAG staff also reviewed the ten-years
PeMS historical counts from 2006 to 2015 to check the data consistency through the years. Traffic
counts which varied significantly over a small period of years were not used to ensure the counts are
reliable. As a result, 918 PeMS counts were retained for model validation.
2.3.2: Caltrans District 11 State Highway Traffic Census Counts
District 11 modeling staff provided the 2012 Caltrans District 11 State Highway Traffic Census counts,
typical weekday hourly volume by station/direction. Among counts at 402 locations, SANDAG staff
matched 256 counts to corresponding highway links after combining two-way counts and removing
duplicated and/or invalid counts.
2.3.3: Local Jurisdiction Counts
Local jurisdictions in the San Diego county collect weekday daily two-way counts on major arterials.
SANDAG staff assembled and compiled these counts from various jurisdictions. Originally, the
assembled 1189 counts were matched to about 1400 highway links because in some cases one count
matched to multiple links. In the cases where one count was matched to multiple links, the raw count
data provided to SANDAG doesn’t allow pinpointing the exact count locations and matching one
count to one link exactly. SANDAG staff picked one link with estimated volume matches count most
closely. For future validation efforts, it is recommended the count locations should be geocoded by
local jurisdictions who collected the counts. By removing a few bad and/or duplicated counts, at the
end, 1181 counts were matched to inks in the highway network for validation purpose.
2.3.4: Other Counts
Periodically, SANDAG collected counts for special modeling purposes. In 2013, SANDAG hired
consultant firm TRA to collect traffic counts along roads crossing specific screen lines for ABM
validation. The traffic was counted for 24-hours on a weekday (Tuesday to Thursday) in July 2013, by
five-minute intervals at 112 stations (counting both directions). These counts were matched to 60
highway links (most of them are two-way links) for validation purposes.
10
2.3.5: Summary of Counts
Overall, 2415 counts were matched to highway links for model validation. Table 2.7 lists the
distribution by count data source. Approximately half of the counts are local jurisdiction counts and
the other half are from PeMS and Caltrans District 11.
Table 2.7 Roadway Traffic Counts by Source
Count Source No. of Counts % of Counts
PeMS 918 38%
Caltrans highway census 256 11%
Local jurisdiction counts 1,181 49%
Other 60 2%
Total 2,415 100%
The counts are from four road types: (1) freeway links, which includes freeway main lanes and ramps;
(2) arterial links, which include prime and major arterials; (3) collector links; and finally, (4) other local
roadway links. Table 2.8 lists the distribution of counts by road type.
Table 2.8 Count Distributions by Road Type
Road Type No. of Counts % of Counts
Freeway 1,139 47%
Arterial 636 26%
Collector 606 25%
Other local 34 1%
Total 2,415 100%
The 2415 links with counts represent 8 percent of all highway links and 12 percent of the total
network length in the SANDAG model. The coverage for freeway main lanes is much higher than that
of other road types, 345 miles or 45 percent of the total length of freeway main lanes. Figure 2.5
shows the count coverage by source. Figure 2.6 shows the count coverage by road type.
11
Figure 2.5 Count Coverage by Source
12
Figure 2.6 Count Coverage by Road Type
2.4: Calibration Target Changes
The American Community Survey (ACS)4 is the premier source for detailed information about the
American people and workforce. ACS helps local officials, community leaders, and businesses
understand the changes taking place in their communities. In previous efforts, calibration targets
were assembled from the 2006 San Diego HHS. San Diego residents’ travel behavior have been
changing as regional transportation systems, policies, transportation technologies, and demographics
and culture change. To make the model reflective of these changes to travel behavior, SANDAG staff
updated calibration targets using the 2010-2014 ACS five-year estimates, specifically auto ownership
and work trip mode choice models targets. The base year 2012 is in the middle year of the 2010-2014
ACS estimates, making the ACS a perfect data source for target setting.
2.4.1: Auto Ownership Model Targets
ACS ‘household size by vehicles available’ data, presented in Table 2.9, are used for deriving auto
ownership model calibration targets.
13
Table 2.9 Household S ize by Vehicles Available
2010-2014 ACS 5-Year Estimates (San Diego)4
San Diego County, California
Estimate Margin of Error
Total: 1,083,811 +/-3,426
No vehicle available 66,596 +/-1,587
1 vehicle available 345,515 +/-3,609
2 vehicles available 431,659 +/-3,958
3 vehicles available 162,600 +/-2,664
4 or more vehicles available 77,441 +/-1,648
1-person household: 266,119 +/-3,378
No vehicle available 38,561 +/-1,129
1 vehicle available 189,387 +/-3,010
2 vehicles available 31,462 +/-1,349
3 vehicles available 4,904 +/-450
4 or more vehicles available 1,805 +/-288
2-person household: 351,404 +/-3,355
No vehicle available 14,121 +/-736
1 vehicle available 82,305 +/-1,990
2 vehicles available 201,932 +/-3,078
3 vehicles available 42,200 +/-1,445
4 or more vehicles available 10,846 +/-634
3-person household: 181,981 +/-2,564
No vehicle available 6,113 +/-503
1 vehicle available 35,931 +/-1,330
2 vehicles available 77,609 +/-1,938
3 vehicles available 48,959 +/-1,508
4 or more vehicles available 13,369 +/-862
4-or-more-person household: 284,307 +/-2,975
No vehicle available 7,801 +/-582
1 vehicle available 37,892 +/-1,368
2 vehicles available 120,656 +/-2,082
3 vehicles available 66,537 +/-1,968
4 or more vehicles available 51,421 +/-1,404
Auto ownership model calibration targets, aggregated from ACS data in Table 2.9, is presented in
Table 2.10. Table 2.10 also presents auto ownership targets assembled from 2000 and 2010 Census
data; the 2010 Census data was used in the last calibration.
14
Table 2.10 Auto Ownership Model Calibration Targets
Autos 2000 Census 2010 Census 2014 ACS
0 7.96% 6.24% 6.14%
1 34.71% 32.13% 31.88%
2 39.55% 39.31% 39.83%
3 12.59% 15.07% 15.00%
4+ 5.19% 7.24% 7.15%
Compared with the 2000 and 2010 Census data, the 2010-2014 ACS five-year release shows San Diego
households own more cars. The comparison between 2010-2014 ACS and 2010 Census data shows
that households which own no car or one car decreased slightly while two-car households increased
by 0.52 percent. Auto ownership is one of the key factors that affect a traveler’s mode choices. If a
household doesn’t own a car or doesn’t have enough cars, then the likelihood of household members
choosing non-motorized and transit modes increases. With the 2010-2014 ACS estimates, it is possible
to calibrate the model to reflect updated auto ownership in San Diego.
2.4.2: Work Trip Mode Choice Model Targets
Several Census Bureau surveys contain questions related to commuting including means of
transportation, time of departure, mean travel time to work, vehicles available, distance traveled,
and expenses associated with commuting. In this effort, ACS means of transportation data in
Table 2.11 was used for updating work trip mode choice model targets.
Table 2.11 Commuting (Journey to Work)
Mode Splits from 2010-2014 ACS 5-Year Release (San Diego)4
Car, truck, or van 85.8%
Drove alone 76.1%
Carpooled 9.7%
In 2-person carpool 7.6%
In 3-person carpool 1.2%
In 4-or-more person carpool 0.8%
Public transportation (excluding taxicab) 3.0%
Walked 2.8%
Bicycle 0.7%
Taxicab, motorcycle, or other means 1.2%
Worked at home 6.5%
Total excludes work at home 93.5%
The modes in ACS and those in SANDAG ABM are slightly different. For example, carpool is divided
into shared ride 2 (SR2) and shared ride 3+ (SR3) in ABM while it is divided into 2-person, 3-person,
and 4-person carpool modes in ACS. The ACS modes splits in Table 2.11 were rescaled to be consistent
15
with ABM modes. Table 2.12 shows targets for work trip mode choice model calibration. Additionally,
Table 2.12 also shows previous work trip mode choice targets derived from the 2006 San Diego HHTS.
Table 2.12 Work Purpose Trip Mode Choice Targets
Mode 2010-2014 ACS 2006 Survey
Drive alone (DA) 82.03% 79.89%
Shared Ride 2 (SR2) 8.77% 11.26%
Shared ride 3+ (SR3) 2.14% 4.41%
Walk 2.99% 2.19%
Bike 0.75% N/A
Transit 3.21% 2.25%
Compared with 2006 HHTS, in 2010-2014 ACS, work trip DA share increased from 79.89 percent to
82.03 percent; SR2 and SR3 shares both decreased; walk share increased from 2.19 percent to 2.99
percent; transit share increased from 2.25 percent to 3.21 percent. Work trip bike mode share target
was not derived from the 2006 HHTS because there weren’t enough bike samples. San Diego workers
tend to use transit and non-motorized modes more in 2012 while they are also more likely to drive
alone to work. With the 2010-2014 ACS commuting data, it is possible to calibrate the ABM to reflect
updated work trip mode choices.
2.5: Software Changes
Since the release of the initial ABM software version 13.0.0 in December 2013, there were bug fixing
and software improvements. SANDAG usually releases a new ABM software version on a quarterly
basis, each with a report that can be provided upon request. Issues and bugs are logged in the
SANDAG JIRA system, a proprietary issue tracking product developed by Atlassian. Since scenario 123,
based off version 13.2.3, there are a few subsequent releases including 13.2.4, 13.2.5, and 13.3.0. The
calibration and validation work described in this document was based off version 13.3.0. The
following are a few examples of software changes and improvements since version 13.2.3:
Modification of drive to transit Park and Ride access file generation, ticket [ABM-634].
Elimination of small value cells in aggregate model trip tables, ticket [ABM-627].
Inclusion of telecommute assumptions for future years, ticket [ABM-676].
Inclusion of transponder ownership model, ticket [ABM-736].
Bug fixes in walk transit walk, walk transit drive, and drive transit walk skim calculators, ticket
[ABM-776].
Bug fix of no early morning highway and transit volumes in the crossborder model, ticket [ABM-
479].
Software changes affected model results, such as estimated link volumes and transit ridership. For
example, the inclusion of transponder ownership model [ABM-676] disallows non-transponder
owners from accessing the I-15 MLs, consequentially changed assigned volumes on I-15 MLs. This
calibration and validation effort reflects all software changes since version 13.2.3.
16
2.6: Network Changes
The roadway, transit, and bike networks, representing the supply side of a transportation system are
an integral part of a travel demand model. Although all model networks attempt to represent the
'real world' accurately, it should be noted that model networks normally are not 100 percent accurate
to ground truth. Like the demand side, the network supply side also evolves. There were many
network changes since the last calibration and validation effort, including improvements made for
the dynamic traffic assignment (DTA) development and network coding improvements on I-15.
17
CHAPTER 3: MODEL CALIBRATION
Calibration consists of the adjustment of constants and other model parameters in estimated or
asserted models to make the model replicate observed data for a base (calibration) year or otherwise
produce more reasonable results.6 In general, each model was calibrated as follows: First, comparisons
between observed data and estimated model results are created and analyzed. Next, models are
calibrated by estimating and applying alternative-specific constants (ASC) to each alternative except
one (the base alternative). A set of ASCs are calculated in each iteration of calibration by dividing the
observed percentage by the estimated percentage for each alternative and taking the natural log of
the result. An adjustment factor (typically set to 0.5) is applied to the constants to help eliminate
oscillating patterns between iterations of the calibration routine. To ensure that the model is not
over-specified, a base alternative for each model is selected. The constant for the base alternative is
added to the constant for each alternative, and the base alternative constant is set to 0. The constant
values for the current iteration are then added to the ASCs for the previous iteration, and entered
into the Utility Expression Calculator spreadsheet (UEC) on a separate line so that the calibrated
constants can be tracked separately from the estimated constants. The following sections describe
calibration results of the following models:
Auto ownership model
Coordinated daily activity pattern model
Tour and trip mode choice models
External-internal trip model
Crossborder model
3.1: Auto Ownership Model
The auto ownership model predicts the number of vehicles owned by each household with five choice
alternatives: (1) no car; (2) one car; (3) two cars; (4) three cars; and (5) four or more cars. There are
two instances of the auto ownership model. The first instance, or the pre-work location instance, is
used to assign a preliminary auto ownership to each household, based upon household demographic
variables, zonal characteristics, and destination-choice accessibilities. The second instance, or the post-
work location instance, is used to assign a final auto ownership to each household based upon chosen
work locations and mode choice logsums. This section describes the calibration of the latter instance.
The base alternative for calibrating the auto ownership models is the one-car alternative. The ASCs
were adjusted in a few iterations. Table 3.1 shows the final AOCs while Table 3.2 and Figure 3.1 show
auto ownership calibration results.
Table 3.1 Auto Ownership Model ASC Adjustments
Auto Ownership Alternatives Adjusted ASC
No car -0.354
1 0.000
2 0.394
3 0.497
4+ 0.491
18
Table 3.2 Auto Ownership Model Calibration Results
2010-2014
ACS Model with
PopSyn II Model with PopSyn III
Calibrated Model with PopSyn III
Autos Target Autos % diff% Autos % diff% Autos % diff%
No car 6.14% 92,149 7.64% 1.50% 102,742 8.52% 2.38% 74,609 6.18% 0.04%
1 31.88% 445,750 36.95% 5.07% 424,516 35.18% 3.30% 386,194 32.01% 0.13%
2 39.83% 430,299 35.66% -4.17% 445,708 36.94% -2.89% 481,295 39.89% 0.06%
3 15.00% 164,915 13.67% -1.33% 155,813 12.91% -2.09% 179,249 14.86% -0.14%
4+ 7.15% 73,390 6.08% -1.07% 77,747 6.44% -0.71% 85,179 7.06% -0.09%
100% 1,206,503 100% 1,206,526 100% 1,206,526
Figure 3.1 Auto Ownership Model Calibration Results
The calibrated auto ownership model results match 2010-2014 ACS targets better than the
uncalibrated models. For example, the gap between estimated 0-car ownership and target was
2.38 percent in the uncalibrated model; it is reduced to 0.04 percent in the calibrated model. If the
auto ownership model was not calibrated, the additional 2.38 percent 0-car households in the
uncalibrated model could have skewed transit and non-motorized mode shares significantly. This
could have affected the forecasting of vehicle miles traveled (VMT) and greenhouse gas emissions
(GHG).
-6.00% -4.00% -2.00% 0.00% 2.00% 4.00% 6.00%
No car-6.14%
1 car-31.88%
2 cars-9.83%
3 cars-15.0%
4+ cars-7.15%
Auto Ownership: Difference b/w Estimated and Observed Targets
Calibrated Model with PopSyn III Model with PopSyn III Model with PopSyn II
19
3.2: Coordinated Daily Activ ity Pattern Model
This model predicts the daily activity pattern type for each household member with three activity
pattern alternatives: (1) mandatory (M); (2) non-mandatory (N); and (3) stay-at-home (H). Because of
the correlation between activity pattern types among different household members, especially for
joint non-mandatory and stay-at-home types, the model is estimated across all household members
simultaneously. The interactions or influences of different types of household members is taken into
account through a specific set of interaction variables. The model was calibrated by adjusting only
individual ASCs by person type: full-time worker, part-timer worker, university student, non-working
adult, non-working senior, driving age student, pre-driving age student, and pre-school child. The
calibration targets are the same as those used in the 2008 calibration.1 The base alternative for
calibrating the Coordinated Daily Activity Pattern (CDAP) model is the H pattern. The ASCs were
adjusted in several iterations. Table 3.3 shows the final ASCs while Table 3.4 and Figure 3.2 show
CDAP model calibration results.
Table 3.3 CDAP Model ASC Adjustments
Person type
Activity Pattern
Mandatory Non Mandatory Home
Full-time worker 0.226 -0.220 0.000
Part-time worker 0.293 -0.293 0.000
University student -0.247 0.170 0.000
Non-working adult -0.029 0.089 0.000
Non-working senior -0.079 -0.091 0.000
Driving age student 0.048 -0.861 0.000
Pre-driving student 0.554 0.235 0.000
Pre-school 0.358 0.067 0.000
20
Table 3.4 Coordinated Daily Activ ity Pattern Model Calibration Results
Person Type CDAP Target: 2008 Calibration
Calibrated w/ PopSyn II
diff% Uncalibrated w/ PopSyn III
diff% Calibrated
w/ PopSyn III diff%
Child H 16.0% 13.5% -2.5% 14.7% -1.3% 14.89% -1.1%
Child M 43.0% 50.0% 7.0% 47.0% 4.0% 45.89% 2.9%
Child N 41.0% 36.5% -4.5% 38.3% -2.7% 39.21% -1.8%
Full-time worker H 5.0% 7.2% 2.2% 7.2% 2.2% 5.04% 0.0%
Full-time worker M 87.0% 81.8% -5.2% 81.7% -5.3% 86.95% 0.0%
Full-time worker N 8.0% 10.9% 2.9% 11.1% 3.1% 8.01% 0.0%
Non-worker H 25.0% 24.7% -0.3% 25.5% 0.5% 24.73% -0.3%
Non-worker N 75.0% 75.3% 0.3% 74.5% -0.5% 75.27% 0.3%
Part-time worker H 7.0% 10.9% 3.9% 10.9% 3.9% 6.99% 0.0%
Part-time worker M 73.0% 63.1% -9.9% 62.5% -10.5% 72.70% -0.3%
Part-time worker N 20.0% 26.0% 6.0% 26.6% 6.6% 20.31% 0.3%
Retired H 27.0% 26.0% -1.0% 25.9% -1.1% 26.96% 0.0%
Retired N 73.0% 74.0% 1.0% 74.1% 1.1% 73.04% 0.0%
Driving age student H 5.0% 5.1% 0.1% 4.9% -0.1% 5.03% 0.0%
Driving age student M 91.0% 92.0% 1.0% 91.1% 0.1% 90.92% -0.1%
Driving age student N 4.0% 2.9% -1.1% 4.0% 0.0% 4.04% 0.0%
Non-driving age student H 2.0% 1.4% -0.6% 1.6% -0.4% 1.26% -0.7%
Non-driving age student M 94.0% 96.3% 2.3% 95.0% 1.0% 94.64% 0.6%
Non-driving age student N 4.0% 2.3% -1.7% 3.4% -0.6% 4.10% 0.1%
University student H 9.0% 10.2% 1.2% 9.7% 0.7% 8.93% -0.1%
University student M 66.0% 67.6% 1.6% 65.9% -0.1% 66.14% 0.1%
University student N 25.0% 22.2% -2.8% 24.4% -0.6% 24.93% -0.1%
21
Figure 3.2 CDAP Model Calibration Results
As shown in Figure 3.2, CDAP results shifted away from the targets in the uncalibrated PopSyn II
model, especially for full-time and part-time workers; more workers assigned with non-mandatory
and stay-home patterns and fewer workers assigned with mandatory patterns compared with the
-12.0% -10.0% -8.0% -6.0% -4.0% -2.0% 0.0% 2.0% 4.0% 6.0% 8.0%
Child_H
Child_M
Child_N
Full-time worker_H
Full-time worker_M
Full-time worker_N
Non-worker_H
Non-worker_N
Part-time worker_H
Part-time worker_M
Part-time worker_N
Retired_H
Retired_N
Driving age student_H
Driving age student_M
Driving age student_N
Non-driving age student_H
Non-driving age student_M
Non-driving age student_N
University student_H
University student_M
University student_N
CDAP: Difference b/w Estimated and Observed Targets
Calibrated Model with PopSyn III Model with PopSyn III Model with PopSyn II
22
targets. Workers assigned with mandatory patterns are more likely travel during peak hours because
of their work/school schedules. If the model was not calibrated, peak hour traffic could have been
skewed. As shown in Figure 3.2, the CDAP results match targets better after calibration.
3.3: Mode Choice Models
Trip mode choice model predicts a mode for each trip with 26 alternatives (Figure 3.3). This model is
also referred as a trip mode ‘switching’ model because a trip mode choice is conditioned by the chosen
tour mode. Both trip and tour mode choice models are segmented by work, university, school,
maintenance, and discretionary purposes as well as work subtour. Trip mode choice ASCs were
adjusted to match calibration targets; and the same set of ASCs were also applied to tour mode choice
model.
Figure 3.3 SANDAG Trip Mode Choice Structure
In the San Diego 2010-2014 ACS, journey to work mode splits are: DA 82.0 percent, SR2 8.8 percent,
SR3 2.2 percent, walk 3.0 percent, bike 0.8 percent, and transit 3.2 percent. Compared with work trip
mode shares summarized from 2006 HHTS, the ACS shows larger DA, walk, and transit mode shares
and smaller SR2 and SR3 mode shares. The mode choice calibration work can be divided into these
categories:
Calibrate work trip mode choice to match ACS targets.
Reduce volume on I-5 and I-15 MLs as the uncalibrated model significantly over estimates ML
volumes.
Increase estimated transit ridership to match updated transit ridership targets.
Choice
Auto
Drive alone
GP(1)
Pay(2)
Shared ride 2
GP(3)
HOV(4)
Pay(5)
Shared ride 3+
GP(6)
HOV(7)
Pay(8)
Non-motorized
Walk(9)
Bike(10)
Transit
Walk access
Local bus(11)
Express bus(12)
BRT(13)
LRT(14)
Commuter rail(15)
PNR access
Local bus(16)
Express bus(17)
BRT(18)
LRT(19)
Commuter rail(20)
KNR access
Local bus(21)
Express bus(22)
BRT(23)
LRT(24)
Commuter rail(25)
School Bus
23
The next sections describe ASC adjustments and calibration results. Although the procedure to adjust
ASC was a stepwise effort for each of the above categories, the calibration results represent
cumulative effects of all ASC adjustments.
3.3.1: Work Trip Mode Choice
Work trip mode choice model was calibrated by adjusting ASCs of modes: DA, SR2, SR3, walk, bike
and transit. The same ASCs were applied to all sub modes of aggregate modes. For example, the
adjusted SR2 ASC -0.338 was applied to all SR2 sub modes: SR2_GP, SR2HOV, and SR2Toll. Table 3.5
shows final AOCs of the calibrated work trip mode choice model. Table 3.6 and Figure 3.4 show work
trip mode choice calibration results.
Table 3.5 Adjusted Work Trip Mode Choice ASCs
Mode Adjusted ASC
Drive alone 0.000
Shared Ride 2 -0.338
Shared ride 3+ -0.680
Walk -1.286
Bike -0.810
Transit 0.082
Table 3.6 Work Trip Mode Choice Calibration Results
Uncalibrated w/
PopSyn II
Uncalibrated w/
PopSyn III
Calibrated w/
PopSyn III
Modes Target:
2010-2014 ACS Estimated % Diff Estimated % Diff Estimated % Diff
Drive alone 82.03% 72.20% -9.83% 71.65% -10.38% 81.52% -0.51%
Shared Ride 2 8.77% 13.56% 4.79% 13.46% 4.69% 8.64% -0.13%
Shared ride 3+ 2.14% 5.83% 3.69% 5.68% 3.54% 2.31% 0.17%
Walk 2.99% 4.77% 1.78% 5.16% 2.17% 3.34% 0.34%
Bike 0.75% 1.06% 0.31% 1.23% 0.48% 0.85% 0.10%
Transit 3.21% 2.57% -0.64% 2.81% -0.40% 3.33% 0.13%
24
Figure 3.4 Work Trip Mode Choice Calibration Results
Uncalibrated models with PopSyn II and III both underestimate DA shares and overestimate SR2, SR3,
and Walk shares. DA shares were underestimated by 10 percent in both cases. Estimated work trip
mode shares of the calibrated PopSyn III model match targets well, with a DA underestimation by
merely 0.5 percent.
3.3.2: Managed Lane (ML) Volume Adjustments
The uncalibrated model overestimates ML volumes on both I-5 and I-15 corridors. ASCs were adjusted
to reduce these modes on MLs: DRIVEALONEPAY, SR2HOV, and SR3HOV. Table 3.7 shows the adjusted
ASCs. Table 3.8 shows the comparisons between estimated and observed counts and on I-5 and I-15
after applying the ASC adjustments.
-12.00% -10.00% -8.00% -6.00% -4.00% -2.00% 0.00% 2.00% 4.00% 6.00%
Drive alone
Shared Ride 2
Shared ride 3+
Walk
Bike
Transit
Work Trip Model Calibration Results: %Difference b/w Estiamted and Target
Calibrated w/ PopSyn III Uncalibrated w/ PopSyn III Uncalibrated w/ PopSyn II
25
Table 3.7 Adjusted ASCs to Reduce ML Volumes
Mode ACS Adjustment
DRIVEALONEFREE 0.000
DRIVEALONEPAY 0.330
SHARED2GP 0.156
SHARED2HOV -0.337
SHARED2PAY -1.034
SHARED3GP 0.173
SHARED3HOV -0.303
SHARED3PAY -1.045
WALK 0.139
BIKE 0.170
WALK_LOC 0.005
WALK_EXP 0.020
WALK_BRT 0.000
WALK_LR 0.000
WALK_CR 0.034
PNR_LOC 0.000
PNR_EXP 0.000
PNR_BRT 0.000
PNR_LR 0.393
PNR_CR 0.034
KNR_LOC 0.180
KNR_EXP 0.000
KNR_BRT 0.000
KNR_LR 0.293
KNR_CR 0.034
26
Table 3.8 I-5 and I-15 ML Validation Results
Uncalibrated models with PopSyn II and III both overestimate ML volumes, compared with counts at
12 locations on I-5 and I-15 corridors. After applying the ASC adjustments, the gap between estimated
and observed counts are smaller. However, in general, the overestimation of ML volumes still exists
and thus suggests further investigations and adjustments are necessary to improve ML volumes.
3.3.3: Transit Mode Adjustments
The 2012 transit ridership targets used in previous calibration and validation efforts were reviewed
by staff familiar with the SANDAG passenger count program; and this led to a decision to update the
targets as shown in Table 3.9.
Table 3.9 Transit Ridership Target Adjustments
Targets Before Adjustment After Adjustment % Diff
Commuter Rail 5,482 5,482 0.00%
Light Rail 123,729 126,861 2.53%
Limited Express Bus 1,430 1,430 0.00%
Local Bus 216,435 229,169 5.88%
Total 347,076 362,942 4.57%
An ASC adjustment of -0.0279 is applied to all transit modes for all trip purposes, including work,
university, school, maintenance, discretionary as well as work subtour. The transit ridership results
from the calibrated model are discussed in section 4.2.
Corridor Hwycov_id from to Count Uncalibrated w/ PopSyn II
Uncalibrated w/ PopSyn III
Calibrated w/ PopSyn III
Vol Diff Vol Diff Vol Diff
I-15 HOV NB 459 Mira Mesa Rancho Penasquitos 21,975 26,900 22% 27,057 23% 19,428 -12%
I-15 HOV NB 464 SR-56 EB Carmel Mtn 14,539 26,024 79% 26,520 82% 19,011 31%
I-15 HOV NB 473 Via Rancho Via Rancho 12,609 27,571 119% 28,058 123% 20,306 61%
I-15 HOV NB 23053 Rancho Bernardo Pomerado 13,170 33,124 152% 32,339 146% 22,338 70%
I-15 HOV SB 25749 SR-78 Valley 9,689 24,959 158% 24,965 158% 16,406 69%
I-15 HOV SB 22275 Carmel Mtn SR-56 EB 20,656 19,577 -5% 20,130 -3% 13,592 -34%
I-5 HOV NB 25381 Del Mar Heights Via De La Valle 8,736 16,824 93% 17,024 95% 11,279 29%
I-5 HOV NB 29565 Carmel Valley Carmel Valley 8,298 16,824 103% 17,024 105% 11,279 36%
I-5 HOV NB 29571 Carmel Valley Via De La Valle 7,307 16,824 130% 17,024 133% 11,279 54%
I-5 HOV SB 29573 Carmel Valley Carmel Valley 7,999 17,236 115% 17,444 118% 11,530 44%
I-5 HOV SB 29577 Via De La Valle Via De La Valle 7,486 17,236 130% 17,444 133% 11,530 54%
I-5 HOV SB 26117 Carmel Valley Carmel Mountain 5,747 17,236 200% 17,444 204% 11,530 101%
Total Total 138,211 260,336 88% 262,473 90% 179,507 30%
27
3.4: Crossborder Model
Crossborder model predicts Mexican residents’ trips inside the San Diego region. In the uncalibrated
models with PopSyn II and III, 9.5 percent and 12.9 percent of Mexican residents have trip ends in
military installations; Naval Base San Diego (NBSD) at 32nd Street is the largest attraction to
crossborder trips. The attractiveness of NBSD can be explained by its proximity to the U.S.-Mexican
border and the large employment and houses/group quarters (GQs) coded on at NBSB. Considering
the strict security checking at military bases, it is hard to justify the significant amount of trips end in
bases. TAZs marked as military use were removed from the crossborder trip destination sample
alternatives, hence significantly reduced the amount of crossborder trips ending in military bases as
shown in Table 3.10 and Figure 3.5. Because military MGRAs are still included in the trip stop location
choice alternatives, it is still possible for a crossborder tour to have a stop in military bases. This
approach effectively reduced the number of crossborder trips ended in military zones. However, a
further investigation of the crossborder survey to understand whether and how Mexican residents
chose military zones as tour and/or stop locations is recommended. Additionally, compiling
distributions of military counts by Mexican and non-Mexican residents could help create crossborder
trip destination choice targets and thus adjust the model to match targets at military zones.
Table 3.10 Cross Border Trips with Destinations in Military Base
Calibrated w/ PopSyn II
Uncalibrated w/ PopSyn III
Calibrated w/ PopSyn III
Xborder trips w/ destinations in military bases 32,814 44,719 17,007
Xborder trips w/ destinations in NBSD 22,897 24,746 12,486
Total Xborder trips 345,642 345,642 375,472
% of Xborder trips w/ destinations in military bases 9.5% 12.9% 4.5%
Figure 3.5 Cross Border Trips with Destinations in Military Bases
32,814 44,719 17,007
345,642 345,642 375,472
9.5%
12.9%
4.5%
0.0%
2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
-
50,000
100,000
150,000
200,000
250,000
300,000
350,000
400,000
Model w/ PopSyn II Model w/ PopSyn III Calibrated Model w/ PopSynIII
Cross Border Trips w/ Destinations in Military Bases
Xborder trips w/ destinations in military bases
Total Xborder trips
% of Xborder trips w/ destinations in military bases
28
CHAPTER 4: MODEL VALIDATION
Travel models are typically defined as a set of mathematical formulas and relationships to replicate
travel decisions. Since travel models (and travel modelers) cannot be omniscient, there will always be
missed information and abstractions resulting in less than perfect models.6 Essentially, all models are
wrong, but some are useful. The practical question is how wrong do they have to be to not be useful.7
The model validation effort described here attempts to improve the model usefulness. Validation is
the application of the calibrated models (discussed in Chapter 3) and comparison of the results against
observed data. Throughout this chapter, the model estimated results refer to results summarized from
a 2012 base year scenario 540. The model estimated results are compared against 2012 observed data
assembled from various sources, including traffic counts, transit ridership data, and regional VMT
data. The difference between model estimated traffic volume and observed count on a link is used
frequently as a validation measure; hereafter it is referred to as the ‘gap.’ The next sections discuss
roadway, transit, and VMT validations.
4.1: Roadway Validation
Roadway validation is to compare assigned volumes against traffic counts. More specifically, the
comparisons in this section are between estimated daily volumes and average weekday traffic counts.
The ‘estimated-count’ gap on each link contributes to two important measures of closeness: percent
root mean squared error (%RMSE) and coefficient of determination (R-squared).
The %RMSE is a measure of accuracy of traffic assignment, representing the average error between
observed and modeled traffic volumes on links with traffic counts. Following the FHWA
recommendations,6 %RMSE is summarized by facility type and by link volume group in this effort.
The coefficient of determination, also known as R-squared, is a statistical measure of how close the
data are to the fitted regression line. The definition of R-squared is fairly straight-forward; it is the
percentage of the response variable variation that is explained by a linear model. R-squared is always
between 0 and 100 percent; 0 percent indicates that the model explains none of the variability of the
response data around its mean; 100 percent indicates that the model explains all the variability of the
response data around its mean.
4.1.1: Roadway Validation by MSA
Table 4.1 presents validation results by Metropolitan Statistical Area (MSA). The last MSA level
validation was conducted for base year 2008;1 the results are presented in Table 4.2.
29
Table 4.1 Roadway Validation Results by MSA Base Year 2012
MSA Observed Count Estimated Volume Difference % Difference
Center City 1,146,404 1,164,888 18,484 1.6%
Central 15,017,450 15,198,199 180,750 1.2%
North City 24,879,428 25,890,523 1,011,095 4.1%
South Suburban 5,290,652 4,927,139 -363,514 -6.9%
East Suburban 7,466,179 7,227,529 -238,650 -3.2%
North County West 6,981,440 6,730,009 -251,431 -3.6%
North County East 8,879,535 8,085,355 -794,180 -8.9%
East County 95,872 129,228 33,356 34.8%
Total 69,756,961 69,352,870 -404,091 -0.6%
Table 4.2 Roadway Validation Results by MSA-Base Year 20081
MSA Observed Count Estimated Volume Difference % Difference
Center City 89,611 104,664 15,053 16.8%
Central 4,800,612 4,938,240 137,628 2.9%
North City 8,201,163 8,173,784 -27,379 -0.3%
South Suburban 1,280,739 1,482,288 201,549 15.7%
East Suburban 3,227,637 3,239,889 12,252 0.4%
North County West 3,647,563 3,191,644 -455,919 -12.5%
North County East 3,144,416 3,169,570 25,154 0.8%
East County 105,138 231,747 126,609 120.4%
Total 24,496,879 24,531,826 34,947 0.1%
The total 2012 regional observed counts increased from 24,496,879 in 2008 to 69,756,961 in 2012, due
to the additional counts staff were able to assemble from various sources. At a regional level, both
2008 and 2012 estimated volumes match counts well; off by 0.1 percent and -0.6 percent respectively.
Validation results at regional level are comparable between base year 2008 and the current 2012
version. However, the current 2012 version fares better at the individual MSA level. Figure 4.1 shows
the comparison between these two versions.
30
Figure 4.1 Validation Results by MSA-Latest 2012 Version vs. 2008 Base Year
Each County MSA is the obvious outlier in both base year 2008 and the current 2012 version. However,
the gap between estimated and observed has been reduced from 120 percent to 35 percent, a
significant improvement. Three other MSAs, Center City, South Suburban, and North County West,
have gaps larger than 10 percent in base year 2008 while no other MSAs except East County MSA
have gaps larger than 10 percent in the current 2012 version.
4.1.2: Roadway Validation by Road Type
Scatterplots of modeled versus observed traffic volumes are useful validation tools often combined
with R-squared summaries. Following the recommendations of FHWA,6 the R-squared statistics was
calculated for links with similar characteristics by facility type (or road class).
Figure 4.2 is a scatter plot of estimated versus observed daily volumes. As there is no systematic
deviation from the 45-degree line, the estimated volumes match counts well. The slope of the
regressed linear line is 1.02, representing a slight overestimation. The R-squared value of 0.959
represents a good fit between the estimated volumes and the linear regression line.
0 0 0 0 0 0 0 0
2008 version' 16.8% 2.9% -0.3% 15.7% 0.4% -12.5% 0.8% 120.4%
'latest scenario' 1.6% 1.2% 4.1% -6.9% -3.2% -3.6% -8.9% 34.8%
16.8%
2.9%
-0.3%
15.7%
0.4%
-12.5%
0.8%
120.4%
2% 1% 4%
-7% -3% -4%-9%
35%
-20.0%
0.0%
20.0%
40.0%
60.0%
80.0%
100.0%
120.0%
140.0%
Dif
fere
nce
by
pe
rce
tns
Validation Results Comparison by MSA - Current Version vs. 2008 Version
2008 version'
'latest scenario'
31
Figure 4.2 Roadway Validation Results -All Road Classes
The ‘ifc’ attribute in the SANDAG network represents a link’s road functional class. For validation
analysis, ten functional classes are aggregated to road classes shown in Table 4.3. Additionally, a
HOV/Toll class is added for validating assigned volumes on HOV/Toll lanes. At the end, road classes
used in validation are: freeway, arterial, collector and local road, ramp, and HOV/Toll lane.
Table 4.3 Correspondence between IFC and Road Class
Functional Class Description Road Class
1 Freeway Freeway
2 Prime Arterial Arterial
3 Major Arterial Arterial
4 Collector Collector & Local
5 Local Collector Collector & Local
6 Rural Collector Collector & Local
7 Local (non-circulation element) Road Collector & Local
8 Freeway Connector Ramp Ramp
9 Local Ramp Ramp
10 Zone Connector N/A
y = 1.0213xR² = 0.9595
y = x
0
20000
40000
60000
80000
100000
120000
140000
160000
180000
200000
0 50000 100000 150000 200000
Esti
mat
ed
Dai
ly V
olu
me
Observed Daily Count
Model Estimated Daily Volumes vs. Counts - All Road Classes
Linear (modeled)
Linear (45 degree)
32
Figure 4.3 is a scatter plot of estimated versus observed daily counts by freeway, ramp, HOV/toll,
arterial, and collector road type. Freeways fare better than other road types with a regressed linear
line slope at 1.043; a slight overestimation on freeways. The model tends to underestimate on arterial,
ramp, and collectors, with linear line slopes at 0.875, 0.93, and 0.755 respectively. The model tends to
overestimate on HOV/Toll links with a linear line slope at 1.1257, reconfirming the findings in section
3.3.2. The R-squared value of freeways is 0.893, the best among all five road types. Unlike freeway
PeMS counts, many collector and arterial counts are collected by local jurisdictions using various
methods. The lack of a systematic approach of collecting arterial and collector counts could be a
contributing factor to the less than ideal performances on arterials and collectors.
Figure 4.3 Roadway Validation Results by Road Class
Validation results are also summarized by gap range between estimated and observed, as shown in
Table 4.4.
y = 1.043xR² = 0.893
y = 0.930xR² = 0.665
y = 1.1257xR² = 0.3578
y = 0.875xR² = 0.765
y = 0.755xR² = 0.4170
20000
40000
60000
80000
100000
120000
140000
160000
180000
0 20000 40000 60000 80000 100000 120000 140000 160000 180000
Esti
mat
ed
Dai
ly V
olu
me
Observed Daily Count
Model Estimated Daily Volumes vs. Counts - by Road Class
freeway
'ramp'
'hov/toll'
arterial
collector
45 Degree
Linear (freeway)
Linear ('ramp')
Linear ('hov/toll')
Linear (arterial)
Linear (collector)
33
Table 4.4 Roadway Validation Results by Road Type
Gap Range
Number of links within Gap Range
freeway ramp hov/toll arterial collector all
Links % Links % Links % Links % Links % Links %
>=100% 1 0% 19 4% 1 7% 9 1% 22 3% 52 2%
50%~100% 9 1% 37 8% 5 33% 22 3% 34 5% 107 4%
30%~50% 31 5% 40 8% 3 20% 28 4% 37 6% 139 6%
20%~30% 39 6% 23 5% 1 7% 26 4% 29 5% 118 5%
10%~20% 138 22% 36 7% 0 0% 38 6% 38 6% 250 10%
0%~10% 216 34% 35 7% 0 0% 103 16% 81 13% 435 18%
0%~-10% 146 23% 64 13% 1 7% 90 14% 55 9% 356 15%
-10%~-20% 34 5% 68 14% 1 7% 95 15% 74 12% 272 11%
-20%~-30% 10 2% 62 13% 0 0% 68 11% 49 8% 189 8%
-30%~-50% 6 1% 76 15% 1 7% 119 19% 102 16% 304 13%
<-50% 1 0% 33 7% 2 13% 38 6% 119 19% 193 8%
total 631 100% 493 100% 15 100% 636 100% 640 100% 2415 100%
-10% ~ +10% 362 57% 99 20% 1 7% 193 30% 136 21% 791 33%
-20% ~ +20% 534 85% 203 41% 2 13% 326 51% 248 39% 1313 54%
-30% ~ +30% 583 92% 288 58% 3 20% 420 66% 326 51% 1620 67%
Average (+) Gaps 13% 48% 55% 24% 64% 33%
Average (-) Gaps -8% -27% -43% -26% -37% -27%
Average of all 7% 2% 22% -8% -8% 1%
%RMSE 13% 40% 47% 30% 55% 23%
Among 2,415 links with counts, estimated volumes on 1,620 links (67%) are within +-30 percent gap
range; 1,313 links (54%) are within +-20 percent gap range; 791 links or (33%) are within +-10 percent
gap range. Among the 631 freeway links, estimated volumes on 583 links (92%) are within
+-30 percent gap range; 534 links (85%) are within in +-20 percent gap range; 362 links (57%) are
within +-10 percent gap range. Among the 636 arterials, 66 percent, 51percent, and 30 percent of
links fall within +-30 percent, +-20 percent, and +-10 percent gap ranges, respectively. Among the 640
collectors, 51 percent, 39 percent, and 21 percent of links fall within +-30 percent, +-20 percent, and
+-10 percent gap ranges, respectively. Among the 493 ramps, 58 percent, 41 percent, and 20 percent
of links fall within +-30 percent, +-20 percent, and +-10 percent, respectively.
34
4.1.3: Roadway Validation by Volume
Validation results are grouped by link volume with incremental units of 10,000 trips. The results are
presented in Table 4.5.
35
Table 4.5 Roadway Validation Results by Volume
Gap Range
Number of links within Gap Range
<10,000 10,000 20,000 30,000 40,000 50,000 60,000 70,000 80,000 90,000 100,000 110000+ all
links % links % links % links % links % links % links % links % links % links % links % links % links %
>=100% 48 5% 4 1% 0 0% 0 0% 0 0% 0 0% 0 0% 0 0% 0 0% 0 0% 0 0% 0 0% 52 2%
50%~100% 77 9% 17 3% 6 2% 4 3% 0 0% 1 1% 1 1% 1 1% 0 0% 0 0% 0 0% 0 0% 107 4%
30%~50% 76 9% 30 6% 4 1% 10 9% 7 8% 2 3% 3 3% 7 8% 0 0% 0 0% 0 0% 0 0% 139 6%
20%~30% 45 5% 18 3% 16 6% 6 5% 10 12% 10 13% 0 0% 6 6% 5 7% 1 1% 1 2% 0 0% 118 5%
10%~20% 57 6% 36 7% 19 7% 20 17% 8 10% 18 23% 21 23% 20 22% 18 24% 18 22% 10 17% 5 13% 250 10%
0%~10% 92 10% 67 12% 42 15% 20 17% 22 27% 17 21% 32 36% 37 40% 33 44% 40 49% 23 38% 10 26% 435 18%
0%~-10% 98 11% 64 12% 29 10% 19 17% 13 16% 21 26% 17 19% 18 19% 15 20% 22 27% 23 38% 17 45% 356 15%
-10%~-20% 114 13% 60 11% 48 17% 11 10% 6 7% 5 6% 14 16% 3 3% 2 3% 1 1% 3 5% 5 13% 272 11%
-20%~-30% 71 8% 67 12% 29 10% 7 6% 10 12% 2 3% 1 1% 0 0% 1 1% 0 0% 0 0% 1 3% 189 8%
-30%~-50% 109 12% 108 20% 60 22% 15 13% 6 7% 3 4% 1 1% 1 1% 1 1% 0 0% 0 0% 0 0% 304 13%
<-50% 95 11% 69 13% 24 9% 3 3% 1 1% 1 1% 0 0% 0 0% 0 0% 0 0% 0 0% 0 0% 193 8%
total 882 100% 540 100% 277 100% 115 100% 83 100% 80 100% 90 100% 93 100% 75 100% 82 100% 60 100% 38 100% 2,415 100%
-10% ~ +10% 190 22% 131 24% 71 26% 39 34% 35 42% 38 48% 49 54% 55 59% 48 64% 62 76% 46 77% 27 71% 791 33%
-20% ~ +20% 361 41% 227 42% 138 50% 70 61% 49 59% 61 76% 84 93% 78 84% 68 91% 81 99% 59 98% 37 97% 1,313 54%
-30% ~ +30% 477 54% 312 58% 183 66% 83 72% 69 83% 73 91% 85 94% 84 90% 74 99% 82 100% 60 100% 38 100% 1,620 67%
Average (+) Gaps 64% 24% 16% 20% 16% 14% 10% 12% 10% 8% 7% 5% 33%
Average (-) Gaps -30% -31% -29% -22% -19% -11% -11% -6% -7% -4% -5% -9% -27%
Average of all 12% -14% -15% 0% 1% 4% 3% 8% 6% 4% 2% -3% 1%
%RMSE 45% 38% 32% 28% 21% 18% 14% 15% 12% 9% 8% 8% 23%
Wu, please note, due to the amount of content of this table, the page setting is: landscape, 8.5 x 14
36
As shown in Table 4.5, as volume increases, validation results improve. For the 90K+ group, 100
percent of the links are within +30 percent gap range, 97 to 99 percent are within +-20 percent gap
range, and 71 to 77 percent are within +-10 percent gap range. For the group with volumes less than
10K, only 54 percent of links are within +-30 percent gap range, 41 percent are within +-20 percent
range, and 22 percent are within +-10 percent range. Out of the 2,415 links with counts, there are
157 (or 6%) outlier with gaps greater than 50 percent; the majority of them (135 out of the 157) are
links with volumes less than 10K. For the groups with volumes larger than 50K, if a gap threshold of
30+ percent is used to identify outliers, then there are only 15 outliers (some of which will be discussed
in the corridor level validation section 4.16).
4.1.4: Roadway Validation by Count Source
Figure 4.4 shows a summary of validation results by data source: PeMS, Caltrans District 11 Census,
local jurisdiction, and special counts.
Figure 4.4 Roadway Validation Results by Count Data Source
Assigned volumes match PeMS and Caltrans District 11 counts well; both show slight overestimations
with slopes of 1.053 and 1.019 and R-squared values 0.967 and 0.952, respectively. Assigned volumes
tend to be smaller than observed on links with local jurisdiction counts. Links with local jurisdiction
counts are mostly on arterial, collector, and local road links. These links typically have smaller traffic
volumes, which present a challenge to be matched to counts. Nevertheless, the less than ideal results
on links with local jurisdiction counts also indicate these counts are subject to further investigation.
y = 1.053xR² = 0.967
y = 1.019xR² = 0.952
y = 0.857xR² = 0.805
y = 0.852xR² = 0.601
0
20000
40000
60000
80000
100000
120000
140000
160000
180000
0 20000 40000 60000 80000 100000 120000 140000 160000 180000
Esti
mat
ed
Dai
ly V
olu
me
Observed Daily Count
Model Estimated Daily Volumes vs. Counts - by Data Source
PEMS
'Caltran'
Jurisdiction
TRA
Linear (PEMS)
Linear ('Caltran')
Linear (Jurisdiction)
Linear (TRA)
37
4.1.5: Roadway Validation by Key Count Locations
Table 4.6 shows validation results at 12 Key count locations used in previous model validations. They
were selected along three major North-South corridors: (1) I-5; (2) I-15; and (3) I-805, plus one
East-West corridor: SR52.
Table 4.6 Validation Results by Key Count Location
ID hwycov_id Corridor Location Count Estimated Gap
1 27195 I-15 NB I-15 at Rainbow Valley Blvd 67,427 59,662 -12%
1 27196 I-15 SB I-15 at Rainbow Valley Blvd 68,438 60,281 -12%
2 27597 I-5 NB I-5 at Camp Pendleton 63,139 59,986 -5%
2 27598 I-5 SB I-5 at Camp Pendleton 63,579 62,672 -1%
3 12929 I-15 NB I-15 at Valley Parkway 97,866 99,143 1%
3 12943 I-15 SB I-15 at Valley Parkway 96,115 102,599 7%
4 14647 I-15 NB I-15 at Carmel Mountain Rd 118,519 119,208 1%
4 14644 I-15 SB I-15 at Carmel Mountain Rd 117,785 119,584 2%
5 14262 I-5 NB I-5 at Carmel Mountain Rd 98,242 108,595 11%
5 30776 I-5 SB I-5 at Carmel Mountain Rd 100,162 94,528 -6%
6 8896 I-5 NB I-5 at Sorrento Valley Rd 76,679 77,251 1%
6 14263 I-5 SB I-5 at Sorrento Valley Rd 72,632 77,502 7%
8 574 I-15 NB I-15 at Junction SR163 165,551 158,258 -4%
8 14419 I-15 SB I-15 at Junction SR163 152,191 153,908 1%
9 15238 SR-52 WB SR52 at Mast Blvd 49,433 64,201 30%
9 7375 SR-52 EB SR52 at Mast Blvd 48,132 59,458 24%
10 11102 SR-52 WB SR52 at Convoy 53,777 67,826 26%
10 11103 SR-52 EB SR52 at Convoy 55,802 69,235 24%
11 26707 I-5 NB I-5 at North of Friars Rd 104,943 103,130 -2%
11 8793 I-5 SB I-5 at North of Friars Rd 102,264 98,104 -4%
12 686 I-805 NB I-805 at North San Ysidro Blvd 26,838 34,181 27%
12 857 I-805 SB I-805 at North San Ysidro Blvd 30,277 35,309 17%
13 27694 I-5 NB I-5 at South Junction I-805 16,461 12,990 -21%
13 27743 I-5 SB I-5 at South Junction I-806 19,442 11,332 -42%
Total 1,865,694 1,908,943 2%
The average gap at all key locations is 2 percent, which is similar to the overall freeway trend line
slope. Overall, 18 out of 24 (or 75%) of the estimated volumes match counts well, within -+15 percent
gap range. However, there are two outlier clusters, the SR-52 corridor cluster (location 9 at Mast Blvd,
and location 10 at Convoy St.) and the Mexican border cluster (location 12 on I-805 at San Ysidro
Blvd., and location 13 on I-5 at I-805 South Junction).
38
Estimated volumes at key locations 9 and 10 on SR-52 are overestimated by 24 percent to 30 percent
in both directions. Similarly, at corridor level analysis in section 4.1.6, activity on SR-52 is
overestimated. Further investigation is needed to understand why so many trips are generated and/or
attracted to SR-52 corridor.
Interestingly, the model underestimates at location 13 on I-5 at South Junction with I-805 in both
directions by 21 percent (NB) and 42 percent (SB); while the model overestimates at a nearby key
count location 12 on I-805 at San Ysidro Boulevard in both directions by 27 percent (NB) and 17
percent (SB). If locations 12 and 13 were combined, the gaps between estimated and counts become
9 percent overestimate (NB) and 6 percent underestimate (SB) as shown in Table 4.7. The combined
results suggest the under and over estimations are likely caused by skewed traffic splits between I-5
and I-805 before (SB) and after (NB) crossing the US-Mexico border. Further investigation of network
coding, especially at I-5 and I-8-5 interchanges, as well as cross border model could help address issues
related to locations 12 and 13.
Table 4.7 Validation Results
with Key Count Locations 12 and 13 Combined
Direction Count Estimated Gap
NB 43299 47171 9%
SB 49719 46641 -6%
4.1.6: Roadway Validation by Corridor
Highway corridor performance is an important metric for regional stakeholders. The examination of
model results by highway corridor help establish a travel model’s precision in regional planning
applications. This section describes validation of following highway corridors:
North-South corridors: I-5, I-15, I-805, SR-67, SR-125, and SR-163
East-West corridors: I-8, SR-52, SR-54, SR-56, SR-78, SR-94, and SR-905.
For each direction of a corridor, model estimated daily volumes are plotted against counts along
milepost axis. Plotting estimated volumes against counts along milepost introduces geographic
context into validation analysis and allows easy identifications of outliers. In each plot, the following
statistics are summarized by comparing estimated results against counts (gaps between estimated
and counts):
%Link(+): percentage of links with positive gaps
%Link(-): percentage of links with negative gaps
AvgGap(+): average percentage of positive gaps
AvgGap(-): average percentage of negative gaps
AvgGap(+-): average percentage of all gaps
GapIn(+-10%): percentage of gaps within +-10% range
GapIn(+-20%): percentage of gaps within +-20% range
GapIn(+-30%): percentage of gaps within +-30% range
Slope: slope of the fitted line between estimated and counts
%RMSE: percent of root-mean-square-error
39
Overall, the model performs well at the corridor level. Twenty out of the 618 links (or 3.2%) with
counts are identified as outliers, using the +-35 percent gap range as a threshold. A systematic trait
in many of these outliers occur at major freeway interchanges with complex configurations of lane
merging, splitting, and HOV only traffic. Further investigation of network coding and count-to-link
matching at freeway interchanges could help understand and address the outlier issues. Secondly,
there are a few outliers near the U.S.-Mexico border where I-5 and I-805 merges. This suggests, besides
network coding and count to link matching, that the crossborder model should also be reviewed to
understand why estimated volumes are off in this special area. Figures 4.5 to 4.30 present validation
results and findings by corridor and direction.
Figure 4.5 Validation Results : I-5 Northbound
Estimated volumes match observed counts well on Northbound I-5. The model slightly overestimates
with a trend line slope value at 1.027 and a %RMSE at 10 percent. Among all links with counts,
95 percent of them have estimated volumes within +-30 percent range of counts; 94 percent within
+-20 percent range; 75 percent within +-10 percent range. There are two notable outliers near the
Mexican border; one between I-805 NB and Via De San Ysidro (49% underestimate) and another one
between Via De San Ysidro and San Ysidro Blvd (53% underestimate). Interestingly, the model
overestimates at a nearby count location on Northbound I-805 by 27 percent. These observations
suggest that the outliers could be caused by the crossborder model, network coding, and/or count-to-
link matching near the U.S.-Mexico border where I-5 and I-805 merge.
40
Figure 4.6 Validation Results : I-5 Southbound
Estimated volumes match observed counts well on Southbound I-5. The model slightly overestimates
with a trend line slope value at 1.031 and a %RMSE at 13 percent. Among all links with counts,
91 percent of them have estimated volumes within +-30 percent range of counts; 86 percent within
+-20 percent range; 62 percent within +-10 percent range. There are five notable outliers at locations
near Park Boulevard (62% overestimate), Imperial Avenue (49% overestimate), SR-75 interchange
(37% overestimate), Bay Boulevard (83% overestimate), and I-5 and I-805 junction (42%
underestimate). The counts at Park Boulevard and Bay Boulevard drop significantly compared to
upstream and downstream counts; indicate a review of counts could help understand these outliers
better. At locations where complex lane configurations occur, such as the Park Boulevard location,
count-to-link matching should be reviewed. The only underestimate outlier occurs at the I-5 and I-805
junction. This could be caused by the crossborder model, network coding, and/or count-to-link
matching near the US-Mexico border where I-5 and I-805 merge.
41
Figure 4.7 Validation Results : I-15 Northbound
Estimated volumes match observed counts well on Northbound I-15. The model overestimates with a
trend line slope value at 1.063 and a %RMSE at 13 percent. Among all links with counts, 94 percent
of them have estimated volumes within +-30% range of counts; 89 percent within +-20 percent range;
43 percent within +-10 percent range. There is only one notable outlier (48% overestimate) at I-15
and SR-56 interchange that has complex configurations of lane merging, splitting, and HOV only
traffic. This indicates network coding and/or count-to-link matching should be reviewed.
42
Figure 4.8 Validation Results : I-15 Southbound
Estimated volumes match observed counts well on Southbound I-15. The model overestimates with a
trend line slope value at 1.084 and a %RMSE at 13 percent. Among all links with counts, 100 percent
of them have estimated volumes within +-30 percent range of counts; 90 percent within +-20 percent
range; 42 percent within +-10 percent range. There is no outlier on Southbound I-15.
43
Figure 4.9 Validation Results : I-805 Northbound
Estimated volumes match observed counts well on Northbound I-805. The model slightly
overestimates with a trend line slope value at 1.046 and a %RMSE at 16 percent. Among all links with
counts, 91 percent of them have estimated volumes within +-30 percent range of counts; 76 percent
within +-20 percent range; 56 percent within +-10 percent range. There are two notable outliers near
El Cajon Boulevard and Miramar Road, both with a 53 percent overestimate. The I-805 and Miramar
Road interchange has complex configurations of lane merging, splitting, and on and off ramps,
indicating network coding and/or count-to-link matching should be reviewed.
44
Figure 4.10 Validation Results : I-805 Southbound
Estimated volumes match observed counts well on South I-805. The model overestimates with a trend
line slope value at 1.089 and a %RMSE at 14 percent. Among all links with counts, 95 percent of them
have estimated volumes within +-30 percent range of counts; 82 percent within +-20 percent range;
33 percent within +-10 percent range. There are two notable outliers; one near East San Ysidro
Boulevard close to the U.S.-Mexico border where I-5 and I-805 merges (36% overestimate) and
another one near the SR-54 interchange (36% overestimate). These two outliers again occur either
near the US-Mexico border or at a major interchange.
45
Figure 4.11 Validation Results : SR-125 Northbound
Estimated volumes match observed counts well on non-toll section (north of SR-54 and SR-125
interchange) of Northbound SR-125. The model overestimates with a trend line slope value at 1.059
and a %RMSE at 11 percent. Among all links with counts, 89 percent of them have estimated volumes
within +-30 percent range of counts; 78 percent within +-20 percent range; 56 percent within +-10
percent range. There is only one notable outlier near SR-52 interchange with a 35 percent
overestimate. This is also a lower volume location that would be harder to match. Again, this outlier
is at a major interchange with complicated lane configurations. Further investigation of network
coding and/or count-to-link matching could help understand why there is a significant gap.
46
Figure 4.12 Validation Results : SR-125 Southbound
Estimated volumes match observed counts well on non-toll section of Southbound SR-125. The model
slightly overestimates with a trend line slope value at 1.048 and a %RMSE at 23 percent. Among all
links with counts, 83 percent of them have estimated volumes within +-30 percent range of counts;
83 percent within +-10 percent range; 67 percent within +-10 percent range. There is only one notable
outlier at I-8 interchange with an 87 percent overestimate. The SR-125 and I-8 interchange has one
of the most complicated lane configurations in the entire San Diego region, with merging and
splitting lanes, on and off ramps, and overflies. Again, further investigation of network coding and/or
count-to-link matching could help understand why there is a significant gap between estimated
volume and count.
47
Figure 4.13 Validation Results : SR-163 Northbound
Estimated volumes match observed counts well on Northbound SR-163. The model slightly
overestimates with a trend line slope value at 1.019 and a %RMSE at 17 percent. Among all links with
counts, 90 percent of them have estimated volumes within +-30 percent range of counts; 80 percent
within +-20 percent range; 60 percent within +-10 percent range. There is only one notable outlier at
I-8 interchange with a 37 percent underestimate. Again, this outlier is at a major interchange between
SR-163 and I-8 with complicated lane configurations.
48
Figure 4.14 Validation Results : SR-163 Southbound
Estimated volumes match observed counts very well on Southbound SR-163. The model slightly
underestimates with a trend line slope value at 0.994 and a %RMSE at 17 percent. Among all links
with counts, 100 percent of them have estimated volumes within +-10 percent range of counts; the
best validation results among all the main corridors. The largest gap between estimated volume and
observed count is only 6 percent, an outstanding validation result.
49
Figure 4.15 Validation Results : SR-67 Northbound
On Northbound SR-67, the model overestimates with a trend line slope value at 1.208 and a %RMSE
at 22 percent. Among all links with counts, 100 percent of them have estimated volumes within
+-30 percent range of counts; 75 percent within +-10 percent range; none within +-10 percent range.
Although there are no outliers, the model overestimates at all four count locations. Further
investigation of the travel behavioral differences among urban, suburban, and rural populations
could help understand why the model overestimates in suburban/rural areas like the areas along
SR-67 corridor.
50
Figure 4.16 Validation Results : SR-67 Southbound
On Southbound SR-67, the model overestimates with a trend line slope value at 1.2 and a %RMSE at
23 percent. Among all links with counts, 100 percent of them have estimated volumes within
+-30 percent range of counts; 50 percent within +-20 percent range; 50 percent within +-10 percent
range. Although there are no outliers, the model consistently overestimates at all four count
locations. Further investigation of the travel behavioral differences among urban, suburban, and rural
populations could help understand why the model overestimates in suburban/rural areas like the
areas along SR-67 corridor.
51
Figure 4.17 Validation Results : I-8 Westbound
Estimated volumes match observed counts well on Westbound I-8. The model slightly underestimates
with a trend line slope value at 0.981 and a %RMSE at 8 percent. Among all links with counts,
95 percent of them have estimated volumes within +-30 percent range of counts; 95 percent within
+-20 percent range; 88 percent within +-10 percent range. There are two notable outliers; one
between Midway Drive and Sunset Cliffs Boulevard with a 90 percent overestimate and another one
near SR-67 interchange. The count location between Midway Drive and Sunset Cliffs Boulevard is in
a unique location where I-8 ends, indicating a further investigation of network coding could help
understand why there is a significant gap.
52
Figure 4.18 Validation Results : I-8 Eastbound
Estimated volumes match observed counts well on Eastbound I-8. The model slightly underestimates
with a trend line slope value at 0.998 and a %RMSE at 14 percent. Among all links with counts,
90 percent of them have estimated volumes within +-30 percent range of counts; 85 percent within
+-20 percent range; 76 percent within +-10 percent range. There are two notable outliers at count
locations near I-5 and I-15 interchanges with overestimates of 45 percent and 47 percent respectively.
Both count locations are at major freeway interchanges with complicated lane configurations. Further
investigation of network coding and/or count-to-link matching could help understand why there are
significant gaps between estimated volumes and counts.
53
Figure 4.19 Validation Results : SR-52 Westbound
On Westbound SR-52, the model overestimates with a trend line slope value at 1.234 and a %RMSE
at 26 percent. Among all links with counts, 68 percent of them have estimated volumes within
+-30 percent range of counts; 37 percent within +-20 percent range; 11 percent within +-10 percent
range. There are two notable outliers, one at Mast Boulevard (51% overestimate) and another one
at Cuyamaca Street (38% overestimate). The model overestimates at almost all count locations,
particularly at the eastern end of the corridor. Further investigation of the demand model and land
use inputs, such as population and employment, along the SR-52 corridor could help understand why
the model overestimates.
54
Figure 4.20 Validation Results : SR-52 Eastbound
On Eastbound SR-52, the model overestimates with a trend line slope value at 1.197 and a %RMSE at
29 percent. Among all links with counts, 69 percent of them have estimated volumes within
+-30 percent range of counts; 25 percent within +-20 percent range; 19 percent within +-10 percent
range. There are three notable outliers at Santo Road (82% overestimate), Mast Boulevard (38%
overestimate) and Cuyamaca Street (37% overestimate). The model overestimates at count locations
east of SR-163. Further investigation of the demand model and land use inputs, such as population
and employment, along the SR-52 corridor could help understand why the model overestimates.
55
Figure 4.21 Validation Results : SR-54 Westbound
Estimated volumes match observed counts well on Westbound SR-54. The model underestimates with
a trend line slope value at 0.935 and a %RMSE at 8 percent. Among all links with counts, 100 percent
of them have estimated volumes within +-30 percent range of counts; 100 percent within +-20 percent
range; 80 percent within +-10 percent range. There are no outliers on this corridor. The largest gap
between estimated volume and count is 17 percent, an excellent validation result.
56
Figure 4.22 Validation Results : SR-54 Eastbound
Estimated volumes match observed counts well on Eastbound SR-54. The model slightly
underestimates with a trend line slope value at 0.967 and a %RMSE at 5 percent. Among all links with
counts, 100 percent of them have estimated volumes within +-10 percent range of counts. There are
no outliers on this corridor. The largest gap between estimated volume and observed is only
9 percent, an excellent validation result.
57
Figure 4.23 Validation Results : SR-56 Westbound
Estimated volumes match observed counts well on Westbound SR-56. The model overestimates with
a trend line slope value at 1.128 and a %RMSE at 14 percent. Among all links with counts, 100 percent
of them have estimated volumes within +-30 percent range of counts; 91 percent within +-20 percent
range; 18 percent within +-10 percent range. Although there are no outliers, the model overestimates
at all count locations. Further investigation of the demand model and land use inputs, such as
population and employment, along the SR-56 corridor could help understand why the model
overestimates.
58
Figure 4.24 Validation Results : SR-56 Eastbound
Estimated volumes match observed counts well on Eastbound SR-56. The model overestimates with a
trend line slope value at 1.13 and a %RMSE at 15 percent. Among all links with counts, 89 percent of
them have estimated volumes within +-30 percent range of counts; 89 percent within +-20 percent
range; 11 percent within +-10 percent range. Although there are no outliers, the model overestimates
at all count locations. Further investigation of the demand model and land use inputs, such as
population and employment, along the SR-56 corridor could help understand why the model
overestimates.
59
Figure 4.25 Validation Results : SR-78 Westbound
Estimated results match counts well on Westbound SR-78. The model slightly underestimates with a
trend line slope value at 0.955 and a %RMSE at 10 percent. Among all links with counts, 100 percent
of them have estimated volumes within +-30 percent range of counts; 96 percent within +-20 percent
range; 64 percent within +-10 percent range. There are no outliers on this corridor. Interestingly, the
model tends to underestimate slightly at the western section of the corridor (west of Melrose Road).
60
Figure 4.26 Validation Results : SR-78 Eastbound
Estimated results match counts well on Eastbound SR-78. The model slightly underestimates with a
trend line slope value at 0.997 and a %RMSE at 8 percent. Among all links with counts, 100 percent
of them have estimated volumes within +-30 percent range of counts; 100 percent within +-20 percent
range; 77 percent within +-10 percent range. There are no outliers on this corridor. The largest
underestimate is only 17 percent at the I-5 interchange, an excellent validation results.
61
Figure 4.27 Validation Results : SR-94 Westbound
Estimated results match counts well on Westbound SR-94. The model overestimates with a trend line
slope value at 1.052 and a %RMSE at 8 percent. Among all links with counts, 100 percent of them
have estimated volumes within +-30 percent range of counts; 100 percent within +-20 percent range;
62 percent within +-10 percent range. There are no outliers on this corridor. The largest
underestimate is 18 percent at 25th Street.
62
Figure 4.28 Validation Results : SR-94 Eastbound
Estimated results match counts well on Eastbound SR-94. The model overestimates with a trend line
slope value at 1.083 and a %RMSE at 11 percent. Among all links with counts, 100 percent of them
have estimated volumes within +-30 percent range of counts; 93 percent within +-20 percent range;
64 percent within +-10 percent range. There are no outliers on this corridor. The largest
underestimate is 26 percent at I-5 interchange.
63
Figure 4.29 Validation Results : SR-905 Westbound
There are only three count locations on Westbound SR-905. The model overestimates with a trend
line slope value at 1.133 and a %RMSE at 26 percent. There is one outlier near the Otay Mesa Port of
Entry (POE) between SR-125 and Siempre Viva Avenue with a 46 percent overestimate, which suggests
the significant overestimates could be caused by the crossborder model.
64
Figure 4.30 Validation Results : SR-905 Eastbound
There are only three count locations on Eastbound SR-905. The model overestimates with a trend line
slope value at 1.304 and a %RMSE at 64 percent. There are two outliers at Siempre Viva Avenue (86%
overestimate) and near SR-125 (127% overestimate) both near the Otay Mesa POE, which suggests
the significant overestimates could be caused by the crossborder model.
4.1.7: Roadway Validation by Corridor Direction
Table 4.8 is a summary of all corridor validation results by corridor direction. The model slightly
overestimates with an average trend line slope value at 1.043 and a %RMSE at 13.3 percent. Among
all highway corridor links with counts, 93 percent of them have estimated volumes within
+-30 percent range of counts; 85 percent within +-20 percent range; 58 percent within +-10 percent
range.
65
Table 4.8 Roadway Validation Results by Corridor Direction
# of Links within Gap Range % of Links within Gap Range
Gap Range NB SB WB EB all NB SB WB EB all
>=40% 3 4 3 5 15 2% 2% 2% 5% 2%
30%~40% 7 6 6 6 25 4% 3% 5% 5% 4%
20%~30% 9 13 9 7 38 5% 7% 7% 6% 6%
10%~20% 46 56 21 15 138 24% 28% 17% 14% 22%
0%~10% 71 67 40 38 216 37% 34% 31% 35% 35%
0%~-10% 41 39 36 29 145 22% 20% 28% 26% 23%
-10%~-20% 7 9 12 6 34 4% 5% 9% 5% 5%
-20%~-30% 3 3 0 3 9 2% 2% 0% 3% 1%
-30%~-40% 1 1 0 1 3 1% 1% 0% 1% 0%
<=-40% 2 1 0 0 3 1% 1% 0% 0% 0%
total 190 199 127 110 626 100% 100% 100% 100% 100%
-10%~10% 112 106 76 67 361 59% 53% 60% 61% 58%
-20%~20% 165 171 109 88 533 87% 86% 86% 80% 85%
-30%~30% 177 187 118 98 580 93% 94% 93% 89% 93%
Average of Gaps and Percent of (±) Links
Average of Gaps Percent of (±) Links
Average (+) gap 12% 13% 14% 16% 13% 72% 73% 62% 65% 69%
Average (-) gap -9% -7% -7% -8% -8% 28% 27% 38% 35% 31%
Average of all 6% 7% 6% 8% 7% 100% 100% 100% 100% 100%
% Root Mean Square Error (RMSE)
RMSE% 12.9% 13.3% 11.6% 14.7% 13.2%
Slope of Trend Line
Slope 1.044 1.060 1.009 1.027 1.043
The analysis of each direction is presented in Tables 4.9 to 4.12. Overall, East-West corridors performs
better than North-South corridors. Out of the 26 corridors (counting both directions), the model
underestimates on 6 corridors, I-8, SR-54, and SR-78, all East-West corridors; the model overestimates
on 20 corridors.
As shown in Table 4.9, estimated volumes match counts well on Northbound corridors. The model
slightly overestimates with an average trend line slope value at 1.044 and a %RMSE at 12.9 percent.
Among all links with counts, 94 percent of them have estimated volumes within +-30 percent range
of counts; 87 percent within +-20 percent range; 59 percent within +-10 percent range. Northbound
SR-67 is the outlier among all northbound corridors with a trend line slope at 1.208, representing a
significant overestimate.
66
Table 4.9 Roadway Validation Results : Northbound Corridors
# of Links by Gap Range
Gap Range I-15 I-5 I-805 SR-125 SR-163 SR-67 all
>=40% 1 0 2 0 0 0 3
30%~40% 2 2 1 1 0 0 6
20%~30% 3 0 3 1 1 1 9
10%~20% 22 13 5 1 2 3 46
0%~10% 13 34 15 4 5 0 71
0%~-10% 10 25 4 1 1 0 41
-10%~-20% 2 2 2 1 0 0 7
-20%~-30% 0 1 2 0 0 0 3
-30%~-40% 0 0 0 0 1 0 1
<-40% 0 2 0 0 0 0 2
total 53 79 34 9 10 4 189
% of Links by Gap Range
Gap Range I-15 I-5 I-805 SR-125 SR-163 SR-67 all
>=40% 2% 0% 6% 0% 0% 0% 2%
30%~40% 4% 3% 3% 11% 0% 0% 3%
20%~30% 6% 0% 9% 11% 10% 25% 5%
10%~20% 42% 16% 15% 11% 20% 75% 24%
0%~10% 25% 43% 44% 44% 50% 0% 38%
0%~-10% 19% 32% 12% 11% 10% 0% 22%
-10%~-20% 4% 3% 6% 11% 0% 0% 4%
-20%~-30% 0% 1% 6% 0% 0% 0% 2%
-30%~-40% 0% 0% 0% 0% 10% 0% 1%
<-40% 0% 3% 0% 0% 0% 0% 1%
total 100% 100% 100% 100% 100% 100% 100%
-10%~10% 43% 75% 56% 56% 60% 0% 59%
-20%~20% 89% 94% 76% 78% 80% 75% 87%
-30%~30% 94% 95% 91% 89% 90% 100% 94%
Average Gap & Percent of (±) Links
% of links with (+) gap 77% 62% 76% 78% 80% 100% 71%
% of links with (-) gap 23% 38% 24% 22% 20% 0% 29%
Ave (+) gap 13% 8% 13% 13% 11% 19% 12%
Ave (-) gap -5% -8% -12% -9% -18% N/A -9%
Ave of all 9% 2% 7% 8% 5% 19% 6%
% Root Mean Square Error (RMSE)
RMSE percent 13% 10% 16% 11% 17% 22% 12.9%
Slope of Trend Line
Slope 1.063 1.027 1.046 1.059 1.019 1.208 1.044
As shown in Table 4.10, estimated volumes match counts well on Southbound corridors. The model
overestimates with an average trend line slope value at 1.06 and a %RMSE at 13.3 percent. Among
all links with counts, 94 percent of them have estimated volumes within +-30 percent range of counts;
86 percent within +-20 percent range; 54 percent within +-10 percent range. Southbound SR-67 is the
67
outlier among all southbound corridors with a trend line slope at 1.2, representing a significant
overestimate. Table 4.10
Road Validation Results : Southbound Corridors
# of Links by Gap Range
Gap Range I-15 I-5 I-805 SR-125 SR-163 SR-67 all
>=40% 0 3 0 1 0 0 4
30%~40% 0 3 2 0 0 0 5
20%~30% 5 1 5 0 0 2 13
10%~20% 23 15 18 0 0 0 56
0%~10% 17 28 8 6 6 2 67
0%~-10% 5 21 5 2 6 0 39
-10%~-20% 2 4 1 2 0 0 9
-20%~-30% 0 3 0 0 0 0 3
-30%~-40% 0 0 0 1 0 0 1
<-40% 0 1 0 0 0 0 1
total 52 79 39 12 12 4 198
% of Links by Gap Range
Gap Range I-15 I-5 I-805 SR-125 SR-163 SR-67 all
>=40% 0% 4% 0% 8% 0% 0% 2%
30%~40% 0% 4% 5% 0% 0% 0% 3%
20%~30% 10% 1% 13% 0% 0% 50% 7%
10%~20% 44% 19% 46% 0% 0% 0% 28%
0%~10% 33% 35% 21% 50% 50% 50% 34%
0%~-10% 10% 27% 13% 17% 50% 0% 20%
-10%~-20% 4% 5% 3% 17% 0% 0% 5%
-20%~-30% 0% 4% 0% 0% 0% 0% 2%
-30%~-40% 0% 0% 0% 8% 0% 0% 1%
<-40% 0% 1% 0% 0% 0% 0% 1%
total 100% 100% 100% 100% 100% 100% 100%
-10%~10% 42% 62% 33% 67% 100% 50% 54%
-20%~20% 90% 86% 82% 83% 100% 50% 86%
-30%~30% 100% 91% 95% 83% 100% 100% 94%
Average Gap & Percent of (±) Links
% of links with (+) gap 87% 59% 85% 50% 50% 100% 71%
% of links with (-) gap 13% 35% 15% 42% 50% 0% 26%
Ave (+) gap 12% 12% 15% 16% 3% 16% 13%
Ave (-) gap -5% -8% -6% -12% -4% N/A -8%
Ave of all 10% 5% 12% 4% 0% 16% 7%
% Root Mean Square Error (RMSE)
RMSE percent 13% 13% 14% 23% 4% 23% 13.3%
Slope of Trend Line
Slope 1.084 1.031 1.089 1.048 0.994 1.200 1.060
As shown in Table 4.11, estimated volumes match counts well on Westbound corridors. The model
slightly overestimates with an average trend line slope value at 1.009 and a %RMSE at 11.6 percent.
Among all links with counts, 93 percent of them have estimated volumes within +-30 percent range
68
of counts; 86 percent within +-20 percent range; 60 percent within +-10 percent range. Westbound
SR-52 is the outlier among all westbound corridors with a trend line slope at 1.234, representing a
significant overestimate.
Table 4.11 Roadway Validation Results : Westbound Corridors
# of Links by Gap Range
Gap Range I-8 SR-52 SR-54 SR-56 SR-78 SR-905 SR-94 all
>=40% 1 1 0 0 0 1 0 3
30%~40% 1 5 0 0 0 0 0 6
20%~30% 0 6 0 1 1 1 0 9
10%~20% 0 5 0 8 0 0 8 21
0%~10% 17 1 0 2 10 0 10 40
0%~-10% 21 1 4 0 6 1 3 36
-10%~-20% 3 0 1 0 8 0 0 12
-20%~-30% 0 0 0 0 0 0 0 0
-30%~-40% 0 0 0 0 0 0 0 0
<-40% 0 0 0 0 0 0 0 0
total 43 19 5 11 25 3 21 127
% of Links by Gap Range
Gap Range I-8 SR-52 SR-54 SR-56 SR-78 SR-905 SR-94 all
>=40% 2% 5% 0% 0% 0% 33% 0% 2%
30%~40% 2% 26% 0% 0% 0% 0% 0% 5%
20%~30% 0% 32% 0% 9% 4% 33% 0% 7%
10%~20% 0% 26% 0% 73% 0% 0% 38% 17%
0%~10% 40% 5% 0% 18% 40% 0% 48% 31%
0%~-10% 49% 5% 80% 0% 24% 33% 14% 28%
-10%~-20% 7% 0% 20% 0% 32% 0% 0% 9%
-20%~-30% 0% 0% 0% 0% 0% 0% 0% 0%
-30%~-40% 0% 0% 0% 0% 0% 0% 0% 0%
<-40% 0% 0% 0% 0% 0% 0% 0% 0%
total 100% 100% 100% 100% 100% 100% 100% 100%
-10%~10% 88% 11% 80% 18% 64% 33% 62% 60%
-20%~20% 95% 37% 100% 91% 96% 33% 100% 86%
-30%~30% 95% 68% 100% 100% 100% 67% 100% 93%
Average Gap & Percent of (±) Links
% of links with (+) gap 44% 95% 0% 100% 44% 67% 86% 62%
% of links with (-) gap 56% 5% 100% 0% 56% 33% 14% 38%
Ave (+) gap 10% 25% N/A 14% 6% 37% 8% 14%
Ave (-) gap -5% -6% -7% N/A -11% -3% -4% -7%
Ave of all 2% 24% -7% 14% -4% 23% 6% 6%
% Root Mean Square Error (RMSE)
RMSE percent 8% 26% 8% 14% 10% 26% 8% 11.6%
Slope of Trend Line
Slope 0.981 1.234 0.935 1.128 0.955 1.133 1.052 1.009
69
As shown in Table 4.12, estimated volumes match counts well on Eastbound corridors. The model
slightly overestimates with an average trend line slope value at 1.027 and a %RMSE at 14.7 percent.
Among all links with counts, 89 percent of them have estimated volumes within +-30 percent range
of counts; 80 percent within +-20 percent range; 61 percent within +-10 percent range. Eastbound
SR-905 is the outlier among all eastbound corridors with a trend line slope at 1.304, representing a
significant overestimate. However, there are only three count locations on Eastbound SR-905,
statistically insufficient for drawing meaningful conclusions.
Table 4.12 Roadway Validation Results : Eastbound Corridors
# of Links by Gap Range
Gap Range I-8 SR-52 SR-54 SR-56 SR-78 SR-905 SR-94 all
>=40% 2 1 0 0 0 2 0 5
30%~40% 1 4 0 1 0 0 0 6
20%~30% 0 6 0 0 0 0 1 7
10%~20% 1 1 0 7 2 0 4 15
0%~10% 16 1 0 1 10 1 9 38
0%~-10% 15 2 5 0 7 0 0 29
-10%~-20% 3 0 0 0 3 0 0 6
-20%~-30% 2 1 0 0 0 0 0 3
-30%~-40% 1 0 0 0 0 0 0 1
<-40% 0 0 0 0 0 0 0 0
total 41 16 5 9 22 3 14 110
% of Links by Gap Range
Gap Range I-8 SR-52 SR-54 SR-56 SR-78 SR-905 SR-94 all
>=40% 5% 6% 0% 0% 0% 67% 0% 5%
30%~40% 2% 25% 0% 11% 0% 0% 0% 5%
20%~30% 0% 38% 0% 0% 0% 0% 7% 6%
10%~20% 2% 6% 0% 78% 9% 0% 29% 14%
0%~10% 39% 6% 0% 11% 45% 33% 64% 35%
0%~-10% 37% 13% 100% 0% 32% 0% 0% 26%
-10%~-20% 7% 0% 0% 0% 14% 0% 0% 5%
-20%~-30% 5% 6% 0% 0% 0% 0% 0% 3%
-30%~-40% 2% 0% 0% 0% 0% 0% 0% 1%
<-40% 0% 0% 0% 0% 0% 0% 0% 0%
total 100% 100% 100% 100% 100% 100% 100% 100%
-10%~10% 76% 19% 100% 11% 77% 33% 64% 61%
-20%~20% 85% 25% 100% 89% 100% 33% 93% 80%
-30%~30% 90% 69% 100% 89% 100% 33% 100% 89%
Average Gap & Percent of (±) Links
% of links with (+) gap 49% 81% 0% 100% 55% 100% 100% 65%
% of links with (-) gap 51% 19% 100% 0% 45% 0% 0% 35%
Ave (+) gap 11% 30% N/A 15% 6% 71% 9% 16%
Ave (-) gap -8% -12% -3% N/A -8% N/A N/A -8%
Ave of all 1% 22% -3% 15% 0% 71% 9% 8%
% Root Mean Square Error (RMSE)
RMSE percent 14% 29% 5% 15% 8% 64% 11% 14.7%
Slope of Trend Line
Slope 0.998 1.197 0.967 1.130 0.997 1.304 1.083 1.027
4.2: Transit Validation
For transit validation, estimated regional daily transit ridership was compared against 2012 observed
daily transit ridership from the SANDAG Passenger Count Program. Figure 4.31 shows the results of
70
three model versions, calibrated model with PopSyn II, uncalibrated model with PopSyn III, and
calibrated model with PopSyn III. The calibrated model with PopSyn III has the best performance with
a slight overestimate of 0.61 percent.
Figure 4.31 Transit Validation Results : Regional Transit Ridership
Additionally, estimated daily transit ridership by line haul mode was compared against 2012 observed
data. Table 4.13 and Figure 4.32 show transit validation results by line haul mode. The calibrated
model with PopSyn III has the best performance across all four modes: (1) commuter rail; (2) light rail;
(3) limited express bus; and (4) local bus, with gaps of 3.7 percent, -1.5 percent, 3.4 percent, and
1.7 percent respectively.
Table 4.13 Transit Validation Results by Line Haul Mode
Observed
Calibrated w/
PopSyn II
Uncalibrated w/
PopSyn III
Calibrated w/
PopSyn III
vol diff% vol diff% vol diff%
Commuter Rail 5,482 4,410 -19.6% 5,287 -3.6% 5,684 3.7%
Light Rail 126,861 115,834 -8.7% 116,188 -8.4% 124,966 -1.5%
Limited Express Bus 1,430 1,297 -9.3% 1,255 -12.2% 1,478 3.4%
Local Bus 229,169 234,877 2.5% 249,253 8.8% 233,018 1.7%
Total 362,942 356,417 -1.8% 371,983 2.5% 365,146 0.6%
362,942
356,417
371,983
365,146
-1.80%
2.49%
0.61%
-2.50%-2.00%-1.50%-1.00%-0.50%0.00%0.50%1.00%1.50%2.00%2.50%3.00%
345,000
350,000
355,000
360,000
365,000
370,000
375,000
Observed Model w/ PopSynII
Model w/ PopSynIII
Calibrated Modelw/ PopSyn III
Transit Ridership
Transit Ridership Diff%
71
Figure 4.32 Transit Validation Results by Line Haul Mode
4.3: Regional VMT Validation
Model estimated daily VMT was compared against observed VMT, factored to a weekday average
from a 7-day weekly average, from 2012 California Public Road Data. 8 Figure 4.33 shows VMT
validation results of the three model versions. The calibrated model with PopSyn III has the best
performance with a slight overestimate of 0.1 percent.
-19.55%
-8.69%
-9.30%
2.49%
-3.56%
-8.41%
-12.24%
8.76%
3.68%
-1.49%
3.36%
1.68%
-25.00% -20.00% -15.00% -10.00% -5.00% 0.00% 5.00% 10.00% 15.00%
Commuter Rail
Light Rail
Limited Express Bus
Local Bus
Estimated vs. Observed Transit Ridership by Line Haul Mode
Calibrated Model w/ PopSyn III Model w/ PopSyn III Model w/ PopSyn II
72
Figure 4.33 Regional VMT Validation Results
4.4: Comparisons of SANDAG Validation Results with FHWA Guidelines
Although there are no universal recommended validation standards in the FHWA validation
guideline4, there are validation standard examples used by various state DOTs and MPOs listed in the
guideline, including the Volume-Over-Count Ratio guidance used in Florida.
Table 4.14 Volume-Over-Count Ratios and Percent Error (Florida S ample6)
Standards Statistic Acceptable Preferable
Freeway Volume-over-Count (FT1x, FT8x, FT9x) +/- 7% +/- 6%
Divided Arterial Volume-over-Count (FT2x) +/- 15% +/- 10%
Undivided Arterial Volume-over-Count (FT3x) +/- 15% +/- 10%
Collector Volume-over-Count (FT4x) +/- 25% +/- 20%
One way/Frontage Road Volume-over-Count (FT6x) +/- 25% +/- 20%
Freeway Peak Volume-over-Count 75% of links @ +/-20% 50% of links @ +/-10%
Major Arterial Peak Volume-over-Count 75% of links @ +/-30% 50% of links @ +/-15%
Assigned VMT-over-Count Area wide +/-5% +/-2%
Assigned VHT-over-Count Area wide +/-5% +/-2%
Source: FSUTMS-Cube Framework Phase II, Model Calibration and Validation Standards: Model
Validation Guidelines and Standards, prepared for Florida Department of Transportation, prepared
by Cambridge Systematics, Inc., December 31, 2007.
79,434,621
79,289,103
79,036,085
79,513,965
-0.18%
-0.50%
0.10%
-0.60%
-0.50%
-0.40%
-0.30%
-0.20%
-0.10%
0.00%
0.10%
0.20%
78,700,000
78,800,000
78,900,000
79,000,000
79,100,000
79,200,000
79,300,000
79,400,000
79,500,000
79,600,000
Target Model w/ PopSyn II Model w/ PopSyn III Calibrated Model w/PopSyn III
Vehicle Miles Traveled
VMT Diff%
73
Similar validation measures are summarized for the SANDAG ABM in Table 4.15. The SANDAG model
outperforms the ‘preferable’ Florida example in the ‘Assigned VMT-over-Count Area wide’ category.
The SANDAG model performs on par with or outperforms the ‘acceptable’ Florida example in
‘Freeway Volume-over-Count,’ ‘Arterial Volume-over-Count,’ and ‘Collector Volume-over-Count’
categories.
Table 4.15 Volume-Over-Count Ratios and Percent Error (SANDAG 2012 Base Year Model)
Standards SANDAG 2012 Model Results
Freeway Volume-over-Count 7%
Arterial Volume-over-Count -8%
Collector Volume-over-Count -8%
Freeway Daily Volume-over-Count 57% of links @ +/-10%; 85% of links @ +/-20%;
Arterial Daily Volume-over-Count 51% of links @ +/-20%; 66% of links @ +/-30%;
Assigned VMT-over-Count Area wide 0.10%
There are example %RMSE by volume guidelines used by various states in the FHWA report,6 including
those used in Ohio, Florida, and Oregon (Figure 4.34)6. Similar %RMSE results were summarized from
the SANDAG ABM as shown in Figure 4.35.
Figure 4.34 %RMSE by Volume Examples in FHWA Validation Guideline6
74
Figure 4.35 %RMSE by Volume-SANDAG 2012 Base Year Model
The following conclusions can be made by comparing SANDAG %RMSE and the example %RMSE
guidelines in the FHWA report:
Similar to example %RMSE curves given in the FHWA report, the SANDAG %RMSEs decrease as
the volumes increase.
For volumes less than 40K, the SANDAG %RMSEs (10K at 37.7%, 20K at 31.5%, 30K at 28.0%, and
40K at 21.5%) are on par with the average of the FHWA examples, represented by the yellow
curve from Ohio.
For volumes between 50K and 80K, the SANDAG %RMSEs (50K at 17.8%, 60K at 14.4%, 70K at
15.2%, and 80K at 12.2% outperforms the average of FHWA examples.
For volumes larger than 90K, the SANDAG %RMSEs (all less than 10%) outperforms even the
strictest guideline, the Florida-Desirable example.
45.4%
37.7%
31.5%28.0%
21.5%17.8%
14.4%15.2%12.2%
8.7% 8.3% 8.8% 7.2% 8.4%3.6% 1.1%
4.4%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0 20000 40000 60000 80000 100000 120000 140000 160000
RM
SE%
Link Volume Group
75
CHAPTER 5: MILITARY TRAVEL CALIBRATION AND VALIDATION
The objective of this effort is to improve modeling of trips to/from military bases. In consultation with
people familiar with local traffic conditions, initial model estimated trips to/from military bases were
determined to be unsatisfactory. This can be explained as follows:
Unlike travel activities of civilians, travel activities of military personnel are likely contained within
the base the traveler works or lives in. Military bases usually provide housing and support facilities
such as restaurants, gas stations, schools, hospitals or clinics, and shopping and convenience stores
for military personnel. Travel behavior of military personnel might be quite different from those
of the civilians. Unfortunately, there were not enough military household and group quarter (GQ)
samples in the 2006 HHTS to allow the development of a special military personnel travel model.
Unlike destinations in civilian areas, military bases typically have controlled access due to security
reasons. Consequently, in the travel model, military and civilian zones should be treated
differently as location choice alternatives. Currently the SANDAG destination choice models do
not reflect the highly controlled access to military zones.
Lastly, there were hardly any military base traffic counts prior to this effort. The lack of counts
presents a challenge to calibrate and validate the model to observed ground conditions at the
military bases.
In this effort, staff reviewed and modified roadway network coding at military bases. The network
coding changes improved model results at both gate and base levels. One key limitation of this
exercise is the lack of a military travel model that reflects the unique military personnel travel
behavior. Fortunately, in the 2016 HHTS, military GQs will be oversampled to allow the development
of a model component in the updated ABM that reflects military personnel travel behavior.
5.1: Military Base Traffic Count8
In support of this task, a consultant was hired to conduct traffic counts at 10 military bases, including
most military installations in the San Diego region. Traffic counts at 48 gates were collected from
May 11, 2015, to May 21, 2015. Table 5.1 shows the participating bases.
Table 5.1 Military Bases Participated in Traffic Count9
Military Base Gate(s)
Marine Corps Base (MCB) Camp Pendleton 1-8
Marine Corps Recruit Depot (MCRD) 9-11
Marine Corps Air Station (MCAS) Miramar 12-15
Naval Base Point Loma (NBPL) 16-23
Naval Base San Diego (NBSD) 24-37
Naval Air Station (NAS) North Island 38-41
Naval Amphibious Base (NAB) Coronado 42-45
Silver Strand Training Complex (SSTC)-South 46
Naval Outlying Landing Field (NOLF) Imperial Beach 47
U.S. Coast Guard (USCG) San Diego 48
76
Support from the ten participating military bases is critical to the success of this effort. Members of
SANDAG MWG initiated the contact with the military bases. For each military base, points of contact
(POCs) were identified and coordination initiated to identify the appropriate traffic count locations
(i.e., gates). Each POC reviewed and provided feedback on each gate, including hours
(i.e., operational, peak, and queuing hours) and special events days/hours. Traffic count methods
(i.e., radar, tube, or manual counts) and military base access also were discussed. Manual counts were
performed for two hours during each peak period (i.e., a.m. and/or p.m. peak periods), utilizing peak
hours identified by the POCs or standard peak hours (i.e., 7:00 to 9:00 a.m. and 4:00 to 6:00 p.m.).
Data at these gates was obtained as follows: 19 gates used radar counters; 12 gates used tube
counters; and 17 gates used manual counts. Data was reviewed or modified as follows:
Where previous traffic studies were provided by POCs, the data was reviewed/validated.
Discrepancies were reviewed with the POCs, with differences potentially attributed to the
following: number of people deployed on the traffic count day and revised gate operations
(e.g., hours and lanes).
For manual counts, it is assumed that the peak hour volume of either the a.m. or p.m. peak hour
(whichever was greater) represents 20 percent of the average daily traffic volume.
5.2: Roadway Network Adjustments
The coding of the roadway network was an integral part of completing the military trip validation
task. The importance of proper network coding exists in this case because ground counts for the
military bases were performed at every specific gate location that connected to the civilian roadway
network.
Roadway network edits for military base validation require a specific procedure in order to minimize
the number of model runs. The procedure was approached in the following critical path order:
1. Validate gate locations from ground count results.
2. Ensure all gates have proper connections to the civilian roadway network.
3. Determine the required internal base network; in order to ensure proper connectivity to
gates.
4. Adjust TAZ connector lengths from TAZ centroid to internal base network.
5. Apply speed adjustments, if necessary, on internal base network or driveways to properly
distribute volume.
Performing these steps for each military base in the San Diego region ensured that the roadway
network was coded in the best possible manner to match ground counts.
5.2.1: Gate Location Validation
The validation of gate locations and the physical placement of vehicle count equipment was the first
step taken. This was performed to the extent that the counts collected could be used to compare to
model results. At three separate locations on regional military bases, complications with the count
77
data were discovered. The three bases affected were: (1) Camp Pendleton; (2) Bob Wilson Naval
Hospital; and (3) 32nd Street Naval Base.
At Camp Pendleton, aerial imagery showed that gate No. 8 functioned as an internal access control
gate, and not an access point to the civilian roadway network. Further follow up revealed that gate
No. 8 was a control point between Camp Pendleton and Naval Weapons Station. While the counts
for gate No. 8 provide valuable insight to internal base travel, it was omitted from further
transportation model roadway network validation.
At Bob Wilson Naval Hospital, it was discovered that while gate No. 37 was an actual access point to
the facility, it was limited to pedestrian and ambulances only. The counts collected for gate No. 37
were determined to be from a shared driveway that allowed both access to ambulances and civilian
access to adjacent Balboa Park uses. The configuration at this location can be viewed in Figure 5.1.
Counts for gate No. 37 were omitted from further transportation model roadway network validation.
Figure 5.1* Count & Driveway Configuration at Gate 37
*Image source: Google.com
At 32nd Street Naval Base, aerial imagery showed that a gate was omitted from the ground count
effort. The gate is located at 32nd Street and Norman Scott Road. While the lack of a count for this
gate is of some concern, it did not affect validation results in a deleterious manner.
78
5.2.2: Gate Connections to the Civilian Roadway Network
This component of network validation was the most valuable. The branches of the military that
operate in the San Diego region have complete control over access points to bases and facilities. Over
time, these access points have changed in location and operation. Most of these changes were not
reflected in the roadway network for the transportation model. The best example of the importance
of proper gate location coding can be seen in Figure 5.2, where, before making edits, had four
incorrectly coded gate locations.
Figure 5.2* Ariel Image of 32nd Street Naval Base
*Image source: Google.com
After review, all base gates and the connections to the civilian roadway network were properly coded.
Making sure that the proper access points from the civilian network to base gates was required in
order to proceed to the next three steps.
79
5.2.3: Internal Base Networks
After the gate locations were validated, the next step was to evaluate if there was a need to add a
portion of a military base’s internal road network. Portions of an internal base roadway network may
be required to ensure that trips from military base TAZs have viable paths for traffic assignment to
load onto the civilian roadway network. For military bases that have their entire perimeter within
one TAZ or that offer limited points of access to gates from the civilian roadway network, an internal
network is not required. For larger bases that have multiple points of access spread out over a wide
geographical area along their perimeter.
For every military base except Camp Pendleton, it was determined that proper coding of gate
driveways and TAZ zone connectors eliminated the need to code internal military base networks. For
modeling purposes, adding roadway facilities within an access controlled area should be kept to an
absolute minimum. Figure 5.3 shows how the original internal Camp Pendleton network used in
previous versions of ABM and the 4-step model was limited to a few roadway features.
Figure 5.3 USMC Camp Pendleton Internal Base Network Used in Previous Modeling E fforts
Camp Pendleton has seven perimeter gates distributed on a perimeter of over 50 miles. Because of
the scale of this base, adding an internal base roadway network was required to ensure proper traffic
assignment. One requirement of the internal base network was that it could not allow for “pass thru”
80
traffic. A path that would allow for “pass thru” assignment in ABM would be problematic because
current network attributes do not allow for non-military trip purposes to be excluded from accessing
the base. This is applicable for Camp Pendleton because a viable parallel path to SR-76 exists by
accessing the internal Camp Pendleton road network. The internal base network was limited to
adding features in the ABM roadway network to allow traffic assignment to distribute trips from
zone connectors. Figure 5.4 shows the new configuration for Camp Pendleton with added internal
base network and modified zone centroid connectors.
Figure 5.4 USMC Camp Pendleton After Roadway & Zone Centroid Revis ions
5.2.4: Zone Connector Configuration
ABM assigns auto traffic onto the roadway network from TAZ centroid connectors. The sensitivity of
zone connectors for validation is only applicable in cases where one TAZ has more than one
connection to the roadway network. The best example of this was at the USMC Recruit Depot and
can be seen in Figure 5.5. This military base is all contained in one TAZ and has 3 gates. The coding
for this base requires that each gate have its own centroid connector.
81
Figure 5.5 USMC Recruit Depot Zone Centroid Coding Used in Previous Modeling Efforts
In these cases of multiple zone connectors, the sensitivity of trip assignment is based on the length of
the feature and the upstream impedance of the roadway link. Requirements of coding multiple zone
connectors for the same TAZ are that the A-node must be shared and the A-node must be within the
TAZ boundary. After comparing the gate counts for USMC Recruit Depot to the assigned volumes
from ABM, it was determined that the zone centroids could be moved in order to allow ABM traffic
assignment to better match the gate counts. Figure 5.6 illustrates how the zone connectors were
moved and still share the same common A-node.
82
Figure 5.6 USMC Recruit Depot Zone Centroid Coding After Editing
For some TAZs, the requirements may not allow for ideal coding of zone connectors. The unique
nature of military bases did limit zone connector coding flexibility. This condition set up the need for
one additional step of coding.
5.2.5: Speed Adjustments on Base Driveways or Networks
Speed adjustments on base driveways or internal roadway networks were the last tasks performed to
adjust trip assignment. Speed adjustments are the final, network based measure that can be taken to
calibrate traffic assignment. Imputing a higher or lower travel speed on a roadway link affects the
impedance of auto trips being assigned from the military base zone connectors. It is important to
note that Imputed speed adjustments should be considered an optional final step, after efforts from
the first four steps have been performed.
An illustrated example of speed adjustment can be seen at Naval Air Station North Island. After the
initial steps of network coding were performed, it was determined that the assigned volumes at the
north and main gates were slightly low while the assigned volume at the south gate was too high.
To allow for more volume to be assigned to the main and north gates, a low driveway speed was
coded to the south gate in order to lower the utility on the path from the gate. Driveway speed was
reduced to 15 mph from the originally coded 25 mph. This small change was enough to bring all four
gate volumes within acceptable proportion for the base.
83
Figure 5.7 Speed Adjustment at NAS North Is land South Gate
5.2.6: Summary Matrix for Roadway Network Adjustments
Figure 5.8 indicates what measures were undertaken to adjust roadway coding of the gate locations.
A solid circle in the figure indicates the corresponding measure is applied.
84
Figure 5.8 Roadway Network Adjustments Matrix
● – Network edit method used at this gate location ○ - Network edit method not used at this gate location
85
● – Network edit method used at this gate location ○ - Network edit method not used at this gate location
86
5.3: Military Travel Calibration and Validation Results
The military calibration and validation were performed at base and gate levels.
5.3.1: Validation by Military Base
For base-level validation, the sum of estimated trip to/from each base was compared against the sum
of all gate counts. Silver Strand Training Complex (SSTC) is removed from the validation list because
no housing and employment was coded on the base and the count is low (500 daily). Additionally,
Point Loma Naval Mine and Anti-Submarine and VA Balboa Hospital were separated out from NBPL
and NBSD respectively because they are not adjacent to the rest of the bases. Table 5.2 shows the
final 11 installations used in validation. Table 5.3 and Figure 5.9 present installation-level validation
summaries.
Table 5.2 Validation Results by Military Installation
Base Name Count
Scenario 227 Scenario 540
Estimated diff% Estimated diff%
Marine Corps Base (MCB) Camp Pendleton 76,349 136,786 79% 99,940 31%
Marine Corps Air Station (MCAS) Miramar 35,025 43,768 25% 31,647 -10%
Marine Corps Recruit Depot (MCRD) 19,075 13,079 -31% 16,708 -12%
US Coast Guard (USCG) San Diego 1,925 2,720 41% 2,230 16%
Naval Air Station (NAS) North Island 51,717 34,160 -34% 49,398 -4%
Naval Base Point Loma (NBPL) 22,992 14,961 -35% 21,906 -5%
Naval Base San Diego (NBSD) 64,395 110,756 72% 70,061 9%
Naval Amphibious Base (NAB) Coronado 21,097 23,441 11% 20,472 -3%
Naval Outlying Landing Field (NOLF) Imperial Beach 4,449 6,978 57% 6,789 53%
Point Loma Naval Mine and Anti-Submarine 6,200 8,283 34% 5,826 -6%
VA/Balboa Hospital 11,845 11,962 1% 12,283 4%
Table 5.3 Summary of Validation Results -by Installation
% Diff b/w Estimated and Counts Scenario 227 Scenario 540
within -+10% 1 7
within -+20% 2 9
>20% or <-20% 9 2
Total 11 11
87
Figure 5.9 % Validation Results -By Installation
The number of installations within +-10 percent estimated-count gap range increased from 1 in the
uncalibrated model to 7 in the calibrated model; the number of installations within +-20 percent gap
range increased from 2 to 9; the number of installations with gaps greater than 20 percent or less
than -20 percent reduced from 9 to 2. Although the 53 percent overestimate at NOLF Imperial Beach
is significant, the count at NOLF is relatively small (4000 daily); therefore, the gap is not as concerning.
MCB Camp Pendleton results have been improved significantly with base-level overestimate reduced
from 79 percent to 31 percent. However, the 31 percent overestimate is still significant; further
investigation of land use inputs, such as housing, GQs and employment, could help understand why
large amount of trips were generated from Camp Pendleton.
88
5.3.2: Validation by Gate
Validation was also performed at gate level by comparing each gate count to estimated volume on
corresponding link. Not all 48 gates were used for validation for following reasons:
Some gates are located at minor streets not represented in the highway network (e.g., Tuglia
Road at Naval Amphibious Base Coronado).
Some TAZs cover both civilian and military areas. Estimated trips to/from these TAZs represent
mixed civilian and military trips, which are not comparable with gate counts that represent
exclusive traffic to/from access controlled military areas.
At the end, 30 gates were used for validation. Table 5.4 and Figure 5.10 show gate-level validation
results. Model results in the calibrated model improved at 25 out of the 30 gates. The average gap
between estimated and count at all gates was reduced from 28 percent to 8 percent.
89
Table 5.4 Validation Results by Gate
Gate Base Route Location Count Uncalibrated Scen
227 w/ PopSyn III diff%
Calibrated Scen
540 w/ PopSyn III diff%
1 Marine Corps Base Camp Pendleton VANDEGRIFT BLVD N of SAN RAFAEL DR 31,899 36,508 14% 41,160 29%
2 Marine Corps Base Camp Pendleton VANDEGRIFT BLVD N of PAPAGALLO DR 15,771 41,069 160% 25,398 61%
3 Marine Corps Base Camp Pendleton BASILONE RD N of I-5 6,962 14,180 104% 12,400 78%
4 Marine Corps Base Camp Pendleton LAS PULGAS RD N of I-5 5,192 31,978 516% 3,709 -29%
5 Marine Corps Base Camp Pendleton AMMUNITION RD Btwn SPARROW RD & ALTURAS RD 11,121 405 -96% 14,453 30%
9 Marine Corps Recruit Depot San Diego HENDERSON AVE S of BARNETT AVE 7,130 1,555 -78% 2,731 -62%
10 Marine Corps Recruit Depot San Diego TRIPOLI AVE S of PACIFIC HWY 10,145 9,198 -9% 12,887 27%
11 Marine Corps Recruit Depot San Diego WASHINGTON ST E of GUANTANAMO ST 1,800 75 -96% 1,090 -39%
12 Marine Corps Air Station Miramar MIRAMAR WAY W of KEARNY VILLA RD 17,187 21,020 22% 12,138 -29%
13 Marine Corps Air Station Miramar MITSCHER WAY S of MIRAMAR RD 15,482 13,150 -15% 14,290 -8%
16 Naval Base Point Loma ROSECRANS ST N of GATE RD 8,915 5,738 -36% 11,900 33%
18, 19 Naval Base Point Loma CATALINA BLVD N of ELECTRON DR 14,077 1,620 -88% 10,342 -27%
24 Naval Base San Diego (32nd St) WARD RD S of E HARBOR DR 7,000 2,134 -70% 5,441 -22%
25 Naval Base San Diego (32nd St) S 32ND ST S of E HARBOR DR 11,100 15,356 38% 11,976 8%
26 Naval Base San Diego (32nd St) VESTA ST S of E HARBOR DR 7,185 12,337 72% 7,740 8%
27 Naval Base San Diego (32nd St) W 8TH ST W of E HARBOR DR 6,780 11,180 65% 11,724 73%
28 Naval Base San Diego (32nd St) COLTON AVE E of S 28TH ST 4,840 7,661 58% 4,021 -17%
29 Naval Base San Diego (32nd St) CALLAGAN HWY W of S 32ND ST 5,795 9,639 66% 7,099 23%
30 Naval Base San Diego (32nd St) VESTA ST S of MAIN ST 6,755 3,419 -49% 5,753 -15%
31 Naval Base San Diego (32nd St) W DIVISION ST S of SB I-5 ON-RAMP 8,715 13,840 59% 11,582 33%
33 Naval Medical Center San Diego BOB WILSON DR W of FLORIDA DR 9,415 7,262 -23% 8,393 -11%
38 North Island Air Station STOCKDALE RD W of ALAMEDA BLVD 13,772 1,407 -90% 20,333 48%
39 North Island Air Station MCCAIN BLVD W of ALAMEDA BLVD 18,230 25,892 42% 15,683 -14%
40 North Island Air Station QUAY RD W of ALAMEDA BLVD 14,771 29 -100% 8,085 -45%
41 North Island Air Station OCEAN BLVD E of S O ST 4,944 7,920 60% 5,296 7%
43 Naval Amphibious Base Coronado GUADALCANAL RD E of STRAND WAY 11,145 20,591 85% 10,215 -8%
44 Naval Amphibious Base Coronado TARAWA RD W of SR75 7,453 14,632 96% 10,256 38%
47 Naval Outlying Field Imperial Beach TOWER RD W of 13TH ST 4,449 6,742 52% 6,789 53%
48 US Coast Guard Station San Diego US COAST GUARD GATE S of HARBOR DR 1,925 2,720 41% 2,230 16%
Average 28% 8%
90
Figure 5.10 Validation Results by Gate
-200% -100% 0% 100% 200% 300% 400% 500% 600%
1_VANDEGRIFT BLVD
2_VANDEGRIFT BLVD
3_BASILONE RD
4_LAS PULGAS RD
5_AMMUNITION RD
9_HENDERSON AVE
10_TRIPOLI AVE
11_WASHINGTON ST
12_MIRAMAR WAY
13_MITSCHER WAY
16_ROSECRANS ST
18,19_CATALINA BLVD
24_WARD RD
25_S 32ND ST
26_VESTA ST
27_W 8TH ST
28_COLTON AVE
29_CALLAGAN HWY
30_VESTA ST
31_W DIVISION ST
33_BOB WILSON DR
38_STOCKDALE RD
39_MCCAIN BLVD
40_QUAY RD
41_OCEAN BLVD
43_GUADALCANAL RD
44_TARAWA RD
47_TOWER RD
48_US COAST GUARD GATE
%Difference b/w Model Estimated Trips and Gate Counts
Calibrated Model w/ PopSyn III Model w/ PopSyn III
91
Although base-level validation results improved significantly, the 31 percent overestimate at MCB
Camp Pendleton deems further investigation. At gate level, the following gates miss counts by over
60 percent:
Gate 2 at MCB Camp Pendleton (Vandegrift Blvd. north of Papagallo Drive)
Gate 3 at MCB Camp Pendleton (Basilone Road north of I-5)
Gate 9 at MCRD (Henderson Avenue south of Barnett Avenue)
Gate 27 at NBSD (West 8th Street west of East Harbor Drive)
The potential causes and improvements of these outliers are explained as follows:
Lack of understanding of military households’ and group quarters’ travel behavior.
Population and land use inputs need further improvements on military zones. Because the ABM
is a disaggregate simulation model driven by population and employment, significant
overestimations or underestimations of trips to/from a TAZ often indicate the population and
employment coded on the TAZ don’t have the capacity to generate and/or attract enough trips
as the count suggests.
For mixed-use TAZs, estimated trips to/from these TAZs represent mixed civilian and military trips,
which are not comparable with gate counts that represent exclusive traffic to/from access
controlled military areas. The delineations of military bases and TAZ boundaries need further
improvements to make them consistent.
Further improvement of transportation network coding could also help improve estimated
volume at each gate.
As SANDAG moves forward preparing for the next RTP, some efforts are already underway to improve
military travel modeling. The 2016 HHTS will include an oversample of military GQs, allowing an ABM
update to reflect unique travel behavior of military GQs. In forecast series 14 (SR14), land use inputs
at military bases will be updated and carefully reviewed, hence potentially correcting the
underestimation or overestimation issues at some bases. Additionally, some of the TAZ delineation
and network coding issues will also be addressed in SR14.
92
CHAPTER 6: CONCLUSIONS AND FUTURE WORK
The overall highway and transit assignment results were improved over the 2012 calibration and
validation with Synthetic Population III. The following conclusions can be made for calibration results:
There were three prior calibration and validation efforts for base years 20081, 20102, and 20123.
The effort described in this document is necessary for incorporating changes since the last
calibration and validation. Among all the changes, the transition from PopSyn II to PopSyn III has
the most significant impact on model results.
Auto ownership model was calibrated using updated targets summarized from the 2010-2014
ACS release; Estimated results match targets almost spot-on for all auto ownership categories.
The objectives of trip and tour mode choice calibrations are:
o Calibrate work trip mode choices using updated targets summarized from 2010-2014 ACS
release
o Calibrate transit mode choices using adjusted 2012 transit boardings
o Calibrate shared ride modes to match observed traffic volumes on managed lanes
Estimated work trips mode shares match updated targets well; Estimated regional transit
boardings match adjusted boardings targets by line haul mode well; Although the significantly
overestimated volumes on MLs were improved, the model still overestimates volumes on MLs.
Calibration of crossborder model reduced Mexican resident trips that end in military installations.
The following conclusions can be made for validation results:
The quality and quantity of counts improved after staff’s review and cleaning of counts and
count-to-link matchings. The total number of counts increased from 20613 to 2415; PeMS counts
increased from 167 to 918; Local jurisdiction counts decreased from 1602 to 1181 after cleaning.
Overall validation results are satisfactory with no systematic deviation from the 45-degree line in
Figure 4.2 scatter plot. The slope of the regressed line is 1.02, which represents a slight
overestimation. The R-squared value is 0.959, which represents a good fit between the estimated
volumes and the linear regression line.
Estimated regional VMT matches observed VMT (2012 California Public Road Data7) well with a
slight underestimate of 0.21 percent. Estimated regional transit boardings matches observed
boardings (SANDAG Passenger Count Program) well with a slight 0.6 percent overestimate.
Validation by MSA shows that the results from the 20081 and current validation efforts are
comparable at regional level. Results of current effort fare better at individual MSA level,
especially East County MSA.
Validation by road type shows freeway results fare better than those of other road types with a
regressed line slope of 1.043, representing a slight overestimation. The model tends to
underestimate volumes on arterials, ramps, and collectors, with line slopes at 0.875, 0.93, and
0.755 respectively. The lack of a systematic approach of collecting traffic counts on arterials and
93
collectors could be a contributing factor to the less than ideal performances on arterials and
collectors.
Validation by count data source shows that estimated volumes match PeMS and Caltrans
District 11 counts well; Estimated volumes tend to be smaller than observed on links with local
jurisdiction counts.
Validation by volume shows that the larger estimated link volumes are the better they match the
counts; %RMSEs decrease as the estimated volumes increase. The validation results by volume are
on a par with or outperforms the examples given in the FHWA calibration guideline.6
The 2 percent average gap between estimated and count at 26 (counting both directions) key
locations suggests the model slightly overestimates at key count locations; 70 percent of the
estimated volumes are within +-15 percent range of counts. There are four outliers at location
9 (SR-52 at Mast Blvd), location 10 (SR-52 at Convoy Blvd), location 12 (I-805 at San Ysidro Blvd.)
and location 13 (I-5 at I-805 South Junction).
Validation was performed on 26 corridors (counting both directions): I-5, I-15, I-805, SR-67, SR-125,
SR-163, I-8, SR-52, SR-54, SR-56, SR-78, SR-94, and SR-905. Overall the model performs well at
corridor level. Only 3.2 percent of links (20 out of the 618 links) are identified as outliers. Many
of these outliers are at major freeway interchanges with complex configurations of lane merging,
splitting, and HOV only traffic. There are also a few outliers near the U.S.-Mexico border where
I-5 and I-805 merges.
Validation by corridor direction shows that East-West corridors perform slightly better than
North-South corridors. Out of the 26 corridors, the model underestimates on 6 corridors, all of
which are East-West corridors: I-8, SR-54, and SR-78.
The following conclusions can be made for military travel calibration and validation:
In support of military travel calibration and validation, traffic counts were collected at 48 gates
from 10 military bases. These counts provide a solid foundation for military travel calibration and
validation and is a key achievement in this effort.
The coding of the roadway network was an integral part of completing the military trip validation
task. Roadway network edits for military base validation require a specific procedure in order to
minimize the number of model runs. The procedure was approached in the following critical path
order.
o Validate gate locations from ground count results.
o Ensure all gates have proper connections to the civilian roadway network.
o Determine the required internal base network; in order to ensure proper connectivity to
gates.
o Adjust TAZ connector lengths from TAZ centroid to internal base network.
o Apply speed adjustments, if necessary, on internal base network or driveways to properly
distribute volume.
Military travel calibration and validation show improved results of trips to/from military bases.
o Among the 11 military installations, the number of installations within -+10 percent gap
between estimated trips and counts increased from 1 to 7; the number of installations within
94
-+20 percent gap increased from 2 to 9; the number of installations with gaps greater than
20 percent or less than -20 percent reduced from 9 to 2.
o The gate level validation shows significantly improved results. Compared with the
uncalibrated model, 25 out of the 30 gates match counts better in the calibrated model. The
average gap between model estimated and counts at gate-level was reduced from 28 percent
to 8 percent.
One key limitation of the military calibration and validation effort is the lack of military personnel
survey. This limitation disallows a systematic analysis to understand explicitly how military
personnel travel. Fortunately, in the coming 2016 HHTS, the unique military personnel travel
behavior patterns will be captured by oversampling of military GQs.
The validation analysis has indicated the following areas as recommendations for further work.
Investigate the 20 outliers in the corridor analysis case by case. The majority of these outliers occur
either at major freeway interchanges with complex lane configurations. This suggests network
coding and count-to-link matching at interchanges should be investigation focuses.
There are a few outliers at locations near the U.S.-Mexico border where I-5 and I-805 merges. The
model overestimates on I-805 while it underestimates on I-5 in this area. This suggests the
investigation should focus on network coding and its impact on splitting traffics between I-5 and
I-805. Additionally, the crossborder model and its impact on the I-5 and I-805 traffic near the
border should be another focus.
Validation effort described in this document help establish model credibility for regional and
corridor-level planning purposes. For project-level planning purposes, further validation is
recommended to validate estimated volumes and speeds against observed data by time-of-day to
establish confidence in model’s performance during peak hours.
A major concern for validation of travel models is error inherent in the collection of count data
used for validation. Problems with count data or validation data can lead to erroneous corrections
to models that, ultimately, will damage model performance, credibility, and results.6 Thus, a good
course of action is to check for errors in count and other validation data. A systematic error
checking and cleaning is especially recommended for arterial counts.
Corridor level validation shows consistent overestimates on SR-52 on SR-67. Further work is
recommended to investigate population, employment, and land use inputs along SR-52 and SR-67
corridors to understand why the model consistently overestimates on these corridors.
A key limitation of this calibration and validation effort is the lack of an up-to-date household
travel behavior survey. Fortunately, the new San Diego 2016 HHTS is underway and the data will
be available in June 2017. A complete ABM update will start once the 2016 HHTS data is available.
95
References
1 Activity-Based Travel Model Validation for 2008: Coordinated Travel-Regional Activity Based Modeling Platform (CT-RAMP) for San Diego County; SANDAG, December, 2012.
2 Activity-Based Travel Model Validation for 2010 Using Series 13 Data: Coordinated Travel-Regional Activity Based Modeling Platform (CT-RAMP) for San Diego County; SANDAG, September, 2013.
3 Activity-Based Travel Model Validation for 2012 Using Series 13 Data: Coordinated Travel-Regional Activity Based Modeling Platform (CT-RAMP) for San Diego County; SANDAG, November, 2013.
4 American Community Survey (ACS). United States Census Bureau website www.census.gov/programs-surveys/acs/.
5 Peter Vovsha, Parsons Brinckerhoff, etc. New Features of Population Synthesis, presentation at the 94th Annual Meeting, January 2015.
6 Travel Model Validation and Reasonableness Checking Manual; 2nd Edition, The Travel Model Improvement Program (TMIP), FHWA, May 2010.
7 George Box, Professor Emeritus of Statistics, University of Wisconsin, as quoted in Project Traffic Forecasting, NCHRP 255 Review.
8 2012 California Public Road Data, Statistical Information Derived from the Highway Performance Monitoring System, Caltrans Division of Research, Innovation, and System Information.
9 Military Base Traffic Counts Memo, prepared by Parsons Brinkerhoff for SANDAG, 2015.