18
Evaluation of Travel Information and Prediction System (TIPS) for Work Zone Operations University of Maryland, College Park
Evaluation of Travel Information and Prediction System (TIPS) for Work Zone Operations
University of Maryland, College Park
2
TIPS overview
Traffic Information and Prediction System
• The messages displayed on these three PCMS are the range of travel time from each spot to Spot 5 (I-695 Gore) (e.g., “12 TO 16 MIN TO I-695”).
Spot 1
Spot 2 Spot 3
Spot 4 Spot 5
4.55 miles
7.55 miles
9.75 miles
10.80 miles
PCMS 1
PCMS 2 PCMS 3
SensorTrailer
I-695Gore
3
Criteria of TIPS Evaluation
System Accuracy and Reliability
Data collection for evaluation: Testing vehicle method
• TIPS Accuracy : comparing the actual measured travel times to those displayed from each PCMS.
• TIPS Reliability : consistency between the travel time information reported on the website and those displayed on each PCMS.
4
System Accuracy Evaluation on 11/14
System Accuracy
(Note: “No message” data on PCMS #1 were excluded from the sample size)
PCMS #1 PCMS #2 PCMS #3
LB UB LB UB LB UB
1750 vph
2400 vph
2200 vph
4250 vph
1650 vph
2650 vph
# of incorrect 9* 8 7
# of correct 6* 27 27System accuracy
(%) 40.00 77.14 79.41
• The accuracy drops as PCMS #1 is far away from I-695.
Volume difference from Sensor data
5
System Accuracy Evaluation on 11/14
System Accuracy
• Statistical analysis of predicted travel time errors PCMS #1 PCMS #2 PCMS #3
Average (seconds) 149.11 38.75 206.29
Standard deviation 415.05 308.11 181.70
Deviation of predicted travel time error (sec)
Frequency
PCMS #1 PCMS #2 PCMS #3
<= -480 0 1 0
<= -240 2 0 0
<= 0 2 2 1
<= 240 1 2 3
<= 480 1 3 3
> 480 3 0 0
Total 9 8 7
• Frequency distribution of predicted travel time errors
6
System Accuracy Evaluation on 11/19
System Accuracy
11
PCMS #1 PCMS #2 PCMS #3
LB UB LB UB LB UB
1900 vph
5000 vph
2100 vph
4100 vph
1100 vph
2400 vph
# of incorrect 8 5 4
# of correct 14 15System accuracy
(%) 57.89 73.68 78.95
• The volume difference from the sensor data at PCMS #1 is greater than those at PCMS #2 and #3
• The accuracy drops as PCMS #1 is far away from I-695 and the volume difference increases.
Volume difference from Sensor data
7
System Accuracy Evaluation on 11/19
System Accuracy
PCMS #1 PCMS #2 PCMS #3
Average (seconds) -21.00 85.40 87.25
Standard deviation 315.37 268.95 401.38
Deviation of predicted travel time error (sec)
Frequency
PCMS #1 PCMS #2 PCMS #3
<= -480 0 0 1
<= -240 1 0 0
<= 0 4 2 0
<= 240 1 2 2
<= 480 1 1 1
> 480 1 0 0
Total 8 5 4
• Statistical analysis of predicted travel time errors
• Frequency distribution of predicted travel time errors
8
System Accuracy Evaluation on 11/24
System Accuracy
• The LB and UB of volumes are low and their differences are small, which mean that traffic conditions are uncongested and stable.
• The system can achieve a high accuracy under the stable traffic flow patterns.
21
PCMS #1 PCMS #2 PCMS #3
LB UB LB UB LB UB
1550 vph
2450 vph
1200vph
2400vph
1000vph
2300vph
# of incorrect 0 0 2
# of correct 21 19
System accuracy (%)
100.00 100.00 90.48
Volume difference from Sensor data
9
System Accuracy Evaluation on 11/24
System Accuracy
PCMS #1 PCMS #2 PCMS #3Average (seconds) N/A N/A 148.00Standard deviation N/A N/A 16.97
• Statistical analysis of predicted travel time errors
• Frequency distribution of predicted travel time errorsDeviation of
predicted travel time error (sec)
Frequency
PCMS #1 PCMS #2 PCMS #3
<= -480 0 0 0
<= -240 0 0 0
<= 0 0 0 0
<= 240 0 0 2
<= 480 0 0 0
> 480 0 0 0
Total 0 0 2
10
System Accuracy Evaluation on 11/25
System Accuracy
2
PCMS #1 PCMS #2 PCMS #3
LB UB LB UB LB UB
600 vph
3750 vph
1000 vph
4300 vph
1500 vph
2750 vph
# of incorrect 29 12 6
# of correct 19 25System accuracy
(%) 6.45 61.29 80.65
Volume difference from Sensor data
• The performance of such a system clearly varies with the range of flow rate variation.
• PCMS #1 experiences the largest range of flow variation, and yields the poorest results.
11
System Accuracy Evaluation on 11/25
System Accuracy
PCMS #1 PCMS #2 PCMS #3
Average (seconds) -185.52 103.92 157.50
Standard deviation 792.84 224.49 168.95
• Statistical analysis of predicted travel time errors
• Frequency distribution of predicted travel time errorsDeviation of
predicted travel time error (sec)
Frequency
PCMS #1 PCMS #2 PCMS #3
<= -480 12 0 0
<= -240 4 0 0
<= 0 1 4 1
<= 240 1 5 4
<= 480 1 2 1
> 480 10 1 0
Total 29 12 6
12
System Accuracy Evaluation on 12/04
System Accuracy
• The accuracy drops as PCMS #1 is far away from I-695 and the volume difference increases.
10
PCMS #1 PCMS #2 PCMS #3
LB UB LBUB LB UB
1200 vph
3000 vph
1100 vph
4400 vph
1500 vph
2700 vph
# of incorrect 3 0 1
# of correct 13 12System accuracy
(%) 76.92 100.00 92.31
UB
4400 vph
Volume difference from Sensor data
13
System Accuracy Evaluation on 12/04
System Accuracy
PCMS #1 PCMS #2 PCMS #3
Average (seconds) -70.00 N/A 151.00
Standard deviation 190.13 N/A N/A
• Statistical analysis of predicted travel time errors
• Frequency distribution of predicted travel time errorsDeviation of
predicted travel time error (sec)
Frequency
PCMS #1 PCMS #2 PCMS #3
<= -480 0 0 0
<= -240 0 0 0
<= 0 1 0 0
<= 240 1 0 1
<= 480 0 0 0
> 480 0 0 0
Total 2 0 1
14
System Accuracy Evaluation
Conclusions
• The accuracy is often dropped for those PCMS far away from Spot 5 (I-695 Gore), or experiencing a wide range of the flow rate variation.
• The system doesn’t provide reliable travel time information during congested peak hours, especially for a short peak or a transition period between off-peak and peak hours.
15
System Reliability Evaluation on 11/14 and 11/19
• 11/14
• 11/19
PCMS #1 PCMS #2 PCMS #3
# of Inconsistence 0 0 1
# of Consistence 19 19 18System reliability
(%)100.00 100.00 94.74
PCMS #1 PCMS #2 PCMS #3
# of Inconsistence 4 1 1
# of Consistence 9 33 30System reliability
(%)69.23 97.06 96.77
(Note: “No message” from the PCMS is excluded from the sample size.)
16
System Reliability Evaluation on 11/24 and 11/25
PCMS #1 PCMS #2 PCMS #3
# of Inconsistence 0 0 0
# of Consistence 21 21 21System reliability
(%)100.00 100.00 100.00
• 11/24
• 11/25
PCMS #1 PCMS #2 PCMS #3
# of Inconsistence 24 1 1
# of Consistence 9 33 30System reliability
(%)27.27 97.06 96.77
(Note: “Blank” from the website is included from the sample size.)
17
System Reliability Evaluation on 12/04
• 12/04
PCMS #1 PCMS #2 PCMS #3
# of Inconsistence 7 0 0
# of Consistence 6 13 13System reliability
(%)46.15 100.00 100.00
(Note: “Blank” from the website is included from the sample size.)
18
System Reliability Evaluation
Conclusions
• The system reliability also show patterns similar to the system accuracy.
• That is, the reliability of a PCMS decreases with its distance to Spot 5 (I-695 Gore).
