Download - Evaluation of Travel Information and Prediction System (TIPS) for Work Zone Operations

Evaluation of Travel Information and Prediction System (TIPS) for Work Zone Operations

University of Maryland, College Park

2

TIPS overview

Traffic Information and Prediction System

• The messages displayed on these three PCMS are the range of travel time from each spot to Spot 5 (I-695 Gore) (e.g., “12 TO 16 MIN TO I-695”).

Spot 1

Spot 2 Spot 3

Spot 4 Spot 5

4.55 miles

7.55 miles

9.75 miles

10.80 miles

PCMS 1

PCMS 2 PCMS 3

SensorTrailer

I-695Gore

3

Criteria of TIPS Evaluation

System Accuracy and Reliability

Data collection for evaluation: Testing vehicle method

• TIPS Accuracy : comparing the actual measured travel times to those displayed from each PCMS.

• TIPS Reliability : consistency between the travel time information reported on the website and those displayed on each PCMS.

4

System Accuracy Evaluation on 11/14

System Accuracy

(Note: “No message” data on PCMS #1 were excluded from the sample size)

PCMS #1 PCMS #2 PCMS #3

LB UB LB UB LB UB

1750 vph

2400 vph

2200 vph

4250 vph

1650 vph

2650 vph

# of incorrect 9* 8 7

# of correct 6* 27 27System accuracy

(%) 40.00 77.14 79.41

• The accuracy drops as PCMS #1 is far away from I-695.

Volume difference from Sensor data

5


System Accuracy

• Statistical analysis of predicted travel time errors PCMS #1 PCMS #2 PCMS #3

Average (seconds) 149.11 38.75 206.29

Standard deviation 415.05 308.11 181.70

Deviation of predicted travel time error (sec)

Frequency


<= -480 0 1 0

<= -240 2 0 0

<= 0 2 2 1

<= 240 1 2 3

<= 480 1 3 3

> 480 3 0 0

Total 9 8 7

• Frequency distribution of predicted travel time errors

6


System Accuracy

11


LB UB LB UB LB UB

1900 vph

5000 vph

2100 vph

4100 vph

1100 vph

2400 vph

# of incorrect 8 5 4

# of correct 14 15System accuracy

(%) 57.89 73.68 78.95

• The volume difference from the sensor data at PCMS #1 is greater than those at PCMS #2 and #3

• The accuracy drops as PCMS #1 is far away from I-695 and the volume difference increases.


7


System Accuracy


Average (seconds) -21.00 85.40 87.25


Deviation of predicted travel time error (sec)

Frequency


<= -480 0 0 1

<= -240 1 0 0

<= 0 4 2 0

<= 240 1 2 2

<= 480 1 1 1

> 480 1 0 0

Total 8 5 4

• Statistical analysis of predicted travel time errors

• Frequency distribution of predicted travel time errors

8


System Accuracy

• The LB and UB of volumes are low and their differences are small, which mean that traffic conditions are uncongested and stable.

• The system can achieve a high accuracy under the stable traffic flow patterns.

21


LB UB LB UB LB UB

1550 vph

2450 vph

1200vph

2400vph

1000vph

2300vph


# of correct 21 19

System accuracy (%)

100.00 100.00 90.48


9


System Accuracy

PCMS #1 PCMS #2 PCMS #3Average (seconds) N/A N/A 148.00Standard deviation N/A N/A 16.97


• Frequency distribution of predicted travel time errorsDeviation of

predicted travel time error (sec)

Frequency


<= -480 0 0 0

<= -240 0 0 0

<= 0 0 0 0

<= 240 0 0 2

<= 480 0 0 0

> 480 0 0 0

Total 0 0 2

10


System Accuracy

2


LB UB LB UB LB UB

600 vph

3750 vph

1000 vph

4300 vph

1500 vph

2750 vph



(%) 6.45 61.29 80.65


• The performance of such a system clearly varies with the range of flow rate variation.

• PCMS #1 experiences the largest range of flow variation, and yields the poorest results.

11


System Accuracy


Average (seconds) -185.52 103.92 157.50





Frequency


<= -480 12 0 0

<= -240 4 0 0

<= 0 1 4 1

<= 240 1 5 4

<= 480 1 2 1

> 480 10 1 0

Total 29 12 6

12


System Accuracy

• The accuracy drops as PCMS #1 is far away from I-695 and the volume difference increases.

10


LB UB LBUB LB UB

1200 vph

3000 vph

1100 vph

4400 vph

1500 vph

2700 vph



(%) 76.92 100.00 92.31

UB

4400 vph


13


System Accuracy


Average (seconds) -70.00 N/A 151.00

Standard deviation 190.13 N/A N/A




Frequency


<= -480 0 0 0

<= -240 0 0 0

<= 0 1 0 0

<= 240 1 0 1

<= 480 0 0 0

> 480 0 0 0

Total 2 0 1

14

System Accuracy Evaluation

Conclusions

• The accuracy is often dropped for those PCMS far away from Spot 5 (I-695 Gore), or experiencing a wide range of the flow rate variation.

• The system doesn’t provide reliable travel time information during congested peak hours, especially for a short peak or a transition period between off-peak and peak hours.

15

System Reliability Evaluation on 11/14 and 11/19

• 11/14

• 11/19


# of Inconsistence 0 0 1

# of Consistence 19 19 18System reliability

(%)100.00 100.00 94.74




(%)69.23 97.06 96.77

(Note: “No message” from the PCMS is excluded from the sample size.)

16

System Reliability Evaluation on 11/24 and 11/25




(%)100.00 100.00 100.00

• 11/24

• 11/25




(%)27.27 97.06 96.77

(Note: “Blank” from the website is included from the sample size.)

17

System Reliability Evaluation on 12/04

• 12/04




(%)46.15 100.00 100.00

(Note: “Blank” from the website is included from the sample size.)

18

System Reliability Evaluation

Conclusions

• The system reliability also show patterns similar to the system accuracy.

• That is, the reliability of a PCMS decreases with its distance to Spot 5 (I-695 Gore).