35

Transportation Research Record: Journal of the Transportation Research Board, No. 2315, Transportation Research Board of the National Academies, Washington, D.C., 2012, pp. 35–46.DOI: 10.3141/2315-04

Université de Lyon, French Institute of Science and Technology for Transport, Deve lopment, and Networks, Laboratoire Ingénierie Circulation Transports, F-69675 Bron, F-69518 Vaux-en-Velin, France. Corresponding author: F. Marczak, florian.marczak@entpe.fr.

data to be collected, propose a set of measures of effectiveness, and indicate the procedure leading to calibration and validation of the tool on the specific project.• Validate, particularly on detailed data sets that are not com-

monly available and collected for a specific purpose, the key com-ponents of traffic simulation tools. Among them, the car-following and lane-changing models are probably two of the most important components of traffic simulation tools. The available data sets are NGSIM (1) and TU Delft (2).

This study is devoted to the second direction and contributes to the identification and correction of one of the possible deficiencies in detailed calibration and validation of car-following models: the data errors of individual trajectory data.

The validation process occurs after calibration. The car- following model calibration is made of successive steps. The first step concerns data filtering and enhancement in order to reduce the impact of mea-surement errors on the simulation results. The definition of the mea-sure of performance has to be done in the second step. To evaluate the overall performance of simulation models, the goodness of fit is defined in the third step. Finally one applies an optimization process to perform the calibration.

Here the focus is on the first step of the calibration process. Indeed, the impact of measurement errors on the calibration process can be significant. For example, Ossen and Hoogendoorn evaluate the effects on car-following calibration of adding to vehicle displacement errors from various combinations of random and systematic components (3, 4). They conclude that measurement errors yield a considerable bias in the calibration results. They also show that the presence of measurement errors reduces the sensitivity of the calibration objective.

The aim of this study is to test various filtering techniques and to propose a new one based on I-splines. A comparison is established between the current proposal and a set of filtering techniques pre-sented in the literature. For the comparison, the 5:00 to 5:15 p.m. data file for I-80, available within the NGSIM program, is used. There is a special focus on trajectories, speed profiles, and acceleration profiles. As a quality indicator of the various filtering techniques, velocity dis-tribution, acceleration distribution, acceleration standard deviation for each trajectory of the whole data set, and jerk analysis are used.

State of the art

filtering techniques

This state-of-the-art review is a summary of Section 3.2 of the paper by Punzo et al. (5), updated with some references. For further depth on the subject, the authors direct their readers to that paper (5).

New Filtering Method for Trajectory Measurement Errors and Its Comparison with Existing Methods

Florian Marczak and Christine Buisson

Dynamic traffic simulation tools are increasingly being used to help traffic managers and urban planners to make decisions. Therefore, simulation tool users require a validated methodology guaranteeing that simulation results can be trusted. This study contributes to the identification and correction of a possible deficiency in detailed calibration and validation of car-following models: the data errors of individual trajectory data. Some studies addressed the problem of filtering trajectory data. A new filtering technique to reduce the measurement errors on trajectories, speed pro-files, and acceleration profiles is proposed here. This technique is based on some piecewise polynomials termed “splines.” The proposed tech-nique is compared with a set of filtering techniques found in the literature. A complete trajectory data set available within the NGSIM program is used. As a quality indicator of the various filtering techniques, velocity distribution, acceleration distribution, and jerk analysis are used for the whole data set. Also, analyzing acceleration standard deviations for each trajectory of the data set is suggested. The main findings are as follows: (a) of the methods compared within this work, the I-spline method with the action points most reduces the spikes in the velocity distribution; (b) moreover, the I-spline method most reduces the percentage of jerk values higher than 15 m/s3 as well as the percentage of the 1-s windows with more than one sign inversion of the jerk; and (c) in some cases, this method increases the acceleration variability of smoothed trajectories.

Dynamic traffic simulation tools are increasingly being used to help traffic managers and urban planners to make decisions. For a long time visualization of the current and the forecast situation was most important for decision makers. But nowadays, it seems that by apply-ing good methodologies established in a scientifically sound manner, the predicted impact of the project on the current situation is val-ued by decision makers. Therefore, simulation tool users (consul-tancy agencies, for example) now ask for a validated methodology guaranteeing that simulation results can be trusted.

There are two directions for researchers to provide simulation tool users with some guaranties that the model used can be trusted:

• Provide some guidelines, ideally accompanied by ad hoc devices plugged into the tools. The guidelines must list the minimal amount of

36 Transportation Research Record 2315

Filtering is a general name given to the process that removes from data some unwanted components or features such as the noise due to measurement errors. The difficulty with filtering is to find a trade-off between the correction of unwanted components and respect for the real data dynamics. Averaging and smoothing are two forms of filtering.

The first common filtering technique used in the literature is the moving average. This method consists of creating a series of aver-ages of different subsets of the full data set in order to smooth out short-term fluctuations. The literature presents various definitions of the moving average. For instance, Ossen and Hoogendoorn (3) and Duret et al. (6) apply a simple moving average to reduce the measurement errors. Ossen and Hoogendoorn establish that post-processing the data can compensate for the negative impact of trajectory measurement errors (3). Hamdar and Mahmassani use a Gaussian kernel moving average to eliminate discontinuities in the lateral x-coordinate and speed from NGSIM data (7). In order to correct the measurement errors, Thiemann et al. employ a sym-metric exponential moving average (sEMA) to smooth trajectories, speeds, and accelerations from NGSIM data (8).

Another common filtering technique consists of smoothing the data, that is, creating an approximating function that attempts to cap-ture important patterns in the data while eliminating noise. There-fore, Brockfeld et al. use a Savitsky–Golay filter with a second-order polynomial over a 1-s window to determine acceleration from speeds from Global Positioning System data (9). Toledo et al. propose an approach to processing position data in order to develop vehicle trajectories and consequently speed and acceleration profiles (10). The approach uses locally weighted regression. In order to obtain accurate car-following data, Punzo et al. also use a local regression technique (11). They apply a fourth-order Butterworth filter with a cutoff frequency equal to 0.5 Hz.

Lu and Skabardonis propose an algorithm to estimate the propa-gation speed of shock waves on freeways based on vehicle trajec-tory data (12). The implementation of this algorithm consists first of filtering to smooth the speed–time trajectories. To this purpose, Lu and Skabardonis also use a Butterworth filter. Xin et al. propose a methodology for collecting and processing collision-inclusive tra-jectories (13). The postprocessing algorithm in this methodology employs a bilevel optimization structure to minimize measurement errors and internal inconsistency in position, speed, and acceleration. The authors use some piecewise polynomials termed splines.

Hoogendoorn et al. propose a new data-driven stochastic car- following model based on the principles of psycho-spacing model-

ing (14). This model uses trajectory data. To assess the data correctly, the authors present a filtering technique the assumption of which is that a trajectory can be represented by periods in which the accel-eration is constant. This assumption implies, consequently, that the speed is a continuous piecewise function of time. The aim of the filter is then to determine the time instants at which the acceleration changes and the points (the action points) describing the value of the piecewise linear function at these time instants.

The more advanced filtering techniques are the Kalman filter and the Kalman smoother. Punzo et al. use a Kalman filter in order to obtain accurate car-following data (11). In this study, the results of the Kalman filter are compared with those from the application of a Butterworth filter and a local regression. The authors report better results for the Kalman filter in terms of trajectory consistency. Ma and Andreasson present a project on modeling driver behavior in microscopic traffic simulation (15). The authors apply a Kalman smoother to the positions, speeds, and accelerations in order to filter measurement noise.

Table 1, taken from Punzo et al. (5) and completed, summarizes all studies that propose a filtering technique. Rows show the variable to which each filtering technique is applied. Columns are arranged in increasing order of complexity.

Comparison Indicators

For the comparison, the 5:00 to 5:15 p.m. data file of I-80 available within the NGSIM program was used. Table 2 presents the quality indicators used for the comparison of the various filtering techniques.

New fIlterINg teChNIque

Instead of using one single polynomial for the whole trajectory, the aim proposed here consists of filtering the positions by divid-ing the total time interval into smaller intervals and using lower-degree polynomials in each of these subintervals. Those particular polynomials are termed splines. They are, for instance, used by Xin et al. (13), Craven and Wahba (17), and Reinsch (18) to smooth discrete noisy data. To allow good continuity characteristics, it is suggested that a basis of nonnegative and monotone splines termed “I-splines” be used. Once they are calculated, the smoothed trajec-tories T could be represented as the linear combination T(t) = ∑am, where am are coordinates of smoothed trajectory T in the I-splines basis Ik,m, Ik,m (t; S), where Ik,m is a set of the I-splines basis. Smoothed

TABLE 1 Filtering Techniques Applied in Literature (5)

Variable Averaging Smoothing Kalman Filtering Kalman Smoothing

Coordinates Hamdar and Mahmassani (7 ) — Ervin et al. (16) —

Trajectory Ossen and Hoogendoorn (3) Toledo et al. (10) Ma and Andreasson (15)Duret et al. (6) Punzo et al. (11)Thiemann et al. (8) Xin et al. (13)

Hoogendoorn et al. (14)

Intervehicle spacing — — Punzo et al. (11) Ma and Andreasson (15)

Speed Duret et al. (6) Brockfeld et al. (9) Punzo et al. (11) Ma and Andreasson (15)Hamdar and Mahmassani (7 ) Toledo et al. (10)Thiemann et al. (8) Punzo et al. (11)

Lu and Skabardonis (12)Xin et al. (13)Hoogendoorn et al. (14)

Acceleration Thiemann et al. (8) Toledo et al. (10) — —Xin et al. (13)

Note: — = no studies done or used.

Marczak and Buisson 37

positions are differentiated to calculate smoothed velocities and accelerations.

S = {sm; M = 1, . . . , n} and k are, respectively, the knot sequence, which partitions the total time interval of the trajectory into smaller intervals, and the degree of the I-splines. The knot sequence and the degree condition the I-spline construction. These splines are defined as follows:

I t S

m j

s s M t S

kk mu k u k u

,,;

. ;( ) =

>

−( ) ( )+ + +

0

1 1

if

++− + ≤ ≤

< − +

=∑ 1

1

1 1

u m

j

j k m j

m j k

if

if

where u =index in the sum that defines the I-splines.Mk+1,u(t; S) is defined by the following recursion:

M t S

k t s M t S s t M

k m

m k m m k k

,

,

;

. ; .

( ) =

−( ) ( ) + −( )− +1 −− +

+

+

( ){ }−( ) −( )

≠

1 1

1

0

, ;

.

* m

m k m

m k m

t S

k s ss sif

iff s sm k m+ =

and

M t S

t

s ss s

m

s s

m mm m

m m

1 11

1

0

1

,

,

;( ) =( )

−≠+[ ]

++if

iff s sm m+ =

1

where 1[sm,sm+1](t) is the indicator function, that is, the function defined as

11

01

1

s s

m m

m

m mt

t s s

t s s,

,

,+[ ]

+( ) =∈ [ ]∉

if

if mm+[ ]

1

Two methods to set up the knot sequence were tested. The I-spline degree is a parameter of both of them. It is denoted d1 and d2 for Method 1 and Method 2, respectively. Method 1 is very basic and

consists of determining the time instants at which the speed changes along a trajectory. For this purpose, initially a threshold δ is fixed, which is the second parameter of Method 1. Then, on the assumption that the speed is constant during at least 1 s, 10 observations were con-sidered (NGSIM data observations have a frequency of 0.1 s). Next, a linear regression is carried out on these 10 observations. A knot is positioned at the time instant when the distance between the measured position and the regression curve is higher than the threshold δ.

Method 2 is more elaborate. According to Imai and Iri, it consists first of constructing two bounds around the measured trajectory by sliding it vertically by ω both upward and downward (19). The error bound ω is a parameter of Method 2. Then one finds the best piece-wise linear function with the minimum number of points to approxi-mate the measured trajectory inside the error bounds. The objective of Method 2 is more to find the minimum numbers of points than to approximate the measured trajectory by a piecewise linear function. These points will be used as the knot sequence to build the I-splines. The points are a kind of action point, which are used by Hoogendoorn et al. (14) and Hoogendoorn et al. (20). Therefore the term action point will be used to distinguish Method 2 from Method 1. Figure 1 illustrates both methods to set up the knot sequence.

For each I-spline method, a combination of both parameters was tested over all 500 experimental trajectories randomly extracted from the I-80 NGSIM data set. Such a sample is big enough to be statistically significant. Various values of d1 and d2 were tested ranging between 1 and 13. The feasible values of δ and ω are chosen between 0.1 and 3. For each trajectory of the sample, the optimal parameters are chosen to minimize the function:

Fn

x xi ii

n

= −( ) +

=

=∑min min

1 2

1

mes filacc RMσ SSE acc+( )σ

where

RMSE = root mean square error, σacc = acceleration standard deviation along filtered trajectory, xi

mes = measured data point at time instant ti, and xi

fil = filtered data point at time instant ti.

TABLE 2 Quality Indicators of Various Filtering Techniques Used for Comparison

Indicator Description Remarks and Explanations

Distribution of vehicle velocities over whole data set.

The velocity distribution is a function that describes the probability to each measured velocity value over the whole data set.

If taken as real, the spikes in the velocity distribution would mean that all drivers barely accelerate or decelerate to drive at the same preferred velocities (7). The spikes clearly show unrealistic behavior.

Distribution of vehicle accelerations over whole data set.

The acceleration distribution is a function that describes the probability to each measured acceleration value over the whole data set.

One can observe the percentage of unrealistic acceleration values beyond the threshold of ±3 m/s2 (7) before and after filtering.

Acceleration standard deviation along each trajectory.

The standard deviation is a measure of acceleration variability.

The bigger the acceleration standard deviation, the more unrealistic acceleration evolutions are seen along a given trajectory. Therefore, the acceleration standard deviation value should decrease after correction.

Percentage of observations over the data set with jerk values higher than the limit of ±15 m/s3 (Pjerk>15).

Pjerk>15 = N

Njerk>

tot

15

Njerk>15 is the number of observations with jerk values higher than limit of ±15 m/s3. Ntot is total number of observations.

Values exceeding the limit of ±15 m/s3 are considered not mechanically feasible (15).

Percentage of 1-s windows over the data set containing more than one sign inversion of the jerk (Pinv).

Pinv = N

Ninv

wind

Ninv is the number of 1-s windows containing more than one sign inversion of the jerk.

More than one sign inversion in a 1-s window is not physically consistent with human and mechanical observed responses (15).

Nwind is the total number of 1-s windows.

38 Transportation Research Record 2315

This objective function was considered to obtain a set of param-eters that minimizes (a) the lack of faithfulness to the data (as measured by root-mean-square error) and (b) the acceleration vari-ability (as measured by σacc) along each trajectory of the calibration sample.

Figure 2 presents the calibration results. A sample of 500 experi-mental trajectories randomly extracted from the 5:00 to 5:15 p.m. data file of I-80 was used. The more recurrent values are retained as optimal parameters. In Figure 2, a and b, one can see the distribu-tions for the parameters of the I-spline method without the action points. Figure 2, a and c, shows that one can optimally smooth trajectories with cubic splines. Figure 2d shows that the optimal ω-value is 0.3 for the I-spline method with the action points. The

δ-distribution is uniform (see Figure 2b). The maximum percentage is equal to 8% for δ = 0.2. Therefore, the following values are used: d1 = 3 and δ = 0.2 for Method 1 (without action points); d2 = 3 and ω = 0.3 for Method 2 (with action points).

fIlterINg teChNIqueS CoNSIdered for ComparISoN

Table 1 reports and classifies the studies that address the problem of filtering trajectory data. The rows show the variable to which each technique is applied. A filtering technique to smooth trajectories, speed profiles, and acceleration profiles is proposed. Among the aver-

FIGURE 1 Methods to set up knot sequence: (a) without action points, threshold d fixed at 2.5 m, and (b) constructing two bounds around measured trajectory by sliding it vertically by v both upward and downward, error bound v fixed at 2.5 m.

Measured trajectory

611 612 613 614 615 616 617 618 619 620 621 210

220

230

240

250

260

270

280

290

Time [s]

Pos

ition

[m]

611 612 613 614 615 616 617 618 619 620 621 210

220

230

240

250

260

270

280

290

Time [s]

(b)

(a)

Pos

ition

[m]

Marczak and Buisson 39

aging techniques presented in Table 1, only the sEMA [proposed by Thiemann et al. (8)] is applied to these three variables. Among the smoothing techniques, only the locally weighted regression [proposed by Toledo et al. (10) and Punzo et al. (11)] is applied to trajectories, speed profiles, and acceleration profiles. Therefore the current proposal will be compared with both the sEMA and the locally weighted regression. To compare the proposal with a low-pass filter, the Butterworth filter proposed by Punzo et al. (11) and Lu and Skabardonis (12) will be used.

A Kalman filter approach to filter vehicle trajectories is also proposed. The model can be represented as follows (15):

X t A X t t+( ) = ( ) + ( )1 * Γ

Y t X t t( ) = ( ) + ( )Θ

where

X(t) = [x(t) v(t) a(t)]T = physical state vector of position, speed, and acceleration [v(t) and a(t) are simply calculated by differencing positions];

Γ(t) = noise vector in state transition; Y(t) = measurement vector;

1 2 3 4 5 6 7 8 9 10 11 12 13 0

5

10

15

20

25

30

35

40

d1

Freq

uenc

y (%

)

0 0. 5 1 1. 5 2 2. 5 3 0

5

10

15

20

25

30

35

40

Freq

uenc

y (%

)

1 2 3 4 5 6 7 8 9 10 11 12 13 0

5

10

15

20

25

30

35

40

d2

(a)

(c)

(b)

(d)

Freq

uenc

y (%

)

0 0. 5 1 1. 5 2 2. 5 3 0

5

10

15

20

25

30

35

40 Fr

eque

ncy

(%)

FIGURE 2 Parameter distributions for I-spline methods [(a, b) with action points and (c, d) without action points]: (a) d1 distribution (Fmax 5 24%), (b) d distribution (Fmax 5 8%), (c) d2 distribution (Fmax 5 31%), and (d) v distribution (Fmax 5 15%).

40 Transportation Research Record 2315

Θ(t) = measurement noise vector; and A = state transition matrix with δt = 0.1 s:

At

t

t=

12

00

10 1

2

δ δ

δ

Filtering techniques considered for the comparison were not calibrated in this study, but values obtained from previous stud-ies were applied. The sEMA depends on the smoothing window size. According Thiemann et al. (8), a smoothing width equal to 0.5 s, 1 s, and 4 s for the position, the speed, and the acceleration, respectively, will be considered. The locally weighted regression depends on the smoothing window size and the polynomial order. Toledo et al. used different combinations of both of these parameters to test the sensitivity of the quality of the results (10). According to their results a smoothing window size equal to 9 observations and a polynomial order equal to 6 will be used. The Butterworth filter depends on a cutoff frequency. Punzo et al. obtained a cutoff fre-quency by simulation equal to 0.5 Hz. This value is considered for the comparison (11).

Results

For the comparison, first an example of the trajectory and the 5:00 to 5:15 p.m. data file for I-80 are used.

Figure 3 illustrates the effects of the applied filtering techniques on the missing observations. For this purpose a pair of vehicles in the 5:00 to 5:15 p.m. data file for I-80 was selected. Two trajectories (Ref-erence IDs: 2035 and 2042) with missing observations were chosen. (Some observations in the NGSIM trajectories are missing because of the data processing. One can observe lines on the trajectories where observations are missing.) By comparing the NGSIM trajectory line segments before and after filtering, one can observe if the filtering can correct the missing observations. The smoothed trajectories with the sEMA, the Butterworth filter, the locally weighted regression, and the Kalman filter are close to the original trajectories (see Figure 3, c–f ). The distance between the smoothed trajectories with the I-spline meth-ods and the original trajectories is higher (see Figures 3, a and b). The I-spline methods seem able to correct the missing observations (blue ellipses, Figure 3). But Figure 3, a and b, also show I-spline limits in reproducing vehicles’ trajectories. One can see that the vehicle’s stop (red ellipses, Figure 3) has been almost eliminated by the filter.

Figure 4 reports the acceleration and speed profiles of Vehicle 2042. The applied smoothing techniques eliminate all measurements

(a)

(c)

(b)

(d)

FIGURE 3 Effects of applied smoothing techniques on original trajectories: (a) I-spline, (b) I-spline with action points, (c) sEMA, (d) Butterworth filter. (continued)

Marczak and Buisson 41

630 64 0 65 0 0

2

4

6

8

10

Time [s]

Spe

ed [m

/s]

a1 /

630 64 0 650 0

2

4

6

8

10

Time [s]

Spe

ed [m

/s]

b1 /

630 64 0 65 0 0

2

4

6

8

10

Time [s]

Spe

ed [m

/s]

c1 /

630 64 0 650 0

2

4

6

8

10

Time [s]

Spe

ed [m

/s]

d1 /

630 64 0 65 0 0

2

4

6

8

10

Time [s]

Spe

ed [m

/s]

e1 /

630 64 0 650 0

2

4

6

8

10

Time [s]

Spe

ed [m

/s]

f1 /

63 0 64 0 65 0 -20

-10

0

10

20

30

Time [s]

Acc

eler

atio

n [m

/s²]

a2 /

630 64 0 650 -20

-10

0

10

20

30

Time [s]

Acc

eler

atio

n [m

/s²]

b2 /

63 0 64 0 65 0 -20

-10

0

10

20

30

Time [s]

Acc

eler

atio

n [m

/s²]

c2 /

630 64 0 650 -20

-10

0

10

20

30

Time [s]

Acc

eler

atio

n [m

/s²]

d2 /

63 0 64 0 65 0 -20

-10

0

10

20

30

Time [s]

Acc

eler

atio

n [m

/s²]

e2 /

630 64 0 650 -20

-10

0

10

20

30

Time [s]

Acc

eler

atio

n [m

/s²]

f2 /

(a) (b) (c) (d)

(e) (f) (g) (h)

(i) (j) (k) (l)

FIGURE 4 Effects of applied smoothing techniques on section of speed and acceleration profiles (Vehicle 2042): (a) I-splines, (b) I-splines, (c) I-splines with action points, (d) I-splines with action points, (e) sEMA, (f) sEMA, (g) Butterworth filter, (h) Butterworth filter, (i) locally weighted regression, (j) locally weighted regression, (k) Kalman filter, and (l) Kalman filter.

FIGURE 3 (continued) Effects of applied smoothing techniques on original trajectories: (e) locally weighted regression, and (f) Kalman filter.

(e) (f)

42 Transportation Research Record 2315

0 2 4 6 8 10 12 0

0. 02

0. 04

0. 06

0. 08

0. 1

Speed [m/s]

Pro

b. d

ensi

ty

Or ig in al dat a

Sm oot h ed da ta

0 2 4 6 8 10 12 0

0. 02

0. 04

0. 06

0. 08

0. 1

Speed [m/s] P

rob.

den

sity

0 2 4 6 8 10 12 0

0. 02

0. 04

0. 06

0. 08

0. 1

Speed [m/s]

Pro

b. d

ensi

ty

0 2 4 6 8 10 12 0

0. 02

0. 04

0. 06

0. 08

0. 1

Speed [m/s]

Pro

b. d

ensi

ty

0 2 4 6 8 10 12 0

0. 02

0. 04

0. 06

0. 08

0. 1

Speed [m/s]

Pro

b. d

ensi

ty

0 2 4 6 8 10 12 0

0. 02

0. 04

0. 06

0. 08

0. 1

Speed [m/s]

(a) (b)

(c) (d)

(e) (f)

Pro

b. d

ensi

ty

FIGURE 5 Effects of applied filtering techniques on velocity distribution (dashed line 5 original data): (a) I-splines, (b) I-splines with action points, (c) sEMA, (d) Butterworth filter, (e) locally weighted regression, and (f) Kalman filter.

as very noisy. The Butterworth filter cuts off high frequencies but preserves variations in speed and acceleration profiles (Fig-ure 4, g and h). The locally weighted regression (Figure 4i) and the Butterworth filter (Figure 4g) preserve the constant speed plateaus even if these periods are due to missing observations. I-spline methods remove the vehicle’s stop between 632 s and 638 s (Figure 4, a and b).

Figure 5 presents the effects of the applied filtering techniques on the velocity distribution. Among the filtering methods compared

in this work, the method using the I-splines and the action points is the one that most reduces the spikes in the velocity distribu-tion. Thus this method seems to most reduce the effects of the measurement errors on the velocity distribution. Results for the I-spline method without the action points (Figure 5a) are close to the sEMA and the Kalman filter results (Figure 5, c and f, respec-tively). The Butterworth filter removes the noise from the NGSIM data but preserves the bias, and therefore does not permit the spikes in the velocity distribution to diminish (Figure 5d).

Marczak and Buisson 43

Figure 6 presents the effects of the applied filtering techniques on the acceleration distribution from the whole data set. Figure 6a cor-responds to the original data acceleration distribution. The standard deviation for this distribution is high (8.6) and thus indicates that the NGSIM accelerations are spread out over a large range of values. Some NGSIM acceleration values are beyond the threshold of ±3 m/s2. Each applied filtering technique reduces the acceleration standard devia-tion and the number of unrealistic values beyond ±3 m/s2. However, the I-spline method without the action points reduces the accelera-

tion standard deviation less than the other filtering methods compared in this work. The acceleration standard deviation reaches 3.6 for the I-spline method without the action points, whereas it ranges from 0.76 for the Kalman filter (Figure 6g) to 2.9 for the I-spline method with the action points (Figure 6c). The filtered data tend to be closer to the mean acceleration than the original data. The sEMA reduces the most the standard deviation and the number of unrealistic values.

Figure 7 compares the acceleration standard deviation along each single trajectory of the whole data set. The black diagonal line

-1 0 -5 0 5 10 0

10

20

30

40

50

60

70

Acceleration [m/s²]

Fre

quen

cy (

%)

= 8. 6 Fmax = 47%

-1 0 -5 0 5 10 0

10

20

30

40

50

60

70

Acceleration [m/s²]

Fre

quen

cy (

%)

= 3. 6 Fmax = 47%

-1 0 -5 0 5 10 0

10

20

30

40

50

60

70

Acceleration [m/s²]

Fre

quen

cy (

%)

= 2. 9 Fmax = 54%

-1 0 -5 0 5 10 0

10

20

30

40

50

60

70

Acceleration [m/s²]

Fre

quen

cy (

%)

= 1. 6 Fmax = 70%

-1 0 -5 0 5 10 0

10

20

30

40

50

60

70

Acceleration [m/s²]

Fre

quen

cy (

%)

= 2. 8 Fmax = 42%

-1 0 -5 0 5 10 0

10

20

30

40

50

60

70

Acceleration [m/s²]

Fre

quen

cy (

%)

= 1. 6 Fmax = 53%

-1 0 -5 0 5 10 0

10

20

30

40

50

60

70

Acceleration [m/s²]

(a)

(b) (c)

(d) (e)

(f) (g)

Fre

quen

cy (

%)

= 0. 76 Fmax = 59%

FIGURE 6 Effects of applied filtering techniques on acceleration distribution: (a) original NGSIM data, (b) splines, (c) splines with action points, (d) sEMA, (e) Butterworth filter, (f) locally weighted regression, and (g) Kalman filter.

44 Transportation Research Record 2315

0 5 10 15 20 25 0

5

10

15

20

25

Standard deviation NGSIM

Sta

ndar

d de

viat

ion

Spl

ines

0 5 10 15 20 25 0

5

10

15

20

25

Standard deviation NGSIM

Sta

ndar

d de

viat

ion

Spl

ines

+A

P

0 5 10 15 20 25 0

5

10

15

20

25

Standard deviation NGSIM

Sta

ndar

d de

viat

ion

sEM

A

0 5 10 15 20 25 0

5

10

15

20

25

Standard deviation NGSIM

Sta

ndar

d de

viat

ion

But

terw

orth

0 5 10 15 20 25 0

5

10

15

20

25

Standard deviation NGSIM

Sta

ndar

d de

viat

ion

Loc.

Wei

ght.

Reg

.

0 5 10 15 20 25 0

5

10

15

20

25

Standard deviation NGSIM

(a) (b)

(c) (d)

(e) (f)

Sta

ndar

d de

viat

ion

Kal

man

FIGURE 7 Comparison between acceleration standard deviation (SD) along each trajectory before and after filtering (x-axis 5 original NGSIM data acceleration SD, y-axis 5 acceleration SD after filtering): (a) splines, (b) splines with action points, (c) sEMA, (d) Butterworth filter, (e) locally weighted regression, and (f) Kalman filter.

in Figure 7 represents the identity function. When a point is over the identity function, the filtering technique increases the accelera-tion variability in comparison with the original NGSIM data. The filtering techniques smooth the peaks in the acceleration profiles (see Figure 4) and therefore reduce the acceleration variability. One can, however, observe in Figure 7b that the I-spline method with the action points increases the acceleration variability in par-ticular for some trajectories with a low standard deviation in the

NGSIM data. Through the use of the Kalman filter one can control the acceleration standard deviation: the lower the set noise standard deviation, the lower the filtered acceleration standard deviation. The acceleration standard deviation after filtering is independent of the original acceleration standard deviation. The noise standard deviation has been fixed for the whole data set, so the accelera-tion standard deviation after filtering is approximately the same in Figure 7f.

Marczak and Buisson 45

Moreover, the I-spline method is also the one that most reduces the percentage of 1-s windows with more than one sign inversion of the jerk. The Kalman filter most reduces the percentage of jerk values higher than the threshold of 15 m/s3 (see Table 3). These two indicators reach 98% and 36%, respectively, for the original NGSIM data; they are equal to 59% and 0.8% for the sEMA and 47% and 5.3% for the Butterworth filter; and they reach 14% and 1.3% for the locally weighted regression. Regarding these two indicators, the I-spline method without the action points has better results than the one with the action points. The percentage of 1-s windows with more than one sign inversion of the jerk and the percentage of jerk values higher than the limit of 15 m/s3 reach 0.3% and 0.6%, respectively, for the I-spline method without the action points. The same indica-tors are respectively equal to 0.7% and 1% for the I-spline method with action points. It is claimed that more than one sign inversion of the jerk in a 1-s window is not physically consistent with human and mechanical observed responses. Moreover, jerk values exceeding the threshold of ±15m/s3 are considered not mechanically feasible.

dISCuSSIoN aNd CoNCluSIoN

Measurement errors have an impact on calibration results. The aim of this study was to propose a new filtering technique based on I-splines and to compare it with a set of various filtering techniques proposed in the literature. For this purpose a complete trajectory data set from I-80 was used, which is available within the NGSIM program. Velocity distribution, acceleration distribution, acceleration standard deviation along each trajectory of the data set, and jerk analysis were used as quality indicators of the various filtering techniques.

The main findings of the comparison are as follows. Among the filtering techniques compared within this work, the I-spline method with the action points most reduces the spikes in the velocity distri-bution. The I-spline method without the action points most reduces the percentage of jerk values higher than 15 m/s3 and the percentage of 1-s windows with more than one sign inversion of the jerk. But in some cases, I-spline methods increase the acceleration variability of smoothed trajectories. The smoothed trajectories with the sEMA, the locally weighted regression, and the Butterworth filter are close to the original trajectories even if they are missing observations on these trajectories. I-spline methods seem to have more effect on the missing observations.

Some of the results presented from the application of the new tech-nique are promising. However, the comparison among the different techniques is not completely fair. In the current version of the work, I-spline methods only were calibrated. The other filtering techniques considered for the comparison were not calibrated in this study, but values obtained from previous studies were applied. If one applies to a data set a filter calibrated on a different data set, it is likely to obtain unsatisfactory results. Therefore, to make a valid comparison, filtering techniques must be calibrated on the same data set.

The current version of the work does not take into account the internal inconsistency of the data. Xin et al. also used splines (13). However, the methodology they proposed employs a bilevel opti-mization structure not only to minimize measurement errors but also to resolve internal inconsistency with position, speed, and acceleration data.

Punzo et al. present three kinds of analysis for trajectory data (5). The first, as was presented in this paper, examines the jerk values in order to verify acceleration feasibility. The second aims at verify-ing the platoon and internal consistency of data. The third examines spectral frequencies on speed, acceleration, and jerk. The last two kinds of analyses were not implemented in this paper. This analysis will be done in further research in order to complete the comparison between the filtering techniques.

The first calibration process step—the data filtering and enhancement—was focused on here. An interesting topic for future research is to analyze the effects of real trajectory data enhance-ment on the whole calibration process. For example, the calibration method proposed by Chiabaut et al. could be used (21). To tackle the calibration drawbacks highlighted by Ossen and Hoogendoorn (3, 4), Chiabaut et al. propose a method to estimate Newell’s car-following model parameters. This model assumes that two consecu-tive vehicles are related in congestion by a shift −w¢ with slope w. The proposed calibration method consists simply in determining which values of w provide uniform coordinates of w¢ along the trajectories.

aCkNowledgmeNtS

The authors thank Serge Hoogendoorn, Raymond Hoogendoorn, and Ludovic Leclercq for their help in understanding the principles of the action points. The authors also thank the reviewers for their valuable comments.

refereNCeS

1. NGSIM—Next Generation SIMulation. FHWA, U.S. Department of Transportation, 2008. http://www.ngsim.fhwa.dot.gov.

2. Hoogendoorn, S. P., H. J. van Zuylen, M. Schreuder, B. Gorte, and G. Vosselman. Microscopic Traffic Data Collection by Remote Sens-ing. In Transportation Research Record: Journal of the Transporta-tion Research Board, No. 1855, Transportation Research Board of the National Academies, Washington, D.C., 2002, pp. 121–128.

3. Ossen, S., and S. P. Hoogendoorn. Validity of Trajectory-Based Calibra-tion Approach of Car-Following Models in Presence of Measurement Errors. In Transportation Research Record: Journal of the Transporta-tion Research Board, No. 2088, Transportation Research Board of the National Academies, Washington, D.C., 2008, pp. 117–125.

4. Ossen, S., and S. P. Hoogendoorn. Reliability of Parameter Values Estimated Using Trajectory Observations. In Transportation Research Record: Journal of the Transportation Research Board, No. 2124, Transportation Research Board of the National Academies, Washington, D.C., 2009, pp. 36–44.

TABLE 3 Percentage of 1-s Windows with More Than One Sign Inversion of Jerk and Percentage of Jerk Values Higher Than Threshold of 615 m/s3 for Original NGSIM Data and Applied Filtering Method

VariableOriginal NGSIM Data I-Splines

I-Splines and Action Points sEMA Butterworth

Locally Weighted Regression

Kalman Filter

Percentage of 1-s window with more than one sign inversion of jerk

98.6 0.3 0.7 59.7 47.3 13.9 17

Percentage of jerk values higher than ±15 m/s3 35.7 0.6 1.0 0.8 5.3 1.3 1.5

46 Transportation Research Record 2315

5. Punzo, V., M. T. Boracchiello, and B. Ciuffo. On the Assessment of Vehicle Trajectory Data Accuracy and Application to the Next Genera-tion Simulation (NGSIM) Program Data. Transportation Research Part C, Vol. 19, No. 6, 2011, pp. 1243–1262.

6. Duret, A., C. Buisson, and N. Chiabaut. Estimating Individual Speed–Spacing Relationship and Assessing Ability of Newell’s Car-Following Model to Reproduce Trajectories. In Transportation Research Record: Journal of the Transportation Research Board, No. 2088, Transpor-tation Research Board of the National Academies, Washington, D.C., 2008, pp. 188–197.

7. Hamdar, S. H., and H. S. Mahmassani. Driver Car-Following Behavior: From a Discrete Event Process to Continuous Set of Episodes. Pre-sented at 87th Annual Meeting of the Transportation Research Board, Washington, D.C., 2008.

8. Thiemann, C., M. Treiber, and A. Kesting. Estimating Acceleration and Lane-Changing Dynamics from Next Generation Simulation Trajectory Data. In Transportation Research Record: Journal of the Transporta-tion Research Board, No. 2088, Transportation Research Board of the National Academies, Washington, D.C., 2008, pp. 90–101.

9. Brockfeld, M., R. D. Kühne, and P. Wagner. Calibration and Valida-tion of Microscopic Traffic Flow Models. In Transportation Research Record: Journal of the Transportation Research Board, No. 1876, Transportation Research Board of the National Academies, Washington, D.C., 2004, pp. 62–70.

10. Toledo, T., H. N. Koutsopoulos, and K. I. Ahmed. Estimation of Vehi-cle Trajectories with Locally Weighted Regression. In Transportation Research Record: Journal of the Transportation Research Board, No. 1999, Transportation Research Board of the National Academies, Wash-ington, D.C., 2007, pp. 161–169.

11. Punzo, V., D. J. Formisano, and V. Torrieri. Nonstationary Kalman Filter for Estimation of Accurate and Consistent Car-Following Data. In Transportation Research Record: Journal of the Transportation Research Board, No. 1934, Transportation Research Board of the National Academies, Washington, D.C., 2005, pp. 3–12.

12. Lu, X.-Y., and A. Skabardonis. Freeway Traffic Shockwave Analysis: Exploring the NGSIM Trajectory Data. Presented at 86th Annual Meet-ing of the Transportation Research Board, Washington, D.C., 2007.

13. Xin, W., J. Hourdos, and P. G. Michalopoulos. Vehicle Trajectory Collection and Processing Methodology and Its Implementation. Pre-sented at 87th Annual Meeting of the Transportation Research Board, Washington, D.C., 2008.

14. Hoogendoorn, S., R. G. Hoogendoorn, and W. Daamen. Wiedemann Revisited: New Trajectory Filtering Technique and Its Implications for Car-Following Modeling. In Transportation Research Record: Jour-nal of the Transportation Research Board, No. 2260, Transportation Research Board of the National Academies, Washington, D.C., 2011, pp. 152–162.

15. Ma, X., and I. Andreasson. Dynamic Car Following Data Collection and Noise Cancellation Based on the Kalman Smoothing. Proc., Inter-national Conference on Vehicular Electronics and Safety (ICVES’05), IEEE, New York, 2005.

16. Ervin, R. D., C. C. MacAdam, K. Gilbert, Jr., and P. Tchoryk. Quantita-tive Characterization of the Vehicle Motion Environment (VME). Proc., 2nd Vehicle Navigation and Information Systems Conference, October 20–23, Dearborn, Mich., 1991, pp. 1011–1029.

17. Craven, P., and G. Wahba. Smoothing Noisy Data with Spline Func-tions: Estimating the Correct Degree of Smoothing by the Method of Generalized Cross-Validation. Numerische Mathematik, Vol. 31, 1979, pp. 377–403.

18. Reinsch, C. M. Smoothing by Spline Function. Numerische Mathema-tik, Vol. 10, 1973, pp. 377–403.

19. Imai, H., and M. Iri. An Optimal Algorithm for Approximating a Piece-wise Linear Function. Journal of Information Processing, Vol. 9, No. 3, 1987, pp. 159–162.

20. Hoogendoorn, R. G., S. P. Hoogendoorn, K. A. Brookhuis, and W. Daamen. Adaptation Longitudinal Driving Behavior, Mental Workload, and Psycho-Spacing Models in Fog. In Transportation Research Record: Journal of the Transportation Research Board, No. 2249, Transpor-tation Research Board of the National Academies, Washington, D.C., 2011, pp. 20–28.

21. Chiabaut, N., L. Leclercq, and C. Buisson. From Heterogeneous Drivers to Macroscopic Patterns in Congestion. Transportation Research Part B, Vol. 44, 2010, pp. 299–308.

The Traffic Flow Theory and Characteristics Committee peer-reviewed this paper.