Night Time Vehicle Detection for Real Time Traffic Monitoring Systems: A
Review
Padmavathi.S1, Keerthana Gunasekaran2 1,2Department of Computer Science, Amrita School of Engineering, Coimbatore.
Abstract
Real time traffic surveillance using computer vision
system is an emerging research area. Many new
algorithms are being developed to perform the
surveillance in the most effective manner. The first and
critical step in these road traffic monitoring systems is
to detect and track the vehicles. In this paper, we
provide a brief review on the night time vehicle
detection techniques that have been used in the recent
years. The detection of vehicles in the night time can
prove to be challenging because the usual features of
the vehicles like the vehicle shadows, horizontal and
vertical edges that helps in the identification in day
time cannot be used during the night time. The only
salient features that are visible in the night time are
headlights, rear-lights and their beams, street-lamps,
horizontal signals such as zebra crossings and traffic
scenes with reflectors. Thus, in night time surveillance
the target objects are the vehicle headlights and rear
lights.
1. Introduction
The data for the real time traffic monitoring systems
can come various sources like the loop detectors,
ultrasonic detectors, microwave sensors, radar sensors
or video cameras. Due to the recent advancement in
computer vision and image processing techniques, the
video cameras have been found to be an efficient means
to collect and analyze the traffic data. Video based
camera systems are more sophisticated and robust
because the information that is associated with the
image sequences present in the video allows us to
identify and classify the vehicles in the most effective
manner. The temporal continuity of data in video
stream helps in improving the accuracy during vehicle
detection. A video based monitoring system must be
able to handle various weather and illumination
conditions.
The vehicle detection in day time can be done using
various methods which can be motion based,
knowledge based or appearance based. The various
techniques that are commonly employed in day time
vehicle detection is comprehensively reviewed in [1].
The methods mostly use the edges in the vehicle to
identify the moving objects in the screen through frame
differencing methods. Stauffer and Grimson [2] in 1999
published a novel method to detect the moving vehicles
using background subtraction. In this method each
pixel was modeled as mixture of Gaussians and based
on the variance and persistence, the Gaussians which
corresponds to the background colors were determined.
The pixel values that do not fit the background
distributions were considered to be foreground. This
method was able to handle very slow illumination
changes and the multimodal distributions caused by
swaying trees. This method was further improved in [2]
where an algorithm was proposed to learn the
descriptive mixture of the first few frames and the
result of this algorithm was used in the Gaussian
mixture model. In [3], a spatial temporal technique was
used to detect the vehicles. This method exploits the
information in the moving points, which can be
gathered by the difference between the successive
frames and the variations in the luminance in these
points. The SVM classifiers can also be used after
extracting the required features as suggested by [3].
The vehicle detection can also be obtained by using
Gabor filter and Kalman filter to predict the next
position [4]. Apart from these common methods, the
vehicles can also be detected using the changes in the
optical flow as reviewed in [5] and all these methods
try to accommodate the illumination changes and
weather conditions.
The methods used for the day time vehicle detection
cannot be used for the night time vehicle detection due
to various factors. In the night time, the bad
illumination causes strong noise which increases the
complexity of the detection task. The reflection of the
beams of headlights and rear lights can cause lots of
false alarms during the detection process. The moving
reflections of the headlights can introduce a lot of
foreground or background ambiguities. Moreover, at
night the camera images have very low contrast and a
weak light sensitivity thus making it difficult to use the
normal day time detection methods. An effective
method is to use infra-red thermal cameras as night
vision sensors to collect the traffic data during the night
Keerthana Gunasekaran et al, Int.J.Computer Technology & Applications,Vol 5 (2),451-456
IJCTA | March-April 2014 Available [email protected]
451
ISSN:2229-6093
time. This way the normal detection techniques can be
used. But unfortunately, this method is very expensive
and can be rarely seen in the real time scenarios and
moreover installing two different cameras for night and
day time is futile.
The night time environment can be classified into two
types. The unlit scene where there are no street lights
and the only features visible are the headlights and their
reflections. The headlights appear as a bright round
blob in contrast to the dark surroundings. The second
type is the lit scenes where the streets are illuminated
by the public light poles like most of the urban areas. In
this type of scenes, the background, pedestrians and
other objects are also visible. Here the headlights are
not clearly visible, especially when the vehicles are also
in white or some light colors. Hence the complexity of
extracting the headlights from the images is increased.
In this paper, Section II gives the general description of
the night time vehicle detection algorithms. Section III
sheds light on the various methods used in feature
extraction and Section IV provides an insight into
various methods used for headlight pairing and
verification. Section V elucidates the conclusion and
references.
2. Night Time Vehicle Detection Most of the night time detection algorithms use the
headlights and tail lights as the main features in the
detection of vehicles. The general template of the night
time vehicle detection algorithm as shown in Fig.1
consists of mainly four phases namely the
preprocessing phase, feature extraction and
classification to detect the vehicles.
Fig 1 – General Algorithm for night time vehicle
detection
The preprocessing phase is optional and is responsible
for the offline configuration of the traffic scene as
specified in [6]. This phase mainly consists of the
calibration of the camera and collecting the required
information about the background, lanes and camera
parameters which can be used in later stages to judge
the traffic flow and vehicle status. The preprocessing
stage also removes the noise using smoothing filters
and making the image easy to process.
3. Feature Extraction
The main features that can be used in the night time
vehicle detection are headlights, rear lights and
windshields. The various methods that can be used in
the extraction of these features are described in this
section.
3.1. Headlight Detection
Headlights are strong and consistent features which can
be used to reveal the presence of a vehicle at night. The
major difficulty lies in segmenting the headlights pixels
from the other bright pixels that are present in the
scene. Usually, the contrast between the headlights and
their surrounding is very high and the histogram of the
images of the night environment is bimodal. The
headlights can be identified based on the reflection
identity map and the reflection suppression map as used
in [7] but the method is complex and is not used
frequently. Most of the algorithms have two main steps
to detect the headlights namely bright pixel extraction
and headlight identification.
3.1.1. Bright Pixel Extraction
Third-order headings, as in this paragraph, are
discouraged. However, if you must use them, use 10-
point Times, boldface, initially capitalized, flush left,
preceded by one blank line, followed by a period and
your text on the same line.
3.1.1.1. Non-linear Luminance
The bright pixels can be extracted by extracting the
connected pixels that maximize the value of V/S ratio
(value on saturation) on the different color channels (R,
G, B). This can be done using the non-linear luminance
as the threshold value [8]. This can be followed by
white top hat transform to detect the presence of
vehicles.
) )) Eq. (1)
3.1.1.2. Canny Edge Detection
The first step is to smoothen the image using Gaussian
filter. The smoothened image is then differentiated
along the X and Y directions and non-maximum
suppression is performed as suggested in [8]. Finally
Input Video
Preprocessing
Feature Extraction
Vehicle Identification
(Analysis of Features)
Keerthana Gunasekaran et al, Int.J.Computer Technology & Applications,Vol 5 (2),451-456
IJCTA | March-April 2014 Available [email protected]
452
ISSN:2229-6093
double thresholding is performed to get the edges of the
image. The binary image b(x, y) is created from an
intensity image I(x, y) according to the Eq. (2).
) { )
Eq. (2)
where T is the thresholding level. The thresholding is
done on the obtained region of interest. This method
used in [9] is very effective because the edges are not
missed n this detector and there will not be an image
without any edges present. However, the canny edge
detector can be improved by using the fuzzy
enhancement algorithm instead of smoothening the
image using simple Gaussian filter [10]. Before
performing the fuzzy enhanced canny edge detection,
the spatial homomorphism can be performed to
eliminate the illumination effects [10].
3.1.1.3 Multilevel Thresholding
The bright pixels can be extracted using the
thresholding. Ostu [11] has determined the optimal
threshold by maximizing the between class variance of
bright and dark regions. This method is improved in
[12]. The normalized probability is calculated as Pi.
The image I is split into k+1classes using k threshold
values. Here T is taken as the threshold set and Ci
represents the classes that the pixels are grouped into
using the thresholds. The between class variance and
within class variance can be calculated using the Eq.
(3) and (4) respectively
) ∑ )
Eq. (3)
) ∑
Eq. (4)
Where denotes the total variance and the overall mean is
denoted by .
∑
Eq. (5)
∑
Eq. (6)
The values , and represents the cumulative
probability, mean and standard deviation of class
Cn The separability factor can be calculated as follows
:
)
)
Eq. (7)
When the separability factor approaches to 1, the
classes of gray levels are completely separated. When
the following condition is satisfied, the largest
threshold value is used and the bright objects are
separated.
Eq. (8)
3.1.2. Headlight Identification
The bright pixel extraction extracts all the bright pixels
in the particular frame. But not all of these pixels
correspond to the headlights. They can be street lights,
reflections or other nuisance lights. To separate the
headlights from the other lights the following methods
are used:
3.1.2.1 Morphological Filtering
The morphological filtering can be used to remove
false positives. These functions can be used to remove
objects fewer than „p‟ pixels. The erosion and dilation
operations can be performed with the same structuring
element as mentioned in [9].The default connectivity is
8 for two dimensional image.
) Eq. (9)
3.1.2.2 Temporal Information
After the bright pixels are extracted, temporal
information can be used [13]. The past car light is used
to predict the next light position thus narrowing down
the area in which the headlight can be detected. This
method suggested in [14] helps in finding the vector of
light positions.
3.1.2.3 White Top Hat Transform
This method used in [15] is a powerful method for
detection of contrasted objects on non- uniform
background. The white hat transformation is defined as
the residue between the original image and its opening
[16]. The top hat transform is basically used to modify
the contrast of the image but the false regions can also
be removed using this principle.
) Eq. (10)
3.2. Rear Light Detection
A rear light can be detected effectively with the help of
red light filters. The threshold here is set using HSV
color space and the lights are detected [17] - [18].
Many different color spaces with widely varying
parameters have been used to segment red-color light
regions from images. The luminance(Y) and the red
component (Cr) can be combined to form a single
Keerthana Gunasekaran et al, Int.J.Computer Technology & Applications,Vol 5 (2),451-456
IJCTA | March-April 2014 Available [email protected]
453
ISSN:2229-6093
component RGC. This RGC can be used as a threshold
to extract the lights [19].
Eq. (11)
3.3. Windshield Detection
Along with the headlights, windshield detection can
also aid in the detection of vehicles in the night time. In
the night time, the vehicle consists of a dark windshield
and pair of bright headlights. After determining the
position of windshield (between two headlights), the
median value of all the pixels in the windshield is
calculated. If the median is a dark value, then the
presence of vehicle can be confirmed. In [20] for each
pixel in the image, it is assumed that it is the center of a
front windshield and then a 3D windshield model is
projected to the position of that pixel. The probability
of the vehicle presence is calculated using the shape
and edge matching likelihoods.
4. Vehicle Identification After the identification of the features, the features
should be analyzed and the vehicles should be
identified as a whole. The methods that can be used to
analyze the features are as follows:
4.1. SVM Classification
Many of the vehicle detection algorithms use SVM
Classifier to confirm the presence of vehicle in the
scene [21]-[23]. The group of pixels can be combined
together and the eigenvalues are found. These values
are given as the input to the classifier [24]. The SVM
classifiers can be trained with many attributes of the
vehicles like the
Area in component in pixels
Coordinates (u,v) of the object‟s centroid
Estimated Distance of the Object‟s Centroid
Hat Value
Rectangularity
Aspect Ratio
Length of the object‟s contour
Circularity
Angle value of each light object‟s centroid
An input vector was defined for the classifier.
The output of the support vector machine is simply the
distance from the hyper plane. This distance will help
in the making the decision of whether the analyzed
object corresponds to a vehicle or not and this can also
be used as a threshold for separating nuisance light
sources and vehicles. For creating the training and test
sets, the ratio between positive (vehicles) and negative
(mainly reflections of traffic signs and headlights) must
be set to an appropriate value in order not to produce
wrong learning or a high percentage of false positive
detections (signs classified as vehicles) during the tests.
With the help of the training and testing data sets, the
vehicle presence can be estimated accurately using the
support vector machines.
4.2. Rule Based Component Analysis
A The identified vehicle headlights can be paired with
each other if certain rules are satisfied as mentioned in
[12],[26], [27]. These rules are based on the statistical
features of the vehicle headlights. The rules that can be
used in the estimation are
The components must be horizontally close to
each other and the vertical and horizontal
positions should be considered.
They have highly overlapped vertical projection
profiles.
The components are of similar size.
The width to height ratio of the bounding box
enclosing the two components must be greater.
Area of the pixels must be similar.
The symmetry condition must be satisfied.
The components should have a high horizontal
projection profile.
4.3. Symmetry Based Identification
Symmetry can be used to filter potential vehicle
candidates and make light pairs. The light pairs which
belong to the same vehicle are always symmetrical in
size, shape and intensity in normal conditions. This
property can be used to detect the vehicles in night time
environment. The correlation factor between two lights
can be calculated and if the result is within the allowed
threshold then it is concluded that both the lights
belong to the same vehicle as mentioned in [27]. The
correlation between two components can be calculated
using
∑ ) ̅) ) )̅
Eq. (12)
Where ̅and are the mean and standard deviation of
the component T and ̅and are the mean and
standard deviation of the component I. The correlation
factor is calculated for different color channels to
utilize the color value and to get a better accuracy in
grouping headlights together as a single vehicle. This
method is mainly used to identify the vehicle in back
view after the rear light identification. Apart from the
correlation factor, the intensity of the pixels on either
side of the symmetrical axis can also be compared to
observe the symmetry between two regions.
Keerthana Gunasekaran et al, Int.J.Computer Technology & Applications,Vol 5 (2),451-456
IJCTA | March-April 2014 Available [email protected]
454
ISSN:2229-6093
The below table provides the accuracy of various
vehicle detection methods based on the previous
experiments conducted in [6]-[28].
Table -1
S.No Vehicle Detection
Methods
Accuracy False
Alarms
1 Rule Based
Identification
98% Low
2 Symmetry Based
Identification
92% Medium
3 SVM Classification 95% Low
4
Hypothesis
Generation and
Verification
94% Low
5. Conclusion This paper presented a critical review on the techniques
used in the night time vehicle detection based on the
data obtained from the stationary video surveillance
camera. These detection techniques coupled with the
tracking algorithms can increase the accuracy and
efficiency in great detail. Today many kinds of robust
tracking algorithms are available to aid vehicle
detection and traffic surveillance [29]. With the
reducing cost of the hardware and the increasing
demand for the intelligent transportation systems,
vehicle detection especially in the night when there is
heavy traffic flow will continue to be one of the hottest
research areas.
6. References [1] Wherever Z. Sun, G.Bebis, and R.Miller, “On-road
vehicle detection: A review,” IEEE Transactions
Pattern Analysis and Machine Intelligence, vol.
28, no. 5, pp. 694 –711, May 2006.
[2] C. Stauffer and W. Grimson, “Adaptive
background mixture models for real-time
tracking,” Proc. Conf. Computer Vision and
Pattern Recognition, vol. 2, pp. 246–252, June
1999.
[3] N.Greggio, A.Bernardino, C.Laschi, P.Dario and
J.S. Victor, “Self-Adaptive Gaussian Mixture
Models for Real-Time Video Segmentation and
Background Subtraction” in Proc. 10th
International IEEE conference on Intelligent
System Design and Application, November 2010.
[4] M.Celenk, J.Graham and S.Singh, "Traffic
Surveillance using Gabor Filter Bank and Kalman
Predictor", Proc. of 3rd
International Conference
on Computer Vision Theory and Applications,
January 2008.
[5] Z. Sun, G.Bebis, and R.Miller, “On-road vehicle
detection using optical sensors: A review,” Proc.
of 7th International IEEE Conference on
Intelligent Transportation Systems, October 2004.
[6] D.Neelima, G.Mamidisetti, “A Computer Vision
Model for Vehicle Detection in Traffic
Surveillance”, International Journal of
Engineering Science and Advanced Technology,
vol. 2, issue 5, pp 1203 – 1209, September 2012.
[7] N.Gajalakshmi, “Detection and pairing of Vehicle
Headlight in the Night”, International Journal of
Research in Engineering, Vol. 3, issue 9,
September 2013.
[8] K. Robert "Night-time traffic surveillance: A
robust framework for multi-vehicle detection,
classification and tracking", Proc. of 6th
International IEEE Conference on Advanced Video
Signal Based Surveillance, pp. 1 -6, September
2009.
[9] V.B. Sutar and L.S. Admuthe, “Night time Vehicle
Detection and Classification Using Support Vector
Machine”, IOSR Journal of VLSI and Signal
Processing,vol.1, issue 4, pp.01-09,December
2012.
[10] Wei-zhi Wang and Bing-han Liu, “The Vehicle
Edge Detection Based on Homomorphism
Filtering and Fuzzy Enhancement in Night-time
Environments”, Proc. of IEEE International
Conference on Intelligent Computing and
Intelligent Systems, pp. 714 – 718, October 2010.
[11] N. Otsu, "A Thresholding Selection Method from
Gray Level Histograms," IEEE Transactions on
Systems, Man and Cybernetics, vol.9, pp.62-66,
January 1979.
[12] Yen-Lin Chen, "Nighttime Vehicle Light
Detection on a Moving Vehicle using Image
Segmentation and Analysis Techniques", WSEAS
Transactions On Computers, vol. 8, issue 3, March
2009.
[13] S.Nandhini and P.G.S.Parthiban, “Automatic
Vehicle Detection during Nighttime Using Bright
Pixel Segment with Spatial Temporal Technique”,
International Journal of Electronics and Computer
Science Engineering, Vol. 1, No. 3, 2007.
[14] M.L. Eichner and T.P.Breckon, "Real-time video
analysis for vehicle lights detection using temporal
information", Proc. The Fourth European
Conference on Visual Media Production,
November 2007.
[15] K. Robert, "Video-based traffic monitoring at day
and night vehicle features detection tracking," in
Proc. of the Twelfth IEEE International
Conference on Intelligent Transportation Systems,
pp. 1-6, October 2009.
[16] P. Alcantarilla , L. M. Bergasa , P. Jimenez , M. A.
Sotelo , I. Parra , D. Fernandez and S. S.
Keerthana Gunasekaran et al, Int.J.Computer Technology & Applications,Vol 5 (2),451-456
IJCTA | March-April 2014 Available [email protected]
455
ISSN:2229-6093
Mayoral "Night time vehicle detection for driving
assistance light beam controller", Proc. IEEE
Intelligent. Vehicles Symposium, pp.291 -296, June
2008.
[17] R. O'Malley , E. Jones and M. Glavin "Rear-lamp
vehicle detection and tracking in low exposure
color video for night conditions", IEEE
Transactions on Intelligent Transportation
Systems, vol. 11, no. 2, pp.453 -462, June 2010.
[18] M. Betke and H.Nguyen,” Highway Scene
Analysis from a moving Vehicle under Reduced
Visibility Conditions”, Proc. of IEEE International
Conference on Intelligent Vehicle, October 1998.
[19] H.K.Kim, K.Sagong, J.H. Park, F.Moutarde and
H.Y.Jung, “A Color Based Real-time Vehicle
Detection for Adaptive Headlamps Control”, 2011.
[20] J. Yang, Y. Wang, A. Sowmya and Z. Li, "Vehicle
detection and tracking with low-angle cameras," in
Proceedings of the 17th IEEE International
Conference on Image Processing (ICIP), pp. 685-
688, September 2010.
[21] J. Zhou , D. Gao and D. Zhang "Moving vehicle
detection for automatic traffic monitoring", IEEE
Transactions on Vehicular Technology, vol.
56, no. 1, pp.51 -59, January 2007.
[22] T.H.Thi, K.Robert, S.Lu and J.Zhang,” Vehicle
classification at nighttime using Eigenspaces and
Support Vector Machine”, National ICT of
Australia, 2007.
[23] H.-K Kim, S. Kuk, Mk.Kim and H-Y Jung, “An
Effective Method of Head lamp and Tail Lamp
Recognition for Night Time Vehicle Detection”,
World Academy of Science Engineering and
Technology, Vol. 44, August 2010.
[24] T.H.Thi, K.Robert, S.Lu and J.Zhang, “Vehicle
Classification at Nighttime using Eigenspaces and
Support Vector Machine”, National ICT of
Australia.
[25] J. Firl, M. Hoerter, M. Lauer, and C. Stiller,
“Vehicle detection, classification and position
estimation based on monocular video data during
night-time,” Proc. of the 8th Int. Symposium on
Automotive Lighting., 2009.
[26] S.Zhou, J.Li, Z.Shen and L.Ying, “A Night time
Application for a Real-Time Vehicle Detection
Algorithm Based on Computer Vision”, Research
Journal of Applied Sciences, Engineering and
Technology, March 2013.
[27] J.Wng, X.Sun and J.Guo, “A Region Tracking
Based Vehicle Detection Algorithm in Nighttime
Traffic Scenes”, Proc. of IEEE Sensors, pp. 16474-
16493, November 2013.
[28] R. Cucchiara and M. Piccardi, "Vehicle Detection
under Day and Night Illumination", in Proc. of
IIA-99 – 3rd
International ICSC Symposium on
Intelligent Industrial Automation, Special Session
on Vision Based Intelligent Systems for
Surveillance and Traffic Control, pp. 789-794,
June 1999.
[29] S.Sivaraman and M.M.Trivedi, ”Looking at
Vehicles on the Road: A Survey of Vision-Based
Vehicle Detection, Tracking and Behavior
Analysis”, IEEE Transactions on Intelligent
Transportation Systems, vol. 14, issue 4, pp. 1773-
1795, December 2013.
Keerthana Gunasekaran et al, Int.J.Computer Technology & Applications,Vol 5 (2),451-456
IJCTA | March-April 2014 Available [email protected]
456
ISSN:2229-6093