Abstract—Sliding window approach used in conventionalpedestrian detection samples huge number of sub-windows froman input image to extract features required for classification.In this paper, we propose computationally efficient pedestriandetection based on Markov Chain Monte Carlo. In order toapply Markov Chain Monte Carlo for pedestrian detection, aproposal distribution and a target distribution must be defined.In the proposed method, we define them by random-walk andpedestrian likelihood, respectively. Experimental results showthat the proposed method can detect pedestrians as accurateas a conventional method that uses fixed sampling intervaleven though the proposed method uses only 55.57% samplesand requires 58.17% of computing time compared with theconventional method.
I. INTRODUCTION
Pedestrian detection from visual images is a recent challeng-ing problem in the field of computer vision. Many variationsof pedestrians appearance, such as their clothes, poses, andillumination, make it difficult to distinguish pedestrians fromother objects. For accurate pedestrian detection, selection andcombination of feature descriptors and classification algorithmare important issues. As the methods of feature descrip-tors, many schemes have been proposed, such as Haar-likemethod [19], local self-similarities method [14], contour-basedmethods [8], [15], and gradient-based methods [1], [9].
Among these works, detection algorithms based-on his-tograms of oriented gradients (HOG) [1] are now popularlyused for pedestrian detection. Many HOG-based detectionalgorithms are proposed because of its robustness on de-formation and illumination change. Recently Co-occurrencehistograms of oriented gradients (CoHOG) [20] achieved ex-cellent detection performance as accurate as other non-HOG-based schemes [18], [13], [10]. Most of conventional pedes-trian detection methods combine the above feature descriptorsand sliding window sampling that extracts huge number ofsub-windows for generating feature descriptors in raster scanorder from input image as shown in Fig. 1.
On the other hand, real-time processing of pedestrian detec-tion is indispensable for practical applications such as driverassistance and video surveillance. In order to achieve real-time pedestrian detection without degrading detection accuracyon embedded systems a computationally efficient classifica-tion scheme using cascade classifier [4], parallel processingusing multi-core processors [12], and specialized hardware
engines [6], [7], [5] are proposed. However, computationalcosts to calculate CoHOG features and classify them are toolarge even for specialized hardware engines because dimensionof the feature descriptor is very high (about 35,000 dimensionsif the window size is 18× 36). To solve this problem, a novelmethod to reduce computational costs is needed.
In this paper, to reduce the computational cost of pedestriandetection, we propose a novel pedestrian detection methodusing efficient sampling based on Markov Chain MonteCarlo [3], [2], [11], which generates proper samples from thegiven probability distribution and calculates various expecta-tion values by using convergence of stochastic process. In theproposed scheme, Metropolis-Hastings algorithm, widely usedMCMC, is used as sampling method, which is expected tocontrol the number of sampling without degradation of detec-tion accuracy. The target distribution and proposal distributionof Metropolis-Hastings are defined as pedestrian likelihoodand random-walk, respectively. In order to examine detectionperformance, the proposed scheme is implemented on CPU.
II. PEDESTRIAN DETECTION BASED ON SLIDING WINDOWAPPROACH
This section shows the relation between sampling intervaland detection accuracy of pedestrian detection based on slidingwindow approach.
As shown in Fig. 1, sliding window approach extractsfeature vectors with a fixed sampling interval from an inputimage. The computational cost of detection can be controlledby adjusting the sampling interval. However, if the samplinginterval broaden as so to reduce computational cost, thedetection accuracy is conspicuously degraded.
To examine the relation between a sampling interval anddetection accuracy, we have done preliminary experiments ofCoHOG-based detection at various sampling intervals usingpedestrian dataset provided by INRIA [1]. Fig. 2 shows theresults of them.
These results show that Detection Error Trade-off (DET)curve, which represents the detection accuracy, is improved asthe sampling interval gets dense, though it is usually expectedthat the decrease of detection miss rate and the increase offalse detection rate occur simultaneously. This is becausetrue positives tend to appear densely near the pedestrianregions, but false positives tend to appear discretely due to the
Fig. 1. The sampling sequence of sliding window approach
100
100
Mis
s ra
te
FPPW (false positives per window)
32 pixel step16 pixel step
8 pixel step4 pixel step2 pixel step
Fig. 2. DET curve at several sampling intervals
characteristic of CoHOG-based classification. Therefore, if thesampling interval is dense enough, it is expected to enhancethe detection accuracy. However, in real-time applications,the dense sampling is not always good solution becauseit increases the number of samples and the computationalcomplexity. To solve the above problem, in this paper, wepropose a computationally efficient scheme that enables thereduction of the computational costs without degradation ofdetection accuracy based on Markov Chain Monte Carlo anddemonstrate it by using the INIRIA dataset.
III. PEDESTRIAN DETECTION USING MCMC
Shortly, this section introduces Markov Chain Monte Carlo(MCMC), and proposes a novel sampling method for pedes-trian detection using Metropolis-Hastings, which is widelyused MCMC.
A. Markov Chain Monte Carlo
MCMC can generate proper samples from the given prob-ability distribution and calculate various expectation valuesby using convergence of stochastic process, called Markovprocess. Although diverse methods exist in MCMC, suchas Metropolis-Hastings algorithm [3], Gibbs sampling [2],Slice sampling [11], etc., in this paper, Metropolis-Hastingsalgorithm is adopted.
Metropolis-Hastings is iterative algorithm, and can drawsamples from any probability distribution π(~x), called targetdistribution. In each iteration, it generates candidate samplesfrom current state by using conditional probability distributionq(~y|~x), called proposal distribution, and decides whether toaccept each candidate by using Acceptance-Rejection method.
Metropolis-Hastings algorithm with n iterations can berepresented by the following procedure.
• Set initial sample ~x(0),• repeat the following operations at t (t = 0, 1, . . . , n),
1) generate candidate ~y from proposal distributionq(~y|~x(t)),
2) and generate random number u from uniform ran-dom generator U(0, 1),
~x(t+1) =
{~y when u ≤ α(~x(t), ~y)
~x(t) in other cases, (1)
(2)
α(~x, ~y) = min(1,π(~y)q(~x|~y)
π(~x)q(~y|~x)), (3)
where the value of α(~x, ~y), called acceptance probability,depends only on the ratio of π(~y) to π(~x).
B. Pedestrian detection using Metropolis-Hastings
To apply Metropolis-Hastings algorithm for pedestrian de-tection problem, initial samples, target distribution, and pro-posal distribution must be defined. This subsection explainshow to define these conditions in the proposed method.
In the proposed method, the coordinates of initial samplesare given as identical sparse lattice points for same resolutionimages.
Target distribution π(~x) is defined by pedestrian likelihoodcalculated from CoHOG features and Support Vector Machine.In order to use the pedestrian likelihood as target distributionπ(~x), pedestrian likelihood must satisfy following conditions.First, pedestrian likelihood must be capable of representing thedifference of pedestrian and non-pedestrian clearly. Second,pedestrian likelihood must be non-negative because proposaldistribution is a probability function.
880
100
100
Mis
s ra
te
FPPW (false positives per window)
16 pixel stepCPU MCMC 16 iter.
8 pixel stepCPU MCMC 32 iter.
Fig. 3. The comparison of sliding window and MCMC detection accuracy
In our proposal, target distribution is defined as follows.
π(~xf ) = exp(d(~xf )), (4)
where ~xf represents a feature vector at coordinate (x, y) andscale s, and d represents the signed distance between featurevector and decision hyperplane in feature space. This likeli-hood distribution is the same function that is used in pedestriantracking based on the particle filter proposed in [16], [17],which is proposed by a part of the authors.
In the proposed method, random-walk chain is applied asthe proposal distribution q(~y|~x). Random-walk chain is usuallyrepresented by the following equation.
~y = ~x + ~ε, (5)
where ~ε is independent of x, and the expectation value is 0. Inthis paper, normal distribution is used as the ~ε. Here, random-walk chain satisfies the following equation.
q(~y|~x) = q(~x|~y). (6)
By substituting eq. (6), the eq. (3) is simplified as
α(~x, ~y) = min(1,π(~y)
π(~x)). (7)
As a result, the computational cost for Metropolis-Hastingsalgorithm is decreased.
IV. EVALUATION
The detection accuracy and the computational costs areexamined with pedestrian dataset provided by INRIA.
To evaluate detection accuracy, four DET curves are com-pared in Fig. 3, where the solid line with square points, thedotted line with square points, the solid line with X points,and the dotted line with X points are acquired from 16 pixelstep sliding window, 16 times MCMC iteration, 8 pixel stepsliding window, and 32 times MCMC iteration. In Fig. 3, theproposed method using 32 times MCMC iteration shows thebest accuracy in these four methods.
Examples of detection results are shown in Fig. 4 to Fig. 7.Fig. 4 and Fig. 6 are the results of conventional sampling
Processing time (ms) 23385.80 6003.12 6971.75 13604.50
method using 8 pixel fixed sampling interval. Both of themshow detection failures for some pedestrian area. However, inFig. 5 and Fig. 7, the results of the proposed method using32 times MCMC iteration, almost all pedestrians are detectedcorrectly.
TABLE I shows the number of samples and Processingtime of each method, which indicate computational costs. Thevalues of TABLE I are the average result of 10 times operationusing Intel Core i7 CPU 975 3.33GHz, Linux kernel 2.6.32,and 1060 × 605 images.
The experimental results show that the proposed methodusing 32 times MCMC iteration is more accurate than theconventional method using sliding-window with 8 pixel fixedsampling interval, even though the proposed method used only55.57% samples and is computed in 58.17% of time comparedto the conventional method. Additionally, the processing timerelated to MCMC calculation itself is nearly 0.0% comparedwith the total processing time from profile result.
V. CONCLUSION
This paper proposed an efficient sampling scheme forpedestrian detection based on Metropolis-Hastings algorithm,widely used MCMC, aiming to maximize the detection ac-curacy under the constraint of computational cost. To ap-ply Metropolis-Hastings algorithm for pedestrian detection,it is required to define initial samples, proposal distribution,and target distribution. In this paper, the coordinates of ini-tial samples of the proposed method are given as identicalsparse lattice points for same resolution images for stableperformance, and adopted random-walk chain and pedestrianlikelihood as proposal and target distributions, respectively.Experimental results show that the proposed method can detectpedestrians as accurate as a conventional method, which usesfixed sampling interval, even though the proposed method usedonly 55.57% samples and it’s computation time is 58.17% oftime compared to the conventional method. From the results,the increase of the computational complexity due to MCMCcalculation itself is negligible compared to the total process.
The proposed scheme shows that the computational costcan be reduced efficiently without degradation of detectionaccuracy. However, the experimental results indicate a problemis still remained, which is that some sub-windows classified aspedestrians include one or more sub-windows in themselves.This is caused by insufficient clustering operated as post-processing after detection. On the other hand, over clusteringmay merge several sub-windows that should be recognizedas different pedestrians. In the future, the authors will try tomake a clustering scheme suitable for the proposed detectionscheme using MCMC in order to realize accurate pedestriandetection on practical embedded systems.
881
Fig. 4. Detection result of the conventional method (crop001555.png) Fig. 5. Detection result of the proposed method (crop001555.png)
Fig. 6. Detection result of the conventional method (person 347.png) Fig. 7. Detection result of the proposed method (person 347.png)
ACKNOWLEDGEMENT
This work is partly supported by Grant-in-Aid for YoungScientists (B) 20700048, JSPS Fellows 21·2488, and JSPSFellows 22·6556.
REFERENCES
[1] N. Dalal and B. Triggs. Histograms of oriented gradients for humandetection. In Proc. of CVPR, volume 1, pages 886–893, 2005.
[2] S. Geman and D. Geman. Stochastic relaxation, gibbs distributionsand the bayesian restoration of images. IEEE Transactions on PatternAnalysis and Machine Intelligence, 6(6):721–741, November 1984.
[3] W. K. Hastings. Monte carlo sampling methods using markov chainsand their applications. Biometrika, 57(1):97–109, April 1970.
[4] M. Hiromoto and R. Miyamoto. Cascade classifier using dividedCoHOG features for rapid pedestrian detection. In Proc. of the 7thInternational Conference on Computer Vision Systems, pages 53–62,2009.
[5] M. Hiromoto and R. Miyamoto. Hardware architecture for high-accuracyreal-time pedestrian detection with CoHOG features. In Proc. of the12th International Conference on Computer Vision Workshops, pages894–899, 2009.
[6] M. Hiromoto, H. Sugano, and R. Miyamoto. Partially parallel architec-ture for adaboost-based detection with haar-like features. IEEE Trans.on CSVT, 19(1):41–52, 2009.
[7] R. Kadota, H. Sugano, M. Hiromoto, H. Ochi, R. Miyamoto, andY. Nakamura. Hardware architecture for hog feature extraction. InProc. of the Fifth International Conference on Intelligent InformationHiding and Multimedia Signal Processing, 2009.
[8] M. Leordeanu, M. Hebert, and R. Sukthankar. Beyond local appearance:Category recognition from pairwise interactions of simple features. InProc. of CVPR, 2007.
[9] D. Lowe. Distinctive image features from scale-invariant keypoints. Int’lJournal of Computer Vision, 60(2):91–110, 2004.
[10] S. Maji, A. Berg, and J. Malik. Classification using intersection kernelsupport vector machines is efficient. In Proc. of CVPR, 2008.
[11] R. Neal. Slice sampling. Annals of Statistics, 31:705–767, 2000.[12] V. Prisacariu and I. Reid. fastHOG - a real-time GPU implementation of
HOG. Technical Report 2310/09, Department of Engineering Science,Oxford University, 2009.
[13] P. Sabzmeydani and G. Mori. Detecting pedestrians by learning shapeletfeatures. In Proc. of CVPR, 2007.
[14] E. Shechtman and M. Irani. Matching local self-similarities acrossimages and videos. In Proc. of CVPR, 2007.
[15] J. Shotton, A. Blake, and R. Cipolla. Multiscale categorical objectrecognition using contour fragments. IEEE Trans. on Pattern Analysisand Machine Intelligence, 30(7):1270–1281, 2008.
[16] H. Sugano and R. Miyamoto. Parallel implementation of pedestriantracking using multiple cues on GPGPU. In Proc. of the 12th Inter-national Conference on Computer Vision Workshops, pages 900–906,2009.
[17] H. Sugano, R. Miyamoto, and Y. Nakamura. Optimized parallelimplementation of pedestrian tracking using HOG features on GPU.In Proc. of the 6th Conference on Ph.D. Research in Microelectronics& Electronics, to appear.
[18] O. Tuzel, F. Porikli, and P. Meer. Pedestrian detection via classificationon Riemannian manifolds. IEEE Trans. on Pattern Analysis and MachineIntelligence, 30(10):1713–1727, 2008.
[19] P. Viola, M. Jones, and D. Snow. Detecting pedestrians using patterns ofmotion and appearance. Int’l Journal of Computer Vision, 63:153–161,2005.
[20] T. Watanabe, S. Ito, and K. Yokoi. Co-occurrence histograms of orientedgradients for pedestrian detection. In Proc. of PSIVT, pages 37–47, Jan2009.
