Research ArticleVisual Tracking via Feature Tensor MultimanifoldDiscriminate Analysis
Ting-quan Deng1 Jia-shu Dai2 Tian-zhen Dong2 and Ke-jia Yi3
1 College of Science Harbin Engineering University Harbin 150001 China2 College of Computer Science and Technology Harbin Engineering University Harbin 151001 China3 Science and Technology on Underwater Acoustic Antagonizing Laboratory Systems Engineering Research Institute of CSSCBeijing 100036 China
Correspondence should be addressed to Jia-shu Dai jiashu dai163com
Received 25 June 2014 Accepted 25 August 2014 Published 9 November 2014
Academic Editor Guoqiang Zhang
Copyright copy 2014 Ting-quan Deng et al This is an open access article distributed under the Creative Commons AttributionLicense which permits unrestricted use distribution and reproduction in any medium provided the original work is properlycited
In the visual tracking scenarios if there are multiple objects due to the interference of similar objects tracking may fail in theprogress of occlusion to separation To address this problem this paper proposed a visual tracking algorithm with discriminationthrough multimanifold learning Color-gradient-based feature tensor was used to describe object appearance for accommodationof partial occlusion A prior multimanifold tensor dataset is established through the template matching tracking algorithm Forthe purpose of discrimination tensor distance was defined to determine the intramanifold and intermanifold neighborhoodrelationship in multimanifold space Then multimanifold discriminate analysis was employed to construct multilinear projectionmatrices of submanifolds Finally object states were obtained by combiningwith sequence inferenceMeanwhile themultimanifolddataset and manifold learning embedded projection should be updated online Experiments were conducted on two realvisual surveillance sequences to evaluate the proposed algorithm with three state-of-the-art tracking methods qualitatively andquantitatively Experimental results show that the proposed algorithm can achieve effective and robust effect inmulti-similar-objectmutual occlusion scenarios
1 Introduction
Visual tracking is an important research area in computervision and pattern recognition which can be applied tomanydomains such as visual surveillance trafficmonitoringhuman computer interaction image compression three-dimension reconstruction and weapons automatically track-ing combat To make these applications viable the results ofvisual tracking must be robust and precise
Visual tracking is a challenging problem due to objectappearance variations Many issues can cause object appear-ance variations including cameramotions camera viewpointchanges environmental illumination changes noise distur-bance background clutter pose variation and object shapedeformation and occlusions occur [1]
11 Related Works In recent years there are a wide rangeof tracking algorithms to deal with these object appearance
variations These algorithms can be roughly classified intotwo categories according to the model-construction mecha-nism which are generative and discriminative methods
The generative methods mainly focus on how to robustlydescribe the appearance model and then find the best match-ing appearance model of image patch with that of the objectThe classical template matching tracking algorithm can beviewed as the generative model The earliest template-basedtracking method dates back to the Lucas-Kanade algorithmThe eigen-tracking [2] algorithm demonstrated that trackingcan be considered as finding the minimum distance from theappearance model of tracked object to that of the subspacerepresented Matthews et al [3] show how to update thetemplate which can avoid the ldquodriftingrdquo inherent in the naivemethod The IVT [4] tracking algorithm utilizes subspacelearning to generate a low-dimensional object appearanceand incrementally update it Hu et al [5] proposed a visualobject tracking algorithm which models appearance changes
Hindawi Publishing CorporationMathematical Problems in EngineeringVolume 2014 Article ID 787093 12 pageshttpdxdoiorg1011552014787093
2 Mathematical Problems in Engineering
by incrementally learning a tensor subspace representationIn the tracking procedure the sample mean and an eigen-basis for each unfolding matrix of the tensor are adaptivelyupdated The classical mean-shift [6] tracker uses histogramas the appearance model then the mean-shift procedureis achieved to locate the object The Fragtrack [7] utilizesseveral fragments to design the appearance model which canhandle pose change and partial occlusion The ℓ
1-tracker [8]
casts tracking problem as sparse approximation where theobject is modeled by a sparse linear combination of targetand a set of trivial templates The sparse representation isobtained by solving an ℓ
1-regularized optimization least-
squares problem and the posteriori probability of candidateimage patch belonging to the object class is inversely pro-portional to the residual between the candidate image patchand the reconstructed oneThe ℓ
1-APG tracker [9] developed
the ℓ1-tracker that not only runs in real-time but also
improves the tracking accuracy The S-MTT [10] algorithmregularizes the appearance model representation problememploying sparsity-inducing ℓ
119901119902mixed norms which can
handle particles independentlyThe discriminative methods treat visual tracking as a
binary classification problem It aims to separate the objectfrom its surrounding complex background with a smalllocal region There are many newly proposed visual trackingalgorithms based on boosting classifier because of its pow-erful discriminative learning capabilities Online boostingalgorithmhaswide applications in object detection and visualtracking Grabner et al [11] proposed an online boostingtracker which is firstly given a discriminative evaluation ofeach feature from a candidate feature pool Then onlinesemisupervised boosting method [12] is proposed for thepurpose of alleviating the object drifting problem in visualtracking Ensemble tracking [13] uses weak classifiers toconstruct a confidence map by pixel classification to distin-guish between the foreground and the background The MILtracker [14] represents an object by a set of samples thesesamples corresponding to image patch are considered withinpositive and negative bags Then multiple instance boostingis used to overcome the problem that slight inaccuracies inlabeled training examples can cause object drift However thetracking may fail when the training samples are imprecisePointing to this problem the WMIL tracker [15] whichintegrates the sample important into the multiple instancelearning is proposed The SVM tracker [16] combined sup-port vector machine into optical flow to achieve visualtracking A visual tracking algorithm via an online featureselection mechanism for evaluating multiple object featuresis proposed in [17] The VTD algorithm [18] designs theobservation andmotionmodel based on visual tracking com-position schemeThe TLD tracker [19] explicitly decomposesthe long-term tracking problem into three componentswhichare tracking learning and detection The CT tracker [20]extracted the sparse image feature combined with a naiveclassifier to separate the object from the background
In the multiple moving objects scenarios with the move-ment of one object the reflected lights of other objects whichreach to the camera lens may be hindered making otherobjectsrsquo projection imaging incomplete or even completely
invisible on the imaging planeWhen the occlusion occurredif the tracking object is similar to the occlusion object theobject is vulnerable to the similar objects influence in theprogress of occlusion to separation which can cause driftThus it is necessary to distinguish the tracking object withthe potential similar objects in the scenarios Meanwhilewhen the object is partially occluded the information fromunoccluded part has a large reference value of determiningthe object state Therefore the object feature must maintainthe structural relationship of the original space This paperproposed a visual object tracking algorithm for multiple sim-ilar objects mutual occluded problem which combines thesetwo ideas First of all a feature function is designed for thepurpose of extracting the tensor feature which can maintainthe spatial structure of the object The multimanifold tensordata set is collected by template matching tracking algorithmin the initial few frames A tensor distance is defined todetermine the intramanifold and intermanifold neighbor-hood relationship The object feature tensor is embeddedinto a low-dimensional space by multimanifold discriminateanalysisThen the object state in the next frame is obtained byBayesian sequence inference Considering the changes in theobject appearance an update strategy for the multimanifoldset is needed to be set
12 Plan of the Paper This paper is organized as follows inthe next sectionwe first introduce the notation of tensor alge-bra and feature tensor After that multimanifold discriminateanalysis is reviewed in Section 3 Section 4 details the visualtracking framework In Section 5 comparative experimentalresults and analysis are showed and conclusions are drawnin Section 6
2 Feature Tensor
A tensor is a high-order array which can be maintained theoriginal spatial structure of an object Construct a featuretensor from an object appearance can increase trackingaccuracy
21 Tensor Algebra Tensor can be viewed as multiorderarray which exists in multiple vector spaces the algebracorresponding to tensor is the mathematical foundation ofmultilinear analysis [21] An 119873-order tensor is denoted asX isin R119868
1times1198682timessdotsdotsdottimes119868
119873 each elements in this tensor is representedas 119909
22 Feature Tensor The object appearance image from RGBcolor video sequence is a three-dimensional data whichformed a nature tensor structure The color and edge infor-mation of the object have a better discrimination on theobject class the gradient feature can describe the object edgeinformation For a detailed description of object informationthe feature function of an object appearance image is definedas follows
direction gradients on the 119877 119866 and 119861 color channelsEach pixel (119894 119895) on object appearance image corresponds
to a twelve-dimensional feature vector the size 119886times119887times3 objectappearance image corresponds to a X isin R119886times119887times12 featuretensor
3 Multimanifold Discriminate Analysis
The basic assumption of the manifold learning is that high-dimensional datum can be considered as geometric correla-tion points which lie in low-dimensional smooth manifoldThere is usually a submanifold structure corresponding to asingle object class different objects lie in different subman-ifolds The multimanifold discriminate analysis can projectthe tensor data which is from a submanifold into a low-dimensional space
31 Multimanifold Neighborhood Relationship of Feature Ten-sor The appearance of each object under different poses isusually composed of a submanifold the multiple differentobject appearance spaces formed the multimanifold Eachmoving object appearance image in video sequence canextract a feature tensor X isin R119886times119887times12 The set of feature tensorcalculated by the appearance images from the first 119898 framesis denoted as119872
119894= X
1198941 X
1198942 X
119894119898 then119872
119894can be seen as
a submanifold Because of the presence of multiple movingobjects in the scenarios the set of each submanifold 119872 =
1198721119872
2 119872
119899 is a multimanifold dataset [22]The entries
X11989411198942sdotsdotsdot119894119873
(1 le 119894119895le 119868
119895 1 le 119895 le 119873) in X are corresponding to
the 119897th element in x where
119897 = 1198941+
119873
sum
119895=2
(119894119895minus 1)
119895minus1
prod
119900=1
119868119900
(2 le 119895 le 119873) (5)
The distance between two tensors X and Y is (the order anddimension of X and Y are the same)
119889119879(X Y) = radic
1198681times1198682timessdotsdotsdottimes119868
119873
sum
119897119898=1
119892119897119898
(x119897minus y
119897) (x
119898minus y
119898) (6)
where 119892119897119898
is the measurement coefficient Since there are toomany entries in the tensor data the measurement coefficientis defined by the distance of points which have spatialneighborhood relationship Consider
119892119897119898
= 119890minus119901119897minus1199011198982
221205902
if x119898isin 119873
1198961015840 (x
119897)
0 else(7)
where 120590 is the regularization parameter and 119901119897minus 119901
1198982is the
location distance between x119897and x
119898 If x
119897and x
119898 respectively
correspond to the X11989411198942sdotsdotsdot119894119873
and X1198941015840
11198941015840
2sdotsdotsdot1198941015840
119873
in tensor X then
1003817100381710038171003817119901119897 minus 119901119898
10038171003817100381710038172 =radic(119894
1minus 119894
1015840
1)2
+ (1198942minus 119894
1015840
2)2
+ sdot sdot sdot + (119894119873minus 119894
1015840
119873)2
(8)
The 1198701intramanifold neighborhood 119873
1198701
intra(X119894119895) of the ten-
sor X119894119895is as follows calculate the tensor distance 119889
119895119897=
119889119879(X
119894119895 X
119894119897) 119895 = 119897 between the tensor X
119894119895in submanifold 119872
119894
and another tensor X119894119897in this submanifold then the nearest
1198701intramanifold neighborhood of X
119894119895can be obtained
according to the tensor distance 119889119895119897
The 1198702intermanifold neighborhood 119873
1198702
inter(X119894119895) of the
tensor X119894119895is as follows calculate the tensor distance 119889
119895119904=
119889119879(X
119894119895 X
119897119904) 119894 = 119897 between the tensor X
119894119895in submanifold 119872
119894
and tensor X119897119904(119897 = 119894) in another submanifold 119872
119897(119897 = 119894)
then the nearest 1198702intermanifold neighborhood of X
119894119895can
be obtained according to the tensor distance 119889119895119904
Themultimanifold dataset and its neighborhood relation-ship are shown in Figure 1
As can be seen fromFigure 1 there are four initial movingobjects in the scenarios thus constructing four submani-folds which are 119872
1 119872
2 119872
3 119872
4 these four submanifolds
formed a multimanifold The intramanifold neighborhoodrelationship of tensor X
13in submanifold119872
1is X
12 X
14 X
17
the intermanifold neighborhood relationship of this tensor isX22 X
24 X
41 X
43 X
44
32 Multimanifold Discriminate Analysis The objective ofmanifold learning is to recover the low-dimensional structurefrom the high-dimensional datum space and find a low-dimensional embedding map In the multiple similar objectsscenarios it is hoped that the extracted object feature candistinguish the object and the potential similar objects inthe scenarios The objective of multimanifold learning isthat the difference between a tensor and intramanifoldneighborhood points decreases and the difference betweenthe tensor and intermanifold neighborhood points increases
4 Mathematical Problems in Engineering
Manifold margin
M1
M2
M3
M4
X22 X24
X12X13
X17
X14
X41X43
X44
Figure 1 Multimanifold dataset and neighborhood relationships
in the embedded space Considering these the objectivefunction of multimanifold discriminate analysis is
argmaxU1U2U3
119891 (U1U
2U
3)
= 119891inter (U1U
2U
3) minus 119891intra (U1
U2U
3)
(9)
where U1 U
2 U
3are the multilinear projection matrices
in the first-order second-order and third-order which arecorresponding to the tensor in the submanifold119872
To maximize the 119891(U119901) by solving the eigen-value equation
(Ainter minus Aintra) 119906119901 = 120582119906119901 (16)
obtain U119901
The eigen-values are 1205821
ge 1205822
ge sdot sdot sdot ge 1205821198891015840 ge 0 ge
1205821198891015840+1
ge sdot sdot sdot ge 120582119889 the corresponding eigen-vector of eigen-
value 120582119901is [119906
1199011
1199061199012
119906119901119889
] where 119889 is the dimension 119901thorder in the original feature tensor from submanifold 119872
119894
The directional projection positive along the eigen-vector 119906119901119897
which is corresponding to the eigen-value 120582119897of (AinterminusAintra)
is positive that is intermanifold neighborhood distanceof tensors is bigger than the intramanifold neighborhooddistance which are projected along this direction Thereforethe projection matrix U
119901= [119906
1199011
1199061199012
1199061199011198891015840] consists
of all of the eigen-vectors which are corresponding to thepositive eigen-values Thus the tensor data which are insubmanifold 119872
119894can be embedded in a low-dimensional
space via multilinear projection matrix U1 U
2 U
3 In this
lower-dimensional space the difference between tensor dataand its intramanifold neighborhood points decreases and thedifference between it and its intermanifold neighborhoodpoints increases so that the distinguishing ability between theobject and the similar ones is greater
4 Visual Tracking Framework
In order to achieve tracking of an object in scenariosBayesian sequence inference is used to obtain the object finalstate Meanwhile the multi-manifold datasets and the multi-linear projection matrice which are calculated from multi-manifold discriminate analysis should be updated
41 Sequence Inference In the visual tracking problem themovement of the object is unable to predict the object statein the current frame only related to that in the prior framethen the visual tracking process satisfies the Markov process[23] A bounding box 119900
119905= (119909
119905 119910
119905 119908
119905 ℎ
119905) is used to describe
the object state at the 119905th frame where (119909119905 119910
119905) 119908
119905 ℎ
119905denote
the upper left corner coordinate the width and height of thebounding box
Given a set of observed object appearance images 119878119905=
1199041 119904
2 119904
119905 the objective of visual tracking is to obtain the
optimal estimate value of the hidden state variables 119900119905 There
is a similar result as that of the object state which is obtainedaccording to Bayesrsquo theorem Consider
119875 (119900119905| 119878
119905) prop 119875 (119904
119905| 119900
119905) int119875 (119900
119905| 119900
119905minus1) 119875 (119900
119905minus1| 119878
119905minus1) 119889119900
119905minus1
(17)
where 119875(119900119905| 119900
119905minus1) refers to the state transition model and
119875(119904119905| 119900
119905) refers to the observation model According the
observation model 119875(119904119905
| 119900119905) we can obtain the tracking
results
State Transition Model This was used to model the move-ment of object between consecutive frames Because of theirregular movement of object the object state is difficult topredict and the moving speed of the object is not very fast Itis considered that the object state in the current frame is nearto that in the prior frameThen the object state 119900
119905is modeled
by independent Gaussian distribution around its counterpartin state 119900
119905minus1 described as
119875 (119900119905| 119900
119905minus1) = 119873 (119900
119905 119900
119905minus1 Σ) (18)
where Σ means the diagonal covariance matrix correspond-ing to the variables 119909
119905 119910
119905 119908
119905 ℎ
119905 and the elements are 1205902
119909 1205902
119910
1205902
119908 1205902
ℎ 119873 particles can be randomly generated pointing to
Gaussian distribution Each particle corresponds to an objectstate then 119873 particles can obtain multiple states 119900
119894
119905 119894 =
1 2 119873 During the visual tracking process the morethe particles we generated are the more accurate the objectstate estimate was but at the same time the computationalefficiency was low For the purpose of efficient and effectiveof the visual tracking algorithm there is a balance soughtbetween these factors
Observation Model This was used to measure the differ-ence between the appearance observation and the objectappearance model Given a drawn particle state 119900
119894
119905and the
corresponding cropped image patch 119911119894
119905in the frame image 119868
119905
the probability of an image patch being generated from thesubmanifold space is inversely proportional to the differencebetween image patch and the appearance model and couldbe calculated between the negative exponential distance ofthe projected data and the weighted center of submanifoldConsider
119901 (z119895119905| 119900
119905) = exp
minus
(10038171003817100381710038171003817(z119895
119905minus 119862
119894) times
1U1times2U2times3U3
10038171003817100381710038171003817119865)
1205902
(19)
6 Mathematical Problems in Engineering
where120590 indicates the bandwidth sdot 119865is the Frobenius norm
andU1U
2U
3are themultilinear projectionmatrix of the 119894th
object in submanifold119872119894
The state 119900119894
119905corresponding to the maximum 119901(119911
119894
119905|
119900119905) is the optimal object state at the 119905th frame Let 120576 =
(z119895119905minus 119862
119894) times
1U1times2U2times3U3119865represent the error between
feature tensor which is calculated by observation z119895119905and the
weighted center 119862119894of submanifold119872
119894
42 Multimanifold Data Sets Update The appearance imageof the object changeswith themovement of it in the scenariosthe submanifold of the object should have different postureobject appearance feature tensors Therefore the multiman-ifold data set should be updated in the tracking processBecause of the factors such as occlusion and so forth whichinfluence the object appearance the appearance images ofthe tracked object have the non-object information thenobtained object feature tensor will not be in the submanifoldTherefore the update strategy is necessary From the perspec-tive of the human sensory vision the appearance informationof object changes in the process of occlusion the changes ofobject between consecutive frames are bigger or the objectfeature tensor is far awaywith the center of submanifold in theembedded space while the changing information betweenconsecutive frames is small or the object feature tensor is nearthe center of submanifold in the embedded space that is theobject state is well determined
The image first-order entropy is used to describe the grayvalue distribution of the object image but not to considerit spatial distribution while the image second-order entropyuses the 2-tuple feature (119894 119895) which is calculated by spatialdistributionThe image second-order entropy could describethe changes of the object where 119894 is the gray value (0 le 119894 le
255) and 119895 is the neighborhood gray value (0 le 119895 le 255)119901119894119895
= 119891(119894 119895)119886119887 denotes the gray value and neighborhoodgray distribution where119891(119894 119895) is the counts of the occurrenceof the 2-tuple feature and 119886119887 is the size of imageThe second-order entropy is defined as
119867 =
255
sum
119894=0
119864119894=
255
sum
119894=0
119901119894119895ln119901
119894119895 (20)
Thedifference of the object in consecutive frames is describedby the second-order entropyWhen the second-order entropydifference of the object image in consecutive frames is biggerthe objectmaybe occluded Simultaneously the feature tensorof appearance image would be far away from the weightedcenter of submanifold namely the error is bigger As shownin Figure 2 the object is largely occluded at the frames 33ndash46and 48ndash63 and small part occluded at the frames 69ndash77
For a best state 119900119905of object 119894 which is newly obtained
when the difference of second-order entropy with the priorframe 119867
119889lt 120575119867
119889and the error in low-dimensional tensor
space embedded 120576 gt 120575120576119872119894
the feature tensor calculatedby the newly obtained object state 119900
119905should add into the
submanifold 119872119894 where 119867
119889is mean of the difference of
second-order entropy 120576119872119894
is the mean of the errors and 120575 isthe adjustment factor which takes 12 in this experiment
When the tensor number in a submanifold 119872119894is the
multiples of the initial number the multimanifold discrimi-nate analysis is computed on the newmultimanifold datasetsthen the weighted center of submanifold and multilinearprojectionmatrices are updatedThere will be a small portionof the determined object data abandoned but the tensorswhich added into the data set are essentially the featuretensors of object appearance
The whole tracking algorithm is working as follows
(1) Locate the object state in the first frame eithermanually or by using an automated detector
(2) Tracking objects use template matching trackingalgorithm in the initial119898 frames
1 sdot sdot sdot 119898) from each object appearance images whichare cropped according to the obtained objects states
(4) Construct the multimanifold dataset 119872 using theobtained feature tensors X
119894119895(119894 = 1 sdot sdot sdot 119873
119900 119895 = 1 sdot sdot sdot 119898)
(5) Determine the neighborhood relationship using ten-sor distance in the multimanifold dataset
(6) Calculate the weighted centers of each submanifoldand themultilinear embeddedmatrices throughmul-timanifold discriminate analysis
(7) Advance to the next frame 119905 Draw particles accordingto the object prior state 119900
119905minus1and crop the appearance
images corresponding to each of the particles Extractthe feature tensors of each of the appearance imagesThe best object state in current frame is calculated byBayesian sequence inference
(8) Calculate the difference of second-order entropy withthe prior frame and the error in low-dimensionaltensor space embedded if 119867
119889lt 120575119867
119889and 120576 gt 120575120576
119872119894
the feature tensor calculated by the newly obtainedobject state 119900
119905should add into the submanifold119872
119894
(9) When the tensor number in a submanifold 119872119894is the
multiples of the initial number119898 go to step (3)
5 Comparative Experiments and Analysis
In order to verify the effectiveness of the proposed algorithmCAVIAR data sets and PETS outdoor multiperson data setsare used to be verified The initial state of a moving objectis determined by automatically tracking detectors [24] orartificial markers The initial multimanifold data set is calcu-lated by the object states which come from templatematchingtracking algorithm The proposed algorithm is comparedwith three state-of-the-art trackers which are IVT [4] L1-APG [9] and MIL [14] The Bayesian sequence inferenceneeds to consider the particle number which impacts onthe overall efficiency of the algorithm the particle numberis chosen to be 200 for comprehensive consideration Eachobject appearance image is resized to a 64 times 32 times 3 patch
51 CAVIAR Data Sets In this experiment the experimentscenarios come from the Portugal Mall surveillance video
Mathematical Problems in Engineering 7
The second-order entropy difference of theobject image in consecutive frames
0
10000
20000
30000
40000
50000
60000
70000
0 10 20 30 40 50 60 70 80 900
002
004
006
008
01
012
0 10 20 30 40 50 60 70 80 90
The difference of the object feature tensor and the weighted center of submanifold
Figure 2 The change of the object in consecutive frames
data sets There are object scale change pose variation andocclusion during the three objects walking away from thecamera Testing video sequences are color images of 388 times
284 resolutions The Gaussian variances of the three objectsare (8 8 05 05) (4 4 05 05) (2 2 05 05) The results areshown in Figure 3
As can be seen from the results the threemain objects didnot occlude before the initial 57 frames the three comparisonalgorithms can achieve tracking Since the 57th frame object2 gradually occludes object 3 until object 3 is unable to beseen while the IVT and L1-APG algorithms are all missingobject 3 and offset to object 2 which led to the wrong trackingSince the 87th frame object 1 gradually occludes object 3while the IVT tracker could not distinguish them due tothe fact that object 1 is similar to object 3 and then object3 is mistaken as object 1 which carried the wrong trackingMeanwhile the color of object 2 is largely different fromobject 2 and object 3 the IVT and L1-APG trackers canachieve the better results in tracking object 2TheMIL trackerdid not achieve the accurate tracking on the three objectsdue to the interference of the background The proposedalgorithm achieved complete tracking on the three objectswhich was not subject to the interference of similar object inthe tracking process
52 PETSOutdoorMultipersonData Sets In this experimentthe experiment scenarios come from the PETS2009 surveil-lance video data sets There are multiple human objects thatmove around in multiple directions in the scenarios whichare similar to each other The objects cross occlusion andthe objects scale pose variation during the walking Testingvideo sequences are color images of 768 times 576 resolutions
The Gaussian variances of the four objects are (4 3 05 05)(4 4 05 05) (2 2 05 05) (6 6 05 05) The results are inFigure 4
As can be seen from the results object 2 graduallycompletely occludes object 1 since the 26th frame whichmakes object 1 lost most of its information Then the IVTandL1-APG trackers lost object 1 while they achieved trackingobject 2 which is not occluded The MIL tracker roughlyachieves tracking of objects 1 and 2 Object 1 occludes object3 in the 36th frame then the IVT L1-APG and MIL trackersare disturbed by object 1 when tracking object 3 the threealgorithms are all offset to object 1 because object 1 andobject 3 are very similar Object 1 is occluded by object4 since the 56th frame the IVT and L1-APG trackers aredisturbed by object 1 when tracking object 1The two trackerslost object 4 and offset to object 1 while the MIL trackerachieved tracking object 4 Object 4 and object 2 mutualoccluded since the 64th frame MIL tracker failed to trackobject 4 while the IVT and L1-APG are completely wrongtrackingThis video sequence often occurs an object occludedanother one which made the tracking very difficult theproposed algorithm tracking successfully without excessiveinterference with similar objects and achieved a completetracking of the four objects
53 Quantitative Evaluation Aside from the qualitative com-parison we used two metrics to quantitatively compare theexperimental results of the tracking algorithms which aretracking success ratio and center location error [20] Weinitially manually labeled ldquoground truthrdquo locations in eachexperimental scenario
8 Mathematical Problems in Engineering
Figure 3 Some experiments results on CAVIAR data sets (proposed algorithm results 1st 5th row IVT algorithm results 2nd 6th rowL1-APG algorithm results 3rd 7th row MIL algorithm results 4th 8th row frames 1 42 57 87 93 108 118 148 200 and 282)
Mathematical Problems in Engineering 9
Figure 4 Some experiments results on PETS outdoor multiperson data sets (proposed algorithm results 1st 5th row IVT algorithm results2nd 6th row L1-APG algorithm results 3rd 7th row MIL algorithm results 4th 8th row frames 1 26 31 36 48 56 59 64 68 and 90)
10 Mathematical Problems in Engineering
0 50 100 150 200 250 3000
010203040506070809
1Object success ratio
Frame index
Ratio
(a) Scene 1-object 1
0010203040506070809
1
Ratio
0 50 100 150 200 250 300
Object success ratio
Frame index
(b) Scene 1-object 2
0010203040506070809
1
Ratio
0 50 100 150 200 250 300
Object success ratio
Frame index
(c) Scene 1-object 3
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(d) Scene 2-object 1
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(e) Scene 2-object 2
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(f) Scene 2-object 3
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(g) Scene 2-object 4
Figure 5 Tracking success ratio (the red line is the proposed method results the green line is the IVT results the blue line is the L1-APGresults and the yellow line is MIL results)
The tracking success ratio is
ratio =
area (119877e cap 119877119892)
area (119877e cup 119877119892)
(21)
where 119877e is the experiment tracking bounding box 119877119892is the
ground truth bounding box and area() means the area ofthe region The tracking result in one frame is considered asa success when the tracking success ratio is above 05 Thetracking success ratios of four trackers in two scenarios areshown in Figure 5
As can be seen from Figure 5 the IVT and L1-APGtrackers achieve tracking of object 2 in the first scenarios the
three comparison trackers do not achieve completely trackingof other objects in both scenarios due to the disturbance ofbackground information or the similar objects The trackingsuccess ratios of the proposed algorithm with seven objectsin two scenarios are all greater than 05 which means that thealgorithm achieved accurate tracking and is essentially betterthan the other three trackers
The center location error between experiment boundingbox and ground truth bounding box is
where 119909e 119909119892 119910e 119910119892 are the 119909-axis and 119910-axis coordinates ofthe center of the experiment tracking bounding box and theground truth bounding box
The errors of four trackers in two scenarios are shown inTable 1 S2-O2-err represents the center location error of thesecond object in scenarios 2The data in bold refer to optimalresults
As can be seen from Table 1 the other three trackersrarely achieve a complete tracking so the tracking centerpoint errors is large The errors in the proposed method aresignificantly better than the other three trackers and theerrors are within the acceptable range
Our tracker is implemented in MATLAB 2012a and runsat 11 frames and 08 frames per second on an Inter Xeon24GHz CPU with 8GB RAM which is lacking in real-time
6 Conclusions
In this paper we proposed a visual object tracking algorithmvia feature tensor multimanifold discriminate analysis whichconsiders the tracking is vulnerable to the interference ofsimilar objects The object appearance model described byfeature tensor can maintain the object spatial structuralwhich helps to deal with the partial occlusion problem andhelps better to distinguish the object with similar ones inthe embedded low-dimensional subspace throughmultiman-ifold discriminate analysis In addition the update strategy isdesigned from the perspective of object appearance changewhich is used to determine if it is needed to update themultimanifold datasets As can be seen from the comparisonexperiments the proposed algorithm is able to adapt tothe object pose variation scale change and undisturbedtracking of similar objects in scenarios and also can achievecomplete tracking even if the object was completely occludedThe proposed algorithm exist some defects and when theobject is continuously occluded in the dense moving objectsscenarios the object appearance will be incomplete whichcannot construct an accurate multimanifold datasets thatcaused tracking failure
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
This work is supported by the National Natural ScienceFoundation of China (10771043) and the National Natural
Science Foundation of Inner-Mongolia Autonomous RegionChina under Grant (2012MS0931)
References
[1] H Yang L Shao F Zheng L Wang and Z Song ldquoRecentadvances and trends in visual tracking a reviewrdquoNeurocomput-ing vol 74 no 18 pp 3823ndash3831 2011
[2] M J Black and A D Jepson ldquoEigentracking robust matchingand tracking of articulated objects using a view-based represen-tationrdquo International Journal of Computer Vision vol 26 no 1pp 63ndash84 1998
[3] I Matthews T Ishikawa and S Baker ldquoThe template updateproblemrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 26 no 6 pp 810ndash815 2004
[4] D A Ross J Lim R-S Lin and M-H Yang ldquoIncrementallearning for robust visual trackingrdquo International Journal ofComputer Vision vol 77 no 1ndash3 pp 125ndash141 2008
[5] W Hu X Li X Zhang X Shi S Maybank and Z ZhangldquoIncremental tensor subspace learning and Its applications toforeground segmentation and trackingrdquo International Journal ofComputer Vision vol 91 no 3 pp 303ndash327 2011
[6] D Comaniciu V Ramesh and P Meer ldquoKernel-based objecttrackingrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 25 no 5 pp 564ndash577 2003
[7] A Adam E Rivlin and I Shimshoni ldquoRobust fragments-basedtracking using the integral histogramrdquo in Proceedings of theIEEE Computer Society Conference on Computer Vision andPattern Recognition (CVPR rsquo06) pp 798ndash805 June 2006
[8] X Mei and H Ling ldquoRobust visual tracking and vehicleclassification via sparse representationrdquo IEEE Transactions onPattern Analysis and Machine Intelligence vol 33 no 11 pp2259ndash2272 2011
[9] C Bao Y Wu H Ling and H Ji ldquoReal time robust L1 trackerusing accelerated proximal gradient approachrdquo in Proceedingsof the IEEE Conference on Computer Vision and Pattern Recog-nition (CVPR rsquo12) pp 1830ndash1837 June 2012
[10] T Zhang B Ghanem S Liu and N Ahuja ldquoRobust visualtracking via structured multi-task sparse learningrdquo Interna-tional Journal of Computer Vision vol 101 no 2 pp 367ndash3832013
[11] HGrabnerMGrabner andH Bischof ldquoReal-time tracking viaon-line boostingrdquo in Proceedings of the British Machine VisionConference (BMVC rsquo06) pp 47ndash56 September 2006
[12] H Grabner C Leistner and H Bischof ldquoSemi-supervised on-line boosting for robust trackingrdquo inProceedings of the EuropeanConference on Computer Vision pp 234ndash247 Marseille FranceOctober 2008
[13] S Avidan ldquoEnsemble trackingrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 29 no 2 pp 261ndash2712007
12 Mathematical Problems in Engineering
[14] B Babenko M-H Yang and S Belongie ldquoRobust object track-ing with online multiple instance learningrdquo IEEE Transactionson Pattern Analysis and Machine Intelligence vol 33 no 8 pp1619ndash1632 2011
[15] K Zhang and H Song ldquoReal-time visual tracking via onlineweighted multiple instance learningrdquo Pattern Recognition vol46 no 1 pp 397ndash411 2013
[16] S Avidan ldquoSupport vector trackingrdquo IEEE Transactions onPattern Analysis and Machine Intelligence vol 26 no 8 pp1064ndash1072 2004
[17] R T Collins Y Liu and M Leordeanu ldquoOnline selection ofdiscriminative tracking featuresrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 27 no 10 pp 1631ndash16432005
[18] J Kwon and K M Lee ldquoVisual tracking decompositionrdquoin Proceedings of the IEEE Computer Society Conference onComputer Vision and Pattern Recognition (CVPR 10) pp 1269ndash1276 San Francisco Calif USA June 2010
[19] Z Kalal K Mikolajczyk and J Matas ldquoTracking-learning-detectionrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 34 no 7 pp 1409ndash1422 2012
[20] K Zhang L Zhang and M H Yang ldquoReal-time compressivetrackingrdquo in Proceedings of the European Conference on Com-puter Vision pp 864ndash877 2012
[21] H Lu K N Plataniotis and A N Venetsanopoulos ldquoAsurvey of multilinear subspace learning for tensor datardquo PatternRecognition vol 44 no 7 pp 1540ndash1551 2011
[22] W Yang C Sun and L Zhang ldquoAmulti-manifold discriminantanalysis method for image feature extractionrdquo Pattern Recogni-tion vol 44 no 8 pp 1649ndash1657 2011
[23] J Sherrah B Ristic and N J Redding ldquoParticle filter to trackmultiple people for visual surveillancerdquo IET Computer Visionvol 5 no 4 pp 192ndash200 2011
[24] P Dollar CWojek B Schiele and P Perona ldquoPedestrian detec-tion an evaluation of the state of the artrdquo IEEE Transactions onPatternAnalysis andMachine Intelligence vol 34 no 4 pp 743ndash761 2012
by incrementally learning a tensor subspace representationIn the tracking procedure the sample mean and an eigen-basis for each unfolding matrix of the tensor are adaptivelyupdated The classical mean-shift [6] tracker uses histogramas the appearance model then the mean-shift procedureis achieved to locate the object The Fragtrack [7] utilizesseveral fragments to design the appearance model which canhandle pose change and partial occlusion The ℓ
1-tracker [8]
casts tracking problem as sparse approximation where theobject is modeled by a sparse linear combination of targetand a set of trivial templates The sparse representation isobtained by solving an ℓ
1-regularized optimization least-
squares problem and the posteriori probability of candidateimage patch belonging to the object class is inversely pro-portional to the residual between the candidate image patchand the reconstructed oneThe ℓ
1-APG tracker [9] developed
the ℓ1-tracker that not only runs in real-time but also
improves the tracking accuracy The S-MTT [10] algorithmregularizes the appearance model representation problememploying sparsity-inducing ℓ
119901119902mixed norms which can
handle particles independentlyThe discriminative methods treat visual tracking as a
binary classification problem It aims to separate the objectfrom its surrounding complex background with a smalllocal region There are many newly proposed visual trackingalgorithms based on boosting classifier because of its pow-erful discriminative learning capabilities Online boostingalgorithmhaswide applications in object detection and visualtracking Grabner et al [11] proposed an online boostingtracker which is firstly given a discriminative evaluation ofeach feature from a candidate feature pool Then onlinesemisupervised boosting method [12] is proposed for thepurpose of alleviating the object drifting problem in visualtracking Ensemble tracking [13] uses weak classifiers toconstruct a confidence map by pixel classification to distin-guish between the foreground and the background The MILtracker [14] represents an object by a set of samples thesesamples corresponding to image patch are considered withinpositive and negative bags Then multiple instance boostingis used to overcome the problem that slight inaccuracies inlabeled training examples can cause object drift However thetracking may fail when the training samples are imprecisePointing to this problem the WMIL tracker [15] whichintegrates the sample important into the multiple instancelearning is proposed The SVM tracker [16] combined sup-port vector machine into optical flow to achieve visualtracking A visual tracking algorithm via an online featureselection mechanism for evaluating multiple object featuresis proposed in [17] The VTD algorithm [18] designs theobservation andmotionmodel based on visual tracking com-position schemeThe TLD tracker [19] explicitly decomposesthe long-term tracking problem into three componentswhichare tracking learning and detection The CT tracker [20]extracted the sparse image feature combined with a naiveclassifier to separate the object from the background
In the multiple moving objects scenarios with the move-ment of one object the reflected lights of other objects whichreach to the camera lens may be hindered making otherobjectsrsquo projection imaging incomplete or even completely
invisible on the imaging planeWhen the occlusion occurredif the tracking object is similar to the occlusion object theobject is vulnerable to the similar objects influence in theprogress of occlusion to separation which can cause driftThus it is necessary to distinguish the tracking object withthe potential similar objects in the scenarios Meanwhilewhen the object is partially occluded the information fromunoccluded part has a large reference value of determiningthe object state Therefore the object feature must maintainthe structural relationship of the original space This paperproposed a visual object tracking algorithm for multiple sim-ilar objects mutual occluded problem which combines thesetwo ideas First of all a feature function is designed for thepurpose of extracting the tensor feature which can maintainthe spatial structure of the object The multimanifold tensordata set is collected by template matching tracking algorithmin the initial few frames A tensor distance is defined todetermine the intramanifold and intermanifold neighbor-hood relationship The object feature tensor is embeddedinto a low-dimensional space by multimanifold discriminateanalysisThen the object state in the next frame is obtained byBayesian sequence inference Considering the changes in theobject appearance an update strategy for the multimanifoldset is needed to be set
12 Plan of the Paper This paper is organized as follows inthe next sectionwe first introduce the notation of tensor alge-bra and feature tensor After that multimanifold discriminateanalysis is reviewed in Section 3 Section 4 details the visualtracking framework In Section 5 comparative experimentalresults and analysis are showed and conclusions are drawnin Section 6
2 Feature Tensor
A tensor is a high-order array which can be maintained theoriginal spatial structure of an object Construct a featuretensor from an object appearance can increase trackingaccuracy
21 Tensor Algebra Tensor can be viewed as multiorderarray which exists in multiple vector spaces the algebracorresponding to tensor is the mathematical foundation ofmultilinear analysis [21] An 119873-order tensor is denoted asX isin R119868
1times1198682timessdotsdotsdottimes119868
119873 each elements in this tensor is representedas 119909
22 Feature Tensor The object appearance image from RGBcolor video sequence is a three-dimensional data whichformed a nature tensor structure The color and edge infor-mation of the object have a better discrimination on theobject class the gradient feature can describe the object edgeinformation For a detailed description of object informationthe feature function of an object appearance image is definedas follows
direction gradients on the 119877 119866 and 119861 color channelsEach pixel (119894 119895) on object appearance image corresponds
to a twelve-dimensional feature vector the size 119886times119887times3 objectappearance image corresponds to a X isin R119886times119887times12 featuretensor
3 Multimanifold Discriminate Analysis
The basic assumption of the manifold learning is that high-dimensional datum can be considered as geometric correla-tion points which lie in low-dimensional smooth manifoldThere is usually a submanifold structure corresponding to asingle object class different objects lie in different subman-ifolds The multimanifold discriminate analysis can projectthe tensor data which is from a submanifold into a low-dimensional space
31 Multimanifold Neighborhood Relationship of Feature Ten-sor The appearance of each object under different poses isusually composed of a submanifold the multiple differentobject appearance spaces formed the multimanifold Eachmoving object appearance image in video sequence canextract a feature tensor X isin R119886times119887times12 The set of feature tensorcalculated by the appearance images from the first 119898 framesis denoted as119872
119894= X
1198941 X
1198942 X
119894119898 then119872
119894can be seen as
a submanifold Because of the presence of multiple movingobjects in the scenarios the set of each submanifold 119872 =
1198721119872
2 119872
119899 is a multimanifold dataset [22]The entries
X11989411198942sdotsdotsdot119894119873
(1 le 119894119895le 119868
119895 1 le 119895 le 119873) in X are corresponding to
the 119897th element in x where
119897 = 1198941+
119873
sum
119895=2
(119894119895minus 1)
119895minus1
prod
119900=1
119868119900
(2 le 119895 le 119873) (5)
The distance between two tensors X and Y is (the order anddimension of X and Y are the same)
119889119879(X Y) = radic
1198681times1198682timessdotsdotsdottimes119868
119873
sum
119897119898=1
119892119897119898
(x119897minus y
119897) (x
119898minus y
119898) (6)
where 119892119897119898
is the measurement coefficient Since there are toomany entries in the tensor data the measurement coefficientis defined by the distance of points which have spatialneighborhood relationship Consider
119892119897119898
= 119890minus119901119897minus1199011198982
221205902
if x119898isin 119873
1198961015840 (x
119897)
0 else(7)
where 120590 is the regularization parameter and 119901119897minus 119901
1198982is the
location distance between x119897and x
119898 If x
119897and x
119898 respectively
correspond to the X11989411198942sdotsdotsdot119894119873
and X1198941015840
11198941015840
2sdotsdotsdot1198941015840
119873
in tensor X then
1003817100381710038171003817119901119897 minus 119901119898
10038171003817100381710038172 =radic(119894
1minus 119894
1015840
1)2
+ (1198942minus 119894
1015840
2)2
+ sdot sdot sdot + (119894119873minus 119894
1015840
119873)2
(8)
The 1198701intramanifold neighborhood 119873
1198701
intra(X119894119895) of the ten-
sor X119894119895is as follows calculate the tensor distance 119889
119895119897=
119889119879(X
119894119895 X
119894119897) 119895 = 119897 between the tensor X
119894119895in submanifold 119872
119894
and another tensor X119894119897in this submanifold then the nearest
1198701intramanifold neighborhood of X
119894119895can be obtained
according to the tensor distance 119889119895119897
The 1198702intermanifold neighborhood 119873
1198702
inter(X119894119895) of the
tensor X119894119895is as follows calculate the tensor distance 119889
119895119904=
119889119879(X
119894119895 X
119897119904) 119894 = 119897 between the tensor X
119894119895in submanifold 119872
119894
and tensor X119897119904(119897 = 119894) in another submanifold 119872
119897(119897 = 119894)
then the nearest 1198702intermanifold neighborhood of X
119894119895can
be obtained according to the tensor distance 119889119895119904
Themultimanifold dataset and its neighborhood relation-ship are shown in Figure 1
As can be seen fromFigure 1 there are four initial movingobjects in the scenarios thus constructing four submani-folds which are 119872
1 119872
2 119872
3 119872
4 these four submanifolds
formed a multimanifold The intramanifold neighborhoodrelationship of tensor X
13in submanifold119872
1is X
12 X
14 X
17
the intermanifold neighborhood relationship of this tensor isX22 X
24 X
41 X
43 X
44
32 Multimanifold Discriminate Analysis The objective ofmanifold learning is to recover the low-dimensional structurefrom the high-dimensional datum space and find a low-dimensional embedding map In the multiple similar objectsscenarios it is hoped that the extracted object feature candistinguish the object and the potential similar objects inthe scenarios The objective of multimanifold learning isthat the difference between a tensor and intramanifoldneighborhood points decreases and the difference betweenthe tensor and intermanifold neighborhood points increases
4 Mathematical Problems in Engineering
Manifold margin
M1
M2
M3
M4
X22 X24
X12X13
X17
X14
X41X43
X44
Figure 1 Multimanifold dataset and neighborhood relationships
in the embedded space Considering these the objectivefunction of multimanifold discriminate analysis is
argmaxU1U2U3
119891 (U1U
2U
3)
= 119891inter (U1U
2U
3) minus 119891intra (U1
U2U
3)
(9)
where U1 U
2 U
3are the multilinear projection matrices
in the first-order second-order and third-order which arecorresponding to the tensor in the submanifold119872
To maximize the 119891(U119901) by solving the eigen-value equation
(Ainter minus Aintra) 119906119901 = 120582119906119901 (16)
obtain U119901
The eigen-values are 1205821
ge 1205822
ge sdot sdot sdot ge 1205821198891015840 ge 0 ge
1205821198891015840+1
ge sdot sdot sdot ge 120582119889 the corresponding eigen-vector of eigen-
value 120582119901is [119906
1199011
1199061199012
119906119901119889
] where 119889 is the dimension 119901thorder in the original feature tensor from submanifold 119872
119894
The directional projection positive along the eigen-vector 119906119901119897
which is corresponding to the eigen-value 120582119897of (AinterminusAintra)
is positive that is intermanifold neighborhood distanceof tensors is bigger than the intramanifold neighborhooddistance which are projected along this direction Thereforethe projection matrix U
119901= [119906
1199011
1199061199012
1199061199011198891015840] consists
of all of the eigen-vectors which are corresponding to thepositive eigen-values Thus the tensor data which are insubmanifold 119872
119894can be embedded in a low-dimensional
space via multilinear projection matrix U1 U
2 U
3 In this
lower-dimensional space the difference between tensor dataand its intramanifold neighborhood points decreases and thedifference between it and its intermanifold neighborhoodpoints increases so that the distinguishing ability between theobject and the similar ones is greater
4 Visual Tracking Framework
In order to achieve tracking of an object in scenariosBayesian sequence inference is used to obtain the object finalstate Meanwhile the multi-manifold datasets and the multi-linear projection matrice which are calculated from multi-manifold discriminate analysis should be updated
41 Sequence Inference In the visual tracking problem themovement of the object is unable to predict the object statein the current frame only related to that in the prior framethen the visual tracking process satisfies the Markov process[23] A bounding box 119900
119905= (119909
119905 119910
119905 119908
119905 ℎ
119905) is used to describe
the object state at the 119905th frame where (119909119905 119910
119905) 119908
119905 ℎ
119905denote
the upper left corner coordinate the width and height of thebounding box
Given a set of observed object appearance images 119878119905=
1199041 119904
2 119904
119905 the objective of visual tracking is to obtain the
optimal estimate value of the hidden state variables 119900119905 There
is a similar result as that of the object state which is obtainedaccording to Bayesrsquo theorem Consider
119875 (119900119905| 119878
119905) prop 119875 (119904
119905| 119900
119905) int119875 (119900
119905| 119900
119905minus1) 119875 (119900
119905minus1| 119878
119905minus1) 119889119900
119905minus1
(17)
where 119875(119900119905| 119900
119905minus1) refers to the state transition model and
119875(119904119905| 119900
119905) refers to the observation model According the
observation model 119875(119904119905
| 119900119905) we can obtain the tracking
results
State Transition Model This was used to model the move-ment of object between consecutive frames Because of theirregular movement of object the object state is difficult topredict and the moving speed of the object is not very fast Itis considered that the object state in the current frame is nearto that in the prior frameThen the object state 119900
119905is modeled
by independent Gaussian distribution around its counterpartin state 119900
119905minus1 described as
119875 (119900119905| 119900
119905minus1) = 119873 (119900
119905 119900
119905minus1 Σ) (18)
where Σ means the diagonal covariance matrix correspond-ing to the variables 119909
119905 119910
119905 119908
119905 ℎ
119905 and the elements are 1205902
119909 1205902
119910
1205902
119908 1205902
ℎ 119873 particles can be randomly generated pointing to
Gaussian distribution Each particle corresponds to an objectstate then 119873 particles can obtain multiple states 119900
119894
119905 119894 =
1 2 119873 During the visual tracking process the morethe particles we generated are the more accurate the objectstate estimate was but at the same time the computationalefficiency was low For the purpose of efficient and effectiveof the visual tracking algorithm there is a balance soughtbetween these factors
Observation Model This was used to measure the differ-ence between the appearance observation and the objectappearance model Given a drawn particle state 119900
119894
119905and the
corresponding cropped image patch 119911119894
119905in the frame image 119868
119905
the probability of an image patch being generated from thesubmanifold space is inversely proportional to the differencebetween image patch and the appearance model and couldbe calculated between the negative exponential distance ofthe projected data and the weighted center of submanifoldConsider
119901 (z119895119905| 119900
119905) = exp
minus
(10038171003817100381710038171003817(z119895
119905minus 119862
119894) times
1U1times2U2times3U3
10038171003817100381710038171003817119865)
1205902
(19)
6 Mathematical Problems in Engineering
where120590 indicates the bandwidth sdot 119865is the Frobenius norm
andU1U
2U
3are themultilinear projectionmatrix of the 119894th
object in submanifold119872119894
The state 119900119894
119905corresponding to the maximum 119901(119911
119894
119905|
119900119905) is the optimal object state at the 119905th frame Let 120576 =
(z119895119905minus 119862
119894) times
1U1times2U2times3U3119865represent the error between
feature tensor which is calculated by observation z119895119905and the
weighted center 119862119894of submanifold119872
119894
42 Multimanifold Data Sets Update The appearance imageof the object changeswith themovement of it in the scenariosthe submanifold of the object should have different postureobject appearance feature tensors Therefore the multiman-ifold data set should be updated in the tracking processBecause of the factors such as occlusion and so forth whichinfluence the object appearance the appearance images ofthe tracked object have the non-object information thenobtained object feature tensor will not be in the submanifoldTherefore the update strategy is necessary From the perspec-tive of the human sensory vision the appearance informationof object changes in the process of occlusion the changes ofobject between consecutive frames are bigger or the objectfeature tensor is far awaywith the center of submanifold in theembedded space while the changing information betweenconsecutive frames is small or the object feature tensor is nearthe center of submanifold in the embedded space that is theobject state is well determined
The image first-order entropy is used to describe the grayvalue distribution of the object image but not to considerit spatial distribution while the image second-order entropyuses the 2-tuple feature (119894 119895) which is calculated by spatialdistributionThe image second-order entropy could describethe changes of the object where 119894 is the gray value (0 le 119894 le
255) and 119895 is the neighborhood gray value (0 le 119895 le 255)119901119894119895
= 119891(119894 119895)119886119887 denotes the gray value and neighborhoodgray distribution where119891(119894 119895) is the counts of the occurrenceof the 2-tuple feature and 119886119887 is the size of imageThe second-order entropy is defined as
119867 =
255
sum
119894=0
119864119894=
255
sum
119894=0
119901119894119895ln119901
119894119895 (20)
Thedifference of the object in consecutive frames is describedby the second-order entropyWhen the second-order entropydifference of the object image in consecutive frames is biggerthe objectmaybe occluded Simultaneously the feature tensorof appearance image would be far away from the weightedcenter of submanifold namely the error is bigger As shownin Figure 2 the object is largely occluded at the frames 33ndash46and 48ndash63 and small part occluded at the frames 69ndash77
For a best state 119900119905of object 119894 which is newly obtained
when the difference of second-order entropy with the priorframe 119867
119889lt 120575119867
119889and the error in low-dimensional tensor
space embedded 120576 gt 120575120576119872119894
the feature tensor calculatedby the newly obtained object state 119900
119905should add into the
submanifold 119872119894 where 119867
119889is mean of the difference of
second-order entropy 120576119872119894
is the mean of the errors and 120575 isthe adjustment factor which takes 12 in this experiment
When the tensor number in a submanifold 119872119894is the
multiples of the initial number the multimanifold discrimi-nate analysis is computed on the newmultimanifold datasetsthen the weighted center of submanifold and multilinearprojectionmatrices are updatedThere will be a small portionof the determined object data abandoned but the tensorswhich added into the data set are essentially the featuretensors of object appearance
The whole tracking algorithm is working as follows
(1) Locate the object state in the first frame eithermanually or by using an automated detector
(2) Tracking objects use template matching trackingalgorithm in the initial119898 frames
1 sdot sdot sdot 119898) from each object appearance images whichare cropped according to the obtained objects states
(4) Construct the multimanifold dataset 119872 using theobtained feature tensors X
119894119895(119894 = 1 sdot sdot sdot 119873
119900 119895 = 1 sdot sdot sdot 119898)
(5) Determine the neighborhood relationship using ten-sor distance in the multimanifold dataset
(6) Calculate the weighted centers of each submanifoldand themultilinear embeddedmatrices throughmul-timanifold discriminate analysis
(7) Advance to the next frame 119905 Draw particles accordingto the object prior state 119900
119905minus1and crop the appearance
images corresponding to each of the particles Extractthe feature tensors of each of the appearance imagesThe best object state in current frame is calculated byBayesian sequence inference
(8) Calculate the difference of second-order entropy withthe prior frame and the error in low-dimensionaltensor space embedded if 119867
119889lt 120575119867
119889and 120576 gt 120575120576
119872119894
the feature tensor calculated by the newly obtainedobject state 119900
119905should add into the submanifold119872
119894
(9) When the tensor number in a submanifold 119872119894is the
multiples of the initial number119898 go to step (3)
5 Comparative Experiments and Analysis
In order to verify the effectiveness of the proposed algorithmCAVIAR data sets and PETS outdoor multiperson data setsare used to be verified The initial state of a moving objectis determined by automatically tracking detectors [24] orartificial markers The initial multimanifold data set is calcu-lated by the object states which come from templatematchingtracking algorithm The proposed algorithm is comparedwith three state-of-the-art trackers which are IVT [4] L1-APG [9] and MIL [14] The Bayesian sequence inferenceneeds to consider the particle number which impacts onthe overall efficiency of the algorithm the particle numberis chosen to be 200 for comprehensive consideration Eachobject appearance image is resized to a 64 times 32 times 3 patch
51 CAVIAR Data Sets In this experiment the experimentscenarios come from the Portugal Mall surveillance video
Mathematical Problems in Engineering 7
The second-order entropy difference of theobject image in consecutive frames
0
10000
20000
30000
40000
50000
60000
70000
0 10 20 30 40 50 60 70 80 900
002
004
006
008
01
012
0 10 20 30 40 50 60 70 80 90
The difference of the object feature tensor and the weighted center of submanifold
Figure 2 The change of the object in consecutive frames
data sets There are object scale change pose variation andocclusion during the three objects walking away from thecamera Testing video sequences are color images of 388 times
284 resolutions The Gaussian variances of the three objectsare (8 8 05 05) (4 4 05 05) (2 2 05 05) The results areshown in Figure 3
As can be seen from the results the threemain objects didnot occlude before the initial 57 frames the three comparisonalgorithms can achieve tracking Since the 57th frame object2 gradually occludes object 3 until object 3 is unable to beseen while the IVT and L1-APG algorithms are all missingobject 3 and offset to object 2 which led to the wrong trackingSince the 87th frame object 1 gradually occludes object 3while the IVT tracker could not distinguish them due tothe fact that object 1 is similar to object 3 and then object3 is mistaken as object 1 which carried the wrong trackingMeanwhile the color of object 2 is largely different fromobject 2 and object 3 the IVT and L1-APG trackers canachieve the better results in tracking object 2TheMIL trackerdid not achieve the accurate tracking on the three objectsdue to the interference of the background The proposedalgorithm achieved complete tracking on the three objectswhich was not subject to the interference of similar object inthe tracking process
52 PETSOutdoorMultipersonData Sets In this experimentthe experiment scenarios come from the PETS2009 surveil-lance video data sets There are multiple human objects thatmove around in multiple directions in the scenarios whichare similar to each other The objects cross occlusion andthe objects scale pose variation during the walking Testingvideo sequences are color images of 768 times 576 resolutions
The Gaussian variances of the four objects are (4 3 05 05)(4 4 05 05) (2 2 05 05) (6 6 05 05) The results are inFigure 4
As can be seen from the results object 2 graduallycompletely occludes object 1 since the 26th frame whichmakes object 1 lost most of its information Then the IVTandL1-APG trackers lost object 1 while they achieved trackingobject 2 which is not occluded The MIL tracker roughlyachieves tracking of objects 1 and 2 Object 1 occludes object3 in the 36th frame then the IVT L1-APG and MIL trackersare disturbed by object 1 when tracking object 3 the threealgorithms are all offset to object 1 because object 1 andobject 3 are very similar Object 1 is occluded by object4 since the 56th frame the IVT and L1-APG trackers aredisturbed by object 1 when tracking object 1The two trackerslost object 4 and offset to object 1 while the MIL trackerachieved tracking object 4 Object 4 and object 2 mutualoccluded since the 64th frame MIL tracker failed to trackobject 4 while the IVT and L1-APG are completely wrongtrackingThis video sequence often occurs an object occludedanother one which made the tracking very difficult theproposed algorithm tracking successfully without excessiveinterference with similar objects and achieved a completetracking of the four objects
53 Quantitative Evaluation Aside from the qualitative com-parison we used two metrics to quantitatively compare theexperimental results of the tracking algorithms which aretracking success ratio and center location error [20] Weinitially manually labeled ldquoground truthrdquo locations in eachexperimental scenario
8 Mathematical Problems in Engineering
Figure 3 Some experiments results on CAVIAR data sets (proposed algorithm results 1st 5th row IVT algorithm results 2nd 6th rowL1-APG algorithm results 3rd 7th row MIL algorithm results 4th 8th row frames 1 42 57 87 93 108 118 148 200 and 282)
Mathematical Problems in Engineering 9
Figure 4 Some experiments results on PETS outdoor multiperson data sets (proposed algorithm results 1st 5th row IVT algorithm results2nd 6th row L1-APG algorithm results 3rd 7th row MIL algorithm results 4th 8th row frames 1 26 31 36 48 56 59 64 68 and 90)
10 Mathematical Problems in Engineering
0 50 100 150 200 250 3000
010203040506070809
1Object success ratio
Frame index
Ratio
(a) Scene 1-object 1
0010203040506070809
1
Ratio
0 50 100 150 200 250 300
Object success ratio
Frame index
(b) Scene 1-object 2
0010203040506070809
1
Ratio
0 50 100 150 200 250 300
Object success ratio
Frame index
(c) Scene 1-object 3
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(d) Scene 2-object 1
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(e) Scene 2-object 2
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(f) Scene 2-object 3
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(g) Scene 2-object 4
Figure 5 Tracking success ratio (the red line is the proposed method results the green line is the IVT results the blue line is the L1-APGresults and the yellow line is MIL results)
The tracking success ratio is
ratio =
area (119877e cap 119877119892)
area (119877e cup 119877119892)
(21)
where 119877e is the experiment tracking bounding box 119877119892is the
ground truth bounding box and area() means the area ofthe region The tracking result in one frame is considered asa success when the tracking success ratio is above 05 Thetracking success ratios of four trackers in two scenarios areshown in Figure 5
As can be seen from Figure 5 the IVT and L1-APGtrackers achieve tracking of object 2 in the first scenarios the
three comparison trackers do not achieve completely trackingof other objects in both scenarios due to the disturbance ofbackground information or the similar objects The trackingsuccess ratios of the proposed algorithm with seven objectsin two scenarios are all greater than 05 which means that thealgorithm achieved accurate tracking and is essentially betterthan the other three trackers
The center location error between experiment boundingbox and ground truth bounding box is
where 119909e 119909119892 119910e 119910119892 are the 119909-axis and 119910-axis coordinates ofthe center of the experiment tracking bounding box and theground truth bounding box
The errors of four trackers in two scenarios are shown inTable 1 S2-O2-err represents the center location error of thesecond object in scenarios 2The data in bold refer to optimalresults
As can be seen from Table 1 the other three trackersrarely achieve a complete tracking so the tracking centerpoint errors is large The errors in the proposed method aresignificantly better than the other three trackers and theerrors are within the acceptable range
Our tracker is implemented in MATLAB 2012a and runsat 11 frames and 08 frames per second on an Inter Xeon24GHz CPU with 8GB RAM which is lacking in real-time
6 Conclusions
In this paper we proposed a visual object tracking algorithmvia feature tensor multimanifold discriminate analysis whichconsiders the tracking is vulnerable to the interference ofsimilar objects The object appearance model described byfeature tensor can maintain the object spatial structuralwhich helps to deal with the partial occlusion problem andhelps better to distinguish the object with similar ones inthe embedded low-dimensional subspace throughmultiman-ifold discriminate analysis In addition the update strategy isdesigned from the perspective of object appearance changewhich is used to determine if it is needed to update themultimanifold datasets As can be seen from the comparisonexperiments the proposed algorithm is able to adapt tothe object pose variation scale change and undisturbedtracking of similar objects in scenarios and also can achievecomplete tracking even if the object was completely occludedThe proposed algorithm exist some defects and when theobject is continuously occluded in the dense moving objectsscenarios the object appearance will be incomplete whichcannot construct an accurate multimanifold datasets thatcaused tracking failure
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
This work is supported by the National Natural ScienceFoundation of China (10771043) and the National Natural
Science Foundation of Inner-Mongolia Autonomous RegionChina under Grant (2012MS0931)
References
[1] H Yang L Shao F Zheng L Wang and Z Song ldquoRecentadvances and trends in visual tracking a reviewrdquoNeurocomput-ing vol 74 no 18 pp 3823ndash3831 2011
[2] M J Black and A D Jepson ldquoEigentracking robust matchingand tracking of articulated objects using a view-based represen-tationrdquo International Journal of Computer Vision vol 26 no 1pp 63ndash84 1998
[3] I Matthews T Ishikawa and S Baker ldquoThe template updateproblemrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 26 no 6 pp 810ndash815 2004
[4] D A Ross J Lim R-S Lin and M-H Yang ldquoIncrementallearning for robust visual trackingrdquo International Journal ofComputer Vision vol 77 no 1ndash3 pp 125ndash141 2008
[5] W Hu X Li X Zhang X Shi S Maybank and Z ZhangldquoIncremental tensor subspace learning and Its applications toforeground segmentation and trackingrdquo International Journal ofComputer Vision vol 91 no 3 pp 303ndash327 2011
[6] D Comaniciu V Ramesh and P Meer ldquoKernel-based objecttrackingrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 25 no 5 pp 564ndash577 2003
[7] A Adam E Rivlin and I Shimshoni ldquoRobust fragments-basedtracking using the integral histogramrdquo in Proceedings of theIEEE Computer Society Conference on Computer Vision andPattern Recognition (CVPR rsquo06) pp 798ndash805 June 2006
[8] X Mei and H Ling ldquoRobust visual tracking and vehicleclassification via sparse representationrdquo IEEE Transactions onPattern Analysis and Machine Intelligence vol 33 no 11 pp2259ndash2272 2011
[9] C Bao Y Wu H Ling and H Ji ldquoReal time robust L1 trackerusing accelerated proximal gradient approachrdquo in Proceedingsof the IEEE Conference on Computer Vision and Pattern Recog-nition (CVPR rsquo12) pp 1830ndash1837 June 2012
[10] T Zhang B Ghanem S Liu and N Ahuja ldquoRobust visualtracking via structured multi-task sparse learningrdquo Interna-tional Journal of Computer Vision vol 101 no 2 pp 367ndash3832013
[11] HGrabnerMGrabner andH Bischof ldquoReal-time tracking viaon-line boostingrdquo in Proceedings of the British Machine VisionConference (BMVC rsquo06) pp 47ndash56 September 2006
[12] H Grabner C Leistner and H Bischof ldquoSemi-supervised on-line boosting for robust trackingrdquo inProceedings of the EuropeanConference on Computer Vision pp 234ndash247 Marseille FranceOctober 2008
[13] S Avidan ldquoEnsemble trackingrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 29 no 2 pp 261ndash2712007
12 Mathematical Problems in Engineering
[14] B Babenko M-H Yang and S Belongie ldquoRobust object track-ing with online multiple instance learningrdquo IEEE Transactionson Pattern Analysis and Machine Intelligence vol 33 no 8 pp1619ndash1632 2011
[15] K Zhang and H Song ldquoReal-time visual tracking via onlineweighted multiple instance learningrdquo Pattern Recognition vol46 no 1 pp 397ndash411 2013
[16] S Avidan ldquoSupport vector trackingrdquo IEEE Transactions onPattern Analysis and Machine Intelligence vol 26 no 8 pp1064ndash1072 2004
[17] R T Collins Y Liu and M Leordeanu ldquoOnline selection ofdiscriminative tracking featuresrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 27 no 10 pp 1631ndash16432005
[18] J Kwon and K M Lee ldquoVisual tracking decompositionrdquoin Proceedings of the IEEE Computer Society Conference onComputer Vision and Pattern Recognition (CVPR 10) pp 1269ndash1276 San Francisco Calif USA June 2010
[19] Z Kalal K Mikolajczyk and J Matas ldquoTracking-learning-detectionrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 34 no 7 pp 1409ndash1422 2012
[20] K Zhang L Zhang and M H Yang ldquoReal-time compressivetrackingrdquo in Proceedings of the European Conference on Com-puter Vision pp 864ndash877 2012
[21] H Lu K N Plataniotis and A N Venetsanopoulos ldquoAsurvey of multilinear subspace learning for tensor datardquo PatternRecognition vol 44 no 7 pp 1540ndash1551 2011
[22] W Yang C Sun and L Zhang ldquoAmulti-manifold discriminantanalysis method for image feature extractionrdquo Pattern Recogni-tion vol 44 no 8 pp 1649ndash1657 2011
[23] J Sherrah B Ristic and N J Redding ldquoParticle filter to trackmultiple people for visual surveillancerdquo IET Computer Visionvol 5 no 4 pp 192ndash200 2011
[24] P Dollar CWojek B Schiele and P Perona ldquoPedestrian detec-tion an evaluation of the state of the artrdquo IEEE Transactions onPatternAnalysis andMachine Intelligence vol 34 no 4 pp 743ndash761 2012
22 Feature Tensor The object appearance image from RGBcolor video sequence is a three-dimensional data whichformed a nature tensor structure The color and edge infor-mation of the object have a better discrimination on theobject class the gradient feature can describe the object edgeinformation For a detailed description of object informationthe feature function of an object appearance image is definedas follows
direction gradients on the 119877 119866 and 119861 color channelsEach pixel (119894 119895) on object appearance image corresponds
to a twelve-dimensional feature vector the size 119886times119887times3 objectappearance image corresponds to a X isin R119886times119887times12 featuretensor
3 Multimanifold Discriminate Analysis
The basic assumption of the manifold learning is that high-dimensional datum can be considered as geometric correla-tion points which lie in low-dimensional smooth manifoldThere is usually a submanifold structure corresponding to asingle object class different objects lie in different subman-ifolds The multimanifold discriminate analysis can projectthe tensor data which is from a submanifold into a low-dimensional space
31 Multimanifold Neighborhood Relationship of Feature Ten-sor The appearance of each object under different poses isusually composed of a submanifold the multiple differentobject appearance spaces formed the multimanifold Eachmoving object appearance image in video sequence canextract a feature tensor X isin R119886times119887times12 The set of feature tensorcalculated by the appearance images from the first 119898 framesis denoted as119872
119894= X
1198941 X
1198942 X
119894119898 then119872
119894can be seen as
a submanifold Because of the presence of multiple movingobjects in the scenarios the set of each submanifold 119872 =
1198721119872
2 119872
119899 is a multimanifold dataset [22]The entries
X11989411198942sdotsdotsdot119894119873
(1 le 119894119895le 119868
119895 1 le 119895 le 119873) in X are corresponding to
the 119897th element in x where
119897 = 1198941+
119873
sum
119895=2
(119894119895minus 1)
119895minus1
prod
119900=1
119868119900
(2 le 119895 le 119873) (5)
The distance between two tensors X and Y is (the order anddimension of X and Y are the same)
119889119879(X Y) = radic
1198681times1198682timessdotsdotsdottimes119868
119873
sum
119897119898=1
119892119897119898
(x119897minus y
119897) (x
119898minus y
119898) (6)
where 119892119897119898
is the measurement coefficient Since there are toomany entries in the tensor data the measurement coefficientis defined by the distance of points which have spatialneighborhood relationship Consider
119892119897119898
= 119890minus119901119897minus1199011198982
221205902
if x119898isin 119873
1198961015840 (x
119897)
0 else(7)
where 120590 is the regularization parameter and 119901119897minus 119901
1198982is the
location distance between x119897and x
119898 If x
119897and x
119898 respectively
correspond to the X11989411198942sdotsdotsdot119894119873
and X1198941015840
11198941015840
2sdotsdotsdot1198941015840
119873
in tensor X then
1003817100381710038171003817119901119897 minus 119901119898
10038171003817100381710038172 =radic(119894
1minus 119894
1015840
1)2
+ (1198942minus 119894
1015840
2)2
+ sdot sdot sdot + (119894119873minus 119894
1015840
119873)2
(8)
The 1198701intramanifold neighborhood 119873
1198701
intra(X119894119895) of the ten-
sor X119894119895is as follows calculate the tensor distance 119889
119895119897=
119889119879(X
119894119895 X
119894119897) 119895 = 119897 between the tensor X
119894119895in submanifold 119872
119894
and another tensor X119894119897in this submanifold then the nearest
1198701intramanifold neighborhood of X
119894119895can be obtained
according to the tensor distance 119889119895119897
The 1198702intermanifold neighborhood 119873
1198702
inter(X119894119895) of the
tensor X119894119895is as follows calculate the tensor distance 119889
119895119904=
119889119879(X
119894119895 X
119897119904) 119894 = 119897 between the tensor X
119894119895in submanifold 119872
119894
and tensor X119897119904(119897 = 119894) in another submanifold 119872
119897(119897 = 119894)
then the nearest 1198702intermanifold neighborhood of X
119894119895can
be obtained according to the tensor distance 119889119895119904
Themultimanifold dataset and its neighborhood relation-ship are shown in Figure 1
As can be seen fromFigure 1 there are four initial movingobjects in the scenarios thus constructing four submani-folds which are 119872
1 119872
2 119872
3 119872
4 these four submanifolds
formed a multimanifold The intramanifold neighborhoodrelationship of tensor X
13in submanifold119872
1is X
12 X
14 X
17
the intermanifold neighborhood relationship of this tensor isX22 X
24 X
41 X
43 X
44
32 Multimanifold Discriminate Analysis The objective ofmanifold learning is to recover the low-dimensional structurefrom the high-dimensional datum space and find a low-dimensional embedding map In the multiple similar objectsscenarios it is hoped that the extracted object feature candistinguish the object and the potential similar objects inthe scenarios The objective of multimanifold learning isthat the difference between a tensor and intramanifoldneighborhood points decreases and the difference betweenthe tensor and intermanifold neighborhood points increases
4 Mathematical Problems in Engineering
Manifold margin
M1
M2
M3
M4
X22 X24
X12X13
X17
X14
X41X43
X44
Figure 1 Multimanifold dataset and neighborhood relationships
in the embedded space Considering these the objectivefunction of multimanifold discriminate analysis is
argmaxU1U2U3
119891 (U1U
2U
3)
= 119891inter (U1U
2U
3) minus 119891intra (U1
U2U
3)
(9)
where U1 U
2 U
3are the multilinear projection matrices
in the first-order second-order and third-order which arecorresponding to the tensor in the submanifold119872
To maximize the 119891(U119901) by solving the eigen-value equation
(Ainter minus Aintra) 119906119901 = 120582119906119901 (16)
obtain U119901
The eigen-values are 1205821
ge 1205822
ge sdot sdot sdot ge 1205821198891015840 ge 0 ge
1205821198891015840+1
ge sdot sdot sdot ge 120582119889 the corresponding eigen-vector of eigen-
value 120582119901is [119906
1199011
1199061199012
119906119901119889
] where 119889 is the dimension 119901thorder in the original feature tensor from submanifold 119872
119894
The directional projection positive along the eigen-vector 119906119901119897
which is corresponding to the eigen-value 120582119897of (AinterminusAintra)
is positive that is intermanifold neighborhood distanceof tensors is bigger than the intramanifold neighborhooddistance which are projected along this direction Thereforethe projection matrix U
119901= [119906
1199011
1199061199012
1199061199011198891015840] consists
of all of the eigen-vectors which are corresponding to thepositive eigen-values Thus the tensor data which are insubmanifold 119872
119894can be embedded in a low-dimensional
space via multilinear projection matrix U1 U
2 U
3 In this
lower-dimensional space the difference between tensor dataand its intramanifold neighborhood points decreases and thedifference between it and its intermanifold neighborhoodpoints increases so that the distinguishing ability between theobject and the similar ones is greater
4 Visual Tracking Framework
In order to achieve tracking of an object in scenariosBayesian sequence inference is used to obtain the object finalstate Meanwhile the multi-manifold datasets and the multi-linear projection matrice which are calculated from multi-manifold discriminate analysis should be updated
41 Sequence Inference In the visual tracking problem themovement of the object is unable to predict the object statein the current frame only related to that in the prior framethen the visual tracking process satisfies the Markov process[23] A bounding box 119900
119905= (119909
119905 119910
119905 119908
119905 ℎ
119905) is used to describe
the object state at the 119905th frame where (119909119905 119910
119905) 119908
119905 ℎ
119905denote
the upper left corner coordinate the width and height of thebounding box
Given a set of observed object appearance images 119878119905=
1199041 119904
2 119904
119905 the objective of visual tracking is to obtain the
optimal estimate value of the hidden state variables 119900119905 There
is a similar result as that of the object state which is obtainedaccording to Bayesrsquo theorem Consider
119875 (119900119905| 119878
119905) prop 119875 (119904
119905| 119900
119905) int119875 (119900
119905| 119900
119905minus1) 119875 (119900
119905minus1| 119878
119905minus1) 119889119900
119905minus1
(17)
where 119875(119900119905| 119900
119905minus1) refers to the state transition model and
119875(119904119905| 119900
119905) refers to the observation model According the
observation model 119875(119904119905
| 119900119905) we can obtain the tracking
results
State Transition Model This was used to model the move-ment of object between consecutive frames Because of theirregular movement of object the object state is difficult topredict and the moving speed of the object is not very fast Itis considered that the object state in the current frame is nearto that in the prior frameThen the object state 119900
119905is modeled
by independent Gaussian distribution around its counterpartin state 119900
119905minus1 described as
119875 (119900119905| 119900
119905minus1) = 119873 (119900
119905 119900
119905minus1 Σ) (18)
where Σ means the diagonal covariance matrix correspond-ing to the variables 119909
119905 119910
119905 119908
119905 ℎ
119905 and the elements are 1205902
119909 1205902
119910
1205902
119908 1205902
ℎ 119873 particles can be randomly generated pointing to
Gaussian distribution Each particle corresponds to an objectstate then 119873 particles can obtain multiple states 119900
119894
119905 119894 =
1 2 119873 During the visual tracking process the morethe particles we generated are the more accurate the objectstate estimate was but at the same time the computationalefficiency was low For the purpose of efficient and effectiveof the visual tracking algorithm there is a balance soughtbetween these factors
Observation Model This was used to measure the differ-ence between the appearance observation and the objectappearance model Given a drawn particle state 119900
119894
119905and the
corresponding cropped image patch 119911119894
119905in the frame image 119868
119905
the probability of an image patch being generated from thesubmanifold space is inversely proportional to the differencebetween image patch and the appearance model and couldbe calculated between the negative exponential distance ofthe projected data and the weighted center of submanifoldConsider
119901 (z119895119905| 119900
119905) = exp
minus
(10038171003817100381710038171003817(z119895
119905minus 119862
119894) times
1U1times2U2times3U3
10038171003817100381710038171003817119865)
1205902
(19)
6 Mathematical Problems in Engineering
where120590 indicates the bandwidth sdot 119865is the Frobenius norm
andU1U
2U
3are themultilinear projectionmatrix of the 119894th
object in submanifold119872119894
The state 119900119894
119905corresponding to the maximum 119901(119911
119894
119905|
119900119905) is the optimal object state at the 119905th frame Let 120576 =
(z119895119905minus 119862
119894) times
1U1times2U2times3U3119865represent the error between
feature tensor which is calculated by observation z119895119905and the
weighted center 119862119894of submanifold119872
119894
42 Multimanifold Data Sets Update The appearance imageof the object changeswith themovement of it in the scenariosthe submanifold of the object should have different postureobject appearance feature tensors Therefore the multiman-ifold data set should be updated in the tracking processBecause of the factors such as occlusion and so forth whichinfluence the object appearance the appearance images ofthe tracked object have the non-object information thenobtained object feature tensor will not be in the submanifoldTherefore the update strategy is necessary From the perspec-tive of the human sensory vision the appearance informationof object changes in the process of occlusion the changes ofobject between consecutive frames are bigger or the objectfeature tensor is far awaywith the center of submanifold in theembedded space while the changing information betweenconsecutive frames is small or the object feature tensor is nearthe center of submanifold in the embedded space that is theobject state is well determined
The image first-order entropy is used to describe the grayvalue distribution of the object image but not to considerit spatial distribution while the image second-order entropyuses the 2-tuple feature (119894 119895) which is calculated by spatialdistributionThe image second-order entropy could describethe changes of the object where 119894 is the gray value (0 le 119894 le
255) and 119895 is the neighborhood gray value (0 le 119895 le 255)119901119894119895
= 119891(119894 119895)119886119887 denotes the gray value and neighborhoodgray distribution where119891(119894 119895) is the counts of the occurrenceof the 2-tuple feature and 119886119887 is the size of imageThe second-order entropy is defined as
119867 =
255
sum
119894=0
119864119894=
255
sum
119894=0
119901119894119895ln119901
119894119895 (20)
Thedifference of the object in consecutive frames is describedby the second-order entropyWhen the second-order entropydifference of the object image in consecutive frames is biggerthe objectmaybe occluded Simultaneously the feature tensorof appearance image would be far away from the weightedcenter of submanifold namely the error is bigger As shownin Figure 2 the object is largely occluded at the frames 33ndash46and 48ndash63 and small part occluded at the frames 69ndash77
For a best state 119900119905of object 119894 which is newly obtained
when the difference of second-order entropy with the priorframe 119867
119889lt 120575119867
119889and the error in low-dimensional tensor
space embedded 120576 gt 120575120576119872119894
the feature tensor calculatedby the newly obtained object state 119900
119905should add into the
submanifold 119872119894 where 119867
119889is mean of the difference of
second-order entropy 120576119872119894
is the mean of the errors and 120575 isthe adjustment factor which takes 12 in this experiment
When the tensor number in a submanifold 119872119894is the
multiples of the initial number the multimanifold discrimi-nate analysis is computed on the newmultimanifold datasetsthen the weighted center of submanifold and multilinearprojectionmatrices are updatedThere will be a small portionof the determined object data abandoned but the tensorswhich added into the data set are essentially the featuretensors of object appearance
The whole tracking algorithm is working as follows
(1) Locate the object state in the first frame eithermanually or by using an automated detector
(2) Tracking objects use template matching trackingalgorithm in the initial119898 frames
1 sdot sdot sdot 119898) from each object appearance images whichare cropped according to the obtained objects states
(4) Construct the multimanifold dataset 119872 using theobtained feature tensors X
119894119895(119894 = 1 sdot sdot sdot 119873
119900 119895 = 1 sdot sdot sdot 119898)
(5) Determine the neighborhood relationship using ten-sor distance in the multimanifold dataset
(6) Calculate the weighted centers of each submanifoldand themultilinear embeddedmatrices throughmul-timanifold discriminate analysis
(7) Advance to the next frame 119905 Draw particles accordingto the object prior state 119900
119905minus1and crop the appearance
images corresponding to each of the particles Extractthe feature tensors of each of the appearance imagesThe best object state in current frame is calculated byBayesian sequence inference
(8) Calculate the difference of second-order entropy withthe prior frame and the error in low-dimensionaltensor space embedded if 119867
119889lt 120575119867
119889and 120576 gt 120575120576
119872119894
the feature tensor calculated by the newly obtainedobject state 119900
119905should add into the submanifold119872
119894
(9) When the tensor number in a submanifold 119872119894is the
multiples of the initial number119898 go to step (3)
5 Comparative Experiments and Analysis
In order to verify the effectiveness of the proposed algorithmCAVIAR data sets and PETS outdoor multiperson data setsare used to be verified The initial state of a moving objectis determined by automatically tracking detectors [24] orartificial markers The initial multimanifold data set is calcu-lated by the object states which come from templatematchingtracking algorithm The proposed algorithm is comparedwith three state-of-the-art trackers which are IVT [4] L1-APG [9] and MIL [14] The Bayesian sequence inferenceneeds to consider the particle number which impacts onthe overall efficiency of the algorithm the particle numberis chosen to be 200 for comprehensive consideration Eachobject appearance image is resized to a 64 times 32 times 3 patch
51 CAVIAR Data Sets In this experiment the experimentscenarios come from the Portugal Mall surveillance video
Mathematical Problems in Engineering 7
The second-order entropy difference of theobject image in consecutive frames
0
10000
20000
30000
40000
50000
60000
70000
0 10 20 30 40 50 60 70 80 900
002
004
006
008
01
012
0 10 20 30 40 50 60 70 80 90
The difference of the object feature tensor and the weighted center of submanifold
Figure 2 The change of the object in consecutive frames
data sets There are object scale change pose variation andocclusion during the three objects walking away from thecamera Testing video sequences are color images of 388 times
284 resolutions The Gaussian variances of the three objectsare (8 8 05 05) (4 4 05 05) (2 2 05 05) The results areshown in Figure 3
As can be seen from the results the threemain objects didnot occlude before the initial 57 frames the three comparisonalgorithms can achieve tracking Since the 57th frame object2 gradually occludes object 3 until object 3 is unable to beseen while the IVT and L1-APG algorithms are all missingobject 3 and offset to object 2 which led to the wrong trackingSince the 87th frame object 1 gradually occludes object 3while the IVT tracker could not distinguish them due tothe fact that object 1 is similar to object 3 and then object3 is mistaken as object 1 which carried the wrong trackingMeanwhile the color of object 2 is largely different fromobject 2 and object 3 the IVT and L1-APG trackers canachieve the better results in tracking object 2TheMIL trackerdid not achieve the accurate tracking on the three objectsdue to the interference of the background The proposedalgorithm achieved complete tracking on the three objectswhich was not subject to the interference of similar object inthe tracking process
52 PETSOutdoorMultipersonData Sets In this experimentthe experiment scenarios come from the PETS2009 surveil-lance video data sets There are multiple human objects thatmove around in multiple directions in the scenarios whichare similar to each other The objects cross occlusion andthe objects scale pose variation during the walking Testingvideo sequences are color images of 768 times 576 resolutions
The Gaussian variances of the four objects are (4 3 05 05)(4 4 05 05) (2 2 05 05) (6 6 05 05) The results are inFigure 4
As can be seen from the results object 2 graduallycompletely occludes object 1 since the 26th frame whichmakes object 1 lost most of its information Then the IVTandL1-APG trackers lost object 1 while they achieved trackingobject 2 which is not occluded The MIL tracker roughlyachieves tracking of objects 1 and 2 Object 1 occludes object3 in the 36th frame then the IVT L1-APG and MIL trackersare disturbed by object 1 when tracking object 3 the threealgorithms are all offset to object 1 because object 1 andobject 3 are very similar Object 1 is occluded by object4 since the 56th frame the IVT and L1-APG trackers aredisturbed by object 1 when tracking object 1The two trackerslost object 4 and offset to object 1 while the MIL trackerachieved tracking object 4 Object 4 and object 2 mutualoccluded since the 64th frame MIL tracker failed to trackobject 4 while the IVT and L1-APG are completely wrongtrackingThis video sequence often occurs an object occludedanother one which made the tracking very difficult theproposed algorithm tracking successfully without excessiveinterference with similar objects and achieved a completetracking of the four objects
53 Quantitative Evaluation Aside from the qualitative com-parison we used two metrics to quantitatively compare theexperimental results of the tracking algorithms which aretracking success ratio and center location error [20] Weinitially manually labeled ldquoground truthrdquo locations in eachexperimental scenario
8 Mathematical Problems in Engineering
Figure 3 Some experiments results on CAVIAR data sets (proposed algorithm results 1st 5th row IVT algorithm results 2nd 6th rowL1-APG algorithm results 3rd 7th row MIL algorithm results 4th 8th row frames 1 42 57 87 93 108 118 148 200 and 282)
Mathematical Problems in Engineering 9
Figure 4 Some experiments results on PETS outdoor multiperson data sets (proposed algorithm results 1st 5th row IVT algorithm results2nd 6th row L1-APG algorithm results 3rd 7th row MIL algorithm results 4th 8th row frames 1 26 31 36 48 56 59 64 68 and 90)
10 Mathematical Problems in Engineering
0 50 100 150 200 250 3000
010203040506070809
1Object success ratio
Frame index
Ratio
(a) Scene 1-object 1
0010203040506070809
1
Ratio
0 50 100 150 200 250 300
Object success ratio
Frame index
(b) Scene 1-object 2
0010203040506070809
1
Ratio
0 50 100 150 200 250 300
Object success ratio
Frame index
(c) Scene 1-object 3
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(d) Scene 2-object 1
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(e) Scene 2-object 2
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(f) Scene 2-object 3
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(g) Scene 2-object 4
Figure 5 Tracking success ratio (the red line is the proposed method results the green line is the IVT results the blue line is the L1-APGresults and the yellow line is MIL results)
The tracking success ratio is
ratio =
area (119877e cap 119877119892)
area (119877e cup 119877119892)
(21)
where 119877e is the experiment tracking bounding box 119877119892is the
ground truth bounding box and area() means the area ofthe region The tracking result in one frame is considered asa success when the tracking success ratio is above 05 Thetracking success ratios of four trackers in two scenarios areshown in Figure 5
As can be seen from Figure 5 the IVT and L1-APGtrackers achieve tracking of object 2 in the first scenarios the
three comparison trackers do not achieve completely trackingof other objects in both scenarios due to the disturbance ofbackground information or the similar objects The trackingsuccess ratios of the proposed algorithm with seven objectsin two scenarios are all greater than 05 which means that thealgorithm achieved accurate tracking and is essentially betterthan the other three trackers
The center location error between experiment boundingbox and ground truth bounding box is
where 119909e 119909119892 119910e 119910119892 are the 119909-axis and 119910-axis coordinates ofthe center of the experiment tracking bounding box and theground truth bounding box
The errors of four trackers in two scenarios are shown inTable 1 S2-O2-err represents the center location error of thesecond object in scenarios 2The data in bold refer to optimalresults
As can be seen from Table 1 the other three trackersrarely achieve a complete tracking so the tracking centerpoint errors is large The errors in the proposed method aresignificantly better than the other three trackers and theerrors are within the acceptable range
Our tracker is implemented in MATLAB 2012a and runsat 11 frames and 08 frames per second on an Inter Xeon24GHz CPU with 8GB RAM which is lacking in real-time
6 Conclusions
In this paper we proposed a visual object tracking algorithmvia feature tensor multimanifold discriminate analysis whichconsiders the tracking is vulnerable to the interference ofsimilar objects The object appearance model described byfeature tensor can maintain the object spatial structuralwhich helps to deal with the partial occlusion problem andhelps better to distinguish the object with similar ones inthe embedded low-dimensional subspace throughmultiman-ifold discriminate analysis In addition the update strategy isdesigned from the perspective of object appearance changewhich is used to determine if it is needed to update themultimanifold datasets As can be seen from the comparisonexperiments the proposed algorithm is able to adapt tothe object pose variation scale change and undisturbedtracking of similar objects in scenarios and also can achievecomplete tracking even if the object was completely occludedThe proposed algorithm exist some defects and when theobject is continuously occluded in the dense moving objectsscenarios the object appearance will be incomplete whichcannot construct an accurate multimanifold datasets thatcaused tracking failure
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
This work is supported by the National Natural ScienceFoundation of China (10771043) and the National Natural
Science Foundation of Inner-Mongolia Autonomous RegionChina under Grant (2012MS0931)
References
[1] H Yang L Shao F Zheng L Wang and Z Song ldquoRecentadvances and trends in visual tracking a reviewrdquoNeurocomput-ing vol 74 no 18 pp 3823ndash3831 2011
[2] M J Black and A D Jepson ldquoEigentracking robust matchingand tracking of articulated objects using a view-based represen-tationrdquo International Journal of Computer Vision vol 26 no 1pp 63ndash84 1998
[3] I Matthews T Ishikawa and S Baker ldquoThe template updateproblemrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 26 no 6 pp 810ndash815 2004
[4] D A Ross J Lim R-S Lin and M-H Yang ldquoIncrementallearning for robust visual trackingrdquo International Journal ofComputer Vision vol 77 no 1ndash3 pp 125ndash141 2008
[5] W Hu X Li X Zhang X Shi S Maybank and Z ZhangldquoIncremental tensor subspace learning and Its applications toforeground segmentation and trackingrdquo International Journal ofComputer Vision vol 91 no 3 pp 303ndash327 2011
[6] D Comaniciu V Ramesh and P Meer ldquoKernel-based objecttrackingrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 25 no 5 pp 564ndash577 2003
[7] A Adam E Rivlin and I Shimshoni ldquoRobust fragments-basedtracking using the integral histogramrdquo in Proceedings of theIEEE Computer Society Conference on Computer Vision andPattern Recognition (CVPR rsquo06) pp 798ndash805 June 2006
[8] X Mei and H Ling ldquoRobust visual tracking and vehicleclassification via sparse representationrdquo IEEE Transactions onPattern Analysis and Machine Intelligence vol 33 no 11 pp2259ndash2272 2011
[9] C Bao Y Wu H Ling and H Ji ldquoReal time robust L1 trackerusing accelerated proximal gradient approachrdquo in Proceedingsof the IEEE Conference on Computer Vision and Pattern Recog-nition (CVPR rsquo12) pp 1830ndash1837 June 2012
[10] T Zhang B Ghanem S Liu and N Ahuja ldquoRobust visualtracking via structured multi-task sparse learningrdquo Interna-tional Journal of Computer Vision vol 101 no 2 pp 367ndash3832013
[11] HGrabnerMGrabner andH Bischof ldquoReal-time tracking viaon-line boostingrdquo in Proceedings of the British Machine VisionConference (BMVC rsquo06) pp 47ndash56 September 2006
[12] H Grabner C Leistner and H Bischof ldquoSemi-supervised on-line boosting for robust trackingrdquo inProceedings of the EuropeanConference on Computer Vision pp 234ndash247 Marseille FranceOctober 2008
[13] S Avidan ldquoEnsemble trackingrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 29 no 2 pp 261ndash2712007
12 Mathematical Problems in Engineering
[14] B Babenko M-H Yang and S Belongie ldquoRobust object track-ing with online multiple instance learningrdquo IEEE Transactionson Pattern Analysis and Machine Intelligence vol 33 no 8 pp1619ndash1632 2011
[15] K Zhang and H Song ldquoReal-time visual tracking via onlineweighted multiple instance learningrdquo Pattern Recognition vol46 no 1 pp 397ndash411 2013
[16] S Avidan ldquoSupport vector trackingrdquo IEEE Transactions onPattern Analysis and Machine Intelligence vol 26 no 8 pp1064ndash1072 2004
[17] R T Collins Y Liu and M Leordeanu ldquoOnline selection ofdiscriminative tracking featuresrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 27 no 10 pp 1631ndash16432005
[18] J Kwon and K M Lee ldquoVisual tracking decompositionrdquoin Proceedings of the IEEE Computer Society Conference onComputer Vision and Pattern Recognition (CVPR 10) pp 1269ndash1276 San Francisco Calif USA June 2010
[19] Z Kalal K Mikolajczyk and J Matas ldquoTracking-learning-detectionrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 34 no 7 pp 1409ndash1422 2012
[20] K Zhang L Zhang and M H Yang ldquoReal-time compressivetrackingrdquo in Proceedings of the European Conference on Com-puter Vision pp 864ndash877 2012
[21] H Lu K N Plataniotis and A N Venetsanopoulos ldquoAsurvey of multilinear subspace learning for tensor datardquo PatternRecognition vol 44 no 7 pp 1540ndash1551 2011
[22] W Yang C Sun and L Zhang ldquoAmulti-manifold discriminantanalysis method for image feature extractionrdquo Pattern Recogni-tion vol 44 no 8 pp 1649ndash1657 2011
[23] J Sherrah B Ristic and N J Redding ldquoParticle filter to trackmultiple people for visual surveillancerdquo IET Computer Visionvol 5 no 4 pp 192ndash200 2011
[24] P Dollar CWojek B Schiele and P Perona ldquoPedestrian detec-tion an evaluation of the state of the artrdquo IEEE Transactions onPatternAnalysis andMachine Intelligence vol 34 no 4 pp 743ndash761 2012
To maximize the 119891(U119901) by solving the eigen-value equation
(Ainter minus Aintra) 119906119901 = 120582119906119901 (16)
obtain U119901
The eigen-values are 1205821
ge 1205822
ge sdot sdot sdot ge 1205821198891015840 ge 0 ge
1205821198891015840+1
ge sdot sdot sdot ge 120582119889 the corresponding eigen-vector of eigen-
value 120582119901is [119906
1199011
1199061199012
119906119901119889
] where 119889 is the dimension 119901thorder in the original feature tensor from submanifold 119872
119894
The directional projection positive along the eigen-vector 119906119901119897
which is corresponding to the eigen-value 120582119897of (AinterminusAintra)
is positive that is intermanifold neighborhood distanceof tensors is bigger than the intramanifold neighborhooddistance which are projected along this direction Thereforethe projection matrix U
119901= [119906
1199011
1199061199012
1199061199011198891015840] consists
of all of the eigen-vectors which are corresponding to thepositive eigen-values Thus the tensor data which are insubmanifold 119872
119894can be embedded in a low-dimensional
space via multilinear projection matrix U1 U
2 U
3 In this
lower-dimensional space the difference between tensor dataand its intramanifold neighborhood points decreases and thedifference between it and its intermanifold neighborhoodpoints increases so that the distinguishing ability between theobject and the similar ones is greater
4 Visual Tracking Framework
In order to achieve tracking of an object in scenariosBayesian sequence inference is used to obtain the object finalstate Meanwhile the multi-manifold datasets and the multi-linear projection matrice which are calculated from multi-manifold discriminate analysis should be updated
41 Sequence Inference In the visual tracking problem themovement of the object is unable to predict the object statein the current frame only related to that in the prior framethen the visual tracking process satisfies the Markov process[23] A bounding box 119900
119905= (119909
119905 119910
119905 119908
119905 ℎ
119905) is used to describe
the object state at the 119905th frame where (119909119905 119910
119905) 119908
119905 ℎ
119905denote
the upper left corner coordinate the width and height of thebounding box
Given a set of observed object appearance images 119878119905=
1199041 119904
2 119904
119905 the objective of visual tracking is to obtain the
optimal estimate value of the hidden state variables 119900119905 There
is a similar result as that of the object state which is obtainedaccording to Bayesrsquo theorem Consider
119875 (119900119905| 119878
119905) prop 119875 (119904
119905| 119900
119905) int119875 (119900
119905| 119900
119905minus1) 119875 (119900
119905minus1| 119878
119905minus1) 119889119900
119905minus1
(17)
where 119875(119900119905| 119900
119905minus1) refers to the state transition model and
119875(119904119905| 119900
119905) refers to the observation model According the
observation model 119875(119904119905
| 119900119905) we can obtain the tracking
results
State Transition Model This was used to model the move-ment of object between consecutive frames Because of theirregular movement of object the object state is difficult topredict and the moving speed of the object is not very fast Itis considered that the object state in the current frame is nearto that in the prior frameThen the object state 119900
119905is modeled
by independent Gaussian distribution around its counterpartin state 119900
119905minus1 described as
119875 (119900119905| 119900
119905minus1) = 119873 (119900
119905 119900
119905minus1 Σ) (18)
where Σ means the diagonal covariance matrix correspond-ing to the variables 119909
119905 119910
119905 119908
119905 ℎ
119905 and the elements are 1205902
119909 1205902
119910
1205902
119908 1205902
ℎ 119873 particles can be randomly generated pointing to
Gaussian distribution Each particle corresponds to an objectstate then 119873 particles can obtain multiple states 119900
119894
119905 119894 =
1 2 119873 During the visual tracking process the morethe particles we generated are the more accurate the objectstate estimate was but at the same time the computationalefficiency was low For the purpose of efficient and effectiveof the visual tracking algorithm there is a balance soughtbetween these factors
Observation Model This was used to measure the differ-ence between the appearance observation and the objectappearance model Given a drawn particle state 119900
119894
119905and the
corresponding cropped image patch 119911119894
119905in the frame image 119868
119905
the probability of an image patch being generated from thesubmanifold space is inversely proportional to the differencebetween image patch and the appearance model and couldbe calculated between the negative exponential distance ofthe projected data and the weighted center of submanifoldConsider
119901 (z119895119905| 119900
119905) = exp
minus
(10038171003817100381710038171003817(z119895
119905minus 119862
119894) times
1U1times2U2times3U3
10038171003817100381710038171003817119865)
1205902
(19)
6 Mathematical Problems in Engineering
where120590 indicates the bandwidth sdot 119865is the Frobenius norm
andU1U
2U
3are themultilinear projectionmatrix of the 119894th
object in submanifold119872119894
The state 119900119894
119905corresponding to the maximum 119901(119911
119894
119905|
119900119905) is the optimal object state at the 119905th frame Let 120576 =
(z119895119905minus 119862
119894) times
1U1times2U2times3U3119865represent the error between
feature tensor which is calculated by observation z119895119905and the
weighted center 119862119894of submanifold119872
119894
42 Multimanifold Data Sets Update The appearance imageof the object changeswith themovement of it in the scenariosthe submanifold of the object should have different postureobject appearance feature tensors Therefore the multiman-ifold data set should be updated in the tracking processBecause of the factors such as occlusion and so forth whichinfluence the object appearance the appearance images ofthe tracked object have the non-object information thenobtained object feature tensor will not be in the submanifoldTherefore the update strategy is necessary From the perspec-tive of the human sensory vision the appearance informationof object changes in the process of occlusion the changes ofobject between consecutive frames are bigger or the objectfeature tensor is far awaywith the center of submanifold in theembedded space while the changing information betweenconsecutive frames is small or the object feature tensor is nearthe center of submanifold in the embedded space that is theobject state is well determined
The image first-order entropy is used to describe the grayvalue distribution of the object image but not to considerit spatial distribution while the image second-order entropyuses the 2-tuple feature (119894 119895) which is calculated by spatialdistributionThe image second-order entropy could describethe changes of the object where 119894 is the gray value (0 le 119894 le
255) and 119895 is the neighborhood gray value (0 le 119895 le 255)119901119894119895
= 119891(119894 119895)119886119887 denotes the gray value and neighborhoodgray distribution where119891(119894 119895) is the counts of the occurrenceof the 2-tuple feature and 119886119887 is the size of imageThe second-order entropy is defined as
119867 =
255
sum
119894=0
119864119894=
255
sum
119894=0
119901119894119895ln119901
119894119895 (20)
Thedifference of the object in consecutive frames is describedby the second-order entropyWhen the second-order entropydifference of the object image in consecutive frames is biggerthe objectmaybe occluded Simultaneously the feature tensorof appearance image would be far away from the weightedcenter of submanifold namely the error is bigger As shownin Figure 2 the object is largely occluded at the frames 33ndash46and 48ndash63 and small part occluded at the frames 69ndash77
For a best state 119900119905of object 119894 which is newly obtained
when the difference of second-order entropy with the priorframe 119867
119889lt 120575119867
119889and the error in low-dimensional tensor
space embedded 120576 gt 120575120576119872119894
the feature tensor calculatedby the newly obtained object state 119900
119905should add into the
submanifold 119872119894 where 119867
119889is mean of the difference of
second-order entropy 120576119872119894
is the mean of the errors and 120575 isthe adjustment factor which takes 12 in this experiment
When the tensor number in a submanifold 119872119894is the
multiples of the initial number the multimanifold discrimi-nate analysis is computed on the newmultimanifold datasetsthen the weighted center of submanifold and multilinearprojectionmatrices are updatedThere will be a small portionof the determined object data abandoned but the tensorswhich added into the data set are essentially the featuretensors of object appearance
The whole tracking algorithm is working as follows
(1) Locate the object state in the first frame eithermanually or by using an automated detector
(2) Tracking objects use template matching trackingalgorithm in the initial119898 frames
1 sdot sdot sdot 119898) from each object appearance images whichare cropped according to the obtained objects states
(4) Construct the multimanifold dataset 119872 using theobtained feature tensors X
119894119895(119894 = 1 sdot sdot sdot 119873
119900 119895 = 1 sdot sdot sdot 119898)
(5) Determine the neighborhood relationship using ten-sor distance in the multimanifold dataset
(6) Calculate the weighted centers of each submanifoldand themultilinear embeddedmatrices throughmul-timanifold discriminate analysis
(7) Advance to the next frame 119905 Draw particles accordingto the object prior state 119900
119905minus1and crop the appearance
images corresponding to each of the particles Extractthe feature tensors of each of the appearance imagesThe best object state in current frame is calculated byBayesian sequence inference
(8) Calculate the difference of second-order entropy withthe prior frame and the error in low-dimensionaltensor space embedded if 119867
119889lt 120575119867
119889and 120576 gt 120575120576
119872119894
the feature tensor calculated by the newly obtainedobject state 119900
119905should add into the submanifold119872
119894
(9) When the tensor number in a submanifold 119872119894is the
multiples of the initial number119898 go to step (3)
5 Comparative Experiments and Analysis
In order to verify the effectiveness of the proposed algorithmCAVIAR data sets and PETS outdoor multiperson data setsare used to be verified The initial state of a moving objectis determined by automatically tracking detectors [24] orartificial markers The initial multimanifold data set is calcu-lated by the object states which come from templatematchingtracking algorithm The proposed algorithm is comparedwith three state-of-the-art trackers which are IVT [4] L1-APG [9] and MIL [14] The Bayesian sequence inferenceneeds to consider the particle number which impacts onthe overall efficiency of the algorithm the particle numberis chosen to be 200 for comprehensive consideration Eachobject appearance image is resized to a 64 times 32 times 3 patch
51 CAVIAR Data Sets In this experiment the experimentscenarios come from the Portugal Mall surveillance video
Mathematical Problems in Engineering 7
The second-order entropy difference of theobject image in consecutive frames
0
10000
20000
30000
40000
50000
60000
70000
0 10 20 30 40 50 60 70 80 900
002
004
006
008
01
012
0 10 20 30 40 50 60 70 80 90
The difference of the object feature tensor and the weighted center of submanifold
Figure 2 The change of the object in consecutive frames
data sets There are object scale change pose variation andocclusion during the three objects walking away from thecamera Testing video sequences are color images of 388 times
284 resolutions The Gaussian variances of the three objectsare (8 8 05 05) (4 4 05 05) (2 2 05 05) The results areshown in Figure 3
As can be seen from the results the threemain objects didnot occlude before the initial 57 frames the three comparisonalgorithms can achieve tracking Since the 57th frame object2 gradually occludes object 3 until object 3 is unable to beseen while the IVT and L1-APG algorithms are all missingobject 3 and offset to object 2 which led to the wrong trackingSince the 87th frame object 1 gradually occludes object 3while the IVT tracker could not distinguish them due tothe fact that object 1 is similar to object 3 and then object3 is mistaken as object 1 which carried the wrong trackingMeanwhile the color of object 2 is largely different fromobject 2 and object 3 the IVT and L1-APG trackers canachieve the better results in tracking object 2TheMIL trackerdid not achieve the accurate tracking on the three objectsdue to the interference of the background The proposedalgorithm achieved complete tracking on the three objectswhich was not subject to the interference of similar object inthe tracking process
52 PETSOutdoorMultipersonData Sets In this experimentthe experiment scenarios come from the PETS2009 surveil-lance video data sets There are multiple human objects thatmove around in multiple directions in the scenarios whichare similar to each other The objects cross occlusion andthe objects scale pose variation during the walking Testingvideo sequences are color images of 768 times 576 resolutions
The Gaussian variances of the four objects are (4 3 05 05)(4 4 05 05) (2 2 05 05) (6 6 05 05) The results are inFigure 4
As can be seen from the results object 2 graduallycompletely occludes object 1 since the 26th frame whichmakes object 1 lost most of its information Then the IVTandL1-APG trackers lost object 1 while they achieved trackingobject 2 which is not occluded The MIL tracker roughlyachieves tracking of objects 1 and 2 Object 1 occludes object3 in the 36th frame then the IVT L1-APG and MIL trackersare disturbed by object 1 when tracking object 3 the threealgorithms are all offset to object 1 because object 1 andobject 3 are very similar Object 1 is occluded by object4 since the 56th frame the IVT and L1-APG trackers aredisturbed by object 1 when tracking object 1The two trackerslost object 4 and offset to object 1 while the MIL trackerachieved tracking object 4 Object 4 and object 2 mutualoccluded since the 64th frame MIL tracker failed to trackobject 4 while the IVT and L1-APG are completely wrongtrackingThis video sequence often occurs an object occludedanother one which made the tracking very difficult theproposed algorithm tracking successfully without excessiveinterference with similar objects and achieved a completetracking of the four objects
53 Quantitative Evaluation Aside from the qualitative com-parison we used two metrics to quantitatively compare theexperimental results of the tracking algorithms which aretracking success ratio and center location error [20] Weinitially manually labeled ldquoground truthrdquo locations in eachexperimental scenario
8 Mathematical Problems in Engineering
Figure 3 Some experiments results on CAVIAR data sets (proposed algorithm results 1st 5th row IVT algorithm results 2nd 6th rowL1-APG algorithm results 3rd 7th row MIL algorithm results 4th 8th row frames 1 42 57 87 93 108 118 148 200 and 282)
Mathematical Problems in Engineering 9
Figure 4 Some experiments results on PETS outdoor multiperson data sets (proposed algorithm results 1st 5th row IVT algorithm results2nd 6th row L1-APG algorithm results 3rd 7th row MIL algorithm results 4th 8th row frames 1 26 31 36 48 56 59 64 68 and 90)
10 Mathematical Problems in Engineering
0 50 100 150 200 250 3000
010203040506070809
1Object success ratio
Frame index
Ratio
(a) Scene 1-object 1
0010203040506070809
1
Ratio
0 50 100 150 200 250 300
Object success ratio
Frame index
(b) Scene 1-object 2
0010203040506070809
1
Ratio
0 50 100 150 200 250 300
Object success ratio
Frame index
(c) Scene 1-object 3
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(d) Scene 2-object 1
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(e) Scene 2-object 2
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(f) Scene 2-object 3
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(g) Scene 2-object 4
Figure 5 Tracking success ratio (the red line is the proposed method results the green line is the IVT results the blue line is the L1-APGresults and the yellow line is MIL results)
The tracking success ratio is
ratio =
area (119877e cap 119877119892)
area (119877e cup 119877119892)
(21)
where 119877e is the experiment tracking bounding box 119877119892is the
ground truth bounding box and area() means the area ofthe region The tracking result in one frame is considered asa success when the tracking success ratio is above 05 Thetracking success ratios of four trackers in two scenarios areshown in Figure 5
As can be seen from Figure 5 the IVT and L1-APGtrackers achieve tracking of object 2 in the first scenarios the
three comparison trackers do not achieve completely trackingof other objects in both scenarios due to the disturbance ofbackground information or the similar objects The trackingsuccess ratios of the proposed algorithm with seven objectsin two scenarios are all greater than 05 which means that thealgorithm achieved accurate tracking and is essentially betterthan the other three trackers
The center location error between experiment boundingbox and ground truth bounding box is
where 119909e 119909119892 119910e 119910119892 are the 119909-axis and 119910-axis coordinates ofthe center of the experiment tracking bounding box and theground truth bounding box
The errors of four trackers in two scenarios are shown inTable 1 S2-O2-err represents the center location error of thesecond object in scenarios 2The data in bold refer to optimalresults
As can be seen from Table 1 the other three trackersrarely achieve a complete tracking so the tracking centerpoint errors is large The errors in the proposed method aresignificantly better than the other three trackers and theerrors are within the acceptable range
Our tracker is implemented in MATLAB 2012a and runsat 11 frames and 08 frames per second on an Inter Xeon24GHz CPU with 8GB RAM which is lacking in real-time
6 Conclusions
In this paper we proposed a visual object tracking algorithmvia feature tensor multimanifold discriminate analysis whichconsiders the tracking is vulnerable to the interference ofsimilar objects The object appearance model described byfeature tensor can maintain the object spatial structuralwhich helps to deal with the partial occlusion problem andhelps better to distinguish the object with similar ones inthe embedded low-dimensional subspace throughmultiman-ifold discriminate analysis In addition the update strategy isdesigned from the perspective of object appearance changewhich is used to determine if it is needed to update themultimanifold datasets As can be seen from the comparisonexperiments the proposed algorithm is able to adapt tothe object pose variation scale change and undisturbedtracking of similar objects in scenarios and also can achievecomplete tracking even if the object was completely occludedThe proposed algorithm exist some defects and when theobject is continuously occluded in the dense moving objectsscenarios the object appearance will be incomplete whichcannot construct an accurate multimanifold datasets thatcaused tracking failure
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
This work is supported by the National Natural ScienceFoundation of China (10771043) and the National Natural
Science Foundation of Inner-Mongolia Autonomous RegionChina under Grant (2012MS0931)
References
[1] H Yang L Shao F Zheng L Wang and Z Song ldquoRecentadvances and trends in visual tracking a reviewrdquoNeurocomput-ing vol 74 no 18 pp 3823ndash3831 2011
[2] M J Black and A D Jepson ldquoEigentracking robust matchingand tracking of articulated objects using a view-based represen-tationrdquo International Journal of Computer Vision vol 26 no 1pp 63ndash84 1998
[3] I Matthews T Ishikawa and S Baker ldquoThe template updateproblemrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 26 no 6 pp 810ndash815 2004
[4] D A Ross J Lim R-S Lin and M-H Yang ldquoIncrementallearning for robust visual trackingrdquo International Journal ofComputer Vision vol 77 no 1ndash3 pp 125ndash141 2008
[5] W Hu X Li X Zhang X Shi S Maybank and Z ZhangldquoIncremental tensor subspace learning and Its applications toforeground segmentation and trackingrdquo International Journal ofComputer Vision vol 91 no 3 pp 303ndash327 2011
[6] D Comaniciu V Ramesh and P Meer ldquoKernel-based objecttrackingrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 25 no 5 pp 564ndash577 2003
[7] A Adam E Rivlin and I Shimshoni ldquoRobust fragments-basedtracking using the integral histogramrdquo in Proceedings of theIEEE Computer Society Conference on Computer Vision andPattern Recognition (CVPR rsquo06) pp 798ndash805 June 2006
[8] X Mei and H Ling ldquoRobust visual tracking and vehicleclassification via sparse representationrdquo IEEE Transactions onPattern Analysis and Machine Intelligence vol 33 no 11 pp2259ndash2272 2011
[9] C Bao Y Wu H Ling and H Ji ldquoReal time robust L1 trackerusing accelerated proximal gradient approachrdquo in Proceedingsof the IEEE Conference on Computer Vision and Pattern Recog-nition (CVPR rsquo12) pp 1830ndash1837 June 2012
[10] T Zhang B Ghanem S Liu and N Ahuja ldquoRobust visualtracking via structured multi-task sparse learningrdquo Interna-tional Journal of Computer Vision vol 101 no 2 pp 367ndash3832013
[11] HGrabnerMGrabner andH Bischof ldquoReal-time tracking viaon-line boostingrdquo in Proceedings of the British Machine VisionConference (BMVC rsquo06) pp 47ndash56 September 2006
[12] H Grabner C Leistner and H Bischof ldquoSemi-supervised on-line boosting for robust trackingrdquo inProceedings of the EuropeanConference on Computer Vision pp 234ndash247 Marseille FranceOctober 2008
[13] S Avidan ldquoEnsemble trackingrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 29 no 2 pp 261ndash2712007
12 Mathematical Problems in Engineering
[14] B Babenko M-H Yang and S Belongie ldquoRobust object track-ing with online multiple instance learningrdquo IEEE Transactionson Pattern Analysis and Machine Intelligence vol 33 no 8 pp1619ndash1632 2011
[15] K Zhang and H Song ldquoReal-time visual tracking via onlineweighted multiple instance learningrdquo Pattern Recognition vol46 no 1 pp 397ndash411 2013
[16] S Avidan ldquoSupport vector trackingrdquo IEEE Transactions onPattern Analysis and Machine Intelligence vol 26 no 8 pp1064ndash1072 2004
[17] R T Collins Y Liu and M Leordeanu ldquoOnline selection ofdiscriminative tracking featuresrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 27 no 10 pp 1631ndash16432005
[18] J Kwon and K M Lee ldquoVisual tracking decompositionrdquoin Proceedings of the IEEE Computer Society Conference onComputer Vision and Pattern Recognition (CVPR 10) pp 1269ndash1276 San Francisco Calif USA June 2010
[19] Z Kalal K Mikolajczyk and J Matas ldquoTracking-learning-detectionrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 34 no 7 pp 1409ndash1422 2012
[20] K Zhang L Zhang and M H Yang ldquoReal-time compressivetrackingrdquo in Proceedings of the European Conference on Com-puter Vision pp 864ndash877 2012
[21] H Lu K N Plataniotis and A N Venetsanopoulos ldquoAsurvey of multilinear subspace learning for tensor datardquo PatternRecognition vol 44 no 7 pp 1540ndash1551 2011
[22] W Yang C Sun and L Zhang ldquoAmulti-manifold discriminantanalysis method for image feature extractionrdquo Pattern Recogni-tion vol 44 no 8 pp 1649ndash1657 2011
[23] J Sherrah B Ristic and N J Redding ldquoParticle filter to trackmultiple people for visual surveillancerdquo IET Computer Visionvol 5 no 4 pp 192ndash200 2011
[24] P Dollar CWojek B Schiele and P Perona ldquoPedestrian detec-tion an evaluation of the state of the artrdquo IEEE Transactions onPatternAnalysis andMachine Intelligence vol 34 no 4 pp 743ndash761 2012
To maximize the 119891(U119901) by solving the eigen-value equation
(Ainter minus Aintra) 119906119901 = 120582119906119901 (16)
obtain U119901
The eigen-values are 1205821
ge 1205822
ge sdot sdot sdot ge 1205821198891015840 ge 0 ge
1205821198891015840+1
ge sdot sdot sdot ge 120582119889 the corresponding eigen-vector of eigen-
value 120582119901is [119906
1199011
1199061199012
119906119901119889
] where 119889 is the dimension 119901thorder in the original feature tensor from submanifold 119872
119894
The directional projection positive along the eigen-vector 119906119901119897
which is corresponding to the eigen-value 120582119897of (AinterminusAintra)
is positive that is intermanifold neighborhood distanceof tensors is bigger than the intramanifold neighborhooddistance which are projected along this direction Thereforethe projection matrix U
119901= [119906
1199011
1199061199012
1199061199011198891015840] consists
of all of the eigen-vectors which are corresponding to thepositive eigen-values Thus the tensor data which are insubmanifold 119872
119894can be embedded in a low-dimensional
space via multilinear projection matrix U1 U
2 U
3 In this
lower-dimensional space the difference between tensor dataand its intramanifold neighborhood points decreases and thedifference between it and its intermanifold neighborhoodpoints increases so that the distinguishing ability between theobject and the similar ones is greater
4 Visual Tracking Framework
In order to achieve tracking of an object in scenariosBayesian sequence inference is used to obtain the object finalstate Meanwhile the multi-manifold datasets and the multi-linear projection matrice which are calculated from multi-manifold discriminate analysis should be updated
41 Sequence Inference In the visual tracking problem themovement of the object is unable to predict the object statein the current frame only related to that in the prior framethen the visual tracking process satisfies the Markov process[23] A bounding box 119900
119905= (119909
119905 119910
119905 119908
119905 ℎ
119905) is used to describe
the object state at the 119905th frame where (119909119905 119910
119905) 119908
119905 ℎ
119905denote
the upper left corner coordinate the width and height of thebounding box
Given a set of observed object appearance images 119878119905=
1199041 119904
2 119904
119905 the objective of visual tracking is to obtain the
optimal estimate value of the hidden state variables 119900119905 There
is a similar result as that of the object state which is obtainedaccording to Bayesrsquo theorem Consider
119875 (119900119905| 119878
119905) prop 119875 (119904
119905| 119900
119905) int119875 (119900
119905| 119900
119905minus1) 119875 (119900
119905minus1| 119878
119905minus1) 119889119900
119905minus1
(17)
where 119875(119900119905| 119900
119905minus1) refers to the state transition model and
119875(119904119905| 119900
119905) refers to the observation model According the
observation model 119875(119904119905
| 119900119905) we can obtain the tracking
results
State Transition Model This was used to model the move-ment of object between consecutive frames Because of theirregular movement of object the object state is difficult topredict and the moving speed of the object is not very fast Itis considered that the object state in the current frame is nearto that in the prior frameThen the object state 119900
119905is modeled
by independent Gaussian distribution around its counterpartin state 119900
119905minus1 described as
119875 (119900119905| 119900
119905minus1) = 119873 (119900
119905 119900
119905minus1 Σ) (18)
where Σ means the diagonal covariance matrix correspond-ing to the variables 119909
119905 119910
119905 119908
119905 ℎ
119905 and the elements are 1205902
119909 1205902
119910
1205902
119908 1205902
ℎ 119873 particles can be randomly generated pointing to
Gaussian distribution Each particle corresponds to an objectstate then 119873 particles can obtain multiple states 119900
119894
119905 119894 =
1 2 119873 During the visual tracking process the morethe particles we generated are the more accurate the objectstate estimate was but at the same time the computationalefficiency was low For the purpose of efficient and effectiveof the visual tracking algorithm there is a balance soughtbetween these factors
Observation Model This was used to measure the differ-ence between the appearance observation and the objectappearance model Given a drawn particle state 119900
119894
119905and the
corresponding cropped image patch 119911119894
119905in the frame image 119868
119905
the probability of an image patch being generated from thesubmanifold space is inversely proportional to the differencebetween image patch and the appearance model and couldbe calculated between the negative exponential distance ofthe projected data and the weighted center of submanifoldConsider
119901 (z119895119905| 119900
119905) = exp
minus
(10038171003817100381710038171003817(z119895
119905minus 119862
119894) times
1U1times2U2times3U3
10038171003817100381710038171003817119865)
1205902
(19)
6 Mathematical Problems in Engineering
where120590 indicates the bandwidth sdot 119865is the Frobenius norm
andU1U
2U
3are themultilinear projectionmatrix of the 119894th
object in submanifold119872119894
The state 119900119894
119905corresponding to the maximum 119901(119911
119894
119905|
119900119905) is the optimal object state at the 119905th frame Let 120576 =
(z119895119905minus 119862
119894) times
1U1times2U2times3U3119865represent the error between
feature tensor which is calculated by observation z119895119905and the
weighted center 119862119894of submanifold119872
119894
42 Multimanifold Data Sets Update The appearance imageof the object changeswith themovement of it in the scenariosthe submanifold of the object should have different postureobject appearance feature tensors Therefore the multiman-ifold data set should be updated in the tracking processBecause of the factors such as occlusion and so forth whichinfluence the object appearance the appearance images ofthe tracked object have the non-object information thenobtained object feature tensor will not be in the submanifoldTherefore the update strategy is necessary From the perspec-tive of the human sensory vision the appearance informationof object changes in the process of occlusion the changes ofobject between consecutive frames are bigger or the objectfeature tensor is far awaywith the center of submanifold in theembedded space while the changing information betweenconsecutive frames is small or the object feature tensor is nearthe center of submanifold in the embedded space that is theobject state is well determined
The image first-order entropy is used to describe the grayvalue distribution of the object image but not to considerit spatial distribution while the image second-order entropyuses the 2-tuple feature (119894 119895) which is calculated by spatialdistributionThe image second-order entropy could describethe changes of the object where 119894 is the gray value (0 le 119894 le
255) and 119895 is the neighborhood gray value (0 le 119895 le 255)119901119894119895
= 119891(119894 119895)119886119887 denotes the gray value and neighborhoodgray distribution where119891(119894 119895) is the counts of the occurrenceof the 2-tuple feature and 119886119887 is the size of imageThe second-order entropy is defined as
119867 =
255
sum
119894=0
119864119894=
255
sum
119894=0
119901119894119895ln119901
119894119895 (20)
Thedifference of the object in consecutive frames is describedby the second-order entropyWhen the second-order entropydifference of the object image in consecutive frames is biggerthe objectmaybe occluded Simultaneously the feature tensorof appearance image would be far away from the weightedcenter of submanifold namely the error is bigger As shownin Figure 2 the object is largely occluded at the frames 33ndash46and 48ndash63 and small part occluded at the frames 69ndash77
For a best state 119900119905of object 119894 which is newly obtained
when the difference of second-order entropy with the priorframe 119867
119889lt 120575119867
119889and the error in low-dimensional tensor
space embedded 120576 gt 120575120576119872119894
the feature tensor calculatedby the newly obtained object state 119900
119905should add into the
submanifold 119872119894 where 119867
119889is mean of the difference of
second-order entropy 120576119872119894
is the mean of the errors and 120575 isthe adjustment factor which takes 12 in this experiment
When the tensor number in a submanifold 119872119894is the
multiples of the initial number the multimanifold discrimi-nate analysis is computed on the newmultimanifold datasetsthen the weighted center of submanifold and multilinearprojectionmatrices are updatedThere will be a small portionof the determined object data abandoned but the tensorswhich added into the data set are essentially the featuretensors of object appearance
The whole tracking algorithm is working as follows
(1) Locate the object state in the first frame eithermanually or by using an automated detector
(2) Tracking objects use template matching trackingalgorithm in the initial119898 frames
1 sdot sdot sdot 119898) from each object appearance images whichare cropped according to the obtained objects states
(4) Construct the multimanifold dataset 119872 using theobtained feature tensors X
119894119895(119894 = 1 sdot sdot sdot 119873
119900 119895 = 1 sdot sdot sdot 119898)
(5) Determine the neighborhood relationship using ten-sor distance in the multimanifold dataset
(6) Calculate the weighted centers of each submanifoldand themultilinear embeddedmatrices throughmul-timanifold discriminate analysis
(7) Advance to the next frame 119905 Draw particles accordingto the object prior state 119900
119905minus1and crop the appearance
images corresponding to each of the particles Extractthe feature tensors of each of the appearance imagesThe best object state in current frame is calculated byBayesian sequence inference
(8) Calculate the difference of second-order entropy withthe prior frame and the error in low-dimensionaltensor space embedded if 119867
119889lt 120575119867
119889and 120576 gt 120575120576
119872119894
the feature tensor calculated by the newly obtainedobject state 119900
119905should add into the submanifold119872
119894
(9) When the tensor number in a submanifold 119872119894is the
multiples of the initial number119898 go to step (3)
5 Comparative Experiments and Analysis
In order to verify the effectiveness of the proposed algorithmCAVIAR data sets and PETS outdoor multiperson data setsare used to be verified The initial state of a moving objectis determined by automatically tracking detectors [24] orartificial markers The initial multimanifold data set is calcu-lated by the object states which come from templatematchingtracking algorithm The proposed algorithm is comparedwith three state-of-the-art trackers which are IVT [4] L1-APG [9] and MIL [14] The Bayesian sequence inferenceneeds to consider the particle number which impacts onthe overall efficiency of the algorithm the particle numberis chosen to be 200 for comprehensive consideration Eachobject appearance image is resized to a 64 times 32 times 3 patch
51 CAVIAR Data Sets In this experiment the experimentscenarios come from the Portugal Mall surveillance video
Mathematical Problems in Engineering 7
The second-order entropy difference of theobject image in consecutive frames
0
10000
20000
30000
40000
50000
60000
70000
0 10 20 30 40 50 60 70 80 900
002
004
006
008
01
012
0 10 20 30 40 50 60 70 80 90
The difference of the object feature tensor and the weighted center of submanifold
Figure 2 The change of the object in consecutive frames
data sets There are object scale change pose variation andocclusion during the three objects walking away from thecamera Testing video sequences are color images of 388 times
284 resolutions The Gaussian variances of the three objectsare (8 8 05 05) (4 4 05 05) (2 2 05 05) The results areshown in Figure 3
As can be seen from the results the threemain objects didnot occlude before the initial 57 frames the three comparisonalgorithms can achieve tracking Since the 57th frame object2 gradually occludes object 3 until object 3 is unable to beseen while the IVT and L1-APG algorithms are all missingobject 3 and offset to object 2 which led to the wrong trackingSince the 87th frame object 1 gradually occludes object 3while the IVT tracker could not distinguish them due tothe fact that object 1 is similar to object 3 and then object3 is mistaken as object 1 which carried the wrong trackingMeanwhile the color of object 2 is largely different fromobject 2 and object 3 the IVT and L1-APG trackers canachieve the better results in tracking object 2TheMIL trackerdid not achieve the accurate tracking on the three objectsdue to the interference of the background The proposedalgorithm achieved complete tracking on the three objectswhich was not subject to the interference of similar object inthe tracking process
52 PETSOutdoorMultipersonData Sets In this experimentthe experiment scenarios come from the PETS2009 surveil-lance video data sets There are multiple human objects thatmove around in multiple directions in the scenarios whichare similar to each other The objects cross occlusion andthe objects scale pose variation during the walking Testingvideo sequences are color images of 768 times 576 resolutions
The Gaussian variances of the four objects are (4 3 05 05)(4 4 05 05) (2 2 05 05) (6 6 05 05) The results are inFigure 4
As can be seen from the results object 2 graduallycompletely occludes object 1 since the 26th frame whichmakes object 1 lost most of its information Then the IVTandL1-APG trackers lost object 1 while they achieved trackingobject 2 which is not occluded The MIL tracker roughlyachieves tracking of objects 1 and 2 Object 1 occludes object3 in the 36th frame then the IVT L1-APG and MIL trackersare disturbed by object 1 when tracking object 3 the threealgorithms are all offset to object 1 because object 1 andobject 3 are very similar Object 1 is occluded by object4 since the 56th frame the IVT and L1-APG trackers aredisturbed by object 1 when tracking object 1The two trackerslost object 4 and offset to object 1 while the MIL trackerachieved tracking object 4 Object 4 and object 2 mutualoccluded since the 64th frame MIL tracker failed to trackobject 4 while the IVT and L1-APG are completely wrongtrackingThis video sequence often occurs an object occludedanother one which made the tracking very difficult theproposed algorithm tracking successfully without excessiveinterference with similar objects and achieved a completetracking of the four objects
53 Quantitative Evaluation Aside from the qualitative com-parison we used two metrics to quantitatively compare theexperimental results of the tracking algorithms which aretracking success ratio and center location error [20] Weinitially manually labeled ldquoground truthrdquo locations in eachexperimental scenario
8 Mathematical Problems in Engineering
Figure 3 Some experiments results on CAVIAR data sets (proposed algorithm results 1st 5th row IVT algorithm results 2nd 6th rowL1-APG algorithm results 3rd 7th row MIL algorithm results 4th 8th row frames 1 42 57 87 93 108 118 148 200 and 282)
Mathematical Problems in Engineering 9
Figure 4 Some experiments results on PETS outdoor multiperson data sets (proposed algorithm results 1st 5th row IVT algorithm results2nd 6th row L1-APG algorithm results 3rd 7th row MIL algorithm results 4th 8th row frames 1 26 31 36 48 56 59 64 68 and 90)
10 Mathematical Problems in Engineering
0 50 100 150 200 250 3000
010203040506070809
1Object success ratio
Frame index
Ratio
(a) Scene 1-object 1
0010203040506070809
1
Ratio
0 50 100 150 200 250 300
Object success ratio
Frame index
(b) Scene 1-object 2
0010203040506070809
1
Ratio
0 50 100 150 200 250 300
Object success ratio
Frame index
(c) Scene 1-object 3
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(d) Scene 2-object 1
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(e) Scene 2-object 2
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(f) Scene 2-object 3
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(g) Scene 2-object 4
Figure 5 Tracking success ratio (the red line is the proposed method results the green line is the IVT results the blue line is the L1-APGresults and the yellow line is MIL results)
The tracking success ratio is
ratio =
area (119877e cap 119877119892)
area (119877e cup 119877119892)
(21)
where 119877e is the experiment tracking bounding box 119877119892is the
ground truth bounding box and area() means the area ofthe region The tracking result in one frame is considered asa success when the tracking success ratio is above 05 Thetracking success ratios of four trackers in two scenarios areshown in Figure 5
As can be seen from Figure 5 the IVT and L1-APGtrackers achieve tracking of object 2 in the first scenarios the
three comparison trackers do not achieve completely trackingof other objects in both scenarios due to the disturbance ofbackground information or the similar objects The trackingsuccess ratios of the proposed algorithm with seven objectsin two scenarios are all greater than 05 which means that thealgorithm achieved accurate tracking and is essentially betterthan the other three trackers
The center location error between experiment boundingbox and ground truth bounding box is
where 119909e 119909119892 119910e 119910119892 are the 119909-axis and 119910-axis coordinates ofthe center of the experiment tracking bounding box and theground truth bounding box
The errors of four trackers in two scenarios are shown inTable 1 S2-O2-err represents the center location error of thesecond object in scenarios 2The data in bold refer to optimalresults
As can be seen from Table 1 the other three trackersrarely achieve a complete tracking so the tracking centerpoint errors is large The errors in the proposed method aresignificantly better than the other three trackers and theerrors are within the acceptable range
Our tracker is implemented in MATLAB 2012a and runsat 11 frames and 08 frames per second on an Inter Xeon24GHz CPU with 8GB RAM which is lacking in real-time
6 Conclusions
In this paper we proposed a visual object tracking algorithmvia feature tensor multimanifold discriminate analysis whichconsiders the tracking is vulnerable to the interference ofsimilar objects The object appearance model described byfeature tensor can maintain the object spatial structuralwhich helps to deal with the partial occlusion problem andhelps better to distinguish the object with similar ones inthe embedded low-dimensional subspace throughmultiman-ifold discriminate analysis In addition the update strategy isdesigned from the perspective of object appearance changewhich is used to determine if it is needed to update themultimanifold datasets As can be seen from the comparisonexperiments the proposed algorithm is able to adapt tothe object pose variation scale change and undisturbedtracking of similar objects in scenarios and also can achievecomplete tracking even if the object was completely occludedThe proposed algorithm exist some defects and when theobject is continuously occluded in the dense moving objectsscenarios the object appearance will be incomplete whichcannot construct an accurate multimanifold datasets thatcaused tracking failure
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
This work is supported by the National Natural ScienceFoundation of China (10771043) and the National Natural
Science Foundation of Inner-Mongolia Autonomous RegionChina under Grant (2012MS0931)
References
[1] H Yang L Shao F Zheng L Wang and Z Song ldquoRecentadvances and trends in visual tracking a reviewrdquoNeurocomput-ing vol 74 no 18 pp 3823ndash3831 2011
[2] M J Black and A D Jepson ldquoEigentracking robust matchingand tracking of articulated objects using a view-based represen-tationrdquo International Journal of Computer Vision vol 26 no 1pp 63ndash84 1998
[3] I Matthews T Ishikawa and S Baker ldquoThe template updateproblemrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 26 no 6 pp 810ndash815 2004
[4] D A Ross J Lim R-S Lin and M-H Yang ldquoIncrementallearning for robust visual trackingrdquo International Journal ofComputer Vision vol 77 no 1ndash3 pp 125ndash141 2008
[5] W Hu X Li X Zhang X Shi S Maybank and Z ZhangldquoIncremental tensor subspace learning and Its applications toforeground segmentation and trackingrdquo International Journal ofComputer Vision vol 91 no 3 pp 303ndash327 2011
[6] D Comaniciu V Ramesh and P Meer ldquoKernel-based objecttrackingrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 25 no 5 pp 564ndash577 2003
[7] A Adam E Rivlin and I Shimshoni ldquoRobust fragments-basedtracking using the integral histogramrdquo in Proceedings of theIEEE Computer Society Conference on Computer Vision andPattern Recognition (CVPR rsquo06) pp 798ndash805 June 2006
[8] X Mei and H Ling ldquoRobust visual tracking and vehicleclassification via sparse representationrdquo IEEE Transactions onPattern Analysis and Machine Intelligence vol 33 no 11 pp2259ndash2272 2011
[9] C Bao Y Wu H Ling and H Ji ldquoReal time robust L1 trackerusing accelerated proximal gradient approachrdquo in Proceedingsof the IEEE Conference on Computer Vision and Pattern Recog-nition (CVPR rsquo12) pp 1830ndash1837 June 2012
[10] T Zhang B Ghanem S Liu and N Ahuja ldquoRobust visualtracking via structured multi-task sparse learningrdquo Interna-tional Journal of Computer Vision vol 101 no 2 pp 367ndash3832013
[11] HGrabnerMGrabner andH Bischof ldquoReal-time tracking viaon-line boostingrdquo in Proceedings of the British Machine VisionConference (BMVC rsquo06) pp 47ndash56 September 2006
[12] H Grabner C Leistner and H Bischof ldquoSemi-supervised on-line boosting for robust trackingrdquo inProceedings of the EuropeanConference on Computer Vision pp 234ndash247 Marseille FranceOctober 2008
[13] S Avidan ldquoEnsemble trackingrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 29 no 2 pp 261ndash2712007
12 Mathematical Problems in Engineering
[14] B Babenko M-H Yang and S Belongie ldquoRobust object track-ing with online multiple instance learningrdquo IEEE Transactionson Pattern Analysis and Machine Intelligence vol 33 no 8 pp1619ndash1632 2011
[15] K Zhang and H Song ldquoReal-time visual tracking via onlineweighted multiple instance learningrdquo Pattern Recognition vol46 no 1 pp 397ndash411 2013
[16] S Avidan ldquoSupport vector trackingrdquo IEEE Transactions onPattern Analysis and Machine Intelligence vol 26 no 8 pp1064ndash1072 2004
[17] R T Collins Y Liu and M Leordeanu ldquoOnline selection ofdiscriminative tracking featuresrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 27 no 10 pp 1631ndash16432005
[18] J Kwon and K M Lee ldquoVisual tracking decompositionrdquoin Proceedings of the IEEE Computer Society Conference onComputer Vision and Pattern Recognition (CVPR 10) pp 1269ndash1276 San Francisco Calif USA June 2010
[19] Z Kalal K Mikolajczyk and J Matas ldquoTracking-learning-detectionrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 34 no 7 pp 1409ndash1422 2012
[20] K Zhang L Zhang and M H Yang ldquoReal-time compressivetrackingrdquo in Proceedings of the European Conference on Com-puter Vision pp 864ndash877 2012
[21] H Lu K N Plataniotis and A N Venetsanopoulos ldquoAsurvey of multilinear subspace learning for tensor datardquo PatternRecognition vol 44 no 7 pp 1540ndash1551 2011
[22] W Yang C Sun and L Zhang ldquoAmulti-manifold discriminantanalysis method for image feature extractionrdquo Pattern Recogni-tion vol 44 no 8 pp 1649ndash1657 2011
[23] J Sherrah B Ristic and N J Redding ldquoParticle filter to trackmultiple people for visual surveillancerdquo IET Computer Visionvol 5 no 4 pp 192ndash200 2011
[24] P Dollar CWojek B Schiele and P Perona ldquoPedestrian detec-tion an evaluation of the state of the artrdquo IEEE Transactions onPatternAnalysis andMachine Intelligence vol 34 no 4 pp 743ndash761 2012
where120590 indicates the bandwidth sdot 119865is the Frobenius norm
andU1U
2U
3are themultilinear projectionmatrix of the 119894th
object in submanifold119872119894
The state 119900119894
119905corresponding to the maximum 119901(119911
119894
119905|
119900119905) is the optimal object state at the 119905th frame Let 120576 =
(z119895119905minus 119862
119894) times
1U1times2U2times3U3119865represent the error between
feature tensor which is calculated by observation z119895119905and the
weighted center 119862119894of submanifold119872
119894
42 Multimanifold Data Sets Update The appearance imageof the object changeswith themovement of it in the scenariosthe submanifold of the object should have different postureobject appearance feature tensors Therefore the multiman-ifold data set should be updated in the tracking processBecause of the factors such as occlusion and so forth whichinfluence the object appearance the appearance images ofthe tracked object have the non-object information thenobtained object feature tensor will not be in the submanifoldTherefore the update strategy is necessary From the perspec-tive of the human sensory vision the appearance informationof object changes in the process of occlusion the changes ofobject between consecutive frames are bigger or the objectfeature tensor is far awaywith the center of submanifold in theembedded space while the changing information betweenconsecutive frames is small or the object feature tensor is nearthe center of submanifold in the embedded space that is theobject state is well determined
The image first-order entropy is used to describe the grayvalue distribution of the object image but not to considerit spatial distribution while the image second-order entropyuses the 2-tuple feature (119894 119895) which is calculated by spatialdistributionThe image second-order entropy could describethe changes of the object where 119894 is the gray value (0 le 119894 le
255) and 119895 is the neighborhood gray value (0 le 119895 le 255)119901119894119895
= 119891(119894 119895)119886119887 denotes the gray value and neighborhoodgray distribution where119891(119894 119895) is the counts of the occurrenceof the 2-tuple feature and 119886119887 is the size of imageThe second-order entropy is defined as
119867 =
255
sum
119894=0
119864119894=
255
sum
119894=0
119901119894119895ln119901
119894119895 (20)
Thedifference of the object in consecutive frames is describedby the second-order entropyWhen the second-order entropydifference of the object image in consecutive frames is biggerthe objectmaybe occluded Simultaneously the feature tensorof appearance image would be far away from the weightedcenter of submanifold namely the error is bigger As shownin Figure 2 the object is largely occluded at the frames 33ndash46and 48ndash63 and small part occluded at the frames 69ndash77
For a best state 119900119905of object 119894 which is newly obtained
when the difference of second-order entropy with the priorframe 119867
119889lt 120575119867
119889and the error in low-dimensional tensor
space embedded 120576 gt 120575120576119872119894
the feature tensor calculatedby the newly obtained object state 119900
119905should add into the
submanifold 119872119894 where 119867
119889is mean of the difference of
second-order entropy 120576119872119894
is the mean of the errors and 120575 isthe adjustment factor which takes 12 in this experiment
When the tensor number in a submanifold 119872119894is the
multiples of the initial number the multimanifold discrimi-nate analysis is computed on the newmultimanifold datasetsthen the weighted center of submanifold and multilinearprojectionmatrices are updatedThere will be a small portionof the determined object data abandoned but the tensorswhich added into the data set are essentially the featuretensors of object appearance
The whole tracking algorithm is working as follows
(1) Locate the object state in the first frame eithermanually or by using an automated detector
(2) Tracking objects use template matching trackingalgorithm in the initial119898 frames
1 sdot sdot sdot 119898) from each object appearance images whichare cropped according to the obtained objects states
(4) Construct the multimanifold dataset 119872 using theobtained feature tensors X
119894119895(119894 = 1 sdot sdot sdot 119873
119900 119895 = 1 sdot sdot sdot 119898)
(5) Determine the neighborhood relationship using ten-sor distance in the multimanifold dataset
(6) Calculate the weighted centers of each submanifoldand themultilinear embeddedmatrices throughmul-timanifold discriminate analysis
(7) Advance to the next frame 119905 Draw particles accordingto the object prior state 119900
119905minus1and crop the appearance
images corresponding to each of the particles Extractthe feature tensors of each of the appearance imagesThe best object state in current frame is calculated byBayesian sequence inference
(8) Calculate the difference of second-order entropy withthe prior frame and the error in low-dimensionaltensor space embedded if 119867
119889lt 120575119867
119889and 120576 gt 120575120576
119872119894
the feature tensor calculated by the newly obtainedobject state 119900
119905should add into the submanifold119872
119894
(9) When the tensor number in a submanifold 119872119894is the
multiples of the initial number119898 go to step (3)
5 Comparative Experiments and Analysis
In order to verify the effectiveness of the proposed algorithmCAVIAR data sets and PETS outdoor multiperson data setsare used to be verified The initial state of a moving objectis determined by automatically tracking detectors [24] orartificial markers The initial multimanifold data set is calcu-lated by the object states which come from templatematchingtracking algorithm The proposed algorithm is comparedwith three state-of-the-art trackers which are IVT [4] L1-APG [9] and MIL [14] The Bayesian sequence inferenceneeds to consider the particle number which impacts onthe overall efficiency of the algorithm the particle numberis chosen to be 200 for comprehensive consideration Eachobject appearance image is resized to a 64 times 32 times 3 patch
51 CAVIAR Data Sets In this experiment the experimentscenarios come from the Portugal Mall surveillance video
Mathematical Problems in Engineering 7
The second-order entropy difference of theobject image in consecutive frames
0
10000
20000
30000
40000
50000
60000
70000
0 10 20 30 40 50 60 70 80 900
002
004
006
008
01
012
0 10 20 30 40 50 60 70 80 90
The difference of the object feature tensor and the weighted center of submanifold
Figure 2 The change of the object in consecutive frames
data sets There are object scale change pose variation andocclusion during the three objects walking away from thecamera Testing video sequences are color images of 388 times
284 resolutions The Gaussian variances of the three objectsare (8 8 05 05) (4 4 05 05) (2 2 05 05) The results areshown in Figure 3
As can be seen from the results the threemain objects didnot occlude before the initial 57 frames the three comparisonalgorithms can achieve tracking Since the 57th frame object2 gradually occludes object 3 until object 3 is unable to beseen while the IVT and L1-APG algorithms are all missingobject 3 and offset to object 2 which led to the wrong trackingSince the 87th frame object 1 gradually occludes object 3while the IVT tracker could not distinguish them due tothe fact that object 1 is similar to object 3 and then object3 is mistaken as object 1 which carried the wrong trackingMeanwhile the color of object 2 is largely different fromobject 2 and object 3 the IVT and L1-APG trackers canachieve the better results in tracking object 2TheMIL trackerdid not achieve the accurate tracking on the three objectsdue to the interference of the background The proposedalgorithm achieved complete tracking on the three objectswhich was not subject to the interference of similar object inthe tracking process
52 PETSOutdoorMultipersonData Sets In this experimentthe experiment scenarios come from the PETS2009 surveil-lance video data sets There are multiple human objects thatmove around in multiple directions in the scenarios whichare similar to each other The objects cross occlusion andthe objects scale pose variation during the walking Testingvideo sequences are color images of 768 times 576 resolutions
The Gaussian variances of the four objects are (4 3 05 05)(4 4 05 05) (2 2 05 05) (6 6 05 05) The results are inFigure 4
As can be seen from the results object 2 graduallycompletely occludes object 1 since the 26th frame whichmakes object 1 lost most of its information Then the IVTandL1-APG trackers lost object 1 while they achieved trackingobject 2 which is not occluded The MIL tracker roughlyachieves tracking of objects 1 and 2 Object 1 occludes object3 in the 36th frame then the IVT L1-APG and MIL trackersare disturbed by object 1 when tracking object 3 the threealgorithms are all offset to object 1 because object 1 andobject 3 are very similar Object 1 is occluded by object4 since the 56th frame the IVT and L1-APG trackers aredisturbed by object 1 when tracking object 1The two trackerslost object 4 and offset to object 1 while the MIL trackerachieved tracking object 4 Object 4 and object 2 mutualoccluded since the 64th frame MIL tracker failed to trackobject 4 while the IVT and L1-APG are completely wrongtrackingThis video sequence often occurs an object occludedanother one which made the tracking very difficult theproposed algorithm tracking successfully without excessiveinterference with similar objects and achieved a completetracking of the four objects
53 Quantitative Evaluation Aside from the qualitative com-parison we used two metrics to quantitatively compare theexperimental results of the tracking algorithms which aretracking success ratio and center location error [20] Weinitially manually labeled ldquoground truthrdquo locations in eachexperimental scenario
8 Mathematical Problems in Engineering
Figure 3 Some experiments results on CAVIAR data sets (proposed algorithm results 1st 5th row IVT algorithm results 2nd 6th rowL1-APG algorithm results 3rd 7th row MIL algorithm results 4th 8th row frames 1 42 57 87 93 108 118 148 200 and 282)
Mathematical Problems in Engineering 9
Figure 4 Some experiments results on PETS outdoor multiperson data sets (proposed algorithm results 1st 5th row IVT algorithm results2nd 6th row L1-APG algorithm results 3rd 7th row MIL algorithm results 4th 8th row frames 1 26 31 36 48 56 59 64 68 and 90)
10 Mathematical Problems in Engineering
0 50 100 150 200 250 3000
010203040506070809
1Object success ratio
Frame index
Ratio
(a) Scene 1-object 1
0010203040506070809
1
Ratio
0 50 100 150 200 250 300
Object success ratio
Frame index
(b) Scene 1-object 2
0010203040506070809
1
Ratio
0 50 100 150 200 250 300
Object success ratio
Frame index
(c) Scene 1-object 3
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(d) Scene 2-object 1
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(e) Scene 2-object 2
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(f) Scene 2-object 3
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(g) Scene 2-object 4
Figure 5 Tracking success ratio (the red line is the proposed method results the green line is the IVT results the blue line is the L1-APGresults and the yellow line is MIL results)
The tracking success ratio is
ratio =
area (119877e cap 119877119892)
area (119877e cup 119877119892)
(21)
where 119877e is the experiment tracking bounding box 119877119892is the
ground truth bounding box and area() means the area ofthe region The tracking result in one frame is considered asa success when the tracking success ratio is above 05 Thetracking success ratios of four trackers in two scenarios areshown in Figure 5
As can be seen from Figure 5 the IVT and L1-APGtrackers achieve tracking of object 2 in the first scenarios the
three comparison trackers do not achieve completely trackingof other objects in both scenarios due to the disturbance ofbackground information or the similar objects The trackingsuccess ratios of the proposed algorithm with seven objectsin two scenarios are all greater than 05 which means that thealgorithm achieved accurate tracking and is essentially betterthan the other three trackers
The center location error between experiment boundingbox and ground truth bounding box is
where 119909e 119909119892 119910e 119910119892 are the 119909-axis and 119910-axis coordinates ofthe center of the experiment tracking bounding box and theground truth bounding box
The errors of four trackers in two scenarios are shown inTable 1 S2-O2-err represents the center location error of thesecond object in scenarios 2The data in bold refer to optimalresults
As can be seen from Table 1 the other three trackersrarely achieve a complete tracking so the tracking centerpoint errors is large The errors in the proposed method aresignificantly better than the other three trackers and theerrors are within the acceptable range
Our tracker is implemented in MATLAB 2012a and runsat 11 frames and 08 frames per second on an Inter Xeon24GHz CPU with 8GB RAM which is lacking in real-time
6 Conclusions
In this paper we proposed a visual object tracking algorithmvia feature tensor multimanifold discriminate analysis whichconsiders the tracking is vulnerable to the interference ofsimilar objects The object appearance model described byfeature tensor can maintain the object spatial structuralwhich helps to deal with the partial occlusion problem andhelps better to distinguish the object with similar ones inthe embedded low-dimensional subspace throughmultiman-ifold discriminate analysis In addition the update strategy isdesigned from the perspective of object appearance changewhich is used to determine if it is needed to update themultimanifold datasets As can be seen from the comparisonexperiments the proposed algorithm is able to adapt tothe object pose variation scale change and undisturbedtracking of similar objects in scenarios and also can achievecomplete tracking even if the object was completely occludedThe proposed algorithm exist some defects and when theobject is continuously occluded in the dense moving objectsscenarios the object appearance will be incomplete whichcannot construct an accurate multimanifold datasets thatcaused tracking failure
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
This work is supported by the National Natural ScienceFoundation of China (10771043) and the National Natural
Science Foundation of Inner-Mongolia Autonomous RegionChina under Grant (2012MS0931)
References
[1] H Yang L Shao F Zheng L Wang and Z Song ldquoRecentadvances and trends in visual tracking a reviewrdquoNeurocomput-ing vol 74 no 18 pp 3823ndash3831 2011
[2] M J Black and A D Jepson ldquoEigentracking robust matchingand tracking of articulated objects using a view-based represen-tationrdquo International Journal of Computer Vision vol 26 no 1pp 63ndash84 1998
[3] I Matthews T Ishikawa and S Baker ldquoThe template updateproblemrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 26 no 6 pp 810ndash815 2004
[4] D A Ross J Lim R-S Lin and M-H Yang ldquoIncrementallearning for robust visual trackingrdquo International Journal ofComputer Vision vol 77 no 1ndash3 pp 125ndash141 2008
[5] W Hu X Li X Zhang X Shi S Maybank and Z ZhangldquoIncremental tensor subspace learning and Its applications toforeground segmentation and trackingrdquo International Journal ofComputer Vision vol 91 no 3 pp 303ndash327 2011
[6] D Comaniciu V Ramesh and P Meer ldquoKernel-based objecttrackingrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 25 no 5 pp 564ndash577 2003
[7] A Adam E Rivlin and I Shimshoni ldquoRobust fragments-basedtracking using the integral histogramrdquo in Proceedings of theIEEE Computer Society Conference on Computer Vision andPattern Recognition (CVPR rsquo06) pp 798ndash805 June 2006
[8] X Mei and H Ling ldquoRobust visual tracking and vehicleclassification via sparse representationrdquo IEEE Transactions onPattern Analysis and Machine Intelligence vol 33 no 11 pp2259ndash2272 2011
[9] C Bao Y Wu H Ling and H Ji ldquoReal time robust L1 trackerusing accelerated proximal gradient approachrdquo in Proceedingsof the IEEE Conference on Computer Vision and Pattern Recog-nition (CVPR rsquo12) pp 1830ndash1837 June 2012
[10] T Zhang B Ghanem S Liu and N Ahuja ldquoRobust visualtracking via structured multi-task sparse learningrdquo Interna-tional Journal of Computer Vision vol 101 no 2 pp 367ndash3832013
[11] HGrabnerMGrabner andH Bischof ldquoReal-time tracking viaon-line boostingrdquo in Proceedings of the British Machine VisionConference (BMVC rsquo06) pp 47ndash56 September 2006
[12] H Grabner C Leistner and H Bischof ldquoSemi-supervised on-line boosting for robust trackingrdquo inProceedings of the EuropeanConference on Computer Vision pp 234ndash247 Marseille FranceOctober 2008
[13] S Avidan ldquoEnsemble trackingrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 29 no 2 pp 261ndash2712007
12 Mathematical Problems in Engineering
[14] B Babenko M-H Yang and S Belongie ldquoRobust object track-ing with online multiple instance learningrdquo IEEE Transactionson Pattern Analysis and Machine Intelligence vol 33 no 8 pp1619ndash1632 2011
[15] K Zhang and H Song ldquoReal-time visual tracking via onlineweighted multiple instance learningrdquo Pattern Recognition vol46 no 1 pp 397ndash411 2013
[16] S Avidan ldquoSupport vector trackingrdquo IEEE Transactions onPattern Analysis and Machine Intelligence vol 26 no 8 pp1064ndash1072 2004
[17] R T Collins Y Liu and M Leordeanu ldquoOnline selection ofdiscriminative tracking featuresrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 27 no 10 pp 1631ndash16432005
[18] J Kwon and K M Lee ldquoVisual tracking decompositionrdquoin Proceedings of the IEEE Computer Society Conference onComputer Vision and Pattern Recognition (CVPR 10) pp 1269ndash1276 San Francisco Calif USA June 2010
[19] Z Kalal K Mikolajczyk and J Matas ldquoTracking-learning-detectionrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 34 no 7 pp 1409ndash1422 2012
[20] K Zhang L Zhang and M H Yang ldquoReal-time compressivetrackingrdquo in Proceedings of the European Conference on Com-puter Vision pp 864ndash877 2012
[21] H Lu K N Plataniotis and A N Venetsanopoulos ldquoAsurvey of multilinear subspace learning for tensor datardquo PatternRecognition vol 44 no 7 pp 1540ndash1551 2011
[22] W Yang C Sun and L Zhang ldquoAmulti-manifold discriminantanalysis method for image feature extractionrdquo Pattern Recogni-tion vol 44 no 8 pp 1649ndash1657 2011
[23] J Sherrah B Ristic and N J Redding ldquoParticle filter to trackmultiple people for visual surveillancerdquo IET Computer Visionvol 5 no 4 pp 192ndash200 2011
[24] P Dollar CWojek B Schiele and P Perona ldquoPedestrian detec-tion an evaluation of the state of the artrdquo IEEE Transactions onPatternAnalysis andMachine Intelligence vol 34 no 4 pp 743ndash761 2012
The second-order entropy difference of theobject image in consecutive frames
0
10000
20000
30000
40000
50000
60000
70000
0 10 20 30 40 50 60 70 80 900
002
004
006
008
01
012
0 10 20 30 40 50 60 70 80 90
The difference of the object feature tensor and the weighted center of submanifold
Figure 2 The change of the object in consecutive frames
data sets There are object scale change pose variation andocclusion during the three objects walking away from thecamera Testing video sequences are color images of 388 times
284 resolutions The Gaussian variances of the three objectsare (8 8 05 05) (4 4 05 05) (2 2 05 05) The results areshown in Figure 3
As can be seen from the results the threemain objects didnot occlude before the initial 57 frames the three comparisonalgorithms can achieve tracking Since the 57th frame object2 gradually occludes object 3 until object 3 is unable to beseen while the IVT and L1-APG algorithms are all missingobject 3 and offset to object 2 which led to the wrong trackingSince the 87th frame object 1 gradually occludes object 3while the IVT tracker could not distinguish them due tothe fact that object 1 is similar to object 3 and then object3 is mistaken as object 1 which carried the wrong trackingMeanwhile the color of object 2 is largely different fromobject 2 and object 3 the IVT and L1-APG trackers canachieve the better results in tracking object 2TheMIL trackerdid not achieve the accurate tracking on the three objectsdue to the interference of the background The proposedalgorithm achieved complete tracking on the three objectswhich was not subject to the interference of similar object inthe tracking process
52 PETSOutdoorMultipersonData Sets In this experimentthe experiment scenarios come from the PETS2009 surveil-lance video data sets There are multiple human objects thatmove around in multiple directions in the scenarios whichare similar to each other The objects cross occlusion andthe objects scale pose variation during the walking Testingvideo sequences are color images of 768 times 576 resolutions
The Gaussian variances of the four objects are (4 3 05 05)(4 4 05 05) (2 2 05 05) (6 6 05 05) The results are inFigure 4
As can be seen from the results object 2 graduallycompletely occludes object 1 since the 26th frame whichmakes object 1 lost most of its information Then the IVTandL1-APG trackers lost object 1 while they achieved trackingobject 2 which is not occluded The MIL tracker roughlyachieves tracking of objects 1 and 2 Object 1 occludes object3 in the 36th frame then the IVT L1-APG and MIL trackersare disturbed by object 1 when tracking object 3 the threealgorithms are all offset to object 1 because object 1 andobject 3 are very similar Object 1 is occluded by object4 since the 56th frame the IVT and L1-APG trackers aredisturbed by object 1 when tracking object 1The two trackerslost object 4 and offset to object 1 while the MIL trackerachieved tracking object 4 Object 4 and object 2 mutualoccluded since the 64th frame MIL tracker failed to trackobject 4 while the IVT and L1-APG are completely wrongtrackingThis video sequence often occurs an object occludedanother one which made the tracking very difficult theproposed algorithm tracking successfully without excessiveinterference with similar objects and achieved a completetracking of the four objects
53 Quantitative Evaluation Aside from the qualitative com-parison we used two metrics to quantitatively compare theexperimental results of the tracking algorithms which aretracking success ratio and center location error [20] Weinitially manually labeled ldquoground truthrdquo locations in eachexperimental scenario
8 Mathematical Problems in Engineering
Figure 3 Some experiments results on CAVIAR data sets (proposed algorithm results 1st 5th row IVT algorithm results 2nd 6th rowL1-APG algorithm results 3rd 7th row MIL algorithm results 4th 8th row frames 1 42 57 87 93 108 118 148 200 and 282)
Mathematical Problems in Engineering 9
Figure 4 Some experiments results on PETS outdoor multiperson data sets (proposed algorithm results 1st 5th row IVT algorithm results2nd 6th row L1-APG algorithm results 3rd 7th row MIL algorithm results 4th 8th row frames 1 26 31 36 48 56 59 64 68 and 90)
10 Mathematical Problems in Engineering
0 50 100 150 200 250 3000
010203040506070809
1Object success ratio
Frame index
Ratio
(a) Scene 1-object 1
0010203040506070809
1
Ratio
0 50 100 150 200 250 300
Object success ratio
Frame index
(b) Scene 1-object 2
0010203040506070809
1
Ratio
0 50 100 150 200 250 300
Object success ratio
Frame index
(c) Scene 1-object 3
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(d) Scene 2-object 1
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(e) Scene 2-object 2
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(f) Scene 2-object 3
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(g) Scene 2-object 4
Figure 5 Tracking success ratio (the red line is the proposed method results the green line is the IVT results the blue line is the L1-APGresults and the yellow line is MIL results)
The tracking success ratio is
ratio =
area (119877e cap 119877119892)
area (119877e cup 119877119892)
(21)
where 119877e is the experiment tracking bounding box 119877119892is the
ground truth bounding box and area() means the area ofthe region The tracking result in one frame is considered asa success when the tracking success ratio is above 05 Thetracking success ratios of four trackers in two scenarios areshown in Figure 5
As can be seen from Figure 5 the IVT and L1-APGtrackers achieve tracking of object 2 in the first scenarios the
three comparison trackers do not achieve completely trackingof other objects in both scenarios due to the disturbance ofbackground information or the similar objects The trackingsuccess ratios of the proposed algorithm with seven objectsin two scenarios are all greater than 05 which means that thealgorithm achieved accurate tracking and is essentially betterthan the other three trackers
The center location error between experiment boundingbox and ground truth bounding box is
where 119909e 119909119892 119910e 119910119892 are the 119909-axis and 119910-axis coordinates ofthe center of the experiment tracking bounding box and theground truth bounding box
The errors of four trackers in two scenarios are shown inTable 1 S2-O2-err represents the center location error of thesecond object in scenarios 2The data in bold refer to optimalresults
As can be seen from Table 1 the other three trackersrarely achieve a complete tracking so the tracking centerpoint errors is large The errors in the proposed method aresignificantly better than the other three trackers and theerrors are within the acceptable range
Our tracker is implemented in MATLAB 2012a and runsat 11 frames and 08 frames per second on an Inter Xeon24GHz CPU with 8GB RAM which is lacking in real-time
6 Conclusions
In this paper we proposed a visual object tracking algorithmvia feature tensor multimanifold discriminate analysis whichconsiders the tracking is vulnerable to the interference ofsimilar objects The object appearance model described byfeature tensor can maintain the object spatial structuralwhich helps to deal with the partial occlusion problem andhelps better to distinguish the object with similar ones inthe embedded low-dimensional subspace throughmultiman-ifold discriminate analysis In addition the update strategy isdesigned from the perspective of object appearance changewhich is used to determine if it is needed to update themultimanifold datasets As can be seen from the comparisonexperiments the proposed algorithm is able to adapt tothe object pose variation scale change and undisturbedtracking of similar objects in scenarios and also can achievecomplete tracking even if the object was completely occludedThe proposed algorithm exist some defects and when theobject is continuously occluded in the dense moving objectsscenarios the object appearance will be incomplete whichcannot construct an accurate multimanifold datasets thatcaused tracking failure
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
This work is supported by the National Natural ScienceFoundation of China (10771043) and the National Natural
Science Foundation of Inner-Mongolia Autonomous RegionChina under Grant (2012MS0931)
References
[1] H Yang L Shao F Zheng L Wang and Z Song ldquoRecentadvances and trends in visual tracking a reviewrdquoNeurocomput-ing vol 74 no 18 pp 3823ndash3831 2011
[2] M J Black and A D Jepson ldquoEigentracking robust matchingand tracking of articulated objects using a view-based represen-tationrdquo International Journal of Computer Vision vol 26 no 1pp 63ndash84 1998
[3] I Matthews T Ishikawa and S Baker ldquoThe template updateproblemrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 26 no 6 pp 810ndash815 2004
[4] D A Ross J Lim R-S Lin and M-H Yang ldquoIncrementallearning for robust visual trackingrdquo International Journal ofComputer Vision vol 77 no 1ndash3 pp 125ndash141 2008
[5] W Hu X Li X Zhang X Shi S Maybank and Z ZhangldquoIncremental tensor subspace learning and Its applications toforeground segmentation and trackingrdquo International Journal ofComputer Vision vol 91 no 3 pp 303ndash327 2011
[6] D Comaniciu V Ramesh and P Meer ldquoKernel-based objecttrackingrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 25 no 5 pp 564ndash577 2003
[7] A Adam E Rivlin and I Shimshoni ldquoRobust fragments-basedtracking using the integral histogramrdquo in Proceedings of theIEEE Computer Society Conference on Computer Vision andPattern Recognition (CVPR rsquo06) pp 798ndash805 June 2006
[8] X Mei and H Ling ldquoRobust visual tracking and vehicleclassification via sparse representationrdquo IEEE Transactions onPattern Analysis and Machine Intelligence vol 33 no 11 pp2259ndash2272 2011
[9] C Bao Y Wu H Ling and H Ji ldquoReal time robust L1 trackerusing accelerated proximal gradient approachrdquo in Proceedingsof the IEEE Conference on Computer Vision and Pattern Recog-nition (CVPR rsquo12) pp 1830ndash1837 June 2012
[10] T Zhang B Ghanem S Liu and N Ahuja ldquoRobust visualtracking via structured multi-task sparse learningrdquo Interna-tional Journal of Computer Vision vol 101 no 2 pp 367ndash3832013
[11] HGrabnerMGrabner andH Bischof ldquoReal-time tracking viaon-line boostingrdquo in Proceedings of the British Machine VisionConference (BMVC rsquo06) pp 47ndash56 September 2006
[12] H Grabner C Leistner and H Bischof ldquoSemi-supervised on-line boosting for robust trackingrdquo inProceedings of the EuropeanConference on Computer Vision pp 234ndash247 Marseille FranceOctober 2008
[13] S Avidan ldquoEnsemble trackingrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 29 no 2 pp 261ndash2712007
12 Mathematical Problems in Engineering
[14] B Babenko M-H Yang and S Belongie ldquoRobust object track-ing with online multiple instance learningrdquo IEEE Transactionson Pattern Analysis and Machine Intelligence vol 33 no 8 pp1619ndash1632 2011
[15] K Zhang and H Song ldquoReal-time visual tracking via onlineweighted multiple instance learningrdquo Pattern Recognition vol46 no 1 pp 397ndash411 2013
[16] S Avidan ldquoSupport vector trackingrdquo IEEE Transactions onPattern Analysis and Machine Intelligence vol 26 no 8 pp1064ndash1072 2004
[17] R T Collins Y Liu and M Leordeanu ldquoOnline selection ofdiscriminative tracking featuresrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 27 no 10 pp 1631ndash16432005
[18] J Kwon and K M Lee ldquoVisual tracking decompositionrdquoin Proceedings of the IEEE Computer Society Conference onComputer Vision and Pattern Recognition (CVPR 10) pp 1269ndash1276 San Francisco Calif USA June 2010
[19] Z Kalal K Mikolajczyk and J Matas ldquoTracking-learning-detectionrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 34 no 7 pp 1409ndash1422 2012
[20] K Zhang L Zhang and M H Yang ldquoReal-time compressivetrackingrdquo in Proceedings of the European Conference on Com-puter Vision pp 864ndash877 2012
[21] H Lu K N Plataniotis and A N Venetsanopoulos ldquoAsurvey of multilinear subspace learning for tensor datardquo PatternRecognition vol 44 no 7 pp 1540ndash1551 2011
[22] W Yang C Sun and L Zhang ldquoAmulti-manifold discriminantanalysis method for image feature extractionrdquo Pattern Recogni-tion vol 44 no 8 pp 1649ndash1657 2011
[23] J Sherrah B Ristic and N J Redding ldquoParticle filter to trackmultiple people for visual surveillancerdquo IET Computer Visionvol 5 no 4 pp 192ndash200 2011
[24] P Dollar CWojek B Schiele and P Perona ldquoPedestrian detec-tion an evaluation of the state of the artrdquo IEEE Transactions onPatternAnalysis andMachine Intelligence vol 34 no 4 pp 743ndash761 2012
Figure 3 Some experiments results on CAVIAR data sets (proposed algorithm results 1st 5th row IVT algorithm results 2nd 6th rowL1-APG algorithm results 3rd 7th row MIL algorithm results 4th 8th row frames 1 42 57 87 93 108 118 148 200 and 282)
Mathematical Problems in Engineering 9
Figure 4 Some experiments results on PETS outdoor multiperson data sets (proposed algorithm results 1st 5th row IVT algorithm results2nd 6th row L1-APG algorithm results 3rd 7th row MIL algorithm results 4th 8th row frames 1 26 31 36 48 56 59 64 68 and 90)
10 Mathematical Problems in Engineering
0 50 100 150 200 250 3000
010203040506070809
1Object success ratio
Frame index
Ratio
(a) Scene 1-object 1
0010203040506070809
1
Ratio
0 50 100 150 200 250 300
Object success ratio
Frame index
(b) Scene 1-object 2
0010203040506070809
1
Ratio
0 50 100 150 200 250 300
Object success ratio
Frame index
(c) Scene 1-object 3
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(d) Scene 2-object 1
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(e) Scene 2-object 2
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(f) Scene 2-object 3
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(g) Scene 2-object 4
Figure 5 Tracking success ratio (the red line is the proposed method results the green line is the IVT results the blue line is the L1-APGresults and the yellow line is MIL results)
The tracking success ratio is
ratio =
area (119877e cap 119877119892)
area (119877e cup 119877119892)
(21)
where 119877e is the experiment tracking bounding box 119877119892is the
ground truth bounding box and area() means the area ofthe region The tracking result in one frame is considered asa success when the tracking success ratio is above 05 Thetracking success ratios of four trackers in two scenarios areshown in Figure 5
As can be seen from Figure 5 the IVT and L1-APGtrackers achieve tracking of object 2 in the first scenarios the
three comparison trackers do not achieve completely trackingof other objects in both scenarios due to the disturbance ofbackground information or the similar objects The trackingsuccess ratios of the proposed algorithm with seven objectsin two scenarios are all greater than 05 which means that thealgorithm achieved accurate tracking and is essentially betterthan the other three trackers
The center location error between experiment boundingbox and ground truth bounding box is
where 119909e 119909119892 119910e 119910119892 are the 119909-axis and 119910-axis coordinates ofthe center of the experiment tracking bounding box and theground truth bounding box
The errors of four trackers in two scenarios are shown inTable 1 S2-O2-err represents the center location error of thesecond object in scenarios 2The data in bold refer to optimalresults
As can be seen from Table 1 the other three trackersrarely achieve a complete tracking so the tracking centerpoint errors is large The errors in the proposed method aresignificantly better than the other three trackers and theerrors are within the acceptable range
Our tracker is implemented in MATLAB 2012a and runsat 11 frames and 08 frames per second on an Inter Xeon24GHz CPU with 8GB RAM which is lacking in real-time
6 Conclusions
In this paper we proposed a visual object tracking algorithmvia feature tensor multimanifold discriminate analysis whichconsiders the tracking is vulnerable to the interference ofsimilar objects The object appearance model described byfeature tensor can maintain the object spatial structuralwhich helps to deal with the partial occlusion problem andhelps better to distinguish the object with similar ones inthe embedded low-dimensional subspace throughmultiman-ifold discriminate analysis In addition the update strategy isdesigned from the perspective of object appearance changewhich is used to determine if it is needed to update themultimanifold datasets As can be seen from the comparisonexperiments the proposed algorithm is able to adapt tothe object pose variation scale change and undisturbedtracking of similar objects in scenarios and also can achievecomplete tracking even if the object was completely occludedThe proposed algorithm exist some defects and when theobject is continuously occluded in the dense moving objectsscenarios the object appearance will be incomplete whichcannot construct an accurate multimanifold datasets thatcaused tracking failure
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
This work is supported by the National Natural ScienceFoundation of China (10771043) and the National Natural
Science Foundation of Inner-Mongolia Autonomous RegionChina under Grant (2012MS0931)
References
[1] H Yang L Shao F Zheng L Wang and Z Song ldquoRecentadvances and trends in visual tracking a reviewrdquoNeurocomput-ing vol 74 no 18 pp 3823ndash3831 2011
[2] M J Black and A D Jepson ldquoEigentracking robust matchingand tracking of articulated objects using a view-based represen-tationrdquo International Journal of Computer Vision vol 26 no 1pp 63ndash84 1998
[3] I Matthews T Ishikawa and S Baker ldquoThe template updateproblemrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 26 no 6 pp 810ndash815 2004
[4] D A Ross J Lim R-S Lin and M-H Yang ldquoIncrementallearning for robust visual trackingrdquo International Journal ofComputer Vision vol 77 no 1ndash3 pp 125ndash141 2008
[5] W Hu X Li X Zhang X Shi S Maybank and Z ZhangldquoIncremental tensor subspace learning and Its applications toforeground segmentation and trackingrdquo International Journal ofComputer Vision vol 91 no 3 pp 303ndash327 2011
[6] D Comaniciu V Ramesh and P Meer ldquoKernel-based objecttrackingrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 25 no 5 pp 564ndash577 2003
[7] A Adam E Rivlin and I Shimshoni ldquoRobust fragments-basedtracking using the integral histogramrdquo in Proceedings of theIEEE Computer Society Conference on Computer Vision andPattern Recognition (CVPR rsquo06) pp 798ndash805 June 2006
[8] X Mei and H Ling ldquoRobust visual tracking and vehicleclassification via sparse representationrdquo IEEE Transactions onPattern Analysis and Machine Intelligence vol 33 no 11 pp2259ndash2272 2011
[9] C Bao Y Wu H Ling and H Ji ldquoReal time robust L1 trackerusing accelerated proximal gradient approachrdquo in Proceedingsof the IEEE Conference on Computer Vision and Pattern Recog-nition (CVPR rsquo12) pp 1830ndash1837 June 2012
[10] T Zhang B Ghanem S Liu and N Ahuja ldquoRobust visualtracking via structured multi-task sparse learningrdquo Interna-tional Journal of Computer Vision vol 101 no 2 pp 367ndash3832013
[11] HGrabnerMGrabner andH Bischof ldquoReal-time tracking viaon-line boostingrdquo in Proceedings of the British Machine VisionConference (BMVC rsquo06) pp 47ndash56 September 2006
[12] H Grabner C Leistner and H Bischof ldquoSemi-supervised on-line boosting for robust trackingrdquo inProceedings of the EuropeanConference on Computer Vision pp 234ndash247 Marseille FranceOctober 2008
[13] S Avidan ldquoEnsemble trackingrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 29 no 2 pp 261ndash2712007
12 Mathematical Problems in Engineering
[14] B Babenko M-H Yang and S Belongie ldquoRobust object track-ing with online multiple instance learningrdquo IEEE Transactionson Pattern Analysis and Machine Intelligence vol 33 no 8 pp1619ndash1632 2011
[15] K Zhang and H Song ldquoReal-time visual tracking via onlineweighted multiple instance learningrdquo Pattern Recognition vol46 no 1 pp 397ndash411 2013
[16] S Avidan ldquoSupport vector trackingrdquo IEEE Transactions onPattern Analysis and Machine Intelligence vol 26 no 8 pp1064ndash1072 2004
[17] R T Collins Y Liu and M Leordeanu ldquoOnline selection ofdiscriminative tracking featuresrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 27 no 10 pp 1631ndash16432005
[18] J Kwon and K M Lee ldquoVisual tracking decompositionrdquoin Proceedings of the IEEE Computer Society Conference onComputer Vision and Pattern Recognition (CVPR 10) pp 1269ndash1276 San Francisco Calif USA June 2010
[19] Z Kalal K Mikolajczyk and J Matas ldquoTracking-learning-detectionrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 34 no 7 pp 1409ndash1422 2012
[20] K Zhang L Zhang and M H Yang ldquoReal-time compressivetrackingrdquo in Proceedings of the European Conference on Com-puter Vision pp 864ndash877 2012
[21] H Lu K N Plataniotis and A N Venetsanopoulos ldquoAsurvey of multilinear subspace learning for tensor datardquo PatternRecognition vol 44 no 7 pp 1540ndash1551 2011
[22] W Yang C Sun and L Zhang ldquoAmulti-manifold discriminantanalysis method for image feature extractionrdquo Pattern Recogni-tion vol 44 no 8 pp 1649ndash1657 2011
[23] J Sherrah B Ristic and N J Redding ldquoParticle filter to trackmultiple people for visual surveillancerdquo IET Computer Visionvol 5 no 4 pp 192ndash200 2011
[24] P Dollar CWojek B Schiele and P Perona ldquoPedestrian detec-tion an evaluation of the state of the artrdquo IEEE Transactions onPatternAnalysis andMachine Intelligence vol 34 no 4 pp 743ndash761 2012
Figure 4 Some experiments results on PETS outdoor multiperson data sets (proposed algorithm results 1st 5th row IVT algorithm results2nd 6th row L1-APG algorithm results 3rd 7th row MIL algorithm results 4th 8th row frames 1 26 31 36 48 56 59 64 68 and 90)
10 Mathematical Problems in Engineering
0 50 100 150 200 250 3000
010203040506070809
1Object success ratio
Frame index
Ratio
(a) Scene 1-object 1
0010203040506070809
1
Ratio
0 50 100 150 200 250 300
Object success ratio
Frame index
(b) Scene 1-object 2
0010203040506070809
1
Ratio
0 50 100 150 200 250 300
Object success ratio
Frame index
(c) Scene 1-object 3
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(d) Scene 2-object 1
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(e) Scene 2-object 2
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(f) Scene 2-object 3
0010203040506070809
1
Ratio
0 10 20 30 40 50 60 70 80 90
Object success ratio
Frame index
(g) Scene 2-object 4
Figure 5 Tracking success ratio (the red line is the proposed method results the green line is the IVT results the blue line is the L1-APGresults and the yellow line is MIL results)
The tracking success ratio is
ratio =
area (119877e cap 119877119892)
area (119877e cup 119877119892)
(21)
where 119877e is the experiment tracking bounding box 119877119892is the
ground truth bounding box and area() means the area ofthe region The tracking result in one frame is considered asa success when the tracking success ratio is above 05 Thetracking success ratios of four trackers in two scenarios areshown in Figure 5
As can be seen from Figure 5 the IVT and L1-APGtrackers achieve tracking of object 2 in the first scenarios the
three comparison trackers do not achieve completely trackingof other objects in both scenarios due to the disturbance ofbackground information or the similar objects The trackingsuccess ratios of the proposed algorithm with seven objectsin two scenarios are all greater than 05 which means that thealgorithm achieved accurate tracking and is essentially betterthan the other three trackers
The center location error between experiment boundingbox and ground truth bounding box is
where 119909e 119909119892 119910e 119910119892 are the 119909-axis and 119910-axis coordinates ofthe center of the experiment tracking bounding box and theground truth bounding box
The errors of four trackers in two scenarios are shown inTable 1 S2-O2-err represents the center location error of thesecond object in scenarios 2The data in bold refer to optimalresults
As can be seen from Table 1 the other three trackersrarely achieve a complete tracking so the tracking centerpoint errors is large The errors in the proposed method aresignificantly better than the other three trackers and theerrors are within the acceptable range
Our tracker is implemented in MATLAB 2012a and runsat 11 frames and 08 frames per second on an Inter Xeon24GHz CPU with 8GB RAM which is lacking in real-time
6 Conclusions
In this paper we proposed a visual object tracking algorithmvia feature tensor multimanifold discriminate analysis whichconsiders the tracking is vulnerable to the interference ofsimilar objects The object appearance model described byfeature tensor can maintain the object spatial structuralwhich helps to deal with the partial occlusion problem andhelps better to distinguish the object with similar ones inthe embedded low-dimensional subspace throughmultiman-ifold discriminate analysis In addition the update strategy isdesigned from the perspective of object appearance changewhich is used to determine if it is needed to update themultimanifold datasets As can be seen from the comparisonexperiments the proposed algorithm is able to adapt tothe object pose variation scale change and undisturbedtracking of similar objects in scenarios and also can achievecomplete tracking even if the object was completely occludedThe proposed algorithm exist some defects and when theobject is continuously occluded in the dense moving objectsscenarios the object appearance will be incomplete whichcannot construct an accurate multimanifold datasets thatcaused tracking failure
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
This work is supported by the National Natural ScienceFoundation of China (10771043) and the National Natural
Science Foundation of Inner-Mongolia Autonomous RegionChina under Grant (2012MS0931)
References
[1] H Yang L Shao F Zheng L Wang and Z Song ldquoRecentadvances and trends in visual tracking a reviewrdquoNeurocomput-ing vol 74 no 18 pp 3823ndash3831 2011
[2] M J Black and A D Jepson ldquoEigentracking robust matchingand tracking of articulated objects using a view-based represen-tationrdquo International Journal of Computer Vision vol 26 no 1pp 63ndash84 1998
[3] I Matthews T Ishikawa and S Baker ldquoThe template updateproblemrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 26 no 6 pp 810ndash815 2004
[4] D A Ross J Lim R-S Lin and M-H Yang ldquoIncrementallearning for robust visual trackingrdquo International Journal ofComputer Vision vol 77 no 1ndash3 pp 125ndash141 2008
[5] W Hu X Li X Zhang X Shi S Maybank and Z ZhangldquoIncremental tensor subspace learning and Its applications toforeground segmentation and trackingrdquo International Journal ofComputer Vision vol 91 no 3 pp 303ndash327 2011
[6] D Comaniciu V Ramesh and P Meer ldquoKernel-based objecttrackingrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 25 no 5 pp 564ndash577 2003
[7] A Adam E Rivlin and I Shimshoni ldquoRobust fragments-basedtracking using the integral histogramrdquo in Proceedings of theIEEE Computer Society Conference on Computer Vision andPattern Recognition (CVPR rsquo06) pp 798ndash805 June 2006
[8] X Mei and H Ling ldquoRobust visual tracking and vehicleclassification via sparse representationrdquo IEEE Transactions onPattern Analysis and Machine Intelligence vol 33 no 11 pp2259ndash2272 2011
[9] C Bao Y Wu H Ling and H Ji ldquoReal time robust L1 trackerusing accelerated proximal gradient approachrdquo in Proceedingsof the IEEE Conference on Computer Vision and Pattern Recog-nition (CVPR rsquo12) pp 1830ndash1837 June 2012
[10] T Zhang B Ghanem S Liu and N Ahuja ldquoRobust visualtracking via structured multi-task sparse learningrdquo Interna-tional Journal of Computer Vision vol 101 no 2 pp 367ndash3832013
[11] HGrabnerMGrabner andH Bischof ldquoReal-time tracking viaon-line boostingrdquo in Proceedings of the British Machine VisionConference (BMVC rsquo06) pp 47ndash56 September 2006
[12] H Grabner C Leistner and H Bischof ldquoSemi-supervised on-line boosting for robust trackingrdquo inProceedings of the EuropeanConference on Computer Vision pp 234ndash247 Marseille FranceOctober 2008
[13] S Avidan ldquoEnsemble trackingrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 29 no 2 pp 261ndash2712007
12 Mathematical Problems in Engineering
[14] B Babenko M-H Yang and S Belongie ldquoRobust object track-ing with online multiple instance learningrdquo IEEE Transactionson Pattern Analysis and Machine Intelligence vol 33 no 8 pp1619ndash1632 2011
[15] K Zhang and H Song ldquoReal-time visual tracking via onlineweighted multiple instance learningrdquo Pattern Recognition vol46 no 1 pp 397ndash411 2013
[16] S Avidan ldquoSupport vector trackingrdquo IEEE Transactions onPattern Analysis and Machine Intelligence vol 26 no 8 pp1064ndash1072 2004
[17] R T Collins Y Liu and M Leordeanu ldquoOnline selection ofdiscriminative tracking featuresrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 27 no 10 pp 1631ndash16432005
[18] J Kwon and K M Lee ldquoVisual tracking decompositionrdquoin Proceedings of the IEEE Computer Society Conference onComputer Vision and Pattern Recognition (CVPR 10) pp 1269ndash1276 San Francisco Calif USA June 2010
[19] Z Kalal K Mikolajczyk and J Matas ldquoTracking-learning-detectionrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 34 no 7 pp 1409ndash1422 2012
[20] K Zhang L Zhang and M H Yang ldquoReal-time compressivetrackingrdquo in Proceedings of the European Conference on Com-puter Vision pp 864ndash877 2012
[21] H Lu K N Plataniotis and A N Venetsanopoulos ldquoAsurvey of multilinear subspace learning for tensor datardquo PatternRecognition vol 44 no 7 pp 1540ndash1551 2011
[22] W Yang C Sun and L Zhang ldquoAmulti-manifold discriminantanalysis method for image feature extractionrdquo Pattern Recogni-tion vol 44 no 8 pp 1649ndash1657 2011
[23] J Sherrah B Ristic and N J Redding ldquoParticle filter to trackmultiple people for visual surveillancerdquo IET Computer Visionvol 5 no 4 pp 192ndash200 2011
[24] P Dollar CWojek B Schiele and P Perona ldquoPedestrian detec-tion an evaluation of the state of the artrdquo IEEE Transactions onPatternAnalysis andMachine Intelligence vol 34 no 4 pp 743ndash761 2012
Figure 5 Tracking success ratio (the red line is the proposed method results the green line is the IVT results the blue line is the L1-APGresults and the yellow line is MIL results)
The tracking success ratio is
ratio =
area (119877e cap 119877119892)
area (119877e cup 119877119892)
(21)
where 119877e is the experiment tracking bounding box 119877119892is the
ground truth bounding box and area() means the area ofthe region The tracking result in one frame is considered asa success when the tracking success ratio is above 05 Thetracking success ratios of four trackers in two scenarios areshown in Figure 5
As can be seen from Figure 5 the IVT and L1-APGtrackers achieve tracking of object 2 in the first scenarios the
three comparison trackers do not achieve completely trackingof other objects in both scenarios due to the disturbance ofbackground information or the similar objects The trackingsuccess ratios of the proposed algorithm with seven objectsin two scenarios are all greater than 05 which means that thealgorithm achieved accurate tracking and is essentially betterthan the other three trackers
The center location error between experiment boundingbox and ground truth bounding box is
where 119909e 119909119892 119910e 119910119892 are the 119909-axis and 119910-axis coordinates ofthe center of the experiment tracking bounding box and theground truth bounding box
The errors of four trackers in two scenarios are shown inTable 1 S2-O2-err represents the center location error of thesecond object in scenarios 2The data in bold refer to optimalresults
As can be seen from Table 1 the other three trackersrarely achieve a complete tracking so the tracking centerpoint errors is large The errors in the proposed method aresignificantly better than the other three trackers and theerrors are within the acceptable range
Our tracker is implemented in MATLAB 2012a and runsat 11 frames and 08 frames per second on an Inter Xeon24GHz CPU with 8GB RAM which is lacking in real-time
6 Conclusions
In this paper we proposed a visual object tracking algorithmvia feature tensor multimanifold discriminate analysis whichconsiders the tracking is vulnerable to the interference ofsimilar objects The object appearance model described byfeature tensor can maintain the object spatial structuralwhich helps to deal with the partial occlusion problem andhelps better to distinguish the object with similar ones inthe embedded low-dimensional subspace throughmultiman-ifold discriminate analysis In addition the update strategy isdesigned from the perspective of object appearance changewhich is used to determine if it is needed to update themultimanifold datasets As can be seen from the comparisonexperiments the proposed algorithm is able to adapt tothe object pose variation scale change and undisturbedtracking of similar objects in scenarios and also can achievecomplete tracking even if the object was completely occludedThe proposed algorithm exist some defects and when theobject is continuously occluded in the dense moving objectsscenarios the object appearance will be incomplete whichcannot construct an accurate multimanifold datasets thatcaused tracking failure
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
This work is supported by the National Natural ScienceFoundation of China (10771043) and the National Natural
Science Foundation of Inner-Mongolia Autonomous RegionChina under Grant (2012MS0931)
References
[1] H Yang L Shao F Zheng L Wang and Z Song ldquoRecentadvances and trends in visual tracking a reviewrdquoNeurocomput-ing vol 74 no 18 pp 3823ndash3831 2011
[2] M J Black and A D Jepson ldquoEigentracking robust matchingand tracking of articulated objects using a view-based represen-tationrdquo International Journal of Computer Vision vol 26 no 1pp 63ndash84 1998
[3] I Matthews T Ishikawa and S Baker ldquoThe template updateproblemrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 26 no 6 pp 810ndash815 2004
[4] D A Ross J Lim R-S Lin and M-H Yang ldquoIncrementallearning for robust visual trackingrdquo International Journal ofComputer Vision vol 77 no 1ndash3 pp 125ndash141 2008
[5] W Hu X Li X Zhang X Shi S Maybank and Z ZhangldquoIncremental tensor subspace learning and Its applications toforeground segmentation and trackingrdquo International Journal ofComputer Vision vol 91 no 3 pp 303ndash327 2011
[6] D Comaniciu V Ramesh and P Meer ldquoKernel-based objecttrackingrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 25 no 5 pp 564ndash577 2003
[7] A Adam E Rivlin and I Shimshoni ldquoRobust fragments-basedtracking using the integral histogramrdquo in Proceedings of theIEEE Computer Society Conference on Computer Vision andPattern Recognition (CVPR rsquo06) pp 798ndash805 June 2006
[8] X Mei and H Ling ldquoRobust visual tracking and vehicleclassification via sparse representationrdquo IEEE Transactions onPattern Analysis and Machine Intelligence vol 33 no 11 pp2259ndash2272 2011
[9] C Bao Y Wu H Ling and H Ji ldquoReal time robust L1 trackerusing accelerated proximal gradient approachrdquo in Proceedingsof the IEEE Conference on Computer Vision and Pattern Recog-nition (CVPR rsquo12) pp 1830ndash1837 June 2012
[10] T Zhang B Ghanem S Liu and N Ahuja ldquoRobust visualtracking via structured multi-task sparse learningrdquo Interna-tional Journal of Computer Vision vol 101 no 2 pp 367ndash3832013
[11] HGrabnerMGrabner andH Bischof ldquoReal-time tracking viaon-line boostingrdquo in Proceedings of the British Machine VisionConference (BMVC rsquo06) pp 47ndash56 September 2006
[12] H Grabner C Leistner and H Bischof ldquoSemi-supervised on-line boosting for robust trackingrdquo inProceedings of the EuropeanConference on Computer Vision pp 234ndash247 Marseille FranceOctober 2008
[13] S Avidan ldquoEnsemble trackingrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 29 no 2 pp 261ndash2712007
12 Mathematical Problems in Engineering
[14] B Babenko M-H Yang and S Belongie ldquoRobust object track-ing with online multiple instance learningrdquo IEEE Transactionson Pattern Analysis and Machine Intelligence vol 33 no 8 pp1619ndash1632 2011
[15] K Zhang and H Song ldquoReal-time visual tracking via onlineweighted multiple instance learningrdquo Pattern Recognition vol46 no 1 pp 397ndash411 2013
[16] S Avidan ldquoSupport vector trackingrdquo IEEE Transactions onPattern Analysis and Machine Intelligence vol 26 no 8 pp1064ndash1072 2004
[17] R T Collins Y Liu and M Leordeanu ldquoOnline selection ofdiscriminative tracking featuresrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 27 no 10 pp 1631ndash16432005
[18] J Kwon and K M Lee ldquoVisual tracking decompositionrdquoin Proceedings of the IEEE Computer Society Conference onComputer Vision and Pattern Recognition (CVPR 10) pp 1269ndash1276 San Francisco Calif USA June 2010
[19] Z Kalal K Mikolajczyk and J Matas ldquoTracking-learning-detectionrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 34 no 7 pp 1409ndash1422 2012
[20] K Zhang L Zhang and M H Yang ldquoReal-time compressivetrackingrdquo in Proceedings of the European Conference on Com-puter Vision pp 864ndash877 2012
[21] H Lu K N Plataniotis and A N Venetsanopoulos ldquoAsurvey of multilinear subspace learning for tensor datardquo PatternRecognition vol 44 no 7 pp 1540ndash1551 2011
[22] W Yang C Sun and L Zhang ldquoAmulti-manifold discriminantanalysis method for image feature extractionrdquo Pattern Recogni-tion vol 44 no 8 pp 1649ndash1657 2011
[23] J Sherrah B Ristic and N J Redding ldquoParticle filter to trackmultiple people for visual surveillancerdquo IET Computer Visionvol 5 no 4 pp 192ndash200 2011
[24] P Dollar CWojek B Schiele and P Perona ldquoPedestrian detec-tion an evaluation of the state of the artrdquo IEEE Transactions onPatternAnalysis andMachine Intelligence vol 34 no 4 pp 743ndash761 2012
where 119909e 119909119892 119910e 119910119892 are the 119909-axis and 119910-axis coordinates ofthe center of the experiment tracking bounding box and theground truth bounding box
The errors of four trackers in two scenarios are shown inTable 1 S2-O2-err represents the center location error of thesecond object in scenarios 2The data in bold refer to optimalresults
As can be seen from Table 1 the other three trackersrarely achieve a complete tracking so the tracking centerpoint errors is large The errors in the proposed method aresignificantly better than the other three trackers and theerrors are within the acceptable range
Our tracker is implemented in MATLAB 2012a and runsat 11 frames and 08 frames per second on an Inter Xeon24GHz CPU with 8GB RAM which is lacking in real-time
6 Conclusions
In this paper we proposed a visual object tracking algorithmvia feature tensor multimanifold discriminate analysis whichconsiders the tracking is vulnerable to the interference ofsimilar objects The object appearance model described byfeature tensor can maintain the object spatial structuralwhich helps to deal with the partial occlusion problem andhelps better to distinguish the object with similar ones inthe embedded low-dimensional subspace throughmultiman-ifold discriminate analysis In addition the update strategy isdesigned from the perspective of object appearance changewhich is used to determine if it is needed to update themultimanifold datasets As can be seen from the comparisonexperiments the proposed algorithm is able to adapt tothe object pose variation scale change and undisturbedtracking of similar objects in scenarios and also can achievecomplete tracking even if the object was completely occludedThe proposed algorithm exist some defects and when theobject is continuously occluded in the dense moving objectsscenarios the object appearance will be incomplete whichcannot construct an accurate multimanifold datasets thatcaused tracking failure
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
This work is supported by the National Natural ScienceFoundation of China (10771043) and the National Natural
Science Foundation of Inner-Mongolia Autonomous RegionChina under Grant (2012MS0931)
References
[1] H Yang L Shao F Zheng L Wang and Z Song ldquoRecentadvances and trends in visual tracking a reviewrdquoNeurocomput-ing vol 74 no 18 pp 3823ndash3831 2011
[2] M J Black and A D Jepson ldquoEigentracking robust matchingand tracking of articulated objects using a view-based represen-tationrdquo International Journal of Computer Vision vol 26 no 1pp 63ndash84 1998
[3] I Matthews T Ishikawa and S Baker ldquoThe template updateproblemrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 26 no 6 pp 810ndash815 2004
[4] D A Ross J Lim R-S Lin and M-H Yang ldquoIncrementallearning for robust visual trackingrdquo International Journal ofComputer Vision vol 77 no 1ndash3 pp 125ndash141 2008
[5] W Hu X Li X Zhang X Shi S Maybank and Z ZhangldquoIncremental tensor subspace learning and Its applications toforeground segmentation and trackingrdquo International Journal ofComputer Vision vol 91 no 3 pp 303ndash327 2011
[6] D Comaniciu V Ramesh and P Meer ldquoKernel-based objecttrackingrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 25 no 5 pp 564ndash577 2003
[7] A Adam E Rivlin and I Shimshoni ldquoRobust fragments-basedtracking using the integral histogramrdquo in Proceedings of theIEEE Computer Society Conference on Computer Vision andPattern Recognition (CVPR rsquo06) pp 798ndash805 June 2006
[8] X Mei and H Ling ldquoRobust visual tracking and vehicleclassification via sparse representationrdquo IEEE Transactions onPattern Analysis and Machine Intelligence vol 33 no 11 pp2259ndash2272 2011
[9] C Bao Y Wu H Ling and H Ji ldquoReal time robust L1 trackerusing accelerated proximal gradient approachrdquo in Proceedingsof the IEEE Conference on Computer Vision and Pattern Recog-nition (CVPR rsquo12) pp 1830ndash1837 June 2012
[10] T Zhang B Ghanem S Liu and N Ahuja ldquoRobust visualtracking via structured multi-task sparse learningrdquo Interna-tional Journal of Computer Vision vol 101 no 2 pp 367ndash3832013
[11] HGrabnerMGrabner andH Bischof ldquoReal-time tracking viaon-line boostingrdquo in Proceedings of the British Machine VisionConference (BMVC rsquo06) pp 47ndash56 September 2006
[12] H Grabner C Leistner and H Bischof ldquoSemi-supervised on-line boosting for robust trackingrdquo inProceedings of the EuropeanConference on Computer Vision pp 234ndash247 Marseille FranceOctober 2008
[13] S Avidan ldquoEnsemble trackingrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 29 no 2 pp 261ndash2712007
12 Mathematical Problems in Engineering
[14] B Babenko M-H Yang and S Belongie ldquoRobust object track-ing with online multiple instance learningrdquo IEEE Transactionson Pattern Analysis and Machine Intelligence vol 33 no 8 pp1619ndash1632 2011
[15] K Zhang and H Song ldquoReal-time visual tracking via onlineweighted multiple instance learningrdquo Pattern Recognition vol46 no 1 pp 397ndash411 2013
[16] S Avidan ldquoSupport vector trackingrdquo IEEE Transactions onPattern Analysis and Machine Intelligence vol 26 no 8 pp1064ndash1072 2004
[17] R T Collins Y Liu and M Leordeanu ldquoOnline selection ofdiscriminative tracking featuresrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 27 no 10 pp 1631ndash16432005
[18] J Kwon and K M Lee ldquoVisual tracking decompositionrdquoin Proceedings of the IEEE Computer Society Conference onComputer Vision and Pattern Recognition (CVPR 10) pp 1269ndash1276 San Francisco Calif USA June 2010
[19] Z Kalal K Mikolajczyk and J Matas ldquoTracking-learning-detectionrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 34 no 7 pp 1409ndash1422 2012
[20] K Zhang L Zhang and M H Yang ldquoReal-time compressivetrackingrdquo in Proceedings of the European Conference on Com-puter Vision pp 864ndash877 2012
[21] H Lu K N Plataniotis and A N Venetsanopoulos ldquoAsurvey of multilinear subspace learning for tensor datardquo PatternRecognition vol 44 no 7 pp 1540ndash1551 2011
[22] W Yang C Sun and L Zhang ldquoAmulti-manifold discriminantanalysis method for image feature extractionrdquo Pattern Recogni-tion vol 44 no 8 pp 1649ndash1657 2011
[23] J Sherrah B Ristic and N J Redding ldquoParticle filter to trackmultiple people for visual surveillancerdquo IET Computer Visionvol 5 no 4 pp 192ndash200 2011
[24] P Dollar CWojek B Schiele and P Perona ldquoPedestrian detec-tion an evaluation of the state of the artrdquo IEEE Transactions onPatternAnalysis andMachine Intelligence vol 34 no 4 pp 743ndash761 2012
[14] B Babenko M-H Yang and S Belongie ldquoRobust object track-ing with online multiple instance learningrdquo IEEE Transactionson Pattern Analysis and Machine Intelligence vol 33 no 8 pp1619ndash1632 2011
[15] K Zhang and H Song ldquoReal-time visual tracking via onlineweighted multiple instance learningrdquo Pattern Recognition vol46 no 1 pp 397ndash411 2013
[16] S Avidan ldquoSupport vector trackingrdquo IEEE Transactions onPattern Analysis and Machine Intelligence vol 26 no 8 pp1064ndash1072 2004
[17] R T Collins Y Liu and M Leordeanu ldquoOnline selection ofdiscriminative tracking featuresrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 27 no 10 pp 1631ndash16432005
[18] J Kwon and K M Lee ldquoVisual tracking decompositionrdquoin Proceedings of the IEEE Computer Society Conference onComputer Vision and Pattern Recognition (CVPR 10) pp 1269ndash1276 San Francisco Calif USA June 2010
[19] Z Kalal K Mikolajczyk and J Matas ldquoTracking-learning-detectionrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 34 no 7 pp 1409ndash1422 2012
[20] K Zhang L Zhang and M H Yang ldquoReal-time compressivetrackingrdquo in Proceedings of the European Conference on Com-puter Vision pp 864ndash877 2012
[21] H Lu K N Plataniotis and A N Venetsanopoulos ldquoAsurvey of multilinear subspace learning for tensor datardquo PatternRecognition vol 44 no 7 pp 1540ndash1551 2011
[22] W Yang C Sun and L Zhang ldquoAmulti-manifold discriminantanalysis method for image feature extractionrdquo Pattern Recogni-tion vol 44 no 8 pp 1649ndash1657 2011
[23] J Sherrah B Ristic and N J Redding ldquoParticle filter to trackmultiple people for visual surveillancerdquo IET Computer Visionvol 5 no 4 pp 192ndash200 2011
[24] P Dollar CWojek B Schiele and P Perona ldquoPedestrian detec-tion an evaluation of the state of the artrdquo IEEE Transactions onPatternAnalysis andMachine Intelligence vol 34 no 4 pp 743ndash761 2012
