+ All Categories
Home > Documents > Human Activity Recognition Using Gaussian Mixture Hidden...

Human Activity Recognition Using Gaussian Mixture Hidden...

Date post: 11-Aug-2020
Category:
Upload: others
View: 1 times
Download: 0 times
Share this document with a friend
15
Research Article Human Activity Recognition Using Gaussian Mixture Hidden Conditional Random Fields Muhammad Hameed Siddiqi , 1 Madallah Alruwaili , 1 Amjad Ali, 2 Saad Alanazi , 1 and Furkh Zeshan 2 1 College of Computer and Information Sciences, Jouf University, Sakaka, Saudi Arabia 2 Department of Computer Science, COMSATS University Islamabad, Lahore Campus, Lahore, Pakistan Correspondence should be addressed to Muhammad Hameed Siddiqi; [email protected] Received 12 May 2019; Accepted 10 July 2019; Published 18 August 2019 Academic Editor: Paolo Gastaldo Copyright © 2019 Muhammad Hameed Siddiqi et al. is is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. In healthcare, the analysis of patients’ activities is one of the important factors that offer adequate information to provide better services for managing their illnesses well. Most of the human activity recognition (HAR) systems are completely reliant on recognition module/stage. e inspiration behind the recognition stage is the lack of enhancement in the learning method. In this study, we have proposed the usage of the hidden conditional random fields (HCRFs) for the human activity recognition problem. Moreover, we contend that the existing HCRF model is inadequate by independence assumptions, which may reduce classification accuracy. erefore, we utilized a new algorithm to relax the assumption, allowing our model to use full-covariance distribution. Also, in this work, we proved that computation wise our method has very much lower complexity against the existing methods. For the experiments, we used four publicly available standard datasets to show the performance. We utilized a 10-fold cross- validation scheme to train, assess, and compare the proposed model with the conditional learning method, hidden Markov model (HMM), and existing HCRF model which can only use diagonal-covariance Gaussian distributions. From the experiments, it is obvious that the proposed model showed a substantial improvement with p value 0.2 regarding the classification accuracy. 1. Introduction In real-life environments, there are some fascinating ap- plications in which the analysis of human activities plays a significant role. Some applications include human/object detection and recognition based on vision object analysis and processing areas such as tracking and detection [1, 2], computer engineering [3], physical sciences [4], health-re- lated issues, natural sciences, and industrial academic areas [5]. Most of the authors [6–11] recognized the human ac- tivities in indoor environments based on different meth- odologies. However, in their respective systems, they used stable environment like fixed camera setting and prelighting setting, and most of the activities were performed by the instructions provided by the instructor. Similarly, the au- thors of [10, 12–14] proposed different methods to recognize the human daily activities in outdoor environments. However, in most of the used datasets, they used static background and this is one of the common drawbacks in their systems. Similarly, different sensors were utilized by the authors of [15–17] in order to classify indoor and outdoor human activities. Moreover, in telemedicine and healthcare, human ac- tivity recognition (HAR) can be explained by helping physically disabled persons’ scenario. A paralyzed patient with half of the body critically disturbed by stroke is completely unable to walk and the one way to recover him is through daily exercises. Normally, the daily exercises (ac- tivities) are recommended by the doctors to the stroke patients for getting better improvements in their health. A human activity recognition (HAR) system can correctly train and identify the activities performed by the stroke patients, through which the doctors easily can monitor the improvement scale in the patients’ health. ere are four modules in a typical HAR system: pre- processing (segmentation), feature extraction, feature Hindawi Computational Intelligence and Neuroscience Volume 2019, Article ID 8590560, 14 pages https://doi.org/10.1155/2019/8590560
Transcript
Page 1: Human Activity Recognition Using Gaussian Mixture Hidden …downloads.hindawi.com/journals/cin/2019/8590560.pdf · 2019-08-22 · ResearchArticle Human Activity Recognition Using

Research ArticleHuman Activity Recognition Using Gaussian Mixture HiddenConditional Random Fields

Muhammad Hameed Siddiqi 1 Madallah Alruwaili 1 Amjad Ali2 Saad Alanazi 1

and Furkh Zeshan2

1College of Computer and Information Sciences Jouf University Sakaka Saudi Arabia2Department of Computer Science COMSATS University Islamabad Lahore Campus Lahore Pakistan

Correspondence should be addressed to Muhammad Hameed Siddiqi siddiqi1984gmailcom

Received 12 May 2019 Accepted 10 July 2019 Published 18 August 2019

Academic Editor Paolo Gastaldo

Copyright copy 2019 Muhammad Hameed Siddiqi et al is is an open access article distributed under the Creative CommonsAttribution License which permits unrestricted use distribution and reproduction in anymedium provided the original work isproperly cited

In healthcare the analysis of patientsrsquo activities is one of the important factors that offer adequate information to provide betterservices for managing their illnesses well Most of the human activity recognition (HAR) systems are completely reliant onrecognition modulestage e inspiration behind the recognition stage is the lack of enhancement in the learning method In thisstudy we have proposed the usage of the hidden conditional random fields (HCRFs) for the human activity recognition problemMoreover we contend that the existing HCRFmodel is inadequate by independence assumptions whichmay reduce classificationaccuracy erefore we utilized a new algorithm to relax the assumption allowing our model to use full-covariance distributionAlso in this work we proved that computation wise our method has very much lower complexity against the existing methodsFor the experiments we used four publicly available standard datasets to show the performance We utilized a 10-fold cross-validation scheme to train assess and compare the proposed model with the conditional learning method hidden Markov model(HMM) and existing HCRF model which can only use diagonal-covariance Gaussian distributions From the experiments it isobvious that the proposed model showed a substantial improvement with p value le02 regarding the classification accuracy

1 Introduction

In real-life environments there are some fascinating ap-plications in which the analysis of human activities plays asignificant role Some applications include humanobjectdetection and recognition based on vision object analysisand processing areas such as tracking and detection [1 2]computer engineering [3] physical sciences [4] health-re-lated issues natural sciences and industrial academic areas[5] Most of the authors [6ndash11] recognized the human ac-tivities in indoor environments based on different meth-odologies However in their respective systems they usedstable environment like fixed camera setting and prelightingsetting and most of the activities were performed by theinstructions provided by the instructor Similarly the au-thors of [10 12ndash14] proposed different methods to recognizethe human daily activities in outdoor environmentsHowever in most of the used datasets they used static

background and this is one of the common drawbacks intheir systems Similarly different sensors were utilized by theauthors of [15ndash17] in order to classify indoor and outdoorhuman activities

Moreover in telemedicine and healthcare human ac-tivity recognition (HAR) can be explained by helpingphysically disabled personsrsquo scenario A paralyzed patientwith half of the body critically disturbed by stroke iscompletely unable to walk and the one way to recover him isthrough daily exercises Normally the daily exercises (ac-tivities) are recommended by the doctors to the strokepatients for getting better improvements in their health Ahuman activity recognition (HAR) system can correctlytrain and identify the activities performed by the strokepatients through which the doctors easily can monitor theimprovement scale in the patientsrsquo health

ere are four modules in a typical HAR system pre-processing (segmentation) feature extraction feature

HindawiComputational Intelligence and NeuroscienceVolume 2019 Article ID 8590560 14 pageshttpsdoiorg10115520198590560

selection and recognition as shown in Figure 1 Most of theexisting works [18ndash23] focused on feature extraction andselection however very limited works have been done forthe recognition module Some studies exploited conven-tional techniques [24ndash28] Among them HMM is one of thebest candidates for the activity recognition however HMMis generative in nature and less precise than its matching partlike HCRF model [29]

e inspiration behind the recognition stage is the lack ofenhancement in the learning method erefore we havemade the following contribution

(i) e existing HCRF model is inadequate by in-dependence assumptions which may reduce classi-cation accuracy erefore the rst objective of thisstudy is to propose a recognitionmodel that presents anew algorithm to relax the assumption allowing ourmodel in order to use full -covariance distribution

(ii) Another objective of the work is to prove thatcomputation wise our method has very much lowercomplexity against the existing methods In thismethod our goal was to nd some parameters tomaximize the conditional probability of the trainingdata at the training phase erefore in our workwe utilize limited-memory BroydenndashFletcherndashGoldfarbndashShanno (L-BFGS) method to search forthe optimal point However instead of repeating theforward and backward algorithms to compute thegradients as others did [30] we run the forward andbackward algorithms only when calculating theconditional probability and then we reuse the resultto compute the gradients As a result the compu-tation time is signicantly reduced

(iii) A comprehensive set of experiments which yielded aweighted average classication rate 97 that isbetter improvement in the performance against thestate-of-the-art methods

e rest of the paper is organized as follows Section 2presents related works with their limitations Section 3provides the proposed recognition model with its advan-tages Section 4 describes the experimental setup for the

proposed model against four datasets Based on the setup aseries of experiments are presented in Section 5 FinallySection 6 describes conclusion with some future directions

2 Related Works

In a typical HAR system dierent types of latest segmen-tation methods were used in preprocessing module in orderto extract the human body from the activity frame isprocess helps to improve the performance of the activityrecognition system erefore in the literature the authorsof [31ndash36] utilized the latest methods to segment the humanbody from the video frames Similarly for the feature ex-traction dierent latest methodologies have been employedwhich help the classiers to accurately classify the humanactivities (as the workow shown in Figure 1) [37ndash42] eyshowed better performance on dierent datasets and mostof them achieved average accuracy between 70 and 90

Regarding recognition the researchers have proposeddiverse systems which exploit various classiers such asGaussian mixture model (GMM) [43 44] articial neuralnetwork (ANN) [45 46] and support vector machine(SVMs) [47ndash50] ese classiers were principally employedfor frame-based classication Contrarily in many HARsystems [37 51 52] the eminent hidden Markov model(HMM) has extensively been utilized for sequence-basedclassication In the case of frame-level features HMMs arebeneted over vector-based classiers like SVM GMM andANN in terms of eectively handling the sequential dataHowever the Markovian property implied in the traditionalHMM assumes that the current state is a function of the paststate only is causes the labels of two adjacent states in theobservation sequence to hypothetically appear in successionBut in practical implementation this assumption often doesnot meet satisfaction Besides the generative characteristicof HMM and independence presumptions between obser-vations and states also limit its performance [29] To get ridof these limitations the maximum entropy Markov model(MEMM) had been proposed which comparatively performsbetter than HMM [53] However MEMM is associated withthe well-known disadvantage termed as ldquolabel bias problemrdquo

Video frames

Raw dataPreprocessing

Featureextraction

andselection

Segmentation(body shape sequence)

System testing

System testing

TrainedHCRFs

HCRFtraining

Recognition Activitylabels

Output

OutputSymbolsequences

System trainingSequences of symbolLabel

Figure 1 A typical human activity recognition (HAR) system

2 Computational Intelligence and Neuroscience

Two generalized models of MEMM known as condi-tional random fields (CRFs) [29] and HCRF [54] weredeveloped to fix the shortcoming of ldquolabel bias problemrdquo[29] For learning the hidden structure of the sequential dataHCRF facilitates the effectiveness of CRF with hidden statesHowever in both models the per-state normalization isreplaced with global normalization permitting the weightedscores which in turn result in larger parameter spaces ascompared to HMM and MEMM

For example the CRFs achieved in the HAR systemhaving the observed frames from a video are represented byfeature vectorU resultant labelV and unknown state label K

Suppose the problem image labeling is assumed byoriginal labels K with image features U and parameter of themodel is Λ then the later probability (post(K | UΛ))maximized by CRF is given as

post(K ∣ UΛ) ex(f(K UΛ))

z(UΛ) (1)

where the normalization factor is

z(UΛ) 1113944

Kprime

ex f Kprime UΛ( 1113857( 1113857 (2)

Some issues in HCRF implementation are reviewed andanalyzed in the following description e later probabilityof CRF in (1) has been updated by the post(K ∣ UΛ) in aHCRF model that is the addition of exponentials of latentfunctions with all expected labels L as given below

post(V ∣ UΛ) 1113936Lex Λ middot f(V L U)1113864 1113865

z(UΛ)

z(UΛ) 1113944

VprimeL

ex Λ middot f Vprime L U( 11138571113864 1113865

(3)

e above equations are used to warranty the sum toone rule of the conditional probability Vprime is the possibletag for the series of frames and L l1 l2 lT1113864 1113865 is a seriesof hidden states li i 1 2 T and equations (1) and (2)have constant values from 1 to Q (the number of states) Λis the vector factor and f(V L U) is a feature vector thatwill yield a decision which parameter will be educated bythe model en the feature vector concludes the additionof the existing HCRF model For example the underneathselections will create a Markov restraint HCRF with aGaussian distribution at every state

fPriorl (V L U) δ l1 l( 1113857 middot v uforalll

fTransitionllprime (V L U) 1113944

T

t1δ ltminus 1 l( 1113857δ lt lprime( 1113857 middot v uforalll lprime

fOccurencel (V L U) 1113944

T

t1δ lt l( 1113857 middot v uforalll

(4)where each v isin V is the expected tag and every u isin U is apredicted vector e per-component square of the obser-vation vector v at state t (ie vt) is given as

fM1l 1113944

T

t1δ lt l( 1113857 middot v utforalll

fM2l (V L U) 1113944

T

t1δ lt l( 1113857 middot v u

2tforalll

(5)

It can be seen that along with certain set of parameters(Λ) the HCRF addition is similar to the hidden Markovmodel for instance along with the abovementioned featurevector if we choose

Λl

Prior log(b(l)) (6)

Λl

Transition log C l lprime( 1113857( 1113857 (7)

Λl

Occurence minus

12

log 2πσ2l1113872 1113873 +μ2lσ2l

1113888 1113889 (8)

ΛM1l

μl

σ2l (9)

ΛM2l minus

12σ2l

(10)

where b in (6) is an earlier dissemination of Gaussian HMMand C in (7) is an evolution matrix then conditional pos-sibility numerator might be explained as

1113944

L

ex Λ middot f(V L U)1113864 1113865 1113944

L

b l1( 11138571113945

T

t1C ltminus 1 lt( 1113857N u

2t μLt

σLt1113872 1113873

(11)

In the above equationN represents Gaussian distributionEquation (11) is the conditional probability of U given V iscalculated along with a GaussianHMM through equation (11)which has an earlier distribution bwith a conversionmatrixC

Moreover the authors of [30] proposed a comprehensiveform of the HCRF model to tackle composite scatteringsutilizing a linear combination of Gaussian distribution func-tions which is explained as

post(V ∣ UΛ) 1113936L1113936

Mm1ex Λf(V L m U)1113864 1113865

z(UΛ) (12)

In equation (12)M indicates the number of componentsin Gaussian mixture

Lots of works have been developed which showed betterperformance based on the usage of the abovementionedHCRF [55 56] however most of them did not consider thelimitations of the model It is obvious from the aforemen-tioned equations that the existing model employed diago-nal (sloping)-covariance Gaussian distribution whichmeans that the variables (columns of ui i 1 2 N )were presumed to be couples independent On the otherhand equations (8)ndash(10) suggest that with a specific set ofvalue each state observation density will congregate toGaussian procedure Unluckily there is no training method

Computational Intelligence and Neuroscience 3

designed yet to guarantee this convergence and thosesuppositions might decrease the accuracy results

erefore we proposed the improved version of theHCRF technique that has the ability to openly employ full-covariance Gaussian mixture in the feature function eproposed model will get the benefits of hidden conditionalrandom field model that completely considered the draw-backs of the previous method

3 Proposed Methodology

31 Feature Extraction In our previous work we utilizedsymlet wavelet [37] for extracting various features from theactivity frames ere are number of reasons for using thesymlet wavelet which produces relatively better classificationresults ese include its capability to extract the conspic-uous information from the activity frames in terms of fre-quency and its support to the characteristics of the grayscaleimages like orthogonality biorthogonality and reversebiorthogonality For a certain provision size the symlet ischaracterized with the highest number of vanishing mo-ments and has the least asymmetry

32 Proposed Hidden Conditional Random Fields (HCRFs)Model As described earlier the current Gaussian mixtureHCRF model does not have the capability of utilizing full-covariance distributions and also does not guarantee theconjunction of its factors to certain values upon which theconditional probability is demonstrated as a combination ofthe normal density functions

To address these limitations we explicitly involve amixture of Gaussian distributions in the feature functions asillustrated in the following forms

fl

Prior(V L U) δ l1 l( 1113857 middot v uforalll (13)

fllprime

Transition(V L U) 1113944

T

t1δ ltminus 1 l( 1113857δ lt lprime( 1113857 middot v uforalll lprime

(14)

fl

Observation(V L U) 1113944

T

t1log 1113944

M

m1ΓObslm N u

2t μlmΣlm1113872 1113873⎛⎝ ⎞⎠

δ lt l( 1113857 middot v

(15)

then

N u2t μlmΣlm1113872 1113873

1

(2π)D2 Σlm1113868111386811138681113868

111386811138681113868111386812ex

middot minus12

u2t minus μlm1113872 1113873prime1113944

minus 1

lm

x2t minus μlm1113872 1113873⎛⎝ ⎞⎠

(16)

where N represents the number of density functionsGamma ldquoΓrdquo considers the appropriate information of theentire observations D indicates the dimension of the ob-servation and ΓObslm presents the partying weightiness forthe mth constituent along with mean μlm and covariancematrix Σlm

As indicated in equation (14) when we change some ofthe parameters such as Γ μ and Σ then we may build acombination of the standard densities e resultant con-ditional probability might be written as

post(V ∣ UΛ Γ μΣ) 1113936Lex(P(L) + T(L) + O(L))

z(UΛ Γ μΣ)

P(L) 1113944l

ΛPriorl fPriorl (V L U)

T(L) 1113944

llprime

ΛTransitionllprime fTransitionllprime (V L U)

O(L) 1113944l

fObservationl (V L U)

(17)

therefore

post(V ∣ UΛ Γ μΣ) 1113936Lexp 1113936lΛPrior

l fPriorl (V L U) + 1113936llprimeΛTransitionllprime fTransition

llprime (V L U) + 1113936lfObservationl (V L U)1113872 1113873

z(UΛ Γ μΣ) (18)

post(V ∣ UΛ Γ μΣ) 1113936Ll1 l2 lT

ex ΛPriorl1+ 1113936

Tt1 Λ

Transitionltminus 1 lt

1113872 1113873 + log 1113936Mm1Γ

Obsltm

N u2t μltm

Σltm1113872 11138731113872 11138731113872 1113873

z(UΛ Γ μΣ) (19)

post(V ∣ UΛ Γ μΣ) Score(U|VΛ Γ μΣ)

z(UΛ Γ μΣ) (20)

e forward and backward algorithms are used to cal-culate the conditional probability based on equations (19)and (20) that can be written as

4 Computational Intelligence and Neuroscience

ατ 1113944

Ll1 l2 lτl

exp ΛPriorl1+ 1113944

T

t1ΛTransitionltminus 1 lt

1113872 1113873 + log 1113944

M

m1ΓObsltm

N middot u2t μltm

Σltm1113872 1113873⎛⎝ ⎞⎠⎛⎝ ⎞⎠

1113944lprime

ατminus 1 lprime( 1113857exp ΛTransitionllprime + log 1113944M

m1ΓObsltm

N middot uτ μlmΣlm1113872 1113873⎛⎝ ⎞⎠

(21)

βτ(l) 1113944

L lτl lτ+1 lT

ex ΛPriorl1

+ 1113944T

t1ΛTransitionltminus 1lt

1113872 1113873 + log 1113944M

m1ΓObsltm

N middot u2t μltm

Σltm1113872 1113873⎛⎝ ⎞⎠

1113944lprime

βτ+1 lprime( 1113857exp ΛTransitionllprime + log 1113944M

m1ΓObslm N uτ μlmΣlm1113872 1113873⎛⎝ ⎞⎠⎛⎝ ⎞⎠

(22)

Score(V ∣ VΛ Γ μΣ) 1113944l

αT(l) 1113944l

β1(l) (23)

In the training data to maximize the conditionalprobability we initially focused on calculating the pa-rameters (Λ Γ μ andΣ) In the proposed approachlimited-memory BroydenndashFletcherndashGoldfarbndashShanno(L-BGFS) method has been implemented in order tosearch the optimum point Unlikely the other models[30] both the forward and backward algorithms are usedto compute the conditional probability and the resultswere reused for finding the gradients is makes thealgorithm more significant in reducing the computationtime

At the observation level we particularly incorporated thefull-covariance matrix in the feature function as shown in(16) Equation (17) may be used for getting the normaldistribution which is further elaborated in the followingequations

d Score(V ∣ UΛ Γ μΣ)dΛPriorl

1113944

L

dg(V L U)

dΛPriorl

ex(g(V L U))

1113944

L

fPriorl (V L U)ex(g(V L U)) β1(l)

(24)

e d Score function is a gradient function for a variableof the prior probability vectord Score(V ∣ UΛ Γ μΣ)

dΛTransitionl

1113944

L

dg(V L U)

dΛTransitionllprime

ex(g(V L U))

1113944

L

fTransitionllprime (V L U)exp(g(V L U))

(25)

e d Score function is a gradient function for a variableof the transition probability vector

d Score(V ∣ UΛ Γ μΣ)dΓObslm

1113944

L

dg(V L U)

dΓObslm

ex(g(V L U))

1113936LfObservation

l (V L U)

dΓObslm

ex(g(V L U))

1113944

L

1113944

T

t1

N ut μlmΣlm1113872 1113873

1113936Mm1Γ

Obslm N ut μlmΣlm1113872 1113873

δ lt l( 1113857ex(g(V L U))

1113944T

t1

N ut μlmΣlm1113872 1113873

1113936Mm1Γ

Obslm N ut μlmΣlm1113872 1113873

α(t l)c(t + 1)

(26)

Computational Intelligence and Neuroscience 5

e d Score function is a gradient function for aGaussian mixture weight variable Here a function V(t) canbe determined as

c(t) suml

β(t l) (27)

d Score(V ∣ UΛ Γ μΣ)dμlm

sumT

t1

ΓObslm dN ut μlmΣlm( )dμlmsumMm1ΓObslm N ut μlmΣlm( )

middot α(t l)c(t + 1)(28)

e d Score function is a gradient function for theGaussian distribution mean

d Score(V ∣ UΛ Γ μΣ)dΣlm

sumT

t1

ΓObslm dN ut μlmΣlm( )dΣlmsumMm1ΓObslm N ut μlmΣlm( )

middot α(t l)c(t + 1)(29)

e d Score function is a gradient function for the co-variance of the Gaussian distributions

Equations (24)ndash(27) presented above describe an anal-ysis method algorithm for calculating values of gradients fora feature function the mean of Gaussian distributions and

the covariance of the prior probability vector the transitionprobability vector and the observation probability vectorobtained from the existing HCRF

In our model the recognition of a variety of real-timeactivities can be divided into two steps a training step and aninference step In the rst step data with known labels areinputted for recognizing the target as well as training thehidden conditional eld model In the inference step theinputs to be actually estimated are ordered dependent onparameters determined in the training phase

If the activity frame is acting as an input in the training stepthen in the preprocessing step the applied distinctive lightingeects are decreased for detecting and extracting faces from theactivity frames At that point the movable features are extri-cated from the various facial parts for creating the featurevector After that the feature vector obtained serves as an inputto a full-covariance Gaussian-mixed hidden conditional ran-dom eld model of the suggested recognition model

As mentioned in the earlier discussion a feature gradient isgenerally determined by LBFG approach in the training phaseof the HCRF model Nonetheless in the current gradientcalculation technique a forward and backward iterative exe-cution algorithm is iteratively called upon which needs anexceptionally high computational time and thus leads to re-duction in the computational speed Another analysis ap-proach has been formulated that reduces the invoking of theforward and backward iterative execution algorithm using vegradient functions determined by equations (24)ndash(28) Using

Observed activity labels

Hidden CRFparameter

Modelweights

Hidden CRFtraining engine

Data calibrationand normalization

Unified interface

Raw data 1

2

5

7

6

3

4

4

ClusteringVectorquantization

Feature extraction

Discreteinput

sequence

Normalizeddata

Kernelvectors

Kernelvectors

Feature vectors

External interfaceInternal interface

Data files

Codebookvectors

Figure 2 Workow diagram of the proposed recognition model

6 Computational Intelligence and Neuroscience

this analysis the real-time computation can be carried out at ahigher speed resulting in an enormous decrease in the com-putational time compared to a known analysis approach eoverall workflow of the proposed model is shown in Figure 2

4 Model Validation

41 Datasets Used In this work we employed four open-source standard action datasets like Weizmann actiondatasets [57] KTH action dataset [58] UCF sports dataset[59] and IXMAS action dataset [60] for corroborating theproposed HCRF model performance All the datasets areexplained below

411 Weizmann Action Dataset is dataset consisted of10 actions such as bending running walking skippingplace jumping side movement jumping forward two handwaving and one hand waving that were performed by total9 subjects is dataset comprised of 90 video clips with

average of 15 frames per clip where the frame size is144 times180

412 KTH Action Dataset KTH dataset employed for ac-tivity recognition comprised of 25 subjects who performed 6activities like running walking boxing jogging hand-clapping and hand waving in four distinctive scenariosUsing a static camera in the homogenous background atotal of 2391 sequences were taken with a frame size of160times120

413 UCF Sports Dataset In this dataset there were 182videos which were evaluated by n-fold cross-validation ruleis dataset has been taken from different sports activities inbroadcast television channels Some of the videos had highintraclass similarities is dataset was also collected using astatic camera is dataset covers 9 activities like runningdiving lifting golf swinging skating kicking walking

Table 2 Confusion matrix of the proposed recognition model using KTH action dataset (unit )

Activities Walking Jogging Running Boxing Hand-wave HandclapWalking 100 0 0 0 0 0Jogging 0 98 1 1 0 0Running 2 1 95 1 1 0Boxing 0 2 1 97 0 0Hand-wave 1 0 1 0 98 0Handclap 0 0 1 0 0 99Average 9783

Table 1 Confusion matrix of the proposed recognition model using Weizmann action dataset (unit )

Activities Bend Jack Pjump Run Side Skip Walk Wave 1 Wave 2Bend 98 0 1 0 0 1 0 0 0Jack 1 96 0 0 2 0 1 0 0Pjump 0 1 97 0 1 0 0 1 0Run 0 0 0 99 0 0 0 1 0Side 1 2 0 0 95 0 0 2 0Skip 0 0 0 0 0 100 0 0 0Walk 1 0 0 1 0 1 96 0 1Wave 1 0 1 1 0 0 0 0 98 0Wave 2 0 1 0 0 1 1 1 1 95Average 9711

Table 3 Confusion matrix of the proposed recognition model using UCF sports dataset (unit )

Activities Diving GS Kicking Lifting HBR Run Skating BS WalkDiving 95 1 2 1 0 1 0 0 0GS 1 94 0 0 2 1 1 1 0Kicking 0 2 98 0 0 0 0 0 0Lifting 1 1 1 94 1 0 0 1 1HBR 0 0 2 0 96 1 0 1 0Running 0 3 0 0 0 97 0 0 0Skating 1 0 1 1 0 1 95 0 1BS 0 0 0 1 0 1 0 97 1Walking 0 0 0 0 0 0 0 0 100Average 9622GS golf swinging HBR horseback riding and BS baseball swinging

Computational Intelligence and Neuroscience 7

Table 4 Confusion matrix of the proposed recognition model using IXMAS action dataset (unit )

Activities CA SD GU TA Walk Wave Punch KickCA 97 0 0 1 2 0 0 0SD 0 99 1 0 0 0 0 0GU 1 2 94 3 0 0 0 0TA 0 0 1 95 2 1 1 0Walk 0 1 1 0 98 0 0 0Wave 0 0 2 0 1 97 0 0Punch 0 1 0 1 0 2 96 0Kick 0 0 0 0 1 0 0 99Average 9688CA cross arm SD sit down GU get up and TA turn around

Table 5 Classification results of the proposed system on Weizmann action dataset (A) using ANN (B) using SVM (C) using HMM and(D) using existing HCRF [30] while removing the proposed HCRF model (unit )

Activities Bend Jack Pjump Run Side Skip Walk Wave 1 Wave 2(A)Bend 70 4 5 3 3 2 5 5 3Jack 4 68 3 6 7 2 3 3 4Pjump 2 4 75 6 3 2 2 2 4Run 4 2 3 72 6 2 5 3 3Side 5 3 5 4 65 6 4 6 2Skip 4 6 4 3 5 67 3 2 6Walk 4 2 4 7 3 4 70 3 3Wave 1 2 1 3 3 4 5 7 71 4Wave 2 2 5 3 6 4 4 3 5 68Average 6955(B)Bend 69 3 4 4 6 4 2 3 5Jack 2 72 2 3 4 3 5 4 5Pjump 1 4 75 2 4 5 4 2 3Run 2 4 3 78 2 2 4 2 3Side 2 4 5 3 70 4 3 5 4Skip 2 1 3 2 4 80 3 3 2Walk 2 0 3 4 3 2 82 1 3Wave 1 2 2 3 4 3 2 3 77 4Wave 2 1 2 1 2 3 1 3 4 83Average 7622(C)Bend 82 3 0 2 2 3 1 5 2Jack 3 80 1 2 3 2 3 4 2Pjump 3 4 85 3 0 0 1 2 2Run 5 4 2 79 0 2 1 3 4Side 0 1 5 4 81 3 1 2 3Skip 3 1 2 2 3 88 0 0 1Walk 0 2 3 2 1 2 83 3 4Wave 1 1 3 2 2 4 2 3 78 5Wave 2 1 2 2 2 2 3 1 0 87Average 8256(D)Bend 80 2 3 1 4 0 5 2 3Jack 1 88 0 2 0 3 2 3 1Pjump 0 2 90 1 0 3 0 2 2Run 2 1 2 85 2 3 0 0 5Side 4 1 2 3 80 4 1 2 3Skip 1 4 0 5 1 84 0 3 2Walk 2 1 0 0 1 2 89 2 3Wave 1 3 0 1 2 0 2 0 91 1Wave 2 4 1 3 0 2 3 0 2 85Average 8578

8 Computational Intelligence and Neuroscience

horseback riding and baseball swinging Each frame has asize of 720times 480

414 IXMAS Action Dataset IXMAS (INRIA Xmas motionacquisition sequences) dataset comprised of 13 activity classeswhich were performed by 11 actors each 3 times Every actoropted a free orientation as well as position e dataset hasprovided annotated silhouettes for each person For our ex-periments we have selected only 8 action classes like walkcross arms punch turn around sit down wave get up andkick IXMAS dataset is a multiview dataset for a view-invarianthuman activity recognition where each frame has a size of390times 291 is dataset has a major occlusion and that maycause misclassification therefore we utilized global histogramequalization [61] in order to resolve the occlusion issue

42 Setup For a comprehensive validation we carried outthe following set of experiments executed using Matlab

(i) e first experiment was conducted on each datasetseparately in order to show the performance of theproposed model In this experiment we employed10-fold cross-validation rule which means that data

from 9 subjects were utilized for training data whilethe data from one subject was picked as a testingdata e procedure was reiterated for 10 timesprovided each subject data is utilized for bothtraining and testing

(ii) e second experiment was conducted in the ab-sence of the proposed recognition model on all thefour datasets that will show the importance of thedeveloped model For this purpose we used theexisting eminent classifiers like SVM ANN HMMand existing HCRF [30] as a recognition modelrather than utilizing the proposed HCRF model

(iii) e third experiment was conducted to show theperformance of the proposed approach against thestate-of-the-art methods

(iv) In the last experiment the computational com-plexity of the proposed HCRF model was comparedwith forwardbackward algorithms

5 Results and Discussion

51 First Experiment As described before this experimentvalidates the performance of the proposed recognition model

Table 6 Classification results of the proposed system on KTH action dataset (A) using ANN (B) using SVM (C) using HMM and (D)using existing HCRF [30] while removing the proposed HCRF model (unit )

Activities Walking Jogging Running Boxing Hand-wave Handclap(A)Walking 79 5 6 4 3 3Jogging 3 81 5 3 4 4Running 6 4 77 5 5 3Boxing 6 7 6 69 5 7Hand-wave 4 7 5 5 73 6Handclap 4 6 5 4 6 75Average 7566(B)Walking 82 2 3 5 4 4Jogging 3 86 2 3 2 4Running 5 3 80 4 5 3Boxing 5 3 3 79 4 6Hand-wave 1 4 3 3 89 0Handclap 3 5 2 4 3 83Average 8317(C)Walking 86 3 2 4 2 3Jogging 0 88 3 2 4 3Running 0 3 90 0 4 3Boxing 3 0 4 92 1 0Hand-wave 1 3 2 2 91 1Handclap 1 3 4 1 2 89Average 8933(D)Walking 90 3 0 3 4 0Jogging 2 88 2 3 3 2Running 4 2 92 0 0 2Boxing 1 3 2 91 3 0Hand-wave 0 1 3 2 93 1Handclap 1 3 2 4 3 87Average 9017

Computational Intelligence and Neuroscience 9

on an individual dataset e overall results are shown inTables 1 (using Weizmann dataset) 2 (using KTH dataset) 3(usingUCF sports dataset) and 4 (using IXMAS) respectively

As observed from Tables 1ndash4 the proposed recognitionmodel constantly obtained higher recognition rates on in-dividual dataset is result shows the robustness of theproposed model which means that the model not onlyshowed better performance on one dataset but also showedbetter performance across multiple spontaneous datasets

52 Second Experiment As described before the secondexperiment was conducted in the absence of the proposed

recognition model to show the importance of the proposedmodel using all the four datasets For this purpose we usedthe existing eminent classifiers like SVM ANN HMM andexisting HCRF [30] as a recognition model rather thanutilizing the proposed HCRF model

Tables 5ndash8 show that when the proposed HCRF modelwas substituted with ANN SVM HMM and existing HCRF[30] the system failed to accomplish higher recognitionrates e better performance of the proposed HCRF modelis visualized in Tables 1ndash4 which show that the proposedHCRF model effectively fix the drawbacks of HMM andexisting HCRF that has been extensively utilized for se-quential HAR

Table 7 Classification results of the proposed system onUCF sports dataset (A) using ANN (B) using SVM (C) using HMM and (D) usingexisting HCRF [30] while removing the proposed HCRF model (unit )

Activities Diving GS Kicking Lifting HBR Run Skating BS Walk(A)Diving 68 4 2 5 6 6 4 3 2GS 2 71 2 4 5 4 6 3 3Kicking 3 4 70 3 5 4 2 3 6Lifting 5 4 3 65 5 6 4 6 2HBR 3 4 6 3 66 4 5 5 4Running 3 3 5 4 6 64 6 4 5Skating 2 5 4 5 3 4 69 3 5BS 4 2 5 3 4 6 5 67 4Walking 5 4 2 3 4 3 6 3 70Average 6778(B)Diving 71 4 2 3 5 6 3 2 4GS 3 77 2 4 3 2 5 2 2Kicking 4 2 74 4 5 3 2 3 3Lifting 5 6 3 69 4 3 5 3 2HBR 2 3 3 2 80 2 4 2 2Running 2 3 2 2 5 75 6 2 3Skating 2 1 2 3 4 4 78 2 4BS 3 4 6 3 4 2 3 70 5Walking 4 1 2 4 2 3 0 3 81Average 7500(C)Diving 79 3 2 2 3 4 3 2 2GS 0 83 2 4 3 2 1 3 2Kicking 1 2 85 1 3 3 2 3 0Lifting 3 0 2 82 3 2 4 2 2HBR 0 2 2 4 80 0 5 3 4Running 1 2 1 3 4 84 2 1 2Skating 2 0 3 4 0 1 86 3 1BS 1 1 1 2 0 3 0 88 4Walking 1 2 4 2 5 2 4 3 77Average 8267(D)Diving 90 3 0 1 0 2 2 1 1GS 3 84 2 1 3 1 3 2 1Kicking 3 4 85 0 0 2 3 1 2Lifting 1 2 1 89 1 1 1 2 2HBR 0 2 1 0 91 2 3 0 1Running 2 3 1 2 3 80 4 2 3Skating 2 4 1 2 3 0 84 4 0BS 2 1 1 2 1 0 3 88 2Walking 0 2 1 1 0 1 4 0 91Average 8689GS golf swinging HBR horseback riding BS baseball swinging

10 Computational Intelligence and Neuroscience

53 ird Experiment In this experiment a comparativeanalysis was made between the state-of-the-art methods andthe proposed model All of these approaches were imple-mented by the instructions provided in their particular ar-ticles A 10-fold cross-validation rule was employed on eachdataset as explained in Section 4 e average classicationresults of the existing methods along with the proposedmethod across dierent datasets are summarized in Table 9

Table 8 Classication results of the proposed system on IXMASaction dataset (A) using ANN (B) using SVM (C) using HMMand (D) using existing HCRF [30] while removing the proposedHCRF model (unit )

Activities CA SD GU TA Walk Wave Punch Kick(A)CA 65 5 7 6 5 4 3 5SD 5 72 4 3 5 4 3 4GU 4 3 75 5 3 2 4 4TA 6 7 4 68 3 5 4 3Walk 3 4 5 4 70 6 4 4Wave 4 6 5 3 4 71 3 4Punch 3 5 5 6 7 3 67 4Kick 4 5 4 6 5 4 3 69Average 6962(B)CA 77 3 4 2 2 3 5 4SD 3 79 3 2 4 5 2 2GU 5 6 69 3 4 5 4 4TA 2 3 2 80 4 4 3 2Walk 3 5 4 2 71 5 6 4Wave 2 6 3 5 4 73 4 3Punch 1 5 3 4 1 3 81 2Kick 3 6 7 4 3 5 4 68Average 7475(C)CA 79 3 4 1 2 4 3 4SD 1 84 3 2 3 1 4 2GU 0 1 88 1 2 3 2 3TA 5 2 3 79 2 3 2 4Walk 1 0 3 1 90 2 3 0Wave 2 3 1 0 3 86 2 3Punch 1 0 2 3 0 4 89 1Kick 3 2 4 1 0 2 4 84Average 8477(D)CA 90 1 2 0 3 4 0 0SD 3 85 2 1 3 3 2 1GU 0 1 91 1 0 2 3 2TA 1 3 2 87 1 2 2 2Walk 1 0 3 1 89 2 1 3Wave 0 2 1 0 2 90 3 1Punch 1 4 2 1 2 3 84 3Kick 1 2 4 1 3 2 4 83Average 8738CA cross arm SD sit down GU get up TA turn around

Table 9 Weighted average recognition rates of the proposedmethod with the existing state-of-the-art methods (unit )

State-of-the-art works Average classicationrates

Standarddeviation

GMM 633 plusmn27SVM 675 plusmn44HMM 828 plusmn38Embedded HMM 859 plusmn19[62] 921 plusmn32[63] 843 plusmn49[18] 936 plusmn27[19] 930 plusmn16[22] 927 plusmn25[64] 801 plusmn32Proposed method 972 plusmn28

0

100

200

300

400

500

600

Exec

utio

n tim

e (s)

1 2 3 4 5 6 7 8Number of states

ForwardbackwardProposed HCRF model

(a)Ex

ecut

ion

time (

s)

1 2 3 4 5 6 7 8

ForwardbackwardProposed HCRF model

0

2

4

6

8

10

12

14

16

Number of mixtures

(b)

Figure 3 An illustration of gradient computational time (equation(30)) of the previous forward and backward algorithms and theproposed HCRF model (a)Q 1 minus 5 M 5 T 90 and (b)Q 5 M 1 minus 5 T 90

Computational Intelligence and Neuroscience 11

It is obvious from Table 9 that the proposed methodshowed a significant performance against the existing state-of-the-art methods erefore the proposed method accu-rately and robustly recognizes the human activities usingdifferent video data

54 Fourth Experiment In this experiment we have pre-sented the computational complexity that is also one of thecontributions in this paper e implementations of theprevious HCRF are available in literature which calculatethe gradients by reiterating the forward and backwardtechniques while the proposed HCRF model executes themonce only and cashes the outcomes for the later use From(21) and (22) it is clear that the forward or backwardtechnique has a complexity ofO(TQ2M) where Trepresentsthe input sequence lengthQ represents the number of statesand M indicates the number of mixtures e proposedHCRFmodel however requires a full complexity of O(TM)

to calculate gradients as can be seen from (22)ndash(29)Figure 3 shows a comparison of the execution time when

the gradients are computed by the forward (or backward)algorithm and by our proposed method e computationaltime is calculated by running Matlab R2013a with thespecification of Intelreg Pentiumreg Coretrade i7-6700 (34GHz)with a RAM capacity of 16GB

6 Conclusion

In healthcare and telemedicine the human activity rec-ognition (HAR) can be best explained by helping physi-cally disable personsrsquo scenario A paralyzed patient withhalf of the body critically attacked by paralysis is com-pletely unable to perform their daily exercises e doctorsrecommend specific activities to get better improvement intheir health So for this purpose the doctors need a humanactivity recognition (HAR) system through which they canmonitor the patientsrsquo daily routines (activities) on a reg-ular basis

e accuracy of most of the HAR systems depends uponthe recognition modules For feature extraction and selec-tion modules we used some of the existing well-knownmethods while for the recognition module we proposed theusage of HCRF model which is capable of approximating acomplex distribution using a mixture of Gaussian densityfunctions e proposed model was assessed against fourpublicly available standard action datasets From our ex-periments it is obvious that the proposed full-covarianceGaussian density function showed a significant improve-ment in accuracy than the existing state-of-the-art methodsFurthermore we also proved that such improvement issignificant from statistical point of view by showing valuele02 of the comparison Similarly the complexity analysispoints out that the proposed computational method stronglydecreases the execution time for the hidden conditionalrandom field model

e ultimate goal of this study is to deploy the proposedmodel on smartphones Currently the proposed model isusing full-covariance matrix however this might be time

consuming especially when using on smartphones Using alightweight classifier such as K-nearest neighbor (K-NN)could be one possible solution But K-NN is very muchsensitive to environmental factor (like noise) erefore infuture we will try to investigate further research to reducethe time and sustain the same recognition rate whenemploying on smartphones in real environment

Data Availability

e data used to support the findings of this study areavailable from the corresponding author upon request

Conflicts of Interest

e authors declare there are no conflicts of interest

Acknowledgments

is study was supported by the Jouf University SakakaAljouf Kingdom of Saudi Arabia under the registration no39791

References

[1] Z Chang J Cao and Y Zhang ldquoA novel image segmentationapproach for wood plate surface defect classification throughconvex optimizationrdquo Journal of Forestry Research vol 29no 6 pp 1789ndash1795 2018

[2] Z S Abdallah M M Gaber B Srinivasan andS Krishnaswamy ldquoAdaptive mobile activity recognitionsystemwith evolving data streamsrdquoNeurocomputing vol 150pp 304ndash317 2015

[3] M S Bakli M A Sakr and T H A Soliman ldquoA spatio-temporal algebra in Hadoop for moving objectsrdquo Geo-SpatialInformation Science vol 21 no 2 pp 102ndash114 2018

[4] W Zhao L Yan and Y Zhang ldquoGeometric-constrainedmulti-view image matching method based on semi-globaloptimizationrdquo Geo-Spatial Information Science vol 21 no 2pp 115ndash126 2018

[5] H Gao X Zhang J Zhao and D Li ldquoTechnology of in-telligent driving radar perception based on driving brainrdquoCAAI Transactions on Intelligence Technology vol 2 no 3pp 93ndash100 2017

[6] Y Wang X Jiang R Cao and X Wang ldquoRobust indoorhuman activity recognition using wireless signalsrdquo Sensorsvol 15 no 7 pp 17195ndash17208 2015

[7] J Wang Z Liu Y Wu and J Yuan ldquoMining actionlet ensemblefor action recognition with depth camerasrdquo in Proceedings of theIEEE Conference on Computer Vision and Pattern Recognitionpp 1290ndash1297 IEEE Providence RI USA June 2012

[8] S Karungaru M Daikoku and K Terada ldquoMulti camerasbased indoors human action recognition using fuzzy rulesrdquoJournal of Pattern Recognition Research vol 10 no 1pp 61ndash74 2015

[9] S Shukri L M Kamarudin and M H F Rahiman ldquoDevice-free localization for human activity monitoringrdquo in IntelligentVideo Surveillance IntechOpen London UK 2018

[10] L Fiore D Fehr R Bodor A Drenner G Somasundaramand N Papanikolopoulos ldquoMulti-camera human activitymonitoringrdquo Journal of Intelligent and Robotic Systemsvol 52 no 1 pp 5ndash43 2008

12 Computational Intelligence and Neuroscience

[11] C Zhang and Y Tian ldquoRGB-D camera-based daily livingactivity recognitionrdquo Journal of Computer Vision and ImageProcessing vol 2 no 4 pp 1ndash7 2012

[12] M Leo T DrsquoOrazio and P Spagnolo ldquoHuman activityrecognition for automatic visual surveillance of wide areasrdquo inProceedings of the ACM 2nd International Workshop on VideoSurveillance and Sensor Networks pp 124ndash130 ACM NewYork NY USA October 2004

[13] C-B Jin S Li and H Kim ldquoReal-time action detection invideo surveillance using sub-action descriptor with multi-CNNrdquo 2017 httpsarxivorgabs171003383

[14] W Niu J Long D Han and Y-F Wang ldquoHuman activitydetection and recognition for video surveillancerdquo in Pro-ceedings of the 2004 IEEE International Conference on Mul-timedia and Expo (ICME) (IEEE Cat No 04TH8763) vol 1pp 719ndash722 IEEE Taipei Taiwan June 2004

[15] Q Li M Zheng F Li J Wang Y-A Geng and H JiangldquoRetinal image segmentation using double-scale non-linearthresholding on vessel support regionsrdquo CAAI Transactionson Intelligence Technology vol 2 no 3 pp 109ndash115 2017

[16] L M G Fonseca L M Namikawa and E F CastejonldquoDigital image processing in remote sensingrdquo in Proceedingsof the 2009 Tutorials of the XXII Brazilian Symposium onComputer Graphics and Image Processing pp 59ndash71 IEEERio de Janeiro Brazil October 2009

[17] K Buys C Cagniart A Baksheev T De Laet J De Schutterand C Pantofaru ldquoAn adaptable system for RGB-D basedhuman body detection and pose estimationrdquo Journal of VisualCommunication and Image Representation vol 25 no 1pp 39ndash52 2014

[18] M Sharif M A Khan T Akram M Y Javed T Saba andA Rehman ldquoA framework of human detection and actionrecognition based on uniform segmentation and combinationof Euclidean distance and joint entropy-based features se-lectionrdquo EURASIP Journal on Image and Video Processingvol 2017 no 1 p 89 2017

[19] K-P Chou M Prasad D Wu et al ldquoRobust feature-basedautomated multi-view human action recognition systemrdquoIEEE Access vol 6 pp 15283ndash15296 2018

[20] M Ullah H Ullah and I M Alseadonn ldquoHuman actionrecognition in videos using stable featuresrdquo Signal and ImageProcessing An International Journal vol 8 no 6 pp 1ndash102017

[21] M Sharif M A Khan F Zahid J H Shah and T AkramldquoHuman action recognition a framework of statisticalweighted segmentation and rank correlation-based selectionrdquoPattern Analysis and Applications pp 1ndash14 2019

[22] J Zang L Wang Z Liu Q Zhang G Hua and N ZhengldquoAttention-based temporal weighted convolutional neuralnetwork for action recognitionrdquo in Proceedings of the IFIPInternational Conference on Artificial Intelligence Applicationsand Innovations pp 97ndash108 Springer Rhodes Greece May2018

[23] V-D Hoang ldquoMultiple classifier-based spatiotemporal fea-tures for living activity predictionrdquo Journal of Informationand Telecommunication vol 1 no 1 pp 100ndash112 2017

[24] S Dharmalingam and A Palanisamy ldquoVector space basedaugmented structural kinematic feature descriptor for humanactivity recognition in videosrdquo ETRI Journal vol 40 no 4pp 499ndash510 2018

[25] D Liu Y Yan M-L Shyu G Zhao and M Chen ldquoSpatio-temporal analysis for human action detection and recognitionin uncontrolled environmentsrdquo International Journal of

Multimedia Data Engineering and Management vol 6 no 1pp 1ndash18 2015

[26] C A Ronao and S-B Cho ldquoHuman activity recognition withsmartphone sensors using deep learning neural networksrdquoExpert Systems with Applications vol 59 pp 235ndash244 2016

[27] A arwat H Mahdi M Elhoseny and A E HassanienldquoRecognizing human activity in mobile crowdsensing envi-ronment using optimized k-NN algorithmrdquo Expert Systemswith Applications vol 107 pp 32ndash44 2018

[28] A Jalal Y-H Kim Y-J Kim S Kamal and D Kim ldquoRobusthuman activity recognition from depth video using spatio-temporal multi-fused featuresrdquo Pattern Recognition vol 61pp 295ndash308 2017

[29] J Lafferty A McCallum and F C Pereira ldquoConditionalrandom fields probabilistic models for segmenting and la-beling sequence datardquo in Proceedings of the 18th InternationalConference on Machine Learning (ICML-2001) pp 282ndash289Williamstown MA USA June-July 2001

[30] A Gunawardana M Mahajan A Acero and J C PlattldquoHidden conditional random fields for phone classificationrdquoin Proceedings of the INTERSPEECH vol 2 pp 1117ndash1120Citeseer Lisbon Portugal September 2005

[31] L Yang Q Song Z Wang and M Jiang ldquoParsing r-cnn forinstance-level human analysisrdquo in Proceedings of the IEEEConference on Computer Vision and Pattern Recognitionpp 364ndash373 Long Beach CA USA June 2019

[32] X Liang Y Wei L Lin et al ldquoLearning to segment human bywatching youtuberdquo IEEE Transactions on Pattern Analysisand Machine Intelligence vol 39 no 7 pp 1462ndash1468 2016

[33] M M Requena ldquoHuman segmentation with convolutionalneural networksrdquo esis University of Alicante AlicanteSpain 2018

[34] H Gao S Chen and Z Zhang ldquoParts semantic segmentationaware representation learning for person re-identificationrdquoApplied Sciences vol 9 no 6 p 1239 2019

[35] C Peng S-L Lo J Huang and A C Tsoi ldquoHuman actionsegmentation based on a streaming uniform entropy slicemethodrdquo IEEE Access vol 6 pp 16958ndash16971 2018

[36] M H Siddiqi A M Khan and S-W Lee ldquoActive contourslevel set based still human body segmentation from depthimages for video-based activity recognitionrdquo KSII Trans-actions on Internet and Information Systems (TIIS) vol 7no 11 pp 2839ndash2852 2013

[37] M Siddiqi R Ali M Rana E-K Hong E Kim and S LeeldquoVideo-based human activity recognition using multilevelwavelet decomposition and stepwise linear discriminantanalysisrdquo Sensors vol 14 no 4 pp 6370ndash6392 2014

[38] S Althloothi M H Mahoor X Zhang and R M VoylesldquoHuman activity recognition using multi-features and mul-tiple kernel learningrdquo Pattern Recognition vol 47 no 5pp 1800ndash1812 2014

[39] A Jalal S Kamal and D Kim ldquoA depth video-based humandetection and activity recognition using multi-features andembedded hidden markov models for health care monitoringsystemsrdquo International Journal of Interactive Multimedia andArtificial Intelligence vol 4 no 4 p 54 2017

[40] T Dobhal V Shitole G omas and G Navada ldquoHumanactivity recognition using binary motion image and deeplearningrdquo Procedia Computer Science vol 58 pp 178ndash1852015

[41] S Zhang Z Wei J Nie L Huang S Wang and Z Li ldquoAreview on human activity recognition using vision-basedmethodrdquo Journal of Healthcare Engineering vol 2017 ArticleID 3090343 31 pages 2017

Computational Intelligence and Neuroscience 13

[42] Y Yang B Zhang L Yang C Chen and W Yang ldquoActionrecognition using completed local binary patterns and mul-tiple-class boosting classifierrdquo in Proceedings of the 2015 3rdIAPR Asian Conference on Pattern Recognition (ACPR) IEEEpp 336ndash340 Kuala Lumpur Malaysia November 2015

[43] W Lin M T Sun R Poovandran and Z Zhang ldquoHumanactivity recognition for video surveillancerdquo in Proceedings ofthe 2008 IEEE International Symposium on Circuits andSystems pp 2737ndash2740 IEEE Seattle WA USA May 2008

[44] H Permuter J Francos and I Jermyn ldquoA study of Gaussianmixture models of color and texture features for imageclassification and segmentationrdquo Pattern Recognition vol 39no 4 pp 695ndash706 2006

[45] M Fiaz and B Ijaz ldquoVision based human activity trackingusing artificial neural networksrdquo in Proceedings of the 2010International Conference on Intelligent and Advanced Systemspp 1ndash5 IEEE Kuala Lumpur Malaysia June 2010

[46] H Foroughi A Naseri A Saberi and H S Yazdi ldquoAneigenspace-based approach for human fall detection usingintegrated time motion image and neural networkrdquo in Pro-ceedings of the 2008 9th International Conference on SignalProcessing pp 1499ndash1503 IEEE Beijing China October2008

[47] A K B Sonali ldquoHuman action recognition using supportvector machine and k-nearest neighborrdquo InternationalJournal of Engineering and Technical Research vol 3 no 4pp 423ndash428 2015

[48] V Swarnambigai ldquoAction recognition using ami and supportvector machinerdquo International Journal of Computer Scienceand Technology vol 5 no 4 pp 175ndash179 2014

[49] D Das and S Saharia ldquoHuman gait analysis and recognitionusing support vector machinesrdquo International Journal ofComputer Science and Information Technology vol 6 no 52004

[50] K G M Chathuramali and R Rodrigo ldquoFaster human activityrecognition with SVMrdquo in Proceedings of the InternationalConference on Advances in ICT for Emerging Regions(ICTer2012) pp 197ndash203 IEEE Colombo Sri Lanka De-cember 2012

[51] M Z Uddin D-H Kim J T Kim and T-S Kim ldquoAn indoorhuman activity recognition system for smart home using localbinary pattern features with hidden markov modelsrdquo Indoorand Built Environment vol 22 no 1 pp 289ndash298 2013

[52] M Z Uddin J Lee and T-S Kim ldquoIndependent shapecomponent-based human activity recognition via hiddenMarkov modelrdquo Applied Intelligence vol 33 no 2 pp 193ndash206 2010

[53] S Kumar and M Hebert ldquoDiscriminative Fields for ModelingSpatial Dependencies in Natural Imagesrdquo in Proceedings of theNIPS Vancouver Canada December 2003

[54] A Quattoni S Wang L-P Morency M Collins andT Darrell ldquoHidden conditional random fieldsrdquo IEEETransactions on Pattern Analysis and Machine Intelligencevol 29 no 10 pp 1848ndash1852 2007

[55] M Mahajan A Gunawardana and A Acero ldquoTraining al-gorithms for hidden conditional random fieldsrdquo in Pro-ceedings of the 2006 IEEE International Conference onAcoustics Speed and Signal Processing vol 1 IEEE ToulouseFrance May 2006

[56] S Reiter B Schuller and G Rigoll ldquoHidden conditionalrandom fields for meeting segmentationrdquo in Proceedings ofthe Multimedia and Expo 2007 IEEE International Confer-ence pp 639ndash642 IEEE Beijing China July 2007

[57] L Gorelick M Blank E Shechtman M Irani and R BasrildquoActions as space-time shapesrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 29 no 12 pp 2247ndash2253 2007

[58] I Laptev M Marszalek C Schmid and B RozenfeldldquoLearning realistic human actions from moviesrdquo in Pro-ceedings of the 2008 IEEE Conference on Computer Vision andPattern Recognition pp 1ndash8 IEEE Anchorage AK USAJune 2008

[59] K Soomro and A R Zamir ldquoAction recognition in realisticsports videosrdquo in Computer Vision in Sports pp 181ndash208Springer Berlin Germany 2014

[60] D Weinland E Boyer and R Ronfard ldquoAction recognitionfrom arbitrary views using 3D exemplarsrdquo in Proceedings ofthe 2007 IEEE 11th International Conference on ComputerVision pp 1ndash7 IEEE Rio de Janeiro Brazil October 2007

[61] M Siddiqi S Lee Y-K Lee A Khan and P Truc ldquoHier-archical recognition scheme for human facial expressionrecognition systemsrdquo Sensors vol 13 no 12 pp 16682ndash167132013

[62] M Elmezain and A Al-Hamadi ldquoVision-based human ac-tivity recognition using ldcrfsrdquo International Arab Journal ofInformation Technology (IAJIT) vol 15 no 3 pp 389ndash3952018

[63] A Roy B Banerjee and V Murino ldquoA novel dictionarylearning based multiple instance learning approach to actionrecognition from videosrdquo in Proceedings of the 6th In-ternational Conference on Pattern Recognition Applicationsand Methods pp 519ndash526 Porto Portugal February 2017

[64] A P Sirat ldquoActions as space-time shapesrdquo InternationalJournal of Application or Innovation in Engineering andManagement vol 7 no 8 pp 49ndash54 2018

14 Computational Intelligence and Neuroscience

Computer Games Technology

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

Advances in

FuzzySystems

Hindawiwwwhindawicom

Volume 2018

International Journal of

ReconfigurableComputing

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

thinspArtificial Intelligence

Hindawiwwwhindawicom Volumethinsp2018

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawiwwwhindawicom Volume 2018

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Engineering Mathematics

International Journal of

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Computational Intelligence and Neuroscience

Hindawiwwwhindawicom Volume 2018

Mathematical Problems in Engineering

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Hindawiwwwhindawicom Volume 2018

Human-ComputerInteraction

Advances in

Hindawiwwwhindawicom Volume 2018

Scientic Programming

Submit your manuscripts atwwwhindawicom

Page 2: Human Activity Recognition Using Gaussian Mixture Hidden …downloads.hindawi.com/journals/cin/2019/8590560.pdf · 2019-08-22 · ResearchArticle Human Activity Recognition Using

selection and recognition as shown in Figure 1 Most of theexisting works [18ndash23] focused on feature extraction andselection however very limited works have been done forthe recognition module Some studies exploited conven-tional techniques [24ndash28] Among them HMM is one of thebest candidates for the activity recognition however HMMis generative in nature and less precise than its matching partlike HCRF model [29]

e inspiration behind the recognition stage is the lack ofenhancement in the learning method erefore we havemade the following contribution

(i) e existing HCRF model is inadequate by in-dependence assumptions which may reduce classi-cation accuracy erefore the rst objective of thisstudy is to propose a recognitionmodel that presents anew algorithm to relax the assumption allowing ourmodel in order to use full -covariance distribution

(ii) Another objective of the work is to prove thatcomputation wise our method has very much lowercomplexity against the existing methods In thismethod our goal was to nd some parameters tomaximize the conditional probability of the trainingdata at the training phase erefore in our workwe utilize limited-memory BroydenndashFletcherndashGoldfarbndashShanno (L-BFGS) method to search forthe optimal point However instead of repeating theforward and backward algorithms to compute thegradients as others did [30] we run the forward andbackward algorithms only when calculating theconditional probability and then we reuse the resultto compute the gradients As a result the compu-tation time is signicantly reduced

(iii) A comprehensive set of experiments which yielded aweighted average classication rate 97 that isbetter improvement in the performance against thestate-of-the-art methods

e rest of the paper is organized as follows Section 2presents related works with their limitations Section 3provides the proposed recognition model with its advan-tages Section 4 describes the experimental setup for the

proposed model against four datasets Based on the setup aseries of experiments are presented in Section 5 FinallySection 6 describes conclusion with some future directions

2 Related Works

In a typical HAR system dierent types of latest segmen-tation methods were used in preprocessing module in orderto extract the human body from the activity frame isprocess helps to improve the performance of the activityrecognition system erefore in the literature the authorsof [31ndash36] utilized the latest methods to segment the humanbody from the video frames Similarly for the feature ex-traction dierent latest methodologies have been employedwhich help the classiers to accurately classify the humanactivities (as the workow shown in Figure 1) [37ndash42] eyshowed better performance on dierent datasets and mostof them achieved average accuracy between 70 and 90

Regarding recognition the researchers have proposeddiverse systems which exploit various classiers such asGaussian mixture model (GMM) [43 44] articial neuralnetwork (ANN) [45 46] and support vector machine(SVMs) [47ndash50] ese classiers were principally employedfor frame-based classication Contrarily in many HARsystems [37 51 52] the eminent hidden Markov model(HMM) has extensively been utilized for sequence-basedclassication In the case of frame-level features HMMs arebeneted over vector-based classiers like SVM GMM andANN in terms of eectively handling the sequential dataHowever the Markovian property implied in the traditionalHMM assumes that the current state is a function of the paststate only is causes the labels of two adjacent states in theobservation sequence to hypothetically appear in successionBut in practical implementation this assumption often doesnot meet satisfaction Besides the generative characteristicof HMM and independence presumptions between obser-vations and states also limit its performance [29] To get ridof these limitations the maximum entropy Markov model(MEMM) had been proposed which comparatively performsbetter than HMM [53] However MEMM is associated withthe well-known disadvantage termed as ldquolabel bias problemrdquo

Video frames

Raw dataPreprocessing

Featureextraction

andselection

Segmentation(body shape sequence)

System testing

System testing

TrainedHCRFs

HCRFtraining

Recognition Activitylabels

Output

OutputSymbolsequences

System trainingSequences of symbolLabel

Figure 1 A typical human activity recognition (HAR) system

2 Computational Intelligence and Neuroscience

Two generalized models of MEMM known as condi-tional random fields (CRFs) [29] and HCRF [54] weredeveloped to fix the shortcoming of ldquolabel bias problemrdquo[29] For learning the hidden structure of the sequential dataHCRF facilitates the effectiveness of CRF with hidden statesHowever in both models the per-state normalization isreplaced with global normalization permitting the weightedscores which in turn result in larger parameter spaces ascompared to HMM and MEMM

For example the CRFs achieved in the HAR systemhaving the observed frames from a video are represented byfeature vectorU resultant labelV and unknown state label K

Suppose the problem image labeling is assumed byoriginal labels K with image features U and parameter of themodel is Λ then the later probability (post(K | UΛ))maximized by CRF is given as

post(K ∣ UΛ) ex(f(K UΛ))

z(UΛ) (1)

where the normalization factor is

z(UΛ) 1113944

Kprime

ex f Kprime UΛ( 1113857( 1113857 (2)

Some issues in HCRF implementation are reviewed andanalyzed in the following description e later probabilityof CRF in (1) has been updated by the post(K ∣ UΛ) in aHCRF model that is the addition of exponentials of latentfunctions with all expected labels L as given below

post(V ∣ UΛ) 1113936Lex Λ middot f(V L U)1113864 1113865

z(UΛ)

z(UΛ) 1113944

VprimeL

ex Λ middot f Vprime L U( 11138571113864 1113865

(3)

e above equations are used to warranty the sum toone rule of the conditional probability Vprime is the possibletag for the series of frames and L l1 l2 lT1113864 1113865 is a seriesof hidden states li i 1 2 T and equations (1) and (2)have constant values from 1 to Q (the number of states) Λis the vector factor and f(V L U) is a feature vector thatwill yield a decision which parameter will be educated bythe model en the feature vector concludes the additionof the existing HCRF model For example the underneathselections will create a Markov restraint HCRF with aGaussian distribution at every state

fPriorl (V L U) δ l1 l( 1113857 middot v uforalll

fTransitionllprime (V L U) 1113944

T

t1δ ltminus 1 l( 1113857δ lt lprime( 1113857 middot v uforalll lprime

fOccurencel (V L U) 1113944

T

t1δ lt l( 1113857 middot v uforalll

(4)where each v isin V is the expected tag and every u isin U is apredicted vector e per-component square of the obser-vation vector v at state t (ie vt) is given as

fM1l 1113944

T

t1δ lt l( 1113857 middot v utforalll

fM2l (V L U) 1113944

T

t1δ lt l( 1113857 middot v u

2tforalll

(5)

It can be seen that along with certain set of parameters(Λ) the HCRF addition is similar to the hidden Markovmodel for instance along with the abovementioned featurevector if we choose

Λl

Prior log(b(l)) (6)

Λl

Transition log C l lprime( 1113857( 1113857 (7)

Λl

Occurence minus

12

log 2πσ2l1113872 1113873 +μ2lσ2l

1113888 1113889 (8)

ΛM1l

μl

σ2l (9)

ΛM2l minus

12σ2l

(10)

where b in (6) is an earlier dissemination of Gaussian HMMand C in (7) is an evolution matrix then conditional pos-sibility numerator might be explained as

1113944

L

ex Λ middot f(V L U)1113864 1113865 1113944

L

b l1( 11138571113945

T

t1C ltminus 1 lt( 1113857N u

2t μLt

σLt1113872 1113873

(11)

In the above equationN represents Gaussian distributionEquation (11) is the conditional probability of U given V iscalculated along with a GaussianHMM through equation (11)which has an earlier distribution bwith a conversionmatrixC

Moreover the authors of [30] proposed a comprehensiveform of the HCRF model to tackle composite scatteringsutilizing a linear combination of Gaussian distribution func-tions which is explained as

post(V ∣ UΛ) 1113936L1113936

Mm1ex Λf(V L m U)1113864 1113865

z(UΛ) (12)

In equation (12)M indicates the number of componentsin Gaussian mixture

Lots of works have been developed which showed betterperformance based on the usage of the abovementionedHCRF [55 56] however most of them did not consider thelimitations of the model It is obvious from the aforemen-tioned equations that the existing model employed diago-nal (sloping)-covariance Gaussian distribution whichmeans that the variables (columns of ui i 1 2 N )were presumed to be couples independent On the otherhand equations (8)ndash(10) suggest that with a specific set ofvalue each state observation density will congregate toGaussian procedure Unluckily there is no training method

Computational Intelligence and Neuroscience 3

designed yet to guarantee this convergence and thosesuppositions might decrease the accuracy results

erefore we proposed the improved version of theHCRF technique that has the ability to openly employ full-covariance Gaussian mixture in the feature function eproposed model will get the benefits of hidden conditionalrandom field model that completely considered the draw-backs of the previous method

3 Proposed Methodology

31 Feature Extraction In our previous work we utilizedsymlet wavelet [37] for extracting various features from theactivity frames ere are number of reasons for using thesymlet wavelet which produces relatively better classificationresults ese include its capability to extract the conspic-uous information from the activity frames in terms of fre-quency and its support to the characteristics of the grayscaleimages like orthogonality biorthogonality and reversebiorthogonality For a certain provision size the symlet ischaracterized with the highest number of vanishing mo-ments and has the least asymmetry

32 Proposed Hidden Conditional Random Fields (HCRFs)Model As described earlier the current Gaussian mixtureHCRF model does not have the capability of utilizing full-covariance distributions and also does not guarantee theconjunction of its factors to certain values upon which theconditional probability is demonstrated as a combination ofthe normal density functions

To address these limitations we explicitly involve amixture of Gaussian distributions in the feature functions asillustrated in the following forms

fl

Prior(V L U) δ l1 l( 1113857 middot v uforalll (13)

fllprime

Transition(V L U) 1113944

T

t1δ ltminus 1 l( 1113857δ lt lprime( 1113857 middot v uforalll lprime

(14)

fl

Observation(V L U) 1113944

T

t1log 1113944

M

m1ΓObslm N u

2t μlmΣlm1113872 1113873⎛⎝ ⎞⎠

δ lt l( 1113857 middot v

(15)

then

N u2t μlmΣlm1113872 1113873

1

(2π)D2 Σlm1113868111386811138681113868

111386811138681113868111386812ex

middot minus12

u2t minus μlm1113872 1113873prime1113944

minus 1

lm

x2t minus μlm1113872 1113873⎛⎝ ⎞⎠

(16)

where N represents the number of density functionsGamma ldquoΓrdquo considers the appropriate information of theentire observations D indicates the dimension of the ob-servation and ΓObslm presents the partying weightiness forthe mth constituent along with mean μlm and covariancematrix Σlm

As indicated in equation (14) when we change some ofthe parameters such as Γ μ and Σ then we may build acombination of the standard densities e resultant con-ditional probability might be written as

post(V ∣ UΛ Γ μΣ) 1113936Lex(P(L) + T(L) + O(L))

z(UΛ Γ μΣ)

P(L) 1113944l

ΛPriorl fPriorl (V L U)

T(L) 1113944

llprime

ΛTransitionllprime fTransitionllprime (V L U)

O(L) 1113944l

fObservationl (V L U)

(17)

therefore

post(V ∣ UΛ Γ μΣ) 1113936Lexp 1113936lΛPrior

l fPriorl (V L U) + 1113936llprimeΛTransitionllprime fTransition

llprime (V L U) + 1113936lfObservationl (V L U)1113872 1113873

z(UΛ Γ μΣ) (18)

post(V ∣ UΛ Γ μΣ) 1113936Ll1 l2 lT

ex ΛPriorl1+ 1113936

Tt1 Λ

Transitionltminus 1 lt

1113872 1113873 + log 1113936Mm1Γ

Obsltm

N u2t μltm

Σltm1113872 11138731113872 11138731113872 1113873

z(UΛ Γ μΣ) (19)

post(V ∣ UΛ Γ μΣ) Score(U|VΛ Γ μΣ)

z(UΛ Γ μΣ) (20)

e forward and backward algorithms are used to cal-culate the conditional probability based on equations (19)and (20) that can be written as

4 Computational Intelligence and Neuroscience

ατ 1113944

Ll1 l2 lτl

exp ΛPriorl1+ 1113944

T

t1ΛTransitionltminus 1 lt

1113872 1113873 + log 1113944

M

m1ΓObsltm

N middot u2t μltm

Σltm1113872 1113873⎛⎝ ⎞⎠⎛⎝ ⎞⎠

1113944lprime

ατminus 1 lprime( 1113857exp ΛTransitionllprime + log 1113944M

m1ΓObsltm

N middot uτ μlmΣlm1113872 1113873⎛⎝ ⎞⎠

(21)

βτ(l) 1113944

L lτl lτ+1 lT

ex ΛPriorl1

+ 1113944T

t1ΛTransitionltminus 1lt

1113872 1113873 + log 1113944M

m1ΓObsltm

N middot u2t μltm

Σltm1113872 1113873⎛⎝ ⎞⎠

1113944lprime

βτ+1 lprime( 1113857exp ΛTransitionllprime + log 1113944M

m1ΓObslm N uτ μlmΣlm1113872 1113873⎛⎝ ⎞⎠⎛⎝ ⎞⎠

(22)

Score(V ∣ VΛ Γ μΣ) 1113944l

αT(l) 1113944l

β1(l) (23)

In the training data to maximize the conditionalprobability we initially focused on calculating the pa-rameters (Λ Γ μ andΣ) In the proposed approachlimited-memory BroydenndashFletcherndashGoldfarbndashShanno(L-BGFS) method has been implemented in order tosearch the optimum point Unlikely the other models[30] both the forward and backward algorithms are usedto compute the conditional probability and the resultswere reused for finding the gradients is makes thealgorithm more significant in reducing the computationtime

At the observation level we particularly incorporated thefull-covariance matrix in the feature function as shown in(16) Equation (17) may be used for getting the normaldistribution which is further elaborated in the followingequations

d Score(V ∣ UΛ Γ μΣ)dΛPriorl

1113944

L

dg(V L U)

dΛPriorl

ex(g(V L U))

1113944

L

fPriorl (V L U)ex(g(V L U)) β1(l)

(24)

e d Score function is a gradient function for a variableof the prior probability vectord Score(V ∣ UΛ Γ μΣ)

dΛTransitionl

1113944

L

dg(V L U)

dΛTransitionllprime

ex(g(V L U))

1113944

L

fTransitionllprime (V L U)exp(g(V L U))

(25)

e d Score function is a gradient function for a variableof the transition probability vector

d Score(V ∣ UΛ Γ μΣ)dΓObslm

1113944

L

dg(V L U)

dΓObslm

ex(g(V L U))

1113936LfObservation

l (V L U)

dΓObslm

ex(g(V L U))

1113944

L

1113944

T

t1

N ut μlmΣlm1113872 1113873

1113936Mm1Γ

Obslm N ut μlmΣlm1113872 1113873

δ lt l( 1113857ex(g(V L U))

1113944T

t1

N ut μlmΣlm1113872 1113873

1113936Mm1Γ

Obslm N ut μlmΣlm1113872 1113873

α(t l)c(t + 1)

(26)

Computational Intelligence and Neuroscience 5

e d Score function is a gradient function for aGaussian mixture weight variable Here a function V(t) canbe determined as

c(t) suml

β(t l) (27)

d Score(V ∣ UΛ Γ μΣ)dμlm

sumT

t1

ΓObslm dN ut μlmΣlm( )dμlmsumMm1ΓObslm N ut μlmΣlm( )

middot α(t l)c(t + 1)(28)

e d Score function is a gradient function for theGaussian distribution mean

d Score(V ∣ UΛ Γ μΣ)dΣlm

sumT

t1

ΓObslm dN ut μlmΣlm( )dΣlmsumMm1ΓObslm N ut μlmΣlm( )

middot α(t l)c(t + 1)(29)

e d Score function is a gradient function for the co-variance of the Gaussian distributions

Equations (24)ndash(27) presented above describe an anal-ysis method algorithm for calculating values of gradients fora feature function the mean of Gaussian distributions and

the covariance of the prior probability vector the transitionprobability vector and the observation probability vectorobtained from the existing HCRF

In our model the recognition of a variety of real-timeactivities can be divided into two steps a training step and aninference step In the rst step data with known labels areinputted for recognizing the target as well as training thehidden conditional eld model In the inference step theinputs to be actually estimated are ordered dependent onparameters determined in the training phase

If the activity frame is acting as an input in the training stepthen in the preprocessing step the applied distinctive lightingeects are decreased for detecting and extracting faces from theactivity frames At that point the movable features are extri-cated from the various facial parts for creating the featurevector After that the feature vector obtained serves as an inputto a full-covariance Gaussian-mixed hidden conditional ran-dom eld model of the suggested recognition model

As mentioned in the earlier discussion a feature gradient isgenerally determined by LBFG approach in the training phaseof the HCRF model Nonetheless in the current gradientcalculation technique a forward and backward iterative exe-cution algorithm is iteratively called upon which needs anexceptionally high computational time and thus leads to re-duction in the computational speed Another analysis ap-proach has been formulated that reduces the invoking of theforward and backward iterative execution algorithm using vegradient functions determined by equations (24)ndash(28) Using

Observed activity labels

Hidden CRFparameter

Modelweights

Hidden CRFtraining engine

Data calibrationand normalization

Unified interface

Raw data 1

2

5

7

6

3

4

4

ClusteringVectorquantization

Feature extraction

Discreteinput

sequence

Normalizeddata

Kernelvectors

Kernelvectors

Feature vectors

External interfaceInternal interface

Data files

Codebookvectors

Figure 2 Workow diagram of the proposed recognition model

6 Computational Intelligence and Neuroscience

this analysis the real-time computation can be carried out at ahigher speed resulting in an enormous decrease in the com-putational time compared to a known analysis approach eoverall workflow of the proposed model is shown in Figure 2

4 Model Validation

41 Datasets Used In this work we employed four open-source standard action datasets like Weizmann actiondatasets [57] KTH action dataset [58] UCF sports dataset[59] and IXMAS action dataset [60] for corroborating theproposed HCRF model performance All the datasets areexplained below

411 Weizmann Action Dataset is dataset consisted of10 actions such as bending running walking skippingplace jumping side movement jumping forward two handwaving and one hand waving that were performed by total9 subjects is dataset comprised of 90 video clips with

average of 15 frames per clip where the frame size is144 times180

412 KTH Action Dataset KTH dataset employed for ac-tivity recognition comprised of 25 subjects who performed 6activities like running walking boxing jogging hand-clapping and hand waving in four distinctive scenariosUsing a static camera in the homogenous background atotal of 2391 sequences were taken with a frame size of160times120

413 UCF Sports Dataset In this dataset there were 182videos which were evaluated by n-fold cross-validation ruleis dataset has been taken from different sports activities inbroadcast television channels Some of the videos had highintraclass similarities is dataset was also collected using astatic camera is dataset covers 9 activities like runningdiving lifting golf swinging skating kicking walking

Table 2 Confusion matrix of the proposed recognition model using KTH action dataset (unit )

Activities Walking Jogging Running Boxing Hand-wave HandclapWalking 100 0 0 0 0 0Jogging 0 98 1 1 0 0Running 2 1 95 1 1 0Boxing 0 2 1 97 0 0Hand-wave 1 0 1 0 98 0Handclap 0 0 1 0 0 99Average 9783

Table 1 Confusion matrix of the proposed recognition model using Weizmann action dataset (unit )

Activities Bend Jack Pjump Run Side Skip Walk Wave 1 Wave 2Bend 98 0 1 0 0 1 0 0 0Jack 1 96 0 0 2 0 1 0 0Pjump 0 1 97 0 1 0 0 1 0Run 0 0 0 99 0 0 0 1 0Side 1 2 0 0 95 0 0 2 0Skip 0 0 0 0 0 100 0 0 0Walk 1 0 0 1 0 1 96 0 1Wave 1 0 1 1 0 0 0 0 98 0Wave 2 0 1 0 0 1 1 1 1 95Average 9711

Table 3 Confusion matrix of the proposed recognition model using UCF sports dataset (unit )

Activities Diving GS Kicking Lifting HBR Run Skating BS WalkDiving 95 1 2 1 0 1 0 0 0GS 1 94 0 0 2 1 1 1 0Kicking 0 2 98 0 0 0 0 0 0Lifting 1 1 1 94 1 0 0 1 1HBR 0 0 2 0 96 1 0 1 0Running 0 3 0 0 0 97 0 0 0Skating 1 0 1 1 0 1 95 0 1BS 0 0 0 1 0 1 0 97 1Walking 0 0 0 0 0 0 0 0 100Average 9622GS golf swinging HBR horseback riding and BS baseball swinging

Computational Intelligence and Neuroscience 7

Table 4 Confusion matrix of the proposed recognition model using IXMAS action dataset (unit )

Activities CA SD GU TA Walk Wave Punch KickCA 97 0 0 1 2 0 0 0SD 0 99 1 0 0 0 0 0GU 1 2 94 3 0 0 0 0TA 0 0 1 95 2 1 1 0Walk 0 1 1 0 98 0 0 0Wave 0 0 2 0 1 97 0 0Punch 0 1 0 1 0 2 96 0Kick 0 0 0 0 1 0 0 99Average 9688CA cross arm SD sit down GU get up and TA turn around

Table 5 Classification results of the proposed system on Weizmann action dataset (A) using ANN (B) using SVM (C) using HMM and(D) using existing HCRF [30] while removing the proposed HCRF model (unit )

Activities Bend Jack Pjump Run Side Skip Walk Wave 1 Wave 2(A)Bend 70 4 5 3 3 2 5 5 3Jack 4 68 3 6 7 2 3 3 4Pjump 2 4 75 6 3 2 2 2 4Run 4 2 3 72 6 2 5 3 3Side 5 3 5 4 65 6 4 6 2Skip 4 6 4 3 5 67 3 2 6Walk 4 2 4 7 3 4 70 3 3Wave 1 2 1 3 3 4 5 7 71 4Wave 2 2 5 3 6 4 4 3 5 68Average 6955(B)Bend 69 3 4 4 6 4 2 3 5Jack 2 72 2 3 4 3 5 4 5Pjump 1 4 75 2 4 5 4 2 3Run 2 4 3 78 2 2 4 2 3Side 2 4 5 3 70 4 3 5 4Skip 2 1 3 2 4 80 3 3 2Walk 2 0 3 4 3 2 82 1 3Wave 1 2 2 3 4 3 2 3 77 4Wave 2 1 2 1 2 3 1 3 4 83Average 7622(C)Bend 82 3 0 2 2 3 1 5 2Jack 3 80 1 2 3 2 3 4 2Pjump 3 4 85 3 0 0 1 2 2Run 5 4 2 79 0 2 1 3 4Side 0 1 5 4 81 3 1 2 3Skip 3 1 2 2 3 88 0 0 1Walk 0 2 3 2 1 2 83 3 4Wave 1 1 3 2 2 4 2 3 78 5Wave 2 1 2 2 2 2 3 1 0 87Average 8256(D)Bend 80 2 3 1 4 0 5 2 3Jack 1 88 0 2 0 3 2 3 1Pjump 0 2 90 1 0 3 0 2 2Run 2 1 2 85 2 3 0 0 5Side 4 1 2 3 80 4 1 2 3Skip 1 4 0 5 1 84 0 3 2Walk 2 1 0 0 1 2 89 2 3Wave 1 3 0 1 2 0 2 0 91 1Wave 2 4 1 3 0 2 3 0 2 85Average 8578

8 Computational Intelligence and Neuroscience

horseback riding and baseball swinging Each frame has asize of 720times 480

414 IXMAS Action Dataset IXMAS (INRIA Xmas motionacquisition sequences) dataset comprised of 13 activity classeswhich were performed by 11 actors each 3 times Every actoropted a free orientation as well as position e dataset hasprovided annotated silhouettes for each person For our ex-periments we have selected only 8 action classes like walkcross arms punch turn around sit down wave get up andkick IXMAS dataset is a multiview dataset for a view-invarianthuman activity recognition where each frame has a size of390times 291 is dataset has a major occlusion and that maycause misclassification therefore we utilized global histogramequalization [61] in order to resolve the occlusion issue

42 Setup For a comprehensive validation we carried outthe following set of experiments executed using Matlab

(i) e first experiment was conducted on each datasetseparately in order to show the performance of theproposed model In this experiment we employed10-fold cross-validation rule which means that data

from 9 subjects were utilized for training data whilethe data from one subject was picked as a testingdata e procedure was reiterated for 10 timesprovided each subject data is utilized for bothtraining and testing

(ii) e second experiment was conducted in the ab-sence of the proposed recognition model on all thefour datasets that will show the importance of thedeveloped model For this purpose we used theexisting eminent classifiers like SVM ANN HMMand existing HCRF [30] as a recognition modelrather than utilizing the proposed HCRF model

(iii) e third experiment was conducted to show theperformance of the proposed approach against thestate-of-the-art methods

(iv) In the last experiment the computational com-plexity of the proposed HCRF model was comparedwith forwardbackward algorithms

5 Results and Discussion

51 First Experiment As described before this experimentvalidates the performance of the proposed recognition model

Table 6 Classification results of the proposed system on KTH action dataset (A) using ANN (B) using SVM (C) using HMM and (D)using existing HCRF [30] while removing the proposed HCRF model (unit )

Activities Walking Jogging Running Boxing Hand-wave Handclap(A)Walking 79 5 6 4 3 3Jogging 3 81 5 3 4 4Running 6 4 77 5 5 3Boxing 6 7 6 69 5 7Hand-wave 4 7 5 5 73 6Handclap 4 6 5 4 6 75Average 7566(B)Walking 82 2 3 5 4 4Jogging 3 86 2 3 2 4Running 5 3 80 4 5 3Boxing 5 3 3 79 4 6Hand-wave 1 4 3 3 89 0Handclap 3 5 2 4 3 83Average 8317(C)Walking 86 3 2 4 2 3Jogging 0 88 3 2 4 3Running 0 3 90 0 4 3Boxing 3 0 4 92 1 0Hand-wave 1 3 2 2 91 1Handclap 1 3 4 1 2 89Average 8933(D)Walking 90 3 0 3 4 0Jogging 2 88 2 3 3 2Running 4 2 92 0 0 2Boxing 1 3 2 91 3 0Hand-wave 0 1 3 2 93 1Handclap 1 3 2 4 3 87Average 9017

Computational Intelligence and Neuroscience 9

on an individual dataset e overall results are shown inTables 1 (using Weizmann dataset) 2 (using KTH dataset) 3(usingUCF sports dataset) and 4 (using IXMAS) respectively

As observed from Tables 1ndash4 the proposed recognitionmodel constantly obtained higher recognition rates on in-dividual dataset is result shows the robustness of theproposed model which means that the model not onlyshowed better performance on one dataset but also showedbetter performance across multiple spontaneous datasets

52 Second Experiment As described before the secondexperiment was conducted in the absence of the proposed

recognition model to show the importance of the proposedmodel using all the four datasets For this purpose we usedthe existing eminent classifiers like SVM ANN HMM andexisting HCRF [30] as a recognition model rather thanutilizing the proposed HCRF model

Tables 5ndash8 show that when the proposed HCRF modelwas substituted with ANN SVM HMM and existing HCRF[30] the system failed to accomplish higher recognitionrates e better performance of the proposed HCRF modelis visualized in Tables 1ndash4 which show that the proposedHCRF model effectively fix the drawbacks of HMM andexisting HCRF that has been extensively utilized for se-quential HAR

Table 7 Classification results of the proposed system onUCF sports dataset (A) using ANN (B) using SVM (C) using HMM and (D) usingexisting HCRF [30] while removing the proposed HCRF model (unit )

Activities Diving GS Kicking Lifting HBR Run Skating BS Walk(A)Diving 68 4 2 5 6 6 4 3 2GS 2 71 2 4 5 4 6 3 3Kicking 3 4 70 3 5 4 2 3 6Lifting 5 4 3 65 5 6 4 6 2HBR 3 4 6 3 66 4 5 5 4Running 3 3 5 4 6 64 6 4 5Skating 2 5 4 5 3 4 69 3 5BS 4 2 5 3 4 6 5 67 4Walking 5 4 2 3 4 3 6 3 70Average 6778(B)Diving 71 4 2 3 5 6 3 2 4GS 3 77 2 4 3 2 5 2 2Kicking 4 2 74 4 5 3 2 3 3Lifting 5 6 3 69 4 3 5 3 2HBR 2 3 3 2 80 2 4 2 2Running 2 3 2 2 5 75 6 2 3Skating 2 1 2 3 4 4 78 2 4BS 3 4 6 3 4 2 3 70 5Walking 4 1 2 4 2 3 0 3 81Average 7500(C)Diving 79 3 2 2 3 4 3 2 2GS 0 83 2 4 3 2 1 3 2Kicking 1 2 85 1 3 3 2 3 0Lifting 3 0 2 82 3 2 4 2 2HBR 0 2 2 4 80 0 5 3 4Running 1 2 1 3 4 84 2 1 2Skating 2 0 3 4 0 1 86 3 1BS 1 1 1 2 0 3 0 88 4Walking 1 2 4 2 5 2 4 3 77Average 8267(D)Diving 90 3 0 1 0 2 2 1 1GS 3 84 2 1 3 1 3 2 1Kicking 3 4 85 0 0 2 3 1 2Lifting 1 2 1 89 1 1 1 2 2HBR 0 2 1 0 91 2 3 0 1Running 2 3 1 2 3 80 4 2 3Skating 2 4 1 2 3 0 84 4 0BS 2 1 1 2 1 0 3 88 2Walking 0 2 1 1 0 1 4 0 91Average 8689GS golf swinging HBR horseback riding BS baseball swinging

10 Computational Intelligence and Neuroscience

53 ird Experiment In this experiment a comparativeanalysis was made between the state-of-the-art methods andthe proposed model All of these approaches were imple-mented by the instructions provided in their particular ar-ticles A 10-fold cross-validation rule was employed on eachdataset as explained in Section 4 e average classicationresults of the existing methods along with the proposedmethod across dierent datasets are summarized in Table 9

Table 8 Classication results of the proposed system on IXMASaction dataset (A) using ANN (B) using SVM (C) using HMMand (D) using existing HCRF [30] while removing the proposedHCRF model (unit )

Activities CA SD GU TA Walk Wave Punch Kick(A)CA 65 5 7 6 5 4 3 5SD 5 72 4 3 5 4 3 4GU 4 3 75 5 3 2 4 4TA 6 7 4 68 3 5 4 3Walk 3 4 5 4 70 6 4 4Wave 4 6 5 3 4 71 3 4Punch 3 5 5 6 7 3 67 4Kick 4 5 4 6 5 4 3 69Average 6962(B)CA 77 3 4 2 2 3 5 4SD 3 79 3 2 4 5 2 2GU 5 6 69 3 4 5 4 4TA 2 3 2 80 4 4 3 2Walk 3 5 4 2 71 5 6 4Wave 2 6 3 5 4 73 4 3Punch 1 5 3 4 1 3 81 2Kick 3 6 7 4 3 5 4 68Average 7475(C)CA 79 3 4 1 2 4 3 4SD 1 84 3 2 3 1 4 2GU 0 1 88 1 2 3 2 3TA 5 2 3 79 2 3 2 4Walk 1 0 3 1 90 2 3 0Wave 2 3 1 0 3 86 2 3Punch 1 0 2 3 0 4 89 1Kick 3 2 4 1 0 2 4 84Average 8477(D)CA 90 1 2 0 3 4 0 0SD 3 85 2 1 3 3 2 1GU 0 1 91 1 0 2 3 2TA 1 3 2 87 1 2 2 2Walk 1 0 3 1 89 2 1 3Wave 0 2 1 0 2 90 3 1Punch 1 4 2 1 2 3 84 3Kick 1 2 4 1 3 2 4 83Average 8738CA cross arm SD sit down GU get up TA turn around

Table 9 Weighted average recognition rates of the proposedmethod with the existing state-of-the-art methods (unit )

State-of-the-art works Average classicationrates

Standarddeviation

GMM 633 plusmn27SVM 675 plusmn44HMM 828 plusmn38Embedded HMM 859 plusmn19[62] 921 plusmn32[63] 843 plusmn49[18] 936 plusmn27[19] 930 plusmn16[22] 927 plusmn25[64] 801 plusmn32Proposed method 972 plusmn28

0

100

200

300

400

500

600

Exec

utio

n tim

e (s)

1 2 3 4 5 6 7 8Number of states

ForwardbackwardProposed HCRF model

(a)Ex

ecut

ion

time (

s)

1 2 3 4 5 6 7 8

ForwardbackwardProposed HCRF model

0

2

4

6

8

10

12

14

16

Number of mixtures

(b)

Figure 3 An illustration of gradient computational time (equation(30)) of the previous forward and backward algorithms and theproposed HCRF model (a)Q 1 minus 5 M 5 T 90 and (b)Q 5 M 1 minus 5 T 90

Computational Intelligence and Neuroscience 11

It is obvious from Table 9 that the proposed methodshowed a significant performance against the existing state-of-the-art methods erefore the proposed method accu-rately and robustly recognizes the human activities usingdifferent video data

54 Fourth Experiment In this experiment we have pre-sented the computational complexity that is also one of thecontributions in this paper e implementations of theprevious HCRF are available in literature which calculatethe gradients by reiterating the forward and backwardtechniques while the proposed HCRF model executes themonce only and cashes the outcomes for the later use From(21) and (22) it is clear that the forward or backwardtechnique has a complexity ofO(TQ2M) where Trepresentsthe input sequence lengthQ represents the number of statesand M indicates the number of mixtures e proposedHCRFmodel however requires a full complexity of O(TM)

to calculate gradients as can be seen from (22)ndash(29)Figure 3 shows a comparison of the execution time when

the gradients are computed by the forward (or backward)algorithm and by our proposed method e computationaltime is calculated by running Matlab R2013a with thespecification of Intelreg Pentiumreg Coretrade i7-6700 (34GHz)with a RAM capacity of 16GB

6 Conclusion

In healthcare and telemedicine the human activity rec-ognition (HAR) can be best explained by helping physi-cally disable personsrsquo scenario A paralyzed patient withhalf of the body critically attacked by paralysis is com-pletely unable to perform their daily exercises e doctorsrecommend specific activities to get better improvement intheir health So for this purpose the doctors need a humanactivity recognition (HAR) system through which they canmonitor the patientsrsquo daily routines (activities) on a reg-ular basis

e accuracy of most of the HAR systems depends uponthe recognition modules For feature extraction and selec-tion modules we used some of the existing well-knownmethods while for the recognition module we proposed theusage of HCRF model which is capable of approximating acomplex distribution using a mixture of Gaussian densityfunctions e proposed model was assessed against fourpublicly available standard action datasets From our ex-periments it is obvious that the proposed full-covarianceGaussian density function showed a significant improve-ment in accuracy than the existing state-of-the-art methodsFurthermore we also proved that such improvement issignificant from statistical point of view by showing valuele02 of the comparison Similarly the complexity analysispoints out that the proposed computational method stronglydecreases the execution time for the hidden conditionalrandom field model

e ultimate goal of this study is to deploy the proposedmodel on smartphones Currently the proposed model isusing full-covariance matrix however this might be time

consuming especially when using on smartphones Using alightweight classifier such as K-nearest neighbor (K-NN)could be one possible solution But K-NN is very muchsensitive to environmental factor (like noise) erefore infuture we will try to investigate further research to reducethe time and sustain the same recognition rate whenemploying on smartphones in real environment

Data Availability

e data used to support the findings of this study areavailable from the corresponding author upon request

Conflicts of Interest

e authors declare there are no conflicts of interest

Acknowledgments

is study was supported by the Jouf University SakakaAljouf Kingdom of Saudi Arabia under the registration no39791

References

[1] Z Chang J Cao and Y Zhang ldquoA novel image segmentationapproach for wood plate surface defect classification throughconvex optimizationrdquo Journal of Forestry Research vol 29no 6 pp 1789ndash1795 2018

[2] Z S Abdallah M M Gaber B Srinivasan andS Krishnaswamy ldquoAdaptive mobile activity recognitionsystemwith evolving data streamsrdquoNeurocomputing vol 150pp 304ndash317 2015

[3] M S Bakli M A Sakr and T H A Soliman ldquoA spatio-temporal algebra in Hadoop for moving objectsrdquo Geo-SpatialInformation Science vol 21 no 2 pp 102ndash114 2018

[4] W Zhao L Yan and Y Zhang ldquoGeometric-constrainedmulti-view image matching method based on semi-globaloptimizationrdquo Geo-Spatial Information Science vol 21 no 2pp 115ndash126 2018

[5] H Gao X Zhang J Zhao and D Li ldquoTechnology of in-telligent driving radar perception based on driving brainrdquoCAAI Transactions on Intelligence Technology vol 2 no 3pp 93ndash100 2017

[6] Y Wang X Jiang R Cao and X Wang ldquoRobust indoorhuman activity recognition using wireless signalsrdquo Sensorsvol 15 no 7 pp 17195ndash17208 2015

[7] J Wang Z Liu Y Wu and J Yuan ldquoMining actionlet ensemblefor action recognition with depth camerasrdquo in Proceedings of theIEEE Conference on Computer Vision and Pattern Recognitionpp 1290ndash1297 IEEE Providence RI USA June 2012

[8] S Karungaru M Daikoku and K Terada ldquoMulti camerasbased indoors human action recognition using fuzzy rulesrdquoJournal of Pattern Recognition Research vol 10 no 1pp 61ndash74 2015

[9] S Shukri L M Kamarudin and M H F Rahiman ldquoDevice-free localization for human activity monitoringrdquo in IntelligentVideo Surveillance IntechOpen London UK 2018

[10] L Fiore D Fehr R Bodor A Drenner G Somasundaramand N Papanikolopoulos ldquoMulti-camera human activitymonitoringrdquo Journal of Intelligent and Robotic Systemsvol 52 no 1 pp 5ndash43 2008

12 Computational Intelligence and Neuroscience

[11] C Zhang and Y Tian ldquoRGB-D camera-based daily livingactivity recognitionrdquo Journal of Computer Vision and ImageProcessing vol 2 no 4 pp 1ndash7 2012

[12] M Leo T DrsquoOrazio and P Spagnolo ldquoHuman activityrecognition for automatic visual surveillance of wide areasrdquo inProceedings of the ACM 2nd International Workshop on VideoSurveillance and Sensor Networks pp 124ndash130 ACM NewYork NY USA October 2004

[13] C-B Jin S Li and H Kim ldquoReal-time action detection invideo surveillance using sub-action descriptor with multi-CNNrdquo 2017 httpsarxivorgabs171003383

[14] W Niu J Long D Han and Y-F Wang ldquoHuman activitydetection and recognition for video surveillancerdquo in Pro-ceedings of the 2004 IEEE International Conference on Mul-timedia and Expo (ICME) (IEEE Cat No 04TH8763) vol 1pp 719ndash722 IEEE Taipei Taiwan June 2004

[15] Q Li M Zheng F Li J Wang Y-A Geng and H JiangldquoRetinal image segmentation using double-scale non-linearthresholding on vessel support regionsrdquo CAAI Transactionson Intelligence Technology vol 2 no 3 pp 109ndash115 2017

[16] L M G Fonseca L M Namikawa and E F CastejonldquoDigital image processing in remote sensingrdquo in Proceedingsof the 2009 Tutorials of the XXII Brazilian Symposium onComputer Graphics and Image Processing pp 59ndash71 IEEERio de Janeiro Brazil October 2009

[17] K Buys C Cagniart A Baksheev T De Laet J De Schutterand C Pantofaru ldquoAn adaptable system for RGB-D basedhuman body detection and pose estimationrdquo Journal of VisualCommunication and Image Representation vol 25 no 1pp 39ndash52 2014

[18] M Sharif M A Khan T Akram M Y Javed T Saba andA Rehman ldquoA framework of human detection and actionrecognition based on uniform segmentation and combinationof Euclidean distance and joint entropy-based features se-lectionrdquo EURASIP Journal on Image and Video Processingvol 2017 no 1 p 89 2017

[19] K-P Chou M Prasad D Wu et al ldquoRobust feature-basedautomated multi-view human action recognition systemrdquoIEEE Access vol 6 pp 15283ndash15296 2018

[20] M Ullah H Ullah and I M Alseadonn ldquoHuman actionrecognition in videos using stable featuresrdquo Signal and ImageProcessing An International Journal vol 8 no 6 pp 1ndash102017

[21] M Sharif M A Khan F Zahid J H Shah and T AkramldquoHuman action recognition a framework of statisticalweighted segmentation and rank correlation-based selectionrdquoPattern Analysis and Applications pp 1ndash14 2019

[22] J Zang L Wang Z Liu Q Zhang G Hua and N ZhengldquoAttention-based temporal weighted convolutional neuralnetwork for action recognitionrdquo in Proceedings of the IFIPInternational Conference on Artificial Intelligence Applicationsand Innovations pp 97ndash108 Springer Rhodes Greece May2018

[23] V-D Hoang ldquoMultiple classifier-based spatiotemporal fea-tures for living activity predictionrdquo Journal of Informationand Telecommunication vol 1 no 1 pp 100ndash112 2017

[24] S Dharmalingam and A Palanisamy ldquoVector space basedaugmented structural kinematic feature descriptor for humanactivity recognition in videosrdquo ETRI Journal vol 40 no 4pp 499ndash510 2018

[25] D Liu Y Yan M-L Shyu G Zhao and M Chen ldquoSpatio-temporal analysis for human action detection and recognitionin uncontrolled environmentsrdquo International Journal of

Multimedia Data Engineering and Management vol 6 no 1pp 1ndash18 2015

[26] C A Ronao and S-B Cho ldquoHuman activity recognition withsmartphone sensors using deep learning neural networksrdquoExpert Systems with Applications vol 59 pp 235ndash244 2016

[27] A arwat H Mahdi M Elhoseny and A E HassanienldquoRecognizing human activity in mobile crowdsensing envi-ronment using optimized k-NN algorithmrdquo Expert Systemswith Applications vol 107 pp 32ndash44 2018

[28] A Jalal Y-H Kim Y-J Kim S Kamal and D Kim ldquoRobusthuman activity recognition from depth video using spatio-temporal multi-fused featuresrdquo Pattern Recognition vol 61pp 295ndash308 2017

[29] J Lafferty A McCallum and F C Pereira ldquoConditionalrandom fields probabilistic models for segmenting and la-beling sequence datardquo in Proceedings of the 18th InternationalConference on Machine Learning (ICML-2001) pp 282ndash289Williamstown MA USA June-July 2001

[30] A Gunawardana M Mahajan A Acero and J C PlattldquoHidden conditional random fields for phone classificationrdquoin Proceedings of the INTERSPEECH vol 2 pp 1117ndash1120Citeseer Lisbon Portugal September 2005

[31] L Yang Q Song Z Wang and M Jiang ldquoParsing r-cnn forinstance-level human analysisrdquo in Proceedings of the IEEEConference on Computer Vision and Pattern Recognitionpp 364ndash373 Long Beach CA USA June 2019

[32] X Liang Y Wei L Lin et al ldquoLearning to segment human bywatching youtuberdquo IEEE Transactions on Pattern Analysisand Machine Intelligence vol 39 no 7 pp 1462ndash1468 2016

[33] M M Requena ldquoHuman segmentation with convolutionalneural networksrdquo esis University of Alicante AlicanteSpain 2018

[34] H Gao S Chen and Z Zhang ldquoParts semantic segmentationaware representation learning for person re-identificationrdquoApplied Sciences vol 9 no 6 p 1239 2019

[35] C Peng S-L Lo J Huang and A C Tsoi ldquoHuman actionsegmentation based on a streaming uniform entropy slicemethodrdquo IEEE Access vol 6 pp 16958ndash16971 2018

[36] M H Siddiqi A M Khan and S-W Lee ldquoActive contourslevel set based still human body segmentation from depthimages for video-based activity recognitionrdquo KSII Trans-actions on Internet and Information Systems (TIIS) vol 7no 11 pp 2839ndash2852 2013

[37] M Siddiqi R Ali M Rana E-K Hong E Kim and S LeeldquoVideo-based human activity recognition using multilevelwavelet decomposition and stepwise linear discriminantanalysisrdquo Sensors vol 14 no 4 pp 6370ndash6392 2014

[38] S Althloothi M H Mahoor X Zhang and R M VoylesldquoHuman activity recognition using multi-features and mul-tiple kernel learningrdquo Pattern Recognition vol 47 no 5pp 1800ndash1812 2014

[39] A Jalal S Kamal and D Kim ldquoA depth video-based humandetection and activity recognition using multi-features andembedded hidden markov models for health care monitoringsystemsrdquo International Journal of Interactive Multimedia andArtificial Intelligence vol 4 no 4 p 54 2017

[40] T Dobhal V Shitole G omas and G Navada ldquoHumanactivity recognition using binary motion image and deeplearningrdquo Procedia Computer Science vol 58 pp 178ndash1852015

[41] S Zhang Z Wei J Nie L Huang S Wang and Z Li ldquoAreview on human activity recognition using vision-basedmethodrdquo Journal of Healthcare Engineering vol 2017 ArticleID 3090343 31 pages 2017

Computational Intelligence and Neuroscience 13

[42] Y Yang B Zhang L Yang C Chen and W Yang ldquoActionrecognition using completed local binary patterns and mul-tiple-class boosting classifierrdquo in Proceedings of the 2015 3rdIAPR Asian Conference on Pattern Recognition (ACPR) IEEEpp 336ndash340 Kuala Lumpur Malaysia November 2015

[43] W Lin M T Sun R Poovandran and Z Zhang ldquoHumanactivity recognition for video surveillancerdquo in Proceedings ofthe 2008 IEEE International Symposium on Circuits andSystems pp 2737ndash2740 IEEE Seattle WA USA May 2008

[44] H Permuter J Francos and I Jermyn ldquoA study of Gaussianmixture models of color and texture features for imageclassification and segmentationrdquo Pattern Recognition vol 39no 4 pp 695ndash706 2006

[45] M Fiaz and B Ijaz ldquoVision based human activity trackingusing artificial neural networksrdquo in Proceedings of the 2010International Conference on Intelligent and Advanced Systemspp 1ndash5 IEEE Kuala Lumpur Malaysia June 2010

[46] H Foroughi A Naseri A Saberi and H S Yazdi ldquoAneigenspace-based approach for human fall detection usingintegrated time motion image and neural networkrdquo in Pro-ceedings of the 2008 9th International Conference on SignalProcessing pp 1499ndash1503 IEEE Beijing China October2008

[47] A K B Sonali ldquoHuman action recognition using supportvector machine and k-nearest neighborrdquo InternationalJournal of Engineering and Technical Research vol 3 no 4pp 423ndash428 2015

[48] V Swarnambigai ldquoAction recognition using ami and supportvector machinerdquo International Journal of Computer Scienceand Technology vol 5 no 4 pp 175ndash179 2014

[49] D Das and S Saharia ldquoHuman gait analysis and recognitionusing support vector machinesrdquo International Journal ofComputer Science and Information Technology vol 6 no 52004

[50] K G M Chathuramali and R Rodrigo ldquoFaster human activityrecognition with SVMrdquo in Proceedings of the InternationalConference on Advances in ICT for Emerging Regions(ICTer2012) pp 197ndash203 IEEE Colombo Sri Lanka De-cember 2012

[51] M Z Uddin D-H Kim J T Kim and T-S Kim ldquoAn indoorhuman activity recognition system for smart home using localbinary pattern features with hidden markov modelsrdquo Indoorand Built Environment vol 22 no 1 pp 289ndash298 2013

[52] M Z Uddin J Lee and T-S Kim ldquoIndependent shapecomponent-based human activity recognition via hiddenMarkov modelrdquo Applied Intelligence vol 33 no 2 pp 193ndash206 2010

[53] S Kumar and M Hebert ldquoDiscriminative Fields for ModelingSpatial Dependencies in Natural Imagesrdquo in Proceedings of theNIPS Vancouver Canada December 2003

[54] A Quattoni S Wang L-P Morency M Collins andT Darrell ldquoHidden conditional random fieldsrdquo IEEETransactions on Pattern Analysis and Machine Intelligencevol 29 no 10 pp 1848ndash1852 2007

[55] M Mahajan A Gunawardana and A Acero ldquoTraining al-gorithms for hidden conditional random fieldsrdquo in Pro-ceedings of the 2006 IEEE International Conference onAcoustics Speed and Signal Processing vol 1 IEEE ToulouseFrance May 2006

[56] S Reiter B Schuller and G Rigoll ldquoHidden conditionalrandom fields for meeting segmentationrdquo in Proceedings ofthe Multimedia and Expo 2007 IEEE International Confer-ence pp 639ndash642 IEEE Beijing China July 2007

[57] L Gorelick M Blank E Shechtman M Irani and R BasrildquoActions as space-time shapesrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 29 no 12 pp 2247ndash2253 2007

[58] I Laptev M Marszalek C Schmid and B RozenfeldldquoLearning realistic human actions from moviesrdquo in Pro-ceedings of the 2008 IEEE Conference on Computer Vision andPattern Recognition pp 1ndash8 IEEE Anchorage AK USAJune 2008

[59] K Soomro and A R Zamir ldquoAction recognition in realisticsports videosrdquo in Computer Vision in Sports pp 181ndash208Springer Berlin Germany 2014

[60] D Weinland E Boyer and R Ronfard ldquoAction recognitionfrom arbitrary views using 3D exemplarsrdquo in Proceedings ofthe 2007 IEEE 11th International Conference on ComputerVision pp 1ndash7 IEEE Rio de Janeiro Brazil October 2007

[61] M Siddiqi S Lee Y-K Lee A Khan and P Truc ldquoHier-archical recognition scheme for human facial expressionrecognition systemsrdquo Sensors vol 13 no 12 pp 16682ndash167132013

[62] M Elmezain and A Al-Hamadi ldquoVision-based human ac-tivity recognition using ldcrfsrdquo International Arab Journal ofInformation Technology (IAJIT) vol 15 no 3 pp 389ndash3952018

[63] A Roy B Banerjee and V Murino ldquoA novel dictionarylearning based multiple instance learning approach to actionrecognition from videosrdquo in Proceedings of the 6th In-ternational Conference on Pattern Recognition Applicationsand Methods pp 519ndash526 Porto Portugal February 2017

[64] A P Sirat ldquoActions as space-time shapesrdquo InternationalJournal of Application or Innovation in Engineering andManagement vol 7 no 8 pp 49ndash54 2018

14 Computational Intelligence and Neuroscience

Computer Games Technology

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

Advances in

FuzzySystems

Hindawiwwwhindawicom

Volume 2018

International Journal of

ReconfigurableComputing

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

thinspArtificial Intelligence

Hindawiwwwhindawicom Volumethinsp2018

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawiwwwhindawicom Volume 2018

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Engineering Mathematics

International Journal of

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Computational Intelligence and Neuroscience

Hindawiwwwhindawicom Volume 2018

Mathematical Problems in Engineering

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Hindawiwwwhindawicom Volume 2018

Human-ComputerInteraction

Advances in

Hindawiwwwhindawicom Volume 2018

Scientic Programming

Submit your manuscripts atwwwhindawicom

Page 3: Human Activity Recognition Using Gaussian Mixture Hidden …downloads.hindawi.com/journals/cin/2019/8590560.pdf · 2019-08-22 · ResearchArticle Human Activity Recognition Using

Two generalized models of MEMM known as condi-tional random fields (CRFs) [29] and HCRF [54] weredeveloped to fix the shortcoming of ldquolabel bias problemrdquo[29] For learning the hidden structure of the sequential dataHCRF facilitates the effectiveness of CRF with hidden statesHowever in both models the per-state normalization isreplaced with global normalization permitting the weightedscores which in turn result in larger parameter spaces ascompared to HMM and MEMM

For example the CRFs achieved in the HAR systemhaving the observed frames from a video are represented byfeature vectorU resultant labelV and unknown state label K

Suppose the problem image labeling is assumed byoriginal labels K with image features U and parameter of themodel is Λ then the later probability (post(K | UΛ))maximized by CRF is given as

post(K ∣ UΛ) ex(f(K UΛ))

z(UΛ) (1)

where the normalization factor is

z(UΛ) 1113944

Kprime

ex f Kprime UΛ( 1113857( 1113857 (2)

Some issues in HCRF implementation are reviewed andanalyzed in the following description e later probabilityof CRF in (1) has been updated by the post(K ∣ UΛ) in aHCRF model that is the addition of exponentials of latentfunctions with all expected labels L as given below

post(V ∣ UΛ) 1113936Lex Λ middot f(V L U)1113864 1113865

z(UΛ)

z(UΛ) 1113944

VprimeL

ex Λ middot f Vprime L U( 11138571113864 1113865

(3)

e above equations are used to warranty the sum toone rule of the conditional probability Vprime is the possibletag for the series of frames and L l1 l2 lT1113864 1113865 is a seriesof hidden states li i 1 2 T and equations (1) and (2)have constant values from 1 to Q (the number of states) Λis the vector factor and f(V L U) is a feature vector thatwill yield a decision which parameter will be educated bythe model en the feature vector concludes the additionof the existing HCRF model For example the underneathselections will create a Markov restraint HCRF with aGaussian distribution at every state

fPriorl (V L U) δ l1 l( 1113857 middot v uforalll

fTransitionllprime (V L U) 1113944

T

t1δ ltminus 1 l( 1113857δ lt lprime( 1113857 middot v uforalll lprime

fOccurencel (V L U) 1113944

T

t1δ lt l( 1113857 middot v uforalll

(4)where each v isin V is the expected tag and every u isin U is apredicted vector e per-component square of the obser-vation vector v at state t (ie vt) is given as

fM1l 1113944

T

t1δ lt l( 1113857 middot v utforalll

fM2l (V L U) 1113944

T

t1δ lt l( 1113857 middot v u

2tforalll

(5)

It can be seen that along with certain set of parameters(Λ) the HCRF addition is similar to the hidden Markovmodel for instance along with the abovementioned featurevector if we choose

Λl

Prior log(b(l)) (6)

Λl

Transition log C l lprime( 1113857( 1113857 (7)

Λl

Occurence minus

12

log 2πσ2l1113872 1113873 +μ2lσ2l

1113888 1113889 (8)

ΛM1l

μl

σ2l (9)

ΛM2l minus

12σ2l

(10)

where b in (6) is an earlier dissemination of Gaussian HMMand C in (7) is an evolution matrix then conditional pos-sibility numerator might be explained as

1113944

L

ex Λ middot f(V L U)1113864 1113865 1113944

L

b l1( 11138571113945

T

t1C ltminus 1 lt( 1113857N u

2t μLt

σLt1113872 1113873

(11)

In the above equationN represents Gaussian distributionEquation (11) is the conditional probability of U given V iscalculated along with a GaussianHMM through equation (11)which has an earlier distribution bwith a conversionmatrixC

Moreover the authors of [30] proposed a comprehensiveform of the HCRF model to tackle composite scatteringsutilizing a linear combination of Gaussian distribution func-tions which is explained as

post(V ∣ UΛ) 1113936L1113936

Mm1ex Λf(V L m U)1113864 1113865

z(UΛ) (12)

In equation (12)M indicates the number of componentsin Gaussian mixture

Lots of works have been developed which showed betterperformance based on the usage of the abovementionedHCRF [55 56] however most of them did not consider thelimitations of the model It is obvious from the aforemen-tioned equations that the existing model employed diago-nal (sloping)-covariance Gaussian distribution whichmeans that the variables (columns of ui i 1 2 N )were presumed to be couples independent On the otherhand equations (8)ndash(10) suggest that with a specific set ofvalue each state observation density will congregate toGaussian procedure Unluckily there is no training method

Computational Intelligence and Neuroscience 3

designed yet to guarantee this convergence and thosesuppositions might decrease the accuracy results

erefore we proposed the improved version of theHCRF technique that has the ability to openly employ full-covariance Gaussian mixture in the feature function eproposed model will get the benefits of hidden conditionalrandom field model that completely considered the draw-backs of the previous method

3 Proposed Methodology

31 Feature Extraction In our previous work we utilizedsymlet wavelet [37] for extracting various features from theactivity frames ere are number of reasons for using thesymlet wavelet which produces relatively better classificationresults ese include its capability to extract the conspic-uous information from the activity frames in terms of fre-quency and its support to the characteristics of the grayscaleimages like orthogonality biorthogonality and reversebiorthogonality For a certain provision size the symlet ischaracterized with the highest number of vanishing mo-ments and has the least asymmetry

32 Proposed Hidden Conditional Random Fields (HCRFs)Model As described earlier the current Gaussian mixtureHCRF model does not have the capability of utilizing full-covariance distributions and also does not guarantee theconjunction of its factors to certain values upon which theconditional probability is demonstrated as a combination ofthe normal density functions

To address these limitations we explicitly involve amixture of Gaussian distributions in the feature functions asillustrated in the following forms

fl

Prior(V L U) δ l1 l( 1113857 middot v uforalll (13)

fllprime

Transition(V L U) 1113944

T

t1δ ltminus 1 l( 1113857δ lt lprime( 1113857 middot v uforalll lprime

(14)

fl

Observation(V L U) 1113944

T

t1log 1113944

M

m1ΓObslm N u

2t μlmΣlm1113872 1113873⎛⎝ ⎞⎠

δ lt l( 1113857 middot v

(15)

then

N u2t μlmΣlm1113872 1113873

1

(2π)D2 Σlm1113868111386811138681113868

111386811138681113868111386812ex

middot minus12

u2t minus μlm1113872 1113873prime1113944

minus 1

lm

x2t minus μlm1113872 1113873⎛⎝ ⎞⎠

(16)

where N represents the number of density functionsGamma ldquoΓrdquo considers the appropriate information of theentire observations D indicates the dimension of the ob-servation and ΓObslm presents the partying weightiness forthe mth constituent along with mean μlm and covariancematrix Σlm

As indicated in equation (14) when we change some ofthe parameters such as Γ μ and Σ then we may build acombination of the standard densities e resultant con-ditional probability might be written as

post(V ∣ UΛ Γ μΣ) 1113936Lex(P(L) + T(L) + O(L))

z(UΛ Γ μΣ)

P(L) 1113944l

ΛPriorl fPriorl (V L U)

T(L) 1113944

llprime

ΛTransitionllprime fTransitionllprime (V L U)

O(L) 1113944l

fObservationl (V L U)

(17)

therefore

post(V ∣ UΛ Γ μΣ) 1113936Lexp 1113936lΛPrior

l fPriorl (V L U) + 1113936llprimeΛTransitionllprime fTransition

llprime (V L U) + 1113936lfObservationl (V L U)1113872 1113873

z(UΛ Γ μΣ) (18)

post(V ∣ UΛ Γ μΣ) 1113936Ll1 l2 lT

ex ΛPriorl1+ 1113936

Tt1 Λ

Transitionltminus 1 lt

1113872 1113873 + log 1113936Mm1Γ

Obsltm

N u2t μltm

Σltm1113872 11138731113872 11138731113872 1113873

z(UΛ Γ μΣ) (19)

post(V ∣ UΛ Γ μΣ) Score(U|VΛ Γ μΣ)

z(UΛ Γ μΣ) (20)

e forward and backward algorithms are used to cal-culate the conditional probability based on equations (19)and (20) that can be written as

4 Computational Intelligence and Neuroscience

ατ 1113944

Ll1 l2 lτl

exp ΛPriorl1+ 1113944

T

t1ΛTransitionltminus 1 lt

1113872 1113873 + log 1113944

M

m1ΓObsltm

N middot u2t μltm

Σltm1113872 1113873⎛⎝ ⎞⎠⎛⎝ ⎞⎠

1113944lprime

ατminus 1 lprime( 1113857exp ΛTransitionllprime + log 1113944M

m1ΓObsltm

N middot uτ μlmΣlm1113872 1113873⎛⎝ ⎞⎠

(21)

βτ(l) 1113944

L lτl lτ+1 lT

ex ΛPriorl1

+ 1113944T

t1ΛTransitionltminus 1lt

1113872 1113873 + log 1113944M

m1ΓObsltm

N middot u2t μltm

Σltm1113872 1113873⎛⎝ ⎞⎠

1113944lprime

βτ+1 lprime( 1113857exp ΛTransitionllprime + log 1113944M

m1ΓObslm N uτ μlmΣlm1113872 1113873⎛⎝ ⎞⎠⎛⎝ ⎞⎠

(22)

Score(V ∣ VΛ Γ μΣ) 1113944l

αT(l) 1113944l

β1(l) (23)

In the training data to maximize the conditionalprobability we initially focused on calculating the pa-rameters (Λ Γ μ andΣ) In the proposed approachlimited-memory BroydenndashFletcherndashGoldfarbndashShanno(L-BGFS) method has been implemented in order tosearch the optimum point Unlikely the other models[30] both the forward and backward algorithms are usedto compute the conditional probability and the resultswere reused for finding the gradients is makes thealgorithm more significant in reducing the computationtime

At the observation level we particularly incorporated thefull-covariance matrix in the feature function as shown in(16) Equation (17) may be used for getting the normaldistribution which is further elaborated in the followingequations

d Score(V ∣ UΛ Γ μΣ)dΛPriorl

1113944

L

dg(V L U)

dΛPriorl

ex(g(V L U))

1113944

L

fPriorl (V L U)ex(g(V L U)) β1(l)

(24)

e d Score function is a gradient function for a variableof the prior probability vectord Score(V ∣ UΛ Γ μΣ)

dΛTransitionl

1113944

L

dg(V L U)

dΛTransitionllprime

ex(g(V L U))

1113944

L

fTransitionllprime (V L U)exp(g(V L U))

(25)

e d Score function is a gradient function for a variableof the transition probability vector

d Score(V ∣ UΛ Γ μΣ)dΓObslm

1113944

L

dg(V L U)

dΓObslm

ex(g(V L U))

1113936LfObservation

l (V L U)

dΓObslm

ex(g(V L U))

1113944

L

1113944

T

t1

N ut μlmΣlm1113872 1113873

1113936Mm1Γ

Obslm N ut μlmΣlm1113872 1113873

δ lt l( 1113857ex(g(V L U))

1113944T

t1

N ut μlmΣlm1113872 1113873

1113936Mm1Γ

Obslm N ut μlmΣlm1113872 1113873

α(t l)c(t + 1)

(26)

Computational Intelligence and Neuroscience 5

e d Score function is a gradient function for aGaussian mixture weight variable Here a function V(t) canbe determined as

c(t) suml

β(t l) (27)

d Score(V ∣ UΛ Γ μΣ)dμlm

sumT

t1

ΓObslm dN ut μlmΣlm( )dμlmsumMm1ΓObslm N ut μlmΣlm( )

middot α(t l)c(t + 1)(28)

e d Score function is a gradient function for theGaussian distribution mean

d Score(V ∣ UΛ Γ μΣ)dΣlm

sumT

t1

ΓObslm dN ut μlmΣlm( )dΣlmsumMm1ΓObslm N ut μlmΣlm( )

middot α(t l)c(t + 1)(29)

e d Score function is a gradient function for the co-variance of the Gaussian distributions

Equations (24)ndash(27) presented above describe an anal-ysis method algorithm for calculating values of gradients fora feature function the mean of Gaussian distributions and

the covariance of the prior probability vector the transitionprobability vector and the observation probability vectorobtained from the existing HCRF

In our model the recognition of a variety of real-timeactivities can be divided into two steps a training step and aninference step In the rst step data with known labels areinputted for recognizing the target as well as training thehidden conditional eld model In the inference step theinputs to be actually estimated are ordered dependent onparameters determined in the training phase

If the activity frame is acting as an input in the training stepthen in the preprocessing step the applied distinctive lightingeects are decreased for detecting and extracting faces from theactivity frames At that point the movable features are extri-cated from the various facial parts for creating the featurevector After that the feature vector obtained serves as an inputto a full-covariance Gaussian-mixed hidden conditional ran-dom eld model of the suggested recognition model

As mentioned in the earlier discussion a feature gradient isgenerally determined by LBFG approach in the training phaseof the HCRF model Nonetheless in the current gradientcalculation technique a forward and backward iterative exe-cution algorithm is iteratively called upon which needs anexceptionally high computational time and thus leads to re-duction in the computational speed Another analysis ap-proach has been formulated that reduces the invoking of theforward and backward iterative execution algorithm using vegradient functions determined by equations (24)ndash(28) Using

Observed activity labels

Hidden CRFparameter

Modelweights

Hidden CRFtraining engine

Data calibrationand normalization

Unified interface

Raw data 1

2

5

7

6

3

4

4

ClusteringVectorquantization

Feature extraction

Discreteinput

sequence

Normalizeddata

Kernelvectors

Kernelvectors

Feature vectors

External interfaceInternal interface

Data files

Codebookvectors

Figure 2 Workow diagram of the proposed recognition model

6 Computational Intelligence and Neuroscience

this analysis the real-time computation can be carried out at ahigher speed resulting in an enormous decrease in the com-putational time compared to a known analysis approach eoverall workflow of the proposed model is shown in Figure 2

4 Model Validation

41 Datasets Used In this work we employed four open-source standard action datasets like Weizmann actiondatasets [57] KTH action dataset [58] UCF sports dataset[59] and IXMAS action dataset [60] for corroborating theproposed HCRF model performance All the datasets areexplained below

411 Weizmann Action Dataset is dataset consisted of10 actions such as bending running walking skippingplace jumping side movement jumping forward two handwaving and one hand waving that were performed by total9 subjects is dataset comprised of 90 video clips with

average of 15 frames per clip where the frame size is144 times180

412 KTH Action Dataset KTH dataset employed for ac-tivity recognition comprised of 25 subjects who performed 6activities like running walking boxing jogging hand-clapping and hand waving in four distinctive scenariosUsing a static camera in the homogenous background atotal of 2391 sequences were taken with a frame size of160times120

413 UCF Sports Dataset In this dataset there were 182videos which were evaluated by n-fold cross-validation ruleis dataset has been taken from different sports activities inbroadcast television channels Some of the videos had highintraclass similarities is dataset was also collected using astatic camera is dataset covers 9 activities like runningdiving lifting golf swinging skating kicking walking

Table 2 Confusion matrix of the proposed recognition model using KTH action dataset (unit )

Activities Walking Jogging Running Boxing Hand-wave HandclapWalking 100 0 0 0 0 0Jogging 0 98 1 1 0 0Running 2 1 95 1 1 0Boxing 0 2 1 97 0 0Hand-wave 1 0 1 0 98 0Handclap 0 0 1 0 0 99Average 9783

Table 1 Confusion matrix of the proposed recognition model using Weizmann action dataset (unit )

Activities Bend Jack Pjump Run Side Skip Walk Wave 1 Wave 2Bend 98 0 1 0 0 1 0 0 0Jack 1 96 0 0 2 0 1 0 0Pjump 0 1 97 0 1 0 0 1 0Run 0 0 0 99 0 0 0 1 0Side 1 2 0 0 95 0 0 2 0Skip 0 0 0 0 0 100 0 0 0Walk 1 0 0 1 0 1 96 0 1Wave 1 0 1 1 0 0 0 0 98 0Wave 2 0 1 0 0 1 1 1 1 95Average 9711

Table 3 Confusion matrix of the proposed recognition model using UCF sports dataset (unit )

Activities Diving GS Kicking Lifting HBR Run Skating BS WalkDiving 95 1 2 1 0 1 0 0 0GS 1 94 0 0 2 1 1 1 0Kicking 0 2 98 0 0 0 0 0 0Lifting 1 1 1 94 1 0 0 1 1HBR 0 0 2 0 96 1 0 1 0Running 0 3 0 0 0 97 0 0 0Skating 1 0 1 1 0 1 95 0 1BS 0 0 0 1 0 1 0 97 1Walking 0 0 0 0 0 0 0 0 100Average 9622GS golf swinging HBR horseback riding and BS baseball swinging

Computational Intelligence and Neuroscience 7

Table 4 Confusion matrix of the proposed recognition model using IXMAS action dataset (unit )

Activities CA SD GU TA Walk Wave Punch KickCA 97 0 0 1 2 0 0 0SD 0 99 1 0 0 0 0 0GU 1 2 94 3 0 0 0 0TA 0 0 1 95 2 1 1 0Walk 0 1 1 0 98 0 0 0Wave 0 0 2 0 1 97 0 0Punch 0 1 0 1 0 2 96 0Kick 0 0 0 0 1 0 0 99Average 9688CA cross arm SD sit down GU get up and TA turn around

Table 5 Classification results of the proposed system on Weizmann action dataset (A) using ANN (B) using SVM (C) using HMM and(D) using existing HCRF [30] while removing the proposed HCRF model (unit )

Activities Bend Jack Pjump Run Side Skip Walk Wave 1 Wave 2(A)Bend 70 4 5 3 3 2 5 5 3Jack 4 68 3 6 7 2 3 3 4Pjump 2 4 75 6 3 2 2 2 4Run 4 2 3 72 6 2 5 3 3Side 5 3 5 4 65 6 4 6 2Skip 4 6 4 3 5 67 3 2 6Walk 4 2 4 7 3 4 70 3 3Wave 1 2 1 3 3 4 5 7 71 4Wave 2 2 5 3 6 4 4 3 5 68Average 6955(B)Bend 69 3 4 4 6 4 2 3 5Jack 2 72 2 3 4 3 5 4 5Pjump 1 4 75 2 4 5 4 2 3Run 2 4 3 78 2 2 4 2 3Side 2 4 5 3 70 4 3 5 4Skip 2 1 3 2 4 80 3 3 2Walk 2 0 3 4 3 2 82 1 3Wave 1 2 2 3 4 3 2 3 77 4Wave 2 1 2 1 2 3 1 3 4 83Average 7622(C)Bend 82 3 0 2 2 3 1 5 2Jack 3 80 1 2 3 2 3 4 2Pjump 3 4 85 3 0 0 1 2 2Run 5 4 2 79 0 2 1 3 4Side 0 1 5 4 81 3 1 2 3Skip 3 1 2 2 3 88 0 0 1Walk 0 2 3 2 1 2 83 3 4Wave 1 1 3 2 2 4 2 3 78 5Wave 2 1 2 2 2 2 3 1 0 87Average 8256(D)Bend 80 2 3 1 4 0 5 2 3Jack 1 88 0 2 0 3 2 3 1Pjump 0 2 90 1 0 3 0 2 2Run 2 1 2 85 2 3 0 0 5Side 4 1 2 3 80 4 1 2 3Skip 1 4 0 5 1 84 0 3 2Walk 2 1 0 0 1 2 89 2 3Wave 1 3 0 1 2 0 2 0 91 1Wave 2 4 1 3 0 2 3 0 2 85Average 8578

8 Computational Intelligence and Neuroscience

horseback riding and baseball swinging Each frame has asize of 720times 480

414 IXMAS Action Dataset IXMAS (INRIA Xmas motionacquisition sequences) dataset comprised of 13 activity classeswhich were performed by 11 actors each 3 times Every actoropted a free orientation as well as position e dataset hasprovided annotated silhouettes for each person For our ex-periments we have selected only 8 action classes like walkcross arms punch turn around sit down wave get up andkick IXMAS dataset is a multiview dataset for a view-invarianthuman activity recognition where each frame has a size of390times 291 is dataset has a major occlusion and that maycause misclassification therefore we utilized global histogramequalization [61] in order to resolve the occlusion issue

42 Setup For a comprehensive validation we carried outthe following set of experiments executed using Matlab

(i) e first experiment was conducted on each datasetseparately in order to show the performance of theproposed model In this experiment we employed10-fold cross-validation rule which means that data

from 9 subjects were utilized for training data whilethe data from one subject was picked as a testingdata e procedure was reiterated for 10 timesprovided each subject data is utilized for bothtraining and testing

(ii) e second experiment was conducted in the ab-sence of the proposed recognition model on all thefour datasets that will show the importance of thedeveloped model For this purpose we used theexisting eminent classifiers like SVM ANN HMMand existing HCRF [30] as a recognition modelrather than utilizing the proposed HCRF model

(iii) e third experiment was conducted to show theperformance of the proposed approach against thestate-of-the-art methods

(iv) In the last experiment the computational com-plexity of the proposed HCRF model was comparedwith forwardbackward algorithms

5 Results and Discussion

51 First Experiment As described before this experimentvalidates the performance of the proposed recognition model

Table 6 Classification results of the proposed system on KTH action dataset (A) using ANN (B) using SVM (C) using HMM and (D)using existing HCRF [30] while removing the proposed HCRF model (unit )

Activities Walking Jogging Running Boxing Hand-wave Handclap(A)Walking 79 5 6 4 3 3Jogging 3 81 5 3 4 4Running 6 4 77 5 5 3Boxing 6 7 6 69 5 7Hand-wave 4 7 5 5 73 6Handclap 4 6 5 4 6 75Average 7566(B)Walking 82 2 3 5 4 4Jogging 3 86 2 3 2 4Running 5 3 80 4 5 3Boxing 5 3 3 79 4 6Hand-wave 1 4 3 3 89 0Handclap 3 5 2 4 3 83Average 8317(C)Walking 86 3 2 4 2 3Jogging 0 88 3 2 4 3Running 0 3 90 0 4 3Boxing 3 0 4 92 1 0Hand-wave 1 3 2 2 91 1Handclap 1 3 4 1 2 89Average 8933(D)Walking 90 3 0 3 4 0Jogging 2 88 2 3 3 2Running 4 2 92 0 0 2Boxing 1 3 2 91 3 0Hand-wave 0 1 3 2 93 1Handclap 1 3 2 4 3 87Average 9017

Computational Intelligence and Neuroscience 9

on an individual dataset e overall results are shown inTables 1 (using Weizmann dataset) 2 (using KTH dataset) 3(usingUCF sports dataset) and 4 (using IXMAS) respectively

As observed from Tables 1ndash4 the proposed recognitionmodel constantly obtained higher recognition rates on in-dividual dataset is result shows the robustness of theproposed model which means that the model not onlyshowed better performance on one dataset but also showedbetter performance across multiple spontaneous datasets

52 Second Experiment As described before the secondexperiment was conducted in the absence of the proposed

recognition model to show the importance of the proposedmodel using all the four datasets For this purpose we usedthe existing eminent classifiers like SVM ANN HMM andexisting HCRF [30] as a recognition model rather thanutilizing the proposed HCRF model

Tables 5ndash8 show that when the proposed HCRF modelwas substituted with ANN SVM HMM and existing HCRF[30] the system failed to accomplish higher recognitionrates e better performance of the proposed HCRF modelis visualized in Tables 1ndash4 which show that the proposedHCRF model effectively fix the drawbacks of HMM andexisting HCRF that has been extensively utilized for se-quential HAR

Table 7 Classification results of the proposed system onUCF sports dataset (A) using ANN (B) using SVM (C) using HMM and (D) usingexisting HCRF [30] while removing the proposed HCRF model (unit )

Activities Diving GS Kicking Lifting HBR Run Skating BS Walk(A)Diving 68 4 2 5 6 6 4 3 2GS 2 71 2 4 5 4 6 3 3Kicking 3 4 70 3 5 4 2 3 6Lifting 5 4 3 65 5 6 4 6 2HBR 3 4 6 3 66 4 5 5 4Running 3 3 5 4 6 64 6 4 5Skating 2 5 4 5 3 4 69 3 5BS 4 2 5 3 4 6 5 67 4Walking 5 4 2 3 4 3 6 3 70Average 6778(B)Diving 71 4 2 3 5 6 3 2 4GS 3 77 2 4 3 2 5 2 2Kicking 4 2 74 4 5 3 2 3 3Lifting 5 6 3 69 4 3 5 3 2HBR 2 3 3 2 80 2 4 2 2Running 2 3 2 2 5 75 6 2 3Skating 2 1 2 3 4 4 78 2 4BS 3 4 6 3 4 2 3 70 5Walking 4 1 2 4 2 3 0 3 81Average 7500(C)Diving 79 3 2 2 3 4 3 2 2GS 0 83 2 4 3 2 1 3 2Kicking 1 2 85 1 3 3 2 3 0Lifting 3 0 2 82 3 2 4 2 2HBR 0 2 2 4 80 0 5 3 4Running 1 2 1 3 4 84 2 1 2Skating 2 0 3 4 0 1 86 3 1BS 1 1 1 2 0 3 0 88 4Walking 1 2 4 2 5 2 4 3 77Average 8267(D)Diving 90 3 0 1 0 2 2 1 1GS 3 84 2 1 3 1 3 2 1Kicking 3 4 85 0 0 2 3 1 2Lifting 1 2 1 89 1 1 1 2 2HBR 0 2 1 0 91 2 3 0 1Running 2 3 1 2 3 80 4 2 3Skating 2 4 1 2 3 0 84 4 0BS 2 1 1 2 1 0 3 88 2Walking 0 2 1 1 0 1 4 0 91Average 8689GS golf swinging HBR horseback riding BS baseball swinging

10 Computational Intelligence and Neuroscience

53 ird Experiment In this experiment a comparativeanalysis was made between the state-of-the-art methods andthe proposed model All of these approaches were imple-mented by the instructions provided in their particular ar-ticles A 10-fold cross-validation rule was employed on eachdataset as explained in Section 4 e average classicationresults of the existing methods along with the proposedmethod across dierent datasets are summarized in Table 9

Table 8 Classication results of the proposed system on IXMASaction dataset (A) using ANN (B) using SVM (C) using HMMand (D) using existing HCRF [30] while removing the proposedHCRF model (unit )

Activities CA SD GU TA Walk Wave Punch Kick(A)CA 65 5 7 6 5 4 3 5SD 5 72 4 3 5 4 3 4GU 4 3 75 5 3 2 4 4TA 6 7 4 68 3 5 4 3Walk 3 4 5 4 70 6 4 4Wave 4 6 5 3 4 71 3 4Punch 3 5 5 6 7 3 67 4Kick 4 5 4 6 5 4 3 69Average 6962(B)CA 77 3 4 2 2 3 5 4SD 3 79 3 2 4 5 2 2GU 5 6 69 3 4 5 4 4TA 2 3 2 80 4 4 3 2Walk 3 5 4 2 71 5 6 4Wave 2 6 3 5 4 73 4 3Punch 1 5 3 4 1 3 81 2Kick 3 6 7 4 3 5 4 68Average 7475(C)CA 79 3 4 1 2 4 3 4SD 1 84 3 2 3 1 4 2GU 0 1 88 1 2 3 2 3TA 5 2 3 79 2 3 2 4Walk 1 0 3 1 90 2 3 0Wave 2 3 1 0 3 86 2 3Punch 1 0 2 3 0 4 89 1Kick 3 2 4 1 0 2 4 84Average 8477(D)CA 90 1 2 0 3 4 0 0SD 3 85 2 1 3 3 2 1GU 0 1 91 1 0 2 3 2TA 1 3 2 87 1 2 2 2Walk 1 0 3 1 89 2 1 3Wave 0 2 1 0 2 90 3 1Punch 1 4 2 1 2 3 84 3Kick 1 2 4 1 3 2 4 83Average 8738CA cross arm SD sit down GU get up TA turn around

Table 9 Weighted average recognition rates of the proposedmethod with the existing state-of-the-art methods (unit )

State-of-the-art works Average classicationrates

Standarddeviation

GMM 633 plusmn27SVM 675 plusmn44HMM 828 plusmn38Embedded HMM 859 plusmn19[62] 921 plusmn32[63] 843 plusmn49[18] 936 plusmn27[19] 930 plusmn16[22] 927 plusmn25[64] 801 plusmn32Proposed method 972 plusmn28

0

100

200

300

400

500

600

Exec

utio

n tim

e (s)

1 2 3 4 5 6 7 8Number of states

ForwardbackwardProposed HCRF model

(a)Ex

ecut

ion

time (

s)

1 2 3 4 5 6 7 8

ForwardbackwardProposed HCRF model

0

2

4

6

8

10

12

14

16

Number of mixtures

(b)

Figure 3 An illustration of gradient computational time (equation(30)) of the previous forward and backward algorithms and theproposed HCRF model (a)Q 1 minus 5 M 5 T 90 and (b)Q 5 M 1 minus 5 T 90

Computational Intelligence and Neuroscience 11

It is obvious from Table 9 that the proposed methodshowed a significant performance against the existing state-of-the-art methods erefore the proposed method accu-rately and robustly recognizes the human activities usingdifferent video data

54 Fourth Experiment In this experiment we have pre-sented the computational complexity that is also one of thecontributions in this paper e implementations of theprevious HCRF are available in literature which calculatethe gradients by reiterating the forward and backwardtechniques while the proposed HCRF model executes themonce only and cashes the outcomes for the later use From(21) and (22) it is clear that the forward or backwardtechnique has a complexity ofO(TQ2M) where Trepresentsthe input sequence lengthQ represents the number of statesand M indicates the number of mixtures e proposedHCRFmodel however requires a full complexity of O(TM)

to calculate gradients as can be seen from (22)ndash(29)Figure 3 shows a comparison of the execution time when

the gradients are computed by the forward (or backward)algorithm and by our proposed method e computationaltime is calculated by running Matlab R2013a with thespecification of Intelreg Pentiumreg Coretrade i7-6700 (34GHz)with a RAM capacity of 16GB

6 Conclusion

In healthcare and telemedicine the human activity rec-ognition (HAR) can be best explained by helping physi-cally disable personsrsquo scenario A paralyzed patient withhalf of the body critically attacked by paralysis is com-pletely unable to perform their daily exercises e doctorsrecommend specific activities to get better improvement intheir health So for this purpose the doctors need a humanactivity recognition (HAR) system through which they canmonitor the patientsrsquo daily routines (activities) on a reg-ular basis

e accuracy of most of the HAR systems depends uponthe recognition modules For feature extraction and selec-tion modules we used some of the existing well-knownmethods while for the recognition module we proposed theusage of HCRF model which is capable of approximating acomplex distribution using a mixture of Gaussian densityfunctions e proposed model was assessed against fourpublicly available standard action datasets From our ex-periments it is obvious that the proposed full-covarianceGaussian density function showed a significant improve-ment in accuracy than the existing state-of-the-art methodsFurthermore we also proved that such improvement issignificant from statistical point of view by showing valuele02 of the comparison Similarly the complexity analysispoints out that the proposed computational method stronglydecreases the execution time for the hidden conditionalrandom field model

e ultimate goal of this study is to deploy the proposedmodel on smartphones Currently the proposed model isusing full-covariance matrix however this might be time

consuming especially when using on smartphones Using alightweight classifier such as K-nearest neighbor (K-NN)could be one possible solution But K-NN is very muchsensitive to environmental factor (like noise) erefore infuture we will try to investigate further research to reducethe time and sustain the same recognition rate whenemploying on smartphones in real environment

Data Availability

e data used to support the findings of this study areavailable from the corresponding author upon request

Conflicts of Interest

e authors declare there are no conflicts of interest

Acknowledgments

is study was supported by the Jouf University SakakaAljouf Kingdom of Saudi Arabia under the registration no39791

References

[1] Z Chang J Cao and Y Zhang ldquoA novel image segmentationapproach for wood plate surface defect classification throughconvex optimizationrdquo Journal of Forestry Research vol 29no 6 pp 1789ndash1795 2018

[2] Z S Abdallah M M Gaber B Srinivasan andS Krishnaswamy ldquoAdaptive mobile activity recognitionsystemwith evolving data streamsrdquoNeurocomputing vol 150pp 304ndash317 2015

[3] M S Bakli M A Sakr and T H A Soliman ldquoA spatio-temporal algebra in Hadoop for moving objectsrdquo Geo-SpatialInformation Science vol 21 no 2 pp 102ndash114 2018

[4] W Zhao L Yan and Y Zhang ldquoGeometric-constrainedmulti-view image matching method based on semi-globaloptimizationrdquo Geo-Spatial Information Science vol 21 no 2pp 115ndash126 2018

[5] H Gao X Zhang J Zhao and D Li ldquoTechnology of in-telligent driving radar perception based on driving brainrdquoCAAI Transactions on Intelligence Technology vol 2 no 3pp 93ndash100 2017

[6] Y Wang X Jiang R Cao and X Wang ldquoRobust indoorhuman activity recognition using wireless signalsrdquo Sensorsvol 15 no 7 pp 17195ndash17208 2015

[7] J Wang Z Liu Y Wu and J Yuan ldquoMining actionlet ensemblefor action recognition with depth camerasrdquo in Proceedings of theIEEE Conference on Computer Vision and Pattern Recognitionpp 1290ndash1297 IEEE Providence RI USA June 2012

[8] S Karungaru M Daikoku and K Terada ldquoMulti camerasbased indoors human action recognition using fuzzy rulesrdquoJournal of Pattern Recognition Research vol 10 no 1pp 61ndash74 2015

[9] S Shukri L M Kamarudin and M H F Rahiman ldquoDevice-free localization for human activity monitoringrdquo in IntelligentVideo Surveillance IntechOpen London UK 2018

[10] L Fiore D Fehr R Bodor A Drenner G Somasundaramand N Papanikolopoulos ldquoMulti-camera human activitymonitoringrdquo Journal of Intelligent and Robotic Systemsvol 52 no 1 pp 5ndash43 2008

12 Computational Intelligence and Neuroscience

[11] C Zhang and Y Tian ldquoRGB-D camera-based daily livingactivity recognitionrdquo Journal of Computer Vision and ImageProcessing vol 2 no 4 pp 1ndash7 2012

[12] M Leo T DrsquoOrazio and P Spagnolo ldquoHuman activityrecognition for automatic visual surveillance of wide areasrdquo inProceedings of the ACM 2nd International Workshop on VideoSurveillance and Sensor Networks pp 124ndash130 ACM NewYork NY USA October 2004

[13] C-B Jin S Li and H Kim ldquoReal-time action detection invideo surveillance using sub-action descriptor with multi-CNNrdquo 2017 httpsarxivorgabs171003383

[14] W Niu J Long D Han and Y-F Wang ldquoHuman activitydetection and recognition for video surveillancerdquo in Pro-ceedings of the 2004 IEEE International Conference on Mul-timedia and Expo (ICME) (IEEE Cat No 04TH8763) vol 1pp 719ndash722 IEEE Taipei Taiwan June 2004

[15] Q Li M Zheng F Li J Wang Y-A Geng and H JiangldquoRetinal image segmentation using double-scale non-linearthresholding on vessel support regionsrdquo CAAI Transactionson Intelligence Technology vol 2 no 3 pp 109ndash115 2017

[16] L M G Fonseca L M Namikawa and E F CastejonldquoDigital image processing in remote sensingrdquo in Proceedingsof the 2009 Tutorials of the XXII Brazilian Symposium onComputer Graphics and Image Processing pp 59ndash71 IEEERio de Janeiro Brazil October 2009

[17] K Buys C Cagniart A Baksheev T De Laet J De Schutterand C Pantofaru ldquoAn adaptable system for RGB-D basedhuman body detection and pose estimationrdquo Journal of VisualCommunication and Image Representation vol 25 no 1pp 39ndash52 2014

[18] M Sharif M A Khan T Akram M Y Javed T Saba andA Rehman ldquoA framework of human detection and actionrecognition based on uniform segmentation and combinationof Euclidean distance and joint entropy-based features se-lectionrdquo EURASIP Journal on Image and Video Processingvol 2017 no 1 p 89 2017

[19] K-P Chou M Prasad D Wu et al ldquoRobust feature-basedautomated multi-view human action recognition systemrdquoIEEE Access vol 6 pp 15283ndash15296 2018

[20] M Ullah H Ullah and I M Alseadonn ldquoHuman actionrecognition in videos using stable featuresrdquo Signal and ImageProcessing An International Journal vol 8 no 6 pp 1ndash102017

[21] M Sharif M A Khan F Zahid J H Shah and T AkramldquoHuman action recognition a framework of statisticalweighted segmentation and rank correlation-based selectionrdquoPattern Analysis and Applications pp 1ndash14 2019

[22] J Zang L Wang Z Liu Q Zhang G Hua and N ZhengldquoAttention-based temporal weighted convolutional neuralnetwork for action recognitionrdquo in Proceedings of the IFIPInternational Conference on Artificial Intelligence Applicationsand Innovations pp 97ndash108 Springer Rhodes Greece May2018

[23] V-D Hoang ldquoMultiple classifier-based spatiotemporal fea-tures for living activity predictionrdquo Journal of Informationand Telecommunication vol 1 no 1 pp 100ndash112 2017

[24] S Dharmalingam and A Palanisamy ldquoVector space basedaugmented structural kinematic feature descriptor for humanactivity recognition in videosrdquo ETRI Journal vol 40 no 4pp 499ndash510 2018

[25] D Liu Y Yan M-L Shyu G Zhao and M Chen ldquoSpatio-temporal analysis for human action detection and recognitionin uncontrolled environmentsrdquo International Journal of

Multimedia Data Engineering and Management vol 6 no 1pp 1ndash18 2015

[26] C A Ronao and S-B Cho ldquoHuman activity recognition withsmartphone sensors using deep learning neural networksrdquoExpert Systems with Applications vol 59 pp 235ndash244 2016

[27] A arwat H Mahdi M Elhoseny and A E HassanienldquoRecognizing human activity in mobile crowdsensing envi-ronment using optimized k-NN algorithmrdquo Expert Systemswith Applications vol 107 pp 32ndash44 2018

[28] A Jalal Y-H Kim Y-J Kim S Kamal and D Kim ldquoRobusthuman activity recognition from depth video using spatio-temporal multi-fused featuresrdquo Pattern Recognition vol 61pp 295ndash308 2017

[29] J Lafferty A McCallum and F C Pereira ldquoConditionalrandom fields probabilistic models for segmenting and la-beling sequence datardquo in Proceedings of the 18th InternationalConference on Machine Learning (ICML-2001) pp 282ndash289Williamstown MA USA June-July 2001

[30] A Gunawardana M Mahajan A Acero and J C PlattldquoHidden conditional random fields for phone classificationrdquoin Proceedings of the INTERSPEECH vol 2 pp 1117ndash1120Citeseer Lisbon Portugal September 2005

[31] L Yang Q Song Z Wang and M Jiang ldquoParsing r-cnn forinstance-level human analysisrdquo in Proceedings of the IEEEConference on Computer Vision and Pattern Recognitionpp 364ndash373 Long Beach CA USA June 2019

[32] X Liang Y Wei L Lin et al ldquoLearning to segment human bywatching youtuberdquo IEEE Transactions on Pattern Analysisand Machine Intelligence vol 39 no 7 pp 1462ndash1468 2016

[33] M M Requena ldquoHuman segmentation with convolutionalneural networksrdquo esis University of Alicante AlicanteSpain 2018

[34] H Gao S Chen and Z Zhang ldquoParts semantic segmentationaware representation learning for person re-identificationrdquoApplied Sciences vol 9 no 6 p 1239 2019

[35] C Peng S-L Lo J Huang and A C Tsoi ldquoHuman actionsegmentation based on a streaming uniform entropy slicemethodrdquo IEEE Access vol 6 pp 16958ndash16971 2018

[36] M H Siddiqi A M Khan and S-W Lee ldquoActive contourslevel set based still human body segmentation from depthimages for video-based activity recognitionrdquo KSII Trans-actions on Internet and Information Systems (TIIS) vol 7no 11 pp 2839ndash2852 2013

[37] M Siddiqi R Ali M Rana E-K Hong E Kim and S LeeldquoVideo-based human activity recognition using multilevelwavelet decomposition and stepwise linear discriminantanalysisrdquo Sensors vol 14 no 4 pp 6370ndash6392 2014

[38] S Althloothi M H Mahoor X Zhang and R M VoylesldquoHuman activity recognition using multi-features and mul-tiple kernel learningrdquo Pattern Recognition vol 47 no 5pp 1800ndash1812 2014

[39] A Jalal S Kamal and D Kim ldquoA depth video-based humandetection and activity recognition using multi-features andembedded hidden markov models for health care monitoringsystemsrdquo International Journal of Interactive Multimedia andArtificial Intelligence vol 4 no 4 p 54 2017

[40] T Dobhal V Shitole G omas and G Navada ldquoHumanactivity recognition using binary motion image and deeplearningrdquo Procedia Computer Science vol 58 pp 178ndash1852015

[41] S Zhang Z Wei J Nie L Huang S Wang and Z Li ldquoAreview on human activity recognition using vision-basedmethodrdquo Journal of Healthcare Engineering vol 2017 ArticleID 3090343 31 pages 2017

Computational Intelligence and Neuroscience 13

[42] Y Yang B Zhang L Yang C Chen and W Yang ldquoActionrecognition using completed local binary patterns and mul-tiple-class boosting classifierrdquo in Proceedings of the 2015 3rdIAPR Asian Conference on Pattern Recognition (ACPR) IEEEpp 336ndash340 Kuala Lumpur Malaysia November 2015

[43] W Lin M T Sun R Poovandran and Z Zhang ldquoHumanactivity recognition for video surveillancerdquo in Proceedings ofthe 2008 IEEE International Symposium on Circuits andSystems pp 2737ndash2740 IEEE Seattle WA USA May 2008

[44] H Permuter J Francos and I Jermyn ldquoA study of Gaussianmixture models of color and texture features for imageclassification and segmentationrdquo Pattern Recognition vol 39no 4 pp 695ndash706 2006

[45] M Fiaz and B Ijaz ldquoVision based human activity trackingusing artificial neural networksrdquo in Proceedings of the 2010International Conference on Intelligent and Advanced Systemspp 1ndash5 IEEE Kuala Lumpur Malaysia June 2010

[46] H Foroughi A Naseri A Saberi and H S Yazdi ldquoAneigenspace-based approach for human fall detection usingintegrated time motion image and neural networkrdquo in Pro-ceedings of the 2008 9th International Conference on SignalProcessing pp 1499ndash1503 IEEE Beijing China October2008

[47] A K B Sonali ldquoHuman action recognition using supportvector machine and k-nearest neighborrdquo InternationalJournal of Engineering and Technical Research vol 3 no 4pp 423ndash428 2015

[48] V Swarnambigai ldquoAction recognition using ami and supportvector machinerdquo International Journal of Computer Scienceand Technology vol 5 no 4 pp 175ndash179 2014

[49] D Das and S Saharia ldquoHuman gait analysis and recognitionusing support vector machinesrdquo International Journal ofComputer Science and Information Technology vol 6 no 52004

[50] K G M Chathuramali and R Rodrigo ldquoFaster human activityrecognition with SVMrdquo in Proceedings of the InternationalConference on Advances in ICT for Emerging Regions(ICTer2012) pp 197ndash203 IEEE Colombo Sri Lanka De-cember 2012

[51] M Z Uddin D-H Kim J T Kim and T-S Kim ldquoAn indoorhuman activity recognition system for smart home using localbinary pattern features with hidden markov modelsrdquo Indoorand Built Environment vol 22 no 1 pp 289ndash298 2013

[52] M Z Uddin J Lee and T-S Kim ldquoIndependent shapecomponent-based human activity recognition via hiddenMarkov modelrdquo Applied Intelligence vol 33 no 2 pp 193ndash206 2010

[53] S Kumar and M Hebert ldquoDiscriminative Fields for ModelingSpatial Dependencies in Natural Imagesrdquo in Proceedings of theNIPS Vancouver Canada December 2003

[54] A Quattoni S Wang L-P Morency M Collins andT Darrell ldquoHidden conditional random fieldsrdquo IEEETransactions on Pattern Analysis and Machine Intelligencevol 29 no 10 pp 1848ndash1852 2007

[55] M Mahajan A Gunawardana and A Acero ldquoTraining al-gorithms for hidden conditional random fieldsrdquo in Pro-ceedings of the 2006 IEEE International Conference onAcoustics Speed and Signal Processing vol 1 IEEE ToulouseFrance May 2006

[56] S Reiter B Schuller and G Rigoll ldquoHidden conditionalrandom fields for meeting segmentationrdquo in Proceedings ofthe Multimedia and Expo 2007 IEEE International Confer-ence pp 639ndash642 IEEE Beijing China July 2007

[57] L Gorelick M Blank E Shechtman M Irani and R BasrildquoActions as space-time shapesrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 29 no 12 pp 2247ndash2253 2007

[58] I Laptev M Marszalek C Schmid and B RozenfeldldquoLearning realistic human actions from moviesrdquo in Pro-ceedings of the 2008 IEEE Conference on Computer Vision andPattern Recognition pp 1ndash8 IEEE Anchorage AK USAJune 2008

[59] K Soomro and A R Zamir ldquoAction recognition in realisticsports videosrdquo in Computer Vision in Sports pp 181ndash208Springer Berlin Germany 2014

[60] D Weinland E Boyer and R Ronfard ldquoAction recognitionfrom arbitrary views using 3D exemplarsrdquo in Proceedings ofthe 2007 IEEE 11th International Conference on ComputerVision pp 1ndash7 IEEE Rio de Janeiro Brazil October 2007

[61] M Siddiqi S Lee Y-K Lee A Khan and P Truc ldquoHier-archical recognition scheme for human facial expressionrecognition systemsrdquo Sensors vol 13 no 12 pp 16682ndash167132013

[62] M Elmezain and A Al-Hamadi ldquoVision-based human ac-tivity recognition using ldcrfsrdquo International Arab Journal ofInformation Technology (IAJIT) vol 15 no 3 pp 389ndash3952018

[63] A Roy B Banerjee and V Murino ldquoA novel dictionarylearning based multiple instance learning approach to actionrecognition from videosrdquo in Proceedings of the 6th In-ternational Conference on Pattern Recognition Applicationsand Methods pp 519ndash526 Porto Portugal February 2017

[64] A P Sirat ldquoActions as space-time shapesrdquo InternationalJournal of Application or Innovation in Engineering andManagement vol 7 no 8 pp 49ndash54 2018

14 Computational Intelligence and Neuroscience

Computer Games Technology

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

Advances in

FuzzySystems

Hindawiwwwhindawicom

Volume 2018

International Journal of

ReconfigurableComputing

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

thinspArtificial Intelligence

Hindawiwwwhindawicom Volumethinsp2018

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawiwwwhindawicom Volume 2018

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Engineering Mathematics

International Journal of

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Computational Intelligence and Neuroscience

Hindawiwwwhindawicom Volume 2018

Mathematical Problems in Engineering

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Hindawiwwwhindawicom Volume 2018

Human-ComputerInteraction

Advances in

Hindawiwwwhindawicom Volume 2018

Scientic Programming

Submit your manuscripts atwwwhindawicom

Page 4: Human Activity Recognition Using Gaussian Mixture Hidden …downloads.hindawi.com/journals/cin/2019/8590560.pdf · 2019-08-22 · ResearchArticle Human Activity Recognition Using

designed yet to guarantee this convergence and thosesuppositions might decrease the accuracy results

erefore we proposed the improved version of theHCRF technique that has the ability to openly employ full-covariance Gaussian mixture in the feature function eproposed model will get the benefits of hidden conditionalrandom field model that completely considered the draw-backs of the previous method

3 Proposed Methodology

31 Feature Extraction In our previous work we utilizedsymlet wavelet [37] for extracting various features from theactivity frames ere are number of reasons for using thesymlet wavelet which produces relatively better classificationresults ese include its capability to extract the conspic-uous information from the activity frames in terms of fre-quency and its support to the characteristics of the grayscaleimages like orthogonality biorthogonality and reversebiorthogonality For a certain provision size the symlet ischaracterized with the highest number of vanishing mo-ments and has the least asymmetry

32 Proposed Hidden Conditional Random Fields (HCRFs)Model As described earlier the current Gaussian mixtureHCRF model does not have the capability of utilizing full-covariance distributions and also does not guarantee theconjunction of its factors to certain values upon which theconditional probability is demonstrated as a combination ofthe normal density functions

To address these limitations we explicitly involve amixture of Gaussian distributions in the feature functions asillustrated in the following forms

fl

Prior(V L U) δ l1 l( 1113857 middot v uforalll (13)

fllprime

Transition(V L U) 1113944

T

t1δ ltminus 1 l( 1113857δ lt lprime( 1113857 middot v uforalll lprime

(14)

fl

Observation(V L U) 1113944

T

t1log 1113944

M

m1ΓObslm N u

2t μlmΣlm1113872 1113873⎛⎝ ⎞⎠

δ lt l( 1113857 middot v

(15)

then

N u2t μlmΣlm1113872 1113873

1

(2π)D2 Σlm1113868111386811138681113868

111386811138681113868111386812ex

middot minus12

u2t minus μlm1113872 1113873prime1113944

minus 1

lm

x2t minus μlm1113872 1113873⎛⎝ ⎞⎠

(16)

where N represents the number of density functionsGamma ldquoΓrdquo considers the appropriate information of theentire observations D indicates the dimension of the ob-servation and ΓObslm presents the partying weightiness forthe mth constituent along with mean μlm and covariancematrix Σlm

As indicated in equation (14) when we change some ofthe parameters such as Γ μ and Σ then we may build acombination of the standard densities e resultant con-ditional probability might be written as

post(V ∣ UΛ Γ μΣ) 1113936Lex(P(L) + T(L) + O(L))

z(UΛ Γ μΣ)

P(L) 1113944l

ΛPriorl fPriorl (V L U)

T(L) 1113944

llprime

ΛTransitionllprime fTransitionllprime (V L U)

O(L) 1113944l

fObservationl (V L U)

(17)

therefore

post(V ∣ UΛ Γ μΣ) 1113936Lexp 1113936lΛPrior

l fPriorl (V L U) + 1113936llprimeΛTransitionllprime fTransition

llprime (V L U) + 1113936lfObservationl (V L U)1113872 1113873

z(UΛ Γ μΣ) (18)

post(V ∣ UΛ Γ μΣ) 1113936Ll1 l2 lT

ex ΛPriorl1+ 1113936

Tt1 Λ

Transitionltminus 1 lt

1113872 1113873 + log 1113936Mm1Γ

Obsltm

N u2t μltm

Σltm1113872 11138731113872 11138731113872 1113873

z(UΛ Γ μΣ) (19)

post(V ∣ UΛ Γ μΣ) Score(U|VΛ Γ μΣ)

z(UΛ Γ μΣ) (20)

e forward and backward algorithms are used to cal-culate the conditional probability based on equations (19)and (20) that can be written as

4 Computational Intelligence and Neuroscience

ατ 1113944

Ll1 l2 lτl

exp ΛPriorl1+ 1113944

T

t1ΛTransitionltminus 1 lt

1113872 1113873 + log 1113944

M

m1ΓObsltm

N middot u2t μltm

Σltm1113872 1113873⎛⎝ ⎞⎠⎛⎝ ⎞⎠

1113944lprime

ατminus 1 lprime( 1113857exp ΛTransitionllprime + log 1113944M

m1ΓObsltm

N middot uτ μlmΣlm1113872 1113873⎛⎝ ⎞⎠

(21)

βτ(l) 1113944

L lτl lτ+1 lT

ex ΛPriorl1

+ 1113944T

t1ΛTransitionltminus 1lt

1113872 1113873 + log 1113944M

m1ΓObsltm

N middot u2t μltm

Σltm1113872 1113873⎛⎝ ⎞⎠

1113944lprime

βτ+1 lprime( 1113857exp ΛTransitionllprime + log 1113944M

m1ΓObslm N uτ μlmΣlm1113872 1113873⎛⎝ ⎞⎠⎛⎝ ⎞⎠

(22)

Score(V ∣ VΛ Γ μΣ) 1113944l

αT(l) 1113944l

β1(l) (23)

In the training data to maximize the conditionalprobability we initially focused on calculating the pa-rameters (Λ Γ μ andΣ) In the proposed approachlimited-memory BroydenndashFletcherndashGoldfarbndashShanno(L-BGFS) method has been implemented in order tosearch the optimum point Unlikely the other models[30] both the forward and backward algorithms are usedto compute the conditional probability and the resultswere reused for finding the gradients is makes thealgorithm more significant in reducing the computationtime

At the observation level we particularly incorporated thefull-covariance matrix in the feature function as shown in(16) Equation (17) may be used for getting the normaldistribution which is further elaborated in the followingequations

d Score(V ∣ UΛ Γ μΣ)dΛPriorl

1113944

L

dg(V L U)

dΛPriorl

ex(g(V L U))

1113944

L

fPriorl (V L U)ex(g(V L U)) β1(l)

(24)

e d Score function is a gradient function for a variableof the prior probability vectord Score(V ∣ UΛ Γ μΣ)

dΛTransitionl

1113944

L

dg(V L U)

dΛTransitionllprime

ex(g(V L U))

1113944

L

fTransitionllprime (V L U)exp(g(V L U))

(25)

e d Score function is a gradient function for a variableof the transition probability vector

d Score(V ∣ UΛ Γ μΣ)dΓObslm

1113944

L

dg(V L U)

dΓObslm

ex(g(V L U))

1113936LfObservation

l (V L U)

dΓObslm

ex(g(V L U))

1113944

L

1113944

T

t1

N ut μlmΣlm1113872 1113873

1113936Mm1Γ

Obslm N ut μlmΣlm1113872 1113873

δ lt l( 1113857ex(g(V L U))

1113944T

t1

N ut μlmΣlm1113872 1113873

1113936Mm1Γ

Obslm N ut μlmΣlm1113872 1113873

α(t l)c(t + 1)

(26)

Computational Intelligence and Neuroscience 5

e d Score function is a gradient function for aGaussian mixture weight variable Here a function V(t) canbe determined as

c(t) suml

β(t l) (27)

d Score(V ∣ UΛ Γ μΣ)dμlm

sumT

t1

ΓObslm dN ut μlmΣlm( )dμlmsumMm1ΓObslm N ut μlmΣlm( )

middot α(t l)c(t + 1)(28)

e d Score function is a gradient function for theGaussian distribution mean

d Score(V ∣ UΛ Γ μΣ)dΣlm

sumT

t1

ΓObslm dN ut μlmΣlm( )dΣlmsumMm1ΓObslm N ut μlmΣlm( )

middot α(t l)c(t + 1)(29)

e d Score function is a gradient function for the co-variance of the Gaussian distributions

Equations (24)ndash(27) presented above describe an anal-ysis method algorithm for calculating values of gradients fora feature function the mean of Gaussian distributions and

the covariance of the prior probability vector the transitionprobability vector and the observation probability vectorobtained from the existing HCRF

In our model the recognition of a variety of real-timeactivities can be divided into two steps a training step and aninference step In the rst step data with known labels areinputted for recognizing the target as well as training thehidden conditional eld model In the inference step theinputs to be actually estimated are ordered dependent onparameters determined in the training phase

If the activity frame is acting as an input in the training stepthen in the preprocessing step the applied distinctive lightingeects are decreased for detecting and extracting faces from theactivity frames At that point the movable features are extri-cated from the various facial parts for creating the featurevector After that the feature vector obtained serves as an inputto a full-covariance Gaussian-mixed hidden conditional ran-dom eld model of the suggested recognition model

As mentioned in the earlier discussion a feature gradient isgenerally determined by LBFG approach in the training phaseof the HCRF model Nonetheless in the current gradientcalculation technique a forward and backward iterative exe-cution algorithm is iteratively called upon which needs anexceptionally high computational time and thus leads to re-duction in the computational speed Another analysis ap-proach has been formulated that reduces the invoking of theforward and backward iterative execution algorithm using vegradient functions determined by equations (24)ndash(28) Using

Observed activity labels

Hidden CRFparameter

Modelweights

Hidden CRFtraining engine

Data calibrationand normalization

Unified interface

Raw data 1

2

5

7

6

3

4

4

ClusteringVectorquantization

Feature extraction

Discreteinput

sequence

Normalizeddata

Kernelvectors

Kernelvectors

Feature vectors

External interfaceInternal interface

Data files

Codebookvectors

Figure 2 Workow diagram of the proposed recognition model

6 Computational Intelligence and Neuroscience

this analysis the real-time computation can be carried out at ahigher speed resulting in an enormous decrease in the com-putational time compared to a known analysis approach eoverall workflow of the proposed model is shown in Figure 2

4 Model Validation

41 Datasets Used In this work we employed four open-source standard action datasets like Weizmann actiondatasets [57] KTH action dataset [58] UCF sports dataset[59] and IXMAS action dataset [60] for corroborating theproposed HCRF model performance All the datasets areexplained below

411 Weizmann Action Dataset is dataset consisted of10 actions such as bending running walking skippingplace jumping side movement jumping forward two handwaving and one hand waving that were performed by total9 subjects is dataset comprised of 90 video clips with

average of 15 frames per clip where the frame size is144 times180

412 KTH Action Dataset KTH dataset employed for ac-tivity recognition comprised of 25 subjects who performed 6activities like running walking boxing jogging hand-clapping and hand waving in four distinctive scenariosUsing a static camera in the homogenous background atotal of 2391 sequences were taken with a frame size of160times120

413 UCF Sports Dataset In this dataset there were 182videos which were evaluated by n-fold cross-validation ruleis dataset has been taken from different sports activities inbroadcast television channels Some of the videos had highintraclass similarities is dataset was also collected using astatic camera is dataset covers 9 activities like runningdiving lifting golf swinging skating kicking walking

Table 2 Confusion matrix of the proposed recognition model using KTH action dataset (unit )

Activities Walking Jogging Running Boxing Hand-wave HandclapWalking 100 0 0 0 0 0Jogging 0 98 1 1 0 0Running 2 1 95 1 1 0Boxing 0 2 1 97 0 0Hand-wave 1 0 1 0 98 0Handclap 0 0 1 0 0 99Average 9783

Table 1 Confusion matrix of the proposed recognition model using Weizmann action dataset (unit )

Activities Bend Jack Pjump Run Side Skip Walk Wave 1 Wave 2Bend 98 0 1 0 0 1 0 0 0Jack 1 96 0 0 2 0 1 0 0Pjump 0 1 97 0 1 0 0 1 0Run 0 0 0 99 0 0 0 1 0Side 1 2 0 0 95 0 0 2 0Skip 0 0 0 0 0 100 0 0 0Walk 1 0 0 1 0 1 96 0 1Wave 1 0 1 1 0 0 0 0 98 0Wave 2 0 1 0 0 1 1 1 1 95Average 9711

Table 3 Confusion matrix of the proposed recognition model using UCF sports dataset (unit )

Activities Diving GS Kicking Lifting HBR Run Skating BS WalkDiving 95 1 2 1 0 1 0 0 0GS 1 94 0 0 2 1 1 1 0Kicking 0 2 98 0 0 0 0 0 0Lifting 1 1 1 94 1 0 0 1 1HBR 0 0 2 0 96 1 0 1 0Running 0 3 0 0 0 97 0 0 0Skating 1 0 1 1 0 1 95 0 1BS 0 0 0 1 0 1 0 97 1Walking 0 0 0 0 0 0 0 0 100Average 9622GS golf swinging HBR horseback riding and BS baseball swinging

Computational Intelligence and Neuroscience 7

Table 4 Confusion matrix of the proposed recognition model using IXMAS action dataset (unit )

Activities CA SD GU TA Walk Wave Punch KickCA 97 0 0 1 2 0 0 0SD 0 99 1 0 0 0 0 0GU 1 2 94 3 0 0 0 0TA 0 0 1 95 2 1 1 0Walk 0 1 1 0 98 0 0 0Wave 0 0 2 0 1 97 0 0Punch 0 1 0 1 0 2 96 0Kick 0 0 0 0 1 0 0 99Average 9688CA cross arm SD sit down GU get up and TA turn around

Table 5 Classification results of the proposed system on Weizmann action dataset (A) using ANN (B) using SVM (C) using HMM and(D) using existing HCRF [30] while removing the proposed HCRF model (unit )

Activities Bend Jack Pjump Run Side Skip Walk Wave 1 Wave 2(A)Bend 70 4 5 3 3 2 5 5 3Jack 4 68 3 6 7 2 3 3 4Pjump 2 4 75 6 3 2 2 2 4Run 4 2 3 72 6 2 5 3 3Side 5 3 5 4 65 6 4 6 2Skip 4 6 4 3 5 67 3 2 6Walk 4 2 4 7 3 4 70 3 3Wave 1 2 1 3 3 4 5 7 71 4Wave 2 2 5 3 6 4 4 3 5 68Average 6955(B)Bend 69 3 4 4 6 4 2 3 5Jack 2 72 2 3 4 3 5 4 5Pjump 1 4 75 2 4 5 4 2 3Run 2 4 3 78 2 2 4 2 3Side 2 4 5 3 70 4 3 5 4Skip 2 1 3 2 4 80 3 3 2Walk 2 0 3 4 3 2 82 1 3Wave 1 2 2 3 4 3 2 3 77 4Wave 2 1 2 1 2 3 1 3 4 83Average 7622(C)Bend 82 3 0 2 2 3 1 5 2Jack 3 80 1 2 3 2 3 4 2Pjump 3 4 85 3 0 0 1 2 2Run 5 4 2 79 0 2 1 3 4Side 0 1 5 4 81 3 1 2 3Skip 3 1 2 2 3 88 0 0 1Walk 0 2 3 2 1 2 83 3 4Wave 1 1 3 2 2 4 2 3 78 5Wave 2 1 2 2 2 2 3 1 0 87Average 8256(D)Bend 80 2 3 1 4 0 5 2 3Jack 1 88 0 2 0 3 2 3 1Pjump 0 2 90 1 0 3 0 2 2Run 2 1 2 85 2 3 0 0 5Side 4 1 2 3 80 4 1 2 3Skip 1 4 0 5 1 84 0 3 2Walk 2 1 0 0 1 2 89 2 3Wave 1 3 0 1 2 0 2 0 91 1Wave 2 4 1 3 0 2 3 0 2 85Average 8578

8 Computational Intelligence and Neuroscience

horseback riding and baseball swinging Each frame has asize of 720times 480

414 IXMAS Action Dataset IXMAS (INRIA Xmas motionacquisition sequences) dataset comprised of 13 activity classeswhich were performed by 11 actors each 3 times Every actoropted a free orientation as well as position e dataset hasprovided annotated silhouettes for each person For our ex-periments we have selected only 8 action classes like walkcross arms punch turn around sit down wave get up andkick IXMAS dataset is a multiview dataset for a view-invarianthuman activity recognition where each frame has a size of390times 291 is dataset has a major occlusion and that maycause misclassification therefore we utilized global histogramequalization [61] in order to resolve the occlusion issue

42 Setup For a comprehensive validation we carried outthe following set of experiments executed using Matlab

(i) e first experiment was conducted on each datasetseparately in order to show the performance of theproposed model In this experiment we employed10-fold cross-validation rule which means that data

from 9 subjects were utilized for training data whilethe data from one subject was picked as a testingdata e procedure was reiterated for 10 timesprovided each subject data is utilized for bothtraining and testing

(ii) e second experiment was conducted in the ab-sence of the proposed recognition model on all thefour datasets that will show the importance of thedeveloped model For this purpose we used theexisting eminent classifiers like SVM ANN HMMand existing HCRF [30] as a recognition modelrather than utilizing the proposed HCRF model

(iii) e third experiment was conducted to show theperformance of the proposed approach against thestate-of-the-art methods

(iv) In the last experiment the computational com-plexity of the proposed HCRF model was comparedwith forwardbackward algorithms

5 Results and Discussion

51 First Experiment As described before this experimentvalidates the performance of the proposed recognition model

Table 6 Classification results of the proposed system on KTH action dataset (A) using ANN (B) using SVM (C) using HMM and (D)using existing HCRF [30] while removing the proposed HCRF model (unit )

Activities Walking Jogging Running Boxing Hand-wave Handclap(A)Walking 79 5 6 4 3 3Jogging 3 81 5 3 4 4Running 6 4 77 5 5 3Boxing 6 7 6 69 5 7Hand-wave 4 7 5 5 73 6Handclap 4 6 5 4 6 75Average 7566(B)Walking 82 2 3 5 4 4Jogging 3 86 2 3 2 4Running 5 3 80 4 5 3Boxing 5 3 3 79 4 6Hand-wave 1 4 3 3 89 0Handclap 3 5 2 4 3 83Average 8317(C)Walking 86 3 2 4 2 3Jogging 0 88 3 2 4 3Running 0 3 90 0 4 3Boxing 3 0 4 92 1 0Hand-wave 1 3 2 2 91 1Handclap 1 3 4 1 2 89Average 8933(D)Walking 90 3 0 3 4 0Jogging 2 88 2 3 3 2Running 4 2 92 0 0 2Boxing 1 3 2 91 3 0Hand-wave 0 1 3 2 93 1Handclap 1 3 2 4 3 87Average 9017

Computational Intelligence and Neuroscience 9

on an individual dataset e overall results are shown inTables 1 (using Weizmann dataset) 2 (using KTH dataset) 3(usingUCF sports dataset) and 4 (using IXMAS) respectively

As observed from Tables 1ndash4 the proposed recognitionmodel constantly obtained higher recognition rates on in-dividual dataset is result shows the robustness of theproposed model which means that the model not onlyshowed better performance on one dataset but also showedbetter performance across multiple spontaneous datasets

52 Second Experiment As described before the secondexperiment was conducted in the absence of the proposed

recognition model to show the importance of the proposedmodel using all the four datasets For this purpose we usedthe existing eminent classifiers like SVM ANN HMM andexisting HCRF [30] as a recognition model rather thanutilizing the proposed HCRF model

Tables 5ndash8 show that when the proposed HCRF modelwas substituted with ANN SVM HMM and existing HCRF[30] the system failed to accomplish higher recognitionrates e better performance of the proposed HCRF modelis visualized in Tables 1ndash4 which show that the proposedHCRF model effectively fix the drawbacks of HMM andexisting HCRF that has been extensively utilized for se-quential HAR

Table 7 Classification results of the proposed system onUCF sports dataset (A) using ANN (B) using SVM (C) using HMM and (D) usingexisting HCRF [30] while removing the proposed HCRF model (unit )

Activities Diving GS Kicking Lifting HBR Run Skating BS Walk(A)Diving 68 4 2 5 6 6 4 3 2GS 2 71 2 4 5 4 6 3 3Kicking 3 4 70 3 5 4 2 3 6Lifting 5 4 3 65 5 6 4 6 2HBR 3 4 6 3 66 4 5 5 4Running 3 3 5 4 6 64 6 4 5Skating 2 5 4 5 3 4 69 3 5BS 4 2 5 3 4 6 5 67 4Walking 5 4 2 3 4 3 6 3 70Average 6778(B)Diving 71 4 2 3 5 6 3 2 4GS 3 77 2 4 3 2 5 2 2Kicking 4 2 74 4 5 3 2 3 3Lifting 5 6 3 69 4 3 5 3 2HBR 2 3 3 2 80 2 4 2 2Running 2 3 2 2 5 75 6 2 3Skating 2 1 2 3 4 4 78 2 4BS 3 4 6 3 4 2 3 70 5Walking 4 1 2 4 2 3 0 3 81Average 7500(C)Diving 79 3 2 2 3 4 3 2 2GS 0 83 2 4 3 2 1 3 2Kicking 1 2 85 1 3 3 2 3 0Lifting 3 0 2 82 3 2 4 2 2HBR 0 2 2 4 80 0 5 3 4Running 1 2 1 3 4 84 2 1 2Skating 2 0 3 4 0 1 86 3 1BS 1 1 1 2 0 3 0 88 4Walking 1 2 4 2 5 2 4 3 77Average 8267(D)Diving 90 3 0 1 0 2 2 1 1GS 3 84 2 1 3 1 3 2 1Kicking 3 4 85 0 0 2 3 1 2Lifting 1 2 1 89 1 1 1 2 2HBR 0 2 1 0 91 2 3 0 1Running 2 3 1 2 3 80 4 2 3Skating 2 4 1 2 3 0 84 4 0BS 2 1 1 2 1 0 3 88 2Walking 0 2 1 1 0 1 4 0 91Average 8689GS golf swinging HBR horseback riding BS baseball swinging

10 Computational Intelligence and Neuroscience

53 ird Experiment In this experiment a comparativeanalysis was made between the state-of-the-art methods andthe proposed model All of these approaches were imple-mented by the instructions provided in their particular ar-ticles A 10-fold cross-validation rule was employed on eachdataset as explained in Section 4 e average classicationresults of the existing methods along with the proposedmethod across dierent datasets are summarized in Table 9

Table 8 Classication results of the proposed system on IXMASaction dataset (A) using ANN (B) using SVM (C) using HMMand (D) using existing HCRF [30] while removing the proposedHCRF model (unit )

Activities CA SD GU TA Walk Wave Punch Kick(A)CA 65 5 7 6 5 4 3 5SD 5 72 4 3 5 4 3 4GU 4 3 75 5 3 2 4 4TA 6 7 4 68 3 5 4 3Walk 3 4 5 4 70 6 4 4Wave 4 6 5 3 4 71 3 4Punch 3 5 5 6 7 3 67 4Kick 4 5 4 6 5 4 3 69Average 6962(B)CA 77 3 4 2 2 3 5 4SD 3 79 3 2 4 5 2 2GU 5 6 69 3 4 5 4 4TA 2 3 2 80 4 4 3 2Walk 3 5 4 2 71 5 6 4Wave 2 6 3 5 4 73 4 3Punch 1 5 3 4 1 3 81 2Kick 3 6 7 4 3 5 4 68Average 7475(C)CA 79 3 4 1 2 4 3 4SD 1 84 3 2 3 1 4 2GU 0 1 88 1 2 3 2 3TA 5 2 3 79 2 3 2 4Walk 1 0 3 1 90 2 3 0Wave 2 3 1 0 3 86 2 3Punch 1 0 2 3 0 4 89 1Kick 3 2 4 1 0 2 4 84Average 8477(D)CA 90 1 2 0 3 4 0 0SD 3 85 2 1 3 3 2 1GU 0 1 91 1 0 2 3 2TA 1 3 2 87 1 2 2 2Walk 1 0 3 1 89 2 1 3Wave 0 2 1 0 2 90 3 1Punch 1 4 2 1 2 3 84 3Kick 1 2 4 1 3 2 4 83Average 8738CA cross arm SD sit down GU get up TA turn around

Table 9 Weighted average recognition rates of the proposedmethod with the existing state-of-the-art methods (unit )

State-of-the-art works Average classicationrates

Standarddeviation

GMM 633 plusmn27SVM 675 plusmn44HMM 828 plusmn38Embedded HMM 859 plusmn19[62] 921 plusmn32[63] 843 plusmn49[18] 936 plusmn27[19] 930 plusmn16[22] 927 plusmn25[64] 801 plusmn32Proposed method 972 plusmn28

0

100

200

300

400

500

600

Exec

utio

n tim

e (s)

1 2 3 4 5 6 7 8Number of states

ForwardbackwardProposed HCRF model

(a)Ex

ecut

ion

time (

s)

1 2 3 4 5 6 7 8

ForwardbackwardProposed HCRF model

0

2

4

6

8

10

12

14

16

Number of mixtures

(b)

Figure 3 An illustration of gradient computational time (equation(30)) of the previous forward and backward algorithms and theproposed HCRF model (a)Q 1 minus 5 M 5 T 90 and (b)Q 5 M 1 minus 5 T 90

Computational Intelligence and Neuroscience 11

It is obvious from Table 9 that the proposed methodshowed a significant performance against the existing state-of-the-art methods erefore the proposed method accu-rately and robustly recognizes the human activities usingdifferent video data

54 Fourth Experiment In this experiment we have pre-sented the computational complexity that is also one of thecontributions in this paper e implementations of theprevious HCRF are available in literature which calculatethe gradients by reiterating the forward and backwardtechniques while the proposed HCRF model executes themonce only and cashes the outcomes for the later use From(21) and (22) it is clear that the forward or backwardtechnique has a complexity ofO(TQ2M) where Trepresentsthe input sequence lengthQ represents the number of statesand M indicates the number of mixtures e proposedHCRFmodel however requires a full complexity of O(TM)

to calculate gradients as can be seen from (22)ndash(29)Figure 3 shows a comparison of the execution time when

the gradients are computed by the forward (or backward)algorithm and by our proposed method e computationaltime is calculated by running Matlab R2013a with thespecification of Intelreg Pentiumreg Coretrade i7-6700 (34GHz)with a RAM capacity of 16GB

6 Conclusion

In healthcare and telemedicine the human activity rec-ognition (HAR) can be best explained by helping physi-cally disable personsrsquo scenario A paralyzed patient withhalf of the body critically attacked by paralysis is com-pletely unable to perform their daily exercises e doctorsrecommend specific activities to get better improvement intheir health So for this purpose the doctors need a humanactivity recognition (HAR) system through which they canmonitor the patientsrsquo daily routines (activities) on a reg-ular basis

e accuracy of most of the HAR systems depends uponthe recognition modules For feature extraction and selec-tion modules we used some of the existing well-knownmethods while for the recognition module we proposed theusage of HCRF model which is capable of approximating acomplex distribution using a mixture of Gaussian densityfunctions e proposed model was assessed against fourpublicly available standard action datasets From our ex-periments it is obvious that the proposed full-covarianceGaussian density function showed a significant improve-ment in accuracy than the existing state-of-the-art methodsFurthermore we also proved that such improvement issignificant from statistical point of view by showing valuele02 of the comparison Similarly the complexity analysispoints out that the proposed computational method stronglydecreases the execution time for the hidden conditionalrandom field model

e ultimate goal of this study is to deploy the proposedmodel on smartphones Currently the proposed model isusing full-covariance matrix however this might be time

consuming especially when using on smartphones Using alightweight classifier such as K-nearest neighbor (K-NN)could be one possible solution But K-NN is very muchsensitive to environmental factor (like noise) erefore infuture we will try to investigate further research to reducethe time and sustain the same recognition rate whenemploying on smartphones in real environment

Data Availability

e data used to support the findings of this study areavailable from the corresponding author upon request

Conflicts of Interest

e authors declare there are no conflicts of interest

Acknowledgments

is study was supported by the Jouf University SakakaAljouf Kingdom of Saudi Arabia under the registration no39791

References

[1] Z Chang J Cao and Y Zhang ldquoA novel image segmentationapproach for wood plate surface defect classification throughconvex optimizationrdquo Journal of Forestry Research vol 29no 6 pp 1789ndash1795 2018

[2] Z S Abdallah M M Gaber B Srinivasan andS Krishnaswamy ldquoAdaptive mobile activity recognitionsystemwith evolving data streamsrdquoNeurocomputing vol 150pp 304ndash317 2015

[3] M S Bakli M A Sakr and T H A Soliman ldquoA spatio-temporal algebra in Hadoop for moving objectsrdquo Geo-SpatialInformation Science vol 21 no 2 pp 102ndash114 2018

[4] W Zhao L Yan and Y Zhang ldquoGeometric-constrainedmulti-view image matching method based on semi-globaloptimizationrdquo Geo-Spatial Information Science vol 21 no 2pp 115ndash126 2018

[5] H Gao X Zhang J Zhao and D Li ldquoTechnology of in-telligent driving radar perception based on driving brainrdquoCAAI Transactions on Intelligence Technology vol 2 no 3pp 93ndash100 2017

[6] Y Wang X Jiang R Cao and X Wang ldquoRobust indoorhuman activity recognition using wireless signalsrdquo Sensorsvol 15 no 7 pp 17195ndash17208 2015

[7] J Wang Z Liu Y Wu and J Yuan ldquoMining actionlet ensemblefor action recognition with depth camerasrdquo in Proceedings of theIEEE Conference on Computer Vision and Pattern Recognitionpp 1290ndash1297 IEEE Providence RI USA June 2012

[8] S Karungaru M Daikoku and K Terada ldquoMulti camerasbased indoors human action recognition using fuzzy rulesrdquoJournal of Pattern Recognition Research vol 10 no 1pp 61ndash74 2015

[9] S Shukri L M Kamarudin and M H F Rahiman ldquoDevice-free localization for human activity monitoringrdquo in IntelligentVideo Surveillance IntechOpen London UK 2018

[10] L Fiore D Fehr R Bodor A Drenner G Somasundaramand N Papanikolopoulos ldquoMulti-camera human activitymonitoringrdquo Journal of Intelligent and Robotic Systemsvol 52 no 1 pp 5ndash43 2008

12 Computational Intelligence and Neuroscience

[11] C Zhang and Y Tian ldquoRGB-D camera-based daily livingactivity recognitionrdquo Journal of Computer Vision and ImageProcessing vol 2 no 4 pp 1ndash7 2012

[12] M Leo T DrsquoOrazio and P Spagnolo ldquoHuman activityrecognition for automatic visual surveillance of wide areasrdquo inProceedings of the ACM 2nd International Workshop on VideoSurveillance and Sensor Networks pp 124ndash130 ACM NewYork NY USA October 2004

[13] C-B Jin S Li and H Kim ldquoReal-time action detection invideo surveillance using sub-action descriptor with multi-CNNrdquo 2017 httpsarxivorgabs171003383

[14] W Niu J Long D Han and Y-F Wang ldquoHuman activitydetection and recognition for video surveillancerdquo in Pro-ceedings of the 2004 IEEE International Conference on Mul-timedia and Expo (ICME) (IEEE Cat No 04TH8763) vol 1pp 719ndash722 IEEE Taipei Taiwan June 2004

[15] Q Li M Zheng F Li J Wang Y-A Geng and H JiangldquoRetinal image segmentation using double-scale non-linearthresholding on vessel support regionsrdquo CAAI Transactionson Intelligence Technology vol 2 no 3 pp 109ndash115 2017

[16] L M G Fonseca L M Namikawa and E F CastejonldquoDigital image processing in remote sensingrdquo in Proceedingsof the 2009 Tutorials of the XXII Brazilian Symposium onComputer Graphics and Image Processing pp 59ndash71 IEEERio de Janeiro Brazil October 2009

[17] K Buys C Cagniart A Baksheev T De Laet J De Schutterand C Pantofaru ldquoAn adaptable system for RGB-D basedhuman body detection and pose estimationrdquo Journal of VisualCommunication and Image Representation vol 25 no 1pp 39ndash52 2014

[18] M Sharif M A Khan T Akram M Y Javed T Saba andA Rehman ldquoA framework of human detection and actionrecognition based on uniform segmentation and combinationof Euclidean distance and joint entropy-based features se-lectionrdquo EURASIP Journal on Image and Video Processingvol 2017 no 1 p 89 2017

[19] K-P Chou M Prasad D Wu et al ldquoRobust feature-basedautomated multi-view human action recognition systemrdquoIEEE Access vol 6 pp 15283ndash15296 2018

[20] M Ullah H Ullah and I M Alseadonn ldquoHuman actionrecognition in videos using stable featuresrdquo Signal and ImageProcessing An International Journal vol 8 no 6 pp 1ndash102017

[21] M Sharif M A Khan F Zahid J H Shah and T AkramldquoHuman action recognition a framework of statisticalweighted segmentation and rank correlation-based selectionrdquoPattern Analysis and Applications pp 1ndash14 2019

[22] J Zang L Wang Z Liu Q Zhang G Hua and N ZhengldquoAttention-based temporal weighted convolutional neuralnetwork for action recognitionrdquo in Proceedings of the IFIPInternational Conference on Artificial Intelligence Applicationsand Innovations pp 97ndash108 Springer Rhodes Greece May2018

[23] V-D Hoang ldquoMultiple classifier-based spatiotemporal fea-tures for living activity predictionrdquo Journal of Informationand Telecommunication vol 1 no 1 pp 100ndash112 2017

[24] S Dharmalingam and A Palanisamy ldquoVector space basedaugmented structural kinematic feature descriptor for humanactivity recognition in videosrdquo ETRI Journal vol 40 no 4pp 499ndash510 2018

[25] D Liu Y Yan M-L Shyu G Zhao and M Chen ldquoSpatio-temporal analysis for human action detection and recognitionin uncontrolled environmentsrdquo International Journal of

Multimedia Data Engineering and Management vol 6 no 1pp 1ndash18 2015

[26] C A Ronao and S-B Cho ldquoHuman activity recognition withsmartphone sensors using deep learning neural networksrdquoExpert Systems with Applications vol 59 pp 235ndash244 2016

[27] A arwat H Mahdi M Elhoseny and A E HassanienldquoRecognizing human activity in mobile crowdsensing envi-ronment using optimized k-NN algorithmrdquo Expert Systemswith Applications vol 107 pp 32ndash44 2018

[28] A Jalal Y-H Kim Y-J Kim S Kamal and D Kim ldquoRobusthuman activity recognition from depth video using spatio-temporal multi-fused featuresrdquo Pattern Recognition vol 61pp 295ndash308 2017

[29] J Lafferty A McCallum and F C Pereira ldquoConditionalrandom fields probabilistic models for segmenting and la-beling sequence datardquo in Proceedings of the 18th InternationalConference on Machine Learning (ICML-2001) pp 282ndash289Williamstown MA USA June-July 2001

[30] A Gunawardana M Mahajan A Acero and J C PlattldquoHidden conditional random fields for phone classificationrdquoin Proceedings of the INTERSPEECH vol 2 pp 1117ndash1120Citeseer Lisbon Portugal September 2005

[31] L Yang Q Song Z Wang and M Jiang ldquoParsing r-cnn forinstance-level human analysisrdquo in Proceedings of the IEEEConference on Computer Vision and Pattern Recognitionpp 364ndash373 Long Beach CA USA June 2019

[32] X Liang Y Wei L Lin et al ldquoLearning to segment human bywatching youtuberdquo IEEE Transactions on Pattern Analysisand Machine Intelligence vol 39 no 7 pp 1462ndash1468 2016

[33] M M Requena ldquoHuman segmentation with convolutionalneural networksrdquo esis University of Alicante AlicanteSpain 2018

[34] H Gao S Chen and Z Zhang ldquoParts semantic segmentationaware representation learning for person re-identificationrdquoApplied Sciences vol 9 no 6 p 1239 2019

[35] C Peng S-L Lo J Huang and A C Tsoi ldquoHuman actionsegmentation based on a streaming uniform entropy slicemethodrdquo IEEE Access vol 6 pp 16958ndash16971 2018

[36] M H Siddiqi A M Khan and S-W Lee ldquoActive contourslevel set based still human body segmentation from depthimages for video-based activity recognitionrdquo KSII Trans-actions on Internet and Information Systems (TIIS) vol 7no 11 pp 2839ndash2852 2013

[37] M Siddiqi R Ali M Rana E-K Hong E Kim and S LeeldquoVideo-based human activity recognition using multilevelwavelet decomposition and stepwise linear discriminantanalysisrdquo Sensors vol 14 no 4 pp 6370ndash6392 2014

[38] S Althloothi M H Mahoor X Zhang and R M VoylesldquoHuman activity recognition using multi-features and mul-tiple kernel learningrdquo Pattern Recognition vol 47 no 5pp 1800ndash1812 2014

[39] A Jalal S Kamal and D Kim ldquoA depth video-based humandetection and activity recognition using multi-features andembedded hidden markov models for health care monitoringsystemsrdquo International Journal of Interactive Multimedia andArtificial Intelligence vol 4 no 4 p 54 2017

[40] T Dobhal V Shitole G omas and G Navada ldquoHumanactivity recognition using binary motion image and deeplearningrdquo Procedia Computer Science vol 58 pp 178ndash1852015

[41] S Zhang Z Wei J Nie L Huang S Wang and Z Li ldquoAreview on human activity recognition using vision-basedmethodrdquo Journal of Healthcare Engineering vol 2017 ArticleID 3090343 31 pages 2017

Computational Intelligence and Neuroscience 13

[42] Y Yang B Zhang L Yang C Chen and W Yang ldquoActionrecognition using completed local binary patterns and mul-tiple-class boosting classifierrdquo in Proceedings of the 2015 3rdIAPR Asian Conference on Pattern Recognition (ACPR) IEEEpp 336ndash340 Kuala Lumpur Malaysia November 2015

[43] W Lin M T Sun R Poovandran and Z Zhang ldquoHumanactivity recognition for video surveillancerdquo in Proceedings ofthe 2008 IEEE International Symposium on Circuits andSystems pp 2737ndash2740 IEEE Seattle WA USA May 2008

[44] H Permuter J Francos and I Jermyn ldquoA study of Gaussianmixture models of color and texture features for imageclassification and segmentationrdquo Pattern Recognition vol 39no 4 pp 695ndash706 2006

[45] M Fiaz and B Ijaz ldquoVision based human activity trackingusing artificial neural networksrdquo in Proceedings of the 2010International Conference on Intelligent and Advanced Systemspp 1ndash5 IEEE Kuala Lumpur Malaysia June 2010

[46] H Foroughi A Naseri A Saberi and H S Yazdi ldquoAneigenspace-based approach for human fall detection usingintegrated time motion image and neural networkrdquo in Pro-ceedings of the 2008 9th International Conference on SignalProcessing pp 1499ndash1503 IEEE Beijing China October2008

[47] A K B Sonali ldquoHuman action recognition using supportvector machine and k-nearest neighborrdquo InternationalJournal of Engineering and Technical Research vol 3 no 4pp 423ndash428 2015

[48] V Swarnambigai ldquoAction recognition using ami and supportvector machinerdquo International Journal of Computer Scienceand Technology vol 5 no 4 pp 175ndash179 2014

[49] D Das and S Saharia ldquoHuman gait analysis and recognitionusing support vector machinesrdquo International Journal ofComputer Science and Information Technology vol 6 no 52004

[50] K G M Chathuramali and R Rodrigo ldquoFaster human activityrecognition with SVMrdquo in Proceedings of the InternationalConference on Advances in ICT for Emerging Regions(ICTer2012) pp 197ndash203 IEEE Colombo Sri Lanka De-cember 2012

[51] M Z Uddin D-H Kim J T Kim and T-S Kim ldquoAn indoorhuman activity recognition system for smart home using localbinary pattern features with hidden markov modelsrdquo Indoorand Built Environment vol 22 no 1 pp 289ndash298 2013

[52] M Z Uddin J Lee and T-S Kim ldquoIndependent shapecomponent-based human activity recognition via hiddenMarkov modelrdquo Applied Intelligence vol 33 no 2 pp 193ndash206 2010

[53] S Kumar and M Hebert ldquoDiscriminative Fields for ModelingSpatial Dependencies in Natural Imagesrdquo in Proceedings of theNIPS Vancouver Canada December 2003

[54] A Quattoni S Wang L-P Morency M Collins andT Darrell ldquoHidden conditional random fieldsrdquo IEEETransactions on Pattern Analysis and Machine Intelligencevol 29 no 10 pp 1848ndash1852 2007

[55] M Mahajan A Gunawardana and A Acero ldquoTraining al-gorithms for hidden conditional random fieldsrdquo in Pro-ceedings of the 2006 IEEE International Conference onAcoustics Speed and Signal Processing vol 1 IEEE ToulouseFrance May 2006

[56] S Reiter B Schuller and G Rigoll ldquoHidden conditionalrandom fields for meeting segmentationrdquo in Proceedings ofthe Multimedia and Expo 2007 IEEE International Confer-ence pp 639ndash642 IEEE Beijing China July 2007

[57] L Gorelick M Blank E Shechtman M Irani and R BasrildquoActions as space-time shapesrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 29 no 12 pp 2247ndash2253 2007

[58] I Laptev M Marszalek C Schmid and B RozenfeldldquoLearning realistic human actions from moviesrdquo in Pro-ceedings of the 2008 IEEE Conference on Computer Vision andPattern Recognition pp 1ndash8 IEEE Anchorage AK USAJune 2008

[59] K Soomro and A R Zamir ldquoAction recognition in realisticsports videosrdquo in Computer Vision in Sports pp 181ndash208Springer Berlin Germany 2014

[60] D Weinland E Boyer and R Ronfard ldquoAction recognitionfrom arbitrary views using 3D exemplarsrdquo in Proceedings ofthe 2007 IEEE 11th International Conference on ComputerVision pp 1ndash7 IEEE Rio de Janeiro Brazil October 2007

[61] M Siddiqi S Lee Y-K Lee A Khan and P Truc ldquoHier-archical recognition scheme for human facial expressionrecognition systemsrdquo Sensors vol 13 no 12 pp 16682ndash167132013

[62] M Elmezain and A Al-Hamadi ldquoVision-based human ac-tivity recognition using ldcrfsrdquo International Arab Journal ofInformation Technology (IAJIT) vol 15 no 3 pp 389ndash3952018

[63] A Roy B Banerjee and V Murino ldquoA novel dictionarylearning based multiple instance learning approach to actionrecognition from videosrdquo in Proceedings of the 6th In-ternational Conference on Pattern Recognition Applicationsand Methods pp 519ndash526 Porto Portugal February 2017

[64] A P Sirat ldquoActions as space-time shapesrdquo InternationalJournal of Application or Innovation in Engineering andManagement vol 7 no 8 pp 49ndash54 2018

14 Computational Intelligence and Neuroscience

Computer Games Technology

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

Advances in

FuzzySystems

Hindawiwwwhindawicom

Volume 2018

International Journal of

ReconfigurableComputing

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

thinspArtificial Intelligence

Hindawiwwwhindawicom Volumethinsp2018

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawiwwwhindawicom Volume 2018

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Engineering Mathematics

International Journal of

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Computational Intelligence and Neuroscience

Hindawiwwwhindawicom Volume 2018

Mathematical Problems in Engineering

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Hindawiwwwhindawicom Volume 2018

Human-ComputerInteraction

Advances in

Hindawiwwwhindawicom Volume 2018

Scientic Programming

Submit your manuscripts atwwwhindawicom

Page 5: Human Activity Recognition Using Gaussian Mixture Hidden …downloads.hindawi.com/journals/cin/2019/8590560.pdf · 2019-08-22 · ResearchArticle Human Activity Recognition Using

ατ 1113944

Ll1 l2 lτl

exp ΛPriorl1+ 1113944

T

t1ΛTransitionltminus 1 lt

1113872 1113873 + log 1113944

M

m1ΓObsltm

N middot u2t μltm

Σltm1113872 1113873⎛⎝ ⎞⎠⎛⎝ ⎞⎠

1113944lprime

ατminus 1 lprime( 1113857exp ΛTransitionllprime + log 1113944M

m1ΓObsltm

N middot uτ μlmΣlm1113872 1113873⎛⎝ ⎞⎠

(21)

βτ(l) 1113944

L lτl lτ+1 lT

ex ΛPriorl1

+ 1113944T

t1ΛTransitionltminus 1lt

1113872 1113873 + log 1113944M

m1ΓObsltm

N middot u2t μltm

Σltm1113872 1113873⎛⎝ ⎞⎠

1113944lprime

βτ+1 lprime( 1113857exp ΛTransitionllprime + log 1113944M

m1ΓObslm N uτ μlmΣlm1113872 1113873⎛⎝ ⎞⎠⎛⎝ ⎞⎠

(22)

Score(V ∣ VΛ Γ μΣ) 1113944l

αT(l) 1113944l

β1(l) (23)

In the training data to maximize the conditionalprobability we initially focused on calculating the pa-rameters (Λ Γ μ andΣ) In the proposed approachlimited-memory BroydenndashFletcherndashGoldfarbndashShanno(L-BGFS) method has been implemented in order tosearch the optimum point Unlikely the other models[30] both the forward and backward algorithms are usedto compute the conditional probability and the resultswere reused for finding the gradients is makes thealgorithm more significant in reducing the computationtime

At the observation level we particularly incorporated thefull-covariance matrix in the feature function as shown in(16) Equation (17) may be used for getting the normaldistribution which is further elaborated in the followingequations

d Score(V ∣ UΛ Γ μΣ)dΛPriorl

1113944

L

dg(V L U)

dΛPriorl

ex(g(V L U))

1113944

L

fPriorl (V L U)ex(g(V L U)) β1(l)

(24)

e d Score function is a gradient function for a variableof the prior probability vectord Score(V ∣ UΛ Γ μΣ)

dΛTransitionl

1113944

L

dg(V L U)

dΛTransitionllprime

ex(g(V L U))

1113944

L

fTransitionllprime (V L U)exp(g(V L U))

(25)

e d Score function is a gradient function for a variableof the transition probability vector

d Score(V ∣ UΛ Γ μΣ)dΓObslm

1113944

L

dg(V L U)

dΓObslm

ex(g(V L U))

1113936LfObservation

l (V L U)

dΓObslm

ex(g(V L U))

1113944

L

1113944

T

t1

N ut μlmΣlm1113872 1113873

1113936Mm1Γ

Obslm N ut μlmΣlm1113872 1113873

δ lt l( 1113857ex(g(V L U))

1113944T

t1

N ut μlmΣlm1113872 1113873

1113936Mm1Γ

Obslm N ut μlmΣlm1113872 1113873

α(t l)c(t + 1)

(26)

Computational Intelligence and Neuroscience 5

e d Score function is a gradient function for aGaussian mixture weight variable Here a function V(t) canbe determined as

c(t) suml

β(t l) (27)

d Score(V ∣ UΛ Γ μΣ)dμlm

sumT

t1

ΓObslm dN ut μlmΣlm( )dμlmsumMm1ΓObslm N ut μlmΣlm( )

middot α(t l)c(t + 1)(28)

e d Score function is a gradient function for theGaussian distribution mean

d Score(V ∣ UΛ Γ μΣ)dΣlm

sumT

t1

ΓObslm dN ut μlmΣlm( )dΣlmsumMm1ΓObslm N ut μlmΣlm( )

middot α(t l)c(t + 1)(29)

e d Score function is a gradient function for the co-variance of the Gaussian distributions

Equations (24)ndash(27) presented above describe an anal-ysis method algorithm for calculating values of gradients fora feature function the mean of Gaussian distributions and

the covariance of the prior probability vector the transitionprobability vector and the observation probability vectorobtained from the existing HCRF

In our model the recognition of a variety of real-timeactivities can be divided into two steps a training step and aninference step In the rst step data with known labels areinputted for recognizing the target as well as training thehidden conditional eld model In the inference step theinputs to be actually estimated are ordered dependent onparameters determined in the training phase

If the activity frame is acting as an input in the training stepthen in the preprocessing step the applied distinctive lightingeects are decreased for detecting and extracting faces from theactivity frames At that point the movable features are extri-cated from the various facial parts for creating the featurevector After that the feature vector obtained serves as an inputto a full-covariance Gaussian-mixed hidden conditional ran-dom eld model of the suggested recognition model

As mentioned in the earlier discussion a feature gradient isgenerally determined by LBFG approach in the training phaseof the HCRF model Nonetheless in the current gradientcalculation technique a forward and backward iterative exe-cution algorithm is iteratively called upon which needs anexceptionally high computational time and thus leads to re-duction in the computational speed Another analysis ap-proach has been formulated that reduces the invoking of theforward and backward iterative execution algorithm using vegradient functions determined by equations (24)ndash(28) Using

Observed activity labels

Hidden CRFparameter

Modelweights

Hidden CRFtraining engine

Data calibrationand normalization

Unified interface

Raw data 1

2

5

7

6

3

4

4

ClusteringVectorquantization

Feature extraction

Discreteinput

sequence

Normalizeddata

Kernelvectors

Kernelvectors

Feature vectors

External interfaceInternal interface

Data files

Codebookvectors

Figure 2 Workow diagram of the proposed recognition model

6 Computational Intelligence and Neuroscience

this analysis the real-time computation can be carried out at ahigher speed resulting in an enormous decrease in the com-putational time compared to a known analysis approach eoverall workflow of the proposed model is shown in Figure 2

4 Model Validation

41 Datasets Used In this work we employed four open-source standard action datasets like Weizmann actiondatasets [57] KTH action dataset [58] UCF sports dataset[59] and IXMAS action dataset [60] for corroborating theproposed HCRF model performance All the datasets areexplained below

411 Weizmann Action Dataset is dataset consisted of10 actions such as bending running walking skippingplace jumping side movement jumping forward two handwaving and one hand waving that were performed by total9 subjects is dataset comprised of 90 video clips with

average of 15 frames per clip where the frame size is144 times180

412 KTH Action Dataset KTH dataset employed for ac-tivity recognition comprised of 25 subjects who performed 6activities like running walking boxing jogging hand-clapping and hand waving in four distinctive scenariosUsing a static camera in the homogenous background atotal of 2391 sequences were taken with a frame size of160times120

413 UCF Sports Dataset In this dataset there were 182videos which were evaluated by n-fold cross-validation ruleis dataset has been taken from different sports activities inbroadcast television channels Some of the videos had highintraclass similarities is dataset was also collected using astatic camera is dataset covers 9 activities like runningdiving lifting golf swinging skating kicking walking

Table 2 Confusion matrix of the proposed recognition model using KTH action dataset (unit )

Activities Walking Jogging Running Boxing Hand-wave HandclapWalking 100 0 0 0 0 0Jogging 0 98 1 1 0 0Running 2 1 95 1 1 0Boxing 0 2 1 97 0 0Hand-wave 1 0 1 0 98 0Handclap 0 0 1 0 0 99Average 9783

Table 1 Confusion matrix of the proposed recognition model using Weizmann action dataset (unit )

Activities Bend Jack Pjump Run Side Skip Walk Wave 1 Wave 2Bend 98 0 1 0 0 1 0 0 0Jack 1 96 0 0 2 0 1 0 0Pjump 0 1 97 0 1 0 0 1 0Run 0 0 0 99 0 0 0 1 0Side 1 2 0 0 95 0 0 2 0Skip 0 0 0 0 0 100 0 0 0Walk 1 0 0 1 0 1 96 0 1Wave 1 0 1 1 0 0 0 0 98 0Wave 2 0 1 0 0 1 1 1 1 95Average 9711

Table 3 Confusion matrix of the proposed recognition model using UCF sports dataset (unit )

Activities Diving GS Kicking Lifting HBR Run Skating BS WalkDiving 95 1 2 1 0 1 0 0 0GS 1 94 0 0 2 1 1 1 0Kicking 0 2 98 0 0 0 0 0 0Lifting 1 1 1 94 1 0 0 1 1HBR 0 0 2 0 96 1 0 1 0Running 0 3 0 0 0 97 0 0 0Skating 1 0 1 1 0 1 95 0 1BS 0 0 0 1 0 1 0 97 1Walking 0 0 0 0 0 0 0 0 100Average 9622GS golf swinging HBR horseback riding and BS baseball swinging

Computational Intelligence and Neuroscience 7

Table 4 Confusion matrix of the proposed recognition model using IXMAS action dataset (unit )

Activities CA SD GU TA Walk Wave Punch KickCA 97 0 0 1 2 0 0 0SD 0 99 1 0 0 0 0 0GU 1 2 94 3 0 0 0 0TA 0 0 1 95 2 1 1 0Walk 0 1 1 0 98 0 0 0Wave 0 0 2 0 1 97 0 0Punch 0 1 0 1 0 2 96 0Kick 0 0 0 0 1 0 0 99Average 9688CA cross arm SD sit down GU get up and TA turn around

Table 5 Classification results of the proposed system on Weizmann action dataset (A) using ANN (B) using SVM (C) using HMM and(D) using existing HCRF [30] while removing the proposed HCRF model (unit )

Activities Bend Jack Pjump Run Side Skip Walk Wave 1 Wave 2(A)Bend 70 4 5 3 3 2 5 5 3Jack 4 68 3 6 7 2 3 3 4Pjump 2 4 75 6 3 2 2 2 4Run 4 2 3 72 6 2 5 3 3Side 5 3 5 4 65 6 4 6 2Skip 4 6 4 3 5 67 3 2 6Walk 4 2 4 7 3 4 70 3 3Wave 1 2 1 3 3 4 5 7 71 4Wave 2 2 5 3 6 4 4 3 5 68Average 6955(B)Bend 69 3 4 4 6 4 2 3 5Jack 2 72 2 3 4 3 5 4 5Pjump 1 4 75 2 4 5 4 2 3Run 2 4 3 78 2 2 4 2 3Side 2 4 5 3 70 4 3 5 4Skip 2 1 3 2 4 80 3 3 2Walk 2 0 3 4 3 2 82 1 3Wave 1 2 2 3 4 3 2 3 77 4Wave 2 1 2 1 2 3 1 3 4 83Average 7622(C)Bend 82 3 0 2 2 3 1 5 2Jack 3 80 1 2 3 2 3 4 2Pjump 3 4 85 3 0 0 1 2 2Run 5 4 2 79 0 2 1 3 4Side 0 1 5 4 81 3 1 2 3Skip 3 1 2 2 3 88 0 0 1Walk 0 2 3 2 1 2 83 3 4Wave 1 1 3 2 2 4 2 3 78 5Wave 2 1 2 2 2 2 3 1 0 87Average 8256(D)Bend 80 2 3 1 4 0 5 2 3Jack 1 88 0 2 0 3 2 3 1Pjump 0 2 90 1 0 3 0 2 2Run 2 1 2 85 2 3 0 0 5Side 4 1 2 3 80 4 1 2 3Skip 1 4 0 5 1 84 0 3 2Walk 2 1 0 0 1 2 89 2 3Wave 1 3 0 1 2 0 2 0 91 1Wave 2 4 1 3 0 2 3 0 2 85Average 8578

8 Computational Intelligence and Neuroscience

horseback riding and baseball swinging Each frame has asize of 720times 480

414 IXMAS Action Dataset IXMAS (INRIA Xmas motionacquisition sequences) dataset comprised of 13 activity classeswhich were performed by 11 actors each 3 times Every actoropted a free orientation as well as position e dataset hasprovided annotated silhouettes for each person For our ex-periments we have selected only 8 action classes like walkcross arms punch turn around sit down wave get up andkick IXMAS dataset is a multiview dataset for a view-invarianthuman activity recognition where each frame has a size of390times 291 is dataset has a major occlusion and that maycause misclassification therefore we utilized global histogramequalization [61] in order to resolve the occlusion issue

42 Setup For a comprehensive validation we carried outthe following set of experiments executed using Matlab

(i) e first experiment was conducted on each datasetseparately in order to show the performance of theproposed model In this experiment we employed10-fold cross-validation rule which means that data

from 9 subjects were utilized for training data whilethe data from one subject was picked as a testingdata e procedure was reiterated for 10 timesprovided each subject data is utilized for bothtraining and testing

(ii) e second experiment was conducted in the ab-sence of the proposed recognition model on all thefour datasets that will show the importance of thedeveloped model For this purpose we used theexisting eminent classifiers like SVM ANN HMMand existing HCRF [30] as a recognition modelrather than utilizing the proposed HCRF model

(iii) e third experiment was conducted to show theperformance of the proposed approach against thestate-of-the-art methods

(iv) In the last experiment the computational com-plexity of the proposed HCRF model was comparedwith forwardbackward algorithms

5 Results and Discussion

51 First Experiment As described before this experimentvalidates the performance of the proposed recognition model

Table 6 Classification results of the proposed system on KTH action dataset (A) using ANN (B) using SVM (C) using HMM and (D)using existing HCRF [30] while removing the proposed HCRF model (unit )

Activities Walking Jogging Running Boxing Hand-wave Handclap(A)Walking 79 5 6 4 3 3Jogging 3 81 5 3 4 4Running 6 4 77 5 5 3Boxing 6 7 6 69 5 7Hand-wave 4 7 5 5 73 6Handclap 4 6 5 4 6 75Average 7566(B)Walking 82 2 3 5 4 4Jogging 3 86 2 3 2 4Running 5 3 80 4 5 3Boxing 5 3 3 79 4 6Hand-wave 1 4 3 3 89 0Handclap 3 5 2 4 3 83Average 8317(C)Walking 86 3 2 4 2 3Jogging 0 88 3 2 4 3Running 0 3 90 0 4 3Boxing 3 0 4 92 1 0Hand-wave 1 3 2 2 91 1Handclap 1 3 4 1 2 89Average 8933(D)Walking 90 3 0 3 4 0Jogging 2 88 2 3 3 2Running 4 2 92 0 0 2Boxing 1 3 2 91 3 0Hand-wave 0 1 3 2 93 1Handclap 1 3 2 4 3 87Average 9017

Computational Intelligence and Neuroscience 9

on an individual dataset e overall results are shown inTables 1 (using Weizmann dataset) 2 (using KTH dataset) 3(usingUCF sports dataset) and 4 (using IXMAS) respectively

As observed from Tables 1ndash4 the proposed recognitionmodel constantly obtained higher recognition rates on in-dividual dataset is result shows the robustness of theproposed model which means that the model not onlyshowed better performance on one dataset but also showedbetter performance across multiple spontaneous datasets

52 Second Experiment As described before the secondexperiment was conducted in the absence of the proposed

recognition model to show the importance of the proposedmodel using all the four datasets For this purpose we usedthe existing eminent classifiers like SVM ANN HMM andexisting HCRF [30] as a recognition model rather thanutilizing the proposed HCRF model

Tables 5ndash8 show that when the proposed HCRF modelwas substituted with ANN SVM HMM and existing HCRF[30] the system failed to accomplish higher recognitionrates e better performance of the proposed HCRF modelis visualized in Tables 1ndash4 which show that the proposedHCRF model effectively fix the drawbacks of HMM andexisting HCRF that has been extensively utilized for se-quential HAR

Table 7 Classification results of the proposed system onUCF sports dataset (A) using ANN (B) using SVM (C) using HMM and (D) usingexisting HCRF [30] while removing the proposed HCRF model (unit )

Activities Diving GS Kicking Lifting HBR Run Skating BS Walk(A)Diving 68 4 2 5 6 6 4 3 2GS 2 71 2 4 5 4 6 3 3Kicking 3 4 70 3 5 4 2 3 6Lifting 5 4 3 65 5 6 4 6 2HBR 3 4 6 3 66 4 5 5 4Running 3 3 5 4 6 64 6 4 5Skating 2 5 4 5 3 4 69 3 5BS 4 2 5 3 4 6 5 67 4Walking 5 4 2 3 4 3 6 3 70Average 6778(B)Diving 71 4 2 3 5 6 3 2 4GS 3 77 2 4 3 2 5 2 2Kicking 4 2 74 4 5 3 2 3 3Lifting 5 6 3 69 4 3 5 3 2HBR 2 3 3 2 80 2 4 2 2Running 2 3 2 2 5 75 6 2 3Skating 2 1 2 3 4 4 78 2 4BS 3 4 6 3 4 2 3 70 5Walking 4 1 2 4 2 3 0 3 81Average 7500(C)Diving 79 3 2 2 3 4 3 2 2GS 0 83 2 4 3 2 1 3 2Kicking 1 2 85 1 3 3 2 3 0Lifting 3 0 2 82 3 2 4 2 2HBR 0 2 2 4 80 0 5 3 4Running 1 2 1 3 4 84 2 1 2Skating 2 0 3 4 0 1 86 3 1BS 1 1 1 2 0 3 0 88 4Walking 1 2 4 2 5 2 4 3 77Average 8267(D)Diving 90 3 0 1 0 2 2 1 1GS 3 84 2 1 3 1 3 2 1Kicking 3 4 85 0 0 2 3 1 2Lifting 1 2 1 89 1 1 1 2 2HBR 0 2 1 0 91 2 3 0 1Running 2 3 1 2 3 80 4 2 3Skating 2 4 1 2 3 0 84 4 0BS 2 1 1 2 1 0 3 88 2Walking 0 2 1 1 0 1 4 0 91Average 8689GS golf swinging HBR horseback riding BS baseball swinging

10 Computational Intelligence and Neuroscience

53 ird Experiment In this experiment a comparativeanalysis was made between the state-of-the-art methods andthe proposed model All of these approaches were imple-mented by the instructions provided in their particular ar-ticles A 10-fold cross-validation rule was employed on eachdataset as explained in Section 4 e average classicationresults of the existing methods along with the proposedmethod across dierent datasets are summarized in Table 9

Table 8 Classication results of the proposed system on IXMASaction dataset (A) using ANN (B) using SVM (C) using HMMand (D) using existing HCRF [30] while removing the proposedHCRF model (unit )

Activities CA SD GU TA Walk Wave Punch Kick(A)CA 65 5 7 6 5 4 3 5SD 5 72 4 3 5 4 3 4GU 4 3 75 5 3 2 4 4TA 6 7 4 68 3 5 4 3Walk 3 4 5 4 70 6 4 4Wave 4 6 5 3 4 71 3 4Punch 3 5 5 6 7 3 67 4Kick 4 5 4 6 5 4 3 69Average 6962(B)CA 77 3 4 2 2 3 5 4SD 3 79 3 2 4 5 2 2GU 5 6 69 3 4 5 4 4TA 2 3 2 80 4 4 3 2Walk 3 5 4 2 71 5 6 4Wave 2 6 3 5 4 73 4 3Punch 1 5 3 4 1 3 81 2Kick 3 6 7 4 3 5 4 68Average 7475(C)CA 79 3 4 1 2 4 3 4SD 1 84 3 2 3 1 4 2GU 0 1 88 1 2 3 2 3TA 5 2 3 79 2 3 2 4Walk 1 0 3 1 90 2 3 0Wave 2 3 1 0 3 86 2 3Punch 1 0 2 3 0 4 89 1Kick 3 2 4 1 0 2 4 84Average 8477(D)CA 90 1 2 0 3 4 0 0SD 3 85 2 1 3 3 2 1GU 0 1 91 1 0 2 3 2TA 1 3 2 87 1 2 2 2Walk 1 0 3 1 89 2 1 3Wave 0 2 1 0 2 90 3 1Punch 1 4 2 1 2 3 84 3Kick 1 2 4 1 3 2 4 83Average 8738CA cross arm SD sit down GU get up TA turn around

Table 9 Weighted average recognition rates of the proposedmethod with the existing state-of-the-art methods (unit )

State-of-the-art works Average classicationrates

Standarddeviation

GMM 633 plusmn27SVM 675 plusmn44HMM 828 plusmn38Embedded HMM 859 plusmn19[62] 921 plusmn32[63] 843 plusmn49[18] 936 plusmn27[19] 930 plusmn16[22] 927 plusmn25[64] 801 plusmn32Proposed method 972 plusmn28

0

100

200

300

400

500

600

Exec

utio

n tim

e (s)

1 2 3 4 5 6 7 8Number of states

ForwardbackwardProposed HCRF model

(a)Ex

ecut

ion

time (

s)

1 2 3 4 5 6 7 8

ForwardbackwardProposed HCRF model

0

2

4

6

8

10

12

14

16

Number of mixtures

(b)

Figure 3 An illustration of gradient computational time (equation(30)) of the previous forward and backward algorithms and theproposed HCRF model (a)Q 1 minus 5 M 5 T 90 and (b)Q 5 M 1 minus 5 T 90

Computational Intelligence and Neuroscience 11

It is obvious from Table 9 that the proposed methodshowed a significant performance against the existing state-of-the-art methods erefore the proposed method accu-rately and robustly recognizes the human activities usingdifferent video data

54 Fourth Experiment In this experiment we have pre-sented the computational complexity that is also one of thecontributions in this paper e implementations of theprevious HCRF are available in literature which calculatethe gradients by reiterating the forward and backwardtechniques while the proposed HCRF model executes themonce only and cashes the outcomes for the later use From(21) and (22) it is clear that the forward or backwardtechnique has a complexity ofO(TQ2M) where Trepresentsthe input sequence lengthQ represents the number of statesand M indicates the number of mixtures e proposedHCRFmodel however requires a full complexity of O(TM)

to calculate gradients as can be seen from (22)ndash(29)Figure 3 shows a comparison of the execution time when

the gradients are computed by the forward (or backward)algorithm and by our proposed method e computationaltime is calculated by running Matlab R2013a with thespecification of Intelreg Pentiumreg Coretrade i7-6700 (34GHz)with a RAM capacity of 16GB

6 Conclusion

In healthcare and telemedicine the human activity rec-ognition (HAR) can be best explained by helping physi-cally disable personsrsquo scenario A paralyzed patient withhalf of the body critically attacked by paralysis is com-pletely unable to perform their daily exercises e doctorsrecommend specific activities to get better improvement intheir health So for this purpose the doctors need a humanactivity recognition (HAR) system through which they canmonitor the patientsrsquo daily routines (activities) on a reg-ular basis

e accuracy of most of the HAR systems depends uponthe recognition modules For feature extraction and selec-tion modules we used some of the existing well-knownmethods while for the recognition module we proposed theusage of HCRF model which is capable of approximating acomplex distribution using a mixture of Gaussian densityfunctions e proposed model was assessed against fourpublicly available standard action datasets From our ex-periments it is obvious that the proposed full-covarianceGaussian density function showed a significant improve-ment in accuracy than the existing state-of-the-art methodsFurthermore we also proved that such improvement issignificant from statistical point of view by showing valuele02 of the comparison Similarly the complexity analysispoints out that the proposed computational method stronglydecreases the execution time for the hidden conditionalrandom field model

e ultimate goal of this study is to deploy the proposedmodel on smartphones Currently the proposed model isusing full-covariance matrix however this might be time

consuming especially when using on smartphones Using alightweight classifier such as K-nearest neighbor (K-NN)could be one possible solution But K-NN is very muchsensitive to environmental factor (like noise) erefore infuture we will try to investigate further research to reducethe time and sustain the same recognition rate whenemploying on smartphones in real environment

Data Availability

e data used to support the findings of this study areavailable from the corresponding author upon request

Conflicts of Interest

e authors declare there are no conflicts of interest

Acknowledgments

is study was supported by the Jouf University SakakaAljouf Kingdom of Saudi Arabia under the registration no39791

References

[1] Z Chang J Cao and Y Zhang ldquoA novel image segmentationapproach for wood plate surface defect classification throughconvex optimizationrdquo Journal of Forestry Research vol 29no 6 pp 1789ndash1795 2018

[2] Z S Abdallah M M Gaber B Srinivasan andS Krishnaswamy ldquoAdaptive mobile activity recognitionsystemwith evolving data streamsrdquoNeurocomputing vol 150pp 304ndash317 2015

[3] M S Bakli M A Sakr and T H A Soliman ldquoA spatio-temporal algebra in Hadoop for moving objectsrdquo Geo-SpatialInformation Science vol 21 no 2 pp 102ndash114 2018

[4] W Zhao L Yan and Y Zhang ldquoGeometric-constrainedmulti-view image matching method based on semi-globaloptimizationrdquo Geo-Spatial Information Science vol 21 no 2pp 115ndash126 2018

[5] H Gao X Zhang J Zhao and D Li ldquoTechnology of in-telligent driving radar perception based on driving brainrdquoCAAI Transactions on Intelligence Technology vol 2 no 3pp 93ndash100 2017

[6] Y Wang X Jiang R Cao and X Wang ldquoRobust indoorhuman activity recognition using wireless signalsrdquo Sensorsvol 15 no 7 pp 17195ndash17208 2015

[7] J Wang Z Liu Y Wu and J Yuan ldquoMining actionlet ensemblefor action recognition with depth camerasrdquo in Proceedings of theIEEE Conference on Computer Vision and Pattern Recognitionpp 1290ndash1297 IEEE Providence RI USA June 2012

[8] S Karungaru M Daikoku and K Terada ldquoMulti camerasbased indoors human action recognition using fuzzy rulesrdquoJournal of Pattern Recognition Research vol 10 no 1pp 61ndash74 2015

[9] S Shukri L M Kamarudin and M H F Rahiman ldquoDevice-free localization for human activity monitoringrdquo in IntelligentVideo Surveillance IntechOpen London UK 2018

[10] L Fiore D Fehr R Bodor A Drenner G Somasundaramand N Papanikolopoulos ldquoMulti-camera human activitymonitoringrdquo Journal of Intelligent and Robotic Systemsvol 52 no 1 pp 5ndash43 2008

12 Computational Intelligence and Neuroscience

[11] C Zhang and Y Tian ldquoRGB-D camera-based daily livingactivity recognitionrdquo Journal of Computer Vision and ImageProcessing vol 2 no 4 pp 1ndash7 2012

[12] M Leo T DrsquoOrazio and P Spagnolo ldquoHuman activityrecognition for automatic visual surveillance of wide areasrdquo inProceedings of the ACM 2nd International Workshop on VideoSurveillance and Sensor Networks pp 124ndash130 ACM NewYork NY USA October 2004

[13] C-B Jin S Li and H Kim ldquoReal-time action detection invideo surveillance using sub-action descriptor with multi-CNNrdquo 2017 httpsarxivorgabs171003383

[14] W Niu J Long D Han and Y-F Wang ldquoHuman activitydetection and recognition for video surveillancerdquo in Pro-ceedings of the 2004 IEEE International Conference on Mul-timedia and Expo (ICME) (IEEE Cat No 04TH8763) vol 1pp 719ndash722 IEEE Taipei Taiwan June 2004

[15] Q Li M Zheng F Li J Wang Y-A Geng and H JiangldquoRetinal image segmentation using double-scale non-linearthresholding on vessel support regionsrdquo CAAI Transactionson Intelligence Technology vol 2 no 3 pp 109ndash115 2017

[16] L M G Fonseca L M Namikawa and E F CastejonldquoDigital image processing in remote sensingrdquo in Proceedingsof the 2009 Tutorials of the XXII Brazilian Symposium onComputer Graphics and Image Processing pp 59ndash71 IEEERio de Janeiro Brazil October 2009

[17] K Buys C Cagniart A Baksheev T De Laet J De Schutterand C Pantofaru ldquoAn adaptable system for RGB-D basedhuman body detection and pose estimationrdquo Journal of VisualCommunication and Image Representation vol 25 no 1pp 39ndash52 2014

[18] M Sharif M A Khan T Akram M Y Javed T Saba andA Rehman ldquoA framework of human detection and actionrecognition based on uniform segmentation and combinationof Euclidean distance and joint entropy-based features se-lectionrdquo EURASIP Journal on Image and Video Processingvol 2017 no 1 p 89 2017

[19] K-P Chou M Prasad D Wu et al ldquoRobust feature-basedautomated multi-view human action recognition systemrdquoIEEE Access vol 6 pp 15283ndash15296 2018

[20] M Ullah H Ullah and I M Alseadonn ldquoHuman actionrecognition in videos using stable featuresrdquo Signal and ImageProcessing An International Journal vol 8 no 6 pp 1ndash102017

[21] M Sharif M A Khan F Zahid J H Shah and T AkramldquoHuman action recognition a framework of statisticalweighted segmentation and rank correlation-based selectionrdquoPattern Analysis and Applications pp 1ndash14 2019

[22] J Zang L Wang Z Liu Q Zhang G Hua and N ZhengldquoAttention-based temporal weighted convolutional neuralnetwork for action recognitionrdquo in Proceedings of the IFIPInternational Conference on Artificial Intelligence Applicationsand Innovations pp 97ndash108 Springer Rhodes Greece May2018

[23] V-D Hoang ldquoMultiple classifier-based spatiotemporal fea-tures for living activity predictionrdquo Journal of Informationand Telecommunication vol 1 no 1 pp 100ndash112 2017

[24] S Dharmalingam and A Palanisamy ldquoVector space basedaugmented structural kinematic feature descriptor for humanactivity recognition in videosrdquo ETRI Journal vol 40 no 4pp 499ndash510 2018

[25] D Liu Y Yan M-L Shyu G Zhao and M Chen ldquoSpatio-temporal analysis for human action detection and recognitionin uncontrolled environmentsrdquo International Journal of

Multimedia Data Engineering and Management vol 6 no 1pp 1ndash18 2015

[26] C A Ronao and S-B Cho ldquoHuman activity recognition withsmartphone sensors using deep learning neural networksrdquoExpert Systems with Applications vol 59 pp 235ndash244 2016

[27] A arwat H Mahdi M Elhoseny and A E HassanienldquoRecognizing human activity in mobile crowdsensing envi-ronment using optimized k-NN algorithmrdquo Expert Systemswith Applications vol 107 pp 32ndash44 2018

[28] A Jalal Y-H Kim Y-J Kim S Kamal and D Kim ldquoRobusthuman activity recognition from depth video using spatio-temporal multi-fused featuresrdquo Pattern Recognition vol 61pp 295ndash308 2017

[29] J Lafferty A McCallum and F C Pereira ldquoConditionalrandom fields probabilistic models for segmenting and la-beling sequence datardquo in Proceedings of the 18th InternationalConference on Machine Learning (ICML-2001) pp 282ndash289Williamstown MA USA June-July 2001

[30] A Gunawardana M Mahajan A Acero and J C PlattldquoHidden conditional random fields for phone classificationrdquoin Proceedings of the INTERSPEECH vol 2 pp 1117ndash1120Citeseer Lisbon Portugal September 2005

[31] L Yang Q Song Z Wang and M Jiang ldquoParsing r-cnn forinstance-level human analysisrdquo in Proceedings of the IEEEConference on Computer Vision and Pattern Recognitionpp 364ndash373 Long Beach CA USA June 2019

[32] X Liang Y Wei L Lin et al ldquoLearning to segment human bywatching youtuberdquo IEEE Transactions on Pattern Analysisand Machine Intelligence vol 39 no 7 pp 1462ndash1468 2016

[33] M M Requena ldquoHuman segmentation with convolutionalneural networksrdquo esis University of Alicante AlicanteSpain 2018

[34] H Gao S Chen and Z Zhang ldquoParts semantic segmentationaware representation learning for person re-identificationrdquoApplied Sciences vol 9 no 6 p 1239 2019

[35] C Peng S-L Lo J Huang and A C Tsoi ldquoHuman actionsegmentation based on a streaming uniform entropy slicemethodrdquo IEEE Access vol 6 pp 16958ndash16971 2018

[36] M H Siddiqi A M Khan and S-W Lee ldquoActive contourslevel set based still human body segmentation from depthimages for video-based activity recognitionrdquo KSII Trans-actions on Internet and Information Systems (TIIS) vol 7no 11 pp 2839ndash2852 2013

[37] M Siddiqi R Ali M Rana E-K Hong E Kim and S LeeldquoVideo-based human activity recognition using multilevelwavelet decomposition and stepwise linear discriminantanalysisrdquo Sensors vol 14 no 4 pp 6370ndash6392 2014

[38] S Althloothi M H Mahoor X Zhang and R M VoylesldquoHuman activity recognition using multi-features and mul-tiple kernel learningrdquo Pattern Recognition vol 47 no 5pp 1800ndash1812 2014

[39] A Jalal S Kamal and D Kim ldquoA depth video-based humandetection and activity recognition using multi-features andembedded hidden markov models for health care monitoringsystemsrdquo International Journal of Interactive Multimedia andArtificial Intelligence vol 4 no 4 p 54 2017

[40] T Dobhal V Shitole G omas and G Navada ldquoHumanactivity recognition using binary motion image and deeplearningrdquo Procedia Computer Science vol 58 pp 178ndash1852015

[41] S Zhang Z Wei J Nie L Huang S Wang and Z Li ldquoAreview on human activity recognition using vision-basedmethodrdquo Journal of Healthcare Engineering vol 2017 ArticleID 3090343 31 pages 2017

Computational Intelligence and Neuroscience 13

[42] Y Yang B Zhang L Yang C Chen and W Yang ldquoActionrecognition using completed local binary patterns and mul-tiple-class boosting classifierrdquo in Proceedings of the 2015 3rdIAPR Asian Conference on Pattern Recognition (ACPR) IEEEpp 336ndash340 Kuala Lumpur Malaysia November 2015

[43] W Lin M T Sun R Poovandran and Z Zhang ldquoHumanactivity recognition for video surveillancerdquo in Proceedings ofthe 2008 IEEE International Symposium on Circuits andSystems pp 2737ndash2740 IEEE Seattle WA USA May 2008

[44] H Permuter J Francos and I Jermyn ldquoA study of Gaussianmixture models of color and texture features for imageclassification and segmentationrdquo Pattern Recognition vol 39no 4 pp 695ndash706 2006

[45] M Fiaz and B Ijaz ldquoVision based human activity trackingusing artificial neural networksrdquo in Proceedings of the 2010International Conference on Intelligent and Advanced Systemspp 1ndash5 IEEE Kuala Lumpur Malaysia June 2010

[46] H Foroughi A Naseri A Saberi and H S Yazdi ldquoAneigenspace-based approach for human fall detection usingintegrated time motion image and neural networkrdquo in Pro-ceedings of the 2008 9th International Conference on SignalProcessing pp 1499ndash1503 IEEE Beijing China October2008

[47] A K B Sonali ldquoHuman action recognition using supportvector machine and k-nearest neighborrdquo InternationalJournal of Engineering and Technical Research vol 3 no 4pp 423ndash428 2015

[48] V Swarnambigai ldquoAction recognition using ami and supportvector machinerdquo International Journal of Computer Scienceand Technology vol 5 no 4 pp 175ndash179 2014

[49] D Das and S Saharia ldquoHuman gait analysis and recognitionusing support vector machinesrdquo International Journal ofComputer Science and Information Technology vol 6 no 52004

[50] K G M Chathuramali and R Rodrigo ldquoFaster human activityrecognition with SVMrdquo in Proceedings of the InternationalConference on Advances in ICT for Emerging Regions(ICTer2012) pp 197ndash203 IEEE Colombo Sri Lanka De-cember 2012

[51] M Z Uddin D-H Kim J T Kim and T-S Kim ldquoAn indoorhuman activity recognition system for smart home using localbinary pattern features with hidden markov modelsrdquo Indoorand Built Environment vol 22 no 1 pp 289ndash298 2013

[52] M Z Uddin J Lee and T-S Kim ldquoIndependent shapecomponent-based human activity recognition via hiddenMarkov modelrdquo Applied Intelligence vol 33 no 2 pp 193ndash206 2010

[53] S Kumar and M Hebert ldquoDiscriminative Fields for ModelingSpatial Dependencies in Natural Imagesrdquo in Proceedings of theNIPS Vancouver Canada December 2003

[54] A Quattoni S Wang L-P Morency M Collins andT Darrell ldquoHidden conditional random fieldsrdquo IEEETransactions on Pattern Analysis and Machine Intelligencevol 29 no 10 pp 1848ndash1852 2007

[55] M Mahajan A Gunawardana and A Acero ldquoTraining al-gorithms for hidden conditional random fieldsrdquo in Pro-ceedings of the 2006 IEEE International Conference onAcoustics Speed and Signal Processing vol 1 IEEE ToulouseFrance May 2006

[56] S Reiter B Schuller and G Rigoll ldquoHidden conditionalrandom fields for meeting segmentationrdquo in Proceedings ofthe Multimedia and Expo 2007 IEEE International Confer-ence pp 639ndash642 IEEE Beijing China July 2007

[57] L Gorelick M Blank E Shechtman M Irani and R BasrildquoActions as space-time shapesrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 29 no 12 pp 2247ndash2253 2007

[58] I Laptev M Marszalek C Schmid and B RozenfeldldquoLearning realistic human actions from moviesrdquo in Pro-ceedings of the 2008 IEEE Conference on Computer Vision andPattern Recognition pp 1ndash8 IEEE Anchorage AK USAJune 2008

[59] K Soomro and A R Zamir ldquoAction recognition in realisticsports videosrdquo in Computer Vision in Sports pp 181ndash208Springer Berlin Germany 2014

[60] D Weinland E Boyer and R Ronfard ldquoAction recognitionfrom arbitrary views using 3D exemplarsrdquo in Proceedings ofthe 2007 IEEE 11th International Conference on ComputerVision pp 1ndash7 IEEE Rio de Janeiro Brazil October 2007

[61] M Siddiqi S Lee Y-K Lee A Khan and P Truc ldquoHier-archical recognition scheme for human facial expressionrecognition systemsrdquo Sensors vol 13 no 12 pp 16682ndash167132013

[62] M Elmezain and A Al-Hamadi ldquoVision-based human ac-tivity recognition using ldcrfsrdquo International Arab Journal ofInformation Technology (IAJIT) vol 15 no 3 pp 389ndash3952018

[63] A Roy B Banerjee and V Murino ldquoA novel dictionarylearning based multiple instance learning approach to actionrecognition from videosrdquo in Proceedings of the 6th In-ternational Conference on Pattern Recognition Applicationsand Methods pp 519ndash526 Porto Portugal February 2017

[64] A P Sirat ldquoActions as space-time shapesrdquo InternationalJournal of Application or Innovation in Engineering andManagement vol 7 no 8 pp 49ndash54 2018

14 Computational Intelligence and Neuroscience

Computer Games Technology

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

Advances in

FuzzySystems

Hindawiwwwhindawicom

Volume 2018

International Journal of

ReconfigurableComputing

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

thinspArtificial Intelligence

Hindawiwwwhindawicom Volumethinsp2018

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawiwwwhindawicom Volume 2018

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Engineering Mathematics

International Journal of

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Computational Intelligence and Neuroscience

Hindawiwwwhindawicom Volume 2018

Mathematical Problems in Engineering

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Hindawiwwwhindawicom Volume 2018

Human-ComputerInteraction

Advances in

Hindawiwwwhindawicom Volume 2018

Scientic Programming

Submit your manuscripts atwwwhindawicom

Page 6: Human Activity Recognition Using Gaussian Mixture Hidden …downloads.hindawi.com/journals/cin/2019/8590560.pdf · 2019-08-22 · ResearchArticle Human Activity Recognition Using

e d Score function is a gradient function for aGaussian mixture weight variable Here a function V(t) canbe determined as

c(t) suml

β(t l) (27)

d Score(V ∣ UΛ Γ μΣ)dμlm

sumT

t1

ΓObslm dN ut μlmΣlm( )dμlmsumMm1ΓObslm N ut μlmΣlm( )

middot α(t l)c(t + 1)(28)

e d Score function is a gradient function for theGaussian distribution mean

d Score(V ∣ UΛ Γ μΣ)dΣlm

sumT

t1

ΓObslm dN ut μlmΣlm( )dΣlmsumMm1ΓObslm N ut μlmΣlm( )

middot α(t l)c(t + 1)(29)

e d Score function is a gradient function for the co-variance of the Gaussian distributions

Equations (24)ndash(27) presented above describe an anal-ysis method algorithm for calculating values of gradients fora feature function the mean of Gaussian distributions and

the covariance of the prior probability vector the transitionprobability vector and the observation probability vectorobtained from the existing HCRF

In our model the recognition of a variety of real-timeactivities can be divided into two steps a training step and aninference step In the rst step data with known labels areinputted for recognizing the target as well as training thehidden conditional eld model In the inference step theinputs to be actually estimated are ordered dependent onparameters determined in the training phase

If the activity frame is acting as an input in the training stepthen in the preprocessing step the applied distinctive lightingeects are decreased for detecting and extracting faces from theactivity frames At that point the movable features are extri-cated from the various facial parts for creating the featurevector After that the feature vector obtained serves as an inputto a full-covariance Gaussian-mixed hidden conditional ran-dom eld model of the suggested recognition model

As mentioned in the earlier discussion a feature gradient isgenerally determined by LBFG approach in the training phaseof the HCRF model Nonetheless in the current gradientcalculation technique a forward and backward iterative exe-cution algorithm is iteratively called upon which needs anexceptionally high computational time and thus leads to re-duction in the computational speed Another analysis ap-proach has been formulated that reduces the invoking of theforward and backward iterative execution algorithm using vegradient functions determined by equations (24)ndash(28) Using

Observed activity labels

Hidden CRFparameter

Modelweights

Hidden CRFtraining engine

Data calibrationand normalization

Unified interface

Raw data 1

2

5

7

6

3

4

4

ClusteringVectorquantization

Feature extraction

Discreteinput

sequence

Normalizeddata

Kernelvectors

Kernelvectors

Feature vectors

External interfaceInternal interface

Data files

Codebookvectors

Figure 2 Workow diagram of the proposed recognition model

6 Computational Intelligence and Neuroscience

this analysis the real-time computation can be carried out at ahigher speed resulting in an enormous decrease in the com-putational time compared to a known analysis approach eoverall workflow of the proposed model is shown in Figure 2

4 Model Validation

41 Datasets Used In this work we employed four open-source standard action datasets like Weizmann actiondatasets [57] KTH action dataset [58] UCF sports dataset[59] and IXMAS action dataset [60] for corroborating theproposed HCRF model performance All the datasets areexplained below

411 Weizmann Action Dataset is dataset consisted of10 actions such as bending running walking skippingplace jumping side movement jumping forward two handwaving and one hand waving that were performed by total9 subjects is dataset comprised of 90 video clips with

average of 15 frames per clip where the frame size is144 times180

412 KTH Action Dataset KTH dataset employed for ac-tivity recognition comprised of 25 subjects who performed 6activities like running walking boxing jogging hand-clapping and hand waving in four distinctive scenariosUsing a static camera in the homogenous background atotal of 2391 sequences were taken with a frame size of160times120

413 UCF Sports Dataset In this dataset there were 182videos which were evaluated by n-fold cross-validation ruleis dataset has been taken from different sports activities inbroadcast television channels Some of the videos had highintraclass similarities is dataset was also collected using astatic camera is dataset covers 9 activities like runningdiving lifting golf swinging skating kicking walking

Table 2 Confusion matrix of the proposed recognition model using KTH action dataset (unit )

Activities Walking Jogging Running Boxing Hand-wave HandclapWalking 100 0 0 0 0 0Jogging 0 98 1 1 0 0Running 2 1 95 1 1 0Boxing 0 2 1 97 0 0Hand-wave 1 0 1 0 98 0Handclap 0 0 1 0 0 99Average 9783

Table 1 Confusion matrix of the proposed recognition model using Weizmann action dataset (unit )

Activities Bend Jack Pjump Run Side Skip Walk Wave 1 Wave 2Bend 98 0 1 0 0 1 0 0 0Jack 1 96 0 0 2 0 1 0 0Pjump 0 1 97 0 1 0 0 1 0Run 0 0 0 99 0 0 0 1 0Side 1 2 0 0 95 0 0 2 0Skip 0 0 0 0 0 100 0 0 0Walk 1 0 0 1 0 1 96 0 1Wave 1 0 1 1 0 0 0 0 98 0Wave 2 0 1 0 0 1 1 1 1 95Average 9711

Table 3 Confusion matrix of the proposed recognition model using UCF sports dataset (unit )

Activities Diving GS Kicking Lifting HBR Run Skating BS WalkDiving 95 1 2 1 0 1 0 0 0GS 1 94 0 0 2 1 1 1 0Kicking 0 2 98 0 0 0 0 0 0Lifting 1 1 1 94 1 0 0 1 1HBR 0 0 2 0 96 1 0 1 0Running 0 3 0 0 0 97 0 0 0Skating 1 0 1 1 0 1 95 0 1BS 0 0 0 1 0 1 0 97 1Walking 0 0 0 0 0 0 0 0 100Average 9622GS golf swinging HBR horseback riding and BS baseball swinging

Computational Intelligence and Neuroscience 7

Table 4 Confusion matrix of the proposed recognition model using IXMAS action dataset (unit )

Activities CA SD GU TA Walk Wave Punch KickCA 97 0 0 1 2 0 0 0SD 0 99 1 0 0 0 0 0GU 1 2 94 3 0 0 0 0TA 0 0 1 95 2 1 1 0Walk 0 1 1 0 98 0 0 0Wave 0 0 2 0 1 97 0 0Punch 0 1 0 1 0 2 96 0Kick 0 0 0 0 1 0 0 99Average 9688CA cross arm SD sit down GU get up and TA turn around

Table 5 Classification results of the proposed system on Weizmann action dataset (A) using ANN (B) using SVM (C) using HMM and(D) using existing HCRF [30] while removing the proposed HCRF model (unit )

Activities Bend Jack Pjump Run Side Skip Walk Wave 1 Wave 2(A)Bend 70 4 5 3 3 2 5 5 3Jack 4 68 3 6 7 2 3 3 4Pjump 2 4 75 6 3 2 2 2 4Run 4 2 3 72 6 2 5 3 3Side 5 3 5 4 65 6 4 6 2Skip 4 6 4 3 5 67 3 2 6Walk 4 2 4 7 3 4 70 3 3Wave 1 2 1 3 3 4 5 7 71 4Wave 2 2 5 3 6 4 4 3 5 68Average 6955(B)Bend 69 3 4 4 6 4 2 3 5Jack 2 72 2 3 4 3 5 4 5Pjump 1 4 75 2 4 5 4 2 3Run 2 4 3 78 2 2 4 2 3Side 2 4 5 3 70 4 3 5 4Skip 2 1 3 2 4 80 3 3 2Walk 2 0 3 4 3 2 82 1 3Wave 1 2 2 3 4 3 2 3 77 4Wave 2 1 2 1 2 3 1 3 4 83Average 7622(C)Bend 82 3 0 2 2 3 1 5 2Jack 3 80 1 2 3 2 3 4 2Pjump 3 4 85 3 0 0 1 2 2Run 5 4 2 79 0 2 1 3 4Side 0 1 5 4 81 3 1 2 3Skip 3 1 2 2 3 88 0 0 1Walk 0 2 3 2 1 2 83 3 4Wave 1 1 3 2 2 4 2 3 78 5Wave 2 1 2 2 2 2 3 1 0 87Average 8256(D)Bend 80 2 3 1 4 0 5 2 3Jack 1 88 0 2 0 3 2 3 1Pjump 0 2 90 1 0 3 0 2 2Run 2 1 2 85 2 3 0 0 5Side 4 1 2 3 80 4 1 2 3Skip 1 4 0 5 1 84 0 3 2Walk 2 1 0 0 1 2 89 2 3Wave 1 3 0 1 2 0 2 0 91 1Wave 2 4 1 3 0 2 3 0 2 85Average 8578

8 Computational Intelligence and Neuroscience

horseback riding and baseball swinging Each frame has asize of 720times 480

414 IXMAS Action Dataset IXMAS (INRIA Xmas motionacquisition sequences) dataset comprised of 13 activity classeswhich were performed by 11 actors each 3 times Every actoropted a free orientation as well as position e dataset hasprovided annotated silhouettes for each person For our ex-periments we have selected only 8 action classes like walkcross arms punch turn around sit down wave get up andkick IXMAS dataset is a multiview dataset for a view-invarianthuman activity recognition where each frame has a size of390times 291 is dataset has a major occlusion and that maycause misclassification therefore we utilized global histogramequalization [61] in order to resolve the occlusion issue

42 Setup For a comprehensive validation we carried outthe following set of experiments executed using Matlab

(i) e first experiment was conducted on each datasetseparately in order to show the performance of theproposed model In this experiment we employed10-fold cross-validation rule which means that data

from 9 subjects were utilized for training data whilethe data from one subject was picked as a testingdata e procedure was reiterated for 10 timesprovided each subject data is utilized for bothtraining and testing

(ii) e second experiment was conducted in the ab-sence of the proposed recognition model on all thefour datasets that will show the importance of thedeveloped model For this purpose we used theexisting eminent classifiers like SVM ANN HMMand existing HCRF [30] as a recognition modelrather than utilizing the proposed HCRF model

(iii) e third experiment was conducted to show theperformance of the proposed approach against thestate-of-the-art methods

(iv) In the last experiment the computational com-plexity of the proposed HCRF model was comparedwith forwardbackward algorithms

5 Results and Discussion

51 First Experiment As described before this experimentvalidates the performance of the proposed recognition model

Table 6 Classification results of the proposed system on KTH action dataset (A) using ANN (B) using SVM (C) using HMM and (D)using existing HCRF [30] while removing the proposed HCRF model (unit )

Activities Walking Jogging Running Boxing Hand-wave Handclap(A)Walking 79 5 6 4 3 3Jogging 3 81 5 3 4 4Running 6 4 77 5 5 3Boxing 6 7 6 69 5 7Hand-wave 4 7 5 5 73 6Handclap 4 6 5 4 6 75Average 7566(B)Walking 82 2 3 5 4 4Jogging 3 86 2 3 2 4Running 5 3 80 4 5 3Boxing 5 3 3 79 4 6Hand-wave 1 4 3 3 89 0Handclap 3 5 2 4 3 83Average 8317(C)Walking 86 3 2 4 2 3Jogging 0 88 3 2 4 3Running 0 3 90 0 4 3Boxing 3 0 4 92 1 0Hand-wave 1 3 2 2 91 1Handclap 1 3 4 1 2 89Average 8933(D)Walking 90 3 0 3 4 0Jogging 2 88 2 3 3 2Running 4 2 92 0 0 2Boxing 1 3 2 91 3 0Hand-wave 0 1 3 2 93 1Handclap 1 3 2 4 3 87Average 9017

Computational Intelligence and Neuroscience 9

on an individual dataset e overall results are shown inTables 1 (using Weizmann dataset) 2 (using KTH dataset) 3(usingUCF sports dataset) and 4 (using IXMAS) respectively

As observed from Tables 1ndash4 the proposed recognitionmodel constantly obtained higher recognition rates on in-dividual dataset is result shows the robustness of theproposed model which means that the model not onlyshowed better performance on one dataset but also showedbetter performance across multiple spontaneous datasets

52 Second Experiment As described before the secondexperiment was conducted in the absence of the proposed

recognition model to show the importance of the proposedmodel using all the four datasets For this purpose we usedthe existing eminent classifiers like SVM ANN HMM andexisting HCRF [30] as a recognition model rather thanutilizing the proposed HCRF model

Tables 5ndash8 show that when the proposed HCRF modelwas substituted with ANN SVM HMM and existing HCRF[30] the system failed to accomplish higher recognitionrates e better performance of the proposed HCRF modelis visualized in Tables 1ndash4 which show that the proposedHCRF model effectively fix the drawbacks of HMM andexisting HCRF that has been extensively utilized for se-quential HAR

Table 7 Classification results of the proposed system onUCF sports dataset (A) using ANN (B) using SVM (C) using HMM and (D) usingexisting HCRF [30] while removing the proposed HCRF model (unit )

Activities Diving GS Kicking Lifting HBR Run Skating BS Walk(A)Diving 68 4 2 5 6 6 4 3 2GS 2 71 2 4 5 4 6 3 3Kicking 3 4 70 3 5 4 2 3 6Lifting 5 4 3 65 5 6 4 6 2HBR 3 4 6 3 66 4 5 5 4Running 3 3 5 4 6 64 6 4 5Skating 2 5 4 5 3 4 69 3 5BS 4 2 5 3 4 6 5 67 4Walking 5 4 2 3 4 3 6 3 70Average 6778(B)Diving 71 4 2 3 5 6 3 2 4GS 3 77 2 4 3 2 5 2 2Kicking 4 2 74 4 5 3 2 3 3Lifting 5 6 3 69 4 3 5 3 2HBR 2 3 3 2 80 2 4 2 2Running 2 3 2 2 5 75 6 2 3Skating 2 1 2 3 4 4 78 2 4BS 3 4 6 3 4 2 3 70 5Walking 4 1 2 4 2 3 0 3 81Average 7500(C)Diving 79 3 2 2 3 4 3 2 2GS 0 83 2 4 3 2 1 3 2Kicking 1 2 85 1 3 3 2 3 0Lifting 3 0 2 82 3 2 4 2 2HBR 0 2 2 4 80 0 5 3 4Running 1 2 1 3 4 84 2 1 2Skating 2 0 3 4 0 1 86 3 1BS 1 1 1 2 0 3 0 88 4Walking 1 2 4 2 5 2 4 3 77Average 8267(D)Diving 90 3 0 1 0 2 2 1 1GS 3 84 2 1 3 1 3 2 1Kicking 3 4 85 0 0 2 3 1 2Lifting 1 2 1 89 1 1 1 2 2HBR 0 2 1 0 91 2 3 0 1Running 2 3 1 2 3 80 4 2 3Skating 2 4 1 2 3 0 84 4 0BS 2 1 1 2 1 0 3 88 2Walking 0 2 1 1 0 1 4 0 91Average 8689GS golf swinging HBR horseback riding BS baseball swinging

10 Computational Intelligence and Neuroscience

53 ird Experiment In this experiment a comparativeanalysis was made between the state-of-the-art methods andthe proposed model All of these approaches were imple-mented by the instructions provided in their particular ar-ticles A 10-fold cross-validation rule was employed on eachdataset as explained in Section 4 e average classicationresults of the existing methods along with the proposedmethod across dierent datasets are summarized in Table 9

Table 8 Classication results of the proposed system on IXMASaction dataset (A) using ANN (B) using SVM (C) using HMMand (D) using existing HCRF [30] while removing the proposedHCRF model (unit )

Activities CA SD GU TA Walk Wave Punch Kick(A)CA 65 5 7 6 5 4 3 5SD 5 72 4 3 5 4 3 4GU 4 3 75 5 3 2 4 4TA 6 7 4 68 3 5 4 3Walk 3 4 5 4 70 6 4 4Wave 4 6 5 3 4 71 3 4Punch 3 5 5 6 7 3 67 4Kick 4 5 4 6 5 4 3 69Average 6962(B)CA 77 3 4 2 2 3 5 4SD 3 79 3 2 4 5 2 2GU 5 6 69 3 4 5 4 4TA 2 3 2 80 4 4 3 2Walk 3 5 4 2 71 5 6 4Wave 2 6 3 5 4 73 4 3Punch 1 5 3 4 1 3 81 2Kick 3 6 7 4 3 5 4 68Average 7475(C)CA 79 3 4 1 2 4 3 4SD 1 84 3 2 3 1 4 2GU 0 1 88 1 2 3 2 3TA 5 2 3 79 2 3 2 4Walk 1 0 3 1 90 2 3 0Wave 2 3 1 0 3 86 2 3Punch 1 0 2 3 0 4 89 1Kick 3 2 4 1 0 2 4 84Average 8477(D)CA 90 1 2 0 3 4 0 0SD 3 85 2 1 3 3 2 1GU 0 1 91 1 0 2 3 2TA 1 3 2 87 1 2 2 2Walk 1 0 3 1 89 2 1 3Wave 0 2 1 0 2 90 3 1Punch 1 4 2 1 2 3 84 3Kick 1 2 4 1 3 2 4 83Average 8738CA cross arm SD sit down GU get up TA turn around

Table 9 Weighted average recognition rates of the proposedmethod with the existing state-of-the-art methods (unit )

State-of-the-art works Average classicationrates

Standarddeviation

GMM 633 plusmn27SVM 675 plusmn44HMM 828 plusmn38Embedded HMM 859 plusmn19[62] 921 plusmn32[63] 843 plusmn49[18] 936 plusmn27[19] 930 plusmn16[22] 927 plusmn25[64] 801 plusmn32Proposed method 972 plusmn28

0

100

200

300

400

500

600

Exec

utio

n tim

e (s)

1 2 3 4 5 6 7 8Number of states

ForwardbackwardProposed HCRF model

(a)Ex

ecut

ion

time (

s)

1 2 3 4 5 6 7 8

ForwardbackwardProposed HCRF model

0

2

4

6

8

10

12

14

16

Number of mixtures

(b)

Figure 3 An illustration of gradient computational time (equation(30)) of the previous forward and backward algorithms and theproposed HCRF model (a)Q 1 minus 5 M 5 T 90 and (b)Q 5 M 1 minus 5 T 90

Computational Intelligence and Neuroscience 11

It is obvious from Table 9 that the proposed methodshowed a significant performance against the existing state-of-the-art methods erefore the proposed method accu-rately and robustly recognizes the human activities usingdifferent video data

54 Fourth Experiment In this experiment we have pre-sented the computational complexity that is also one of thecontributions in this paper e implementations of theprevious HCRF are available in literature which calculatethe gradients by reiterating the forward and backwardtechniques while the proposed HCRF model executes themonce only and cashes the outcomes for the later use From(21) and (22) it is clear that the forward or backwardtechnique has a complexity ofO(TQ2M) where Trepresentsthe input sequence lengthQ represents the number of statesand M indicates the number of mixtures e proposedHCRFmodel however requires a full complexity of O(TM)

to calculate gradients as can be seen from (22)ndash(29)Figure 3 shows a comparison of the execution time when

the gradients are computed by the forward (or backward)algorithm and by our proposed method e computationaltime is calculated by running Matlab R2013a with thespecification of Intelreg Pentiumreg Coretrade i7-6700 (34GHz)with a RAM capacity of 16GB

6 Conclusion

In healthcare and telemedicine the human activity rec-ognition (HAR) can be best explained by helping physi-cally disable personsrsquo scenario A paralyzed patient withhalf of the body critically attacked by paralysis is com-pletely unable to perform their daily exercises e doctorsrecommend specific activities to get better improvement intheir health So for this purpose the doctors need a humanactivity recognition (HAR) system through which they canmonitor the patientsrsquo daily routines (activities) on a reg-ular basis

e accuracy of most of the HAR systems depends uponthe recognition modules For feature extraction and selec-tion modules we used some of the existing well-knownmethods while for the recognition module we proposed theusage of HCRF model which is capable of approximating acomplex distribution using a mixture of Gaussian densityfunctions e proposed model was assessed against fourpublicly available standard action datasets From our ex-periments it is obvious that the proposed full-covarianceGaussian density function showed a significant improve-ment in accuracy than the existing state-of-the-art methodsFurthermore we also proved that such improvement issignificant from statistical point of view by showing valuele02 of the comparison Similarly the complexity analysispoints out that the proposed computational method stronglydecreases the execution time for the hidden conditionalrandom field model

e ultimate goal of this study is to deploy the proposedmodel on smartphones Currently the proposed model isusing full-covariance matrix however this might be time

consuming especially when using on smartphones Using alightweight classifier such as K-nearest neighbor (K-NN)could be one possible solution But K-NN is very muchsensitive to environmental factor (like noise) erefore infuture we will try to investigate further research to reducethe time and sustain the same recognition rate whenemploying on smartphones in real environment

Data Availability

e data used to support the findings of this study areavailable from the corresponding author upon request

Conflicts of Interest

e authors declare there are no conflicts of interest

Acknowledgments

is study was supported by the Jouf University SakakaAljouf Kingdom of Saudi Arabia under the registration no39791

References

[1] Z Chang J Cao and Y Zhang ldquoA novel image segmentationapproach for wood plate surface defect classification throughconvex optimizationrdquo Journal of Forestry Research vol 29no 6 pp 1789ndash1795 2018

[2] Z S Abdallah M M Gaber B Srinivasan andS Krishnaswamy ldquoAdaptive mobile activity recognitionsystemwith evolving data streamsrdquoNeurocomputing vol 150pp 304ndash317 2015

[3] M S Bakli M A Sakr and T H A Soliman ldquoA spatio-temporal algebra in Hadoop for moving objectsrdquo Geo-SpatialInformation Science vol 21 no 2 pp 102ndash114 2018

[4] W Zhao L Yan and Y Zhang ldquoGeometric-constrainedmulti-view image matching method based on semi-globaloptimizationrdquo Geo-Spatial Information Science vol 21 no 2pp 115ndash126 2018

[5] H Gao X Zhang J Zhao and D Li ldquoTechnology of in-telligent driving radar perception based on driving brainrdquoCAAI Transactions on Intelligence Technology vol 2 no 3pp 93ndash100 2017

[6] Y Wang X Jiang R Cao and X Wang ldquoRobust indoorhuman activity recognition using wireless signalsrdquo Sensorsvol 15 no 7 pp 17195ndash17208 2015

[7] J Wang Z Liu Y Wu and J Yuan ldquoMining actionlet ensemblefor action recognition with depth camerasrdquo in Proceedings of theIEEE Conference on Computer Vision and Pattern Recognitionpp 1290ndash1297 IEEE Providence RI USA June 2012

[8] S Karungaru M Daikoku and K Terada ldquoMulti camerasbased indoors human action recognition using fuzzy rulesrdquoJournal of Pattern Recognition Research vol 10 no 1pp 61ndash74 2015

[9] S Shukri L M Kamarudin and M H F Rahiman ldquoDevice-free localization for human activity monitoringrdquo in IntelligentVideo Surveillance IntechOpen London UK 2018

[10] L Fiore D Fehr R Bodor A Drenner G Somasundaramand N Papanikolopoulos ldquoMulti-camera human activitymonitoringrdquo Journal of Intelligent and Robotic Systemsvol 52 no 1 pp 5ndash43 2008

12 Computational Intelligence and Neuroscience

[11] C Zhang and Y Tian ldquoRGB-D camera-based daily livingactivity recognitionrdquo Journal of Computer Vision and ImageProcessing vol 2 no 4 pp 1ndash7 2012

[12] M Leo T DrsquoOrazio and P Spagnolo ldquoHuman activityrecognition for automatic visual surveillance of wide areasrdquo inProceedings of the ACM 2nd International Workshop on VideoSurveillance and Sensor Networks pp 124ndash130 ACM NewYork NY USA October 2004

[13] C-B Jin S Li and H Kim ldquoReal-time action detection invideo surveillance using sub-action descriptor with multi-CNNrdquo 2017 httpsarxivorgabs171003383

[14] W Niu J Long D Han and Y-F Wang ldquoHuman activitydetection and recognition for video surveillancerdquo in Pro-ceedings of the 2004 IEEE International Conference on Mul-timedia and Expo (ICME) (IEEE Cat No 04TH8763) vol 1pp 719ndash722 IEEE Taipei Taiwan June 2004

[15] Q Li M Zheng F Li J Wang Y-A Geng and H JiangldquoRetinal image segmentation using double-scale non-linearthresholding on vessel support regionsrdquo CAAI Transactionson Intelligence Technology vol 2 no 3 pp 109ndash115 2017

[16] L M G Fonseca L M Namikawa and E F CastejonldquoDigital image processing in remote sensingrdquo in Proceedingsof the 2009 Tutorials of the XXII Brazilian Symposium onComputer Graphics and Image Processing pp 59ndash71 IEEERio de Janeiro Brazil October 2009

[17] K Buys C Cagniart A Baksheev T De Laet J De Schutterand C Pantofaru ldquoAn adaptable system for RGB-D basedhuman body detection and pose estimationrdquo Journal of VisualCommunication and Image Representation vol 25 no 1pp 39ndash52 2014

[18] M Sharif M A Khan T Akram M Y Javed T Saba andA Rehman ldquoA framework of human detection and actionrecognition based on uniform segmentation and combinationof Euclidean distance and joint entropy-based features se-lectionrdquo EURASIP Journal on Image and Video Processingvol 2017 no 1 p 89 2017

[19] K-P Chou M Prasad D Wu et al ldquoRobust feature-basedautomated multi-view human action recognition systemrdquoIEEE Access vol 6 pp 15283ndash15296 2018

[20] M Ullah H Ullah and I M Alseadonn ldquoHuman actionrecognition in videos using stable featuresrdquo Signal and ImageProcessing An International Journal vol 8 no 6 pp 1ndash102017

[21] M Sharif M A Khan F Zahid J H Shah and T AkramldquoHuman action recognition a framework of statisticalweighted segmentation and rank correlation-based selectionrdquoPattern Analysis and Applications pp 1ndash14 2019

[22] J Zang L Wang Z Liu Q Zhang G Hua and N ZhengldquoAttention-based temporal weighted convolutional neuralnetwork for action recognitionrdquo in Proceedings of the IFIPInternational Conference on Artificial Intelligence Applicationsand Innovations pp 97ndash108 Springer Rhodes Greece May2018

[23] V-D Hoang ldquoMultiple classifier-based spatiotemporal fea-tures for living activity predictionrdquo Journal of Informationand Telecommunication vol 1 no 1 pp 100ndash112 2017

[24] S Dharmalingam and A Palanisamy ldquoVector space basedaugmented structural kinematic feature descriptor for humanactivity recognition in videosrdquo ETRI Journal vol 40 no 4pp 499ndash510 2018

[25] D Liu Y Yan M-L Shyu G Zhao and M Chen ldquoSpatio-temporal analysis for human action detection and recognitionin uncontrolled environmentsrdquo International Journal of

Multimedia Data Engineering and Management vol 6 no 1pp 1ndash18 2015

[26] C A Ronao and S-B Cho ldquoHuman activity recognition withsmartphone sensors using deep learning neural networksrdquoExpert Systems with Applications vol 59 pp 235ndash244 2016

[27] A arwat H Mahdi M Elhoseny and A E HassanienldquoRecognizing human activity in mobile crowdsensing envi-ronment using optimized k-NN algorithmrdquo Expert Systemswith Applications vol 107 pp 32ndash44 2018

[28] A Jalal Y-H Kim Y-J Kim S Kamal and D Kim ldquoRobusthuman activity recognition from depth video using spatio-temporal multi-fused featuresrdquo Pattern Recognition vol 61pp 295ndash308 2017

[29] J Lafferty A McCallum and F C Pereira ldquoConditionalrandom fields probabilistic models for segmenting and la-beling sequence datardquo in Proceedings of the 18th InternationalConference on Machine Learning (ICML-2001) pp 282ndash289Williamstown MA USA June-July 2001

[30] A Gunawardana M Mahajan A Acero and J C PlattldquoHidden conditional random fields for phone classificationrdquoin Proceedings of the INTERSPEECH vol 2 pp 1117ndash1120Citeseer Lisbon Portugal September 2005

[31] L Yang Q Song Z Wang and M Jiang ldquoParsing r-cnn forinstance-level human analysisrdquo in Proceedings of the IEEEConference on Computer Vision and Pattern Recognitionpp 364ndash373 Long Beach CA USA June 2019

[32] X Liang Y Wei L Lin et al ldquoLearning to segment human bywatching youtuberdquo IEEE Transactions on Pattern Analysisand Machine Intelligence vol 39 no 7 pp 1462ndash1468 2016

[33] M M Requena ldquoHuman segmentation with convolutionalneural networksrdquo esis University of Alicante AlicanteSpain 2018

[34] H Gao S Chen and Z Zhang ldquoParts semantic segmentationaware representation learning for person re-identificationrdquoApplied Sciences vol 9 no 6 p 1239 2019

[35] C Peng S-L Lo J Huang and A C Tsoi ldquoHuman actionsegmentation based on a streaming uniform entropy slicemethodrdquo IEEE Access vol 6 pp 16958ndash16971 2018

[36] M H Siddiqi A M Khan and S-W Lee ldquoActive contourslevel set based still human body segmentation from depthimages for video-based activity recognitionrdquo KSII Trans-actions on Internet and Information Systems (TIIS) vol 7no 11 pp 2839ndash2852 2013

[37] M Siddiqi R Ali M Rana E-K Hong E Kim and S LeeldquoVideo-based human activity recognition using multilevelwavelet decomposition and stepwise linear discriminantanalysisrdquo Sensors vol 14 no 4 pp 6370ndash6392 2014

[38] S Althloothi M H Mahoor X Zhang and R M VoylesldquoHuman activity recognition using multi-features and mul-tiple kernel learningrdquo Pattern Recognition vol 47 no 5pp 1800ndash1812 2014

[39] A Jalal S Kamal and D Kim ldquoA depth video-based humandetection and activity recognition using multi-features andembedded hidden markov models for health care monitoringsystemsrdquo International Journal of Interactive Multimedia andArtificial Intelligence vol 4 no 4 p 54 2017

[40] T Dobhal V Shitole G omas and G Navada ldquoHumanactivity recognition using binary motion image and deeplearningrdquo Procedia Computer Science vol 58 pp 178ndash1852015

[41] S Zhang Z Wei J Nie L Huang S Wang and Z Li ldquoAreview on human activity recognition using vision-basedmethodrdquo Journal of Healthcare Engineering vol 2017 ArticleID 3090343 31 pages 2017

Computational Intelligence and Neuroscience 13

[42] Y Yang B Zhang L Yang C Chen and W Yang ldquoActionrecognition using completed local binary patterns and mul-tiple-class boosting classifierrdquo in Proceedings of the 2015 3rdIAPR Asian Conference on Pattern Recognition (ACPR) IEEEpp 336ndash340 Kuala Lumpur Malaysia November 2015

[43] W Lin M T Sun R Poovandran and Z Zhang ldquoHumanactivity recognition for video surveillancerdquo in Proceedings ofthe 2008 IEEE International Symposium on Circuits andSystems pp 2737ndash2740 IEEE Seattle WA USA May 2008

[44] H Permuter J Francos and I Jermyn ldquoA study of Gaussianmixture models of color and texture features for imageclassification and segmentationrdquo Pattern Recognition vol 39no 4 pp 695ndash706 2006

[45] M Fiaz and B Ijaz ldquoVision based human activity trackingusing artificial neural networksrdquo in Proceedings of the 2010International Conference on Intelligent and Advanced Systemspp 1ndash5 IEEE Kuala Lumpur Malaysia June 2010

[46] H Foroughi A Naseri A Saberi and H S Yazdi ldquoAneigenspace-based approach for human fall detection usingintegrated time motion image and neural networkrdquo in Pro-ceedings of the 2008 9th International Conference on SignalProcessing pp 1499ndash1503 IEEE Beijing China October2008

[47] A K B Sonali ldquoHuman action recognition using supportvector machine and k-nearest neighborrdquo InternationalJournal of Engineering and Technical Research vol 3 no 4pp 423ndash428 2015

[48] V Swarnambigai ldquoAction recognition using ami and supportvector machinerdquo International Journal of Computer Scienceand Technology vol 5 no 4 pp 175ndash179 2014

[49] D Das and S Saharia ldquoHuman gait analysis and recognitionusing support vector machinesrdquo International Journal ofComputer Science and Information Technology vol 6 no 52004

[50] K G M Chathuramali and R Rodrigo ldquoFaster human activityrecognition with SVMrdquo in Proceedings of the InternationalConference on Advances in ICT for Emerging Regions(ICTer2012) pp 197ndash203 IEEE Colombo Sri Lanka De-cember 2012

[51] M Z Uddin D-H Kim J T Kim and T-S Kim ldquoAn indoorhuman activity recognition system for smart home using localbinary pattern features with hidden markov modelsrdquo Indoorand Built Environment vol 22 no 1 pp 289ndash298 2013

[52] M Z Uddin J Lee and T-S Kim ldquoIndependent shapecomponent-based human activity recognition via hiddenMarkov modelrdquo Applied Intelligence vol 33 no 2 pp 193ndash206 2010

[53] S Kumar and M Hebert ldquoDiscriminative Fields for ModelingSpatial Dependencies in Natural Imagesrdquo in Proceedings of theNIPS Vancouver Canada December 2003

[54] A Quattoni S Wang L-P Morency M Collins andT Darrell ldquoHidden conditional random fieldsrdquo IEEETransactions on Pattern Analysis and Machine Intelligencevol 29 no 10 pp 1848ndash1852 2007

[55] M Mahajan A Gunawardana and A Acero ldquoTraining al-gorithms for hidden conditional random fieldsrdquo in Pro-ceedings of the 2006 IEEE International Conference onAcoustics Speed and Signal Processing vol 1 IEEE ToulouseFrance May 2006

[56] S Reiter B Schuller and G Rigoll ldquoHidden conditionalrandom fields for meeting segmentationrdquo in Proceedings ofthe Multimedia and Expo 2007 IEEE International Confer-ence pp 639ndash642 IEEE Beijing China July 2007

[57] L Gorelick M Blank E Shechtman M Irani and R BasrildquoActions as space-time shapesrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 29 no 12 pp 2247ndash2253 2007

[58] I Laptev M Marszalek C Schmid and B RozenfeldldquoLearning realistic human actions from moviesrdquo in Pro-ceedings of the 2008 IEEE Conference on Computer Vision andPattern Recognition pp 1ndash8 IEEE Anchorage AK USAJune 2008

[59] K Soomro and A R Zamir ldquoAction recognition in realisticsports videosrdquo in Computer Vision in Sports pp 181ndash208Springer Berlin Germany 2014

[60] D Weinland E Boyer and R Ronfard ldquoAction recognitionfrom arbitrary views using 3D exemplarsrdquo in Proceedings ofthe 2007 IEEE 11th International Conference on ComputerVision pp 1ndash7 IEEE Rio de Janeiro Brazil October 2007

[61] M Siddiqi S Lee Y-K Lee A Khan and P Truc ldquoHier-archical recognition scheme for human facial expressionrecognition systemsrdquo Sensors vol 13 no 12 pp 16682ndash167132013

[62] M Elmezain and A Al-Hamadi ldquoVision-based human ac-tivity recognition using ldcrfsrdquo International Arab Journal ofInformation Technology (IAJIT) vol 15 no 3 pp 389ndash3952018

[63] A Roy B Banerjee and V Murino ldquoA novel dictionarylearning based multiple instance learning approach to actionrecognition from videosrdquo in Proceedings of the 6th In-ternational Conference on Pattern Recognition Applicationsand Methods pp 519ndash526 Porto Portugal February 2017

[64] A P Sirat ldquoActions as space-time shapesrdquo InternationalJournal of Application or Innovation in Engineering andManagement vol 7 no 8 pp 49ndash54 2018

14 Computational Intelligence and Neuroscience

Computer Games Technology

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

Advances in

FuzzySystems

Hindawiwwwhindawicom

Volume 2018

International Journal of

ReconfigurableComputing

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

thinspArtificial Intelligence

Hindawiwwwhindawicom Volumethinsp2018

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawiwwwhindawicom Volume 2018

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Engineering Mathematics

International Journal of

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Computational Intelligence and Neuroscience

Hindawiwwwhindawicom Volume 2018

Mathematical Problems in Engineering

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Hindawiwwwhindawicom Volume 2018

Human-ComputerInteraction

Advances in

Hindawiwwwhindawicom Volume 2018

Scientic Programming

Submit your manuscripts atwwwhindawicom

Page 7: Human Activity Recognition Using Gaussian Mixture Hidden …downloads.hindawi.com/journals/cin/2019/8590560.pdf · 2019-08-22 · ResearchArticle Human Activity Recognition Using

this analysis the real-time computation can be carried out at ahigher speed resulting in an enormous decrease in the com-putational time compared to a known analysis approach eoverall workflow of the proposed model is shown in Figure 2

4 Model Validation

41 Datasets Used In this work we employed four open-source standard action datasets like Weizmann actiondatasets [57] KTH action dataset [58] UCF sports dataset[59] and IXMAS action dataset [60] for corroborating theproposed HCRF model performance All the datasets areexplained below

411 Weizmann Action Dataset is dataset consisted of10 actions such as bending running walking skippingplace jumping side movement jumping forward two handwaving and one hand waving that were performed by total9 subjects is dataset comprised of 90 video clips with

average of 15 frames per clip where the frame size is144 times180

412 KTH Action Dataset KTH dataset employed for ac-tivity recognition comprised of 25 subjects who performed 6activities like running walking boxing jogging hand-clapping and hand waving in four distinctive scenariosUsing a static camera in the homogenous background atotal of 2391 sequences were taken with a frame size of160times120

413 UCF Sports Dataset In this dataset there were 182videos which were evaluated by n-fold cross-validation ruleis dataset has been taken from different sports activities inbroadcast television channels Some of the videos had highintraclass similarities is dataset was also collected using astatic camera is dataset covers 9 activities like runningdiving lifting golf swinging skating kicking walking

Table 2 Confusion matrix of the proposed recognition model using KTH action dataset (unit )

Activities Walking Jogging Running Boxing Hand-wave HandclapWalking 100 0 0 0 0 0Jogging 0 98 1 1 0 0Running 2 1 95 1 1 0Boxing 0 2 1 97 0 0Hand-wave 1 0 1 0 98 0Handclap 0 0 1 0 0 99Average 9783

Table 1 Confusion matrix of the proposed recognition model using Weizmann action dataset (unit )

Activities Bend Jack Pjump Run Side Skip Walk Wave 1 Wave 2Bend 98 0 1 0 0 1 0 0 0Jack 1 96 0 0 2 0 1 0 0Pjump 0 1 97 0 1 0 0 1 0Run 0 0 0 99 0 0 0 1 0Side 1 2 0 0 95 0 0 2 0Skip 0 0 0 0 0 100 0 0 0Walk 1 0 0 1 0 1 96 0 1Wave 1 0 1 1 0 0 0 0 98 0Wave 2 0 1 0 0 1 1 1 1 95Average 9711

Table 3 Confusion matrix of the proposed recognition model using UCF sports dataset (unit )

Activities Diving GS Kicking Lifting HBR Run Skating BS WalkDiving 95 1 2 1 0 1 0 0 0GS 1 94 0 0 2 1 1 1 0Kicking 0 2 98 0 0 0 0 0 0Lifting 1 1 1 94 1 0 0 1 1HBR 0 0 2 0 96 1 0 1 0Running 0 3 0 0 0 97 0 0 0Skating 1 0 1 1 0 1 95 0 1BS 0 0 0 1 0 1 0 97 1Walking 0 0 0 0 0 0 0 0 100Average 9622GS golf swinging HBR horseback riding and BS baseball swinging

Computational Intelligence and Neuroscience 7

Table 4 Confusion matrix of the proposed recognition model using IXMAS action dataset (unit )

Activities CA SD GU TA Walk Wave Punch KickCA 97 0 0 1 2 0 0 0SD 0 99 1 0 0 0 0 0GU 1 2 94 3 0 0 0 0TA 0 0 1 95 2 1 1 0Walk 0 1 1 0 98 0 0 0Wave 0 0 2 0 1 97 0 0Punch 0 1 0 1 0 2 96 0Kick 0 0 0 0 1 0 0 99Average 9688CA cross arm SD sit down GU get up and TA turn around

Table 5 Classification results of the proposed system on Weizmann action dataset (A) using ANN (B) using SVM (C) using HMM and(D) using existing HCRF [30] while removing the proposed HCRF model (unit )

Activities Bend Jack Pjump Run Side Skip Walk Wave 1 Wave 2(A)Bend 70 4 5 3 3 2 5 5 3Jack 4 68 3 6 7 2 3 3 4Pjump 2 4 75 6 3 2 2 2 4Run 4 2 3 72 6 2 5 3 3Side 5 3 5 4 65 6 4 6 2Skip 4 6 4 3 5 67 3 2 6Walk 4 2 4 7 3 4 70 3 3Wave 1 2 1 3 3 4 5 7 71 4Wave 2 2 5 3 6 4 4 3 5 68Average 6955(B)Bend 69 3 4 4 6 4 2 3 5Jack 2 72 2 3 4 3 5 4 5Pjump 1 4 75 2 4 5 4 2 3Run 2 4 3 78 2 2 4 2 3Side 2 4 5 3 70 4 3 5 4Skip 2 1 3 2 4 80 3 3 2Walk 2 0 3 4 3 2 82 1 3Wave 1 2 2 3 4 3 2 3 77 4Wave 2 1 2 1 2 3 1 3 4 83Average 7622(C)Bend 82 3 0 2 2 3 1 5 2Jack 3 80 1 2 3 2 3 4 2Pjump 3 4 85 3 0 0 1 2 2Run 5 4 2 79 0 2 1 3 4Side 0 1 5 4 81 3 1 2 3Skip 3 1 2 2 3 88 0 0 1Walk 0 2 3 2 1 2 83 3 4Wave 1 1 3 2 2 4 2 3 78 5Wave 2 1 2 2 2 2 3 1 0 87Average 8256(D)Bend 80 2 3 1 4 0 5 2 3Jack 1 88 0 2 0 3 2 3 1Pjump 0 2 90 1 0 3 0 2 2Run 2 1 2 85 2 3 0 0 5Side 4 1 2 3 80 4 1 2 3Skip 1 4 0 5 1 84 0 3 2Walk 2 1 0 0 1 2 89 2 3Wave 1 3 0 1 2 0 2 0 91 1Wave 2 4 1 3 0 2 3 0 2 85Average 8578

8 Computational Intelligence and Neuroscience

horseback riding and baseball swinging Each frame has asize of 720times 480

414 IXMAS Action Dataset IXMAS (INRIA Xmas motionacquisition sequences) dataset comprised of 13 activity classeswhich were performed by 11 actors each 3 times Every actoropted a free orientation as well as position e dataset hasprovided annotated silhouettes for each person For our ex-periments we have selected only 8 action classes like walkcross arms punch turn around sit down wave get up andkick IXMAS dataset is a multiview dataset for a view-invarianthuman activity recognition where each frame has a size of390times 291 is dataset has a major occlusion and that maycause misclassification therefore we utilized global histogramequalization [61] in order to resolve the occlusion issue

42 Setup For a comprehensive validation we carried outthe following set of experiments executed using Matlab

(i) e first experiment was conducted on each datasetseparately in order to show the performance of theproposed model In this experiment we employed10-fold cross-validation rule which means that data

from 9 subjects were utilized for training data whilethe data from one subject was picked as a testingdata e procedure was reiterated for 10 timesprovided each subject data is utilized for bothtraining and testing

(ii) e second experiment was conducted in the ab-sence of the proposed recognition model on all thefour datasets that will show the importance of thedeveloped model For this purpose we used theexisting eminent classifiers like SVM ANN HMMand existing HCRF [30] as a recognition modelrather than utilizing the proposed HCRF model

(iii) e third experiment was conducted to show theperformance of the proposed approach against thestate-of-the-art methods

(iv) In the last experiment the computational com-plexity of the proposed HCRF model was comparedwith forwardbackward algorithms

5 Results and Discussion

51 First Experiment As described before this experimentvalidates the performance of the proposed recognition model

Table 6 Classification results of the proposed system on KTH action dataset (A) using ANN (B) using SVM (C) using HMM and (D)using existing HCRF [30] while removing the proposed HCRF model (unit )

Activities Walking Jogging Running Boxing Hand-wave Handclap(A)Walking 79 5 6 4 3 3Jogging 3 81 5 3 4 4Running 6 4 77 5 5 3Boxing 6 7 6 69 5 7Hand-wave 4 7 5 5 73 6Handclap 4 6 5 4 6 75Average 7566(B)Walking 82 2 3 5 4 4Jogging 3 86 2 3 2 4Running 5 3 80 4 5 3Boxing 5 3 3 79 4 6Hand-wave 1 4 3 3 89 0Handclap 3 5 2 4 3 83Average 8317(C)Walking 86 3 2 4 2 3Jogging 0 88 3 2 4 3Running 0 3 90 0 4 3Boxing 3 0 4 92 1 0Hand-wave 1 3 2 2 91 1Handclap 1 3 4 1 2 89Average 8933(D)Walking 90 3 0 3 4 0Jogging 2 88 2 3 3 2Running 4 2 92 0 0 2Boxing 1 3 2 91 3 0Hand-wave 0 1 3 2 93 1Handclap 1 3 2 4 3 87Average 9017

Computational Intelligence and Neuroscience 9

on an individual dataset e overall results are shown inTables 1 (using Weizmann dataset) 2 (using KTH dataset) 3(usingUCF sports dataset) and 4 (using IXMAS) respectively

As observed from Tables 1ndash4 the proposed recognitionmodel constantly obtained higher recognition rates on in-dividual dataset is result shows the robustness of theproposed model which means that the model not onlyshowed better performance on one dataset but also showedbetter performance across multiple spontaneous datasets

52 Second Experiment As described before the secondexperiment was conducted in the absence of the proposed

recognition model to show the importance of the proposedmodel using all the four datasets For this purpose we usedthe existing eminent classifiers like SVM ANN HMM andexisting HCRF [30] as a recognition model rather thanutilizing the proposed HCRF model

Tables 5ndash8 show that when the proposed HCRF modelwas substituted with ANN SVM HMM and existing HCRF[30] the system failed to accomplish higher recognitionrates e better performance of the proposed HCRF modelis visualized in Tables 1ndash4 which show that the proposedHCRF model effectively fix the drawbacks of HMM andexisting HCRF that has been extensively utilized for se-quential HAR

Table 7 Classification results of the proposed system onUCF sports dataset (A) using ANN (B) using SVM (C) using HMM and (D) usingexisting HCRF [30] while removing the proposed HCRF model (unit )

Activities Diving GS Kicking Lifting HBR Run Skating BS Walk(A)Diving 68 4 2 5 6 6 4 3 2GS 2 71 2 4 5 4 6 3 3Kicking 3 4 70 3 5 4 2 3 6Lifting 5 4 3 65 5 6 4 6 2HBR 3 4 6 3 66 4 5 5 4Running 3 3 5 4 6 64 6 4 5Skating 2 5 4 5 3 4 69 3 5BS 4 2 5 3 4 6 5 67 4Walking 5 4 2 3 4 3 6 3 70Average 6778(B)Diving 71 4 2 3 5 6 3 2 4GS 3 77 2 4 3 2 5 2 2Kicking 4 2 74 4 5 3 2 3 3Lifting 5 6 3 69 4 3 5 3 2HBR 2 3 3 2 80 2 4 2 2Running 2 3 2 2 5 75 6 2 3Skating 2 1 2 3 4 4 78 2 4BS 3 4 6 3 4 2 3 70 5Walking 4 1 2 4 2 3 0 3 81Average 7500(C)Diving 79 3 2 2 3 4 3 2 2GS 0 83 2 4 3 2 1 3 2Kicking 1 2 85 1 3 3 2 3 0Lifting 3 0 2 82 3 2 4 2 2HBR 0 2 2 4 80 0 5 3 4Running 1 2 1 3 4 84 2 1 2Skating 2 0 3 4 0 1 86 3 1BS 1 1 1 2 0 3 0 88 4Walking 1 2 4 2 5 2 4 3 77Average 8267(D)Diving 90 3 0 1 0 2 2 1 1GS 3 84 2 1 3 1 3 2 1Kicking 3 4 85 0 0 2 3 1 2Lifting 1 2 1 89 1 1 1 2 2HBR 0 2 1 0 91 2 3 0 1Running 2 3 1 2 3 80 4 2 3Skating 2 4 1 2 3 0 84 4 0BS 2 1 1 2 1 0 3 88 2Walking 0 2 1 1 0 1 4 0 91Average 8689GS golf swinging HBR horseback riding BS baseball swinging

10 Computational Intelligence and Neuroscience

53 ird Experiment In this experiment a comparativeanalysis was made between the state-of-the-art methods andthe proposed model All of these approaches were imple-mented by the instructions provided in their particular ar-ticles A 10-fold cross-validation rule was employed on eachdataset as explained in Section 4 e average classicationresults of the existing methods along with the proposedmethod across dierent datasets are summarized in Table 9

Table 8 Classication results of the proposed system on IXMASaction dataset (A) using ANN (B) using SVM (C) using HMMand (D) using existing HCRF [30] while removing the proposedHCRF model (unit )

Activities CA SD GU TA Walk Wave Punch Kick(A)CA 65 5 7 6 5 4 3 5SD 5 72 4 3 5 4 3 4GU 4 3 75 5 3 2 4 4TA 6 7 4 68 3 5 4 3Walk 3 4 5 4 70 6 4 4Wave 4 6 5 3 4 71 3 4Punch 3 5 5 6 7 3 67 4Kick 4 5 4 6 5 4 3 69Average 6962(B)CA 77 3 4 2 2 3 5 4SD 3 79 3 2 4 5 2 2GU 5 6 69 3 4 5 4 4TA 2 3 2 80 4 4 3 2Walk 3 5 4 2 71 5 6 4Wave 2 6 3 5 4 73 4 3Punch 1 5 3 4 1 3 81 2Kick 3 6 7 4 3 5 4 68Average 7475(C)CA 79 3 4 1 2 4 3 4SD 1 84 3 2 3 1 4 2GU 0 1 88 1 2 3 2 3TA 5 2 3 79 2 3 2 4Walk 1 0 3 1 90 2 3 0Wave 2 3 1 0 3 86 2 3Punch 1 0 2 3 0 4 89 1Kick 3 2 4 1 0 2 4 84Average 8477(D)CA 90 1 2 0 3 4 0 0SD 3 85 2 1 3 3 2 1GU 0 1 91 1 0 2 3 2TA 1 3 2 87 1 2 2 2Walk 1 0 3 1 89 2 1 3Wave 0 2 1 0 2 90 3 1Punch 1 4 2 1 2 3 84 3Kick 1 2 4 1 3 2 4 83Average 8738CA cross arm SD sit down GU get up TA turn around

Table 9 Weighted average recognition rates of the proposedmethod with the existing state-of-the-art methods (unit )

State-of-the-art works Average classicationrates

Standarddeviation

GMM 633 plusmn27SVM 675 plusmn44HMM 828 plusmn38Embedded HMM 859 plusmn19[62] 921 plusmn32[63] 843 plusmn49[18] 936 plusmn27[19] 930 plusmn16[22] 927 plusmn25[64] 801 plusmn32Proposed method 972 plusmn28

0

100

200

300

400

500

600

Exec

utio

n tim

e (s)

1 2 3 4 5 6 7 8Number of states

ForwardbackwardProposed HCRF model

(a)Ex

ecut

ion

time (

s)

1 2 3 4 5 6 7 8

ForwardbackwardProposed HCRF model

0

2

4

6

8

10

12

14

16

Number of mixtures

(b)

Figure 3 An illustration of gradient computational time (equation(30)) of the previous forward and backward algorithms and theproposed HCRF model (a)Q 1 minus 5 M 5 T 90 and (b)Q 5 M 1 minus 5 T 90

Computational Intelligence and Neuroscience 11

It is obvious from Table 9 that the proposed methodshowed a significant performance against the existing state-of-the-art methods erefore the proposed method accu-rately and robustly recognizes the human activities usingdifferent video data

54 Fourth Experiment In this experiment we have pre-sented the computational complexity that is also one of thecontributions in this paper e implementations of theprevious HCRF are available in literature which calculatethe gradients by reiterating the forward and backwardtechniques while the proposed HCRF model executes themonce only and cashes the outcomes for the later use From(21) and (22) it is clear that the forward or backwardtechnique has a complexity ofO(TQ2M) where Trepresentsthe input sequence lengthQ represents the number of statesand M indicates the number of mixtures e proposedHCRFmodel however requires a full complexity of O(TM)

to calculate gradients as can be seen from (22)ndash(29)Figure 3 shows a comparison of the execution time when

the gradients are computed by the forward (or backward)algorithm and by our proposed method e computationaltime is calculated by running Matlab R2013a with thespecification of Intelreg Pentiumreg Coretrade i7-6700 (34GHz)with a RAM capacity of 16GB

6 Conclusion

In healthcare and telemedicine the human activity rec-ognition (HAR) can be best explained by helping physi-cally disable personsrsquo scenario A paralyzed patient withhalf of the body critically attacked by paralysis is com-pletely unable to perform their daily exercises e doctorsrecommend specific activities to get better improvement intheir health So for this purpose the doctors need a humanactivity recognition (HAR) system through which they canmonitor the patientsrsquo daily routines (activities) on a reg-ular basis

e accuracy of most of the HAR systems depends uponthe recognition modules For feature extraction and selec-tion modules we used some of the existing well-knownmethods while for the recognition module we proposed theusage of HCRF model which is capable of approximating acomplex distribution using a mixture of Gaussian densityfunctions e proposed model was assessed against fourpublicly available standard action datasets From our ex-periments it is obvious that the proposed full-covarianceGaussian density function showed a significant improve-ment in accuracy than the existing state-of-the-art methodsFurthermore we also proved that such improvement issignificant from statistical point of view by showing valuele02 of the comparison Similarly the complexity analysispoints out that the proposed computational method stronglydecreases the execution time for the hidden conditionalrandom field model

e ultimate goal of this study is to deploy the proposedmodel on smartphones Currently the proposed model isusing full-covariance matrix however this might be time

consuming especially when using on smartphones Using alightweight classifier such as K-nearest neighbor (K-NN)could be one possible solution But K-NN is very muchsensitive to environmental factor (like noise) erefore infuture we will try to investigate further research to reducethe time and sustain the same recognition rate whenemploying on smartphones in real environment

Data Availability

e data used to support the findings of this study areavailable from the corresponding author upon request

Conflicts of Interest

e authors declare there are no conflicts of interest

Acknowledgments

is study was supported by the Jouf University SakakaAljouf Kingdom of Saudi Arabia under the registration no39791

References

[1] Z Chang J Cao and Y Zhang ldquoA novel image segmentationapproach for wood plate surface defect classification throughconvex optimizationrdquo Journal of Forestry Research vol 29no 6 pp 1789ndash1795 2018

[2] Z S Abdallah M M Gaber B Srinivasan andS Krishnaswamy ldquoAdaptive mobile activity recognitionsystemwith evolving data streamsrdquoNeurocomputing vol 150pp 304ndash317 2015

[3] M S Bakli M A Sakr and T H A Soliman ldquoA spatio-temporal algebra in Hadoop for moving objectsrdquo Geo-SpatialInformation Science vol 21 no 2 pp 102ndash114 2018

[4] W Zhao L Yan and Y Zhang ldquoGeometric-constrainedmulti-view image matching method based on semi-globaloptimizationrdquo Geo-Spatial Information Science vol 21 no 2pp 115ndash126 2018

[5] H Gao X Zhang J Zhao and D Li ldquoTechnology of in-telligent driving radar perception based on driving brainrdquoCAAI Transactions on Intelligence Technology vol 2 no 3pp 93ndash100 2017

[6] Y Wang X Jiang R Cao and X Wang ldquoRobust indoorhuman activity recognition using wireless signalsrdquo Sensorsvol 15 no 7 pp 17195ndash17208 2015

[7] J Wang Z Liu Y Wu and J Yuan ldquoMining actionlet ensemblefor action recognition with depth camerasrdquo in Proceedings of theIEEE Conference on Computer Vision and Pattern Recognitionpp 1290ndash1297 IEEE Providence RI USA June 2012

[8] S Karungaru M Daikoku and K Terada ldquoMulti camerasbased indoors human action recognition using fuzzy rulesrdquoJournal of Pattern Recognition Research vol 10 no 1pp 61ndash74 2015

[9] S Shukri L M Kamarudin and M H F Rahiman ldquoDevice-free localization for human activity monitoringrdquo in IntelligentVideo Surveillance IntechOpen London UK 2018

[10] L Fiore D Fehr R Bodor A Drenner G Somasundaramand N Papanikolopoulos ldquoMulti-camera human activitymonitoringrdquo Journal of Intelligent and Robotic Systemsvol 52 no 1 pp 5ndash43 2008

12 Computational Intelligence and Neuroscience

[11] C Zhang and Y Tian ldquoRGB-D camera-based daily livingactivity recognitionrdquo Journal of Computer Vision and ImageProcessing vol 2 no 4 pp 1ndash7 2012

[12] M Leo T DrsquoOrazio and P Spagnolo ldquoHuman activityrecognition for automatic visual surveillance of wide areasrdquo inProceedings of the ACM 2nd International Workshop on VideoSurveillance and Sensor Networks pp 124ndash130 ACM NewYork NY USA October 2004

[13] C-B Jin S Li and H Kim ldquoReal-time action detection invideo surveillance using sub-action descriptor with multi-CNNrdquo 2017 httpsarxivorgabs171003383

[14] W Niu J Long D Han and Y-F Wang ldquoHuman activitydetection and recognition for video surveillancerdquo in Pro-ceedings of the 2004 IEEE International Conference on Mul-timedia and Expo (ICME) (IEEE Cat No 04TH8763) vol 1pp 719ndash722 IEEE Taipei Taiwan June 2004

[15] Q Li M Zheng F Li J Wang Y-A Geng and H JiangldquoRetinal image segmentation using double-scale non-linearthresholding on vessel support regionsrdquo CAAI Transactionson Intelligence Technology vol 2 no 3 pp 109ndash115 2017

[16] L M G Fonseca L M Namikawa and E F CastejonldquoDigital image processing in remote sensingrdquo in Proceedingsof the 2009 Tutorials of the XXII Brazilian Symposium onComputer Graphics and Image Processing pp 59ndash71 IEEERio de Janeiro Brazil October 2009

[17] K Buys C Cagniart A Baksheev T De Laet J De Schutterand C Pantofaru ldquoAn adaptable system for RGB-D basedhuman body detection and pose estimationrdquo Journal of VisualCommunication and Image Representation vol 25 no 1pp 39ndash52 2014

[18] M Sharif M A Khan T Akram M Y Javed T Saba andA Rehman ldquoA framework of human detection and actionrecognition based on uniform segmentation and combinationof Euclidean distance and joint entropy-based features se-lectionrdquo EURASIP Journal on Image and Video Processingvol 2017 no 1 p 89 2017

[19] K-P Chou M Prasad D Wu et al ldquoRobust feature-basedautomated multi-view human action recognition systemrdquoIEEE Access vol 6 pp 15283ndash15296 2018

[20] M Ullah H Ullah and I M Alseadonn ldquoHuman actionrecognition in videos using stable featuresrdquo Signal and ImageProcessing An International Journal vol 8 no 6 pp 1ndash102017

[21] M Sharif M A Khan F Zahid J H Shah and T AkramldquoHuman action recognition a framework of statisticalweighted segmentation and rank correlation-based selectionrdquoPattern Analysis and Applications pp 1ndash14 2019

[22] J Zang L Wang Z Liu Q Zhang G Hua and N ZhengldquoAttention-based temporal weighted convolutional neuralnetwork for action recognitionrdquo in Proceedings of the IFIPInternational Conference on Artificial Intelligence Applicationsand Innovations pp 97ndash108 Springer Rhodes Greece May2018

[23] V-D Hoang ldquoMultiple classifier-based spatiotemporal fea-tures for living activity predictionrdquo Journal of Informationand Telecommunication vol 1 no 1 pp 100ndash112 2017

[24] S Dharmalingam and A Palanisamy ldquoVector space basedaugmented structural kinematic feature descriptor for humanactivity recognition in videosrdquo ETRI Journal vol 40 no 4pp 499ndash510 2018

[25] D Liu Y Yan M-L Shyu G Zhao and M Chen ldquoSpatio-temporal analysis for human action detection and recognitionin uncontrolled environmentsrdquo International Journal of

Multimedia Data Engineering and Management vol 6 no 1pp 1ndash18 2015

[26] C A Ronao and S-B Cho ldquoHuman activity recognition withsmartphone sensors using deep learning neural networksrdquoExpert Systems with Applications vol 59 pp 235ndash244 2016

[27] A arwat H Mahdi M Elhoseny and A E HassanienldquoRecognizing human activity in mobile crowdsensing envi-ronment using optimized k-NN algorithmrdquo Expert Systemswith Applications vol 107 pp 32ndash44 2018

[28] A Jalal Y-H Kim Y-J Kim S Kamal and D Kim ldquoRobusthuman activity recognition from depth video using spatio-temporal multi-fused featuresrdquo Pattern Recognition vol 61pp 295ndash308 2017

[29] J Lafferty A McCallum and F C Pereira ldquoConditionalrandom fields probabilistic models for segmenting and la-beling sequence datardquo in Proceedings of the 18th InternationalConference on Machine Learning (ICML-2001) pp 282ndash289Williamstown MA USA June-July 2001

[30] A Gunawardana M Mahajan A Acero and J C PlattldquoHidden conditional random fields for phone classificationrdquoin Proceedings of the INTERSPEECH vol 2 pp 1117ndash1120Citeseer Lisbon Portugal September 2005

[31] L Yang Q Song Z Wang and M Jiang ldquoParsing r-cnn forinstance-level human analysisrdquo in Proceedings of the IEEEConference on Computer Vision and Pattern Recognitionpp 364ndash373 Long Beach CA USA June 2019

[32] X Liang Y Wei L Lin et al ldquoLearning to segment human bywatching youtuberdquo IEEE Transactions on Pattern Analysisand Machine Intelligence vol 39 no 7 pp 1462ndash1468 2016

[33] M M Requena ldquoHuman segmentation with convolutionalneural networksrdquo esis University of Alicante AlicanteSpain 2018

[34] H Gao S Chen and Z Zhang ldquoParts semantic segmentationaware representation learning for person re-identificationrdquoApplied Sciences vol 9 no 6 p 1239 2019

[35] C Peng S-L Lo J Huang and A C Tsoi ldquoHuman actionsegmentation based on a streaming uniform entropy slicemethodrdquo IEEE Access vol 6 pp 16958ndash16971 2018

[36] M H Siddiqi A M Khan and S-W Lee ldquoActive contourslevel set based still human body segmentation from depthimages for video-based activity recognitionrdquo KSII Trans-actions on Internet and Information Systems (TIIS) vol 7no 11 pp 2839ndash2852 2013

[37] M Siddiqi R Ali M Rana E-K Hong E Kim and S LeeldquoVideo-based human activity recognition using multilevelwavelet decomposition and stepwise linear discriminantanalysisrdquo Sensors vol 14 no 4 pp 6370ndash6392 2014

[38] S Althloothi M H Mahoor X Zhang and R M VoylesldquoHuman activity recognition using multi-features and mul-tiple kernel learningrdquo Pattern Recognition vol 47 no 5pp 1800ndash1812 2014

[39] A Jalal S Kamal and D Kim ldquoA depth video-based humandetection and activity recognition using multi-features andembedded hidden markov models for health care monitoringsystemsrdquo International Journal of Interactive Multimedia andArtificial Intelligence vol 4 no 4 p 54 2017

[40] T Dobhal V Shitole G omas and G Navada ldquoHumanactivity recognition using binary motion image and deeplearningrdquo Procedia Computer Science vol 58 pp 178ndash1852015

[41] S Zhang Z Wei J Nie L Huang S Wang and Z Li ldquoAreview on human activity recognition using vision-basedmethodrdquo Journal of Healthcare Engineering vol 2017 ArticleID 3090343 31 pages 2017

Computational Intelligence and Neuroscience 13

[42] Y Yang B Zhang L Yang C Chen and W Yang ldquoActionrecognition using completed local binary patterns and mul-tiple-class boosting classifierrdquo in Proceedings of the 2015 3rdIAPR Asian Conference on Pattern Recognition (ACPR) IEEEpp 336ndash340 Kuala Lumpur Malaysia November 2015

[43] W Lin M T Sun R Poovandran and Z Zhang ldquoHumanactivity recognition for video surveillancerdquo in Proceedings ofthe 2008 IEEE International Symposium on Circuits andSystems pp 2737ndash2740 IEEE Seattle WA USA May 2008

[44] H Permuter J Francos and I Jermyn ldquoA study of Gaussianmixture models of color and texture features for imageclassification and segmentationrdquo Pattern Recognition vol 39no 4 pp 695ndash706 2006

[45] M Fiaz and B Ijaz ldquoVision based human activity trackingusing artificial neural networksrdquo in Proceedings of the 2010International Conference on Intelligent and Advanced Systemspp 1ndash5 IEEE Kuala Lumpur Malaysia June 2010

[46] H Foroughi A Naseri A Saberi and H S Yazdi ldquoAneigenspace-based approach for human fall detection usingintegrated time motion image and neural networkrdquo in Pro-ceedings of the 2008 9th International Conference on SignalProcessing pp 1499ndash1503 IEEE Beijing China October2008

[47] A K B Sonali ldquoHuman action recognition using supportvector machine and k-nearest neighborrdquo InternationalJournal of Engineering and Technical Research vol 3 no 4pp 423ndash428 2015

[48] V Swarnambigai ldquoAction recognition using ami and supportvector machinerdquo International Journal of Computer Scienceand Technology vol 5 no 4 pp 175ndash179 2014

[49] D Das and S Saharia ldquoHuman gait analysis and recognitionusing support vector machinesrdquo International Journal ofComputer Science and Information Technology vol 6 no 52004

[50] K G M Chathuramali and R Rodrigo ldquoFaster human activityrecognition with SVMrdquo in Proceedings of the InternationalConference on Advances in ICT for Emerging Regions(ICTer2012) pp 197ndash203 IEEE Colombo Sri Lanka De-cember 2012

[51] M Z Uddin D-H Kim J T Kim and T-S Kim ldquoAn indoorhuman activity recognition system for smart home using localbinary pattern features with hidden markov modelsrdquo Indoorand Built Environment vol 22 no 1 pp 289ndash298 2013

[52] M Z Uddin J Lee and T-S Kim ldquoIndependent shapecomponent-based human activity recognition via hiddenMarkov modelrdquo Applied Intelligence vol 33 no 2 pp 193ndash206 2010

[53] S Kumar and M Hebert ldquoDiscriminative Fields for ModelingSpatial Dependencies in Natural Imagesrdquo in Proceedings of theNIPS Vancouver Canada December 2003

[54] A Quattoni S Wang L-P Morency M Collins andT Darrell ldquoHidden conditional random fieldsrdquo IEEETransactions on Pattern Analysis and Machine Intelligencevol 29 no 10 pp 1848ndash1852 2007

[55] M Mahajan A Gunawardana and A Acero ldquoTraining al-gorithms for hidden conditional random fieldsrdquo in Pro-ceedings of the 2006 IEEE International Conference onAcoustics Speed and Signal Processing vol 1 IEEE ToulouseFrance May 2006

[56] S Reiter B Schuller and G Rigoll ldquoHidden conditionalrandom fields for meeting segmentationrdquo in Proceedings ofthe Multimedia and Expo 2007 IEEE International Confer-ence pp 639ndash642 IEEE Beijing China July 2007

[57] L Gorelick M Blank E Shechtman M Irani and R BasrildquoActions as space-time shapesrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 29 no 12 pp 2247ndash2253 2007

[58] I Laptev M Marszalek C Schmid and B RozenfeldldquoLearning realistic human actions from moviesrdquo in Pro-ceedings of the 2008 IEEE Conference on Computer Vision andPattern Recognition pp 1ndash8 IEEE Anchorage AK USAJune 2008

[59] K Soomro and A R Zamir ldquoAction recognition in realisticsports videosrdquo in Computer Vision in Sports pp 181ndash208Springer Berlin Germany 2014

[60] D Weinland E Boyer and R Ronfard ldquoAction recognitionfrom arbitrary views using 3D exemplarsrdquo in Proceedings ofthe 2007 IEEE 11th International Conference on ComputerVision pp 1ndash7 IEEE Rio de Janeiro Brazil October 2007

[61] M Siddiqi S Lee Y-K Lee A Khan and P Truc ldquoHier-archical recognition scheme for human facial expressionrecognition systemsrdquo Sensors vol 13 no 12 pp 16682ndash167132013

[62] M Elmezain and A Al-Hamadi ldquoVision-based human ac-tivity recognition using ldcrfsrdquo International Arab Journal ofInformation Technology (IAJIT) vol 15 no 3 pp 389ndash3952018

[63] A Roy B Banerjee and V Murino ldquoA novel dictionarylearning based multiple instance learning approach to actionrecognition from videosrdquo in Proceedings of the 6th In-ternational Conference on Pattern Recognition Applicationsand Methods pp 519ndash526 Porto Portugal February 2017

[64] A P Sirat ldquoActions as space-time shapesrdquo InternationalJournal of Application or Innovation in Engineering andManagement vol 7 no 8 pp 49ndash54 2018

14 Computational Intelligence and Neuroscience

Computer Games Technology

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

Advances in

FuzzySystems

Hindawiwwwhindawicom

Volume 2018

International Journal of

ReconfigurableComputing

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

thinspArtificial Intelligence

Hindawiwwwhindawicom Volumethinsp2018

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawiwwwhindawicom Volume 2018

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Engineering Mathematics

International Journal of

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Computational Intelligence and Neuroscience

Hindawiwwwhindawicom Volume 2018

Mathematical Problems in Engineering

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Hindawiwwwhindawicom Volume 2018

Human-ComputerInteraction

Advances in

Hindawiwwwhindawicom Volume 2018

Scientic Programming

Submit your manuscripts atwwwhindawicom

Page 8: Human Activity Recognition Using Gaussian Mixture Hidden …downloads.hindawi.com/journals/cin/2019/8590560.pdf · 2019-08-22 · ResearchArticle Human Activity Recognition Using

Table 4 Confusion matrix of the proposed recognition model using IXMAS action dataset (unit )

Activities CA SD GU TA Walk Wave Punch KickCA 97 0 0 1 2 0 0 0SD 0 99 1 0 0 0 0 0GU 1 2 94 3 0 0 0 0TA 0 0 1 95 2 1 1 0Walk 0 1 1 0 98 0 0 0Wave 0 0 2 0 1 97 0 0Punch 0 1 0 1 0 2 96 0Kick 0 0 0 0 1 0 0 99Average 9688CA cross arm SD sit down GU get up and TA turn around

Table 5 Classification results of the proposed system on Weizmann action dataset (A) using ANN (B) using SVM (C) using HMM and(D) using existing HCRF [30] while removing the proposed HCRF model (unit )

Activities Bend Jack Pjump Run Side Skip Walk Wave 1 Wave 2(A)Bend 70 4 5 3 3 2 5 5 3Jack 4 68 3 6 7 2 3 3 4Pjump 2 4 75 6 3 2 2 2 4Run 4 2 3 72 6 2 5 3 3Side 5 3 5 4 65 6 4 6 2Skip 4 6 4 3 5 67 3 2 6Walk 4 2 4 7 3 4 70 3 3Wave 1 2 1 3 3 4 5 7 71 4Wave 2 2 5 3 6 4 4 3 5 68Average 6955(B)Bend 69 3 4 4 6 4 2 3 5Jack 2 72 2 3 4 3 5 4 5Pjump 1 4 75 2 4 5 4 2 3Run 2 4 3 78 2 2 4 2 3Side 2 4 5 3 70 4 3 5 4Skip 2 1 3 2 4 80 3 3 2Walk 2 0 3 4 3 2 82 1 3Wave 1 2 2 3 4 3 2 3 77 4Wave 2 1 2 1 2 3 1 3 4 83Average 7622(C)Bend 82 3 0 2 2 3 1 5 2Jack 3 80 1 2 3 2 3 4 2Pjump 3 4 85 3 0 0 1 2 2Run 5 4 2 79 0 2 1 3 4Side 0 1 5 4 81 3 1 2 3Skip 3 1 2 2 3 88 0 0 1Walk 0 2 3 2 1 2 83 3 4Wave 1 1 3 2 2 4 2 3 78 5Wave 2 1 2 2 2 2 3 1 0 87Average 8256(D)Bend 80 2 3 1 4 0 5 2 3Jack 1 88 0 2 0 3 2 3 1Pjump 0 2 90 1 0 3 0 2 2Run 2 1 2 85 2 3 0 0 5Side 4 1 2 3 80 4 1 2 3Skip 1 4 0 5 1 84 0 3 2Walk 2 1 0 0 1 2 89 2 3Wave 1 3 0 1 2 0 2 0 91 1Wave 2 4 1 3 0 2 3 0 2 85Average 8578

8 Computational Intelligence and Neuroscience

horseback riding and baseball swinging Each frame has asize of 720times 480

414 IXMAS Action Dataset IXMAS (INRIA Xmas motionacquisition sequences) dataset comprised of 13 activity classeswhich were performed by 11 actors each 3 times Every actoropted a free orientation as well as position e dataset hasprovided annotated silhouettes for each person For our ex-periments we have selected only 8 action classes like walkcross arms punch turn around sit down wave get up andkick IXMAS dataset is a multiview dataset for a view-invarianthuman activity recognition where each frame has a size of390times 291 is dataset has a major occlusion and that maycause misclassification therefore we utilized global histogramequalization [61] in order to resolve the occlusion issue

42 Setup For a comprehensive validation we carried outthe following set of experiments executed using Matlab

(i) e first experiment was conducted on each datasetseparately in order to show the performance of theproposed model In this experiment we employed10-fold cross-validation rule which means that data

from 9 subjects were utilized for training data whilethe data from one subject was picked as a testingdata e procedure was reiterated for 10 timesprovided each subject data is utilized for bothtraining and testing

(ii) e second experiment was conducted in the ab-sence of the proposed recognition model on all thefour datasets that will show the importance of thedeveloped model For this purpose we used theexisting eminent classifiers like SVM ANN HMMand existing HCRF [30] as a recognition modelrather than utilizing the proposed HCRF model

(iii) e third experiment was conducted to show theperformance of the proposed approach against thestate-of-the-art methods

(iv) In the last experiment the computational com-plexity of the proposed HCRF model was comparedwith forwardbackward algorithms

5 Results and Discussion

51 First Experiment As described before this experimentvalidates the performance of the proposed recognition model

Table 6 Classification results of the proposed system on KTH action dataset (A) using ANN (B) using SVM (C) using HMM and (D)using existing HCRF [30] while removing the proposed HCRF model (unit )

Activities Walking Jogging Running Boxing Hand-wave Handclap(A)Walking 79 5 6 4 3 3Jogging 3 81 5 3 4 4Running 6 4 77 5 5 3Boxing 6 7 6 69 5 7Hand-wave 4 7 5 5 73 6Handclap 4 6 5 4 6 75Average 7566(B)Walking 82 2 3 5 4 4Jogging 3 86 2 3 2 4Running 5 3 80 4 5 3Boxing 5 3 3 79 4 6Hand-wave 1 4 3 3 89 0Handclap 3 5 2 4 3 83Average 8317(C)Walking 86 3 2 4 2 3Jogging 0 88 3 2 4 3Running 0 3 90 0 4 3Boxing 3 0 4 92 1 0Hand-wave 1 3 2 2 91 1Handclap 1 3 4 1 2 89Average 8933(D)Walking 90 3 0 3 4 0Jogging 2 88 2 3 3 2Running 4 2 92 0 0 2Boxing 1 3 2 91 3 0Hand-wave 0 1 3 2 93 1Handclap 1 3 2 4 3 87Average 9017

Computational Intelligence and Neuroscience 9

on an individual dataset e overall results are shown inTables 1 (using Weizmann dataset) 2 (using KTH dataset) 3(usingUCF sports dataset) and 4 (using IXMAS) respectively

As observed from Tables 1ndash4 the proposed recognitionmodel constantly obtained higher recognition rates on in-dividual dataset is result shows the robustness of theproposed model which means that the model not onlyshowed better performance on one dataset but also showedbetter performance across multiple spontaneous datasets

52 Second Experiment As described before the secondexperiment was conducted in the absence of the proposed

recognition model to show the importance of the proposedmodel using all the four datasets For this purpose we usedthe existing eminent classifiers like SVM ANN HMM andexisting HCRF [30] as a recognition model rather thanutilizing the proposed HCRF model

Tables 5ndash8 show that when the proposed HCRF modelwas substituted with ANN SVM HMM and existing HCRF[30] the system failed to accomplish higher recognitionrates e better performance of the proposed HCRF modelis visualized in Tables 1ndash4 which show that the proposedHCRF model effectively fix the drawbacks of HMM andexisting HCRF that has been extensively utilized for se-quential HAR

Table 7 Classification results of the proposed system onUCF sports dataset (A) using ANN (B) using SVM (C) using HMM and (D) usingexisting HCRF [30] while removing the proposed HCRF model (unit )

Activities Diving GS Kicking Lifting HBR Run Skating BS Walk(A)Diving 68 4 2 5 6 6 4 3 2GS 2 71 2 4 5 4 6 3 3Kicking 3 4 70 3 5 4 2 3 6Lifting 5 4 3 65 5 6 4 6 2HBR 3 4 6 3 66 4 5 5 4Running 3 3 5 4 6 64 6 4 5Skating 2 5 4 5 3 4 69 3 5BS 4 2 5 3 4 6 5 67 4Walking 5 4 2 3 4 3 6 3 70Average 6778(B)Diving 71 4 2 3 5 6 3 2 4GS 3 77 2 4 3 2 5 2 2Kicking 4 2 74 4 5 3 2 3 3Lifting 5 6 3 69 4 3 5 3 2HBR 2 3 3 2 80 2 4 2 2Running 2 3 2 2 5 75 6 2 3Skating 2 1 2 3 4 4 78 2 4BS 3 4 6 3 4 2 3 70 5Walking 4 1 2 4 2 3 0 3 81Average 7500(C)Diving 79 3 2 2 3 4 3 2 2GS 0 83 2 4 3 2 1 3 2Kicking 1 2 85 1 3 3 2 3 0Lifting 3 0 2 82 3 2 4 2 2HBR 0 2 2 4 80 0 5 3 4Running 1 2 1 3 4 84 2 1 2Skating 2 0 3 4 0 1 86 3 1BS 1 1 1 2 0 3 0 88 4Walking 1 2 4 2 5 2 4 3 77Average 8267(D)Diving 90 3 0 1 0 2 2 1 1GS 3 84 2 1 3 1 3 2 1Kicking 3 4 85 0 0 2 3 1 2Lifting 1 2 1 89 1 1 1 2 2HBR 0 2 1 0 91 2 3 0 1Running 2 3 1 2 3 80 4 2 3Skating 2 4 1 2 3 0 84 4 0BS 2 1 1 2 1 0 3 88 2Walking 0 2 1 1 0 1 4 0 91Average 8689GS golf swinging HBR horseback riding BS baseball swinging

10 Computational Intelligence and Neuroscience

53 ird Experiment In this experiment a comparativeanalysis was made between the state-of-the-art methods andthe proposed model All of these approaches were imple-mented by the instructions provided in their particular ar-ticles A 10-fold cross-validation rule was employed on eachdataset as explained in Section 4 e average classicationresults of the existing methods along with the proposedmethod across dierent datasets are summarized in Table 9

Table 8 Classication results of the proposed system on IXMASaction dataset (A) using ANN (B) using SVM (C) using HMMand (D) using existing HCRF [30] while removing the proposedHCRF model (unit )

Activities CA SD GU TA Walk Wave Punch Kick(A)CA 65 5 7 6 5 4 3 5SD 5 72 4 3 5 4 3 4GU 4 3 75 5 3 2 4 4TA 6 7 4 68 3 5 4 3Walk 3 4 5 4 70 6 4 4Wave 4 6 5 3 4 71 3 4Punch 3 5 5 6 7 3 67 4Kick 4 5 4 6 5 4 3 69Average 6962(B)CA 77 3 4 2 2 3 5 4SD 3 79 3 2 4 5 2 2GU 5 6 69 3 4 5 4 4TA 2 3 2 80 4 4 3 2Walk 3 5 4 2 71 5 6 4Wave 2 6 3 5 4 73 4 3Punch 1 5 3 4 1 3 81 2Kick 3 6 7 4 3 5 4 68Average 7475(C)CA 79 3 4 1 2 4 3 4SD 1 84 3 2 3 1 4 2GU 0 1 88 1 2 3 2 3TA 5 2 3 79 2 3 2 4Walk 1 0 3 1 90 2 3 0Wave 2 3 1 0 3 86 2 3Punch 1 0 2 3 0 4 89 1Kick 3 2 4 1 0 2 4 84Average 8477(D)CA 90 1 2 0 3 4 0 0SD 3 85 2 1 3 3 2 1GU 0 1 91 1 0 2 3 2TA 1 3 2 87 1 2 2 2Walk 1 0 3 1 89 2 1 3Wave 0 2 1 0 2 90 3 1Punch 1 4 2 1 2 3 84 3Kick 1 2 4 1 3 2 4 83Average 8738CA cross arm SD sit down GU get up TA turn around

Table 9 Weighted average recognition rates of the proposedmethod with the existing state-of-the-art methods (unit )

State-of-the-art works Average classicationrates

Standarddeviation

GMM 633 plusmn27SVM 675 plusmn44HMM 828 plusmn38Embedded HMM 859 plusmn19[62] 921 plusmn32[63] 843 plusmn49[18] 936 plusmn27[19] 930 plusmn16[22] 927 plusmn25[64] 801 plusmn32Proposed method 972 plusmn28

0

100

200

300

400

500

600

Exec

utio

n tim

e (s)

1 2 3 4 5 6 7 8Number of states

ForwardbackwardProposed HCRF model

(a)Ex

ecut

ion

time (

s)

1 2 3 4 5 6 7 8

ForwardbackwardProposed HCRF model

0

2

4

6

8

10

12

14

16

Number of mixtures

(b)

Figure 3 An illustration of gradient computational time (equation(30)) of the previous forward and backward algorithms and theproposed HCRF model (a)Q 1 minus 5 M 5 T 90 and (b)Q 5 M 1 minus 5 T 90

Computational Intelligence and Neuroscience 11

It is obvious from Table 9 that the proposed methodshowed a significant performance against the existing state-of-the-art methods erefore the proposed method accu-rately and robustly recognizes the human activities usingdifferent video data

54 Fourth Experiment In this experiment we have pre-sented the computational complexity that is also one of thecontributions in this paper e implementations of theprevious HCRF are available in literature which calculatethe gradients by reiterating the forward and backwardtechniques while the proposed HCRF model executes themonce only and cashes the outcomes for the later use From(21) and (22) it is clear that the forward or backwardtechnique has a complexity ofO(TQ2M) where Trepresentsthe input sequence lengthQ represents the number of statesand M indicates the number of mixtures e proposedHCRFmodel however requires a full complexity of O(TM)

to calculate gradients as can be seen from (22)ndash(29)Figure 3 shows a comparison of the execution time when

the gradients are computed by the forward (or backward)algorithm and by our proposed method e computationaltime is calculated by running Matlab R2013a with thespecification of Intelreg Pentiumreg Coretrade i7-6700 (34GHz)with a RAM capacity of 16GB

6 Conclusion

In healthcare and telemedicine the human activity rec-ognition (HAR) can be best explained by helping physi-cally disable personsrsquo scenario A paralyzed patient withhalf of the body critically attacked by paralysis is com-pletely unable to perform their daily exercises e doctorsrecommend specific activities to get better improvement intheir health So for this purpose the doctors need a humanactivity recognition (HAR) system through which they canmonitor the patientsrsquo daily routines (activities) on a reg-ular basis

e accuracy of most of the HAR systems depends uponthe recognition modules For feature extraction and selec-tion modules we used some of the existing well-knownmethods while for the recognition module we proposed theusage of HCRF model which is capable of approximating acomplex distribution using a mixture of Gaussian densityfunctions e proposed model was assessed against fourpublicly available standard action datasets From our ex-periments it is obvious that the proposed full-covarianceGaussian density function showed a significant improve-ment in accuracy than the existing state-of-the-art methodsFurthermore we also proved that such improvement issignificant from statistical point of view by showing valuele02 of the comparison Similarly the complexity analysispoints out that the proposed computational method stronglydecreases the execution time for the hidden conditionalrandom field model

e ultimate goal of this study is to deploy the proposedmodel on smartphones Currently the proposed model isusing full-covariance matrix however this might be time

consuming especially when using on smartphones Using alightweight classifier such as K-nearest neighbor (K-NN)could be one possible solution But K-NN is very muchsensitive to environmental factor (like noise) erefore infuture we will try to investigate further research to reducethe time and sustain the same recognition rate whenemploying on smartphones in real environment

Data Availability

e data used to support the findings of this study areavailable from the corresponding author upon request

Conflicts of Interest

e authors declare there are no conflicts of interest

Acknowledgments

is study was supported by the Jouf University SakakaAljouf Kingdom of Saudi Arabia under the registration no39791

References

[1] Z Chang J Cao and Y Zhang ldquoA novel image segmentationapproach for wood plate surface defect classification throughconvex optimizationrdquo Journal of Forestry Research vol 29no 6 pp 1789ndash1795 2018

[2] Z S Abdallah M M Gaber B Srinivasan andS Krishnaswamy ldquoAdaptive mobile activity recognitionsystemwith evolving data streamsrdquoNeurocomputing vol 150pp 304ndash317 2015

[3] M S Bakli M A Sakr and T H A Soliman ldquoA spatio-temporal algebra in Hadoop for moving objectsrdquo Geo-SpatialInformation Science vol 21 no 2 pp 102ndash114 2018

[4] W Zhao L Yan and Y Zhang ldquoGeometric-constrainedmulti-view image matching method based on semi-globaloptimizationrdquo Geo-Spatial Information Science vol 21 no 2pp 115ndash126 2018

[5] H Gao X Zhang J Zhao and D Li ldquoTechnology of in-telligent driving radar perception based on driving brainrdquoCAAI Transactions on Intelligence Technology vol 2 no 3pp 93ndash100 2017

[6] Y Wang X Jiang R Cao and X Wang ldquoRobust indoorhuman activity recognition using wireless signalsrdquo Sensorsvol 15 no 7 pp 17195ndash17208 2015

[7] J Wang Z Liu Y Wu and J Yuan ldquoMining actionlet ensemblefor action recognition with depth camerasrdquo in Proceedings of theIEEE Conference on Computer Vision and Pattern Recognitionpp 1290ndash1297 IEEE Providence RI USA June 2012

[8] S Karungaru M Daikoku and K Terada ldquoMulti camerasbased indoors human action recognition using fuzzy rulesrdquoJournal of Pattern Recognition Research vol 10 no 1pp 61ndash74 2015

[9] S Shukri L M Kamarudin and M H F Rahiman ldquoDevice-free localization for human activity monitoringrdquo in IntelligentVideo Surveillance IntechOpen London UK 2018

[10] L Fiore D Fehr R Bodor A Drenner G Somasundaramand N Papanikolopoulos ldquoMulti-camera human activitymonitoringrdquo Journal of Intelligent and Robotic Systemsvol 52 no 1 pp 5ndash43 2008

12 Computational Intelligence and Neuroscience

[11] C Zhang and Y Tian ldquoRGB-D camera-based daily livingactivity recognitionrdquo Journal of Computer Vision and ImageProcessing vol 2 no 4 pp 1ndash7 2012

[12] M Leo T DrsquoOrazio and P Spagnolo ldquoHuman activityrecognition for automatic visual surveillance of wide areasrdquo inProceedings of the ACM 2nd International Workshop on VideoSurveillance and Sensor Networks pp 124ndash130 ACM NewYork NY USA October 2004

[13] C-B Jin S Li and H Kim ldquoReal-time action detection invideo surveillance using sub-action descriptor with multi-CNNrdquo 2017 httpsarxivorgabs171003383

[14] W Niu J Long D Han and Y-F Wang ldquoHuman activitydetection and recognition for video surveillancerdquo in Pro-ceedings of the 2004 IEEE International Conference on Mul-timedia and Expo (ICME) (IEEE Cat No 04TH8763) vol 1pp 719ndash722 IEEE Taipei Taiwan June 2004

[15] Q Li M Zheng F Li J Wang Y-A Geng and H JiangldquoRetinal image segmentation using double-scale non-linearthresholding on vessel support regionsrdquo CAAI Transactionson Intelligence Technology vol 2 no 3 pp 109ndash115 2017

[16] L M G Fonseca L M Namikawa and E F CastejonldquoDigital image processing in remote sensingrdquo in Proceedingsof the 2009 Tutorials of the XXII Brazilian Symposium onComputer Graphics and Image Processing pp 59ndash71 IEEERio de Janeiro Brazil October 2009

[17] K Buys C Cagniart A Baksheev T De Laet J De Schutterand C Pantofaru ldquoAn adaptable system for RGB-D basedhuman body detection and pose estimationrdquo Journal of VisualCommunication and Image Representation vol 25 no 1pp 39ndash52 2014

[18] M Sharif M A Khan T Akram M Y Javed T Saba andA Rehman ldquoA framework of human detection and actionrecognition based on uniform segmentation and combinationof Euclidean distance and joint entropy-based features se-lectionrdquo EURASIP Journal on Image and Video Processingvol 2017 no 1 p 89 2017

[19] K-P Chou M Prasad D Wu et al ldquoRobust feature-basedautomated multi-view human action recognition systemrdquoIEEE Access vol 6 pp 15283ndash15296 2018

[20] M Ullah H Ullah and I M Alseadonn ldquoHuman actionrecognition in videos using stable featuresrdquo Signal and ImageProcessing An International Journal vol 8 no 6 pp 1ndash102017

[21] M Sharif M A Khan F Zahid J H Shah and T AkramldquoHuman action recognition a framework of statisticalweighted segmentation and rank correlation-based selectionrdquoPattern Analysis and Applications pp 1ndash14 2019

[22] J Zang L Wang Z Liu Q Zhang G Hua and N ZhengldquoAttention-based temporal weighted convolutional neuralnetwork for action recognitionrdquo in Proceedings of the IFIPInternational Conference on Artificial Intelligence Applicationsand Innovations pp 97ndash108 Springer Rhodes Greece May2018

[23] V-D Hoang ldquoMultiple classifier-based spatiotemporal fea-tures for living activity predictionrdquo Journal of Informationand Telecommunication vol 1 no 1 pp 100ndash112 2017

[24] S Dharmalingam and A Palanisamy ldquoVector space basedaugmented structural kinematic feature descriptor for humanactivity recognition in videosrdquo ETRI Journal vol 40 no 4pp 499ndash510 2018

[25] D Liu Y Yan M-L Shyu G Zhao and M Chen ldquoSpatio-temporal analysis for human action detection and recognitionin uncontrolled environmentsrdquo International Journal of

Multimedia Data Engineering and Management vol 6 no 1pp 1ndash18 2015

[26] C A Ronao and S-B Cho ldquoHuman activity recognition withsmartphone sensors using deep learning neural networksrdquoExpert Systems with Applications vol 59 pp 235ndash244 2016

[27] A arwat H Mahdi M Elhoseny and A E HassanienldquoRecognizing human activity in mobile crowdsensing envi-ronment using optimized k-NN algorithmrdquo Expert Systemswith Applications vol 107 pp 32ndash44 2018

[28] A Jalal Y-H Kim Y-J Kim S Kamal and D Kim ldquoRobusthuman activity recognition from depth video using spatio-temporal multi-fused featuresrdquo Pattern Recognition vol 61pp 295ndash308 2017

[29] J Lafferty A McCallum and F C Pereira ldquoConditionalrandom fields probabilistic models for segmenting and la-beling sequence datardquo in Proceedings of the 18th InternationalConference on Machine Learning (ICML-2001) pp 282ndash289Williamstown MA USA June-July 2001

[30] A Gunawardana M Mahajan A Acero and J C PlattldquoHidden conditional random fields for phone classificationrdquoin Proceedings of the INTERSPEECH vol 2 pp 1117ndash1120Citeseer Lisbon Portugal September 2005

[31] L Yang Q Song Z Wang and M Jiang ldquoParsing r-cnn forinstance-level human analysisrdquo in Proceedings of the IEEEConference on Computer Vision and Pattern Recognitionpp 364ndash373 Long Beach CA USA June 2019

[32] X Liang Y Wei L Lin et al ldquoLearning to segment human bywatching youtuberdquo IEEE Transactions on Pattern Analysisand Machine Intelligence vol 39 no 7 pp 1462ndash1468 2016

[33] M M Requena ldquoHuman segmentation with convolutionalneural networksrdquo esis University of Alicante AlicanteSpain 2018

[34] H Gao S Chen and Z Zhang ldquoParts semantic segmentationaware representation learning for person re-identificationrdquoApplied Sciences vol 9 no 6 p 1239 2019

[35] C Peng S-L Lo J Huang and A C Tsoi ldquoHuman actionsegmentation based on a streaming uniform entropy slicemethodrdquo IEEE Access vol 6 pp 16958ndash16971 2018

[36] M H Siddiqi A M Khan and S-W Lee ldquoActive contourslevel set based still human body segmentation from depthimages for video-based activity recognitionrdquo KSII Trans-actions on Internet and Information Systems (TIIS) vol 7no 11 pp 2839ndash2852 2013

[37] M Siddiqi R Ali M Rana E-K Hong E Kim and S LeeldquoVideo-based human activity recognition using multilevelwavelet decomposition and stepwise linear discriminantanalysisrdquo Sensors vol 14 no 4 pp 6370ndash6392 2014

[38] S Althloothi M H Mahoor X Zhang and R M VoylesldquoHuman activity recognition using multi-features and mul-tiple kernel learningrdquo Pattern Recognition vol 47 no 5pp 1800ndash1812 2014

[39] A Jalal S Kamal and D Kim ldquoA depth video-based humandetection and activity recognition using multi-features andembedded hidden markov models for health care monitoringsystemsrdquo International Journal of Interactive Multimedia andArtificial Intelligence vol 4 no 4 p 54 2017

[40] T Dobhal V Shitole G omas and G Navada ldquoHumanactivity recognition using binary motion image and deeplearningrdquo Procedia Computer Science vol 58 pp 178ndash1852015

[41] S Zhang Z Wei J Nie L Huang S Wang and Z Li ldquoAreview on human activity recognition using vision-basedmethodrdquo Journal of Healthcare Engineering vol 2017 ArticleID 3090343 31 pages 2017

Computational Intelligence and Neuroscience 13

[42] Y Yang B Zhang L Yang C Chen and W Yang ldquoActionrecognition using completed local binary patterns and mul-tiple-class boosting classifierrdquo in Proceedings of the 2015 3rdIAPR Asian Conference on Pattern Recognition (ACPR) IEEEpp 336ndash340 Kuala Lumpur Malaysia November 2015

[43] W Lin M T Sun R Poovandran and Z Zhang ldquoHumanactivity recognition for video surveillancerdquo in Proceedings ofthe 2008 IEEE International Symposium on Circuits andSystems pp 2737ndash2740 IEEE Seattle WA USA May 2008

[44] H Permuter J Francos and I Jermyn ldquoA study of Gaussianmixture models of color and texture features for imageclassification and segmentationrdquo Pattern Recognition vol 39no 4 pp 695ndash706 2006

[45] M Fiaz and B Ijaz ldquoVision based human activity trackingusing artificial neural networksrdquo in Proceedings of the 2010International Conference on Intelligent and Advanced Systemspp 1ndash5 IEEE Kuala Lumpur Malaysia June 2010

[46] H Foroughi A Naseri A Saberi and H S Yazdi ldquoAneigenspace-based approach for human fall detection usingintegrated time motion image and neural networkrdquo in Pro-ceedings of the 2008 9th International Conference on SignalProcessing pp 1499ndash1503 IEEE Beijing China October2008

[47] A K B Sonali ldquoHuman action recognition using supportvector machine and k-nearest neighborrdquo InternationalJournal of Engineering and Technical Research vol 3 no 4pp 423ndash428 2015

[48] V Swarnambigai ldquoAction recognition using ami and supportvector machinerdquo International Journal of Computer Scienceand Technology vol 5 no 4 pp 175ndash179 2014

[49] D Das and S Saharia ldquoHuman gait analysis and recognitionusing support vector machinesrdquo International Journal ofComputer Science and Information Technology vol 6 no 52004

[50] K G M Chathuramali and R Rodrigo ldquoFaster human activityrecognition with SVMrdquo in Proceedings of the InternationalConference on Advances in ICT for Emerging Regions(ICTer2012) pp 197ndash203 IEEE Colombo Sri Lanka De-cember 2012

[51] M Z Uddin D-H Kim J T Kim and T-S Kim ldquoAn indoorhuman activity recognition system for smart home using localbinary pattern features with hidden markov modelsrdquo Indoorand Built Environment vol 22 no 1 pp 289ndash298 2013

[52] M Z Uddin J Lee and T-S Kim ldquoIndependent shapecomponent-based human activity recognition via hiddenMarkov modelrdquo Applied Intelligence vol 33 no 2 pp 193ndash206 2010

[53] S Kumar and M Hebert ldquoDiscriminative Fields for ModelingSpatial Dependencies in Natural Imagesrdquo in Proceedings of theNIPS Vancouver Canada December 2003

[54] A Quattoni S Wang L-P Morency M Collins andT Darrell ldquoHidden conditional random fieldsrdquo IEEETransactions on Pattern Analysis and Machine Intelligencevol 29 no 10 pp 1848ndash1852 2007

[55] M Mahajan A Gunawardana and A Acero ldquoTraining al-gorithms for hidden conditional random fieldsrdquo in Pro-ceedings of the 2006 IEEE International Conference onAcoustics Speed and Signal Processing vol 1 IEEE ToulouseFrance May 2006

[56] S Reiter B Schuller and G Rigoll ldquoHidden conditionalrandom fields for meeting segmentationrdquo in Proceedings ofthe Multimedia and Expo 2007 IEEE International Confer-ence pp 639ndash642 IEEE Beijing China July 2007

[57] L Gorelick M Blank E Shechtman M Irani and R BasrildquoActions as space-time shapesrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 29 no 12 pp 2247ndash2253 2007

[58] I Laptev M Marszalek C Schmid and B RozenfeldldquoLearning realistic human actions from moviesrdquo in Pro-ceedings of the 2008 IEEE Conference on Computer Vision andPattern Recognition pp 1ndash8 IEEE Anchorage AK USAJune 2008

[59] K Soomro and A R Zamir ldquoAction recognition in realisticsports videosrdquo in Computer Vision in Sports pp 181ndash208Springer Berlin Germany 2014

[60] D Weinland E Boyer and R Ronfard ldquoAction recognitionfrom arbitrary views using 3D exemplarsrdquo in Proceedings ofthe 2007 IEEE 11th International Conference on ComputerVision pp 1ndash7 IEEE Rio de Janeiro Brazil October 2007

[61] M Siddiqi S Lee Y-K Lee A Khan and P Truc ldquoHier-archical recognition scheme for human facial expressionrecognition systemsrdquo Sensors vol 13 no 12 pp 16682ndash167132013

[62] M Elmezain and A Al-Hamadi ldquoVision-based human ac-tivity recognition using ldcrfsrdquo International Arab Journal ofInformation Technology (IAJIT) vol 15 no 3 pp 389ndash3952018

[63] A Roy B Banerjee and V Murino ldquoA novel dictionarylearning based multiple instance learning approach to actionrecognition from videosrdquo in Proceedings of the 6th In-ternational Conference on Pattern Recognition Applicationsand Methods pp 519ndash526 Porto Portugal February 2017

[64] A P Sirat ldquoActions as space-time shapesrdquo InternationalJournal of Application or Innovation in Engineering andManagement vol 7 no 8 pp 49ndash54 2018

14 Computational Intelligence and Neuroscience

Computer Games Technology

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

Advances in

FuzzySystems

Hindawiwwwhindawicom

Volume 2018

International Journal of

ReconfigurableComputing

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

thinspArtificial Intelligence

Hindawiwwwhindawicom Volumethinsp2018

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawiwwwhindawicom Volume 2018

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Engineering Mathematics

International Journal of

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Computational Intelligence and Neuroscience

Hindawiwwwhindawicom Volume 2018

Mathematical Problems in Engineering

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Hindawiwwwhindawicom Volume 2018

Human-ComputerInteraction

Advances in

Hindawiwwwhindawicom Volume 2018

Scientic Programming

Submit your manuscripts atwwwhindawicom

Page 9: Human Activity Recognition Using Gaussian Mixture Hidden …downloads.hindawi.com/journals/cin/2019/8590560.pdf · 2019-08-22 · ResearchArticle Human Activity Recognition Using

horseback riding and baseball swinging Each frame has asize of 720times 480

414 IXMAS Action Dataset IXMAS (INRIA Xmas motionacquisition sequences) dataset comprised of 13 activity classeswhich were performed by 11 actors each 3 times Every actoropted a free orientation as well as position e dataset hasprovided annotated silhouettes for each person For our ex-periments we have selected only 8 action classes like walkcross arms punch turn around sit down wave get up andkick IXMAS dataset is a multiview dataset for a view-invarianthuman activity recognition where each frame has a size of390times 291 is dataset has a major occlusion and that maycause misclassification therefore we utilized global histogramequalization [61] in order to resolve the occlusion issue

42 Setup For a comprehensive validation we carried outthe following set of experiments executed using Matlab

(i) e first experiment was conducted on each datasetseparately in order to show the performance of theproposed model In this experiment we employed10-fold cross-validation rule which means that data

from 9 subjects were utilized for training data whilethe data from one subject was picked as a testingdata e procedure was reiterated for 10 timesprovided each subject data is utilized for bothtraining and testing

(ii) e second experiment was conducted in the ab-sence of the proposed recognition model on all thefour datasets that will show the importance of thedeveloped model For this purpose we used theexisting eminent classifiers like SVM ANN HMMand existing HCRF [30] as a recognition modelrather than utilizing the proposed HCRF model

(iii) e third experiment was conducted to show theperformance of the proposed approach against thestate-of-the-art methods

(iv) In the last experiment the computational com-plexity of the proposed HCRF model was comparedwith forwardbackward algorithms

5 Results and Discussion

51 First Experiment As described before this experimentvalidates the performance of the proposed recognition model

Table 6 Classification results of the proposed system on KTH action dataset (A) using ANN (B) using SVM (C) using HMM and (D)using existing HCRF [30] while removing the proposed HCRF model (unit )

Activities Walking Jogging Running Boxing Hand-wave Handclap(A)Walking 79 5 6 4 3 3Jogging 3 81 5 3 4 4Running 6 4 77 5 5 3Boxing 6 7 6 69 5 7Hand-wave 4 7 5 5 73 6Handclap 4 6 5 4 6 75Average 7566(B)Walking 82 2 3 5 4 4Jogging 3 86 2 3 2 4Running 5 3 80 4 5 3Boxing 5 3 3 79 4 6Hand-wave 1 4 3 3 89 0Handclap 3 5 2 4 3 83Average 8317(C)Walking 86 3 2 4 2 3Jogging 0 88 3 2 4 3Running 0 3 90 0 4 3Boxing 3 0 4 92 1 0Hand-wave 1 3 2 2 91 1Handclap 1 3 4 1 2 89Average 8933(D)Walking 90 3 0 3 4 0Jogging 2 88 2 3 3 2Running 4 2 92 0 0 2Boxing 1 3 2 91 3 0Hand-wave 0 1 3 2 93 1Handclap 1 3 2 4 3 87Average 9017

Computational Intelligence and Neuroscience 9

on an individual dataset e overall results are shown inTables 1 (using Weizmann dataset) 2 (using KTH dataset) 3(usingUCF sports dataset) and 4 (using IXMAS) respectively

As observed from Tables 1ndash4 the proposed recognitionmodel constantly obtained higher recognition rates on in-dividual dataset is result shows the robustness of theproposed model which means that the model not onlyshowed better performance on one dataset but also showedbetter performance across multiple spontaneous datasets

52 Second Experiment As described before the secondexperiment was conducted in the absence of the proposed

recognition model to show the importance of the proposedmodel using all the four datasets For this purpose we usedthe existing eminent classifiers like SVM ANN HMM andexisting HCRF [30] as a recognition model rather thanutilizing the proposed HCRF model

Tables 5ndash8 show that when the proposed HCRF modelwas substituted with ANN SVM HMM and existing HCRF[30] the system failed to accomplish higher recognitionrates e better performance of the proposed HCRF modelis visualized in Tables 1ndash4 which show that the proposedHCRF model effectively fix the drawbacks of HMM andexisting HCRF that has been extensively utilized for se-quential HAR

Table 7 Classification results of the proposed system onUCF sports dataset (A) using ANN (B) using SVM (C) using HMM and (D) usingexisting HCRF [30] while removing the proposed HCRF model (unit )

Activities Diving GS Kicking Lifting HBR Run Skating BS Walk(A)Diving 68 4 2 5 6 6 4 3 2GS 2 71 2 4 5 4 6 3 3Kicking 3 4 70 3 5 4 2 3 6Lifting 5 4 3 65 5 6 4 6 2HBR 3 4 6 3 66 4 5 5 4Running 3 3 5 4 6 64 6 4 5Skating 2 5 4 5 3 4 69 3 5BS 4 2 5 3 4 6 5 67 4Walking 5 4 2 3 4 3 6 3 70Average 6778(B)Diving 71 4 2 3 5 6 3 2 4GS 3 77 2 4 3 2 5 2 2Kicking 4 2 74 4 5 3 2 3 3Lifting 5 6 3 69 4 3 5 3 2HBR 2 3 3 2 80 2 4 2 2Running 2 3 2 2 5 75 6 2 3Skating 2 1 2 3 4 4 78 2 4BS 3 4 6 3 4 2 3 70 5Walking 4 1 2 4 2 3 0 3 81Average 7500(C)Diving 79 3 2 2 3 4 3 2 2GS 0 83 2 4 3 2 1 3 2Kicking 1 2 85 1 3 3 2 3 0Lifting 3 0 2 82 3 2 4 2 2HBR 0 2 2 4 80 0 5 3 4Running 1 2 1 3 4 84 2 1 2Skating 2 0 3 4 0 1 86 3 1BS 1 1 1 2 0 3 0 88 4Walking 1 2 4 2 5 2 4 3 77Average 8267(D)Diving 90 3 0 1 0 2 2 1 1GS 3 84 2 1 3 1 3 2 1Kicking 3 4 85 0 0 2 3 1 2Lifting 1 2 1 89 1 1 1 2 2HBR 0 2 1 0 91 2 3 0 1Running 2 3 1 2 3 80 4 2 3Skating 2 4 1 2 3 0 84 4 0BS 2 1 1 2 1 0 3 88 2Walking 0 2 1 1 0 1 4 0 91Average 8689GS golf swinging HBR horseback riding BS baseball swinging

10 Computational Intelligence and Neuroscience

53 ird Experiment In this experiment a comparativeanalysis was made between the state-of-the-art methods andthe proposed model All of these approaches were imple-mented by the instructions provided in their particular ar-ticles A 10-fold cross-validation rule was employed on eachdataset as explained in Section 4 e average classicationresults of the existing methods along with the proposedmethod across dierent datasets are summarized in Table 9

Table 8 Classication results of the proposed system on IXMASaction dataset (A) using ANN (B) using SVM (C) using HMMand (D) using existing HCRF [30] while removing the proposedHCRF model (unit )

Activities CA SD GU TA Walk Wave Punch Kick(A)CA 65 5 7 6 5 4 3 5SD 5 72 4 3 5 4 3 4GU 4 3 75 5 3 2 4 4TA 6 7 4 68 3 5 4 3Walk 3 4 5 4 70 6 4 4Wave 4 6 5 3 4 71 3 4Punch 3 5 5 6 7 3 67 4Kick 4 5 4 6 5 4 3 69Average 6962(B)CA 77 3 4 2 2 3 5 4SD 3 79 3 2 4 5 2 2GU 5 6 69 3 4 5 4 4TA 2 3 2 80 4 4 3 2Walk 3 5 4 2 71 5 6 4Wave 2 6 3 5 4 73 4 3Punch 1 5 3 4 1 3 81 2Kick 3 6 7 4 3 5 4 68Average 7475(C)CA 79 3 4 1 2 4 3 4SD 1 84 3 2 3 1 4 2GU 0 1 88 1 2 3 2 3TA 5 2 3 79 2 3 2 4Walk 1 0 3 1 90 2 3 0Wave 2 3 1 0 3 86 2 3Punch 1 0 2 3 0 4 89 1Kick 3 2 4 1 0 2 4 84Average 8477(D)CA 90 1 2 0 3 4 0 0SD 3 85 2 1 3 3 2 1GU 0 1 91 1 0 2 3 2TA 1 3 2 87 1 2 2 2Walk 1 0 3 1 89 2 1 3Wave 0 2 1 0 2 90 3 1Punch 1 4 2 1 2 3 84 3Kick 1 2 4 1 3 2 4 83Average 8738CA cross arm SD sit down GU get up TA turn around

Table 9 Weighted average recognition rates of the proposedmethod with the existing state-of-the-art methods (unit )

State-of-the-art works Average classicationrates

Standarddeviation

GMM 633 plusmn27SVM 675 plusmn44HMM 828 plusmn38Embedded HMM 859 plusmn19[62] 921 plusmn32[63] 843 plusmn49[18] 936 plusmn27[19] 930 plusmn16[22] 927 plusmn25[64] 801 plusmn32Proposed method 972 plusmn28

0

100

200

300

400

500

600

Exec

utio

n tim

e (s)

1 2 3 4 5 6 7 8Number of states

ForwardbackwardProposed HCRF model

(a)Ex

ecut

ion

time (

s)

1 2 3 4 5 6 7 8

ForwardbackwardProposed HCRF model

0

2

4

6

8

10

12

14

16

Number of mixtures

(b)

Figure 3 An illustration of gradient computational time (equation(30)) of the previous forward and backward algorithms and theproposed HCRF model (a)Q 1 minus 5 M 5 T 90 and (b)Q 5 M 1 minus 5 T 90

Computational Intelligence and Neuroscience 11

It is obvious from Table 9 that the proposed methodshowed a significant performance against the existing state-of-the-art methods erefore the proposed method accu-rately and robustly recognizes the human activities usingdifferent video data

54 Fourth Experiment In this experiment we have pre-sented the computational complexity that is also one of thecontributions in this paper e implementations of theprevious HCRF are available in literature which calculatethe gradients by reiterating the forward and backwardtechniques while the proposed HCRF model executes themonce only and cashes the outcomes for the later use From(21) and (22) it is clear that the forward or backwardtechnique has a complexity ofO(TQ2M) where Trepresentsthe input sequence lengthQ represents the number of statesand M indicates the number of mixtures e proposedHCRFmodel however requires a full complexity of O(TM)

to calculate gradients as can be seen from (22)ndash(29)Figure 3 shows a comparison of the execution time when

the gradients are computed by the forward (or backward)algorithm and by our proposed method e computationaltime is calculated by running Matlab R2013a with thespecification of Intelreg Pentiumreg Coretrade i7-6700 (34GHz)with a RAM capacity of 16GB

6 Conclusion

In healthcare and telemedicine the human activity rec-ognition (HAR) can be best explained by helping physi-cally disable personsrsquo scenario A paralyzed patient withhalf of the body critically attacked by paralysis is com-pletely unable to perform their daily exercises e doctorsrecommend specific activities to get better improvement intheir health So for this purpose the doctors need a humanactivity recognition (HAR) system through which they canmonitor the patientsrsquo daily routines (activities) on a reg-ular basis

e accuracy of most of the HAR systems depends uponthe recognition modules For feature extraction and selec-tion modules we used some of the existing well-knownmethods while for the recognition module we proposed theusage of HCRF model which is capable of approximating acomplex distribution using a mixture of Gaussian densityfunctions e proposed model was assessed against fourpublicly available standard action datasets From our ex-periments it is obvious that the proposed full-covarianceGaussian density function showed a significant improve-ment in accuracy than the existing state-of-the-art methodsFurthermore we also proved that such improvement issignificant from statistical point of view by showing valuele02 of the comparison Similarly the complexity analysispoints out that the proposed computational method stronglydecreases the execution time for the hidden conditionalrandom field model

e ultimate goal of this study is to deploy the proposedmodel on smartphones Currently the proposed model isusing full-covariance matrix however this might be time

consuming especially when using on smartphones Using alightweight classifier such as K-nearest neighbor (K-NN)could be one possible solution But K-NN is very muchsensitive to environmental factor (like noise) erefore infuture we will try to investigate further research to reducethe time and sustain the same recognition rate whenemploying on smartphones in real environment

Data Availability

e data used to support the findings of this study areavailable from the corresponding author upon request

Conflicts of Interest

e authors declare there are no conflicts of interest

Acknowledgments

is study was supported by the Jouf University SakakaAljouf Kingdom of Saudi Arabia under the registration no39791

References

[1] Z Chang J Cao and Y Zhang ldquoA novel image segmentationapproach for wood plate surface defect classification throughconvex optimizationrdquo Journal of Forestry Research vol 29no 6 pp 1789ndash1795 2018

[2] Z S Abdallah M M Gaber B Srinivasan andS Krishnaswamy ldquoAdaptive mobile activity recognitionsystemwith evolving data streamsrdquoNeurocomputing vol 150pp 304ndash317 2015

[3] M S Bakli M A Sakr and T H A Soliman ldquoA spatio-temporal algebra in Hadoop for moving objectsrdquo Geo-SpatialInformation Science vol 21 no 2 pp 102ndash114 2018

[4] W Zhao L Yan and Y Zhang ldquoGeometric-constrainedmulti-view image matching method based on semi-globaloptimizationrdquo Geo-Spatial Information Science vol 21 no 2pp 115ndash126 2018

[5] H Gao X Zhang J Zhao and D Li ldquoTechnology of in-telligent driving radar perception based on driving brainrdquoCAAI Transactions on Intelligence Technology vol 2 no 3pp 93ndash100 2017

[6] Y Wang X Jiang R Cao and X Wang ldquoRobust indoorhuman activity recognition using wireless signalsrdquo Sensorsvol 15 no 7 pp 17195ndash17208 2015

[7] J Wang Z Liu Y Wu and J Yuan ldquoMining actionlet ensemblefor action recognition with depth camerasrdquo in Proceedings of theIEEE Conference on Computer Vision and Pattern Recognitionpp 1290ndash1297 IEEE Providence RI USA June 2012

[8] S Karungaru M Daikoku and K Terada ldquoMulti camerasbased indoors human action recognition using fuzzy rulesrdquoJournal of Pattern Recognition Research vol 10 no 1pp 61ndash74 2015

[9] S Shukri L M Kamarudin and M H F Rahiman ldquoDevice-free localization for human activity monitoringrdquo in IntelligentVideo Surveillance IntechOpen London UK 2018

[10] L Fiore D Fehr R Bodor A Drenner G Somasundaramand N Papanikolopoulos ldquoMulti-camera human activitymonitoringrdquo Journal of Intelligent and Robotic Systemsvol 52 no 1 pp 5ndash43 2008

12 Computational Intelligence and Neuroscience

[11] C Zhang and Y Tian ldquoRGB-D camera-based daily livingactivity recognitionrdquo Journal of Computer Vision and ImageProcessing vol 2 no 4 pp 1ndash7 2012

[12] M Leo T DrsquoOrazio and P Spagnolo ldquoHuman activityrecognition for automatic visual surveillance of wide areasrdquo inProceedings of the ACM 2nd International Workshop on VideoSurveillance and Sensor Networks pp 124ndash130 ACM NewYork NY USA October 2004

[13] C-B Jin S Li and H Kim ldquoReal-time action detection invideo surveillance using sub-action descriptor with multi-CNNrdquo 2017 httpsarxivorgabs171003383

[14] W Niu J Long D Han and Y-F Wang ldquoHuman activitydetection and recognition for video surveillancerdquo in Pro-ceedings of the 2004 IEEE International Conference on Mul-timedia and Expo (ICME) (IEEE Cat No 04TH8763) vol 1pp 719ndash722 IEEE Taipei Taiwan June 2004

[15] Q Li M Zheng F Li J Wang Y-A Geng and H JiangldquoRetinal image segmentation using double-scale non-linearthresholding on vessel support regionsrdquo CAAI Transactionson Intelligence Technology vol 2 no 3 pp 109ndash115 2017

[16] L M G Fonseca L M Namikawa and E F CastejonldquoDigital image processing in remote sensingrdquo in Proceedingsof the 2009 Tutorials of the XXII Brazilian Symposium onComputer Graphics and Image Processing pp 59ndash71 IEEERio de Janeiro Brazil October 2009

[17] K Buys C Cagniart A Baksheev T De Laet J De Schutterand C Pantofaru ldquoAn adaptable system for RGB-D basedhuman body detection and pose estimationrdquo Journal of VisualCommunication and Image Representation vol 25 no 1pp 39ndash52 2014

[18] M Sharif M A Khan T Akram M Y Javed T Saba andA Rehman ldquoA framework of human detection and actionrecognition based on uniform segmentation and combinationof Euclidean distance and joint entropy-based features se-lectionrdquo EURASIP Journal on Image and Video Processingvol 2017 no 1 p 89 2017

[19] K-P Chou M Prasad D Wu et al ldquoRobust feature-basedautomated multi-view human action recognition systemrdquoIEEE Access vol 6 pp 15283ndash15296 2018

[20] M Ullah H Ullah and I M Alseadonn ldquoHuman actionrecognition in videos using stable featuresrdquo Signal and ImageProcessing An International Journal vol 8 no 6 pp 1ndash102017

[21] M Sharif M A Khan F Zahid J H Shah and T AkramldquoHuman action recognition a framework of statisticalweighted segmentation and rank correlation-based selectionrdquoPattern Analysis and Applications pp 1ndash14 2019

[22] J Zang L Wang Z Liu Q Zhang G Hua and N ZhengldquoAttention-based temporal weighted convolutional neuralnetwork for action recognitionrdquo in Proceedings of the IFIPInternational Conference on Artificial Intelligence Applicationsand Innovations pp 97ndash108 Springer Rhodes Greece May2018

[23] V-D Hoang ldquoMultiple classifier-based spatiotemporal fea-tures for living activity predictionrdquo Journal of Informationand Telecommunication vol 1 no 1 pp 100ndash112 2017

[24] S Dharmalingam and A Palanisamy ldquoVector space basedaugmented structural kinematic feature descriptor for humanactivity recognition in videosrdquo ETRI Journal vol 40 no 4pp 499ndash510 2018

[25] D Liu Y Yan M-L Shyu G Zhao and M Chen ldquoSpatio-temporal analysis for human action detection and recognitionin uncontrolled environmentsrdquo International Journal of

Multimedia Data Engineering and Management vol 6 no 1pp 1ndash18 2015

[26] C A Ronao and S-B Cho ldquoHuman activity recognition withsmartphone sensors using deep learning neural networksrdquoExpert Systems with Applications vol 59 pp 235ndash244 2016

[27] A arwat H Mahdi M Elhoseny and A E HassanienldquoRecognizing human activity in mobile crowdsensing envi-ronment using optimized k-NN algorithmrdquo Expert Systemswith Applications vol 107 pp 32ndash44 2018

[28] A Jalal Y-H Kim Y-J Kim S Kamal and D Kim ldquoRobusthuman activity recognition from depth video using spatio-temporal multi-fused featuresrdquo Pattern Recognition vol 61pp 295ndash308 2017

[29] J Lafferty A McCallum and F C Pereira ldquoConditionalrandom fields probabilistic models for segmenting and la-beling sequence datardquo in Proceedings of the 18th InternationalConference on Machine Learning (ICML-2001) pp 282ndash289Williamstown MA USA June-July 2001

[30] A Gunawardana M Mahajan A Acero and J C PlattldquoHidden conditional random fields for phone classificationrdquoin Proceedings of the INTERSPEECH vol 2 pp 1117ndash1120Citeseer Lisbon Portugal September 2005

[31] L Yang Q Song Z Wang and M Jiang ldquoParsing r-cnn forinstance-level human analysisrdquo in Proceedings of the IEEEConference on Computer Vision and Pattern Recognitionpp 364ndash373 Long Beach CA USA June 2019

[32] X Liang Y Wei L Lin et al ldquoLearning to segment human bywatching youtuberdquo IEEE Transactions on Pattern Analysisand Machine Intelligence vol 39 no 7 pp 1462ndash1468 2016

[33] M M Requena ldquoHuman segmentation with convolutionalneural networksrdquo esis University of Alicante AlicanteSpain 2018

[34] H Gao S Chen and Z Zhang ldquoParts semantic segmentationaware representation learning for person re-identificationrdquoApplied Sciences vol 9 no 6 p 1239 2019

[35] C Peng S-L Lo J Huang and A C Tsoi ldquoHuman actionsegmentation based on a streaming uniform entropy slicemethodrdquo IEEE Access vol 6 pp 16958ndash16971 2018

[36] M H Siddiqi A M Khan and S-W Lee ldquoActive contourslevel set based still human body segmentation from depthimages for video-based activity recognitionrdquo KSII Trans-actions on Internet and Information Systems (TIIS) vol 7no 11 pp 2839ndash2852 2013

[37] M Siddiqi R Ali M Rana E-K Hong E Kim and S LeeldquoVideo-based human activity recognition using multilevelwavelet decomposition and stepwise linear discriminantanalysisrdquo Sensors vol 14 no 4 pp 6370ndash6392 2014

[38] S Althloothi M H Mahoor X Zhang and R M VoylesldquoHuman activity recognition using multi-features and mul-tiple kernel learningrdquo Pattern Recognition vol 47 no 5pp 1800ndash1812 2014

[39] A Jalal S Kamal and D Kim ldquoA depth video-based humandetection and activity recognition using multi-features andembedded hidden markov models for health care monitoringsystemsrdquo International Journal of Interactive Multimedia andArtificial Intelligence vol 4 no 4 p 54 2017

[40] T Dobhal V Shitole G omas and G Navada ldquoHumanactivity recognition using binary motion image and deeplearningrdquo Procedia Computer Science vol 58 pp 178ndash1852015

[41] S Zhang Z Wei J Nie L Huang S Wang and Z Li ldquoAreview on human activity recognition using vision-basedmethodrdquo Journal of Healthcare Engineering vol 2017 ArticleID 3090343 31 pages 2017

Computational Intelligence and Neuroscience 13

[42] Y Yang B Zhang L Yang C Chen and W Yang ldquoActionrecognition using completed local binary patterns and mul-tiple-class boosting classifierrdquo in Proceedings of the 2015 3rdIAPR Asian Conference on Pattern Recognition (ACPR) IEEEpp 336ndash340 Kuala Lumpur Malaysia November 2015

[43] W Lin M T Sun R Poovandran and Z Zhang ldquoHumanactivity recognition for video surveillancerdquo in Proceedings ofthe 2008 IEEE International Symposium on Circuits andSystems pp 2737ndash2740 IEEE Seattle WA USA May 2008

[44] H Permuter J Francos and I Jermyn ldquoA study of Gaussianmixture models of color and texture features for imageclassification and segmentationrdquo Pattern Recognition vol 39no 4 pp 695ndash706 2006

[45] M Fiaz and B Ijaz ldquoVision based human activity trackingusing artificial neural networksrdquo in Proceedings of the 2010International Conference on Intelligent and Advanced Systemspp 1ndash5 IEEE Kuala Lumpur Malaysia June 2010

[46] H Foroughi A Naseri A Saberi and H S Yazdi ldquoAneigenspace-based approach for human fall detection usingintegrated time motion image and neural networkrdquo in Pro-ceedings of the 2008 9th International Conference on SignalProcessing pp 1499ndash1503 IEEE Beijing China October2008

[47] A K B Sonali ldquoHuman action recognition using supportvector machine and k-nearest neighborrdquo InternationalJournal of Engineering and Technical Research vol 3 no 4pp 423ndash428 2015

[48] V Swarnambigai ldquoAction recognition using ami and supportvector machinerdquo International Journal of Computer Scienceand Technology vol 5 no 4 pp 175ndash179 2014

[49] D Das and S Saharia ldquoHuman gait analysis and recognitionusing support vector machinesrdquo International Journal ofComputer Science and Information Technology vol 6 no 52004

[50] K G M Chathuramali and R Rodrigo ldquoFaster human activityrecognition with SVMrdquo in Proceedings of the InternationalConference on Advances in ICT for Emerging Regions(ICTer2012) pp 197ndash203 IEEE Colombo Sri Lanka De-cember 2012

[51] M Z Uddin D-H Kim J T Kim and T-S Kim ldquoAn indoorhuman activity recognition system for smart home using localbinary pattern features with hidden markov modelsrdquo Indoorand Built Environment vol 22 no 1 pp 289ndash298 2013

[52] M Z Uddin J Lee and T-S Kim ldquoIndependent shapecomponent-based human activity recognition via hiddenMarkov modelrdquo Applied Intelligence vol 33 no 2 pp 193ndash206 2010

[53] S Kumar and M Hebert ldquoDiscriminative Fields for ModelingSpatial Dependencies in Natural Imagesrdquo in Proceedings of theNIPS Vancouver Canada December 2003

[54] A Quattoni S Wang L-P Morency M Collins andT Darrell ldquoHidden conditional random fieldsrdquo IEEETransactions on Pattern Analysis and Machine Intelligencevol 29 no 10 pp 1848ndash1852 2007

[55] M Mahajan A Gunawardana and A Acero ldquoTraining al-gorithms for hidden conditional random fieldsrdquo in Pro-ceedings of the 2006 IEEE International Conference onAcoustics Speed and Signal Processing vol 1 IEEE ToulouseFrance May 2006

[56] S Reiter B Schuller and G Rigoll ldquoHidden conditionalrandom fields for meeting segmentationrdquo in Proceedings ofthe Multimedia and Expo 2007 IEEE International Confer-ence pp 639ndash642 IEEE Beijing China July 2007

[57] L Gorelick M Blank E Shechtman M Irani and R BasrildquoActions as space-time shapesrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 29 no 12 pp 2247ndash2253 2007

[58] I Laptev M Marszalek C Schmid and B RozenfeldldquoLearning realistic human actions from moviesrdquo in Pro-ceedings of the 2008 IEEE Conference on Computer Vision andPattern Recognition pp 1ndash8 IEEE Anchorage AK USAJune 2008

[59] K Soomro and A R Zamir ldquoAction recognition in realisticsports videosrdquo in Computer Vision in Sports pp 181ndash208Springer Berlin Germany 2014

[60] D Weinland E Boyer and R Ronfard ldquoAction recognitionfrom arbitrary views using 3D exemplarsrdquo in Proceedings ofthe 2007 IEEE 11th International Conference on ComputerVision pp 1ndash7 IEEE Rio de Janeiro Brazil October 2007

[61] M Siddiqi S Lee Y-K Lee A Khan and P Truc ldquoHier-archical recognition scheme for human facial expressionrecognition systemsrdquo Sensors vol 13 no 12 pp 16682ndash167132013

[62] M Elmezain and A Al-Hamadi ldquoVision-based human ac-tivity recognition using ldcrfsrdquo International Arab Journal ofInformation Technology (IAJIT) vol 15 no 3 pp 389ndash3952018

[63] A Roy B Banerjee and V Murino ldquoA novel dictionarylearning based multiple instance learning approach to actionrecognition from videosrdquo in Proceedings of the 6th In-ternational Conference on Pattern Recognition Applicationsand Methods pp 519ndash526 Porto Portugal February 2017

[64] A P Sirat ldquoActions as space-time shapesrdquo InternationalJournal of Application or Innovation in Engineering andManagement vol 7 no 8 pp 49ndash54 2018

14 Computational Intelligence and Neuroscience

Computer Games Technology

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

Advances in

FuzzySystems

Hindawiwwwhindawicom

Volume 2018

International Journal of

ReconfigurableComputing

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

thinspArtificial Intelligence

Hindawiwwwhindawicom Volumethinsp2018

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawiwwwhindawicom Volume 2018

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Engineering Mathematics

International Journal of

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Computational Intelligence and Neuroscience

Hindawiwwwhindawicom Volume 2018

Mathematical Problems in Engineering

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Hindawiwwwhindawicom Volume 2018

Human-ComputerInteraction

Advances in

Hindawiwwwhindawicom Volume 2018

Scientic Programming

Submit your manuscripts atwwwhindawicom

Page 10: Human Activity Recognition Using Gaussian Mixture Hidden …downloads.hindawi.com/journals/cin/2019/8590560.pdf · 2019-08-22 · ResearchArticle Human Activity Recognition Using

on an individual dataset e overall results are shown inTables 1 (using Weizmann dataset) 2 (using KTH dataset) 3(usingUCF sports dataset) and 4 (using IXMAS) respectively

As observed from Tables 1ndash4 the proposed recognitionmodel constantly obtained higher recognition rates on in-dividual dataset is result shows the robustness of theproposed model which means that the model not onlyshowed better performance on one dataset but also showedbetter performance across multiple spontaneous datasets

52 Second Experiment As described before the secondexperiment was conducted in the absence of the proposed

recognition model to show the importance of the proposedmodel using all the four datasets For this purpose we usedthe existing eminent classifiers like SVM ANN HMM andexisting HCRF [30] as a recognition model rather thanutilizing the proposed HCRF model

Tables 5ndash8 show that when the proposed HCRF modelwas substituted with ANN SVM HMM and existing HCRF[30] the system failed to accomplish higher recognitionrates e better performance of the proposed HCRF modelis visualized in Tables 1ndash4 which show that the proposedHCRF model effectively fix the drawbacks of HMM andexisting HCRF that has been extensively utilized for se-quential HAR

Table 7 Classification results of the proposed system onUCF sports dataset (A) using ANN (B) using SVM (C) using HMM and (D) usingexisting HCRF [30] while removing the proposed HCRF model (unit )

Activities Diving GS Kicking Lifting HBR Run Skating BS Walk(A)Diving 68 4 2 5 6 6 4 3 2GS 2 71 2 4 5 4 6 3 3Kicking 3 4 70 3 5 4 2 3 6Lifting 5 4 3 65 5 6 4 6 2HBR 3 4 6 3 66 4 5 5 4Running 3 3 5 4 6 64 6 4 5Skating 2 5 4 5 3 4 69 3 5BS 4 2 5 3 4 6 5 67 4Walking 5 4 2 3 4 3 6 3 70Average 6778(B)Diving 71 4 2 3 5 6 3 2 4GS 3 77 2 4 3 2 5 2 2Kicking 4 2 74 4 5 3 2 3 3Lifting 5 6 3 69 4 3 5 3 2HBR 2 3 3 2 80 2 4 2 2Running 2 3 2 2 5 75 6 2 3Skating 2 1 2 3 4 4 78 2 4BS 3 4 6 3 4 2 3 70 5Walking 4 1 2 4 2 3 0 3 81Average 7500(C)Diving 79 3 2 2 3 4 3 2 2GS 0 83 2 4 3 2 1 3 2Kicking 1 2 85 1 3 3 2 3 0Lifting 3 0 2 82 3 2 4 2 2HBR 0 2 2 4 80 0 5 3 4Running 1 2 1 3 4 84 2 1 2Skating 2 0 3 4 0 1 86 3 1BS 1 1 1 2 0 3 0 88 4Walking 1 2 4 2 5 2 4 3 77Average 8267(D)Diving 90 3 0 1 0 2 2 1 1GS 3 84 2 1 3 1 3 2 1Kicking 3 4 85 0 0 2 3 1 2Lifting 1 2 1 89 1 1 1 2 2HBR 0 2 1 0 91 2 3 0 1Running 2 3 1 2 3 80 4 2 3Skating 2 4 1 2 3 0 84 4 0BS 2 1 1 2 1 0 3 88 2Walking 0 2 1 1 0 1 4 0 91Average 8689GS golf swinging HBR horseback riding BS baseball swinging

10 Computational Intelligence and Neuroscience

53 ird Experiment In this experiment a comparativeanalysis was made between the state-of-the-art methods andthe proposed model All of these approaches were imple-mented by the instructions provided in their particular ar-ticles A 10-fold cross-validation rule was employed on eachdataset as explained in Section 4 e average classicationresults of the existing methods along with the proposedmethod across dierent datasets are summarized in Table 9

Table 8 Classication results of the proposed system on IXMASaction dataset (A) using ANN (B) using SVM (C) using HMMand (D) using existing HCRF [30] while removing the proposedHCRF model (unit )

Activities CA SD GU TA Walk Wave Punch Kick(A)CA 65 5 7 6 5 4 3 5SD 5 72 4 3 5 4 3 4GU 4 3 75 5 3 2 4 4TA 6 7 4 68 3 5 4 3Walk 3 4 5 4 70 6 4 4Wave 4 6 5 3 4 71 3 4Punch 3 5 5 6 7 3 67 4Kick 4 5 4 6 5 4 3 69Average 6962(B)CA 77 3 4 2 2 3 5 4SD 3 79 3 2 4 5 2 2GU 5 6 69 3 4 5 4 4TA 2 3 2 80 4 4 3 2Walk 3 5 4 2 71 5 6 4Wave 2 6 3 5 4 73 4 3Punch 1 5 3 4 1 3 81 2Kick 3 6 7 4 3 5 4 68Average 7475(C)CA 79 3 4 1 2 4 3 4SD 1 84 3 2 3 1 4 2GU 0 1 88 1 2 3 2 3TA 5 2 3 79 2 3 2 4Walk 1 0 3 1 90 2 3 0Wave 2 3 1 0 3 86 2 3Punch 1 0 2 3 0 4 89 1Kick 3 2 4 1 0 2 4 84Average 8477(D)CA 90 1 2 0 3 4 0 0SD 3 85 2 1 3 3 2 1GU 0 1 91 1 0 2 3 2TA 1 3 2 87 1 2 2 2Walk 1 0 3 1 89 2 1 3Wave 0 2 1 0 2 90 3 1Punch 1 4 2 1 2 3 84 3Kick 1 2 4 1 3 2 4 83Average 8738CA cross arm SD sit down GU get up TA turn around

Table 9 Weighted average recognition rates of the proposedmethod with the existing state-of-the-art methods (unit )

State-of-the-art works Average classicationrates

Standarddeviation

GMM 633 plusmn27SVM 675 plusmn44HMM 828 plusmn38Embedded HMM 859 plusmn19[62] 921 plusmn32[63] 843 plusmn49[18] 936 plusmn27[19] 930 plusmn16[22] 927 plusmn25[64] 801 plusmn32Proposed method 972 plusmn28

0

100

200

300

400

500

600

Exec

utio

n tim

e (s)

1 2 3 4 5 6 7 8Number of states

ForwardbackwardProposed HCRF model

(a)Ex

ecut

ion

time (

s)

1 2 3 4 5 6 7 8

ForwardbackwardProposed HCRF model

0

2

4

6

8

10

12

14

16

Number of mixtures

(b)

Figure 3 An illustration of gradient computational time (equation(30)) of the previous forward and backward algorithms and theproposed HCRF model (a)Q 1 minus 5 M 5 T 90 and (b)Q 5 M 1 minus 5 T 90

Computational Intelligence and Neuroscience 11

It is obvious from Table 9 that the proposed methodshowed a significant performance against the existing state-of-the-art methods erefore the proposed method accu-rately and robustly recognizes the human activities usingdifferent video data

54 Fourth Experiment In this experiment we have pre-sented the computational complexity that is also one of thecontributions in this paper e implementations of theprevious HCRF are available in literature which calculatethe gradients by reiterating the forward and backwardtechniques while the proposed HCRF model executes themonce only and cashes the outcomes for the later use From(21) and (22) it is clear that the forward or backwardtechnique has a complexity ofO(TQ2M) where Trepresentsthe input sequence lengthQ represents the number of statesand M indicates the number of mixtures e proposedHCRFmodel however requires a full complexity of O(TM)

to calculate gradients as can be seen from (22)ndash(29)Figure 3 shows a comparison of the execution time when

the gradients are computed by the forward (or backward)algorithm and by our proposed method e computationaltime is calculated by running Matlab R2013a with thespecification of Intelreg Pentiumreg Coretrade i7-6700 (34GHz)with a RAM capacity of 16GB

6 Conclusion

In healthcare and telemedicine the human activity rec-ognition (HAR) can be best explained by helping physi-cally disable personsrsquo scenario A paralyzed patient withhalf of the body critically attacked by paralysis is com-pletely unable to perform their daily exercises e doctorsrecommend specific activities to get better improvement intheir health So for this purpose the doctors need a humanactivity recognition (HAR) system through which they canmonitor the patientsrsquo daily routines (activities) on a reg-ular basis

e accuracy of most of the HAR systems depends uponthe recognition modules For feature extraction and selec-tion modules we used some of the existing well-knownmethods while for the recognition module we proposed theusage of HCRF model which is capable of approximating acomplex distribution using a mixture of Gaussian densityfunctions e proposed model was assessed against fourpublicly available standard action datasets From our ex-periments it is obvious that the proposed full-covarianceGaussian density function showed a significant improve-ment in accuracy than the existing state-of-the-art methodsFurthermore we also proved that such improvement issignificant from statistical point of view by showing valuele02 of the comparison Similarly the complexity analysispoints out that the proposed computational method stronglydecreases the execution time for the hidden conditionalrandom field model

e ultimate goal of this study is to deploy the proposedmodel on smartphones Currently the proposed model isusing full-covariance matrix however this might be time

consuming especially when using on smartphones Using alightweight classifier such as K-nearest neighbor (K-NN)could be one possible solution But K-NN is very muchsensitive to environmental factor (like noise) erefore infuture we will try to investigate further research to reducethe time and sustain the same recognition rate whenemploying on smartphones in real environment

Data Availability

e data used to support the findings of this study areavailable from the corresponding author upon request

Conflicts of Interest

e authors declare there are no conflicts of interest

Acknowledgments

is study was supported by the Jouf University SakakaAljouf Kingdom of Saudi Arabia under the registration no39791

References

[1] Z Chang J Cao and Y Zhang ldquoA novel image segmentationapproach for wood plate surface defect classification throughconvex optimizationrdquo Journal of Forestry Research vol 29no 6 pp 1789ndash1795 2018

[2] Z S Abdallah M M Gaber B Srinivasan andS Krishnaswamy ldquoAdaptive mobile activity recognitionsystemwith evolving data streamsrdquoNeurocomputing vol 150pp 304ndash317 2015

[3] M S Bakli M A Sakr and T H A Soliman ldquoA spatio-temporal algebra in Hadoop for moving objectsrdquo Geo-SpatialInformation Science vol 21 no 2 pp 102ndash114 2018

[4] W Zhao L Yan and Y Zhang ldquoGeometric-constrainedmulti-view image matching method based on semi-globaloptimizationrdquo Geo-Spatial Information Science vol 21 no 2pp 115ndash126 2018

[5] H Gao X Zhang J Zhao and D Li ldquoTechnology of in-telligent driving radar perception based on driving brainrdquoCAAI Transactions on Intelligence Technology vol 2 no 3pp 93ndash100 2017

[6] Y Wang X Jiang R Cao and X Wang ldquoRobust indoorhuman activity recognition using wireless signalsrdquo Sensorsvol 15 no 7 pp 17195ndash17208 2015

[7] J Wang Z Liu Y Wu and J Yuan ldquoMining actionlet ensemblefor action recognition with depth camerasrdquo in Proceedings of theIEEE Conference on Computer Vision and Pattern Recognitionpp 1290ndash1297 IEEE Providence RI USA June 2012

[8] S Karungaru M Daikoku and K Terada ldquoMulti camerasbased indoors human action recognition using fuzzy rulesrdquoJournal of Pattern Recognition Research vol 10 no 1pp 61ndash74 2015

[9] S Shukri L M Kamarudin and M H F Rahiman ldquoDevice-free localization for human activity monitoringrdquo in IntelligentVideo Surveillance IntechOpen London UK 2018

[10] L Fiore D Fehr R Bodor A Drenner G Somasundaramand N Papanikolopoulos ldquoMulti-camera human activitymonitoringrdquo Journal of Intelligent and Robotic Systemsvol 52 no 1 pp 5ndash43 2008

12 Computational Intelligence and Neuroscience

[11] C Zhang and Y Tian ldquoRGB-D camera-based daily livingactivity recognitionrdquo Journal of Computer Vision and ImageProcessing vol 2 no 4 pp 1ndash7 2012

[12] M Leo T DrsquoOrazio and P Spagnolo ldquoHuman activityrecognition for automatic visual surveillance of wide areasrdquo inProceedings of the ACM 2nd International Workshop on VideoSurveillance and Sensor Networks pp 124ndash130 ACM NewYork NY USA October 2004

[13] C-B Jin S Li and H Kim ldquoReal-time action detection invideo surveillance using sub-action descriptor with multi-CNNrdquo 2017 httpsarxivorgabs171003383

[14] W Niu J Long D Han and Y-F Wang ldquoHuman activitydetection and recognition for video surveillancerdquo in Pro-ceedings of the 2004 IEEE International Conference on Mul-timedia and Expo (ICME) (IEEE Cat No 04TH8763) vol 1pp 719ndash722 IEEE Taipei Taiwan June 2004

[15] Q Li M Zheng F Li J Wang Y-A Geng and H JiangldquoRetinal image segmentation using double-scale non-linearthresholding on vessel support regionsrdquo CAAI Transactionson Intelligence Technology vol 2 no 3 pp 109ndash115 2017

[16] L M G Fonseca L M Namikawa and E F CastejonldquoDigital image processing in remote sensingrdquo in Proceedingsof the 2009 Tutorials of the XXII Brazilian Symposium onComputer Graphics and Image Processing pp 59ndash71 IEEERio de Janeiro Brazil October 2009

[17] K Buys C Cagniart A Baksheev T De Laet J De Schutterand C Pantofaru ldquoAn adaptable system for RGB-D basedhuman body detection and pose estimationrdquo Journal of VisualCommunication and Image Representation vol 25 no 1pp 39ndash52 2014

[18] M Sharif M A Khan T Akram M Y Javed T Saba andA Rehman ldquoA framework of human detection and actionrecognition based on uniform segmentation and combinationof Euclidean distance and joint entropy-based features se-lectionrdquo EURASIP Journal on Image and Video Processingvol 2017 no 1 p 89 2017

[19] K-P Chou M Prasad D Wu et al ldquoRobust feature-basedautomated multi-view human action recognition systemrdquoIEEE Access vol 6 pp 15283ndash15296 2018

[20] M Ullah H Ullah and I M Alseadonn ldquoHuman actionrecognition in videos using stable featuresrdquo Signal and ImageProcessing An International Journal vol 8 no 6 pp 1ndash102017

[21] M Sharif M A Khan F Zahid J H Shah and T AkramldquoHuman action recognition a framework of statisticalweighted segmentation and rank correlation-based selectionrdquoPattern Analysis and Applications pp 1ndash14 2019

[22] J Zang L Wang Z Liu Q Zhang G Hua and N ZhengldquoAttention-based temporal weighted convolutional neuralnetwork for action recognitionrdquo in Proceedings of the IFIPInternational Conference on Artificial Intelligence Applicationsand Innovations pp 97ndash108 Springer Rhodes Greece May2018

[23] V-D Hoang ldquoMultiple classifier-based spatiotemporal fea-tures for living activity predictionrdquo Journal of Informationand Telecommunication vol 1 no 1 pp 100ndash112 2017

[24] S Dharmalingam and A Palanisamy ldquoVector space basedaugmented structural kinematic feature descriptor for humanactivity recognition in videosrdquo ETRI Journal vol 40 no 4pp 499ndash510 2018

[25] D Liu Y Yan M-L Shyu G Zhao and M Chen ldquoSpatio-temporal analysis for human action detection and recognitionin uncontrolled environmentsrdquo International Journal of

Multimedia Data Engineering and Management vol 6 no 1pp 1ndash18 2015

[26] C A Ronao and S-B Cho ldquoHuman activity recognition withsmartphone sensors using deep learning neural networksrdquoExpert Systems with Applications vol 59 pp 235ndash244 2016

[27] A arwat H Mahdi M Elhoseny and A E HassanienldquoRecognizing human activity in mobile crowdsensing envi-ronment using optimized k-NN algorithmrdquo Expert Systemswith Applications vol 107 pp 32ndash44 2018

[28] A Jalal Y-H Kim Y-J Kim S Kamal and D Kim ldquoRobusthuman activity recognition from depth video using spatio-temporal multi-fused featuresrdquo Pattern Recognition vol 61pp 295ndash308 2017

[29] J Lafferty A McCallum and F C Pereira ldquoConditionalrandom fields probabilistic models for segmenting and la-beling sequence datardquo in Proceedings of the 18th InternationalConference on Machine Learning (ICML-2001) pp 282ndash289Williamstown MA USA June-July 2001

[30] A Gunawardana M Mahajan A Acero and J C PlattldquoHidden conditional random fields for phone classificationrdquoin Proceedings of the INTERSPEECH vol 2 pp 1117ndash1120Citeseer Lisbon Portugal September 2005

[31] L Yang Q Song Z Wang and M Jiang ldquoParsing r-cnn forinstance-level human analysisrdquo in Proceedings of the IEEEConference on Computer Vision and Pattern Recognitionpp 364ndash373 Long Beach CA USA June 2019

[32] X Liang Y Wei L Lin et al ldquoLearning to segment human bywatching youtuberdquo IEEE Transactions on Pattern Analysisand Machine Intelligence vol 39 no 7 pp 1462ndash1468 2016

[33] M M Requena ldquoHuman segmentation with convolutionalneural networksrdquo esis University of Alicante AlicanteSpain 2018

[34] H Gao S Chen and Z Zhang ldquoParts semantic segmentationaware representation learning for person re-identificationrdquoApplied Sciences vol 9 no 6 p 1239 2019

[35] C Peng S-L Lo J Huang and A C Tsoi ldquoHuman actionsegmentation based on a streaming uniform entropy slicemethodrdquo IEEE Access vol 6 pp 16958ndash16971 2018

[36] M H Siddiqi A M Khan and S-W Lee ldquoActive contourslevel set based still human body segmentation from depthimages for video-based activity recognitionrdquo KSII Trans-actions on Internet and Information Systems (TIIS) vol 7no 11 pp 2839ndash2852 2013

[37] M Siddiqi R Ali M Rana E-K Hong E Kim and S LeeldquoVideo-based human activity recognition using multilevelwavelet decomposition and stepwise linear discriminantanalysisrdquo Sensors vol 14 no 4 pp 6370ndash6392 2014

[38] S Althloothi M H Mahoor X Zhang and R M VoylesldquoHuman activity recognition using multi-features and mul-tiple kernel learningrdquo Pattern Recognition vol 47 no 5pp 1800ndash1812 2014

[39] A Jalal S Kamal and D Kim ldquoA depth video-based humandetection and activity recognition using multi-features andembedded hidden markov models for health care monitoringsystemsrdquo International Journal of Interactive Multimedia andArtificial Intelligence vol 4 no 4 p 54 2017

[40] T Dobhal V Shitole G omas and G Navada ldquoHumanactivity recognition using binary motion image and deeplearningrdquo Procedia Computer Science vol 58 pp 178ndash1852015

[41] S Zhang Z Wei J Nie L Huang S Wang and Z Li ldquoAreview on human activity recognition using vision-basedmethodrdquo Journal of Healthcare Engineering vol 2017 ArticleID 3090343 31 pages 2017

Computational Intelligence and Neuroscience 13

[42] Y Yang B Zhang L Yang C Chen and W Yang ldquoActionrecognition using completed local binary patterns and mul-tiple-class boosting classifierrdquo in Proceedings of the 2015 3rdIAPR Asian Conference on Pattern Recognition (ACPR) IEEEpp 336ndash340 Kuala Lumpur Malaysia November 2015

[43] W Lin M T Sun R Poovandran and Z Zhang ldquoHumanactivity recognition for video surveillancerdquo in Proceedings ofthe 2008 IEEE International Symposium on Circuits andSystems pp 2737ndash2740 IEEE Seattle WA USA May 2008

[44] H Permuter J Francos and I Jermyn ldquoA study of Gaussianmixture models of color and texture features for imageclassification and segmentationrdquo Pattern Recognition vol 39no 4 pp 695ndash706 2006

[45] M Fiaz and B Ijaz ldquoVision based human activity trackingusing artificial neural networksrdquo in Proceedings of the 2010International Conference on Intelligent and Advanced Systemspp 1ndash5 IEEE Kuala Lumpur Malaysia June 2010

[46] H Foroughi A Naseri A Saberi and H S Yazdi ldquoAneigenspace-based approach for human fall detection usingintegrated time motion image and neural networkrdquo in Pro-ceedings of the 2008 9th International Conference on SignalProcessing pp 1499ndash1503 IEEE Beijing China October2008

[47] A K B Sonali ldquoHuman action recognition using supportvector machine and k-nearest neighborrdquo InternationalJournal of Engineering and Technical Research vol 3 no 4pp 423ndash428 2015

[48] V Swarnambigai ldquoAction recognition using ami and supportvector machinerdquo International Journal of Computer Scienceand Technology vol 5 no 4 pp 175ndash179 2014

[49] D Das and S Saharia ldquoHuman gait analysis and recognitionusing support vector machinesrdquo International Journal ofComputer Science and Information Technology vol 6 no 52004

[50] K G M Chathuramali and R Rodrigo ldquoFaster human activityrecognition with SVMrdquo in Proceedings of the InternationalConference on Advances in ICT for Emerging Regions(ICTer2012) pp 197ndash203 IEEE Colombo Sri Lanka De-cember 2012

[51] M Z Uddin D-H Kim J T Kim and T-S Kim ldquoAn indoorhuman activity recognition system for smart home using localbinary pattern features with hidden markov modelsrdquo Indoorand Built Environment vol 22 no 1 pp 289ndash298 2013

[52] M Z Uddin J Lee and T-S Kim ldquoIndependent shapecomponent-based human activity recognition via hiddenMarkov modelrdquo Applied Intelligence vol 33 no 2 pp 193ndash206 2010

[53] S Kumar and M Hebert ldquoDiscriminative Fields for ModelingSpatial Dependencies in Natural Imagesrdquo in Proceedings of theNIPS Vancouver Canada December 2003

[54] A Quattoni S Wang L-P Morency M Collins andT Darrell ldquoHidden conditional random fieldsrdquo IEEETransactions on Pattern Analysis and Machine Intelligencevol 29 no 10 pp 1848ndash1852 2007

[55] M Mahajan A Gunawardana and A Acero ldquoTraining al-gorithms for hidden conditional random fieldsrdquo in Pro-ceedings of the 2006 IEEE International Conference onAcoustics Speed and Signal Processing vol 1 IEEE ToulouseFrance May 2006

[56] S Reiter B Schuller and G Rigoll ldquoHidden conditionalrandom fields for meeting segmentationrdquo in Proceedings ofthe Multimedia and Expo 2007 IEEE International Confer-ence pp 639ndash642 IEEE Beijing China July 2007

[57] L Gorelick M Blank E Shechtman M Irani and R BasrildquoActions as space-time shapesrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 29 no 12 pp 2247ndash2253 2007

[58] I Laptev M Marszalek C Schmid and B RozenfeldldquoLearning realistic human actions from moviesrdquo in Pro-ceedings of the 2008 IEEE Conference on Computer Vision andPattern Recognition pp 1ndash8 IEEE Anchorage AK USAJune 2008

[59] K Soomro and A R Zamir ldquoAction recognition in realisticsports videosrdquo in Computer Vision in Sports pp 181ndash208Springer Berlin Germany 2014

[60] D Weinland E Boyer and R Ronfard ldquoAction recognitionfrom arbitrary views using 3D exemplarsrdquo in Proceedings ofthe 2007 IEEE 11th International Conference on ComputerVision pp 1ndash7 IEEE Rio de Janeiro Brazil October 2007

[61] M Siddiqi S Lee Y-K Lee A Khan and P Truc ldquoHier-archical recognition scheme for human facial expressionrecognition systemsrdquo Sensors vol 13 no 12 pp 16682ndash167132013

[62] M Elmezain and A Al-Hamadi ldquoVision-based human ac-tivity recognition using ldcrfsrdquo International Arab Journal ofInformation Technology (IAJIT) vol 15 no 3 pp 389ndash3952018

[63] A Roy B Banerjee and V Murino ldquoA novel dictionarylearning based multiple instance learning approach to actionrecognition from videosrdquo in Proceedings of the 6th In-ternational Conference on Pattern Recognition Applicationsand Methods pp 519ndash526 Porto Portugal February 2017

[64] A P Sirat ldquoActions as space-time shapesrdquo InternationalJournal of Application or Innovation in Engineering andManagement vol 7 no 8 pp 49ndash54 2018

14 Computational Intelligence and Neuroscience

Computer Games Technology

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

Advances in

FuzzySystems

Hindawiwwwhindawicom

Volume 2018

International Journal of

ReconfigurableComputing

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

thinspArtificial Intelligence

Hindawiwwwhindawicom Volumethinsp2018

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawiwwwhindawicom Volume 2018

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Engineering Mathematics

International Journal of

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Computational Intelligence and Neuroscience

Hindawiwwwhindawicom Volume 2018

Mathematical Problems in Engineering

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Hindawiwwwhindawicom Volume 2018

Human-ComputerInteraction

Advances in

Hindawiwwwhindawicom Volume 2018

Scientic Programming

Submit your manuscripts atwwwhindawicom

Page 11: Human Activity Recognition Using Gaussian Mixture Hidden …downloads.hindawi.com/journals/cin/2019/8590560.pdf · 2019-08-22 · ResearchArticle Human Activity Recognition Using

53 ird Experiment In this experiment a comparativeanalysis was made between the state-of-the-art methods andthe proposed model All of these approaches were imple-mented by the instructions provided in their particular ar-ticles A 10-fold cross-validation rule was employed on eachdataset as explained in Section 4 e average classicationresults of the existing methods along with the proposedmethod across dierent datasets are summarized in Table 9

Table 8 Classication results of the proposed system on IXMASaction dataset (A) using ANN (B) using SVM (C) using HMMand (D) using existing HCRF [30] while removing the proposedHCRF model (unit )

Activities CA SD GU TA Walk Wave Punch Kick(A)CA 65 5 7 6 5 4 3 5SD 5 72 4 3 5 4 3 4GU 4 3 75 5 3 2 4 4TA 6 7 4 68 3 5 4 3Walk 3 4 5 4 70 6 4 4Wave 4 6 5 3 4 71 3 4Punch 3 5 5 6 7 3 67 4Kick 4 5 4 6 5 4 3 69Average 6962(B)CA 77 3 4 2 2 3 5 4SD 3 79 3 2 4 5 2 2GU 5 6 69 3 4 5 4 4TA 2 3 2 80 4 4 3 2Walk 3 5 4 2 71 5 6 4Wave 2 6 3 5 4 73 4 3Punch 1 5 3 4 1 3 81 2Kick 3 6 7 4 3 5 4 68Average 7475(C)CA 79 3 4 1 2 4 3 4SD 1 84 3 2 3 1 4 2GU 0 1 88 1 2 3 2 3TA 5 2 3 79 2 3 2 4Walk 1 0 3 1 90 2 3 0Wave 2 3 1 0 3 86 2 3Punch 1 0 2 3 0 4 89 1Kick 3 2 4 1 0 2 4 84Average 8477(D)CA 90 1 2 0 3 4 0 0SD 3 85 2 1 3 3 2 1GU 0 1 91 1 0 2 3 2TA 1 3 2 87 1 2 2 2Walk 1 0 3 1 89 2 1 3Wave 0 2 1 0 2 90 3 1Punch 1 4 2 1 2 3 84 3Kick 1 2 4 1 3 2 4 83Average 8738CA cross arm SD sit down GU get up TA turn around

Table 9 Weighted average recognition rates of the proposedmethod with the existing state-of-the-art methods (unit )

State-of-the-art works Average classicationrates

Standarddeviation

GMM 633 plusmn27SVM 675 plusmn44HMM 828 plusmn38Embedded HMM 859 plusmn19[62] 921 plusmn32[63] 843 plusmn49[18] 936 plusmn27[19] 930 plusmn16[22] 927 plusmn25[64] 801 plusmn32Proposed method 972 plusmn28

0

100

200

300

400

500

600

Exec

utio

n tim

e (s)

1 2 3 4 5 6 7 8Number of states

ForwardbackwardProposed HCRF model

(a)Ex

ecut

ion

time (

s)

1 2 3 4 5 6 7 8

ForwardbackwardProposed HCRF model

0

2

4

6

8

10

12

14

16

Number of mixtures

(b)

Figure 3 An illustration of gradient computational time (equation(30)) of the previous forward and backward algorithms and theproposed HCRF model (a)Q 1 minus 5 M 5 T 90 and (b)Q 5 M 1 minus 5 T 90

Computational Intelligence and Neuroscience 11

It is obvious from Table 9 that the proposed methodshowed a significant performance against the existing state-of-the-art methods erefore the proposed method accu-rately and robustly recognizes the human activities usingdifferent video data

54 Fourth Experiment In this experiment we have pre-sented the computational complexity that is also one of thecontributions in this paper e implementations of theprevious HCRF are available in literature which calculatethe gradients by reiterating the forward and backwardtechniques while the proposed HCRF model executes themonce only and cashes the outcomes for the later use From(21) and (22) it is clear that the forward or backwardtechnique has a complexity ofO(TQ2M) where Trepresentsthe input sequence lengthQ represents the number of statesand M indicates the number of mixtures e proposedHCRFmodel however requires a full complexity of O(TM)

to calculate gradients as can be seen from (22)ndash(29)Figure 3 shows a comparison of the execution time when

the gradients are computed by the forward (or backward)algorithm and by our proposed method e computationaltime is calculated by running Matlab R2013a with thespecification of Intelreg Pentiumreg Coretrade i7-6700 (34GHz)with a RAM capacity of 16GB

6 Conclusion

In healthcare and telemedicine the human activity rec-ognition (HAR) can be best explained by helping physi-cally disable personsrsquo scenario A paralyzed patient withhalf of the body critically attacked by paralysis is com-pletely unable to perform their daily exercises e doctorsrecommend specific activities to get better improvement intheir health So for this purpose the doctors need a humanactivity recognition (HAR) system through which they canmonitor the patientsrsquo daily routines (activities) on a reg-ular basis

e accuracy of most of the HAR systems depends uponthe recognition modules For feature extraction and selec-tion modules we used some of the existing well-knownmethods while for the recognition module we proposed theusage of HCRF model which is capable of approximating acomplex distribution using a mixture of Gaussian densityfunctions e proposed model was assessed against fourpublicly available standard action datasets From our ex-periments it is obvious that the proposed full-covarianceGaussian density function showed a significant improve-ment in accuracy than the existing state-of-the-art methodsFurthermore we also proved that such improvement issignificant from statistical point of view by showing valuele02 of the comparison Similarly the complexity analysispoints out that the proposed computational method stronglydecreases the execution time for the hidden conditionalrandom field model

e ultimate goal of this study is to deploy the proposedmodel on smartphones Currently the proposed model isusing full-covariance matrix however this might be time

consuming especially when using on smartphones Using alightweight classifier such as K-nearest neighbor (K-NN)could be one possible solution But K-NN is very muchsensitive to environmental factor (like noise) erefore infuture we will try to investigate further research to reducethe time and sustain the same recognition rate whenemploying on smartphones in real environment

Data Availability

e data used to support the findings of this study areavailable from the corresponding author upon request

Conflicts of Interest

e authors declare there are no conflicts of interest

Acknowledgments

is study was supported by the Jouf University SakakaAljouf Kingdom of Saudi Arabia under the registration no39791

References

[1] Z Chang J Cao and Y Zhang ldquoA novel image segmentationapproach for wood plate surface defect classification throughconvex optimizationrdquo Journal of Forestry Research vol 29no 6 pp 1789ndash1795 2018

[2] Z S Abdallah M M Gaber B Srinivasan andS Krishnaswamy ldquoAdaptive mobile activity recognitionsystemwith evolving data streamsrdquoNeurocomputing vol 150pp 304ndash317 2015

[3] M S Bakli M A Sakr and T H A Soliman ldquoA spatio-temporal algebra in Hadoop for moving objectsrdquo Geo-SpatialInformation Science vol 21 no 2 pp 102ndash114 2018

[4] W Zhao L Yan and Y Zhang ldquoGeometric-constrainedmulti-view image matching method based on semi-globaloptimizationrdquo Geo-Spatial Information Science vol 21 no 2pp 115ndash126 2018

[5] H Gao X Zhang J Zhao and D Li ldquoTechnology of in-telligent driving radar perception based on driving brainrdquoCAAI Transactions on Intelligence Technology vol 2 no 3pp 93ndash100 2017

[6] Y Wang X Jiang R Cao and X Wang ldquoRobust indoorhuman activity recognition using wireless signalsrdquo Sensorsvol 15 no 7 pp 17195ndash17208 2015

[7] J Wang Z Liu Y Wu and J Yuan ldquoMining actionlet ensemblefor action recognition with depth camerasrdquo in Proceedings of theIEEE Conference on Computer Vision and Pattern Recognitionpp 1290ndash1297 IEEE Providence RI USA June 2012

[8] S Karungaru M Daikoku and K Terada ldquoMulti camerasbased indoors human action recognition using fuzzy rulesrdquoJournal of Pattern Recognition Research vol 10 no 1pp 61ndash74 2015

[9] S Shukri L M Kamarudin and M H F Rahiman ldquoDevice-free localization for human activity monitoringrdquo in IntelligentVideo Surveillance IntechOpen London UK 2018

[10] L Fiore D Fehr R Bodor A Drenner G Somasundaramand N Papanikolopoulos ldquoMulti-camera human activitymonitoringrdquo Journal of Intelligent and Robotic Systemsvol 52 no 1 pp 5ndash43 2008

12 Computational Intelligence and Neuroscience

[11] C Zhang and Y Tian ldquoRGB-D camera-based daily livingactivity recognitionrdquo Journal of Computer Vision and ImageProcessing vol 2 no 4 pp 1ndash7 2012

[12] M Leo T DrsquoOrazio and P Spagnolo ldquoHuman activityrecognition for automatic visual surveillance of wide areasrdquo inProceedings of the ACM 2nd International Workshop on VideoSurveillance and Sensor Networks pp 124ndash130 ACM NewYork NY USA October 2004

[13] C-B Jin S Li and H Kim ldquoReal-time action detection invideo surveillance using sub-action descriptor with multi-CNNrdquo 2017 httpsarxivorgabs171003383

[14] W Niu J Long D Han and Y-F Wang ldquoHuman activitydetection and recognition for video surveillancerdquo in Pro-ceedings of the 2004 IEEE International Conference on Mul-timedia and Expo (ICME) (IEEE Cat No 04TH8763) vol 1pp 719ndash722 IEEE Taipei Taiwan June 2004

[15] Q Li M Zheng F Li J Wang Y-A Geng and H JiangldquoRetinal image segmentation using double-scale non-linearthresholding on vessel support regionsrdquo CAAI Transactionson Intelligence Technology vol 2 no 3 pp 109ndash115 2017

[16] L M G Fonseca L M Namikawa and E F CastejonldquoDigital image processing in remote sensingrdquo in Proceedingsof the 2009 Tutorials of the XXII Brazilian Symposium onComputer Graphics and Image Processing pp 59ndash71 IEEERio de Janeiro Brazil October 2009

[17] K Buys C Cagniart A Baksheev T De Laet J De Schutterand C Pantofaru ldquoAn adaptable system for RGB-D basedhuman body detection and pose estimationrdquo Journal of VisualCommunication and Image Representation vol 25 no 1pp 39ndash52 2014

[18] M Sharif M A Khan T Akram M Y Javed T Saba andA Rehman ldquoA framework of human detection and actionrecognition based on uniform segmentation and combinationof Euclidean distance and joint entropy-based features se-lectionrdquo EURASIP Journal on Image and Video Processingvol 2017 no 1 p 89 2017

[19] K-P Chou M Prasad D Wu et al ldquoRobust feature-basedautomated multi-view human action recognition systemrdquoIEEE Access vol 6 pp 15283ndash15296 2018

[20] M Ullah H Ullah and I M Alseadonn ldquoHuman actionrecognition in videos using stable featuresrdquo Signal and ImageProcessing An International Journal vol 8 no 6 pp 1ndash102017

[21] M Sharif M A Khan F Zahid J H Shah and T AkramldquoHuman action recognition a framework of statisticalweighted segmentation and rank correlation-based selectionrdquoPattern Analysis and Applications pp 1ndash14 2019

[22] J Zang L Wang Z Liu Q Zhang G Hua and N ZhengldquoAttention-based temporal weighted convolutional neuralnetwork for action recognitionrdquo in Proceedings of the IFIPInternational Conference on Artificial Intelligence Applicationsand Innovations pp 97ndash108 Springer Rhodes Greece May2018

[23] V-D Hoang ldquoMultiple classifier-based spatiotemporal fea-tures for living activity predictionrdquo Journal of Informationand Telecommunication vol 1 no 1 pp 100ndash112 2017

[24] S Dharmalingam and A Palanisamy ldquoVector space basedaugmented structural kinematic feature descriptor for humanactivity recognition in videosrdquo ETRI Journal vol 40 no 4pp 499ndash510 2018

[25] D Liu Y Yan M-L Shyu G Zhao and M Chen ldquoSpatio-temporal analysis for human action detection and recognitionin uncontrolled environmentsrdquo International Journal of

Multimedia Data Engineering and Management vol 6 no 1pp 1ndash18 2015

[26] C A Ronao and S-B Cho ldquoHuman activity recognition withsmartphone sensors using deep learning neural networksrdquoExpert Systems with Applications vol 59 pp 235ndash244 2016

[27] A arwat H Mahdi M Elhoseny and A E HassanienldquoRecognizing human activity in mobile crowdsensing envi-ronment using optimized k-NN algorithmrdquo Expert Systemswith Applications vol 107 pp 32ndash44 2018

[28] A Jalal Y-H Kim Y-J Kim S Kamal and D Kim ldquoRobusthuman activity recognition from depth video using spatio-temporal multi-fused featuresrdquo Pattern Recognition vol 61pp 295ndash308 2017

[29] J Lafferty A McCallum and F C Pereira ldquoConditionalrandom fields probabilistic models for segmenting and la-beling sequence datardquo in Proceedings of the 18th InternationalConference on Machine Learning (ICML-2001) pp 282ndash289Williamstown MA USA June-July 2001

[30] A Gunawardana M Mahajan A Acero and J C PlattldquoHidden conditional random fields for phone classificationrdquoin Proceedings of the INTERSPEECH vol 2 pp 1117ndash1120Citeseer Lisbon Portugal September 2005

[31] L Yang Q Song Z Wang and M Jiang ldquoParsing r-cnn forinstance-level human analysisrdquo in Proceedings of the IEEEConference on Computer Vision and Pattern Recognitionpp 364ndash373 Long Beach CA USA June 2019

[32] X Liang Y Wei L Lin et al ldquoLearning to segment human bywatching youtuberdquo IEEE Transactions on Pattern Analysisand Machine Intelligence vol 39 no 7 pp 1462ndash1468 2016

[33] M M Requena ldquoHuman segmentation with convolutionalneural networksrdquo esis University of Alicante AlicanteSpain 2018

[34] H Gao S Chen and Z Zhang ldquoParts semantic segmentationaware representation learning for person re-identificationrdquoApplied Sciences vol 9 no 6 p 1239 2019

[35] C Peng S-L Lo J Huang and A C Tsoi ldquoHuman actionsegmentation based on a streaming uniform entropy slicemethodrdquo IEEE Access vol 6 pp 16958ndash16971 2018

[36] M H Siddiqi A M Khan and S-W Lee ldquoActive contourslevel set based still human body segmentation from depthimages for video-based activity recognitionrdquo KSII Trans-actions on Internet and Information Systems (TIIS) vol 7no 11 pp 2839ndash2852 2013

[37] M Siddiqi R Ali M Rana E-K Hong E Kim and S LeeldquoVideo-based human activity recognition using multilevelwavelet decomposition and stepwise linear discriminantanalysisrdquo Sensors vol 14 no 4 pp 6370ndash6392 2014

[38] S Althloothi M H Mahoor X Zhang and R M VoylesldquoHuman activity recognition using multi-features and mul-tiple kernel learningrdquo Pattern Recognition vol 47 no 5pp 1800ndash1812 2014

[39] A Jalal S Kamal and D Kim ldquoA depth video-based humandetection and activity recognition using multi-features andembedded hidden markov models for health care monitoringsystemsrdquo International Journal of Interactive Multimedia andArtificial Intelligence vol 4 no 4 p 54 2017

[40] T Dobhal V Shitole G omas and G Navada ldquoHumanactivity recognition using binary motion image and deeplearningrdquo Procedia Computer Science vol 58 pp 178ndash1852015

[41] S Zhang Z Wei J Nie L Huang S Wang and Z Li ldquoAreview on human activity recognition using vision-basedmethodrdquo Journal of Healthcare Engineering vol 2017 ArticleID 3090343 31 pages 2017

Computational Intelligence and Neuroscience 13

[42] Y Yang B Zhang L Yang C Chen and W Yang ldquoActionrecognition using completed local binary patterns and mul-tiple-class boosting classifierrdquo in Proceedings of the 2015 3rdIAPR Asian Conference on Pattern Recognition (ACPR) IEEEpp 336ndash340 Kuala Lumpur Malaysia November 2015

[43] W Lin M T Sun R Poovandran and Z Zhang ldquoHumanactivity recognition for video surveillancerdquo in Proceedings ofthe 2008 IEEE International Symposium on Circuits andSystems pp 2737ndash2740 IEEE Seattle WA USA May 2008

[44] H Permuter J Francos and I Jermyn ldquoA study of Gaussianmixture models of color and texture features for imageclassification and segmentationrdquo Pattern Recognition vol 39no 4 pp 695ndash706 2006

[45] M Fiaz and B Ijaz ldquoVision based human activity trackingusing artificial neural networksrdquo in Proceedings of the 2010International Conference on Intelligent and Advanced Systemspp 1ndash5 IEEE Kuala Lumpur Malaysia June 2010

[46] H Foroughi A Naseri A Saberi and H S Yazdi ldquoAneigenspace-based approach for human fall detection usingintegrated time motion image and neural networkrdquo in Pro-ceedings of the 2008 9th International Conference on SignalProcessing pp 1499ndash1503 IEEE Beijing China October2008

[47] A K B Sonali ldquoHuman action recognition using supportvector machine and k-nearest neighborrdquo InternationalJournal of Engineering and Technical Research vol 3 no 4pp 423ndash428 2015

[48] V Swarnambigai ldquoAction recognition using ami and supportvector machinerdquo International Journal of Computer Scienceand Technology vol 5 no 4 pp 175ndash179 2014

[49] D Das and S Saharia ldquoHuman gait analysis and recognitionusing support vector machinesrdquo International Journal ofComputer Science and Information Technology vol 6 no 52004

[50] K G M Chathuramali and R Rodrigo ldquoFaster human activityrecognition with SVMrdquo in Proceedings of the InternationalConference on Advances in ICT for Emerging Regions(ICTer2012) pp 197ndash203 IEEE Colombo Sri Lanka De-cember 2012

[51] M Z Uddin D-H Kim J T Kim and T-S Kim ldquoAn indoorhuman activity recognition system for smart home using localbinary pattern features with hidden markov modelsrdquo Indoorand Built Environment vol 22 no 1 pp 289ndash298 2013

[52] M Z Uddin J Lee and T-S Kim ldquoIndependent shapecomponent-based human activity recognition via hiddenMarkov modelrdquo Applied Intelligence vol 33 no 2 pp 193ndash206 2010

[53] S Kumar and M Hebert ldquoDiscriminative Fields for ModelingSpatial Dependencies in Natural Imagesrdquo in Proceedings of theNIPS Vancouver Canada December 2003

[54] A Quattoni S Wang L-P Morency M Collins andT Darrell ldquoHidden conditional random fieldsrdquo IEEETransactions on Pattern Analysis and Machine Intelligencevol 29 no 10 pp 1848ndash1852 2007

[55] M Mahajan A Gunawardana and A Acero ldquoTraining al-gorithms for hidden conditional random fieldsrdquo in Pro-ceedings of the 2006 IEEE International Conference onAcoustics Speed and Signal Processing vol 1 IEEE ToulouseFrance May 2006

[56] S Reiter B Schuller and G Rigoll ldquoHidden conditionalrandom fields for meeting segmentationrdquo in Proceedings ofthe Multimedia and Expo 2007 IEEE International Confer-ence pp 639ndash642 IEEE Beijing China July 2007

[57] L Gorelick M Blank E Shechtman M Irani and R BasrildquoActions as space-time shapesrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 29 no 12 pp 2247ndash2253 2007

[58] I Laptev M Marszalek C Schmid and B RozenfeldldquoLearning realistic human actions from moviesrdquo in Pro-ceedings of the 2008 IEEE Conference on Computer Vision andPattern Recognition pp 1ndash8 IEEE Anchorage AK USAJune 2008

[59] K Soomro and A R Zamir ldquoAction recognition in realisticsports videosrdquo in Computer Vision in Sports pp 181ndash208Springer Berlin Germany 2014

[60] D Weinland E Boyer and R Ronfard ldquoAction recognitionfrom arbitrary views using 3D exemplarsrdquo in Proceedings ofthe 2007 IEEE 11th International Conference on ComputerVision pp 1ndash7 IEEE Rio de Janeiro Brazil October 2007

[61] M Siddiqi S Lee Y-K Lee A Khan and P Truc ldquoHier-archical recognition scheme for human facial expressionrecognition systemsrdquo Sensors vol 13 no 12 pp 16682ndash167132013

[62] M Elmezain and A Al-Hamadi ldquoVision-based human ac-tivity recognition using ldcrfsrdquo International Arab Journal ofInformation Technology (IAJIT) vol 15 no 3 pp 389ndash3952018

[63] A Roy B Banerjee and V Murino ldquoA novel dictionarylearning based multiple instance learning approach to actionrecognition from videosrdquo in Proceedings of the 6th In-ternational Conference on Pattern Recognition Applicationsand Methods pp 519ndash526 Porto Portugal February 2017

[64] A P Sirat ldquoActions as space-time shapesrdquo InternationalJournal of Application or Innovation in Engineering andManagement vol 7 no 8 pp 49ndash54 2018

14 Computational Intelligence and Neuroscience

Computer Games Technology

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

Advances in

FuzzySystems

Hindawiwwwhindawicom

Volume 2018

International Journal of

ReconfigurableComputing

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

thinspArtificial Intelligence

Hindawiwwwhindawicom Volumethinsp2018

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawiwwwhindawicom Volume 2018

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Engineering Mathematics

International Journal of

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Computational Intelligence and Neuroscience

Hindawiwwwhindawicom Volume 2018

Mathematical Problems in Engineering

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Hindawiwwwhindawicom Volume 2018

Human-ComputerInteraction

Advances in

Hindawiwwwhindawicom Volume 2018

Scientic Programming

Submit your manuscripts atwwwhindawicom

Page 12: Human Activity Recognition Using Gaussian Mixture Hidden …downloads.hindawi.com/journals/cin/2019/8590560.pdf · 2019-08-22 · ResearchArticle Human Activity Recognition Using

It is obvious from Table 9 that the proposed methodshowed a significant performance against the existing state-of-the-art methods erefore the proposed method accu-rately and robustly recognizes the human activities usingdifferent video data

54 Fourth Experiment In this experiment we have pre-sented the computational complexity that is also one of thecontributions in this paper e implementations of theprevious HCRF are available in literature which calculatethe gradients by reiterating the forward and backwardtechniques while the proposed HCRF model executes themonce only and cashes the outcomes for the later use From(21) and (22) it is clear that the forward or backwardtechnique has a complexity ofO(TQ2M) where Trepresentsthe input sequence lengthQ represents the number of statesand M indicates the number of mixtures e proposedHCRFmodel however requires a full complexity of O(TM)

to calculate gradients as can be seen from (22)ndash(29)Figure 3 shows a comparison of the execution time when

the gradients are computed by the forward (or backward)algorithm and by our proposed method e computationaltime is calculated by running Matlab R2013a with thespecification of Intelreg Pentiumreg Coretrade i7-6700 (34GHz)with a RAM capacity of 16GB

6 Conclusion

In healthcare and telemedicine the human activity rec-ognition (HAR) can be best explained by helping physi-cally disable personsrsquo scenario A paralyzed patient withhalf of the body critically attacked by paralysis is com-pletely unable to perform their daily exercises e doctorsrecommend specific activities to get better improvement intheir health So for this purpose the doctors need a humanactivity recognition (HAR) system through which they canmonitor the patientsrsquo daily routines (activities) on a reg-ular basis

e accuracy of most of the HAR systems depends uponthe recognition modules For feature extraction and selec-tion modules we used some of the existing well-knownmethods while for the recognition module we proposed theusage of HCRF model which is capable of approximating acomplex distribution using a mixture of Gaussian densityfunctions e proposed model was assessed against fourpublicly available standard action datasets From our ex-periments it is obvious that the proposed full-covarianceGaussian density function showed a significant improve-ment in accuracy than the existing state-of-the-art methodsFurthermore we also proved that such improvement issignificant from statistical point of view by showing valuele02 of the comparison Similarly the complexity analysispoints out that the proposed computational method stronglydecreases the execution time for the hidden conditionalrandom field model

e ultimate goal of this study is to deploy the proposedmodel on smartphones Currently the proposed model isusing full-covariance matrix however this might be time

consuming especially when using on smartphones Using alightweight classifier such as K-nearest neighbor (K-NN)could be one possible solution But K-NN is very muchsensitive to environmental factor (like noise) erefore infuture we will try to investigate further research to reducethe time and sustain the same recognition rate whenemploying on smartphones in real environment

Data Availability

e data used to support the findings of this study areavailable from the corresponding author upon request

Conflicts of Interest

e authors declare there are no conflicts of interest

Acknowledgments

is study was supported by the Jouf University SakakaAljouf Kingdom of Saudi Arabia under the registration no39791

References

[1] Z Chang J Cao and Y Zhang ldquoA novel image segmentationapproach for wood plate surface defect classification throughconvex optimizationrdquo Journal of Forestry Research vol 29no 6 pp 1789ndash1795 2018

[2] Z S Abdallah M M Gaber B Srinivasan andS Krishnaswamy ldquoAdaptive mobile activity recognitionsystemwith evolving data streamsrdquoNeurocomputing vol 150pp 304ndash317 2015

[3] M S Bakli M A Sakr and T H A Soliman ldquoA spatio-temporal algebra in Hadoop for moving objectsrdquo Geo-SpatialInformation Science vol 21 no 2 pp 102ndash114 2018

[4] W Zhao L Yan and Y Zhang ldquoGeometric-constrainedmulti-view image matching method based on semi-globaloptimizationrdquo Geo-Spatial Information Science vol 21 no 2pp 115ndash126 2018

[5] H Gao X Zhang J Zhao and D Li ldquoTechnology of in-telligent driving radar perception based on driving brainrdquoCAAI Transactions on Intelligence Technology vol 2 no 3pp 93ndash100 2017

[6] Y Wang X Jiang R Cao and X Wang ldquoRobust indoorhuman activity recognition using wireless signalsrdquo Sensorsvol 15 no 7 pp 17195ndash17208 2015

[7] J Wang Z Liu Y Wu and J Yuan ldquoMining actionlet ensemblefor action recognition with depth camerasrdquo in Proceedings of theIEEE Conference on Computer Vision and Pattern Recognitionpp 1290ndash1297 IEEE Providence RI USA June 2012

[8] S Karungaru M Daikoku and K Terada ldquoMulti camerasbased indoors human action recognition using fuzzy rulesrdquoJournal of Pattern Recognition Research vol 10 no 1pp 61ndash74 2015

[9] S Shukri L M Kamarudin and M H F Rahiman ldquoDevice-free localization for human activity monitoringrdquo in IntelligentVideo Surveillance IntechOpen London UK 2018

[10] L Fiore D Fehr R Bodor A Drenner G Somasundaramand N Papanikolopoulos ldquoMulti-camera human activitymonitoringrdquo Journal of Intelligent and Robotic Systemsvol 52 no 1 pp 5ndash43 2008

12 Computational Intelligence and Neuroscience

[11] C Zhang and Y Tian ldquoRGB-D camera-based daily livingactivity recognitionrdquo Journal of Computer Vision and ImageProcessing vol 2 no 4 pp 1ndash7 2012

[12] M Leo T DrsquoOrazio and P Spagnolo ldquoHuman activityrecognition for automatic visual surveillance of wide areasrdquo inProceedings of the ACM 2nd International Workshop on VideoSurveillance and Sensor Networks pp 124ndash130 ACM NewYork NY USA October 2004

[13] C-B Jin S Li and H Kim ldquoReal-time action detection invideo surveillance using sub-action descriptor with multi-CNNrdquo 2017 httpsarxivorgabs171003383

[14] W Niu J Long D Han and Y-F Wang ldquoHuman activitydetection and recognition for video surveillancerdquo in Pro-ceedings of the 2004 IEEE International Conference on Mul-timedia and Expo (ICME) (IEEE Cat No 04TH8763) vol 1pp 719ndash722 IEEE Taipei Taiwan June 2004

[15] Q Li M Zheng F Li J Wang Y-A Geng and H JiangldquoRetinal image segmentation using double-scale non-linearthresholding on vessel support regionsrdquo CAAI Transactionson Intelligence Technology vol 2 no 3 pp 109ndash115 2017

[16] L M G Fonseca L M Namikawa and E F CastejonldquoDigital image processing in remote sensingrdquo in Proceedingsof the 2009 Tutorials of the XXII Brazilian Symposium onComputer Graphics and Image Processing pp 59ndash71 IEEERio de Janeiro Brazil October 2009

[17] K Buys C Cagniart A Baksheev T De Laet J De Schutterand C Pantofaru ldquoAn adaptable system for RGB-D basedhuman body detection and pose estimationrdquo Journal of VisualCommunication and Image Representation vol 25 no 1pp 39ndash52 2014

[18] M Sharif M A Khan T Akram M Y Javed T Saba andA Rehman ldquoA framework of human detection and actionrecognition based on uniform segmentation and combinationof Euclidean distance and joint entropy-based features se-lectionrdquo EURASIP Journal on Image and Video Processingvol 2017 no 1 p 89 2017

[19] K-P Chou M Prasad D Wu et al ldquoRobust feature-basedautomated multi-view human action recognition systemrdquoIEEE Access vol 6 pp 15283ndash15296 2018

[20] M Ullah H Ullah and I M Alseadonn ldquoHuman actionrecognition in videos using stable featuresrdquo Signal and ImageProcessing An International Journal vol 8 no 6 pp 1ndash102017

[21] M Sharif M A Khan F Zahid J H Shah and T AkramldquoHuman action recognition a framework of statisticalweighted segmentation and rank correlation-based selectionrdquoPattern Analysis and Applications pp 1ndash14 2019

[22] J Zang L Wang Z Liu Q Zhang G Hua and N ZhengldquoAttention-based temporal weighted convolutional neuralnetwork for action recognitionrdquo in Proceedings of the IFIPInternational Conference on Artificial Intelligence Applicationsand Innovations pp 97ndash108 Springer Rhodes Greece May2018

[23] V-D Hoang ldquoMultiple classifier-based spatiotemporal fea-tures for living activity predictionrdquo Journal of Informationand Telecommunication vol 1 no 1 pp 100ndash112 2017

[24] S Dharmalingam and A Palanisamy ldquoVector space basedaugmented structural kinematic feature descriptor for humanactivity recognition in videosrdquo ETRI Journal vol 40 no 4pp 499ndash510 2018

[25] D Liu Y Yan M-L Shyu G Zhao and M Chen ldquoSpatio-temporal analysis for human action detection and recognitionin uncontrolled environmentsrdquo International Journal of

Multimedia Data Engineering and Management vol 6 no 1pp 1ndash18 2015

[26] C A Ronao and S-B Cho ldquoHuman activity recognition withsmartphone sensors using deep learning neural networksrdquoExpert Systems with Applications vol 59 pp 235ndash244 2016

[27] A arwat H Mahdi M Elhoseny and A E HassanienldquoRecognizing human activity in mobile crowdsensing envi-ronment using optimized k-NN algorithmrdquo Expert Systemswith Applications vol 107 pp 32ndash44 2018

[28] A Jalal Y-H Kim Y-J Kim S Kamal and D Kim ldquoRobusthuman activity recognition from depth video using spatio-temporal multi-fused featuresrdquo Pattern Recognition vol 61pp 295ndash308 2017

[29] J Lafferty A McCallum and F C Pereira ldquoConditionalrandom fields probabilistic models for segmenting and la-beling sequence datardquo in Proceedings of the 18th InternationalConference on Machine Learning (ICML-2001) pp 282ndash289Williamstown MA USA June-July 2001

[30] A Gunawardana M Mahajan A Acero and J C PlattldquoHidden conditional random fields for phone classificationrdquoin Proceedings of the INTERSPEECH vol 2 pp 1117ndash1120Citeseer Lisbon Portugal September 2005

[31] L Yang Q Song Z Wang and M Jiang ldquoParsing r-cnn forinstance-level human analysisrdquo in Proceedings of the IEEEConference on Computer Vision and Pattern Recognitionpp 364ndash373 Long Beach CA USA June 2019

[32] X Liang Y Wei L Lin et al ldquoLearning to segment human bywatching youtuberdquo IEEE Transactions on Pattern Analysisand Machine Intelligence vol 39 no 7 pp 1462ndash1468 2016

[33] M M Requena ldquoHuman segmentation with convolutionalneural networksrdquo esis University of Alicante AlicanteSpain 2018

[34] H Gao S Chen and Z Zhang ldquoParts semantic segmentationaware representation learning for person re-identificationrdquoApplied Sciences vol 9 no 6 p 1239 2019

[35] C Peng S-L Lo J Huang and A C Tsoi ldquoHuman actionsegmentation based on a streaming uniform entropy slicemethodrdquo IEEE Access vol 6 pp 16958ndash16971 2018

[36] M H Siddiqi A M Khan and S-W Lee ldquoActive contourslevel set based still human body segmentation from depthimages for video-based activity recognitionrdquo KSII Trans-actions on Internet and Information Systems (TIIS) vol 7no 11 pp 2839ndash2852 2013

[37] M Siddiqi R Ali M Rana E-K Hong E Kim and S LeeldquoVideo-based human activity recognition using multilevelwavelet decomposition and stepwise linear discriminantanalysisrdquo Sensors vol 14 no 4 pp 6370ndash6392 2014

[38] S Althloothi M H Mahoor X Zhang and R M VoylesldquoHuman activity recognition using multi-features and mul-tiple kernel learningrdquo Pattern Recognition vol 47 no 5pp 1800ndash1812 2014

[39] A Jalal S Kamal and D Kim ldquoA depth video-based humandetection and activity recognition using multi-features andembedded hidden markov models for health care monitoringsystemsrdquo International Journal of Interactive Multimedia andArtificial Intelligence vol 4 no 4 p 54 2017

[40] T Dobhal V Shitole G omas and G Navada ldquoHumanactivity recognition using binary motion image and deeplearningrdquo Procedia Computer Science vol 58 pp 178ndash1852015

[41] S Zhang Z Wei J Nie L Huang S Wang and Z Li ldquoAreview on human activity recognition using vision-basedmethodrdquo Journal of Healthcare Engineering vol 2017 ArticleID 3090343 31 pages 2017

Computational Intelligence and Neuroscience 13

[42] Y Yang B Zhang L Yang C Chen and W Yang ldquoActionrecognition using completed local binary patterns and mul-tiple-class boosting classifierrdquo in Proceedings of the 2015 3rdIAPR Asian Conference on Pattern Recognition (ACPR) IEEEpp 336ndash340 Kuala Lumpur Malaysia November 2015

[43] W Lin M T Sun R Poovandran and Z Zhang ldquoHumanactivity recognition for video surveillancerdquo in Proceedings ofthe 2008 IEEE International Symposium on Circuits andSystems pp 2737ndash2740 IEEE Seattle WA USA May 2008

[44] H Permuter J Francos and I Jermyn ldquoA study of Gaussianmixture models of color and texture features for imageclassification and segmentationrdquo Pattern Recognition vol 39no 4 pp 695ndash706 2006

[45] M Fiaz and B Ijaz ldquoVision based human activity trackingusing artificial neural networksrdquo in Proceedings of the 2010International Conference on Intelligent and Advanced Systemspp 1ndash5 IEEE Kuala Lumpur Malaysia June 2010

[46] H Foroughi A Naseri A Saberi and H S Yazdi ldquoAneigenspace-based approach for human fall detection usingintegrated time motion image and neural networkrdquo in Pro-ceedings of the 2008 9th International Conference on SignalProcessing pp 1499ndash1503 IEEE Beijing China October2008

[47] A K B Sonali ldquoHuman action recognition using supportvector machine and k-nearest neighborrdquo InternationalJournal of Engineering and Technical Research vol 3 no 4pp 423ndash428 2015

[48] V Swarnambigai ldquoAction recognition using ami and supportvector machinerdquo International Journal of Computer Scienceand Technology vol 5 no 4 pp 175ndash179 2014

[49] D Das and S Saharia ldquoHuman gait analysis and recognitionusing support vector machinesrdquo International Journal ofComputer Science and Information Technology vol 6 no 52004

[50] K G M Chathuramali and R Rodrigo ldquoFaster human activityrecognition with SVMrdquo in Proceedings of the InternationalConference on Advances in ICT for Emerging Regions(ICTer2012) pp 197ndash203 IEEE Colombo Sri Lanka De-cember 2012

[51] M Z Uddin D-H Kim J T Kim and T-S Kim ldquoAn indoorhuman activity recognition system for smart home using localbinary pattern features with hidden markov modelsrdquo Indoorand Built Environment vol 22 no 1 pp 289ndash298 2013

[52] M Z Uddin J Lee and T-S Kim ldquoIndependent shapecomponent-based human activity recognition via hiddenMarkov modelrdquo Applied Intelligence vol 33 no 2 pp 193ndash206 2010

[53] S Kumar and M Hebert ldquoDiscriminative Fields for ModelingSpatial Dependencies in Natural Imagesrdquo in Proceedings of theNIPS Vancouver Canada December 2003

[54] A Quattoni S Wang L-P Morency M Collins andT Darrell ldquoHidden conditional random fieldsrdquo IEEETransactions on Pattern Analysis and Machine Intelligencevol 29 no 10 pp 1848ndash1852 2007

[55] M Mahajan A Gunawardana and A Acero ldquoTraining al-gorithms for hidden conditional random fieldsrdquo in Pro-ceedings of the 2006 IEEE International Conference onAcoustics Speed and Signal Processing vol 1 IEEE ToulouseFrance May 2006

[56] S Reiter B Schuller and G Rigoll ldquoHidden conditionalrandom fields for meeting segmentationrdquo in Proceedings ofthe Multimedia and Expo 2007 IEEE International Confer-ence pp 639ndash642 IEEE Beijing China July 2007

[57] L Gorelick M Blank E Shechtman M Irani and R BasrildquoActions as space-time shapesrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 29 no 12 pp 2247ndash2253 2007

[58] I Laptev M Marszalek C Schmid and B RozenfeldldquoLearning realistic human actions from moviesrdquo in Pro-ceedings of the 2008 IEEE Conference on Computer Vision andPattern Recognition pp 1ndash8 IEEE Anchorage AK USAJune 2008

[59] K Soomro and A R Zamir ldquoAction recognition in realisticsports videosrdquo in Computer Vision in Sports pp 181ndash208Springer Berlin Germany 2014

[60] D Weinland E Boyer and R Ronfard ldquoAction recognitionfrom arbitrary views using 3D exemplarsrdquo in Proceedings ofthe 2007 IEEE 11th International Conference on ComputerVision pp 1ndash7 IEEE Rio de Janeiro Brazil October 2007

[61] M Siddiqi S Lee Y-K Lee A Khan and P Truc ldquoHier-archical recognition scheme for human facial expressionrecognition systemsrdquo Sensors vol 13 no 12 pp 16682ndash167132013

[62] M Elmezain and A Al-Hamadi ldquoVision-based human ac-tivity recognition using ldcrfsrdquo International Arab Journal ofInformation Technology (IAJIT) vol 15 no 3 pp 389ndash3952018

[63] A Roy B Banerjee and V Murino ldquoA novel dictionarylearning based multiple instance learning approach to actionrecognition from videosrdquo in Proceedings of the 6th In-ternational Conference on Pattern Recognition Applicationsand Methods pp 519ndash526 Porto Portugal February 2017

[64] A P Sirat ldquoActions as space-time shapesrdquo InternationalJournal of Application or Innovation in Engineering andManagement vol 7 no 8 pp 49ndash54 2018

14 Computational Intelligence and Neuroscience

Computer Games Technology

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

Advances in

FuzzySystems

Hindawiwwwhindawicom

Volume 2018

International Journal of

ReconfigurableComputing

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

thinspArtificial Intelligence

Hindawiwwwhindawicom Volumethinsp2018

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawiwwwhindawicom Volume 2018

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Engineering Mathematics

International Journal of

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Computational Intelligence and Neuroscience

Hindawiwwwhindawicom Volume 2018

Mathematical Problems in Engineering

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Hindawiwwwhindawicom Volume 2018

Human-ComputerInteraction

Advances in

Hindawiwwwhindawicom Volume 2018

Scientic Programming

Submit your manuscripts atwwwhindawicom

Page 13: Human Activity Recognition Using Gaussian Mixture Hidden …downloads.hindawi.com/journals/cin/2019/8590560.pdf · 2019-08-22 · ResearchArticle Human Activity Recognition Using

[11] C Zhang and Y Tian ldquoRGB-D camera-based daily livingactivity recognitionrdquo Journal of Computer Vision and ImageProcessing vol 2 no 4 pp 1ndash7 2012

[12] M Leo T DrsquoOrazio and P Spagnolo ldquoHuman activityrecognition for automatic visual surveillance of wide areasrdquo inProceedings of the ACM 2nd International Workshop on VideoSurveillance and Sensor Networks pp 124ndash130 ACM NewYork NY USA October 2004

[13] C-B Jin S Li and H Kim ldquoReal-time action detection invideo surveillance using sub-action descriptor with multi-CNNrdquo 2017 httpsarxivorgabs171003383

[14] W Niu J Long D Han and Y-F Wang ldquoHuman activitydetection and recognition for video surveillancerdquo in Pro-ceedings of the 2004 IEEE International Conference on Mul-timedia and Expo (ICME) (IEEE Cat No 04TH8763) vol 1pp 719ndash722 IEEE Taipei Taiwan June 2004

[15] Q Li M Zheng F Li J Wang Y-A Geng and H JiangldquoRetinal image segmentation using double-scale non-linearthresholding on vessel support regionsrdquo CAAI Transactionson Intelligence Technology vol 2 no 3 pp 109ndash115 2017

[16] L M G Fonseca L M Namikawa and E F CastejonldquoDigital image processing in remote sensingrdquo in Proceedingsof the 2009 Tutorials of the XXII Brazilian Symposium onComputer Graphics and Image Processing pp 59ndash71 IEEERio de Janeiro Brazil October 2009

[17] K Buys C Cagniart A Baksheev T De Laet J De Schutterand C Pantofaru ldquoAn adaptable system for RGB-D basedhuman body detection and pose estimationrdquo Journal of VisualCommunication and Image Representation vol 25 no 1pp 39ndash52 2014

[18] M Sharif M A Khan T Akram M Y Javed T Saba andA Rehman ldquoA framework of human detection and actionrecognition based on uniform segmentation and combinationof Euclidean distance and joint entropy-based features se-lectionrdquo EURASIP Journal on Image and Video Processingvol 2017 no 1 p 89 2017

[19] K-P Chou M Prasad D Wu et al ldquoRobust feature-basedautomated multi-view human action recognition systemrdquoIEEE Access vol 6 pp 15283ndash15296 2018

[20] M Ullah H Ullah and I M Alseadonn ldquoHuman actionrecognition in videos using stable featuresrdquo Signal and ImageProcessing An International Journal vol 8 no 6 pp 1ndash102017

[21] M Sharif M A Khan F Zahid J H Shah and T AkramldquoHuman action recognition a framework of statisticalweighted segmentation and rank correlation-based selectionrdquoPattern Analysis and Applications pp 1ndash14 2019

[22] J Zang L Wang Z Liu Q Zhang G Hua and N ZhengldquoAttention-based temporal weighted convolutional neuralnetwork for action recognitionrdquo in Proceedings of the IFIPInternational Conference on Artificial Intelligence Applicationsand Innovations pp 97ndash108 Springer Rhodes Greece May2018

[23] V-D Hoang ldquoMultiple classifier-based spatiotemporal fea-tures for living activity predictionrdquo Journal of Informationand Telecommunication vol 1 no 1 pp 100ndash112 2017

[24] S Dharmalingam and A Palanisamy ldquoVector space basedaugmented structural kinematic feature descriptor for humanactivity recognition in videosrdquo ETRI Journal vol 40 no 4pp 499ndash510 2018

[25] D Liu Y Yan M-L Shyu G Zhao and M Chen ldquoSpatio-temporal analysis for human action detection and recognitionin uncontrolled environmentsrdquo International Journal of

Multimedia Data Engineering and Management vol 6 no 1pp 1ndash18 2015

[26] C A Ronao and S-B Cho ldquoHuman activity recognition withsmartphone sensors using deep learning neural networksrdquoExpert Systems with Applications vol 59 pp 235ndash244 2016

[27] A arwat H Mahdi M Elhoseny and A E HassanienldquoRecognizing human activity in mobile crowdsensing envi-ronment using optimized k-NN algorithmrdquo Expert Systemswith Applications vol 107 pp 32ndash44 2018

[28] A Jalal Y-H Kim Y-J Kim S Kamal and D Kim ldquoRobusthuman activity recognition from depth video using spatio-temporal multi-fused featuresrdquo Pattern Recognition vol 61pp 295ndash308 2017

[29] J Lafferty A McCallum and F C Pereira ldquoConditionalrandom fields probabilistic models for segmenting and la-beling sequence datardquo in Proceedings of the 18th InternationalConference on Machine Learning (ICML-2001) pp 282ndash289Williamstown MA USA June-July 2001

[30] A Gunawardana M Mahajan A Acero and J C PlattldquoHidden conditional random fields for phone classificationrdquoin Proceedings of the INTERSPEECH vol 2 pp 1117ndash1120Citeseer Lisbon Portugal September 2005

[31] L Yang Q Song Z Wang and M Jiang ldquoParsing r-cnn forinstance-level human analysisrdquo in Proceedings of the IEEEConference on Computer Vision and Pattern Recognitionpp 364ndash373 Long Beach CA USA June 2019

[32] X Liang Y Wei L Lin et al ldquoLearning to segment human bywatching youtuberdquo IEEE Transactions on Pattern Analysisand Machine Intelligence vol 39 no 7 pp 1462ndash1468 2016

[33] M M Requena ldquoHuman segmentation with convolutionalneural networksrdquo esis University of Alicante AlicanteSpain 2018

[34] H Gao S Chen and Z Zhang ldquoParts semantic segmentationaware representation learning for person re-identificationrdquoApplied Sciences vol 9 no 6 p 1239 2019

[35] C Peng S-L Lo J Huang and A C Tsoi ldquoHuman actionsegmentation based on a streaming uniform entropy slicemethodrdquo IEEE Access vol 6 pp 16958ndash16971 2018

[36] M H Siddiqi A M Khan and S-W Lee ldquoActive contourslevel set based still human body segmentation from depthimages for video-based activity recognitionrdquo KSII Trans-actions on Internet and Information Systems (TIIS) vol 7no 11 pp 2839ndash2852 2013

[37] M Siddiqi R Ali M Rana E-K Hong E Kim and S LeeldquoVideo-based human activity recognition using multilevelwavelet decomposition and stepwise linear discriminantanalysisrdquo Sensors vol 14 no 4 pp 6370ndash6392 2014

[38] S Althloothi M H Mahoor X Zhang and R M VoylesldquoHuman activity recognition using multi-features and mul-tiple kernel learningrdquo Pattern Recognition vol 47 no 5pp 1800ndash1812 2014

[39] A Jalal S Kamal and D Kim ldquoA depth video-based humandetection and activity recognition using multi-features andembedded hidden markov models for health care monitoringsystemsrdquo International Journal of Interactive Multimedia andArtificial Intelligence vol 4 no 4 p 54 2017

[40] T Dobhal V Shitole G omas and G Navada ldquoHumanactivity recognition using binary motion image and deeplearningrdquo Procedia Computer Science vol 58 pp 178ndash1852015

[41] S Zhang Z Wei J Nie L Huang S Wang and Z Li ldquoAreview on human activity recognition using vision-basedmethodrdquo Journal of Healthcare Engineering vol 2017 ArticleID 3090343 31 pages 2017

Computational Intelligence and Neuroscience 13

[42] Y Yang B Zhang L Yang C Chen and W Yang ldquoActionrecognition using completed local binary patterns and mul-tiple-class boosting classifierrdquo in Proceedings of the 2015 3rdIAPR Asian Conference on Pattern Recognition (ACPR) IEEEpp 336ndash340 Kuala Lumpur Malaysia November 2015

[43] W Lin M T Sun R Poovandran and Z Zhang ldquoHumanactivity recognition for video surveillancerdquo in Proceedings ofthe 2008 IEEE International Symposium on Circuits andSystems pp 2737ndash2740 IEEE Seattle WA USA May 2008

[44] H Permuter J Francos and I Jermyn ldquoA study of Gaussianmixture models of color and texture features for imageclassification and segmentationrdquo Pattern Recognition vol 39no 4 pp 695ndash706 2006

[45] M Fiaz and B Ijaz ldquoVision based human activity trackingusing artificial neural networksrdquo in Proceedings of the 2010International Conference on Intelligent and Advanced Systemspp 1ndash5 IEEE Kuala Lumpur Malaysia June 2010

[46] H Foroughi A Naseri A Saberi and H S Yazdi ldquoAneigenspace-based approach for human fall detection usingintegrated time motion image and neural networkrdquo in Pro-ceedings of the 2008 9th International Conference on SignalProcessing pp 1499ndash1503 IEEE Beijing China October2008

[47] A K B Sonali ldquoHuman action recognition using supportvector machine and k-nearest neighborrdquo InternationalJournal of Engineering and Technical Research vol 3 no 4pp 423ndash428 2015

[48] V Swarnambigai ldquoAction recognition using ami and supportvector machinerdquo International Journal of Computer Scienceand Technology vol 5 no 4 pp 175ndash179 2014

[49] D Das and S Saharia ldquoHuman gait analysis and recognitionusing support vector machinesrdquo International Journal ofComputer Science and Information Technology vol 6 no 52004

[50] K G M Chathuramali and R Rodrigo ldquoFaster human activityrecognition with SVMrdquo in Proceedings of the InternationalConference on Advances in ICT for Emerging Regions(ICTer2012) pp 197ndash203 IEEE Colombo Sri Lanka De-cember 2012

[51] M Z Uddin D-H Kim J T Kim and T-S Kim ldquoAn indoorhuman activity recognition system for smart home using localbinary pattern features with hidden markov modelsrdquo Indoorand Built Environment vol 22 no 1 pp 289ndash298 2013

[52] M Z Uddin J Lee and T-S Kim ldquoIndependent shapecomponent-based human activity recognition via hiddenMarkov modelrdquo Applied Intelligence vol 33 no 2 pp 193ndash206 2010

[53] S Kumar and M Hebert ldquoDiscriminative Fields for ModelingSpatial Dependencies in Natural Imagesrdquo in Proceedings of theNIPS Vancouver Canada December 2003

[54] A Quattoni S Wang L-P Morency M Collins andT Darrell ldquoHidden conditional random fieldsrdquo IEEETransactions on Pattern Analysis and Machine Intelligencevol 29 no 10 pp 1848ndash1852 2007

[55] M Mahajan A Gunawardana and A Acero ldquoTraining al-gorithms for hidden conditional random fieldsrdquo in Pro-ceedings of the 2006 IEEE International Conference onAcoustics Speed and Signal Processing vol 1 IEEE ToulouseFrance May 2006

[56] S Reiter B Schuller and G Rigoll ldquoHidden conditionalrandom fields for meeting segmentationrdquo in Proceedings ofthe Multimedia and Expo 2007 IEEE International Confer-ence pp 639ndash642 IEEE Beijing China July 2007

[57] L Gorelick M Blank E Shechtman M Irani and R BasrildquoActions as space-time shapesrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 29 no 12 pp 2247ndash2253 2007

[58] I Laptev M Marszalek C Schmid and B RozenfeldldquoLearning realistic human actions from moviesrdquo in Pro-ceedings of the 2008 IEEE Conference on Computer Vision andPattern Recognition pp 1ndash8 IEEE Anchorage AK USAJune 2008

[59] K Soomro and A R Zamir ldquoAction recognition in realisticsports videosrdquo in Computer Vision in Sports pp 181ndash208Springer Berlin Germany 2014

[60] D Weinland E Boyer and R Ronfard ldquoAction recognitionfrom arbitrary views using 3D exemplarsrdquo in Proceedings ofthe 2007 IEEE 11th International Conference on ComputerVision pp 1ndash7 IEEE Rio de Janeiro Brazil October 2007

[61] M Siddiqi S Lee Y-K Lee A Khan and P Truc ldquoHier-archical recognition scheme for human facial expressionrecognition systemsrdquo Sensors vol 13 no 12 pp 16682ndash167132013

[62] M Elmezain and A Al-Hamadi ldquoVision-based human ac-tivity recognition using ldcrfsrdquo International Arab Journal ofInformation Technology (IAJIT) vol 15 no 3 pp 389ndash3952018

[63] A Roy B Banerjee and V Murino ldquoA novel dictionarylearning based multiple instance learning approach to actionrecognition from videosrdquo in Proceedings of the 6th In-ternational Conference on Pattern Recognition Applicationsand Methods pp 519ndash526 Porto Portugal February 2017

[64] A P Sirat ldquoActions as space-time shapesrdquo InternationalJournal of Application or Innovation in Engineering andManagement vol 7 no 8 pp 49ndash54 2018

14 Computational Intelligence and Neuroscience

Computer Games Technology

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

Advances in

FuzzySystems

Hindawiwwwhindawicom

Volume 2018

International Journal of

ReconfigurableComputing

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

thinspArtificial Intelligence

Hindawiwwwhindawicom Volumethinsp2018

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawiwwwhindawicom Volume 2018

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Engineering Mathematics

International Journal of

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Computational Intelligence and Neuroscience

Hindawiwwwhindawicom Volume 2018

Mathematical Problems in Engineering

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Hindawiwwwhindawicom Volume 2018

Human-ComputerInteraction

Advances in

Hindawiwwwhindawicom Volume 2018

Scientic Programming

Submit your manuscripts atwwwhindawicom

Page 14: Human Activity Recognition Using Gaussian Mixture Hidden …downloads.hindawi.com/journals/cin/2019/8590560.pdf · 2019-08-22 · ResearchArticle Human Activity Recognition Using

[42] Y Yang B Zhang L Yang C Chen and W Yang ldquoActionrecognition using completed local binary patterns and mul-tiple-class boosting classifierrdquo in Proceedings of the 2015 3rdIAPR Asian Conference on Pattern Recognition (ACPR) IEEEpp 336ndash340 Kuala Lumpur Malaysia November 2015

[43] W Lin M T Sun R Poovandran and Z Zhang ldquoHumanactivity recognition for video surveillancerdquo in Proceedings ofthe 2008 IEEE International Symposium on Circuits andSystems pp 2737ndash2740 IEEE Seattle WA USA May 2008

[44] H Permuter J Francos and I Jermyn ldquoA study of Gaussianmixture models of color and texture features for imageclassification and segmentationrdquo Pattern Recognition vol 39no 4 pp 695ndash706 2006

[45] M Fiaz and B Ijaz ldquoVision based human activity trackingusing artificial neural networksrdquo in Proceedings of the 2010International Conference on Intelligent and Advanced Systemspp 1ndash5 IEEE Kuala Lumpur Malaysia June 2010

[46] H Foroughi A Naseri A Saberi and H S Yazdi ldquoAneigenspace-based approach for human fall detection usingintegrated time motion image and neural networkrdquo in Pro-ceedings of the 2008 9th International Conference on SignalProcessing pp 1499ndash1503 IEEE Beijing China October2008

[47] A K B Sonali ldquoHuman action recognition using supportvector machine and k-nearest neighborrdquo InternationalJournal of Engineering and Technical Research vol 3 no 4pp 423ndash428 2015

[48] V Swarnambigai ldquoAction recognition using ami and supportvector machinerdquo International Journal of Computer Scienceand Technology vol 5 no 4 pp 175ndash179 2014

[49] D Das and S Saharia ldquoHuman gait analysis and recognitionusing support vector machinesrdquo International Journal ofComputer Science and Information Technology vol 6 no 52004

[50] K G M Chathuramali and R Rodrigo ldquoFaster human activityrecognition with SVMrdquo in Proceedings of the InternationalConference on Advances in ICT for Emerging Regions(ICTer2012) pp 197ndash203 IEEE Colombo Sri Lanka De-cember 2012

[51] M Z Uddin D-H Kim J T Kim and T-S Kim ldquoAn indoorhuman activity recognition system for smart home using localbinary pattern features with hidden markov modelsrdquo Indoorand Built Environment vol 22 no 1 pp 289ndash298 2013

[52] M Z Uddin J Lee and T-S Kim ldquoIndependent shapecomponent-based human activity recognition via hiddenMarkov modelrdquo Applied Intelligence vol 33 no 2 pp 193ndash206 2010

[53] S Kumar and M Hebert ldquoDiscriminative Fields for ModelingSpatial Dependencies in Natural Imagesrdquo in Proceedings of theNIPS Vancouver Canada December 2003

[54] A Quattoni S Wang L-P Morency M Collins andT Darrell ldquoHidden conditional random fieldsrdquo IEEETransactions on Pattern Analysis and Machine Intelligencevol 29 no 10 pp 1848ndash1852 2007

[55] M Mahajan A Gunawardana and A Acero ldquoTraining al-gorithms for hidden conditional random fieldsrdquo in Pro-ceedings of the 2006 IEEE International Conference onAcoustics Speed and Signal Processing vol 1 IEEE ToulouseFrance May 2006

[56] S Reiter B Schuller and G Rigoll ldquoHidden conditionalrandom fields for meeting segmentationrdquo in Proceedings ofthe Multimedia and Expo 2007 IEEE International Confer-ence pp 639ndash642 IEEE Beijing China July 2007

[57] L Gorelick M Blank E Shechtman M Irani and R BasrildquoActions as space-time shapesrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 29 no 12 pp 2247ndash2253 2007

[58] I Laptev M Marszalek C Schmid and B RozenfeldldquoLearning realistic human actions from moviesrdquo in Pro-ceedings of the 2008 IEEE Conference on Computer Vision andPattern Recognition pp 1ndash8 IEEE Anchorage AK USAJune 2008

[59] K Soomro and A R Zamir ldquoAction recognition in realisticsports videosrdquo in Computer Vision in Sports pp 181ndash208Springer Berlin Germany 2014

[60] D Weinland E Boyer and R Ronfard ldquoAction recognitionfrom arbitrary views using 3D exemplarsrdquo in Proceedings ofthe 2007 IEEE 11th International Conference on ComputerVision pp 1ndash7 IEEE Rio de Janeiro Brazil October 2007

[61] M Siddiqi S Lee Y-K Lee A Khan and P Truc ldquoHier-archical recognition scheme for human facial expressionrecognition systemsrdquo Sensors vol 13 no 12 pp 16682ndash167132013

[62] M Elmezain and A Al-Hamadi ldquoVision-based human ac-tivity recognition using ldcrfsrdquo International Arab Journal ofInformation Technology (IAJIT) vol 15 no 3 pp 389ndash3952018

[63] A Roy B Banerjee and V Murino ldquoA novel dictionarylearning based multiple instance learning approach to actionrecognition from videosrdquo in Proceedings of the 6th In-ternational Conference on Pattern Recognition Applicationsand Methods pp 519ndash526 Porto Portugal February 2017

[64] A P Sirat ldquoActions as space-time shapesrdquo InternationalJournal of Application or Innovation in Engineering andManagement vol 7 no 8 pp 49ndash54 2018

14 Computational Intelligence and Neuroscience

Computer Games Technology

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

Advances in

FuzzySystems

Hindawiwwwhindawicom

Volume 2018

International Journal of

ReconfigurableComputing

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

thinspArtificial Intelligence

Hindawiwwwhindawicom Volumethinsp2018

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawiwwwhindawicom Volume 2018

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Engineering Mathematics

International Journal of

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Computational Intelligence and Neuroscience

Hindawiwwwhindawicom Volume 2018

Mathematical Problems in Engineering

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Hindawiwwwhindawicom Volume 2018

Human-ComputerInteraction

Advances in

Hindawiwwwhindawicom Volume 2018

Scientic Programming

Submit your manuscripts atwwwhindawicom

Page 15: Human Activity Recognition Using Gaussian Mixture Hidden …downloads.hindawi.com/journals/cin/2019/8590560.pdf · 2019-08-22 · ResearchArticle Human Activity Recognition Using

Computer Games Technology

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

Advances in

FuzzySystems

Hindawiwwwhindawicom

Volume 2018

International Journal of

ReconfigurableComputing

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

thinspArtificial Intelligence

Hindawiwwwhindawicom Volumethinsp2018

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications

Hindawiwwwhindawicom Volume 2018

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Engineering Mathematics

International Journal of

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Computational Intelligence and Neuroscience

Hindawiwwwhindawicom Volume 2018

Mathematical Problems in Engineering

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Hindawiwwwhindawicom Volume 2018

Human-ComputerInteraction

Advances in

Hindawiwwwhindawicom Volume 2018

Scientic Programming

Submit your manuscripts atwwwhindawicom


Recommended