GESTURE ANALYSIS

SHESHADRI M. (07MCMC02)JAGADEESHWAR CH. (07MCMC07)

Under the guidance of Prof. Bapi Raju

• Abstract• Tools used• Object Tracking• Tracking Methodology• Future work• References

Abstract:• The project aims to analyze human hand gestures from video

recordings. Video is recorded using web camera attached to a computer and off-line analysis is carried out in Matlab.

• As a pilot study, we restricted our work to slow, pre-defined hand gestures such as moving the hand sideways or vertically.

• Although the overall goal is to automatically recognize and segment various gestures, in the current work we present results from tracking of a hand gesture using centroid-based approach along with image subtraction.

Cont…..• We have done preliminary study of tracking a

moving ball using Kalman filter and attempted to apply the same procedure for hand gesture tracking.

• The future work includes implementing a stand-alone application of Kalman filter based tracking for hand gestures both for off-line as well as on-line cases.

TOOLS USED

• MATLAB R2008a• Microsoft Lifecam for capturing video.

I/O Specification

• Input : video file (.avi format) of resolution 320x240.

• storing each frame into 4D array (x, y, color, frame no.)• Output :plotting the graph

Object Tracking:

• Dividing the Video into series of image frames. We select the object of interest (specified by the user) to track. We find centroid of the object of interest using pixel values (colour) of the object. And analyzing its motion during the entire sequence of frames.

Fig.1 Fig. 2

Tracking the yellow color object from the frame

if(Z(i,j,1,z) >= 250 ) if (Z(i,j,2,z) >= 250) if (Z(i,j,3,z) >= 80 && Z(i,j,3,z) <= 200)

Fig.1 is input frameIn fig2 the green color indicates centroid of the object and we tracking the Vertical motion of the object

Subtraction technique:

img1 img2 [img1-img2]

(a) an input image with objects, (b) input image without objects and(c) difference image. Note that the foreground objects are clearly identifiable.

Motivation:It is observed that to make the tracking method more general, we need to remove other objects from the scene that are static or of no interest. Subtraction of images of successive frames will take care of this problem

Subtraction technique for object tracking

Img1, img2 are two successive frames from the input video

img1

Diff img= (img2-img1)

img2

Output of the image subtraction method.

Object Tracking using Image Subtraction

Drawbacks observed in this method:• Some times the successive frame do not show the difference due to slow motion, in this we cant track the object.

• If some objects of not interested are start moving the difference image will track the motion of those objects, it will be problem if the object is same color.

Proposal for modification:Using Windowing approach for tracking

Object Tracking by windowing approach

Using the centroid we are taking a boundary for the object. With this boundary we predict the likely location of the object in the next frame. In successive frames we search in the predicted window boundary so that the searching time Complexity is reduced to just the window of interest rather than the entire frame.

Output figure

Input image and windowing of the object

Starting point

End point

Kalman Filter

• It is a recursive algorithm for estimating the state of the moving object in the image sequences. By using the previous state, kalman filter can predict the object position in the frame.

• In order to use the kalman filter to estimate the state of a object given only a sequence of noisy observations, one must model the process in accordance with the framework of the kalman filter. This means specifying the matrices A,Hk,Qk,Rk and sometimes Bk for each time stamp k.

• It smoothens the effect of the noise in the state variable.

Where

Xk+1 is the current state of the object

A is a matrix relating the state variables at the previous time step to the state variables at the current time stepXk is the previous state of the objectB is a matrix relating the input to the state variables.u is input vector.Bu is the input that we receive (distance).

Ball tracking using Kalman filter:• We have done the preliminary test of ball tracking using the

kalman filter and we applied the same code to our video frames which has moment of object attached to hand.

Input image Output image

Input image output image

•In our video, object and hand are moving but in ball tracking only the ball is moving and background is stable.

•Because of this problem we are unable to track the yellow colored object in frame, so there is a need for refinement of the code of kalman filter.

FUTURE WORK• The future work includes implementing a

stand-alone application of Kalman filter based tracking for hand gestures both for off-line as well as on-line cases. Adaptation of the proposed centroid, subtraction and window-based approaches for on-line tracking is also left for future scope.

References• Cristin Apler Yilmaz, Omar Javed and Mubarak Shah, “Object tracking: A survey”, ACM

Computing Surveys, Vol. 38, No. 4, Article 13, Publication date: December 2006.• Mahmoud Elmezain, Ayub K. Al-Hamadi and Bernd Michaelis, “Hand tracking using kalman

filter and mean shift algorithm”, World Academy of Science, Engineering and Technology 59 2009.

• Cristina Manresa, Javier Varona, Ramon Mas and Francisco J.Perales, “Real-Time Hand Tracking and Gesture Recognition for Human-Computer Interaction”, Electronic Letters on Computer Vision and Image Analysis 0(0):1-7, 2000.

• Image processing Using MATLAB by TechSource Systems Sdn.Bhd. , 2005. • Recent advances in kalman filter theory and applications, Version 1: March 9, 2005; Version

2: May, 2006; Version 3: July 22, 2009. • Olivier Cadet, Transocean Inc. “Introduction to Kalman Filter and its Use in Dynamic

Positioning Systems”, Dynamic Positioning Conference, September 16-17, 2003.• Ball tracking:

http://www.mathworks.co.kr/matlabcentral/fileexchange/14243-2d-target-tracking-using-kalman-filter.

Thank You