3D Motion Capture Assisted Video human motion recognition

based on the Layered HMM

Myunghoon Suk & Ashok Ramadass

Advisor : Dr. B. Prabhakaran Multimedia and Networking LabThe University of Texas at Dallas

Contents

• Motivation• Previous Work• Current Work

– Extracting 2D feature data (MHI)– Classifying human motions

Motivation

CleanedSemantic Data

Easy to get, but

Quite noisy

3D MOCAP data

Video Human Motion data

Recognizing Video Human Motion

+

HMM Modeling

T 1 2 3 4 … t

1s1O2O

3O 4OmO

2s3s

4s

ns

MHI

K-means (WEKA)

2D Motion Shape data

3D Motion Capture data

ObservationSequence data

Hidden state-transition Sequence data

Quantization

Which Motion?Forehand, Backhand, Smash,

Left kick, Right Kick, Left punch, Right punch

Test data

3D Motion Capture Assisted Video Human Motion Recognition Enhancement

Current Work

• The system for falling-down detection of elderly or patient at home

• Lower layered HMMs with 3D motion capture data are to estimate one of atomic activities (e.g. movement of human hip portion)

• Higher layer recognizes exactly the falling-down motion with much longer time granularity

Layered HMM

HMM (A)HMM (A)

HMM (B) (Baum-Welch)HMM (B) (Baum-Welch)

2D Feature Vector2D Feature Vector

Horizontal directionUp directionDown direction

Normal ActionAbnormal Action(Falling down)

Classification ResultsClassification Results

Position of human hip

Movement directions

Background Techniques

• Extracting 2D feature (Computer Vision)– Motion History Images (MHI)

• Classification (Machine Learning)– Hidden Markov Model (HMM)– Layered Hidden Markov Model (LHMM)

Motion History Images

• Keywords:– Motion Energy Image (MEI)– Motion History Image (MHI)– 2D Image feature data with suggestion of possible

actions.

Motion History Images

• Motion Energy Image :-– Describes the motion energy for a given view of

action– Spatial distribution of motion – WHERE

Motion History Images

• Motion History Image :-– Pixel intensity – HOW the spatial distribution has occurred

Motion History Images

MEIMEI MHIMHIWHEREWHERE HOWHOW

2D Image Feature Date

2D Image Feature Date

Suggestion of Possible actions

Suggestion of Possible actions

Motion History Images

• Reference Paper :-– Hierarchical Motion History Images for

recognizing Human Motion.

Project - Face detection using HAAR like Features & AdaBoost algorithm

• deals with the application of one of the four AdaBoost algorithms in boosting the classifiers based on the paper "Robust Real Time Face detection by viola & jones“

• OpenCV Visual C++• Available Source files: Face detection using available HAAR like Features. • PreRequisites: Basic knowledge of using OpenCV library. Knowledge on AdaBoost(Adaptive Boosting) – A Machine Learning

Algorithm.• Other References:

http://cmp.felk.cvut.cz/~sochmj1/adaboost_talk.pdf

Project – Contour detection using Background Subtraction and Edge Detection Techniques

• OpenCV Visual C++• Available Source files:

Reading a video file.• PreRequisites:• Basic knowledge of using OpenCV library.• Other References:• Introduction to opencv programming -

http://www.cs.iit.edu/~agam/cs512/lect-notes/opencv-intro/opencv-intro.html

Thank you

Myunghoon Suk (mhs071000@utdallas.edu)Ashok Ramadass (axr081000@utdallas.edu)