Introduction to Acitivity Recognition

8/12/2019 Introduction to Acitivity Recognition

1/63

Week 2 Introduction to ActivityRecognition

RICHARD DAVIES COM815ACTIVITY MODELLING & RECOGNITION


2/63

Week 1 - Summary Activity Recognition

Why is this field expanding?

P e r v a s i v e

T e c h n o

l o g y

Activity Modelling & Recognition

P r o c e s s o r

p o w e r

B a t t e r y S i z e

C o s t

Increased Functionality

Environment

Wearable


3/63

Coursework 1

Investigative Study Report

Identify new and emerging approach Potential Barriers Summary of results

Topics Health Security Sport Science


4/63

Coursework 1 - Specification


5/63

Coursework 1 - FormatPaper Title* (use style: paper titl e )

Subtitle as needed (paper subtitle)

Authors Name/s per 1st Affiliation ( Author )line 1 (of Affiliation ): dept. name of organizationline 2-name of organization, acronyms acceptable

line 3-City, Countryline 4-e-mail address if desired

Authors Name/s per 2nd Affiliation ( Author )line 1 (of Affiliation ): dept. name of organizationline 2-name of organization, acronyms acceptable

line 3-City, Countryline 4-e-mail address if desired

Abstract This electronic document i s a live template andalready defines the components of your paper [title, text, heads,etc.] in its style sheet. *CRI TI CAL: Do Not Use Symbols, SpecialCharacters, or Math in Paper Titl e or Abstract . ( Abstract )

Keywords component; formatting; style; styling; insert (keywords)

I. I NTRODUCTION ( Heading 1 )This template, modified in MS Word 2007 and saved as a

Word 97 -2003 Document for the PC, provides authors withmost of the formatting specifications needed for preparingelectronic versions of their papers. All standard papercomponents have been specified for three reasons: (1) ease ofuse when formatting individual papers, (2) automaticcompliance to electronic requirements that facilitate theconcurrent or later production of electronic products, and (3)conformity of style throughout a conference proceedings.Margins, column widths, line spacing, and type styles are built-in; examples of the type styles are provided throughout thisdocument and are identified in italic type, within parentheses,following the example. Some components, such as multi-leveled equations, graphics, and tables are not prescribed,although the various table text styles are provided. Theformatter will need to create these components, incorporatingthe applicable criteria that follow.

II. EASE OF USE

A. Selecting a Template (Heading 2)First, confirm that you have the correct template for your

paper size. This template has been tailored for output on the A4 paper size. If you are using US letter-sized paper, please closethis file and download the file MSW_USltr_format .

B. Maintaining the Integrity of the SpecificationsThe template is used to format your paper and style the text.

All margins, column widths, line spaces, and text fonts are prescribed; please do not alter them. You may note peculiarities. For example, the head margin in this templatemeasures proportionately more than is customary. This

measurement and others are deliberate, using specificationsthat anticipate your paper as one part of the entire proceedings,

and not as an independent document. Please do not revise anyof the current designations.

III. PREPARE YOUR PAPER BEFORE STYLING Before you begin to format your paper, first write and save

the content as a separate text file. Keep your text and graphic

files separate until after the text has been formatted and styled.Do not use hard tabs, and limit use of hard returns to only onereturn at the end of a paragraph. Do not add any kind of

pagination anywhere in the paper. Do not number text heads-the template will do that for you.

Finally, complete content and organizational editing beforeformatting. Please take note of the following items when

proofreading spelling and grammar:

A. Abbreviations and AcronymsDefine abbreviations and acronyms the first time they are

used in the text, even after they have been defined in theabstract. Abbreviations such as IEEE, SI, MKS, CGS, sc, dc,and rms do not have to be defined. Do not use abbreviations inthe title or heads unless they are unavoidable.

B. Units Use either SI (MKS) or CGS as primary units. (SI units

are encouraged.) English units may be used assecondary units (in parentheses). An exception would

be the use of English units as identifiers in trade, suchas 3.5-inch disk drive.

Avoid combining SI and CGS units, such as current inamperes and magnetic field in oersteds. This often leadsto confusion because equations do not balancedimensionally. If you must use mixed units, clearlystate the units for each quantity that you use in anequation.

Do not mix complete spellings and abbreviations ounits: Wb/m2 or webers per square meter, notwebers/m2. Spell units when they appear in text: ...afew henries, not ...a few H.

Use a zero before decimal points: 0.25, not .25. Usecm3, not cc. (bullet list )

Identify applicable sponsor/s here. If no sponsors, delete this text box( sponsors).


6/63

Systematic Literature Review

Background

Search Strategy

Selection Criteria

Research Question

Discussion


7/63

Research Question

Systematic Reviews

Clinical TrialsExperimental Studies

Qualitative Studies

Observational Studies


8/63

Systematic Research Questions

1. Is strength training effective, ie, dostrengthening interventions increase strengthin people who are suffering the effects ofacute and chronic stroke?

2. Is therapeutic exercise of benefit inimproving activity and increasing societalparticipation for people who would beexpected to consult a physiotherapist?


9/63

Clinical Trials Questions

1. Does eight hours of stretch per day for threemonths reduce thumb web spacecontractures in neurological conditions?

2. Is the Mapleson C circuit more effective thanthe Laerdal circuit in removing secretions andimproving ventilation and gas exchangeduring manual hyperinflation?


10/63

Your Research Question

Is there any potential in the use ofaccelerometers to improve dental health?


11/63

Background

Define the problem Sign, symptoms etc.

Highlight importance of problem Cite research papers or government reports with

statistics of incidence levels. How were things normally managed in the

past. Where is the gap between the old technique

and the new one.


12/63

Search Strategy Specify your keyword search terms Select sources to be included

INSPEC IET IEEE etc.

Specify search dates 2003 onwards.

automated searches, identify resources to beused (digital libraries and search engines) manual searches, identify the journals and

conferences to be searched


13/63

Search Strategy

identify any ancillary search procedures, e.g.asking leading researchers or research groups,or accessing their web sites; or checkingreference lists of primary studies


14/63

Selection Criteria

You should have a bunch of papers. Identify inclusion criteria

Identify exclusion criteria Only use the titles and abstracts

Type of study

Number of participants Type of patients


15/63

Discussion

A handful of papers Read the full paper

Comparable outcomes Quantitative e.g. accuracy.

Potential

Barriers Results


16/63

Activity Recognition

Activity recognition is the process whereby an

actors behaviour and their situatedenvironment are monitored and analysed to

infer the undergoing activities

Activity Recognition in Pervasive Intelligent Environments, Luke Chen et al


17/63

Activity Recognition Tasks

Activity Modelling

Behaviour & Environment Monitoring

Data Processing

Pattern Recognition


18/63



Vision-based Sensor-based


19/63

Single Camera - Vision

Object

Camera


20/63


21/63

Stereo Vision - Active

Object

Laser

CameraA

CameraB


22/63

Camera Configurations

Object

Camera

Camera

Inward Looking


23/63


Camera

Outward Looking

Scene


24/63


Moving camera

Scene

Camera


25/63

Infrared Camera


26/63

Electromagnetic Spectrum


27/63

Microsoft Kinect


28/63

Microsoft Kinect


29/63

Microsoft Kinect

RGB camera 1280x960 resolution, 12 fps 640x480 resolution, 30 fps

IR emitter, IR depth sensor Image depth

Multi array microphone Direction /location of sound

3 Axis accelerometer Orientation of sensor


30/63

Kinect SDK

Supports human understanding Skeletal Facial recognition Gesture recognition Voice recognition


31/63

Healthcare Applications

Stroke Recovery

with Kinect

Activity Detection by PR2

Koppula et al, 2013

Learning Human Activities andObject Affordances from RGB-D

Videos

Microsoft Research


32/63

Vision-based

Learning Human Activities and Object Affordancesfrom RGB-D Videos

Hema Swetha Koppula, Rudhir Gupta, Ashutosh Saxena

Cornell University


33/63

MIT Vision Research

Eulerian Video Magnication for Revealing Subtle Changes in the World


34/63

Radial Artery Application

Eulerian video magnication used to amplify subtle motions of blood vessels

arising from blood ow


35/63

MIT Invisible Motion Technique

MIT Computer Program RevealsInvisible Motion in Video


36/63

Core Technology

Human Activity Recognition

ObjectSegmentation

FeatureExtraction &

Representation

ActivityDetection &Classification

Algorithms

Human Activity Recognition Systems

SingleMultiple Person

& Crowd

AbnormalActivity

Recognition

Human Activity Recognition Systems

Surveillance &Security

Healthcare Sports

Low

Medium

High


37/63

Computer Vision

ObjectDetection

Behaviourtracking

Activityrecognition

High levelactivityevaluation


38/63

Segmentation or Grouping Tokens

Whatever we are grouping. Pixels, points, surface elements etc.

Which pixels/edges/textures are useful and whichare not? Top down segmentation

Tokens belong together because they lie on the same

object. Bottom up segmentation

Tokens belong together because they are locallycoherent


39/63

Segmentation or Grouping

Why do these tokens belong together?

It is very difficult to tell whether a pixel (token) lieson a surface by simply looking at the pixel.


40/63

Gestalt Theory

Gestalt definition 1

A physical, biological, psychological, or symbolic

configuration or pattern of elements so unified as awhole that its properties cannot be derived from asimple summation of its parts .


41/63

Gestalt Theory


A perceptual pattern or structure possessingqualities as a whole that cannot be describedmerely as a sum of its parts .


42/63

Gestalt Theory


A form or configuration having properties thatcannot be derived by the summation of itscomponent parts .

Gestalt means shape in German.


43/63

Gestalt Theory Gestalt means when parts identified

individually have differentcharacteristics to the whole (Gestaltmeans "organised whole")

e.g. describing a tree - it's parts aretrunk, branches, leaves, perhapsblossoms or fruitBut when you look at an entire tree, you

are not conscious of the parts, you areaware of the overall object - the tree. Parts are of secondary importance even

though they can be clearly seen.


44/63

Which of these two pictures is easierto remember?


45/63

Principles of Gestalt Perception

Identify the figurefrom the background

This text on the slideis the figure and thegrey space is thebackground.

Rubens vase

Figure/ground


46/63

Principles of Gestalt Perception

Proximity

1 2 431+2 = one group3+4 = another group

3 groups of dots in a row

Proximity or contiguity states as things arecloser together will be seen as belongingtogether. Last example is one group.


47/63

Six Principles of Gestalt Perception

Similarity

Similarity means there is a tendency to see groupswhich have the same characteristics so in this example,

there are three groups of black squares and threegroups of white squares arranged in lines.

The principle of similarity states that things whichshare visual characteristics such as:

Shape Size Color Texture Value Orientation

will be seen as belonging together.

Anomaly


48/63


Common fateSuppose both principles of proximity and similarity arein place - then a movement takes place - the dots beginto move down the page.

They appear to change grouping.


49/63


Good continuation

Seeing things as whole lines (sequential) is clearlyimportant. But 'being in wholes means' that fewinterruptions change the reading of the wholelines.A to O and Oto D are two lines. Similarly,C to O and O to B are two lines.

The principle of continuity predicts the preference forcontinuous figures. We perceive the figure as twocrossed lines instead of 4 lines meeting at the center.

C

DA

B

O


50/63


Closure Related to principle of good continuation, there isa tendency to close simple figures, independent ofcontinuity or similarity.

This results in a effect of filling in missing information

or organising information which is present to make awhole.

In the circle at the top its seen easily. In the other tofigures it's a little more complex.

The second figure can be read as two overlappingrectangles (the gestalt) whereas it can also be seen asthree shapes touching; a square and two otherirregular shapes.

The final shape can be seen as a curve joining three

squares or as three uneven shapes touching.


51/63

Examples


52/63

Background subtraction

Naive Approach

Subtract


53/63


Provided the objectintensity/colour issufficiently different from

the background. Objects that enter the

scene and stop willcontinue to be detected.

New objects that pass infront of them will bedifficult to detect


54/63


If part of the assumedstatic backgroundstarts moving, both

the object and itsnegative ghost (therevealed background)

are detected


55/63


Background subtraction is sensitive to anychanges even unimportant ones:

Lighting e.g. sunlight Wind e.g. trees moving.

The camera cannot be moved Vibrations Unwanted behaviour


56/63

Frame Differencing

Subtract

B(t-1)

Object(t)


57/63

Frame Differencing

Frame differencing is very quick to adapt tochanges in lighting or camera motion.

Objects that stop are no longer detected. Objects

that start up do not leave behind ghosts. However, frame differencing only detects the

leading and trailing edge of a uniformly coloured

object. As a result very few pixels on the objectare labelled, and it is very hard to detect anobject moving towards or away from the camera


58/63

Frame Differencing

Object(t) Diff(t-1) Diff(t-3) Diff(t-5) Diff(t-9) Diff(t-15)

We now get a more complete silhouette

But we have two copies, a ghost. One where the object is now

One where the object used to be 15 frames ago.


59/63

3 Frame Differencing

ANDDiff(t-15)

Diff(t)

Diff(t+15)

Choice of good frame-rate for three-frame differencing depends on the sizeand speed of the object.


60/63

Background Subtraction

Adaptive Background Estimation - Parking Lot


61/63

Colour segmentation

Computer Vision - Colour Segmentation


62/63

Vision

Open discussion on ethics


63/63

Computer Vision

A Review on Video-Based Human ActivityRecognition

Date post:	03-Jun-2018
Category:	Documents
Upload:	shuja-khan
View:	224 times
Download:	0 times

Introduction to Acitivity Recognition

Documents