Summary & Homework

Jinxiang Chai

Outline

Motion data process paper summary

Presentation tips

Homework

Paper assignment

Skeleton Model Extraction

Estimate the skeleton model from motion capture data

Motion Data Processing I

Operations on single motion sequence

• Motion warping (cannot satisfy user constraints)

Motion Data Processing I

Operations on single motion sequence

• Motion warping (cannot satisfy user constraints)

• Motion retargeting (can satisfy user constraints but slow)

Motion Data Processing I

Operations on single motion sequence

• Motion warping (cannot satisfy user constraints)

• Motion retargeting (can satisfy user constraints but slow)

• Motion edit (can satisfy user constraints and fast)

Motion Data Processing I

Operations on single motion sequence

• Motion warping (cannot satisfy user constraints)

• Motion retargeting (can satisfy user constraints but slow)

• Motion edit (can satisfy user constraints and fast)

However: does not work for many user-defined constraints

does not utilize spatial-temporal correlation in human motion

Motion Data Processing I

Operations on single motion sequence

• Motion warping (cannot satisfy user constraints)

• Motion retargeting (can satisfy user constraints but slow)

• Motion edit (can satisfy user constraints and fast)

• Expression cloning (facial data) (cannot satisfy user constraints)

Motion Data Processing I

Operations on single motion sequence

• Motion warping (cannot satisfy user constraints)

• Motion retargeting (can satisfy user constraints but slow)

• Motion edit (can satisfy user constraints and fast)

• Expression cloning (facial data) (cannot satisfy user constraints)

• Motion synopsis

Motion Data Processing I

Operations on single motion sequence

• Motion warping (cannot satisfy user constraints)

• Motion retargeting (can satisfy user constraints but slow)

• Motion edit (can satisfy user constraints and fast)

• Expression cloning (facial data) (cannot satisfy user constraints)

• Motion synopsis

• What else?

Motion Data Processing I

Operations on single motion sequence

• Motion segmentation?

• Motion recognition?

• Motion filtering?

• More operations in facial data

Motion Data Processing II

Operations between two motion sequence

• Motion style translation

• What else?

Motion Data Processing II

Operations between two motion sequence

• Motion style translation

• What else?

Motion Data Processing II

Operations between two motion sequence

• Motion style translation

• Motion interpolation?

Motion Data Processing III

Operations on multiple motion sequences/database

• Motion data compression

Motion Data Processing III

Operations on multiple motion sequences/database

• Motion data compression

• What else?

Motion Data Processing III

Operations on multiple motion sequences/database

• Motion data compression

• Motion Retrieval?

• Motion synthesis?

Presentation Tips

Choose right background• Use blue or black or dark colors

Focus on important things• Always talk about what, why, and how

• Detailize the most important technical content

• Always talk about the limitations of the paper

How to deal with equations• Avoid using too many equations

• Talk about the intuition of the equations

• Explain each term of the equations

Presentation Tips (Cont.)

Choose the right background• Use dark colors (blue or black)

Be consistent• Font type, size, color, bullets, figure captions etc.

Use short & concise sentences• Do not copy & paste from the paper

• Use images, diagrams, figures, videos to demonstrate ideas

Presentation Tips (Cont.)

Play videos• Explain each video

• Move the mouse away from the window

Example Slides

Goal: everyone can generate and control human motion easily and quickly

Online animation control

Interfaces for Controlling Human Motion

Applications: Online Animation

Performance-based facial animation for home use

Tiger Woods PGA Tour 2005

Mike Tyson Heavyweight Boxing

Friday Night 3D Bowling

Teleconferencing (from BT)

Xbox Outlaw Tennis

Multi-user Virtual Worlds

Virtual Poker Room

Virtual Presenter

Reordering motion clip [Lee et al. SIG02, Kovar et al. SIG02, Pullen & Bregler SIG02, Arikan et al. SIG03]

Learning model from human motion [Brand & Hertzmann SIG00, Li et al. SIG02]

Interpolating motions [Rose et al. CG&A98, Kovar et al. SIG04]

Animation from Mocap Data

Video Analysis

Overview

Online motion synthesis

Online local modeling

Low-dimensional control signals

Preprocessedmotion capture

dataPreprocessed

motion capturedata

Online local models

Online Motion Optimization

2

21

21 ~~2~)(minarg tttttttttt

Tt

Ttw qqpwUcwfwUUw

t

Optimizing a nonlinear function in about 7 dimensional space wt :

Control term

Smoothness term

Pose prior term

1~q tq

~

tc~…

…

1~c

?

Outline

Motion data paper summary

Presentation tips

Homework

Paper assignment

Dimensionality reduction using PCA

tttt wUpq ~Current pose Mean pose

Eigen-poses Low-dimensional space

Linear model:

Mean pose

Eigen-poses

Dim(wt)?

Linear Dimensionality Reduction

Linear Dimensionality Reduction

0 5 10 15 20 25 30 35 40 45 500

5

10

15

20

25

30

Ave

rage

rec

on

stru

ctio

n e

rro

r(d

egre

e p

er jo

int

ang

le)

A large heterogeneous database (1 hour of data)

PCA (38D for error<1o)

Number of dimensions

Reconstruction Error Curve

0 10 20 30 40 50 600

0.5

1

1.5

2

2.5

Ave

rage

rec

on

stru

ctio

n e

rro

r(d

egre

e p

er jo

int

ang

le)

A large database

Number of dimensions of Wt

tttt wUpq ~

Minimal Dimensionality

0 10 20 30 40 50 600

0.5

1

1.5

2

2.5

Ave

rage

rec

on

stru

ctio

n e

rro

r(d

egre

e p

er jo

int

ang

le)

A large database

Number of dimensions of Wt

tttt wUpq ~

7

0.8

tttt wUpq ~

56 dim 7 dim

Videos

Two Animation videos rendered by Maya• Original mocap sequence

• Reconstructed mocap sequence

Startup Codes & Data

Motion capture data (.amc files and .asf file)

A skinned character model

Visualize amc/asf file• Read asf/amc file

• Visualize mocap data

• Forward kinematics

Matlab codes• Read and write .amc file

Softwares

Maya• To be installed in Rm 220, HRBB

• Render each frame based on mocap data and character model

• Instruction on how to render animation

Adobe Premier • Installed in Rm 220, HRBB

• Making video from image sequences

Outline

Motion data paper summary

Presentation tips

Homework

Paper assignment