+ All Categories
Home > Documents > 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler...

2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler...

Date post: 17-Dec-2015
Category:
Upload: brett-shelton
View: 219 times
Download: 2 times
Share this document with a friend
Popular Tags:
62
2/14/00 Vision based Animation Vision based Animation The Inverse of an Inverse Problem The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang
Transcript
Page 1: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Vision based AnimationVision based AnimationThe Inverse of an Inverse ProblemThe Inverse of an Inverse Problem

Henning BiermannChris BreglerAaron HertzmannLorie LoebKathy PullenDanny Yang

Page 2: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

The Standard Vision PipelineThe Standard Vision Pipeline

Image / Video

Recognition

Modeling

Capture

Page 3: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Vision for Graphics / AnimationVision for Graphics / Animation

Image / Video

Image / Video

Synthesis

Modeling

Capture

Page 4: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Vision for Graphics / AnimationVision for Graphics / Animation

Image / Video

Expressive Animations

Modeling

Capture

Page 5: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

The Standard AnalogyThe Standard Analogy

Page 6: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

The Graphics ProblemThe Graphics Problem

Page 7: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

The Vision ProblemThe Vision Problem

Page 8: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Vision Based AnimationVision Based Animation

Page 9: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Vision for GraphicsVision for Graphics

Image / Video

Image / Video

Synthesis

Modeling

Capture

Page 10: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Vision for GraphicsVision for Graphics

Image / Video

Image / Video

Synthesis

RicherModeling

Capture

Page 11: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Vision for AnimationVision for Animation

Image / Video

Image / Video

KinematicsDynamics

Synthesis

Capture

RicherModeling

Page 12: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Models: 2 IssuesModels: 2 Issues

• Explicit / Implicit tradeoff:

Hand Coded Learned

Page 13: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Models: 2 IssuesModels: 2 Issues

• Explicit / Implicit tradeoff:

Hand Coded Learned

MX Y

Analytically derived Fit Black-Box

Page 14: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Models: 2 IssuesModels: 2 Issues

• Explicit / Implicit tradeoff:

• Model / Data tradeoff:

Pure Model Pure Data

Analytical Learned

Page 15: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Models: 2 IssuesModels: 2 Issues

• Explicit / Implicit tradeoff:

• Model / Data tradeoff:

Pure Model Pure Data

Analytical Learned

Page 16: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Case #1: Models of Kinematics Case #1: Models of Kinematics

- Approximate Kinematic Chains.

- Fine-tune Kinematic Chains.

- Acquisition of Nonrigid Models.

Page 17: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Approximate Kinematic Chains Approximate Kinematic Chains

- Enough for Animation ?

Page 18: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Case #1: Models of Kinematics Case #1: Models of Kinematics

- Approximate Kinematic Chains.

- Fine-tune Kinematic Chains.

- Acquisition of Nonrigid Models.

Page 19: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Fine-Tune KinematicsFine-Tune Kinematics

Anna Rabinowicz

Page 20: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Fine-Tune KinematicsFine-Tune Kinematics

Kathy Pullen

Page 21: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Case #1: Models of Kinematics Case #1: Models of Kinematics

- Approximate Kinematic Chains.

- Fine-tune Kinematic Chains.

- Acquisition of Nonrigid Models.

Page 22: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

-

Acquisition of Nonrigid 3D Kinematics Acquisition of Nonrigid 3D Kinematics

With Gene Alexander, Henning Biermann, Aaron Hertzmann, Lorenzo Torresani, Danny Yang

Page 23: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Solution for Non-rigid Tracking / ReconstructionSolution for Non-rigid Tracking / Reconstruction

- Prior Model:

2D: PCA-based (Cootes et al 95) 3D: Blanz+Vetter-99

Page 24: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

3D Model Acquisition3D Model Acquisition

- Multi-view input: Pighin et al 98

Page 25: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Solution for Rigid 3D AcquisitionSolution for Rigid 3D Acquisition

Structure from Motion:

- Tomasi-Kanade-92

Factorization

3D Pose3D rigid Object

Page 26: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Acquisition without prior model ?Acquisition without prior model ?

• No Model available ?

• Model too generic/specific ?

• Stock-Footage only in 2D ?

Page 27: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Solution based on Factorization Solution based on Factorization

- We want 3 things:- 3D non-rigid shape model- for each frame:

- 3D Pose- non-rigid configuration (deformation)

-> Tomasi-Kanade-92:

W = P S

Rank 3

Page 28: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Solution based on Factorization Solution based on Factorization

- We want 3 things:- 3D non-rigid shape model- for each frame:

- 3D Pose- non-rigid configuration (deformation)

-> PCA-based representations:

W = P non-rigid S

Rank K

Page 29: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Factoring twice Factoring twice

-

Page 30: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Nonrigid 3D Kinematics from point tracks Nonrigid 3D Kinematics from point tracks

-

Page 31: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

-

Nonrigid 3D Kinematics from dense flow Nonrigid 3D Kinematics from dense flow

Page 32: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

-

Nonrigid 3D Kinematics from dense flow Nonrigid 3D Kinematics from dense flow

Page 33: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

-

Nonrigid 3D Kinematics from dense flow Nonrigid 3D Kinematics from dense flow

Page 34: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Motion Capture

Modeling

Synthesis

Nonrigid 3D Kinematics from dense flow Nonrigid 3D Kinematics from dense flow

Page 35: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Case #2: Models of Dynamics Case #2: Models of Dynamics

Pure Model Pure Data

Page 36: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Case #2: Models of Dynamics Case #2: Models of Dynamics

Pure Model Pure Data

Learning Controller Mocap. Library

Page 37: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Case #2: Models of Dynamics Case #2: Models of Dynamics

Pure Model Pure Data

Parametric Non-Parametric

Page 38: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Case #2: Models of Dynamics Case #2: Models of Dynamics

Pure Model Pure Data

Kernel-Based Densities

Page 39: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Case #2: Models of Dynamics Case #2: Models of Dynamics

Kathy Pullen

Page 40: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Case #2: Models of Dynamics Case #2: Models of Dynamics

“Motion Texture”

Sampling /Matching

Page 41: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Case #2: Models of Dynamics Case #2: Models of Dynamics

P(Li | Lj)

Sampling / Matching:

Page 42: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Synthesizing Synthesizing Subtleties + VariationsSubtleties + Variations

Kathy Pullen

Motion Capture

Modeling

Synthesis

Page 43: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Synthesizing Synthesizing Subtleties + VariationsSubtleties + Variations

Kathy Pullen

Motion Capture

Modeling

Synthesis

Page 44: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Synthesizing Synthesizing Subtleties + VariationsSubtleties + Variations

Kathy Pullen

Motion Capture

Modeling

Synthesis

Page 45: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Synthesizing Synthesizing Subtleties + VariationsSubtleties + Variations

Kathy Pullen

Motion Capture

Modeling

Synthesis

Page 46: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Synthesizing Synthesizing Subtleties + VariationsSubtleties + Variations

Kathy Pullen

Motion Capture

Modeling

Synthesis

Page 47: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Vision Based AnimationVision Based Animation

Animations

KinematicsDynamics

RicherModeling

Page 48: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Human Movement ResearchHuman Movement Research

Computer Science:Vision / Graphics

Bio-Mechanics:Motion Models/Analysis

Art:Animation / Dance

Medicine/ Neurology:Movement Disorders

Page 49: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Model

Data

Page 50: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Model

Data

Expressive AnimationExpressive Animation

Page 51: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Human Movement ResearchHuman Movement Research

Computer Science:Vision / Graphics

Bio-Mechanics:Motion Models/Analysis

Art:Animation / Dance

Medicine/ Neurology:Movement Disorders

Page 52: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Project Mariposa: Animation PrinciplesProject Mariposa: Animation Principles

Lorie Loeb

Page 53: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Project Mariposa: Animation PrinciplesProject Mariposa: Animation Principles

Lorie Loeb

Realism is out

Page 54: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Project Mariposa: Animation PrinciplesProject Mariposa: Animation Principles

Lorie Loeb

Realism is out

Expressiveness is in

Page 55: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Model

Data

Expressive AnimationExpressive Animation

Page 56: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Model

Data

Controversy: Motion-Capture based AnimationControversy: Motion-Capture based Animation

Page 57: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Model

Data

Controversy: Motion-Capture based AnimationControversy: Motion-Capture based Animation

Page 58: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Model

Data

Controversy: Motion-Capture based AnimationControversy: Motion-Capture based Animation

Page 59: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Model

Data

Controversy: Motion-Capture based AnimationControversy: Motion-Capture based Animation

Page 60: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Synthesis by AnalysisSynthesis by Analysis

Animations

Page 61: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Enough for Animation ?Enough for Animation ?

Expressive Animations

Video/Mocap

Page 62: 2/14/00 Vision based Animation The Inverse of an Inverse Problem Henning Biermann Chris Bregler Aaron Hertzmann Lorie Loeb Kathy Pullen Danny Yang.

2/14/00

Expressive AnimationExpressive Animation

Expressive Output

CartoonCapture


Recommended