©2003-2010, The Ohio State University

Motion Capture History, Pipeline, and Areas of Interest

Advanced Computing Center for the Arts and Design

Ohio State University

©2010, The Ohio State University

Motion Capture

• Motion capture (mocap) is sampling and recording motion of humans, animals and inanimate objects as 3d data for analysis, playback and remapping.

©2010, The Ohio State University

History of Motion Capture

• Eadweard Muybridge (1830-1904)• Etienne-Jules Marey (1830-1904)• Harold Edgerton (1903-1990)

©2010, The Ohio State University

Eadward Muybridge

• The Flying Horse • Zoopraxiscope• Animal and Human

Locomotion

© Kingston Museum

©2010, The Ohio State University

Etienne-Jules Marey

• First person to analyze human and animal motion with video

• Created chronophotographic gun and fixed plate camera

©2010, The Ohio State University

Harold Edgerton

© Palm Press Inc.

©2010, The Ohio State University

Rotoscope

• Allowed animators to trace cartoon characters over photographed frames of live performances.

• Invented in 1915 by Max Fleischer• Koko the Clown• Snow White

©2010, The Ohio State University

1980’s

• Military and medical research purposes• First computer graphics use in research labs• First production use

o Brilliance by Robert Abel (1985 Superbowl ad)o http://citinite.wordpress.com/2007/12/28/robert-abel-ass

ociates/o Waldo C. Graphic (1988) PDI for Jim Henson touro Mike the Talking Head (Siggraph 88) o http://mambo.ucsc.edu/psl/mike.htmlo Don’t Touch Me (1989)o http://www.poetv.com/video.php?vid=15102

©2010, The Ohio State University

ElectromechanicalOptical fiber

Strobing LEDAcoustic

OpticalAcoustic

Motion Capture Systems

©2010, The Ohio State University

Electromechanical suits

• Linked structures• Potentiometers determine rotation

of each link • No occlusion• No magnetic or electrical

interference• Large capture range, portable

• No global translation• Restricted movement• Fixed configuration of sensors• Low sampling rate• Inaccurate joints

http://www.metamotion.com/gypsy/gypsy-motion-capture-system-mocap.htm

©2010, The Ohio State University

Optical fiber system

• Fiber-optic sensor • Bend and twist sensors measure

transmitted light• No occlusion• Multiple subjects• Portable

• Adjustment to individual proportions is limited

• Less accurate data http://www.finger-motion-capture.com/

©2010, The Ohio State University

Electromagnetic systems

• Electromagnetic sensors placed on joints of moving object

• Measures orientation and position of sensor relative to electromagnetic field generated by transmitter

• No occlusion, multiple subjects• Electromagnetic interference,

small volume http://www.ascension-tech.com/

©2010, The Ohio State University

Acoustic system

• Set of microphones receive sound wave• Other sounds in frequency range can

disrupt capture• Accuracy not as high as other systems

©2010, The Ohio State University

Strobing LED marker system

• Fewer Occlusion Problems

• Limited volume

http://www.inition.co.uk/inition/product.php?URL_=product_mocaptrack_ascension_reactor2&SubCatID_=59

©2010, The Ohio State University

Optical motion capture system

• Reflective markers• Multiple cameras digitize

different views of performance

• High speed capture• High data accuracy

• Occlusion• Position sensitive

cameras• Limited volume http://www.vicon.com

Video-based Motion Analysis

• Research problem for 20+ years

• Difficultieso Complex varying environmento Segmentation issueso Occlusion

©2010, The Ohio State University

©2010, The Ohio State University

Video-based Motion Analysis

Research Areaso Trackingo 3D surface reconstruction (surface scanning)o 3D movement reconstruction (video-based motion

capture)o Recognition

Challenges:o Complex environment variabilityo Body segmentationo Occlusion

©2010, The Ohio State University

3D Reconstruction

• Single camerao Model assumptions required

• Multiple cameraso Require at least 2 cameras, unique with 3o Camera calibration

• Motion capture with markerso Use retroreflective markers to simplify video

information

©2010, The Ohio State University

Optical Mocap Process

• Calibrationo Specifying how each camera is related to the others in

reference to origin

• Trackingo Tracking of points…acceleration and identification

problems

• Datao 3D positional…needs to be converted to rotational

data

©2010, The Ohio State University

Problems Related to Marker Occlusion

©2010, The Ohio State University

Skeletal Fit (remapping mocap data to a character model)

• How to make markers move a skeletono Photo reference of performero CG modelo Motion Buildero Vicon Bladeo Other methods

• Problems with detecting joint centers… • Organization of joint hierarchies

©2009, The Ohio State University

Motion Capture Pipeline

• Planning• Shooting (Vicon IQ)• Data processing (ViconIQ, MotionBuilder)• Skeleton creation (Maya) • Mapping to characters (MotionBuilder)• Binding skeleton to a model (Maya,

MotionBuilder)

©2010, The Ohio State University

Planning

• Character/Prop setupo Target skeleton/character topologyo Ready stance considerations

• Marker setupo Heavy marker redundancy o Three markers per segmento Place markers close to bone

©2009, The Ohio State University

Planning

• Shot listo Number of characters involved in performance o Performance space dimensionso Interactions in shoto Shots to be blended or looped o Length of shotso Size and location of propso Gross proportional differences for retargeting

©2009, The Ohio State University

Planning

• Technical Issueso Output formato File naming conventionso Frames per secondo Target software platformo Database management

©2010, The Ohio State University

Other Areas of Interest

• Re-use of data• Generating 3d

geometry • Scene tracking with

motion capture

http://accad.osu.edu/~vberezin/bm/lp_anim.html

©2010, The Ohio State University

New Optical Technology

• 16 megapixel cameras ( T-series Vicon and Motion Analysis), 2000 fps

• 10 bit greyscale – Vicon (better centroid calculation)

©2010, The Ohio State University

Applications

• Medicine Mixed Reality Rehabilitation http://ame2.asu.edu/projects/mrrehab/

NIH Clinical Movement Analysis Lab http://pdb.cc.nih.gov/

• Military http://www.motionrealityinc.com/

• Sports http://www.highwaygames.com/products/view.php?id=2807

• Video Gameshttp://www.spike.com/video/nba-live-08-mo-cap/2897719

http://vimeo.com/1096600

©2010, The Ohio State University

Other uses

Cognitive Psychologyhttp://www.mcgill.ca/spl/projects/motion/

Historical Preservationhttp://accad.osu.edu/research/mocap_htmls/NAD.htm

Arthttp://www.openendedgroup.com/index.php/artworks/

©2009, The Ohio State University

Feature Films

• Avatar • Curious Case of Benjamin Button• Pirates of the Carribean• The Incredible Hulk• Lord of the Rings• Polar Express• Matrix• Titanic

©2009, The Ohio State University

Video Games

• Kinect

• Move

• Wii

©2010, The Ohio State University

Future

• Univesity of South California (Paul Debevec)

• NYU (Chris Briegler)• Stanford University (Vlad Koltun)• Carnegie Mellon (Jessica Hodgins)• Max Planck Center

• Organic Motion• Mova• Image Metrics• Captive Motion

©2010, The Ohio State University

References

1.Menache, Alberto, “Understanding Motion Capture for Computer Animation and Video Games”

2.http://www.kingston.ac.uk/Muybridge/3.http://www.anotherscene.com/cinema/firsts/marey.html4.http://cmp1.ucr.edu/exhibitions/edgerton/edgerton.html5.Siggraph 2001 Course 51: