Active Appearance Models
Dhruv BatraECE CMU
Active Appearance Models
1. T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc. European Conference on Computer Vision 1998 (H.Burkhardt & B. Neumann Ed.s). Vol. 2, pp. 484-498, Springer, 1998
2. T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", IEEE PAMI, Vol.23, No.6, pp.681-685, 2001
3. G.J. Edwards, A. Lanitis, C.J. Taylor, T. F. Cootes. “Statistical Models of Face Images Improving Specificity”, BMVC (1996)
Essence of the Idea “Interpretation through synthesis”
Form a model of the object/image (Learnt from the training dataset)
I. Matthews and S. Baker, "Active Appearance Models Revisited," International Journal of Computer Vision, Vol. 60, No. 2, November, 2004, pp. 135 - 164.
Essence of the Idea (cont.) Explain a new example in terms of the model parameters
So what’s a model
Model
“Shape” “texture”
Active Shape Modelstraining set
Texture Models
warp to mean shape
Random Aside Shape Vector provides alignment
=
43Alexei (Alyosha) Efros, 15-463 (15-862): Computational Photography, http://graphics.cs.cmu.edu/courses/15-463/2005_fall/www/Lectures/faces.ppt
Random Aside Alignment is the key
1. Warp to mean shape
2. Average pixels
Alexei (Alyosha) Efros, 15-463 (15-862): Computational Photography, http://graphics.cs.cmu.edu/courses/15-463/2005_fall/www/Lectures/faces.ppt
Random Aside Enhancing Gender
more same original androgynous more opposite
D. Rowland, D. Perrett. “Manipulating Facial Appearance through Shape and Color”, IEEE Computer Graphics and Applications, Vol. 15, No. 5: September 1995, pp. 70-76
Random Aside (can’t escape structure!)
Alexei (Alyosha) Efros, 15-463 (15-862): Computational Photography, http://graphics.cs.cmu.edu/courses/15-463/2005_fall/www/Lectures/faces.ppt
Antonio Torralba & Aude Oliva (2002)
Averages: Hundreds of images containing a person are
averaged to reveal regularities in the intensity patterns across
all the images.
Random Aside (can’t escape structure!)
Tomasz Malisiewicz, http://www.cs.cmu.edu/~tmalisie/pascal/trainval_mean_large.png
Random Aside (can’t escape structure!)“100 Special Moments” by Jason Salavon
Jason Salavon, http://salavon.com/PlayboyDecades/PlayboyDecades.shtml
Random Aside (can’t escape structure!)“Every Playboy Centerfold, The Decades (normalized)” by Jason Salavon
1960s 1970s 1980sJason Salavon, http://salavon.com/PlayboyDecades/PlayboyDecades.shtml
Back (sadly) to Texture Models
raster scan
Normalizations
PCA Galore
Reduce Dimensions of shape vector
Reduce Dimension of “texture” vector
They are still correlated; repeat..
Object/Image to Parameters
modeling
~80
Playing with the Parameters
First two modes of shape variation First two modes of gray-level variation
First four modes of appearance variation
Active Appearance Model Search Given: Full training model set, new image to be interpreted,
“reasonable” starting approximation
Goal: Find model with least approximation error
High Dimensional Search: Curse of the dimensions strikes again
Active Appearance Model Search Trick: Each optimization is a similar problem, can be learnt
Assumption: Linearity
Perturb model parameters with known amount
Generate perturbed image and sample error
Learn multivariate regression for many such perterbuations
Active Appearance Model Search Algorithm: current estimate of model parameters: normalized image sample at current estimate
Active Appearance Model Search Slightly different modeling:
Error term:
Taylor expansion (with linear assumption)
Min (RMS sense) error:
Systematically perturb and estimate by numerical differentiation
Active Appearance Model Search (Results)