Computer Visioncs-courses.mines.edu/csci508/schedule/17/StatisticalModels.pdf · A. Szymczak, W....

transcript

Computer VisionColorado School of Mines

Colorado School of Mines

Computer Vision

Professor William HoffDept of Electrical Engineering &Computer Science

http://inside.mines.edu/~whoff/ 1

Statistical Models for Shape and Appearance

Note – some material for these slides came from• Computer Vision Algorithms and Applications, by Szeliski, Springer 2011• Computer vision: models, learning and inference, by Prince, Cambridge University,

Statistical Models for Shape and Appearance

• We often have knowledge about the possible ways that shape and appearance can vary

• When matching to a new image, we can use this to constrain the shape and possibly give us better results

Principal Components

• One way to model the variation in your data, is to use principal components

• Principal components:– Assume we have a population of measurements vectors – The covariance matrix is – Find the eigenvectors and eigenvalues of – The are called “principal components”

• We can express any of the original vectors as a linear combination of PCs:

where , , … , , , , … ,

Generative Model

• This is a “generative” model (as opposed to a “discriminative” model)

• Given coefficients h (which represent the parameters of the model in the world), we can generate examples of measurements x:

where is normally distributed noise with covariance

Approaches

• We’ll look at 3 approaches that use PC‐based statistical models:1. Appearance‐based (use images)

• Also called “eigenimages”

2. Shape‐based (use contour points or surface points)• Also called “active shape models” or “point distribution models”

3. Shape and appearance‐based (use both)• Also called “active appearance models”

Appearance‐Based (eigenimages)

• We have a collection of images, I1..IK– We subtract off the mean of the collection of images

• We transform each MxN image Ii into a column vectorxi = [Ii(1,1), Ii(1,2), … Ii(1,N), Ii(2,1), … Ii(M,N)]T

• Find the covariance matrix Cx

• The eigenvectors (principal components) represent “basis” images (or “eigenimages”)

x1 x2 x3 xK

Application ‐ Face images

from: http://www.cs.princeton.edu/~cdecoro/eigenfaces/

Input images (Yale database)

Eigenfaces

• 10 PC’s capture 36% of the variance

• 25 PC’s capture 56% of the variance

Reconstruction with a small number of Principal Components

Each new image from left to right corresponds to using 1 additional PC for reconstruction

Each new image from left to right corresponds to using 8 additional PC for reconstruction

Recognition

• To recognize a new image x, project it onto the linear subspace spanned by the eigenimages (i.e., find the coefficients ):

DFFS: test if x is a face

DIFS: measure how close x is to a training image face

Shape‐based (Active Shape Models)

• Represent a shape with a set of points– For example, evenly spaced points along a contour

• Let x be the vector consisting of the set of points• We compute the covariance of a set of training vectors, and

get the eigenvectors (principal components)

Examples of contours

• Resistors

• Hand

Recognition

• We assume that the coefficients of the PCs (the shape parameters) are normally distributed

• To recognize a new shape x, we project it onto the linear subspace formed by the PCs, and compute the Mahalanobisdistances to each category center

Alignment of shapes

• The shapes must be registered and points sampled in the same places

Registration to image

• To register a point distribution model with an image, each control point searches in a direction normal to the contour to find the most likely corresponding image edge point

• Then search over (a) the shape parameters and (b) the parameters of a transform to align the shapes

Examples

Computer VisionColorado School of Mines19

3D Shape Models

• Example: Creating bone models from 2D silhouettes using a statistical atlas

Josh King (MS 2007), Colorado School of Mines. Thesis title: “Bi‐Planar Image Registration and Modeling of Bones”

A. Szymczak, W. Hoff and M. Mahfouz, “3D Shape from Silhouette Points in Registered 2D Images Using Conjugate Gradient Method,” Medical Imaging 2010: Image Processing. Proceedings of the SPIE, Volume 7623, pp. 762316‐762316‐8, 2010.

Shape and Appearance‐Based

• You get better results if you model both shape andappearance– This is called an “Active Appearance Model (AAM)”

• We use principal components for shape (encoded by a set of control points) and appearance (ie, the image texture)– The texture is normalized to a canonical shape before being analyzed

Active Appearance Model

• Both shape and texture are represented as deviations from a mean shape and mean texture

• Note – the same appearance parameters h simultaneously control both the shape and texture deformations – these are correlated!

Example – face modeling

Computer VisionColorado School of Mines24

Warping images

To align the image texture according to shape displacements, use a piecewise affine transformation

Triangulate image points using Delaunay triangulation.

Image in each triangle is warped by an affine transformation.

Registering a new image

To register the face model to a new image, search over (a) the appearance parameters and (b) the warping parameters of a transform to align the shapes

Example

• AAMs have been used directly for recognition, but their main use is to align faces into a canonical pose so that other methods of face recognition can be used

Computer Visioncs-courses.mines.edu/csci508/schedule/17/StatisticalModels.pdf · A. Szymczak, W....

Documents