Single View Metrology A. Criminisi, I. Reid, and A. Zisserman University of Oxford IJCV Nov 2000...

Single View MetrologySingle View Metrology A. Criminisi, I. Reid, and A. Zisserman A. Criminisi, I. Reid, and A. Zisserman

University of OxfordUniversity of OxfordIJCV Nov 2000IJCV Nov 2000

Presentation by Kenton AndersonPresentation by Kenton Anderson

CMPT820CMPT820

March 24, 2005March 24, 2005

March 24, 2005March 24, 2005 Single View MetrologySingle View Metrology 22

OverviewOverview

IntroductionIntroduction GeometryGeometry Algebraic RepresentationAlgebraic Representation Uncertainty AnalysisUncertainty Analysis ApplicationsApplications ConclusionsConclusions


ProblemProblem

Is it possible to extract 3D geometric Is it possible to extract 3D geometric information from single images?information from single images?

YESYES How?How? Why?Why?


BackgroundBackground

2D

3D

Opticalcentre

Painter,Linear perspective

Real or imaginaryobject

Painting

Camera,Laws of Optics

Real object

Photograph

Architect,Descriptive Geometry

A mental model

Drawing

Projective GeometryReconstructed

3D model

Flat image


IntroductionIntroduction

3D affine measurements may be 3D affine measurements may be measured from a single perspective measured from a single perspective imageimage


IntroductionIntroduction

1.1. Measurements of the distance Measurements of the distance between any of the planesbetween any of the planes

2.2. Measurements on these planesMeasurements on these planes

3.3. Determine the camera’s positionDetermine the camera’s position

Results are sufficient for a partial or complete 3D Results are sufficient for a partial or complete 3D reconstruction of the observed scenereconstruction of the observed scene


SampleSampleLa Flagellazione di Cristo


GeometryGeometry OverviewOverview Measurements between parallel linesMeasurements between parallel lines Measurements on parallel planesMeasurements on parallel planes Determining the camera positionDetermining the camera position


Geometry OverviewGeometry Overview

Possible to obtain geometric Possible to obtain geometric interpretations for key features in a interpretations for key features in a scenescene

Derive how 3D affine measurements Derive how 3D affine measurements may be extracted from the imagemay be extracted from the image

Use results to analyze and/or model Use results to analyze and/or model the scenethe scene


AssumptionsAssumptions

Assume that images are obtained by Assume that images are obtained by perspective projectionperspective projection

Assume that, from the image, a:Assume that, from the image, a:• vanishing linevanishing line of a of a reference planereference plane• vanishing pointvanishing point of another of another reference reference

directiondirection

may be determined from the imagemay be determined from the image


Geometric CuesGeometric Cues

Vanishing Line ℓVanishing Line ℓ• Projection of the line at infinity of the reference Projection of the line at infinity of the reference

plane into the imageplane into the image



VanishingVanishing Point(s) v Point(s) v• A point at infinity in the reference directionA point at infinity in the reference direction• Reference direction is NOT parallel to reference Reference direction is NOT parallel to reference

planeplane• Also known as the vertical vanishing pointAlso known as the vertical vanishing point


Vanishing point

Vanishing

line

Vanishing point

Vertical vanishing point

(at infinity)



Generic AlgorithmGeneric Algorithm

1)1) Edge detection and straight line fitting to obtain Edge detection and straight line fitting to obtain the set of straight edge segments Sthe set of straight edge segments SAA

2)2) RepeatRepeat

a)a) Randomly select two segments sRandomly select two segments s11, s, s22 €€ S SAA and and intersect them to give point intersect them to give point pp

b)b) The support set SThe support set Spp is the set of straight edges in S is the set of straight edges in SAA going through point going through point pp

3)3) Set the dominant vanishing point as the point Set the dominant vanishing point as the point pp with the largest support Swith the largest support Spp

4)4) Remove all edges in SRemove all edges in Spp from S from SAA and repeat step and repeat step 22


Automatic estimation of vanishing Automatic estimation of vanishing points and linespoints and lines

RANSAC algorithm

Candidate vanishing point


Automatic estimation of vanishing Automatic estimation of vanishing points and linespoints and lines


Measurements between Measurements between Parallel LinesParallel Lines

Wish to measure the distance Wish to measure the distance between two parallel planes, in the between two parallel planes, in the reference directionreference direction• The aim is to compute the height of an The aim is to compute the height of an

object relative to a referenceobject relative to a reference


Cross RatioCross Ratio

Point Point bb on plane ∏ on plane ∏ correspond to point correspond to point tt on plane ∏’on plane ∏’

Aligned to vanishing Aligned to vanishing point point vv

Point Point ii is the is the intersection with the intersection with the vanishing linevanishing line


Cross RatioCross Ratio

The The cross ratiocross ratio is between the is between the points provides an affine length ratiopoints provides an affine length ratio• The value of the cross ratio determines a ratio The value of the cross ratio determines a ratio

of distances between planes in the worldof distances between planes in the world

Thus, if we know the length for an Thus, if we know the length for an object in the scene, we can use it as object in the scene, we can use it as a reference to calculate the length of a reference to calculate the length of other objectsother objects


Estimating HeightEstimating Height

•The distance || tr – br || is known•Used to estimate the height of the man in the scene


Measurements on Parallel PlanesMeasurements on Parallel Planes

If the reference plane is affine calibrated, If the reference plane is affine calibrated, then from the image measurements the then from the image measurements the following can be computed:following can be computed:

i.i. Ratios of lengths of parallel line segments on Ratios of lengths of parallel line segments on the planethe plane

ii.ii. Ratios of areas on the planeRatios of areas on the plane


Parallel Line SegmentsParallel Line Segments

•Basis points are manually selected and measured in the real world•Using ratios of lengths, the size of the windows are calculated


Planar HomologyPlanar Homology

Using the same principals, affine Using the same principals, affine measurements can be made on two measurements can be made on two separate planes, so long as the separate planes, so long as the planes are parallel to each otherplanes are parallel to each other

A map in the world between parallel A map in the world between parallel planes induces a map between planes induces a map between images of points on the two planesimages of points on the two planes


Homology Mapping between Homology Mapping between Parallel PlanesParallel Planes

A point A point XX on plane on plane ∏∏ is mapped into the point is mapped into the point XX’ ’ on on ∏∏’ by a parallel projection’ by a parallel projection


Planar HomologyPlanar Homology

Points in one plane are mapped into Points in one plane are mapped into the corresponding points in the other the corresponding points in the other plane as follows:plane as follows:

X’ = HXX’ = HXwhere (in homogeneous coordinates):where (in homogeneous coordinates):• XX is an image point is an image point• X’X’ is its corresponding point is its corresponding point• HH is the 3 x 3 matrix representing the is the 3 x 3 matrix representing the

homography transformationhomography transformation



This means that we can This means that we can comparecompare measurements made on two measurements made on two separate planes by mapping between separate planes by mapping between the planes in the reference direction the planes in the reference direction via the homologyvia the homology


Parallel Line Segments lying on two Parallel Line Segments lying on two Parallel PlanesParallel Planes


Camera PositionCamera Position

Using the techniques we developed Using the techniques we developed in the previous sections, we can:in the previous sections, we can:• Determine the distance of the camera Determine the distance of the camera

from the scenefrom the scene• Determine the height of the camera Determine the height of the camera

relative to the reference planerelative to the reference plane


Camera Distance from SceneCamera Distance from Scene

In In Measurements between Measurements between Parallel LinesParallel Lines, distances between , distances between planes are computed as a ratio planes are computed as a ratio relative to the camera’s distance relative to the camera’s distance from the reference planefrom the reference plane

Thus we can compute the camera’s Thus we can compute the camera’s distance from a particular frame distance from a particular frame knowing a single reference distanceknowing a single reference distance


Camera Position Relative to Camera Position Relative to Reference PlaneReference Plane

The location of the camera relative to the The location of the camera relative to the reference plane is the back-projection of the reference plane is the back-projection of the vanishing point onto the reference planevanishing point onto the reference plane


Algebraic RepresentationAlgebraic Representation OverviewOverview Measurements between parallel linesMeasurements between parallel lines Measurements on parallel planesMeasurements on parallel planes Determining the camera positionDetermining the camera position


OverviewOverview

Algebraic approach offers many Algebraic approach offers many advantages (over direct geometry):advantages (over direct geometry):1.1. Avoid potential problems with ordering for the Avoid potential problems with ordering for the

cross ratiocross ratio

2.2. Minimal and over-constrained configurations Minimal and over-constrained configurations can be dealt with uniformlycan be dealt with uniformly

3.3. Unifies the different types of measurementsUnifies the different types of measurements

4.4. Are able to develop an Are able to develop an uncertainty analysisuncertainty analysis


Coordinate SystemsCoordinate Systems

Define an affine coordinate system Define an affine coordinate system XYZ in spaceXYZ in space• Origin lies on Origin lies on reference planereference plane• X, Y axes span the X, Y axes span the reference planereference plane• Z axis is the Z axis is the reference directionreference direction

Define image coordinate system xyDefine image coordinate system xy• y in the vertical directiony in the vertical direction• x in the horizontal directionx in the horizontal direction


Coordinate SystemsCoordinate Systems


Projection MatrixProjection Matrix

If If X’X’ is a point in world space, it is is a point in world space, it is projected to an image point projected to an image point x’x’ in in image space via a 3 x 4 projection image space via a 3 x 4 projection matrix Pmatrix P

x’ = PX’ = [ px’ = PX’ = [ p11 p p22 p p33 p p44 ]X’ ]X’

where x’ and X’ are homogeneous vectors:where x’ and X’ are homogeneous vectors:

x’ = (x’ = (x, y ,wx, y ,w) and X’ = () and X’ = (X, Y, Z, WX, Y, Z, W))


Vanishing PointsVanishing Points

Denote the vanishing points for the X, Y Denote the vanishing points for the X, Y and Z directions as vand Z directions as vXX, v, vYY, and v, and v

By inspection, the first 3 columns of matrix By inspection, the first 3 columns of matrix P are the vanishing points:P are the vanishing points:• pp11 = v = vXX

• pp22 = v = vY Y

• pp33 = v = v

Origin of the world coordinate system is pOrigin of the world coordinate system is p44


Vanishing LineVanishing Line

Furthermore, vFurthermore, vXX and v and vYY are on the are on the vanishing line vanishing line ll• Choosing these points fixes the X and Y Choosing these points fixes the X and Y

affine coordinate axesaffine coordinate axes

• Denote them as Denote them as ll11 , , ll22 where where llii · · ll = 0 = 0

Note:Note:• Columns 1, 2 and 4 make up the Columns 1, 2 and 4 make up the

reference plane to image homography reference plane to image homography matrix Hmatrix H

TT T


Projection Matrix ReduxProjection Matrix Redux

o = p4 = o = p4 = ll//|||| ll || = || = ll^̂

• oo is the Origin of the coordinate system is the Origin of the coordinate system

Thus, the parametrization of P is:Thus, the parametrization of P is:

P = [ P = [ ll11 ll22 ααv v ll^̂ ] ]

T T

αα is the affine scale factoris the affine scale factor



The aim is to compute the height of The aim is to compute the height of an object relative to a referencean object relative to a reference

Height is measured in the Z directionHeight is measured in the Z direction



Base point B on the reference planeBase point B on the reference plane Top point T in the sceneTop point T in the scene

b = n(Xpb = n(Xp11 + Yp + Yp22 + p + p44))

t = m(Xpt = m(Xp11 + Yp + Yp22 + Zp + Zp33 + p + p44))

nn and and mm are unknown scale factors are unknown scale factors


Affine Scale FactorAffine Scale Factor

If If αα is known, then we can obtain Z is known, then we can obtain Z If Z is known, we can compute If Z is known, we can compute αα, removing , removing

affine ambiguityaffine ambiguity


RepresentationRepresentation



Projection matrix P from the world to Projection matrix P from the world to the image is defined with respect to the image is defined with respect to a coordinate frame on the reference a coordinate frame on the reference planeplane

The translation from the reference The translation from the reference plane to another plane along the plane to another plane along the reference direction can be reference direction can be parametrized into a new projection parametrized into a new projection matrix P’matrix P’


Plane to Image HomographiesPlane to Image Homographies

P = [ P = [ ll11 ll22 ααv v ll^̂ ] ]

P’ = [ P’ = [ ll11 ll22 ααv v ααZZv + v + ll^̂ ] ]

where Z is the distance between the planeswhere Z is the distance between the planesT T

TT


Plane to Image HomographiesPlane to Image Homographies

Homographies can be extracted:Homographies can be extracted:

H = [ pH = [ p11 p p22 ααZZv + v + ll^̂ ]]

H’ = [ pH’ = [ p11 p p22 ll^̂ ]]

Then Then H” = H’HH” = H’H-1-1 maps points from maps points from the reference plane to the second the reference plane to the second plane, and so defines the homologyplane, and so defines the homology


Generic AlgorithmGeneric Algorithm

1.1. Given an image of a planar surface Given an image of a planar surface estimate the image-to-world estimate the image-to-world homography matrix Hhomography matrix H

2.2. RepeatRepeata)a) Select two points xSelect two points x11 and x and x22 on the image plane on the image plane

b)b) Back-project each image point into the world plane Back-project each image point into the world plane using H to obtain the two world points Xusing H to obtain the two world points X11 and X and X22

c)c) Compute the Euclidean distance Compute the Euclidean distance distdist(X(X11, X, X22))

i.i. distdist(A, B) = || A – B ||(A, B) = || A – B ||


Camera PositionCamera Position

Camera position C = (XCamera position C = (Xcc, Y, Ycc, Z, Zcc, W, Wcc)) PC = 0PC = 0 Implies:Implies:

Using Cramer’s Rule:Using Cramer’s Rule:


Camera In SceneCamera In Scene


Uncertainty AnalysisUncertainty Analysis

Errors arise from the finite accuracy Errors arise from the finite accuracy of the feature detection and of the feature detection and extractionextraction• ie- edge detection, point specificationsie- edge detection, point specifications

Uncertainty analysisUncertainty analysis attempts to attempts to quantify this errorquantify this error


Uncertainty AnalysisUncertainty Analysis

Uncertainty in Uncertainty in • Projection matrix Projection matrix PP• Top point Top point tt• Base point Base point bb• Location of vanishing line Location of vanishing line ll

• Affine scale factor Affine scale factor αα

As the number of reference distances increases, so the As the number of reference distances increases, so the uncertainty decreasesuncertainty decreases


IllustrationIllustration Ellipses are user specifiedEllipses are user specified

tt and and bb are then aligned to the vertical vanishing point are then aligned to the vertical vanishing pointAlignment constraint Alignment constraint v · (t X b) = 0v · (t X b) = 0


ApplicationsApplications

Forensic ScienceForensic Science• Height of suspectHeight of suspect

Virtual ModelingVirtual Modeling• 3D reconstruction of a scene3D reconstruction of a scene

Art HistoryArt History• Modeling paintingsModeling paintings


Forensic ScienceForensic Science


Virtual ModelingVirtual Modeling


Art HistoryArt History


ConclusionsConclusions

Affine structure of 3D space may be Affine structure of 3D space may be partially recovered from perspective partially recovered from perspective imagesimages

Measurements between and on Measurements between and on parallel planes can be determinedparallel planes can be determined

Practical applications can be derivedPractical applications can be derived

Date post:	18-Dec-2015
Category:	Documents
View:	218 times
Download:	2 times

Single View Metrology A. Criminisi, I. Reid, and A. Zisserman University of Oxford IJCV Nov 2000...

Documents