Single-view geometry

Odilon Redon, Cyclops, 1914

Our goal: Recovery of 3D structure• Recovery of structure from one image is

inherently ambiguous

x

X?X?

X?

Our goal: Recovery of 3D structure• Recovery of structure from one image is

inherently ambiguous

Our goal: Recovery of 3D structure• Recovery of structure from one image is

inherently ambiguous

Ames Room

http://en.wikipedia.org/wiki/Ames_room

Our goal: Recovery of 3D structure• We will need multi-view geometry

Recall: Pinhole camera model

• Principal axis: line from the camera center perpendicular to the image plane

• Normalized (camera) coordinate system: camera center is at the origin and the principal axis is the z-axis

)/,/(),,( ZYfZXfZYX

101

0

0

1

Z

Y

X

f

f

Z

Yf

Xf

Z

Y

X

Recall: Pinhole camera model

PXx

Principal point

• Principal point (p): point where principal axis intersects the image plane

• Normalized coordinate system: origin of the image is at the principal point

• Image coordinate system: origin is in the corner• How to go from normalized coordinate system to image

coordinate system?

px

py

)/,/(),,( yx pZYfpZXfZYX

Z

pZYf

pZXf

Z

Y

X

y

x

1

Principal point offset

principal point: ),( yx pp

px

py

101

0

0

Z

Y

X

pf

pf

y

x

101

01

01

1Z

Y

X

pf

pf

Z

ZpYf

ZpXf

y

x

y

x

Principal point offset

1y

x

pf

pf

K calibration matrix 0|IKP

principal point: ),( yx pp

111yy

xx

y

x

y

x

pf

pf

m

m

K

Pixel coordinates

mx pixels per meter in horizontal direction, my pixels per meter in vertical direction

Pixel size: yx mm

11

pixels/m m pixels

C~

-X~

RX~

cam

Camera rotation and translation

• In general, the camera coordinate frame will be related to the world coordinate frame by a rotation and a translation

coords. of point in camera frame

coords. of camera center in world frame

coords. of a pointin world frame

• Conversion from world to camera coordinate system (in non-homogeneous coordinates):

C~

-X~

RX~

cam

X10

C~

RR

1

X~

10

C~

RRXcam

XC~

R|RKX0|IKx cam ,t|RKP C~

Rt

Camera rotation and translation

Note: C is the null space of the camera projection matrix (PC=0)

Camera parameters

• Intrinsic parameters• Principal point coordinates• Focal length• Pixel magnification factors• Skew (non-rectangular pixels)• Radial distortion

111yy

xx

y

x

y

x

pf

pf

m

m

K

tRKP

Camera parameters

• Intrinsic parameters• Principal point coordinates• Focal length• Pixel magnification factors• Skew (non-rectangular pixels)• Radial distortion

• Extrinsic parameters• Rotation and translation relative to world coordinate system

tRKP

Camera calibration

1****

****

****

Z

Y

X

y

x

XtRKx

Source: D. Hoiem

Camera calibration

• Given n points with known 3D coordinates Xi and known image projections xi, estimate the camera parameters

? P

Xi

xi

ii PXx

Camera calibration: Linear method

0PXx ii0

XP

XP

XP

1 3

2

1

iT

iT

iT

i

i

y

x

0

P

P

P

0XX

X0X

XX0

3

2

1

Tii

Tii

Tii

Ti

Tii

Ti

xy

x

y

Two linearly independent equations

Camera calibration: Linear method

• P has 11 degrees of freedom• One 2D/3D correspondence gives us two linearly

independent equations• Homogeneous least squares: find p minimizing ||Ap||2

• Solution given by eigenvector of ATA with smallest eigenvalue• 6 correspondences needed for a minimal solution

0pA 0

P

P

P

X0X

XX0

X0X

XX0

3

2

1111

111

Tnn

TTn

Tnn

Tn

T

TTT

TTT

x

y

x

y

Camera calibration: Linear method

• Note: for coplanar points that satisfy ΠTX=0,we will get degenerate solutions (Π,0,0), (0,Π,0), or (0,0,Π)

0Ap0

P

P

P

X0X

XX0

X0X

XX0

3

2

1111

111

Tnn

TTn

Tnn

Tn

T

TTT

TTT

x

y

x

y

Camera calibration: Linear method

• Advantages: easy to formulate and solve• Disadvantages

• Doesn’t directly tell you camera parameters• Doesn’t model radial distortion• Can’t impose constraints, such as known focal length and

orthogonality

• Non-linear methods are preferred• Define error as squared distance between projected points

and measured points• Minimize error using Newton’s method or other non-linear

optimization

Source: D. Hoiem