1

CS252A, Fall 2011 Computer Vision I

Cameras and Radiometry

Computer Vision I CSE 252A Lecture 6

CS252A, Fall 2011 Computer Vision I

Announcements •  Assignment 1 Posted to the web page: Due

Tuesday, 10/18/10 •  Reading Szeliski Chapter 9, particularly section

9.1

CS252A, Fall 2011 Computer Vision I

When we last met … and what Andrew covered.

CS252A, Fall 2011 Computer Vision I

Homogenous coordinates A way to represent points in a projective space

Use three numbers to represent a point on a projective plane

Add an extra coordinate e.g., (x,y) -> (x,y,1)

Impose equivalence relation (x,y,z) ≈ λ*(x,y,z) such that (λ not 0)

i.e., (x,y,1) ≈ (λx, λy, λ) X

Y (x,y)

(x,y,1)

1 Z

CS252A, Fall 2011 Computer Vision I

Conversion Euclidean -> Homogenous -> Euclidean In 2-D •  Euclidean -> Homogenous:

(x, y) -> k (x,y,1)

•  Homogenous -> Euclidean: (x, y, z) -> (x/z, y/z)

In 3-D •  Euclidean -> Homogenous:

(x, y, z) -> k (x,y,z,1)

•  Homogenous -> Euclidean: (x, y, z, w) -> (x/w, y/w, z/w)

X

Y (x,y)

(x,y,1)

1 Z

CS252A, Fall 2011 Computer Vision I

Points at infinity

X

Y

(x,y,1)

1 Z

Point at infinity – zero for last coordinate (x,y,0)

and equivalence relation (x,y,0) ≈ λ*(x,y,0)

No corresponding Euclidean point

(x,y,0)

Projective Plane

Affine Plane = + Line at

Infinity

2

CS252A, Fall 2011 Computer Vision I

The equation of projection

Cartesian coordinates:

€

(x,y,z)→( f ' xz, f ' y

z)

€

UVW

⎛

⎝

⎜ ⎜ ⎜

⎞

⎠

⎟ ⎟ ⎟

=

1 0 0 00 1 0 00 0 1

f ' 0

⎛

⎝

⎜ ⎜ ⎜ ⎜

⎞

⎠

⎟ ⎟ ⎟ ⎟

XYZT

⎛

⎝

⎜ ⎜ ⎜ ⎜

⎞

⎠

⎟ ⎟ ⎟ ⎟

Homogenous Coordinates and Camera matrix

CS252A, Fall 2011 Computer Vision I

A convenient notation

Points: AP1 Leading superscript indicates the coordinate system that the coordinates are with respect to Subscript – an identifier

Rotation Matrices Lower left (Going from this system) Upper left (Going to this system)

To add vectors, coordinate systems must agree To rotate a vector, points coordinate system must agree with lower left of rotation matrix

CS252A, Fall 2011 Computer Vision I

SO(3) •  RTR = I, det(R) = 1 -> R-1 = RT

•  SO(3) is 3-D manifold •  Parameterizations of SO(3)

– Roll-Pitch-Yaw – Euler Angles – Axis Angle (Rodrigues formula) – Cayley’s formula – Matrix Exponential – Quaternions (four parameters + one constraint)

•  Not commutative CS252A, Fall 2011 Computer Vision I

Block Matrix Multiplication

What is AB ?

Homogeneous Representation of Rigid Transformations

€

BP1

⎡

⎣ ⎢

⎤

⎦ ⎥ = A

BR BOA

0T 1

⎡

⎣ ⎢

⎤

⎦ ⎥

AP1

⎡

⎣ ⎢

⎤

⎦ ⎥ = A

BR AP + BOA

1

⎡

⎣ ⎢

⎤

⎦ ⎥ = A

BTAP1

⎡

⎣ ⎢

⎤

⎦ ⎥

CS252A, Fall 2011 Computer Vision I

Camera parameters •  Issue

–  World units (e.g., cm), camera units (pixels) –  camera may not be at the origin, looking down the z-axis

•  extrinsic parameters –  one unit in camera coordinates may not be the same as one

unit in world coordinates •  intrinsic parameters - focal length, principal point, aspect ratio,

angle between axes, etc.

•  Can be written as u = Mp where u=( u v w)T, p= (x,y,z,t)T

, and M is 3x4 camera matrix

3 x 3 4 x 4: Rigid transformation

CS252A, Fall 2011 Computer Vision I

, estimate intrinsic and extrinsic camera parameters

•  See Text book for how to do it. •  Camera Calibration Toolbox for Matlab (Bouguet) http://www.vision.caltech.edu/bouguetj/calib_doc/

Camera Calibration

3

CS252A, Fall 2011 Computer Vision I

A Hierarchy of 2-D to 2D transformations

€

u1u2u3

⎡

⎣

⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥

=

a11 a12 a13a21 a22 a23a31 a32 a33

⎡

⎣

⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥

x1x2x3

⎡

⎣

⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥

Projective (Homography)

Affine

Similarity

Rigid

€

u1u2u3

⎡

⎣

⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥

=

a11 a12 a13a21 a22 a230 0 1

⎡

⎣

⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥

x1x2x3

⎡

⎣

⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥

€

u1u2u3

⎡

⎣

⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥

=

a11 a12 a13−a12 a11 a230 0 1

⎡

⎣

⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥

x1x2x3

⎡

⎣

⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥

€

u1u2u3

⎡

⎣

⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥

=

a11 a12 a13a21 a22 a230 0 1

⎡

⎣

⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥

x1x2x3

⎡

⎣

⎢ ⎢ ⎢

⎤

⎦

⎥ ⎥ ⎥

where

€

a11 a12a21 a22

⎡

⎣ ⎢

⎤

⎦ ⎥ =

cosθ sinθ−sinθ cosθ⎡

⎣ ⎢

⎤

⎦ ⎥

CS252A, Fall 2011 Computer Vision I

Figure borrowed from Hartley and Zisserman “Multiple View Geometry in computer vision”

Central Projection

CS252A, Fall 2011 Computer Vision I

Figure borrowed from Hartley and Zisserman “Multiple View Geometry in computer vision”

Planar Homography

CS252A, Fall 2011 Computer Vision I

Figure borrowed from Hartley and Zisserman “Multiple View Geometry in computer vision”

Planar Homography: Pure Rotation

CS252A, Fall 2011 Computer Vision I

Radiometry Read Chapter 4 of Ponce & Forsyth

•  Solid Angle •  Irradiance •  Radiance •  BRDF •  Lambertian/Phong BRDF

CS252A, Fall 2011 Computer Vision I

, estimate intrinsic and extrinsic camera parameters

•  See Text book for how to do it. •  Camera Calibration Toolbox for Matlab (Bouguet) http://www.vision.caltech.edu/bouguetj/calib_doc/

Camera Calibration

4

CS252A, Fall 2011 Computer Vision I

Solid Angle

•  By analogy with angle (in radians), the solid angle subtended by a region at a point is the area projected on a unit sphere centered at that point

•  The solid angle dω subtended by a small patch with area dA at distance r from the origin and at angle θ is:

•  Power is energy per unit time

•  Radiance: Power traveling at some point in a specified direction, per unit area perpendicular to the direction of travel, per unit solid angle

•  Symbol: L(x,θ,φ)

•  Units: watts per square meter per steradian : w/(m2sr1)

– CSE 252A

€

L =P

(dAcosα)dω

x

dA

α

dω

(θ, φ)

Power emitted from patch, but radiance in direction different from surface normal

CS252A, Fall 2010 Computer Vision I

Radiance transfer

From definition of radiance

– What is the power received by a small area dA2 at distance r from a small area dA1 emitting radiance L?

€

L =P

(dAcosθ)dω

–  From definition of solid angle

•  How much light is arriving at a surface?

•  Units of irradiance: Watts/m2 •  This is a function of incoming angle. •  A surface experiencing radiance L

(x,θ,φ) coming in from solid angle dω experiences irradiance:

•  Crucial property: Total Irradiance arriving at the surface is given by adding irradiance over all incoming angles Total irradiance is

– CSE 252A

N φ

L(x,θ,φ)

x x

θ

– CSE 252A, Winter 2007 – Image Irradiance !

E ="4

dz'#

$ % &

' ( 2

cos4)*

+ ,

-

. / L

– E: Image irradiance – L: emitted radiance – d : Lens diameter – z’: depth of image plane – α: Angle of patch from optical axis

– δA

– δA’

CS252A, Fall 2011 Computer Vision I

Camera’s sensor

– CSE 252A

•  Measured pixel intensity is a function of irradiance integrated over –  pixel’s area –  over a range of wavelengths –  for some period of time

•  Ideally, it’s linear to the radiance, but the camera response C(.) may not be linear

5

CS252A, Fall 2011 Computer Vision I

Color Cameras

We considered four concepts:

1.  Prism (with 3 sensors) 2.  Filter mosaic 3.  Filter wheel\ 4.  Fovean

CS252A, Fall 2011 Computer Vision I

Light at surfaces Many effects when light strikes a

surface -- could be: •  transmitted

–  Skin, glass •  reflected

–  mirror •  scattered

–  milk •  travel along the surface and

leave at some other point •  absorbed

–  sweaty skin

Assume that •  surfaces don’t fluoresce

–  e.g. scorpions, detergents •  surfaces don’t emit light (i.e.

are cool) •  all the light leaving a point is

due to that arriving at that point

CS252A, Fall 2011 Computer Vision I

BRDF

^ n (θin,φin)

(θout,φout)

Where ρ is sometimes denoted fr.