Recovering metric and affine properties from images

• Affine preserves:• Parallelism• Parallel length ratios

• Similarity preserves:• Angles • Length ratios

Will show:• Projective distortion can be removed once image of line at infinity is specified• Affine distortion removed once image of circular points is specified• Then the remaining distortion is only similarity

The line at infinity

l

1

0

0

1t

0ll

A

AH

TT

A

The line at infinity l is a fixed line under a projective transformation H if and only if H is an affinity

Note: not fixed pointwise

For an affine transformation line at infinity maps onto line at infinity

A point on line at infinity is mapped to ANOTHER point on the line at infinity, not necessarily the same point

Affine properties from imagesprojection rectification

APA

lll

HH

321

010

001

0,l 3321 llll T

Euclidean planeTwo step process:1.Find l the image of line at infinity in plane 22. Transform l to its canonical position(0,0,1) T by plugging into HPA and applying it to the entire image to get a “rectified” image3. Make affine measurements on the rectified image

Affine rectificationv1 v2

l1

l2 l4

l3

l∞

21 vvl

211 llv

432 llv

c

a b

The circular points

“circular points”

0=++++ 233231

22

21 fxxexxdxxx 0=+ 2

221 xx

l∞

T

T

0,-,1J

0,,1I

i

i

03 x

Two points on l_inf: Every circle intersects l_inf at circular points

Line at infinity

Circle:

The circular points

0

1

I i

0

1

J i

I

0

1

0

1

100

cossin

sincos

II

iseitss

tssi

y

x

S

H

The circular points I, J are fixed points under the projective transformation H iff H is a similarity

Circular points are fixed under any similarity transformation

Canonical coordinates of circular points

Identifying circular points allows recovery of similarity properties i.e. anglesratios of lengths

Conic dual to the circular points

TT JIIJ* C

000

010

001*C

TSS HCHC **

The dual conic is fixed conic under the projective transformation H iff H is a similarity

*C

Note: has 4DOF (3x3 homogeneous; symmetric, determinant is zero)

l∞ is the nullvector

*C

Angles

22

21

22

21

2211cosmmll

mlml

T321 ,,l lll T321 ,,m mmm

Euclidean Geometry: not invariant under

projective transformation

Projective: mmll

mlcos

**

*

CC

CTT

T

0ml * CT (orthogonal)

Once is identified in the projective plane, then Euclidean angles may be measured by equation (1)

*C

The above equation is invariant under projective transformation can be applied after projective transformation of the plane.

Length ratios

sin

sin

),(

),(

cad

cbd

Metric properties from images

vvv

v

'

*

*

**

TT

TT

T

TT

T

K

KKK

HHCHH

HHHCHHH

HHHCHHHC

APAP

APSSAP

SAPSAP

TUUC

000

010

001

'* UH

• Upshot: projective (v) and affine (K) components directly determined from the image of C*

∞

• Once C*∞ is identified on the projective plane then projective distortion may be

rectified up to a similarity• Can show that the rectifying transformation is obtained by applying SVD to

• Apply U to the pixels in the projective plane to rectify the image up to a Similarity

'*C

SVD Decompose to get U

'*C

Euclidean

Perspectivetransformation

Rectifying Transformation: U

Similarity transformation

C*∞

Rectified imageProjectivelyDistortedImage

'*C

Recovering up to a similarity from Projective

Recovering up to a similarity from Affine

'*C

Euclidean

affinetransformation

Rectifying Transformation: U

Similarity transformation

C*∞

Rectified imageAffinely DistortedImage

Metric from affine

000

0

3

2

1

321

m

m

m

lllTKK

0,,,, 2221211

212

21122122111 T

kkkkkmlmlmlml

Affine rectified

Metric from projective

0c,,,,, 332332133122122111 5.05.05.0 mlmlmlmlmlmlmlmlml

0vvv

v

3

2

1

321

m

m

m

lllTT

TT

K

KKK

Projective 3D geometry

Singular Value Decomposition

Tnnnmmmnm VΣUA

IUU T

021 n

IVV T

nm

XXVT XVΣ T XVUΣ T

TTTnnn VUVUVUA 222111

000

00

00

00

Σ

2

1

n

Singular Value Decomposition

• Homogeneous least-squares

• Span and null-space

• Closest rank r approximation

• Pseudo inverse

1X AXmin subject to nVX solution

nrrdiag ,,,,,, 121 0 ,, 0 ,,,,~

21 rdiag TVΣ~

UA~ TVUΣA

0000

0000

000

000

Σ 2

1

4321 UU;UU LL NS 4321 VV;VV RR NS

TUVΣA 0 ,, 0 ,,,, 112

11 rdiag

TVUΣA

Projective 3D Geometry

• Points, lines, planes and quadrics

• Transformations

• П∞, ω∞ and Ω ∞

3D points

TT

1 ,,,1,,,X4

3

4

2

4

1 ZYXX

X

X

X

X

X

in R3

04 X

TZYX ,,

in P3

XX' H (4x4-1=15 dof)

projective transformation

3D point

T4321 ,,,X XXXX

Planes

0ππππ 4321 ZYX

0ππππ 44332211 XXXX

0Xπ T

Dual: points ↔ planes, lines ↔ lines

3D plane

0X~

.n d T321 π,π,πn TZYX ,,X~

14 Xd4π

Euclidean representation

n/d

XX' Hππ' -TH

Transformation

Planes from points

0π

X

X

X

3

2

1

T

T

T

2341DX T123124134234 ,,,π DDDD

0det

4342414

3332313

2322212

1312111

XXXX

XXXX

XXXX

XXXX

0πX 0πX 0,πX π 321 TTT andfromSolve

(solve as right nullspace of )π

T

T

T

3

2

1

X

X

X

0XXX Xdet 321

Or implicitly from coplanarity condition

124313422341 DXDXDX 01234124313422341 DXDXDXDX 13422341 DXDX

Points from planes

0X

π

π

π

3

2

1

T

T

T

xX M 321 XXXM

0π MT

0Xπ 0Xπ 0,Xπ X 321 TTT andfromSolve

(solve as right nullspace of )X

T

T

T

3

2

1

π

π

π

Representing a plane by its span

• M is 4x3 matrix. Columns of M are null space of• X is a point on plane• x ( a point on projective plane P2 ) parameterizes

points on the plane π • M is not unique

Tπ

Lines

T

T

B

AW μBλA

2×2** 0=WW=WWTT

0001

1000W

0010

0100W*

Example: X-axis:

(4dof)

Representing a line by its span: two vectors A, B for two space points

T

T

Q

PW* μQλPDual representation: P and Q are planes; line

is span of row space of W*

join of (0,0,0) and (1,0,0) points Intersection of y=0 and z=0

planes

• Line is either joint of two points or intersection of two planes

• Span of WT is the pencil of points on the line• Span of the 2D right null space of W is the pencil of the planes with the line as axis

• Span of W*T is the pencil of Planes with the line as axis

2x4

2x4

Points, lines and planes

TX

WM 0π M

Tπ

W*

M 0X M

W

X

*W

π

Plane π defined by the join of the point X and line W is obtained from the null space of M:

• Point X defined by the intersection of line W with plane is the null space of Mπ

Quadrics and dual quadrics

(Q : 4x4 symmetric matrix)0QXX T

1. 9 d.o.f.

2. in general 9 points define quadric

3. det Q=0 ↔ degenerate quadric

4. (plane ∩ quadric)=conic

5. transformation

Q

QMMC T MxX:π -1-TQHHQ'

0πQπ * T

-1* QQ 1. relation to quadric (non-degenerate)

2. transformation THHQQ' **

• Dual Quadric: defines equation on planes: tangent planes π to the point quadric Q satisfy:

Quadratic surface in P3 defined by:

Quadric classification

Rank Sign.

Diagonal Equation Realization

4 4 (1,1,1,1) X2+ Y2+ Z2+1=0 No real points

2 (1,1,1,-1) X2+ Y2+ Z2=1 Sphere

0 (1,1,-1,-1) X2+ Y2= Z2+1 Hyperboloid (1S)

3 3 (1,1,1,0) X2+ Y2+ Z2=0 Single point

1 (1,1,-1,0) X2+ Y2= Z2 Cone

2 2 (1,1,0,0) X2+ Y2= 0 Single line

0 (1,-1,0,0) X2= Y2 Two planes

1 1 (1,0,0,0) X2=0 Single plane

Quadric classificationProjectively equivalent to sphere:

hyperboloid of two sheets

paraboloidsphere ellipsoid

Degenerate ruled quadrics:

cone two planes

Ruled quadrics:

hyperboloids of one sheet

Hierarchy of transformations

vTv

tAProjective15dof

Affine12dof

Similarity7dof

Euclidean6dof

Intersection and tangency

Parallellism of planes,Volume ratios, centroids,The plane at infinity π∞

The absolute conic Ω∞

Volume

10

tAT

10

tRT

s

10

tRT

group transform distortion invariants properties

The plane at infinity

π

1

0

0

0

1t

0ππ

A

AH

TT

A

The plane at infinity π is a fixed plane under a projective transformation H iff H is an affinity

1. canonical position2. contains directions 3. two planes are parallel line of intersection in π∞

4. line // line (or plane) point of intersection in π∞

T1,0,0,0π

T0,,,D 321 XXX

The absolute conic

The absolute conic Ω∞ is a fixed conic under the projective transformation H iff H is a similarity

04

23

22

21

X

XXX

The absolute conic Ω∞ is a (point) conic on π. In a metric frame:

1. Ω∞ is only fixed as a set, not pointwise2. Circle intersect Ω∞ in two points3. All Spheres intersect π∞ in Ω∞

The absolute conic

2211

21

dddd

ddcos

TT

T

2211

21

dddd

ddcos

TT

T

0dd 21 T

Euclidean:

Projective:

(orthogonality=conjugacy)

d1 and d2 are lines

The absolute dual quadric

00

0I*TQ

The absolute conic Q*∞ is a fixed conic under the

projective transformation H iff H is a similarity

1. 8 dof2. plane at infinity π∞ is the nullvector of 3. Angles:

2*

21*

1

2*

1

ππππ

ππcos

QQ

QTT

T

*Q

Absolute dual Quadric = All planes tangent to Ω∞