Camera calibration technique

transcript

Camera calibration techniquewprowadzenie teoretyczne

Krzysztof WegnerChair of Multimedia Telecommunications and Microelectronics

Poznań University of Technology, Poland

Goal of the calibration Knowledge about

Intrinsic camera parameters Focal length Optical center

Extrinsic camera parameters -Position of the camera in 3D world

Orientation of the camera Translation

Goal of the calibration Knowledge about

Intrinsic camera parameters Focal length Optical center

Orientation of the camera Translation Common word coordinate system

Camera parameters Intrinsic camera parameters

Focal length Optical center

Orientation of the camera Translation

𝑨=[ 𝑓 𝑢 𝛾 𝑜𝑢

0 𝑓 𝑣 𝑜𝑣

0 0 1 ]

𝑹=[𝒓𝟏 𝒓𝟐 𝒓𝟑 ]𝑻=[𝑡𝑥𝑡𝑦𝑡 𝑧 ]

Camera model Projection of a 3D point Onto a point at image plane

s is a scale – distance to the point

𝑠 ∙[𝑢𝑣1 ]=𝐴 ∙ [𝑅 −𝑅 ∙𝑇 ] ∙ [𝑋𝑌𝑍1 ]

Zhang’s Algorithm Allows estimation of

intrinsic parameters - matrix extrinsic parameters – rotation matrix and

translation vector Planar template

6Z. Zhang, “A flexible new technique for camera calibration”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330–1334, 2000

𝑠 ∙[𝑢𝑣1 ]=𝐴 ∙ [𝑟 1 𝑟2 𝑟 3~𝑡 ] ∙ [ 𝑋𝑌𝑍1 ]

Zhang’s algorithm Planar template

Estimating Homography H We know position of the pattern’s feature points From registrated image we know

Estimating Homography H

Let’s assign

We have 2 equations and 9 variables so we need at least 5 points to solve uniquely for

Estimating Homography H Defined up to a scale factor We know position of the pattern’s feature points From registrated image we know

Multiplication of both side by don’t change known

So we don’t know whether we obtain or So we don’t know scale of the scene

𝜆 ∙𝑠 ∙ [𝑢𝑣1 ]=𝝀 ∙𝑯 ∙[ 𝑋𝑌1 ] 𝜆 ∙𝑠 ∙[𝑢𝑣1 ]=~𝑯 ∙[ 𝑋𝑌1 ]

Homography H Defined up to a scale factor We know position of the pattern’s feature points From registrated image we know

𝜆 ∙𝑠 ∙ [𝑢𝑣1 ]=𝝀 ∙𝑯 ∙[ 𝑋𝑌1 ]𝜆 ∙𝑠 ∙ [𝑢𝑣1 ]=~𝑯 ∙[ 𝑋𝑌1 ]

Homography H Defined up to a scale factor

So we don’t know whether we obtain or So we don’t know Z scale of the scene

𝜆 ∙𝑠 ∙ [𝑢𝑣1 ]=𝝀 ∙𝑯 ∙[ 𝑋𝑌1 ]𝜆 ∙𝑠 ∙ [𝑢𝑣1 ]=~𝑯 ∙[ 𝑋𝑌1 ]

Constraints on the intrinsic Vectors , are orthonormal so taking dot product

and length of , should be the same

we have

𝒓𝟏𝑻 ∙𝒓𝟐=𝟎

‖𝒓 𝟏‖𝟐=‖𝒓𝟐‖

𝟐⟹𝒓𝟏𝑻 ∙𝒓 𝟏=𝒓𝟐

𝑻 ∙𝒓𝟐

Constraints on the intrinsic Puting all together

𝒓𝟏𝑻 ∙𝒓𝟐=𝟎

𝒓𝟏𝑻 ∙𝒓𝟏=𝒓 𝟐

𝑻 ∙𝒓𝟐

(𝜆−1 ∙𝑨−𝟏 ∙𝒉𝟏 )𝑻 ∙𝜆−1 ∙ 𝑨−𝟏 ∙𝒉𝟐=𝟎𝒉𝟏

𝑻 ∙𝑨−𝑻 ∙𝜆−1∙ 𝜆− 1 ∙ 𝑨−𝟏 ∙𝒉𝟐=𝟎𝒉𝟏

𝑻 ∙𝑨−𝑻 ∙ 𝑨−𝟏 ∙𝒉𝟐=𝟎

(𝜆−1 ∙𝑨−𝟏 ∙𝒉𝟏 )𝑻 ∙ 𝜆−1 ∙ 𝑨−𝟏 ∙𝒉𝟏=(𝜆− 1 ∙ 𝑨−𝟏 ∙𝒉𝟐 )𝑻 ∙𝜆−1 ∙𝑨−𝟏 ∙𝒉𝟐

𝒉𝟏𝑻 ∙𝑨−𝑻 ∙ 𝜆−1 ∙𝜆− 1 ∙ 𝑨−𝟏 ∙𝒉𝟏=𝒉𝟐

𝑻 ∙𝑨−𝑻 ∙𝜆− 1 ∙𝜆− 1 ∙ 𝑨−𝟏 ∙𝒉𝟐

𝒉𝟏𝑻 ∙𝑨−𝑻 ∙ 𝑨−𝟏 ∙𝒉𝟏=𝒉𝟐

𝑻 ∙ 𝑨−𝑻 ∙𝑨−𝟏 ∙𝒉𝟐

Constraints on the intrinsic Puting all together

𝒓𝟏𝑻 ∙𝒓𝟐=𝟎

𝒓𝟏𝑻 ∙𝒓𝟏=𝒓 𝟐

𝑻 ∙𝒓𝟐

(𝜆−1 ∙𝑨−𝟏 ∙𝒉𝟏 )𝑻 ∙𝜆−1 ∙ 𝑨−𝟏 ∙𝒉𝟐=𝟎𝒉𝟏

𝑻 ∙𝑨−𝑻 ∙𝜆−1∙ 𝜆− 1 ∙ 𝑨−𝟏 ∙𝒉𝟐=𝟎𝒉𝟏

𝑻 ∙𝑨−𝑻 ∙ 𝑨−𝟏 ∙𝒉𝟐=𝟎

(𝜆−1 ∙𝑨−𝟏 ∙𝒉𝟏 )𝑻 ∙ 𝜆−1 ∙ 𝑨−𝟏 ∙𝒉𝟏=(𝜆− 1 ∙ 𝑨−𝟏 ∙𝒉𝟐 )𝑻 ∙𝜆−1 ∙𝑨−𝟏 ∙𝒉𝟐

𝒉𝟏𝑻 ∙𝑨−𝑻 ∙ 𝜆−1 ∙𝜆− 1 ∙ 𝑨−𝟏 ∙𝒉𝟏=𝒉𝟐

𝑻 ∙𝑨−𝑻 ∙𝜆− 1 ∙𝜆− 1 ∙ 𝑨−𝟏 ∙𝒉𝟐

𝒉𝟏𝑻 ∙𝑨−𝑻 ∙ 𝑨−𝟏 ∙𝒉𝟏=𝒉𝟐

𝑻 ∙ 𝑨−𝑻 ∙𝑨−𝟏 ∙𝒉𝟐

Closed form solution Try to solve

Intrinsic matrix

Let’s assign

𝑨=[ 𝑓 𝑢 𝛾 𝑜𝑢

0 𝑓 𝑣 𝑜𝑣

0 0 1 ]𝒉𝟏

𝑻 ∙𝑨−𝑻 ∙ 𝑨−𝟏 ∙𝒉𝟐=𝟎𝒉𝟏

𝑻 ∙𝑨−𝑻 ∙ 𝑨−𝟏 ∙𝒉𝟏=𝒉𝟐𝑻 ∙ 𝑨−𝑻 ∙𝑨−𝟏 ∙𝒉𝟐

𝑩=𝑨−𝑻 ∙ 𝑨−𝟏

𝑩=[1𝑓 𝑢2

−𝛾𝑓 𝑢2 𝑓 𝑣

𝑜𝑣𝛾− 𝑓 𝑣𝑜𝑢

𝑓 𝑢2 𝑓 𝑣

1𝑓 𝑣

2+𝛾2

−𝑜𝑣❑

𝑓 𝑣2 −

𝛾 (𝑜𝑣𝛾− 𝑓 𝑣𝑜𝑢)𝑓 𝑢2 𝑓 𝑣

−𝑜𝑣❑

𝑓 𝑣2 −

𝛾 (𝑜𝑣𝛾− 𝑓 𝑣𝑜𝑢 )𝑓 𝑢2 𝑓 𝑣

2 1+𝑜𝑣

𝑓 𝑣2 +

(𝑜𝑣𝛾− 𝑓 𝑣𝑜𝑢 )2

2]=[𝐵11 𝐵12 𝐵13

𝐵12 𝐵22 𝐵23

𝐵13 𝐵23 𝐵33]

Closed form solution Let’s assign

is symetrical Let’s assign

𝑩=𝑨−𝑻 ∙ 𝑨−𝟏

𝑩=[1𝑓 𝑢2

1𝑓 𝑣

2+𝛾2

−𝑜𝑣❑

𝑓 𝑣2 −

−𝑜𝑣❑

𝑓 𝑣2 −

2 1+𝑜𝑣

𝑓 𝑣2 +

2]=[𝐵11 𝐵12 𝐵13

𝐵12 𝐵22 𝐵23

𝐵13 𝐵23 𝐵33]

𝒃=[𝐵11 𝐵12 𝐵13 𝐵22 𝐵23 𝐵33 ]

Closed form solution We try to solve

Let’s see pattern in the equations

𝒉𝟏𝑻 ∙𝑨−𝑻 ∙ 𝑨−𝟏 ∙𝒉𝟐=𝟎

𝒉𝟏𝑻 ∙𝑨−𝑻 ∙ 𝑨−𝟏 ∙𝒉𝟏=𝒉𝟐

𝑻 ∙ 𝑨−𝑻 ∙𝑨−𝟏 ∙𝒉𝟐

𝑩=𝑨−𝑻 ∙ 𝑨−𝟏

𝒉𝟏𝑻 ∙𝑩 ∙𝒉𝟐=𝟎𝒉𝟏

𝑻 ∙𝑩 ∙𝒉𝟏=𝒉𝟐𝑻 ∙𝑩 ∙𝒉𝟐

𝒉𝒊𝑻 ∙𝑩 ∙𝒉 𝒋=𝒗𝒊𝒋

𝑻 ∙𝒃=

¿𝐵11 ∙ h𝑖1 ∙ h 𝑗 1+𝐵12 ∙ h𝑖1 ∙ h 𝑗2+𝐵13 ∙ h𝑖1 ∙ h 𝑗3+¿

¿ 𝒗𝒊𝒋𝑻 ∙𝒃

𝒗 𝒊𝒋𝑻=[h𝑖 1 ∙ h 𝑗 1 h𝑖1 ∙ h 𝑗 2+h𝑖2 ∙ h 𝑗1 h𝑖1 ∙h 𝑗3+h𝑖3 ∙ h 𝑗 1 h𝑖2 ∙ h 𝑗 2 h𝑖 2 ∙ h 𝑗 3+h𝑖 3 ∙ h 𝑗2 h𝑖3 ∙ h 𝑗3 ]

Closed form solution We try to solve

𝒉𝟏𝑻 ∙𝑩 ∙𝒉𝟐=𝟎𝒉𝟏

𝑻 ∙𝑩 ∙𝒉𝟏−𝒉𝟐𝑻 ∙𝑩 ∙𝒉𝟐=𝟎

𝒉𝒊𝑻 ∙𝑩 ∙𝒉 𝒋=𝒗𝒊𝒋

𝑻 ∙𝒃

𝒗𝟏𝟐𝑻 ∙𝒃=𝟎𝒗𝟏𝟏𝑻 ∙𝒃−𝒗𝟐𝟐

𝑻 ∙𝒃=𝟎𝒗𝟏𝟐𝑻 ∙𝒃=𝟎

(𝒗𝟏𝟏𝑻 −𝒗𝟐𝟐

𝑻 ) ∙𝒃=𝟎

[ 𝒗𝟏𝟐𝑻

(𝒗𝟏𝟏−𝒗𝟐𝟐 )𝑻 ] ∙𝒃=𝟎

Solving for b

Two equation are defined but have 6 unknowns. So at least 3 images are required to uniquly solve for Because all images are captured with the same

camera we can stack equation for together

[ 𝒗𝟏𝟐𝑻

(𝒗𝟏𝟏−𝒗𝟐𝟐 )𝑻 ] ∙𝒃=𝟎

[ 𝒗𝟏𝟐′𝑻

(𝒗 ′𝟏𝟏−𝒗 ′𝟐𝟐)𝑻 ] ∙𝒃=𝟎

[ 𝒗 ′ ′𝟏𝟐𝑻

(𝒗 ′ ′𝟏𝟏−𝒗 ′ ′𝟐𝟐 )𝑻 ]∙𝒃=𝟎

[ 𝒗𝟏𝟐𝑻

(𝒗𝟏𝟏−𝒗𝟐𝟐 )𝑻 ] ∙𝒃=𝟎

[𝒗𝟏𝟐𝑻

(𝒗𝟏𝟏−𝒗𝟐𝟐 )𝑻

𝒗𝟏𝟐′𝑻

(𝒗 ′𝟏𝟏−𝒗 ′𝟐𝟐 )𝑻

𝒗 ′ ′𝟏𝟐𝑻

(𝒗 ′ ′𝟏𝟏−𝒗 ′ ′𝟐𝟐)𝑻] ∙𝒃=𝟎𝑽=[

𝒗𝟏𝟐𝑻

(𝒗𝟏𝟏−𝒗𝟐𝟐 )𝑻

𝒗𝟏𝟐′𝑻

(𝒗 ′𝟏𝟏−𝒗 ′𝟐𝟐)𝑻

𝒗 ′ ′𝟏𝟐𝑻

(𝒗 ′ ′𝟏𝟏−𝒗 ′ ′𝟐𝟐 )𝑻]

Solving for b

There is trivia solution

But we look for non trivial solution so Such solution is given by eigenvector of

asociated with the smallest eigenvalue (right singulart vector of

𝑽 ∙𝒃=𝟎

𝑽 𝑻 ∙𝑽

Retriving intrinsic parameters Once we have We can calculate intrinsic parameters from

𝑩=[1𝑓 𝑢2

1𝑓 𝑣

2+𝛾2

−𝑜𝑣❑

𝑓 𝑣2 −

−𝑜𝑣❑

𝑓 𝑣2 −

2 1+𝑜𝑣

𝑓 𝑣2 +

2]=[𝐵11 𝐵12 𝐵13

𝐵12 𝐵22 𝐵23

𝐵13 𝐵23 𝐵33]

𝒃=[𝐵11 𝐵12 𝐵13 𝐵22 𝐵23 𝐵33 ]

𝑜𝑣=𝐵12 ∙𝐵13−𝐵11 ∙𝐵23

𝐵11 ∙𝐵22−𝐵122𝑓 𝑢=√ 𝜆

𝐵11

𝜆=𝐵33−𝐵132 +𝑜𝑣 (𝐵12 ∙𝐵13−𝐵11 ∙𝐵23 )

𝐵11

𝑓 𝑣=√ 𝜆 ∙𝐵11

𝐵11 ∙𝐵22−𝐵122

𝛾=−𝐵12 ∙ 𝑓 𝑢2 ∙ 𝑓 𝑣

𝜆𝑜𝑢=

𝛾 ∙𝑜𝑣

𝑓 𝑣−𝐵13

𝑓 𝑢2

Retriving position We know constraints

To complete rotation matrix we calculater third column

Finally we ortonormalize rotation matrix

𝒓𝟑=𝒓𝟏×𝒓 𝟐

‖𝒓 𝟏‖=‖𝒓𝟐‖=‖𝒓𝟑‖=𝟏

Summary Camera parameters requires at least 5

point pattern Intrinsic camera parameters estimation

requires at least 3 images at different orientation

All parameters are defined up to a unknown scale

M37232, October 2015, Geneve 27

Camera calibration technique

Education