+ All Categories
Home > Documents > Steerable Projector Calibration

Steerable Projector Calibration

Date post: 10-Jan-2016
Category:
Upload: coy
View: 69 times
Download: 0 times
Share this document with a friend
Description:
Steerable Projector Calibration. Talk for Procams 2005 workshop, 25 June 2005. Mark ASHDOWN www.mark.ashdown.name. Yoichi SATO www.hci.iis.u-tokyo.ac.jp/~ysato/. Sato Lab, Institute of Industrial Science, University of Tokyo, Japan. Overview. Steerable projectors Previous work - PowerPoint PPT Presentation
Popular Tags:
21
Steerable Projector Steerable Projector Calibration Calibration Talk for Procams 2005 workshop, 25 June 2005 Mark ASHDOWN www.mark.ashdown.name Yoichi SATO www.hci.iis.u-tokyo.ac.jp/ ~ysato/ Sato Lab, Institute of Industrial Science, University of Tokyo, Japan
Transcript
Page 1: Steerable Projector Calibration

Steerable Projector CalibrationSteerable Projector CalibrationTalk for Procams 2005 workshop, 25 June 2005

Mark ASHDOWNwww.mark.ashdown.name

Yoichi SATOwww.hci.iis.u-tokyo.ac.jp/~ysato/

Sato Lab, Institute of Industrial Science, University of Tokyo, Japan

Page 2: Steerable Projector Calibration

Steerable Projector CalibrationProcams, 25 Jun 2005

www.mark.ashdown.name

OverviewOverview

• Steerable projectors• Previous work• Overview of algorithm• Calibrating the camera and projector• Obtaining the pan-tilt mirror parameters• Iterative refinement• Performance• Future work

Page 3: Steerable Projector Calibration

Steerable Projector CalibrationProcams, 25 Jun 2005

www.mark.ashdown.name

Steerable projectorSteerable projector

• Can use mirror or moving projector• Projector may or may not rotate about is optical centre

Borkowski, Riff, and Crowley (INRIA Rhone-Alpes), Procams 2003.

Mitsugami et al (Nara Inst. Sci. Tech.), MIRU 2004.

Page 4: Steerable Projector Calibration

Steerable Projector CalibrationProcams, 25 Jun 2005

www.mark.ashdown.name

ApplicationsApplications

Everywhere Display, http://www.research.ibm.com/ed/ Escritoire, http://www.mark.ashdown.name/

Butz, Scheinder, Spassova;

SearchLight, Pervasive 2004

Page 5: Steerable Projector Calibration

Steerable Projector CalibrationProcams, 25 Jun 2005

www.mark.ashdown.name

Previous workPrevious work

• Planar homographies, homography trees, non-planar surfaces, continuous registration.

• Use calibrated projector to get camerascreen homography• Assume limited projector model to calibrate display wall• Projector rotating around optical centre

Andrew Raij and Marc Pollefeys. Auto-Calibration of Multi-Projector Display Walls. In Proceedings of ICPR 2004, 2004.

Takayuki Okatani and Koichiro Deguchi. Autocalibration of a Projector-Screen-Camera System: Theory and Algorithm for Screen-to-Camera Homography Estimation. In Proceedings of ICCV 2003, 2003.

100

0

00

df

f

Page 6: Steerable Projector Calibration

Steerable Projector CalibrationProcams, 25 Jun 2005

www.mark.ashdown.name

Review of transformationsReview of transformations

• Homogeneous co-ordinates• 2D transformations• Projective camera

PXtRK

10100

0010

0001

100

0

0

100

0

0

Z

Y

X

Z

Y

X

pf

pf

w

y

x

Z

Y

X

pf

pf

w

y

x

yy

xx

yy

xx

Page 7: Steerable Projector Calibration

Steerable Projector CalibrationProcams, 25 Jun 2005

www.mark.ashdown.name

Steerable projector modelSteerable projector model

• Internal projector parameters (8)• Projector pose (6)• Pose of mirror system (6)• Details of mirror system (3)

Page 8: Steerable Projector Calibration

Steerable Projector CalibrationProcams, 25 Jun 2005

www.mark.ashdown.name

Stages of the algorithmStages of the algorithm

• Camera• Projector• Pan-tilt mirror

– Projector pose– Tilt axis– Pan axis

• Refine result

Calibrate camera

Calibrate projector

Obtain reflected projector poses

Cluster projector poses

Find pose of tilt axis for fixed φ

Find pose of pan axis and thusthe full calibration

Pick best coarse result

Search for projector pose fromrandomized start position

Repeat around30 times

Repeat for eachφ value

Optionally repeatcoarse result

Iteratively refine the result

Page 9: Steerable Projector Calibration

Steerable Projector CalibrationProcams, 25 Jun 2005

www.mark.ashdown.name

Calibrating the cameraCalibrating the camera

• Use Matlab toolbox• Model radial and tangential distortion

Placing the board in various poses to calibrate the camera

Camera Calibration Toolbox for Matlab http://www.vision.caltech.edu/bouguetj/calib_doc/

ZY

ZX

y

x

/

/

xyyryrr

xrxyxrr

y

x

d

d

222

14

22

1

2221

42

21

2)2()1(

)2(2)1(

1100

0

0

22

11

d

d

y

x

cf

cf

Page 10: Steerable Projector Calibration

Steerable Projector CalibrationProcams, 25 Jun 2005

www.mark.ashdown.name

Calibrating the projectorCalibrating the projector

• Project pattern onto surface• Extract two images• Get camerasurface homography• Map projected points to surface• Do standard calibration• Reflecting the scene in the mirror does not affect calibration

Separating the image of the surface from the projected pattern

Placing the surface in different poses

Page 11: Steerable Projector Calibration

Steerable Projector CalibrationProcams, 25 Jun 2005

www.mark.ashdown.name

Stages of the algorithmStages of the algorithm

• Camera• Projector• Pan-tilt mirror

– Projector pose– Tilt axis– Pan axis

• Refine result

Calibrate camera

Calibrate projector

Obtain reflected projector poses

Cluster projector poses

Find pose of tilt axis for fixed φ

Find pose of pan axis and thusthe full calibration

Pick best coarse result

Search for projector pose fromrandomized start position

Repeat around30 times

Repeat for eachφ value

Optionally repeatcoarse result

Iteratively refine the result

Page 12: Steerable Projector Calibration

Steerable Projector CalibrationProcams, 25 Jun 2005

www.mark.ashdown.name

Locating the projectorLocating the projector

• Obtain projector pose from projector-to-surface homography

XtRKx

trrKHHxX ,, ,

1

,

1

0 21

Y

XY

X

2121 ,, rrrr

XtRKXnnnI

tRKx

10

22 dT

When mirror is used:

Page 13: Steerable Projector Calibration

Steerable Projector CalibrationProcams, 25 Jun 2005

www.mark.ashdown.name

Moving the mirrorMoving the mirror

• As the mirror moves many reflected positions are generated• Mirror has two degrees of freedom: pan and tilt (θ and φ)

Page 14: Steerable Projector Calibration

Steerable Projector CalibrationProcams, 25 Jun 2005

www.mark.ashdown.name

Find projectorFind projector

• We have data points each with a position and orientation• Generate temporary mirror half-way between real projector and

reflected projector• Iteratively minimize variance of reflected quaternions• Cluster like RANSAC

))((min perrqp

Page 15: Steerable Projector Calibration

Steerable Projector CalibrationProcams, 25 Jun 2005

www.mark.ashdown.name

Rotation using Rotation using θθ

• Fix φ, then projector and reflections lie in a plane• Assume mirror planes between projector and reflected positions• Define tilt axis by 3D point u• Use linear constraints to get u• Iteratively refine the solution

1td

i

ii

mn

um

Page 16: Steerable Projector Calibration

Steerable Projector CalibrationProcams, 25 Jun 2005

www.mark.ashdown.name

Rotation using Rotation using φφ

• Fit plane to previously calculated tilt-axis positions• Calculate offset angle β• Use linear constraints to get axis position v

122 ii ba

2

1

tv

v

cba y

x

iii

Page 17: Steerable Projector Calibration

Steerable Projector CalibrationProcams, 25 Jun 2005

www.mark.ashdown.name

Full solution is obtainedFull solution is obtained

• W matrix and α can be calculated from the positions of the two axes• Finally we obtain the 23 parameters

Page 18: Steerable Projector Calibration

Steerable Projector CalibrationProcams, 25 Jun 2005

www.mark.ashdown.name

Final iterative refinementFinal iterative refinement

• Refinement minimizes error in camera image• fminunc in Matlab

Page 19: Steerable Projector Calibration

Steerable Projector CalibrationProcams, 25 Jun 2005

www.mark.ashdown.name

PerformancePerformance

• Camera is 2000x1312 pixels• Projector is 1024x768• 30 degree pan, 10 tilt, 36 positions• Takes about 30 seconds for initial result and 2 minutes for

iterative refinement• Camera: 0.25 camera pixels• Projector: 0.47 projector pixels• Steerable projector: 9.4 camera pixels

Page 20: Steerable Projector Calibration

Steerable Projector CalibrationProcams, 25 Jun 2005

www.mark.ashdown.name

Future workFuture work

• Increasing accuracy• Completely automate the calibration• Finding pose of steerable projector• Combine with Raij & Pollefeys work• Calibrate focus and zoom settings of

projector

Open Source Computer Vision Library (OpenCV)http://www.intel.com/research/mrl/research/opencv/

Atienza and Zelinsky. A Practical Zoom Camera

Calibration Technique: An Application on Active Vision

for Human-Robot Interaction. Proc. Australian Conf.

Robotics and Automation

Page 21: Steerable Projector Calibration

Steerable Projector CalibrationProcams, 25 Jun 2005

www.mark.ashdown.name

Thanks for listening.

Questions?


Recommended