January 29, 2007 Videometrics IX, Electronic Imaging 2007, San Jose, CA, U.S.A.

Real-Time Range Imaging by Phase-Stamp Method

Using Correlation Image Sensor

Akira Kimachi1 and Shigeru Ando2

1Osaka Electro-Communication University, Japan 2The University of Tokyo, Japan

Active Range Imaging• Active range imaging — Image sensor + Active light

– Time-of-flight methods– Triangulation methods

# of frames (scans)

Interpixel image processing

Image intensity-dependence

Light spot Many None Yes

Light stripe Many Stripe detection Yes

Time stamp light stripe

1 None Yes

Structured light 1 Fringe detection

Fringe order

Yes

Phase-shifting structured light

3-4 or more None No

DepthScan angle of sheet beam ijStripe image location ),( ji

Light Stripe Range Finder (LSRF)

Time Stamp LSRF (TSRF)

• Frame-rate operation• Pixel-wise sensing

– One image for one beam scan– Record a “Time Stamp” for lig

ht stripe arrival at each pixel

• “VLSI silicon range finder”– Gruss et al. (1991)– Kang et al. (1994)– Yokoyama et al. (1994)

ijzdepth

ij

beamangle

ijt

timestamp

TSRF

threshold

threshold

LSRF

Effect of Spatially Nonuniform Reflectance/Background

• Intensity-dependent detection of light stripe causes errors in LSRF and TSRF

nonuniformsurface reflectance

nonuniformbackground illumination

nonuniformsurface reflectance

nonuniformbackground illumination

Objectives

• Goal — Real-time active range imaging– Frame-rate operation– Pixel-wise sensing (image processing error-free)– Robustness to spatial nonuniformity in

• Surface reflectance• Background illumination

• Solution — Phase-Stamp Range Finder (PSRF)– Three-Phase Correlation Image Sensor (3PCIS)– Detect the sheet beam angle by “phase stamp”

Three-Phase Time-Domain Correlation Image Sensor (3PCIS)

temporalcorrelation

S. Ando and A. Kimachi,IEEE Trans. ED (2003)

averageintensity

Phase-Stamp Range Finder (PSRF)

• Three-phasereference signals

0f : Frame rate

),( ji

: Object surface reflectance

: Sheet beam intensity

: Background light intensity

ijR

ijI

)(tI bgij

: Arrival time of light stripeijt

• Image intensity at

Phase-Stamp Imaging(1) Sheet beam arrives

(2) Phase stamped

(3) Converted to beam angleone scan

one image

Real-Time Range Imaging by PSRF

3PCIS output

Phase stamp

Total light image

Sheet beam image

Range image

Beam angle imageknown functionby design

• Frame-rate range imaging• Pixel-wise beam angle detection• Robust to spatially nonuniform reflectance/background

Phase stamp imageuncorrelated to

)(tgk

3PCIS Output Images in PSRF

• Sheet beam-only intensity image

• Total light intensity image

: Surface reflectance

: Sheet beam intensity

: Background intensity

ijR

ijI

)(tI bgij

0

/2

/2

• Phase stamp image

Experimental PSRF System• 3PCIS camera

– 200x200 pixels– Frame rate @12.5 fps

• Sheet beam– 7 mW laser diode– Cylindrical lens– Scanning mirror

@25 scans/s

• Reference signals– PC ＋ D/A board– @50 Hz

• Camera and geometry calibration– Based on Zhang’s meth

od

Referencesignals

Sheet beamangle

Real-Time 3D Video Capture

Phase stampTotal light intensity Sheet beam intensity

3D Imaging Examples

left thumb & palm

cone-shaped surface

23 mm

17 mm

40 mm40 mm

3D Measurement Accuracy• Flat object @540 mm from the camera

phase stamp total lightsheet beam

Phase Std. dev. 4.08 deg

DepthStd. dev. 2.35 mm

Syst. error -1.96 mm

0

/2

/2

3D surface plot

Nonuniform Background Illumination

Background absent striped

Std. dev. [mm] 2.35 3.11

Syst. err. [mm]

-1.96 -1.85

phase stamptotal light sheet beam

Nonuniform Surface Reflectance

Reflectance uniform striped

Std. dev. [mm] 2.35 2.39

Syst. err. [mm]

-1.96 -1.62

phase stamptotal light sheet beam

Conclusions

• Phase-stamp range finder using the 3PCIS– Frame-rate operation– Pixel-wise sensing– Robustness to spatial nonuniformity in

• Surface reflectance• Background illumination

• Evaluation by an experimental system– Real-time range imaging @12.5 fps– Accuracy (@540mm)

• Deviation — 2.35 mm → fixed pattern noise (~4 deg) removal

• Systematic error — -1.96 mm → geometry calibration