Depth from Diffusion
Supported by ONR
Changyin Zhou Shree NayarOliver Cossairt
Columbia University
Optical Diffuser
Optical Diffuser
Micrograph of a Holographic Diffuser(RPC Photonics)
~ 10 micron
[Gray, 1978] [Chang et al., 2006] [Garcia-Guerrero et al. 2007]
Diffusers as Accessories
Diffuser to preview the image
(B&H)Diffusers to soften the image
Diffusers for illumination(B&H)
CameraDiffuser
Object
CameraDiffuser
Object
The amount of diffusion varies with depth.
Diffusion Encodes Depth
Geometry of Diffusion: A Pinhole Camera
Sensor
Pinhole
Object
P
Q
Miss
Object
P
Geometry of Diffusion: A Pinhole Camera
Sensor
θ
Pinhole
Diffuser
Geometry of Diffusion: A Pinhole Camera
Pinhole
θ
θ
Sensor
Diffuser
Object
PA
B
Geometry of Diffusion: A Pinhole Camera
O
V U Z
θ
θPinhole
Diffusion Law:
Sensor
A
B
Object
2r
P
ObjectDiffuser
Geometry of Diffusion: A Pinhole Camera
O
V U Z
Pinhole
Sensor
A
B
Object
2r
P
Diffusion Size and Depth:
ObjectDiffuser
θ
θ
Geometry of Diffusion: A Pinhole Camera
O
V U
Pinhole
SensorObject
2r
Diffuser as a proxy object
Diffusion Size and Depth:
PZ
Diffuser
Diffusion as Convolution: A Pinhole Camera
Latent clear image
Captured Image
Diffusion Size
Diffusion PSF
Assume field angle and depth are constant for small Assume field angle and depth are constant for small
image patches, we have:image patches, we have:
Geometry of Diffusion: A Lens Camera
Geometry of Diffusion: A Lens Camera
O
V U
Pinhole
SensorObject
2r
Diffuser as a proxy object
PZ
Diffuser
Geometry of Diffusion: A Lens Camera
V USensorObject
2r
Diffuser as a proxy object
PZ
Diffuser
Lens
The captured image can be
further blurred due to defocus.
Diffusion as Convolution: A Lens Camera
The Final PSF
Defocus PSF
Diffusion PSF
For a lens camera with a diffuser, we have:
is the diffusion PSF if a pinhole were used.
is the defocus PSF if the diffuser were removed.
Depth from Diffusion (DFDiff) Algorithm
Same form as in DFD
2. Estimate
Blur Size r
3. Compute
Depth Z
1. Capture
Two Images
With a diffuser Without a diffuser
Depth from Diffusion vs. Depth from Defocus
[Pentland, 1987] [Subbarao, 1988] [Watanabe & Nayar, 1996]
[Chaudhuri & Rajagopalan, 1999] [Favaro & Soatto, 2005] [Schechner & Kiryati, 2000]
Depth from Defocus
Aperture pattern
P
Lens Focal PlaneSensor
Zr
Depth from Diffusion
Diffusion pattern
Sensor
Pinhole
θ
θ
DiffuserZ
P
r
Depth from Diffusion vs. Depth from Defocus
Depth from Diffusion
P
Any lens is fine!
Suppose 22.5x15mm Sensor, 10 um pixel, 100 mm EFL
Object distance = 1000 mm
ObjectA Diffuser of 21.8
o
Depth precision is about 0.1 mm.
Field of View
Depth from Diffusion vs. Depth from Defocus
Depth from Defocus
P
Lens
Aperture diameter
?
Suppose 22.5x15mm Sensor, 10 um pixel, 100 mm EFL
Depth precision is about 0.1 mm.
Object distance = 1000 mm
Object
Field of View
Depth from Diffusion vs. Depth from Defocus
Depth from Defocus
Object distance = 1000 mm
Suppose 22.5x15mm Sensor, 10 um pixel, 100 mm EFL
Depth precision is about 0.1 mm.
Aperture diameter
800 mm
Object
P
Depth from Diffusion vs. Depth from Defocus
Depth from DiffusionSuppose 22.5x15mm Sensor, 10 um pixel, 100 mm EFL
P
Depth precision is about 1.0 mm.
Object distance = 5000 mm
ObjectAny lens is fine!A Diffuser of 11.2
o
Depth from Diffusion vs. Depth from Defocus
Depth from Defocus
P
Lens
Aperture diameter
?
Suppose 22.5x15mm Sensor, 10 um pixel, 100 mm EFL
Depth precision is about 1.0 mm.
Object distance = 5000 mm
Object
Depth from Diffusion vs. Depth from Defocus
Depth from DefocusSuppose 22.5x15mm Sensor, 10 um pixel, 100 mm EFL
Depth precision is about 1.0 mm.
Aperture diameter
2000 mmP
Object distance = 5000 mm
Object
PSF Measurement: A Pinhole Camera
F/22 , Field Angle = 0 o
Z = 2 mm Z = 5 mm
Captured Modeled
- Canon EOS T1i; EF 50mm F/1.8 Lens;
- Luminit Holographic Diffuser (10o Gaussian);
- Diffuser distance: U = 1m
Z = 2 mm Z = 5 mm
PSF Measurement: A Pinhole Camera
Captured Modeled
F/22 , Field Angle = 10 o
- Canon EOS T1i; EF 50mm F/1.8 Lens;
- Luminit Holographic Diffuser (10o Gaussian);
- Diffuser distance: U = 1m
PSF Measurement: A Lens Camera
F/1.8 , Field Angle = 10 o
Captured Modeled
- Canon EOS T1i; EF 50mm F/1.8 Lens;
- Luminit Holographic Diffuser (10o Gaussian);
- Diffuser distance: U = 1m
Experiments
Five playing cards,
0.29mm thick each
Canon 20D + 50mm Lens
Luminit Diffuser (20o)
Experiments
Captured WITHOUT a Diffuser Captured WITH a Diffuser
Experiments
Computed Depth Map
(~ 0.1 mm precision)
(mm)Five playing cards,
0.29mm thick each
Experiments
A small sculpture of
about 4mm thickness
Canon G5 Compact Camera
Luminit Diffuser (5o)
Experiments
Captured WITHOUT a Diffuser Captured WITH a Diffuser
Experiments
Computed Depth Map
A 3D View of Depth Map
A small sculpture of
about 4mm thickness
Experiments
Canon 20D; Gaussian Diffuser (10o)
450
mm
650 mm
Experiments
Stitched Depth Map
(precision)
(mm)
Summary
Formulated the image formation Formulated the image formation with optical diffuserswith optical diffusers
Proposed Depth from DiffusionProposed Depth from Diffusion- Require a diffuser on the object side - Require a diffuser on the object side
+ High-precision depth estimation+ High-precision depth estimation
+ Distant objects+ Distant objects
+ Less sensitive to lens aberrations+ Less sensitive to lens aberrations
Demonstrated high-precision Demonstrated high-precision depth estimationdepth estimation
CameraDiffuser
Object
Depth from Diffusion
Supported by ONR
Changyin Zhou Shree NayarOliver Cossairt
Columbia University