Date post: | 14-Dec-2015 |
Category: |
Documents |
Upload: | bathsheba-goodwin |
View: | 218 times |
Download: | 1 times |
The 18th Meeting on Image Recognition and Understanding2015/7/29
Depth Image Enhancement
Using Local Tangent Plane Approximations
Kiyoshi Matsuo Yoshimitsu AokiHokuyo Automatic Co., Ltd. Keio University
○
The 18th Meeting on Image Recognition and Understanding2015/7/29
Unique Product & Best SensingHOKUYO
• Background– Research purpose– Related work
• Proposed method (overview and highlight)– Local coordinates based depth image enhancement
• Experimental results
• Conclusion
2
Contents
The 18th Meeting on Image Recognition and Understanding2015/7/29
Unique Product & Best SensingHOKUYO
Depth image enhancement for consumer RGB-D cameras
3
Research purpose
Ex. Soft Kinetic DS311
> Designed for human interaction applications
* Consumer RGB-D camera
+ low cost+ portable
- low accuracy- low resolution
Enhancement of both the accuracy and resolution could help many applications.
Ex. fine-grained object recognition, precise indoor navigation, and etc…
The 18th Meeting on Image Recognition and Understanding2015/7/29
Unique Product & Best SensingHOKUYO
4
Related work
“An application of Markov random fields to range sensing” [ NIPS 2006 Diebel et al. ]“High quality depth map upsampling for 3d-tof cameras” [ ICCV 2011 Park et al. ]“A Joint intensity and depth co-sparse analysis model for depth map surperresolution” [ ICCV 2013 Kiechle et al. ]“Depth map inpainting under a second-order smoothness prior” [ Image Analysis vol. 7944 Herrera et al. ] etc.
1. Measured depth data is used as optimization data prior.2. Various pixel based information is used as optimization smoothness prior. (gradient, segmentation, edge saliency, non-local mean, and co-sparseness etc.)
3. Depth enhancement is achieved by an optimization of over the image plane.
[ Diebel et al. 2006 ]
Global optimization based enhancement Markov Random Field
“Joint bilateral upsampling” [ ACM transactions on Graphics 2007 Kopf et al. ]“Spatial-depth super resolution for range images” [ CVPR 2007 Yang et al. ]“Pixel weighted average strategy for depth sensor data fusion” [ ICIP 2010 Garcia et al. ]“Upsampling range data in dynamic environments” [ CVPR 2010 Dolson et al. ]“Joint geodesic upsampling of depth images” [ CVPR 2013 Liu et al. ] etc.
Local filter based enhancement1. Local measured depth data is summarized by using similarity weights.2. Similarity weights are defined on the global image plane coordinates. (pixel-distance, color-difference, gradient, and color geodesic distance, etc.)
3. Depth enhancement is achieved by local calculations on the image plane.
Joint bilateral upsampling filter
[ Kopf et al. 2007 ]
The 18th Meeting on Image Recognition and Understanding2015/7/29
Unique Product & Best SensingHOKUYO
5
Related work
Previous enhancement methods use pixel-coordinates. These methods smooth the surfaces to be parallel to the image plane.
Measuring object
Image plane
No geometric relationship
The geometries of surfaces are not take into account,(and are sometimes corrupted by them)
The 18th Meeting on Image Recognition and Understanding2015/7/29
Unique Product & Best SensingHOKUYO
6
Contributions
To improve accuracy, we introduce local tangent planes as local coordinates to handle the geometries.
Local coordinatesGlobal coordinates
* pixel-coordinates of the image * charts defined on each local tangent * depend only on the measuring device (independent of the measuring surfaces)
* depend on the local geometries of measuring surfacesAccurate depth image enhancement is achieved.
(especially in noisy cases)
The 18th Meeting on Image Recognition and Understanding2015/7/29
Unique Product & Best SensingHOKUYO
7
Proposed method (overview)
A pair of depth and color images
* Input data
2. Surface reconstruction * Output data
A high-resolution accurate depth image
Only overview of our method(more details will be shown at our poster.)
1. Estimation of local tangent planes
Proposed in ICCVW 2013
The 18th Meeting on Image Recognition and Understanding2015/7/29
Unique Product & Best SensingHOKUYO
8
How to use local coordinates ?
Measured surface
Height smoothed surface
To recover smooth surfaces, we smooth normal components of surfaces on each local coordinates
Colored by the height fromthe tangent plane
Smoothed surface
These filters on local coordinates smooth the surfaces while preserving the local geometries of local tangent planes.
The 18th Meeting on Image Recognition and Understanding2015/7/29
Unique Product & Best SensingHOKUYO
The interior regions of surfaces were reconstructed more accurately. 9
Experimental results
MRF [Diebel and Thrun]
PWAS [Garcia et al]
Previous ours [Matsuo and Aoki] Our method
Source RGB-D image
Captured by SoftKinetic DS311
The 18th Meeting on Image Recognition and Understanding2015/7/29
Unique Product & Best SensingHOKUYO
10
Conclusion
• Local coordinate based depth image enhancement.– Local tangent planes are used as local coordinates.
• Normal component smoothing reduces granular noises.– These filters preserve the local geometries of local tangents.
• The effectiveness is shown by real sensor data.
The 18th Meeting on Image Recognition and Understanding2015/7/29
Thank you for this opportunity to present to you today.And thank you for your attention !
If you have any questions and comments,visit our poster IS 2-4.