Depth Enhancement via Low-rank Matrix Completion Si Lu1, Xiaofeng Ren2, and Feng Liu1
Department of Computer Science, Portland State University1 Department of Computer Science and Engineering, University of Washington2
Depth maps captured by consumer RGB-D cameras are often noisy and miss values at some pixels. This paper presents a depth enhancement algorithm via low rank matrix completion that performs depth map completion and de-noising simultaneously.
SUMMARY
EXPERIMENTS
This work was supported in part by NSF grants IIS-1321119, CNS-1205746, and CNS-1218589.
FRAMEWORK
OBSERVATIONS: Similar RGB-D patches
approximately lie in a low-dimensional subspace.
The subspace constraint essentially captures the potentially scene-dependent image structures in the RGB-D patches in both the color and depth domain.
This low-rank subspace constraint can be enforced through incomplete matrix factorization.
Patch samples. (a): clean patch. (b): noisy patch. (c): the top ten eigen-vectors of the patch matrix formed by similar patches to each noisy patch.
Similar patch searching Rank prediction for patch matrix
Enhancement via low-rank matrix completion
Output color Output depth
Input color Input depth
Output color
Output depth
Input color Input depth
Output color
Output depth
Input color Input depth
Output color
Output depth
Features capturing
patch structure properties
Predicted rank
Training data
Noisy color Color edge
Depth edgeNoisy depth
(a) Ground truth (b) Noisy RGB-D image BM3D Our method
(c) Color denoising
(d) Joint bilateral filter (e) BM3D + joint bilateral filter
(f) Our depth enhancement result
Rank distribution of 30,000 RGB-D patch matrices
(a) (b) (c) (a) (b) (c) (a) (b) (c)
�̂�=𝐴𝐵
Regression model
Comparisons among depth enhancement methods.
patch matrix M matrix rank r