A Bayesian Approach For 3D Reconstruction From a Single Image Presented By: Erick Delage Supervisor:...

transcript

A Bayesian Approach A Bayesian Approach For 3D For 3D

Reconstruction From Reconstruction From a Single Imagea Single Image

Presented By: Erick DelagePresented By: Erick Delage

Supervisor: Prof. Andrew Y. NgSupervisor: Prof. Andrew Y. NgAI Laboratory, Stanford UniversityAI Laboratory, Stanford University

Erick Delage, Stanford University, 2Erick Delage, Stanford University, 2005005

Can a robot reconstruct 3D from a single image?

Autonomous Monocular Vision Autonomous Monocular Vision Depth Reconstruction for Indoor Depth Reconstruction for Indoor

ImageImage

Review of PublicationsReview of PublicationsPopular 3d reconstruction

Stereo Vision (Trucco et Verri, 1998) Structure from Motion

Single View 3d reconstruction

Shape from Shading (Zhang et al., 1999) 3d Metrology (Criminisi et al., 2000)

Our Goal

To develop an autonomous algorithm that recovers 3D information from a single image in a complex environment

Simplification of the ProblemSimplification of the Problem

Assumptions

1. Image contains flat floor and walls2. Camera is parallel to the ground plane3. The camera is at a known height above the ground4. The image is obtained by perspective projection

Our Theory:

Given the floor boundary position, the 3D coordinates in an image of all points can be recovered

General ApproachGeneral Approach

floorB

General Approach (2)General Approach (2)

Prior Knowledge about Indoor + Machine Learning

Image Analysis Floor Boundary detection (Machine Learning) 3D reconstruction

Input Image

Difference from the floor color

Floor Boundary DetectionFloor Boundary DetectionMagnitude of Image gradient

Difference in chromatic space

How can we combine these image features for floor boundary detection ?

Input Image

Training Mask

)),(()),(|),(( jisigmoidjiedgejiP T

Using Logistic Regression :

(Martin, D. R., et al., 2002)

The model was trained using 25 labeled images of a diverse range of indoor environments on Stanford’s campus

xexsigmoid

Floor Boundary DetectionFloor Boundary Detection

Floor Boundary Detection : Floor Boundary Detection : ResultsResults

Precision = “true positives” / “all positives”

Recall = “true positives” / “all true’s”

Bayesian Inference on FloorBayesian Inference on Floor BoundaryBoundary

Can we use prior knowledge about the structure of floors and their boundaries?

Di : Direction of the floor boundary in column i

Yi : Position of floor boundary in column i

Xi : Local image features

C : Color of the floor

Bayesian InferenceBayesian Inference

: initial distribution of variables)(),(),( 11 CPYPDP

11 ,)|( iiii DDDDP verticalDi ,

1,1 ii DD

verticalDYN ii 121 ,),( verticalDDYNDYYP iiiiii 111111 ,),(),|(

),|(),|( CXYPCYXP iiii from the detection algorithm

),,(argmax),,( :1:1,,

:1:1:1:1

CDXPCDX NNCDX

Training / Bayesian InferenceTraining / Bayesian Inference

60 images of indoor environment in 8 different buildings of Stanford’s campus

Leave-one-out cross-validation: train on 7 buildings, test on 1

Parameters for density models estimated from training data using Maximum Likelihood

Exact inference on graph done using Viterbi-like algorithm

Results – Floor Boundary Results – Floor Boundary DetectionDetection

3D Reconstruction3D Reconstruction

Extra Material:

Exemples #1, #2, #3

Or at:

http://www.stanford.edu/~edelage/indoor3drecon

PerformancePerformance

Precision of floor boundary in segmentation

Precision of floor boundary in 3d localization

ConclusionConclusion

Monocular 3d reconstruction is a good example of an Monocular 3d reconstruction is a good example of an ambiguous problem that can be resolved using prior ambiguous problem that can be resolved using prior knowledge about the domainknowledge about the domain

The presented Bayesian network proves high efficiency The presented Bayesian network proves high efficiency in learning prior knowledge necessary for this in learning prior knowledge necessary for this applicationapplication

This is the first autonomous algorithm for depth recovery This is the first autonomous algorithm for depth recovery in a rich, textured indoor scene.in a rich, textured indoor scene.

Future WorkFuture Work

Apply graphical modeling for more complex geometry.Apply graphical modeling for more complex geometry. Formulate the problem in a form that scales precision Formulate the problem in a form that scales precision

performance with depth of objects.performance with depth of objects. Embed this approach in real robot navigation problem Embed this approach in real robot navigation problem

(ex. RC car, night indoor navigation)(ex. RC car, night indoor navigation)

Questions ?Questions ?

ReferencesReferencesCriminisi, A., Reid, I., & Zisserman, A. (2000). Single View Metrology. IJCV, 40, 123-148.

Martin, D. R., Fowlkes, C. C., & Malik, J. (2002). Learning to Detect Natural Image Boundaries using Brightness and Texture. NIPS.

Trucco, E., & Verri, A. (1998). Introductory techniques for 3d computer vision. Prentice Hall.

Zhang, R., Tsai, P.-S., Cryer, J. E., & M. Shah (1999). Shape from shading: a survey. IEEE Trans. On PAMI, 21, 690-706.

A Bayesian Approach For 3D Reconstruction From a Single Image Presented By: Erick Delage Supervisor:...

Documents