Visual Detection of Lintel-Occluded Doors from a Single Image

Zhichao Chen and Stan Birchfield

Dept. of Electrical and Computer EngineeringClemson University

Clemson, South Carolina USA

Types of MapsTopological mapMetric map

Either way, doors are semantically meaningful

landmarks

Previous Approaches to Detecting Doors

• Range-based approaches– sonar [Stoeter et al.1995] – stereo [Kim et al. 1994]– laser [Anguelov et al. 2004]

• Vision-only (uncalibrated)– low cost– low power– non-contact (passive) measurement– rich capturing ability

Vision-Based Door Detection

fuzzy logic [Munoz-Salinas et al. 2004]

color segmentation [Rous et al. 2005]

neural network [Cicirelli et al 2003]

Limitations: • require different colors for doors and walls• simplified environment (untextured floor, no reflections) • limited viewing angle• high computational load • assume lintel visible

Problem Statement Detect doors in complex environments: • textured and untextured floors• walls and doors with similar colors• specular reflections• variable lighting conditions• wide range of robot poses

lintel

post

Another ChallengeLintel-occluded post-and-lintel architecture camera is low to ground cannot point upward b/c obstacles

Video

Our Approach

color

texture

vertical lines

concavity

door gap

vanishing point

kick plate

Adaboost

Standard features

Novel features

Door detected

Assumptions:• Both door posts are visible• Posts appear nearly vertical• The door is at least a certain width

,)(sign)(N

1nn

xhx n

Pairs of Vertical Lines

1. Edges detected by Canny2. Line segments detected by modified Douglas-Peucker algorithm3. Clean up (merge lines across small gaps, discard short lines)4. Separate vertical and non-vertical lines5. Door candidates given by all the vertical line pairs whose spacing

is within a given range

Canny edges detected lines

vertical lines

non-vertical lines

Cue #1: Color

N

i door

N

i doorwall

i

ii

1

1

][

][ ],[min

1. Threshold the histogram intersection between the wall color model φwall and the color histogram φdoor computed between two vertical line segments.

2. Wall color model can be built either by hand, or automatically.

positive

negative

• The bottom part of the door is usually untextured.

• Texture energy is computed by summing the magnitude of the gradient in the lower region of the door.

Cue #2: Texture

Ap

pIA

)(||

1

positive negative

Cue #3: Gap Below the DoorIntensity along the line

darker (light off)

positive

negative

brighter (light on)

no gap

Cue #4: Kick Plate

A region R in the segmented image is considered as kick plate if:

1. the region R is located between two vertical lines2. the bottom of R is near the bottom of the two vertical lines3. the width and height of R are within a specified range

R

• Kick plates occur in 30% of images• Segmentation algorithm of Felzenszwalb et al., IJCV 2004 is

used (15 fps on 160x120 downsampled image)

positive negative

Cue #5: Vanishing Point• The vanishing point is computed as the mean of the intersection

of pairs of non-vertical lines.

The bottom line of a door should intersect the vanishing point

A distracting line caused by shadowsdoes not intersect the vanishing point

j

j

j

jii

i

i

y

x

cba

cba

wwvwv

,

• If the bottom door edge passes near the vanishing point, then the test succeeds.

Cue #6: Concavity

wall walldoor

Slim “U”

intersection line of wall and floor

extension of intersection line

bottom door edge

vertical door lines

ε

Lleft

LRight

floor

Detect Concavity

Concavity is declared if at least two of the three tests (Hrec(L), Hrec(R), and HU) succeed.

Slim “U”

Extended line

positive negative

U"" slim Find

),( ),( maxmin

U

rec

HRLH

Douglas-Peucker Algorithm

Canny edges Edge labeling Line segmenting

dallowed

Modified Douglas-Peucker Algorithm

Note: important for concavity cue

original algorithm: dallowed is constantmodified algorithm: dallowed is given by half-sigmoid function

dallowed

original modified

• The strong classifier is given by a weighted sum of the weak classifiers:

where weight , error

Adaboost• Bayes decision rule: declare a door if the a posteriori

probability of the predicate is greater than that of its complement:

where )|()|(ψ

1

IDhIDn

iii

,1)|(ψexp)|(

)|(

ID

IDPIDP

)(sign)(ψ1

xhxN

nnn

n

nn

1ln21

iin

niyxhiniinDrn iDyxhP

)(:~ )()(

Experimental Results: Similar or Different Door/Wall Color

Experimental Results: High Reflection / Textured Floors

Experimental Results: Different Viewpoints

Experimental Results: Cluttered Environments

Results

20 different buildings 309 images:• 100 training• 209 testing

90% accuracy with 0.05 FP per image

Speed: 5 fps (unoptimized)

False Negatives and Positives

distracting reflectionconcavity and bottom gap tests fail

strong reflection

concavity erroneously detected

two vertical lines unavailable

distracting reflection

Navigation in a Corridor• Doors were detected and tracked from frame to frame.• Fasle positives are discarded if doors were not

repeatedly detected.

Video

Conclusion• Conclusion

– Detect doors using a single uncalibrated camera in a variety of environments

– Augment standard features (color, texture, and vertical edges) with novel features (concavity, door gap, kick plate, and vanishing point)

– The features are combined in an Adaboost framework– Suitable for real-time mobile robot applications using an off-

the-shelf camera• Future work

– on-line learning of hall color– building of a geometric map– detecting open doors