Date post: | 17-Dec-2015 |
Category: |
Documents |
Upload: | reynard-powers |
View: | 215 times |
Download: | 0 times |
Segmentation of Floor in Corridor Images for Mobile Robot Navigation
Yinxiao Li
Clemson University
Committee Members:
Dr. Stanley Birchfield (Chair)
Dr. Adam Hoover
Dr. John Gowdy
MotivationGoal: Segment the floor in a single corridor image
Why?• obstacle avoidance• mapping• autonomous exploration and navigation
Motivation
Why?• obstacle avoidance• mapping• autonomous exploration and navigation
Goal: Segment the floor in a single corridor image
Outline
• Previous Work
• Detecting Line Segments
• Score Model for Evaluating Line Segments
– Structure Score
– Bottom Score
– Homogeneous Score
• Experimental Results
• Extension
Outline
• Previous Work
• Detecting Line Segments
• Score Model for Evaluating Line Segments
– Structure Score
– Bottom Score
– Homogeneous Score
• Experimental Results
• Extension
Previous WorkFree space detection
• Combination of color and histogram (Lorigo 1997)
• Optical flow (Stoffler 2000, Santos-Victor 1995)
• Stereo matching (Sabe 2004)
Floor detection• Apply planar homographies to
optical flow vectors (Kim 2009, Zhou 2006)
• Stereo homographies (Fazl-Ersi 2009)
• Geometric reasoning (Lee 2009)
Limitations• Multiple images for motion• Multiple cameras for stereo• Require different colors for floor and wall• Computational efficiency• Calibrated cameras • Assume ceiling visible
Our contribution: Segment the floor in real time using a single image captured by a low-height mobile robot
Challenging problem: Reflections
Also:• variety of poses (vanishing point, ceiling not always visible)• sometimes wall and floor color are nearly the same
Douglas-Peucker Algorithm
Purpose: Reduce the number of points in a curve (polyline)
Algorithm:
1. Connect farthest endpoints2. Repeat
1. Find maximum distance between the original curve and the simplified curve
2. Split curve at this point Until max distance is less than
threshold
David Douglas & Thomas Peucker, "Algorithms for the reduction of the number of points required to represent a digitized line or its caricature", The Canadian Cartographer 10(2), 112–122 (1973)
Douglas-Peucker Algorithm
Purpose: Reduce the number of points in a curve (polyline)
Algorithm:
1. Connect farthest endpoints2. Repeat
1. Find maximum distance between the original curve and the simplified curve
2. Split curve at this point Until max distance is less than
threshold
Douglas-Peucker Algorithm
Purpose: Reduce the number of points in a curve (polyline)
Algorithm:
1. Connect farthest endpoints2. Repeat
1. Find maximum distance between the original curve and the simplified curve
2. Split curve at this point Until max distance is less than
threshold
Douglas-Peucker Algorithm
Purpose: Reduce the number of points in a curve (polyline)
Algorithm:
1. Connect farthest endpoints2. Repeat
1. Find maximum distance between the original curve and the simplified curve
2. Split curve at this point Until max distance is less than
threshold
Douglas-Peucker Algorithm
Purpose: Reduce the number of points in a curve (polyline)
Algorithm:
1. Connect farthest endpoints2. Repeat
1. Find maximum distance between the original curve and the simplified curve
2. Split curve at this point Until max distance is less than
threshold
Douglas-Peucker Algorithm
Purpose: Reduce the number of points in a curve (polyline)
Algorithm:
1. Connect farthest endpoints2. Repeat
1. Find maximum distance between the original curve and the simplified curve
2. Split curve at this point Until max distance is less than
threshold
Detecting Line Segments (LS)
Detecting Line Segments (LS)
1. Compute Canny edges
Detecting Line Segments (LS)
1. Compute Canny edges
2. Douglas-Peucker algorithm to detect line segments
Vertical LS: slope within ±5° of vertical direction Horizontal LS: Slope within ±45° of horizontal direction
Detecting Line Segments (LS)
1. Compute Canny edges
2. Douglas-Peucker algorithm to detect line segments
Vertical LS: slope within ±5° of vertical direction Horizontal LS: Slope within ±45° of horizontal direction
3. Pruning line segments (320x240) Vertical LS: minimum length 60 pixels Horizontal LS: minimum length 15 pixels Vanishing point (height selection): The vanishing point is computed as the mean of the intersection of pairs of non-vertical lines. It is used for throwing away horizontal line segments coming from windows, ceiling lights, etc.
Outline
• Previous Work
• Detecting Line Segments
• Score Model for Evaluating Line Segments
– Structure Score
– Bottom Score
– Homogeneous Score
• Experimental Results
• Extension
Score Model
is assigned to each horizontal line
Structure Score Bottom Score Homogeneous Score
Weightshorizontal line
Score Model – Structure
Given a typical corridor image
1. |I(x,y)| > T1 (gradient magnitude)
(How to set threshold T1?)
|I(x,y)| > T1
Score Model – Structure
Given a typical corridor image
1. |I(x,y)| > T1 (gradient magnitude)
(How to set threshold T1?)
2. I(x,y) < T2 (image intensity)
(How to set threshold T2?)
|I(x,y)| > T1 and I(x,y) < T2
Score Model – Structure
Given a typical corridor image
1. |I(x,y)| > T1 (gradient magnitude)
(How to set threshold T1?)
2. I(x,y) < T2 (image intensity)
(How to set threshold T2?)
We applied SVM to 800 points in 200 images:
|I(x,y)| > T1 and I(x,y) < T2
Score Model – Structure
Given a typical corridor image
1. |I(x,y)| > T1 (gradient magnitude)
(How to set threshold T1?)
2. I(x,y) < T2 (image intensity)
(How to set threshold T2?)
Approximate using two thresholds:
|I(x,y)| > T1 and I(x,y) < T2 T2
T1
Score Model – Structure
Given a typical corridor image
1. |I(x,y)| > T1 (gradient magnitude)
2. I(x,y) < T2 (image intensity)
3. Now threshold original image using TLC (average gray level of pixels satisfying #1 and #2)
I(x,y) < TLC
Score Model – Structure
Given a typical corridor image
1. |I(x,y)| > T1 (gradient magnitude)
2. I(x,y) < T2 (image intensity)
3. Now threshold original image using TLC (average gray level of pixels satisfying #1 and #2)
4. Compute the chamfer distance between the line segments and structure blocks
Score Model – Structure
Ridler-Calvard Otsu
Our method
Our method is able to remove the spurious pixels on the floor caused by reflections or shadows
Method R-C Otsu Ours
Correctness 62% 66% 82%
Comparison of different threshold methods
Score Model – Bottom
1. Connect the bottom points of consecutive vertical line segments to create “bottom” wall-floor boundary
Score Model – Bottom
1. Connect the bottom points of consecutive vertical line segments to create “bottom” wall-floor boundary
Score Model – Bottom
1. Connect the bottom points of consecutive vertical line segments to create “bottom” wall-floor boundary
2. Compute the distance of each horizontal line segment to the “bottom” wall-floor boundary
(red circled horizontal line segments are more likely on the real wall-floor boundary)
Score Model – HomogeneousIdea: The floor tends to have larger
regions (due to decorations, posters, windows on the wall).
Algorithm:
1. Color-based segmentation of the image (using Felzenszwalb-Huttenlocher’s minimum spanning tree algorithm)
2. For each horizontal line segment, compute size of region
just belowsegment
size of largestsegment
Score Model – HomogeneousIdea: The floor tends to have larger
regions (due to decorations, posters, windows on the wall).
Algorithm:
1. Color-based segmentation of the image (using Felzenszwalb-Huttenlocher’s minimum spanning tree algorithm)
2. For each horizontal line segment, compute size of region
just belowsegment
size of largestsegment
Segmenting the floor
• Normalize the scores and sum• Threshold the final score:
• Connect the remaining line segments and extend the endpoints to the edges of the image
Evaluation Criterion• Wall- floor Boundary
– Green line: Detected wall-floor boundary
– Red line: Ground truth
Evaluation Criterion• Wall- floor Boundary
– Green line: Detected wall-floor boundary
– Red line: Ground truth
• Area– Blue shade area: misclassified area
– Red shade area: ground truth floor area
• Error Rate
• Segmentation is considered successful if rerr < 10%
%err
bluer
red
Outline
• Previous Work
• Detecting Line Segments
• Score Model for Evaluating Line Segments
– Structure Score
– Bottom Score
– Homogeneous Score
• Experimental Results
• Extension
Sample Results89.1% success on database of 426 images:
Sample ResultsImages downloaded from the internet:
Sample ResultsSome successful results on failure examples from Lee et al. 2009
Original Image
Lee’s Ours
D. C. Lee, M. Hebert, and T. Kanade. “Geometric Reasoning for Single Image Structure Recovery”. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009.
Sample Results• Failure examples
Checkered floor
Textured wall
Darkimage Bright
lights
Sample Videos
• Video clip 1
Sample Videos
• Video clip 2
Outline
• Previous Work
• Detecting Line Segments
• Score Model for Evaluating Line Segments
– Structure Score
– Bottom Score
– Homogeneous Score
• Experimental Results
• Extension
Floor Segmentation Algorithm on Low-Res Images
• Apply floor-segmentation algorithm to varying image resolutions – Simply change one parameter: Minimum length of a
horizontal line segment:
• Combine with Corridor Orientation line estimation*
* V. N. Murali and S. T. Birchfield, “Autonomous Exploration Using Rapid Perception of Low-Resolution Image Information”, Autonomous Robots (In review)
Results for varying resolutions
• Wall-floor Boundary
Results for minimalistic geometry
• Minimalistic geometry estimation*
*V. Murali, Y. Li, and S.T. Birchfield, “Extracting Minimalistic Corridor Geometry from Low-Resolution Images”, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2011.(In review)
Sample Videos 1
Sample Videos 2
Conclusion
• Image-Based Floor Segmentation
- Edge-based approach to segmenting floors in
corridors
- Correctly handles specular reflections on floor
- Nearly 90% of the corridor images in our database
can be correctly detected.
- Speed: approximately 7 frames / sec
- Extension
- Three line “geometry”---enough for basic robot navigation
Future Work• Image-Based Floor Segmentation
- Speed up the current high-res algorithm
- Improving score model by add more visual cues
(highly textured floors, low resolution, or dark environment)
- Adapt weights by Adaboost training
- Use floor segmentation for mapping
Thanks!
Questions?
Segmentation of Floor in Corridor Images for Mobile Robot Navigation
---By Yinxiao Li