Introduction to RoboticsFeatures

February 22, 2010

Review: Vision

• Convolution-based filters• Thresholds• Morphological operators• Goal: condensing information

OpenCV morphology demo

Example: Edge Detection

OpenCV example edge

Today: higher level features

• Features allow to reason about the environment– Where am I– Where can I go– What is this– Where is this

• N.B. Features can be extracted from ANY sensor

Example: Visual Servoing/Grasping

- Servo to fruit using image Jacobian

- Rely on radius estimate for depth

- Close gripper / retract arm when arrived

2

F. Chaumette and S. Hutchinson, “Visual servo control part i: Basic approaches,” Robotics & Automation Magazine, vol. 13, no. 4, pp. 82–90

€

dx

dy

dz

⎛

⎝

⎜ ⎜ ⎜

⎞

⎠

⎟ ⎟ ⎟=

dx

du

dx

dv

dx

dwdy

du

dy

dv

dy

dwdz

du

dy

dv

dz

dw

⎛

⎝

⎜ ⎜ ⎜ ⎜ ⎜ ⎜

⎞

⎠

⎟ ⎟ ⎟ ⎟ ⎟ ⎟

du

dv

dw

⎛

⎝

⎜ ⎜ ⎜

⎞

⎠

⎟ ⎟ ⎟

u, vw

Function of arm kinematics

Detection of Fruits

• Objects are defined by features

• Simple: filters “vote” for object locations

• Depth estimated from radius

Sobel Color SpectralHighlights

Segmentation

• Pyramid, mean-shift, graph-cut• Here: Watershed

Gary Bradski (c) 2008 77

Watershed algorithm

http://cmm.ensmp.fr/~beucher/wtshed.html

Demo OpenCV pyramid_segmentation

Contours

Gary Bradski (c) 2008 99

From points to geometry

Least-Square FittingLeast-Squares Fitting of Circles and Ellipses, Walter Gander, Gene H. Golub, Rolf Strebel.

Demo: OpenCV convexhull, squares

Hough Transform

Demo: OpenCV houghlines

Hough Transform

Source: K. Grauma / D. Scaramuzza

So far

• Low-level image features– Convolution-based• Edge detection• Color detection

– Watershed transform– Hough Transform– Morphology

• What about convolution with more complex features?

Face Detection with Viola-Jones Rejection Cascade and Boosting

1414by Gary Bradski

by Viola & Jones

Robust Real-time Object Detection. Paul Viola Michael Jones. 2nd Int. Workshop on statistical and computational theories of vision – Modeling, Learning, Computing and Sampling, 2001.

•Select features using learning (AdaBoost)•Narrow down objects using detection cascade

Example: tomato detection

• Learn dominant features from labeled set– Take random features– Learn features that

generalize best• Detect features using

convolution• Features “vote” on object

centroid good for partially

occluded objectsA. Torralba, K. Murphy, and W. Freeman, “Sharing features: efficientboosting procedures for multiclass object detection,” in Proceedingsof the IEEE Computer Society Conference on Computer Vision andPattern Recognition (CVPR), 2004, pp. 762–769.

Unique Point features

Example: D. Scaramuzza

Harris corner detection

• Corners: edge gradients in two directions

• Idea: corners are repeatable and distinctive

C.Harris and M.Stephens. A Combined Corner and Edge Detector.“ Proceedings of the 4th Alvey Vision Conference: pages 147--151.

MATLAB example: Harris Corner Detector by Ali Ganoun

Harris corner detectorInvestigate gradients in moving window

Flat regions: no changein any direction

Images: A. Efros

Edge: change along edgedirection

Corner: change alongtwo directions

Harris corner detector

• Corners are invariant to rotation of the image, but distance-based matching is NOT

• Corners are NOT invariant to scale• Corners are NOT invariant to illumination

D. Scaramuzza

SIFT detector

• Scale-free detector• Key idea: average intensity will be the same

independent of rotation and scale

D. Scaramuzza

Approach: Difference of Gaussians

D. Scaramuzza

Performance

K.Mikolajczyk, C.Schmid. “Indexing Based on Scale Invariant Interest Points”. ICCV 2001.

D. Scaramuzza

Algorithm

• Find common maxima in scaled DoG images

• Extract regional keypoint descriptor

• Store/Compare descriptor in database

D. Scaramuzzahttp://www.cs.ubc.ca/~lowe/keypoints/

Project Assignments

• 4-5 groups• 1-2 graduate students per group• Balance of CS/EE/ME and AE students• Goal: implement a controller for

RobotStadium• Grad students: independent project focusing

on one aspect of the controller

Homework

• Read chapter 5 -> Section 5.5 (pages 181-212)