Alessandro Franchi, Federico Tombari, Luigi Di Stefano

BOLD features to detecttexture-less objects

Motivations• Object detection is among the most widely studied topics in

computer vision.

2

The established paradigm for textured object detection relies on matching descriptors of local features•e.g. SIFT, SURF, BRIEF, ORB…•do not work well with texture-less objects

Texture-less object detection is still an open issue in computer vision literature.State-of-the-art approaches operate by means of edge-based template matching•do not scale well w.r.t. the size of the model library and the pose space to be explored

BOLD(Bunch Of Lines Descriptor)

• Objective: envision new features aimed at the detection of texture-less objects, to be injected in a classic descriptor-based object detection pipeline

• Descriptor for line segments

• Aggregates geometric primitives computedover pairs of neighboring segments

• Characteristics:– Invariant to rotation, translation and scale– Robust to noise and blur– Efficient to compute

3

Pose EstimationPose EstimationPose EstimationPose EstimationGeneralized

Hough Transform

Generalized Hough

Transform

Generalized Hough

Transform

Generalized Hough

Transform

Descriptor-based Object Detection

Keypoint DetectionKeypoint Detection

KeypointDescription

KeypointDescription

Descriptor MatchingDescriptor Matching

Keypoint DetectionKeypoint Detection

KeypointDescription

KeypointDescription

Descriptor MatchingDescriptor Matching

BOLD - Detection

5

Input imageInput image Extracted line segments

Extracted line segments• polygonal approximation

of the output of an edge detector

• specific line detection algorithms

• …

Canonical Orientation assignment

Canonical Orientation assignment

BOLD - Description (1)Pairwise geometric primitives

• Different geometric primitives have been evaluated– Normalized segment length– Relative midpoint distance– Relative angles– …

• Selected primitives: α,β angles– encode simultaneously

• relative orientations• relative segment displacements• contrast polarity

6

sj

si

mj

mi

g(mj)

g(mi)

ej1

ej2

ei1

ei2

β

αt

BOLD - Description (2)Bunch of segments construction

• A “bunch” of a segment is defined as the set of its k nearest neighboring (kNN) segments

k = 6k = 6 k = 6k = 6

αβ

45

90

135

180

225

270

315

0

360

36031545 90 135 180 225 270

s2

s1

BOLD - Description (3)Aggregating pairwise primitives

• For each segment pair formed by a segment si and one of the k segments in its bunch we accumulate α and β angles in a 2D joint histogram

α1

β1

α2

β2

EvaluationDatasets

9

“CVLab texture-less dataset” with clutter and occlusions9 models – 55 scenes

“CVLab texture-less dataset” with clutter and occlusions9 models – 55 scenes

“Caltech Covers” synthetic textured dataset with clutter and max 90% occlusions

80 models – 50 scenes

“Caltech Covers” synthetic textured dataset with clutter and max 90% occlusions

80 models – 50 scenes

EvaluationTested methods

Name Type Reference

BOLD Descriptor-based ‐‐‐

SIFT Descriptor-based D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” IJCV, 2004

SURF Descriptor-based H. Bay, T. Tuytelaars, and L. Van Gool, “SURF: Speeded up robust features,” ECCV, 2006

ORB Descriptor-based E. Rublee, V. Rabaud, K. Konolige, G. Bradski, “ORB: An efficient alternative to SIFT or SURF,” ICCV, 2011

Line2D Edge-based Template matching

S. Hinterstoisser, S. Holzer, C. Cagniart, S. Ilic, K. Konolige, N. Navab, V. Lepetit, "Multimodal Templates for Real-Time Detection of Texture-less Objects in Heavily Cluttered Scenes," ICCV, 2011

S. Hinterstoisser, C. Cagniart, S. Ilic, P. Sturm, N. Navab, P. Fua, V. Lepetit, "Gradient Response Maps for Real-Time Detection of Texture-Less Objects," PAMI, 2011

HALCON Edge-based Template matching http://www.mvtec.com/halcon/

10

BOLD detection examples

Texture-less object detection evaluation"CVLab textureless“ dataset

Detection results over a textured dataset with 9 models and 55 scenes.Detection time include

matching, validation and pose estimation

Detection results over a textured dataset with 9 models and 55 scenes.Detection time include

matching, validation and pose estimation

Textured object detection evaluation"Caltech Covers“ dataset

Detection results over a textured dataset with 80 models and 50 scenes.Detection time include

matching, validation and pose estimation

Detection results over a textured dataset with 80 models and 50 scenes.Detection time include

matching, validation and pose estimation

Scalability w.r.t. number of models"CVLab textureless“ dataset

15

Conclusions• BOLD features allows for leveraging on a standard descriptor-

based pipeline to detect effectively also texture-less objects– robustness to clutter and occlusion– high scalability w.r.t. the size of the model library

• Limitations– Curvilinear shapes (e.g. round objects)

• Linear approximation of highly curved contours is imprecise– Very simple shapes

• Too few segments, limited informativecontent of associated BOLDs

• Possible future developments– Include description of circular and elliptical arcs– Deploying BOLD to attain 3D object detection

based on a multi-view approach– Left-Right bunches

Right bunchRight bunch Left bunchLeft bunch

Datalogic AutomationVia Lavino 26540050 Monte San PietroBologna – ItalyTel. +39 051 3147011 – Fax +39 051 6759324www.datalogic.automation.com

This presentation contains statements that are neither reported financial results nor other historical information. These statements are forward-looking

statements. These forward-looking statements rely on a number of assumptions and are subject to a number of risks and uncertainties, many of which are

outside the control of Datalogic S.p.A., that could cause actual results to differ materially from those expressed in or implied by such statements, such as

future market conditions, currency fluctuations, the behavior of other market participants and the actions of governmental and state regulators

© 2011 Datalogic S.p.A. - All rights reserved. • Protected to the fullest extent under U.S. and international laws. • Copying, or altering of this document is

prohibited without express written consent from Datalogic S.p.A. Datalogic and the Datalogic logo are registered trademarks of Datalogic S.p.A. in many

countries, including the U.S.A. and the E.U. All other brand and product names may be trademarks of their respective owners.

Thank You!