Dollár, Piotr, and C. Lawrence Zitnick. "Structured forests for fast edge detection.“

Computer Vision (ICCV), 2013 IEEE International Conference on. IEEE, 2013.

Main

Contribution

Compute edge maps in realtime,

faster than the competing state-of-the-art

Proposed

Method

Structured Random Forests

This presentation is inspired by the talk: http://techtalks.tv/talks/structured-forest-for-fast-edge-detection/59412/

Edge Definition

Source: http://upload.wikimedia.org/wikipedia/en/8/8e/EdgeDetectionMathematica.png

Edge Definition

Source: http://upload.wikimedia.org/wikipedia/en/8/8e/EdgeDetectionMathematica.png

Where this work excels

A c c u r a c y & S p e e d [ re a l t i m e ]

Where this work excels

A c c u r a c y & S p e e d [ re a l t i m e ]

Where this work excels

A c c u r a c y & S p e e d [ re a l t i m e ]

Where this work excels

A c c u r a c y & S p e e d [ re a l t i m e ]

Edge Detection

as

Classification Problem

Edge Detection as Classification Problem

• {0, 1}

Edge Detection as Classification Problem

• {0, 1}

• Binary classification ignoring the local structures of the edges

Edges have Structures

Clustering Sketch Tokens

Sketch Tokens: A Learned Mid-level Representation for Contour and Object Detection, Joseph J. Lim et al. 2013

Random Forests

Random Forests

ℎ 𝑥, 𝜃 = 𝑥 𝑘1 − 𝑥 𝑘2 < 𝜏

Random Forests

Random Forests

Random Forests

Random Forests For Edge Detection

Random Forests For Edge Detection

Random Forests For Edge Detection

Random Forests For Edge Detection

Random Forests For Edge Detection

Random Forests For Edge Detection

Decision:

Structured Random Forests

The Output Space

{0, 1} 2

{ ,….} 151

DimensionalityInput Space

The Output Space

{0, 1} 2

{ ,….} 151

2256

DimensionalityInput Space

Node Split

Low entropy split

Training Model

Bad split

Training Model

Go od split

Training Model

Cluster the

structured labels

Training Model

Just one difference to random forests:

cluster the output into a binary or multiclass output using distance function

Clustering

𝑌: Structured space where information gain not well defined

𝐶: Discrete space where information space is good defined

𝑍: Intermediate space where similarity measurement is easy to compute

Π ∶ 𝑌 → 𝑍 , 𝑍 → 𝐶

Training Model

• Computing information gain

– Labels 𝐶 are discrete, standard entropy criterions used.

• Combining predictions

– To combine 𝑦1… 𝑦𝑛 ∈ 𝑌 into a prediction:

• Compute 𝑧𝑖 = Π𝜑(𝑦𝑖) of dimension 𝑚

• Select 𝑦𝑘 , whose 𝑧𝑘 = 𝑎𝑟𝑔𝑚𝑖𝑛𝑧𝑘 𝑖,𝑗(𝑧𝑘𝑗 − 𝑧𝑖𝑗)2

(medoid)

+ Computing medoids is fast, 𝑂(𝑛𝑚)

Training Structured Forests For

Edge Detection

Training Structured Forests For Edge Detection

32x32 RGB image patch

→ 7228 features

Training Structured Forests For Edge Detection

32x32 RGB image patch

→ 7228 features

Π ∶ 𝑌 → 𝑍

Dimension of 𝑍 = 2562

Down-sampled to m = 256

Training Structured Forests For Edge Detection

32x32 RGB image patch

→ 7228 features

Π ∶ 𝑌 → 𝑍

Dimension of 𝑍 = 2562

Down-sampled to m = 256

Edge Detection with Structured Forests

32x32 RGB image patch

→ 7228 features

Edge Detection with Structured Forests

32x32 RGB image patch

→ 7228 features

𝑌 is a 16x16 segmentation

mask

Multi-scale Detection

Multi-scale Detection

Multi-scale Detection

Results

• BSDS 500 image set

– Multi-scale ties or outperforms the accuracy of the state of the art.

– Single-scale improves runtime by 5x to 10x

Results

• BSDS 500 image set

– Multi-scale ties or outperforms the accuracy of the state of the art.

– Single-scale improves runtime by 5x to 10x

Results

• BSDS 500 image set

– Multi-scale ties or outperforms the accuracy of the state of the art.

– Single-scale improves runtime by 5x to 10x

Results

• NYU image set

– Multi-scale is slightly better than the state of the art.

– Improved performance by multiple orders of magnitude

Conclusions

• Realtime structured learning method for edge detection

• General purpose method for learning structured random forests

• Real time + state of the art accuracy → new applications possible

• Novel learning approach may be applicable to other problems.

T h a n k yo u