Segmentation and Perceptual Grouping

Kaniza

(Introduction to Computer Vision, 11.1.04)

The image of this cube contradicts the optical image

Perceptual Organization

• Atomism, reductionism:– Perception is a process of decomposing an

image into its parts.– The whole is equal to the sum of its parts.

• Gestalt (Wertheimer, Köhler, Koffka 1912)– The whole is larger than the sum of its parts.

Gestalt: apparent motion

Gestalt: apparent motion

Gestalt Principles

• Proximity

Gestalt Principles

• Proximity• Proximity• Similarity

Gestalt Principles

• Proximity• Similarity

• Proximity• Similarity• Continuity

Gestalt Principles

• Closure• Proximity• Similarity• Continuity

Gestalt Principles

• Proximity• Similarity• Continuity

• Closure• Closure• Common Fate

Gestalt Principles

• Proximity• Similarity• Continuity

• Closure• Common Fate• Simplicity

• Closure• Common Fate

Mona Lisa

Mona Lisa

Smooth Completion

• Isotropic

• Smoothness

• Minimal curvature

• Extensibility

Elastica:

2min ( )k s ds

Elastica

• Scale invariant (Weiss, Bruckstein & Netravali)

• Approximation (Sharon, Brandt & Basri)

2 21 2 1 24( )

2min ( )l k s ds

(Sharon, Brandt & Basri)

Hough Transform

Hough Transform

Curve Salience

Saliency Network

Encourage

• Length

• Low curvature

• Closure

(Shashua & Ullman)

Saliency Network(Shashua & Ullman)

Tensor Voting

• Every edge element votes to all its circular edge completions

• Vote attenuates with distance: e-αd

• Vote attenuates with curvature: e-βk

• Determine salience at every point using principal moments

(Guy & Medioni)

Tensor Voting(Guy & Medioni)

Stochastic Completion Field

• Random walk:

• In addition, a particle may die with probability:

2

cos

sin

(0, )

x

y

N

1/ re

(Mumford; Williams & Jacobs)

Stochastic Completion Fields

• Most probable path:

with

2

2

( )

1

21

log( 2 )

k s ds ds

r

(Mumford; Williams & Jacobs)

Stochastic Completion Fields(Mumford; Williams & Jacobs)

Stochastic Completion Fields(Mumford; Williams & Jacobs)

Stochastic Completion Fields(Mumford; Williams & Jacobs)

Shortest Path(Hu, Sakoda & Pavlidis)

Snakes

• Given a curve Г(s)=(x(s),y(s)), define:1

0

int

22 2

int 2

( ( ))

( ( ))

( , )

( ) ( )

image ext

image

E s ds

E s E E E

E I x y

E s ss s

(Kass, Witkin & Terzopolous)

Snakes: Curve Evolution

Snakes: Curve Evolution

Thresholding

Histogram

0 50 100 150 200 250

0

200

400

600

800

1000

1200

Thresholding

Thresholding

125

15699

Image Segmentation

Camouflage

Minimum Cut(Wu & Leahy)

Texture Examples

Filter Bank(Malik & Perona)

Normalized Cuts(Malik et al.)

Segmentation by Weighted Aggregation

A multiscale algorithm:• Optimizes a global measure• Returns a full hierarchy of segments• Linear complexity• Combines multiscale measurements:

– Texture– Boundary integrity

(Galun, Sharon, Brandt & Basri)

Segmentation by Weighted Aggregation(Galun, Sharon, Brandt & Basri)

Leopards

And More…

Malik’s Ncuts