Date post: | 30-Dec-2015 |
Category: |
Documents |
Upload: | reynard-preston |
View: | 223 times |
Download: | 5 times |
May I introduce myself…
• Rolf Lakaemper
• PhD (Doctorate Degree) 2000Hamburg University, Germany
• Currently Assist. Professor at Departmentof Computer and Information Sciences,Temple University, Philadelphia, USA
• Main Research Area: Computer Vision
Why Shape ?
Several applications in computer vision use shape processing:
• Object recognition• Image retrieval
• Processing of pictorial information• Video compression (eg. MPEG-7)
…
This presentation focuses on object recognition and image retrieval.
Motivation
Typical Application: Multimedia: Image Database
Query by Shape / Texture / …(Color / Keyword)
Blobworld
Example 1: Blobworldhttp://elib.cs.berkeley.edu/photos/blobworld/start.html
BLOB = “Binary Large Object”, “an indistinct shapeless (really ?) form”
Blobworld
Blobworld: Query by Shape / Texture / Location / Color
Selected Blob
Query:
by Color and Texture of Blob
Result:
Blobs with similar Color and Texture
Satisfying ?
Blobworld
Blobworld: Query by Shape / Texture / Location / Color
Selected Blob
Query:
by Shape of Blob
Result:
…are these shapes similar ?
Satisfying ?
ISS Database
Example 2: ISS-Databasehttp://knight.cis.temple.edu/~shape
ISS Database
ISS: Query by Shape / Texture
Sketch of Shape
Query:
by Shape only
Result:
Satisfying ?
Overview
Overview Part 1
• Why shape ?• What is shape ?• Shape similarity• Metrices• Classes of similarity measures• Feature Based Coding• Examples for global similarity
Why Shape ?
Why Shape ?
• Shape is probably the most important property that is perceived about objects. It allows to predict more facts about an object than other features, e.g. color (Palmer 1999)
• Thus, recognizing shape is crucial for object recognition. In some applications it may be the only feature present, e.g. logo recognition
Why Shape ?
Shape is not only perceived by visual means:
• tactical sensors can also provide shape information that are processed in a similar way.
• robots’ range sensor provide shape information, too.
Shape
Typical problems:
• How to describe shape ? • What is the matching
transformation?• No one-to-one correspondence• Occlusion• Noise
What is Shape ?
What is Shape ?
Plato, "Meno", 380 BC:
• "figure is the only existing thing that is found always following color“
• "figure is limit of solid"
What is Shape ?
… let’s start with some properties easier to agree on:
• Shape describes a spatial regionShape is a (the ?) specific part of spatial cognition
• Typically addresses 2D spacewhy ?
What is Shape ?
Moving on from the naive understanding, some questions arise:
• Is there a maximum size for a shape to be a shape?
• Can a shape have holes?• Does shape always describe a connected
region?• How to deal with/represent partial shapes
(occlusion / partial match) ?
What is Shape ?
Shape or Not ?
Continuous transformation from shape to no shape: Is there a point when it stops being a shape?
What is Shape ?
Shape or Not ?
Continuous transformation from shape to two shapes: Is there a point when it stops being a single shape?
What is Shape ?
• There’s no easy, single definition of shape• In difference to geometry, arbitrary shape is not
covered by an axiomatic system
• Different applications in object recognition focus on different shape related features• Special shapes can be handled
• Typically, applications in object recognition employ a similarity measure to determine a plausibility that two shapes correspond to each other
Similarity
which similarity measure,
depends onwhich required properties,
depends onwhich particular matching problem,
depends onwhich application
Similarity
... robustness
... invariance to basic transformations
Simple Recognition (yes / no)
Common Rating (best of ...)
Analytical Rating (best of, but...)
…which application
Similarity
…which problem
• computation problem: d(A,B)
• decision problem: d(A,B) <e ?
• decision problem: is there g: d(g(A),B) <e ?
• optimization problem: find g: min d(g(A),B)
Similarity Measure
Requirements to a similarity measure
• Should not incorporate context knowledge (no AI), thus computes generic shape similarity
Similarity Measure
Requirements to a similarity measure
• Must be able to deal with noise• Must be invariant with respect to basic
transformations
Next:StrategyScaling (or resolution)
Rotation
Rigid / non-rigid deformation
Similarity Measure
Requirements to a similarity measure
• Must be able to deal with noise
• Must be invariant with respect to basic transformations
• Must be in accord with human perception
Similarity Measure
Some other aspects worth consideration:
• Similarity of structure• Similarity of area
Can all these aspects be expressed by a single number?
Similarity Measure
Desired Properties of a Similarity Function C(Basri et al. 1998)
• C should be a metric• C should be continous• C should be invariant (to…)
Properties
Metric Properties
S set of patternsMetric: d: S S R satisfying1. Self-identity: xS, d(x,x)=02. Positivity: x yS, d(x,y)>03. Symmetry: x, yS, d(x,y)= d(y,x)4. Triangle inequality: x, y, zS, d(x,z)d(x,y)
+d(y,z)
• Semi-metric: 1, 2, 3• Pseudo-metric: 1, 3, 4• S with fixed metric d is called metric space
Properties
In general:
• a similarity measure in accordance with human perception is NOT a metric. This leads to deep problems in further processing, e.g. clustering, since most of these algorithms need metric spaces !
Properties
Continuity:
“usually useful”, although sometimes not in accordance with principles of Gestalt properties, e.g. symmetry,
collinearity.
Properties
Some more properties:
• One major difference should cause a greater dissimilarity than some minor ones.
• S must not diverge for curves that are not smooth (e.g. polygons).
However, these demands are contradictory (proof is left as an exercise)
Similarity Measures
Classes of Similarity Measures:
Similarity Measure depends on
• Shape Representation
• Boundary
• Area (discrete: = point set)
• Structural (e.g. Skeleton)
• Comparison Model
• feature vector
• direct
Similarity Measures
direct feature based
Boundary Spring model, Cum. Angular Function, Chaincode, Arc Decomposition (ASR-Algorithm)
Central Dist. Fourier
Distance histogram
…
Area (point set) Hausdorff
…
Moments
Zernike Moments
…
Structure Skeleton
…
---
Feature Based Coding
Feature Based Coding
This category defines all approaches that determine a feature-vector for a given shape.
Two operations need to be defined: a mapping of shape into the feature space and a similarity of feature vectors.
Representation Feature Extraction Vector Comparison
Vector Comparison
More Vector Distances:
• Quadratic Form Distance
• Earth Movers Distance
• Proportional Transportation Distance
• …
Feature Based Coding
Again:
Two operations need to be defined: a mapping of shape into the feature space and a similarity of feature vectors.
We hence have TWO TIMES an information reduction of the basic representation, which by itself is already a mapping of the ‘reality’.
Representation Feature Extraction Vector Comparison
Vector Comparison
Example 1
Elementary Descriptors
Shape A,B given as
• Area (continous) or
• Point Sets (discrete)(Elementary Descriptors are 1dimensional feature
vectors)
Vector Comparison
Example 2
(Discrete) Moments
Shape A,B given as
• Area (continous) or
• Point Sets (discrete)
Discrete Moments
Exercise:
Please compute all 7 moments for the following shapes, compare the vectors using different comparison techniques
Discrete Moments
Result: each shape is transformed to a 7-dimensional vector. To compare the shapes, compare the vectors (how ?).
Vector Comparison
Contour is given as list of euclidean coordinates:
0,0; 1,0; 2,0; 2,1; 2,2; 3,3; 4,3; 5,2; …
0 1 2
3
4
5 6
7 ...
...
Vector Comparison
All Feature Vector approaches have similar properties:
• Provide a compact representation
• this is especially interesting for database indexing !
• Works for any shape
• Requires complete shapes (global comparison)
• Sensible to noise (except Zernike moments which are computationally demanding)
• Map dissimilar shapes to similar feature vectors (!)• They can be used as a prefilter for database applications !
• Make the choice of a similarity function difficult
Vector ComparisonDirect Comparison
Example 1
Hausdorff Distance
Shape A,B given as point sets
A={a1,a2,…}
B={b1,b2,…}
Vector ComparisonBoundary Representation
Hausdorff:
Unstable with respect to noise(This is easy to fix ! How ?)
Problem: Invariance !Nevertheless: Hausdorff is the motor behind many applications in specific fields (e.g. character recognition)
Vector ComparisonBoundary Representation
Example 2
Chaincode Comparison
Shape A,B given as chaincode
Vector ComparisonBoundary Representation
A binary image can be converted into a ‘chain code’ representing the boundary. The boundary is traversed and a string representing the curvature is constructed.
0
123
4
5 6 7
C
5,6,6,3,3,4,3,2,3,4,5,3,…
Vector ComparisonBoundary Representation
To extend this measure to strings, two steps are carried out.
1. Extend the measure to character against string, for example by summing up individual similarity measures.
2. Employing a matching to compute a correspondence of sub-strings. Hereby, the matching constitutes from 1-to-1, 1-to-many, and many-to-1-matchings. It is computed as string matching by means of dynamic programming.
Vector ComparisonBoundary Representation
Digital curves suffer from effects caused by digitalization, e.g. rotation:
Vector ComparisonBoundary Representation
Compare chaincodes by string matchingAs string-matching is not able to model a matching of digital curves adequately, more sophisticated matching algorithms are employed in “real applications” using chain codes:
Weighted Levensthein DistanceDefines an edit distance for transforming one string into another.
Costs are defined for altering, deleting, or inserting a character.
Extended DistanceFormal translation system with costs assigned to individual production rules.
Vector ComparisonStructural Representation
Example 3
Skeletons
Shape A,B primarily given as area or boundary, structure is derived from
representation
Vector ComparisonStructural Representation
Structural approaches capture the structure of a shape, typically by rep-resenting shape as a graph.
Typical example: skeletons
Vector ComparisonStructural Representation
The computation can be described as a medial axis transform, a kind of discrete generalized voronoi.
Vector ComparisonStructural Representation
The graph is constructed mirroring the adjacency of the skeleton’s parts. Edges are labeled according to the qualitative classes.
Matching two shapes requires matching two usually different graphs against each other.