www.cs.technion.ac.il/~ron
Numerical Geometry in Image Processing
Ron Kimmel
Geometric Image Processing Lab
Computer Science Department Technion-Israel Institute of Technology
Heat Equation in Image Analysis
Linear scale space (T. Iijima 59, Witkin 83, Koenderink 84)
)(tIIt )0(*)()( ItGtI
Geometric Heat Equation in Image Analysis
Geometric scale space, Euclidean (Gage-Hamilton 86, Grayson 89, Osher-Sethian 88, Evans Spruck 91, Alvarez-Guichard-Lions-Morel 93)
Geometric Heat Equation in Image Analysis
Gabor 65 anisotropic reaction-diffusion Geometric, Special Affine. (Alvarez-Guichard-Lions-Morel
93, Sapiro-Tannenbaum 93)
Geometric Heat Equation in Image Analysis
Multi Channel, Euclidean.(Chambolle 94, Whitaker-Gerig 94, Proesmans-Pauwels-van Gool 94,Sapiro-Ringach 96, Shah 96, Blomgren-Chan 96, Sochen-Kimmel-Malladi 96, Weickert, Romeny, Lopez, and van Enk 97,…)
Geometric, Bending.(Curves: Grayson 89, Kimmel-Sapiro 95 (via Osher-Sethian),Images: Kimmel 97)
Bending Invariant Scale Space
Invariant to surface bending. Embedding: The gray level sets embedding is preserved. Existence: The level sets exist for all evolution time,
disappear at points or converge into geodesics. Topology: Image topology is simplified. Shortening flow:The scale space is a shortening flow of the
image level sets. Implementation: Simple, consistent, and stable numerical
implementation.
Curves on Surfaces: The Geodesic Curvature
From Curve to Image Evolution
Geodesic curvature flow
The Beltrami Framework
Brief history of color line element theories. A simplified color image formation model. The importance of channel alignment. Images as surfaces. Surface area minimization via Beltrami flow. Applications: Enhancement and scale space. Beyond the metric, the Gabor connection
Images as Surfaces Gray level analysis is sometimes misleading…
Is there a `right way’ to link color channels? process texture? enhance volumetric data?
We view images as embedded maps that flow towards minimal surfaces: Gray scale images are surfaces in (x,y, I), and color images are surfaces embedded in (x,y,R,G,B).
Joint with Sochen & Malladi, IEEE T-IP 98, IJCV 2000.
Helmholtz 1896: Schrodinger 1920:
Stiles 1946: Vos and Walraven 1972: inductive line elements (above), empirical line
elements (MacAdam 1942, CIELAB 1976). Define: the simplest hybrid spatial- color space:
Spatial-Spectral Arclength
Color Image Formation
F. Guichard 93Mondrian world:Lambertian surface patches
Image formationLambetian
model
V
lN
)cos(,),( lNyxI
)cos(,),(
)cos(,),(
)cos(,),(
BB
GG
RR
lNyxB
lNyxG
lNyxR
Color Image Formation
The gradient directions should agree since
Example: Demosaicing
Color image reconstruction Solution: Edges support the colors and the colors support the edges
Color Image Formation
Lambertian shading model: R(x,y) = <N,L> G(x,y) = <N,L> B(x,y) = <N,L>Thus Within an object R/G= / =constant We preserve color ratio weighted by an edge
indication function.
R
G
B
R G
Demosaicing ResultsOriginal Bilinear interpolation Weighted interpolation
Demosaicing ResultsBilinear interpolation Weighted interpolation
Demosaicing ResultsOriginal Bilinear interpolation Weighted interpolation
Demosaicing ResultsBilinear interpolation Weighted interpolation
Demosaicing ResultsOriginal Bilinear interpolation Weighted interpolation
Demosaicing ResultsBilinear interpolation Weighted interpolation
From Arclength to Area
Gray level arclength:
Color arclength
Area
Multi Channel Model
The Beltrami Flow
Gray level:
The Beltrami Flow
Color :
where
Matlab Program
Signal processing viewpoint
Beltrami Smoothing
Gaussian Smoothing
Sochen, Kimmel, Bruckstein, JMIV, 2001.
The Beltrami Flow
Texture:
Inverse Diffusion Across the Edge
Inverse Diffusion Across the Edge
Summary: Geometric Framework
From color image formation to the importance of channel alignment.
From color line element theories to the definition of area in color images.
Area minimization as a unified framework for enhancement and scale space.
Inverse heat operator across the edges. Related applications: Color movies segmentation
and demosaicing
www.cs.technion.ac.il/~ron
Open Questions
Is there a maximum principle to the Beltrami flow?
Are there simple geometric measures to minimize in color image processing subject to more complicated image formation models?
Can we really invert the geometric heat operator?
Is there a real-time numerical implementation for the Beltrami flow in color?
www.cs.technion.ac.il/~ron