Part 3: Fourier Transform and Filtering in the Frequency...

transcript

2011-04-19

Digital Image Processing

Achim J. Lilienthal

AASS Learning Systems Lab, Dep. Teknik

Room T1209 (Fr, 11-12 o'clock)

achim.lilienthal@oru.se

Digital Image ProcessingPart 3: Fourier Transform and

Filtering in the Frequency Domain

Course Book Chapter 4

Achim J. Lilienthal

Derivatives and their Fourier Transform

Laplacian in the Fourier Domain

[ ])()()( xfujdx

xfd nn

)(),( 22 vuvuH Laplacian +−=⇒

),()()],([ 222 vuFvuyxf +−=∇F

2 Relation Between Spatial and Frequency Filters

Achim J. Lilienthal

Derivatives and their Fourier Transform

[ ])()()( xfujdx

xfd nn

( ) ( ) ( )vuFNvMuyxf ,22),(222

−+−−⇔∇

Achim J. Lilienthal

Laplacian in the Spatial Domain

( ) ( ) yxvuvu ++− −−+− 1]1)([ 221F

Achim J. Lilienthal

Deriving Spatial Filter Masks

General Idea select a filter in the frequency domain

transform this filter to the spatial domain

try to specify a small filter mask that captures the "essence" of the filter function

Achim J. Lilienthal

Filtering in the Frequency Domain

Other Filters bandpass

allows frequencies in a band between the two frequencies D0 and D1

bandstop: stops frequencies in a band

between the two frequencies D0 and D1

non-symmetric filters: allow different frequencies in the u and v direction

Achim J. Lilienthal

Recovering Intrinsic Images,Homomorphic Filtering

→ Contents

Achim J. Lilienthal

Reminder: Image Formation Model illumination i(x,y) from a source reflectivity r(x,y) = reflection / absorption in the scene

f(x,y) = r(x,y) i(x,y)

Image Formation3

Achim J. Lilienthal

Intrinsic Images "midlevel description" of scenes

proposed by Barrow and Tenebaum[H.G. Barrow and J.M. Tenenbaum. Recovering Intrinsic Scene Characteristics from Images". In Computer Vision Systems. Academic Press, 1978]

not a full 3D description of the scene

viewpoint dependent

physical causes of changes in illumination are not made explicit

Intrinsic Images3

Achim J. Lilienthal

proposed by Barrow and Tenebaum[H.G. Barrow and J.M. Tenenbaum. Recovering Intrinsic Scene Characteristics from Images". In Computer Vision Systems. Academic Press, 1978]

"The observed image is a product of two images: an illumination image and a reflectance image."

Intrinsic Images3

Achim J. Lilienthal

(input) image is decomposed into two images …

Intrinsic Images3

from "Deriving Intrinsic Images From Image Sequences",

Yair Weiss , Proc. ICCV 2001

Achim J. Lilienthal

(input) image is decomposed into two images … a reflectance image

Intrinsic Images3

r(x,y)

Achim J. Lilienthal

(input) image is decomposed into two images … a reflectance image and

an illumination image

Intrinsic Images3

r(x,y) i(x,y)

Achim J. Lilienthal

"The observed image is a product of two images: an illumination image and a reflectance image." segmentation on the intrinsic reflectance should be much simpler

than on the original image

3D information can be obtained from the illumination picture

Intrinsic Images3

r(x,y) i(x,y)

Achim J. Lilienthal

(input) image is decomposed into two images … a reflectance image and

an illumination image

but: decomposition is an ill-posed problem number of unknowns is twice as high as the number of equations

(for example: set i(x,y) = 1 r(x,y) = f(x,y))

Intrinsic Images3

),(),(),( yxryxiyxf =

Achim J. Lilienthal

Homomorphic Filtering

Idea: Separate Illumination and Reflectance

not separable directly …

… but the logarithm is separable

)],([)],([)],([ yxryxiyxf FFF ≠

[ ]),(ln),( yxfyxz ≡

[ ][ ] [ ][ ]),(),(),(),(ln),(ln)],([

vuFvuFvuZyxryxiyxz

ri +==+= FFF

Achim J. Lilienthal

Frequency Domain Approximation to Homomorphic Filtering – Assumption illumination component varies slowly

reflectance component tends to vary abruptly

⇒ use filter that affects low- and high-frequency components in a different way (decreases influence of illumination, increases influence of reflectance)

γH >1

Achim J. Lilienthal

Idea: Separate Illumination and Reflectance

[ ][ ] [ ][ ]),(ln),(ln),(),(),( yxryxivuFvuFvuZ ri FF +=+=

),(),(),(),(),(),(),( vuFvuHvuFvuHvuZvuHvuS ri +==

[ ] [ ] [ ]),('),('

),(),(),(),(),(),( 111

yxryxivuFvuHvuFvuHvuSyxs ri

+=+== −−− FFF

),('),('),(),( zxrzxiyxs eeeyxg ==

Achim J. Lilienthal

Example consider non-uniform illumination

Homomorphic Filtering3

disturbance pattern

Achim J. Lilienthal

Example histogram equalization does not perform well

histogram equalization

Achim J. Lilienthal

Example homomorphic filtering

Homomorphic Filtering3

homomorphic filtering

Achim J. Lilienthal

Homomorphic Filtering Assumption True?["Deriving Intrinsic Images From Image Sequences", Yair Weiss, Proc. ICCV 2001]

edges due to illumination often have as high a contrast as those due to reflectance changes possible solution: deriving intrinsic images from image sequences

Remarks3

Achim J. Lilienthal

Recovering Intrinsic Images from a Single Image["Recovering Intrinsic Images from a Single Image", Marshall F. Tappen, William T Freeman, Edward H Adelson, MIT AI Memo, 2002, http://hdl.handle.net/1721.1/6703]

use colour information

Remarks3

Achim J. Lilienthal

Recovering Intrinsic Images from a Single Image[" Recovering Intrinsic Images from a Single Image ", Marshall F. Tappen, William T Freeman, Edward H Adelson, MIT AI Memo, 2002, http://hdl.handle.net/1721.1/6703]

use colour information find chromaticity changes by classifying image derivatives by

thresholding scalar product of normalized RGB vector neighbours

Remarks3

Achim J. Lilienthal

use colour information find chromaticity changes by classifying image derivatives by

thresholding scalar product of normalized RGB vector neighbours

Remarks3

Achim J. Lilienthal

learn appearance models of shading patterns classify gray-scale image

Remarks3

Achim J. Lilienthal

Remarks3

Achim J. Lilienthal

Remarks3

Achim J. Lilienthal

propagate evidence (MRF model with learned parameters)

Remarks3

Achim J. Lilienthal

use colour information +

learn appearance models of shading patterns +

Remarks3

Achim J. Lilienthal

Remarks3

Achim J. Lilienthal

Remarks3

Achim J. Lilienthal

Remarks3

Achim J. Lilienthal

Remarks3

Achim J. Lilienthal

Remarks3

Achim J. Lilienthal

Properties of the Fourier Transform

→ Contents

Achim J. Lilienthal

a shift in f(x,y) does not affect |F(u,v)|

)//(200

00),(),( NyvMxujevuFyyxxf +−⇔−− π

Translation

Achim J. Lilienthal

Translation a shift in f(x,y)

does not affect the spectrum |F(u,v)|

Achim J. Lilienthal

Distributive Over Addition

Not Distributive Over Multiplication

Scaling

Rotation

)],([)],([)],(),([ 2121 yxfyxfyxfyxf FFF +=+

)],([)],([)],(),([ 2121 yxfyxfyxfyxf FFF ≠

),(),( vuaFyxaf ⇔ )/,/(1),( bvauFab

byaxf ⇔

),(),( 00 θϕωθθ +⇔+ Frf

Achim J. Lilienthal

rotating f(x,y) rotates F(u,v) by the same angle

F(u,v)

f(x,y)

Rotation

Achim J. Lilienthal

Periodicity

the discrete Fourier transform is periodic

also the inverse of the discrete Fourier transform is periodic

Conjugate Symmetry

the spectrum is symmetric about the origin

),(),(),(),( NvMuFNvuFvMuFvuF ++=+=+=

),(),(),(),( NyMxfNyxfyMxfyxf ++=+=+=

),(),( * vuFvuF −−= ),(),( * vuFvuF −−=⇒

Achim J. Lilienthal

Separability

F(x,v) is the Fourier transform along one row

F(u,v) can be obtained by two successive applications of the simple 1D Fourier transform instead of by one application of the more complex 2D Fourier transform

[ ][ ]),(),(1

),(1),(

yxfvxFeM

eyxfMN

NvyMuxj

∑∑

−−

Achim J. Lilienthal

Fourier Transform

from "Computer Vision – A Modern Approach", Forsyth and Ponce,

Prentice Hall, 2002

( )1 ,box x ysin sinu v

u v( )( ),u vF f a b

Achim J. Lilienthal

Correlation

Definition of Correlation

compare with convolution

also the need for padding

∑∑−

* ),(),(1),(),(M

nnymxhnmf

MNyxhyxf

∑∑−

−−=∗1

0),(),(),(),(

nnymxhnmfyxhyxf

Achim J. Lilienthal

Correlation

Template Matching

f(x,y) is the image

h(x,y) is a template

if h(x,y) matches somewhere in f(x,y) the correlation will be maximal there

∑∑−

* ),(),(1),(),(M

nnymxhnmf

MNyxhyxf

),(),(),(),( * vuHvuFyxhyxf ⇔

),(),(),(),(* vuHvuFyxhyxf ⇔

Achim J. Lilienthal

Correlation

Template Matching – Example 1

f1(x,y) – padded h(x,y) – padded

Achim J. Lilienthal

Correlation

F -1[F1*(u,v)H(u,v)] – rescaledf1(x,y) – padded

Achim J. Lilienthal

Correlation

F -1[F1*(u,v)H(u,v)] 4 – rescaled4f1(x,y) – padded

Achim J. Lilienthal

Correlation

Definition of Correlation (from previous slide)

Correlation Theorem

∑∑−

* ),(),(1),(),(M

nnymxhnmf

MNyxhyxf

),(),(),(),( * vuHvuFyxhyxf ⇔

),(),(),(),(* vuHvuFyxhyxf ⇔

Achim J. Lilienthal

Correlation

Achim J. Lilienthal

Correlation

f2(x,y) – padded F -1[F2*(u,v)H(u,v)] 4 – rescaled4

Achim J. Lilienthal

Correlation

Achim J. Lilienthal

Correlation

f3(x,y) – padded F -1[F3*(u,v)H(u,v)] 8 – rescaled8

Achim J. Lilienthal

Filters as Templates

Filters as Templates filters respond most strongly to patterns

that look like the filter

the kernel looks like the effect it is intended to detect

filtering has an analogy to computing a dot product measures similarity to the filter kernel

stronger response in brighter areas normalized correlation

filtering as changing the basis convolution can be seen as changing the base of an image

• base: vectors δ-functions base: shifted versions of the filter

• this process will typically loose information (coefficients on the new base can be redundant) but it might expose image structurein a useful way

Achim J. Lilienthal

Nyquist-Shannon Sampling Theorem

→ Contents

Achim J. Lilienthal

Aliasing / Undersampling, Moiré Pattern

decrease resolution

Achim J. Lilienthal

How to avoid Aliasing Problems? sampling

continuous function (irradiance in the camera) discrete grid

number of samples relative tothe function seems important a signal sampled too slowly is

misrepresented by the samples

from "Computer Vision – A Modern Approach", Forsyth and Ponce, Prentice Hall, 2002

Achim J. Lilienthal

Sampling a Signal in 1D

Reconstruction of the Original Continuous Signal which sample rate?

how to derive the continuous signal from the samples?

how to model the sampling process?

x… …

Achim J. Lilienthal

Sampling a Signal in 1D

How to Model the Sampling Process? continuous model of a sampled signal needed

sum of delta functions (2D: "bed-of-nails function") sampling process = multiplication with a sampling function fIII(x)

x x… …

… …

Achim J. Lilienthal

Considering Band-Limited Signals

signal band-width: band/range of non-zero frequencies

a band-limited signal is constrained in terms of how fast it can change

Achim J. Lilienthal

Fourier Transform of a Sampled Signal

sampling = multiplication with the sampling function in the spatial domain

f(x)⋅fIII(x)

x… …

Achim J. Lilienthal

equals convolution in the frequency domain

f(x)⋅fIII(x)

x… …

F(u)∗FIII(u)

→Fourier transform of a

"Dirac comb" is again a Dirac comb

( Poisson summation)

Achim J. Lilienthal

equals convolution in the frequency domain

f(x)⋅fIII(x)

x… …

F(u)∗FIII(u)

Achim J. Lilienthal

non-overlapping support of the "shifted Fourier Transforms" we can reconstruct the signal from the sampled versions

f(x)⋅fIII(x)

x… …

F(u)∗FIII(u)

Achim J. Lilienthal

Reconstruction of the Signal

Cut out by multiplication with box filterInverse

Fourier Transform

from "Computer Vision – A Modern Approach", Forsyth and Ponce, Prentice Hall, 2002

Achim J. Lilienthal

Reconstruction of the Signal but if support regions do overlap?

we can't reconstruct the signal

Fourier transform in the regions that overlap can't be determined

Achim J. Lilienthal

Fourier Transform of a "Dirac comb"

reciprocal behaviour of Δx and Δu

fIII(x)

… …

FIII(u)

… …

Achim J. Lilienthal

Sampling Theorem there should be no overlap

between the repetitions of the FT of the signal

the sampling interval should be at least the double of the highest frequency (1/w) present in the signal

wxxw 2112 ≤∆⇒∆≤⇒

FIII(u)

… …

1/Δx2 w

Part 3: Fourier Transform and Filtering in the Frequency...

Documents