46
Spectral Domain Image Processing ● Continuous and Discrete Fourier Transform ● DFT issues – Aliasing – Smoothing/Leakage – Edge Effects – … ● Solutions – Zero-padding – Centering – …
Spectral Domain Image Processing
● Continuous and Discrete Fourier Transform● DFT issues
– Aliasing
– Smoothing/Leakage
– Edge Effects
– …
● Solutions– Zero-padding
– Centering
– …
Book a bit slow start, but good if you are not familiar with the Fourier Transform.
I will use 1D examples, easy to visualize, principle generally applicable to 2D.
Wikipedia/Wikimedia commons
Fourier transform
NON-PERIODIC functions, with finite area under curve can be expressed as the integral of waves multiplied by a coefficient.
Images have finite extent.
Images have two dimensions, so not a 1D wave.
More like a ...
Fourier series+++
+
=
Joseph Fourier: The Fourier series decompose a PERIODIC continuous signal into waves with frequency, amplitude and phase.
c.f. e.g. Taylor series/expansion
Fig. 4.1 in book
Fourier transform
NON-PERIODIC functions, with finite area under curve can be expressed as the integral of waves multiplied by a coefficient.
Images have finite extent.
Images have two dimensions, so not a 1D wave.
More like a ...
Ruffled potato chips!
Wikimedia Commons
Defined for a sampled image f(x, y) of MxN pixels:
where x = 0, 1, 2…M-1, y = 0,1,2…N-1 and u = 0, 1, 2…M-1, v = 0, 1, 2…N-1.
F (u , v )=∑x=0
M−1
∑y=0
N −1
f ( x , y )e− j 2 π (ux /M +vy /N )
The 2D Discrete Fourier Transform
(Book: eq. 4.5-15)
Defined for a sampled image f(x, y) of MxN pixels:
where x = 0, 1, 2…M-1, y = 0,1,2…N-1 and u = 0, 1, 2…M-1, v = 0, 1, 2…N-1.
F (u , v )=∑x=0
M−1
∑y=0
N −1
f ( x , y )e− j 2 π (ux /M +vy /N )
The 2D Discrete Fourier Transform
(Book: eq. 4.5-15)
DFT (complex) IMAGE Wave: freq. u,v in x,y directioni.e. the ruffled potato chips
F(u,v) is also periodic!
F(0,0) and basis functions
A sum of corrugated surfaces with different frequency, amplitude and phase. Also remember
Euler’s formula.
e j θ=cosθ+ j sinθ
Amplitude and PhaseEach Fourier component (i.e. u,v coordinate) is a
complex number
C = Re + jIm , C* = Re - jIm
Amplitude : |F(u,v)| = (Re**2 + Im**2)**0.5
Phase : φ(u,v) = arctan(Im/Re)
Amplitude=Spectrum=Magnitude
atan2 in
Matlab!
= +
DFT ( )
Amplitude Phase
Reversible process!
The center F(u,v) is the average intensity of the image
Defined for a sampled image f(x, y) of MxN pixels:
How do you get back? Inverse transform! i.e. sum those sine and cosines at each x,y coordinate.
F (u , v )=∑x=0
M−1
∑y=0
N −1
f ( x , y )e− j 2 π (ux /M +vy /N )
Inverse DFT
f ( x , y )=1
MN∑u=0
M−1
∑v=0
N−1
F (u , v )e j2 π (ux / M+ vy/ N )
(Book: eq. 4.5-15)
(Book: eq. 4.5-16)
DFT (complex) IMAGE Wave: freq. u,v in x,y direction
+
A signal/function/image that can be represented by a finite number of Fourier components/terms.
Band limited function?
A Fourier transformed band limited function can be fully restored to its original. (μ=u)
Figure 4.6 in book
● DFT works on finite images with MxN pixels→ Frequency smoothing/leakage
● DFT uses discrete sampled images i.e. pixels→ Aliasing
● DFT assumes periodic boundary conditions→ Centering, Edge effects, Convolution
DFT effects
Images have borders, they are truncated (finite).This causes freq. smoothing and freq. leakage.
|F(u)|f(x)
Aliasing - short version
Images consists of pixels, they are sampled.
Too few pixels → fake signals (aliasing)!
Aliasing example (1D)
Figure 4.10 in book
Aliasing examples (2D)
Aliasing explained with DFT (1D)
Figure 4.6 in book
Aliasing explained with DFT (1D)
Can then be inversed to get spatial function back. Or do analysis.
Figure 4.8 in book
Aliasing explained with DFT (1D)
Figure 4.9 in book
Aliasing explained with DFT (1D)
Remember!
ΔT <1
2μmax
Figure 4.7 in book
Figure 4.10 in book
Aliasing explained with DFT - in 2D!
Figure 4.15 in book
Aliasing explained with DFT - in 2D!
and
Aliasing: Take home message
How do you avoid aliasing?1. Make sure signal is band limited. How?2. Sample with enough pixels!
What is enough pixels? Nyquist Theorem: The signal must be measured at least twice per period, i.e.:
Sampling theorem
“A continuous, band-limited function can be recovered with no error from a set of its samples if the sampling intervals exceed twice the highest frequency present.”
and
Quick pause to discuss
● What is frequency leakage? Why is it important?
● What is aliasing? Why is it important?How do you avoid aliasing?
● What do you think you can you do if your signal/image is not band-limited?
Centering: Looking at DFTs
DFT
Centering: Looking at DFTs
DFT
Centering: Looking at DFTs
Edge effects: Example
DFT
What do you see?
Edge effects: Example
Edge effects
● Spikes or lines in FT because of sharp-edged objects in image, spike is perpendicular to direction of the edge.
● Vertical spikes when there is a large difference in brightness between top and bottom of picture.
● Horizontal spikes when there is a large difference in brightness between left and right.
Edge effects
● Spikes or lines in FT because of sharp-edged objects in image, spike is perpendicular to direction of the edge.
● Vertical spikes when there is a large difference in brightness between top and bottom of picture.
● Horizontal spikes when there is a large difference in brightness between left and right.
Edge effects
● Spikes or lines in FT because of sharp-edged objects in image, spike is perpendicular to direction of the edge.
● Vertical spikes when there is a large difference in brightness between top and bottom of picture.
● Horizontal spikes when there is a large difference in brightness between left and right.
Quick pause to discuss
● What does centering mean?Why is it useful?
● Give an example of edge effects. Explain why this happens.
● What would the main Fourier strikes be if we Fourier transform an image of a “T”?
Fourier transform of characters
The convolution theorem is your friend!
Convolution in spatial domain is equivalent to multiplication in frequency domain!
Much more convenient!
Convolution (c.f. last week)
Convolution: Image vs DFT
Convolution is a multiplication in the spectral domain. However, we still do filtering in the spatial domain sometimes, why?
Convolution: Image vs DFT
A general linear convolution of N1xN1 image with N2xN2 convolving function (e.g. smoothing filter) requires in the image domain of order
N12N2
2 operations.
Instead using DFT, multiplication, inverse DFT one needs of order
4N2Log2N operations.
Here N is the smallest 2n number greater or equal to N1+N2-1.
Rule of thumb: Use Image convolution for small convolving functions, and DFT for large convolving functions.
Convolution example
What artifacts do you see in the convolved image?
Convolution: Wrap-around errors
● Why? DFT assumes periodic functions/data.● Avoid by using zero padding! How much needed?● Consider two NxM images. If image 1 is nonzero
over region N1xM1 and image 2 is nonzero over region N2xM2 then we will not get any wrap-around errors if
If the above is not true we need to zero-pad the images to make the condition true!
Convolution: Wrap-around errors
N should be at least 256+256-1=511
DFT lecture 1
● What is 1D continuous FT?● The 2D Discrete Fourier Transform● Important things in a discrete world:
– Freq. Smoothing and leakage
– Aliasing
– Centering
– Edge effects
– Convolution
● Coming up: Matlab exercises
