Spectral Domain Image Processing

● Continuous and Discrete Fourier Transform● DFT issues

– Aliasing

– Smoothing/Leakage

– Edge Effects

– …

● Solutions– Zero-padding

– Centering

– …

Book a bit slow start, but good if you are not familiar with the Fourier Transform.

I will use 1D examples, easy to visualize, principle generally applicable to 2D.

Wikipedia/Wikimedia commons

Fourier transform

NON-PERIODIC functions, with finite area under curve can be expressed as the integral of waves multiplied by a coefficient.

Images have finite extent.

Images have two dimensions, so not a 1D wave.

More like a ...

Fourier series+++

+

=

Joseph Fourier: The Fourier series decompose a PERIODIC continuous signal into waves with frequency, amplitude and phase.

c.f. e.g. Taylor series/expansion

Fig. 4.1 in book

Fourier transform

NON-PERIODIC functions, with finite area under curve can be expressed as the integral of waves multiplied by a coefficient.

Images have finite extent.

Images have two dimensions, so not a 1D wave.

More like a ...

Ruffled potato chips!

Wikimedia Commons

Defined for a sampled image f(x, y) of MxN pixels:

where x = 0, 1, 2…M-1, y = 0,1,2…N-1 and u = 0, 1, 2…M-1, v = 0, 1, 2…N-1.

F (u , v )=∑x=0

M−1

∑y=0

N −1

f ( x , y )e− j 2 π (ux /M +vy /N )

The 2D Discrete Fourier Transform

(Book: eq. 4.5-15)

Defined for a sampled image f(x, y) of MxN pixels:

where x = 0, 1, 2…M-1, y = 0,1,2…N-1 and u = 0, 1, 2…M-1, v = 0, 1, 2…N-1.

F (u , v )=∑x=0

M−1

∑y=0

N −1

f ( x , y )e− j 2 π (ux /M +vy /N )

The 2D Discrete Fourier Transform

(Book: eq. 4.5-15)

DFT (complex) IMAGE Wave: freq. u,v in x,y directioni.e. the ruffled potato chips

F(u,v) is also periodic!

F(0,0) and basis functions

A sum of corrugated surfaces with different frequency, amplitude and phase. Also remember

Euler’s formula.

e j θ=cosθ+ j sinθ

Amplitude and PhaseEach Fourier component (i.e. u,v coordinate) is a

complex number

C = Re + jIm , C* = Re - jIm

Amplitude : |F(u,v)| = (Re**2 + Im**2)**0.5

Phase : φ(u,v) = arctan(Im/Re)

Amplitude=Spectrum=Magnitude

atan2 in

Matlab!

= +

DFT ( )

Amplitude Phase

Reversible process!

The center F(u,v) is the average intensity of the image

Defined for a sampled image f(x, y) of MxN pixels:

How do you get back? Inverse transform! i.e. sum those sine and cosines at each x,y coordinate.

F (u , v )=∑x=0

M−1

∑y=0

N −1

f ( x , y )e− j 2 π (ux /M +vy /N )

Inverse DFT

f ( x , y )=1

MN∑u=0

M−1

∑v=0

N−1

F (u , v )e j2 π (ux / M+ vy/ N )

(Book: eq. 4.5-15)

(Book: eq. 4.5-16)

DFT (complex) IMAGE Wave: freq. u,v in x,y direction

+

A signal/function/image that can be represented by a finite number of Fourier components/terms.

Band limited function?

A Fourier transformed band limited function can be fully restored to its original. (μ=u)

Figure 4.6 in book

● DFT works on finite images with MxN pixels→ Frequency smoothing/leakage

● DFT uses discrete sampled images i.e. pixels→ Aliasing

● DFT assumes periodic boundary conditions→ Centering, Edge effects, Convolution

DFT effects

Images have borders, they are truncated (finite).This causes freq. smoothing and freq. leakage.

|F(u)|f(x)

Aliasing - short version

Images consists of pixels, they are sampled.

Too few pixels → fake signals (aliasing)!

Aliasing example (1D)

Figure 4.10 in book

Aliasing examples (2D)

Aliasing explained with DFT (1D)

Figure 4.6 in book

Aliasing explained with DFT (1D)

Can then be inversed to get spatial function back. Or do analysis.

Figure 4.8 in book

Aliasing explained with DFT (1D)

Figure 4.9 in book

Aliasing explained with DFT (1D)

Remember!

ΔT <1

2μmax

Figure 4.7 in book

Figure 4.10 in book

Aliasing explained with DFT - in 2D!

Figure 4.15 in book

Aliasing explained with DFT - in 2D!

and

Aliasing: Take home message

How do you avoid aliasing?1. Make sure signal is band limited. How?2. Sample with enough pixels!

What is enough pixels? Nyquist Theorem: The signal must be measured at least twice per period, i.e.:

Sampling theorem

“A continuous, band-limited function can be recovered with no error from a set of its samples if the sampling intervals exceed twice the highest frequency present.”

and

Quick pause to discuss

● What is frequency leakage? Why is it important?

● What is aliasing? Why is it important?How do you avoid aliasing?

● What do you think you can you do if your signal/image is not band-limited?

Centering: Looking at DFTs

DFT

Centering: Looking at DFTs

DFT

Centering: Looking at DFTs

Edge effects: Example

DFT

What do you see?

Edge effects: Example

Edge effects

● Spikes or lines in FT because of sharp-edged objects in image, spike is perpendicular to direction of the edge.

● Vertical spikes when there is a large difference in brightness between top and bottom of picture.

● Horizontal spikes when there is a large difference in brightness between left and right.

Edge effects

● Spikes or lines in FT because of sharp-edged objects in image, spike is perpendicular to direction of the edge.

● Vertical spikes when there is a large difference in brightness between top and bottom of picture.

● Horizontal spikes when there is a large difference in brightness between left and right.

Edge effects

● Spikes or lines in FT because of sharp-edged objects in image, spike is perpendicular to direction of the edge.

● Vertical spikes when there is a large difference in brightness between top and bottom of picture.

● Horizontal spikes when there is a large difference in brightness between left and right.

Quick pause to discuss

● What does centering mean?Why is it useful?

● Give an example of edge effects. Explain why this happens.

● What would the main Fourier strikes be if we Fourier transform an image of a “T”?

Fourier transform of characters

The convolution theorem is your friend!

Convolution in spatial domain is equivalent to multiplication in frequency domain!

Much more convenient!

Convolution (c.f. last week)

Convolution: Image vs DFT

Convolution is a multiplication in the spectral domain. However, we still do filtering in the spatial domain sometimes, why?

Convolution: Image vs DFT

A general linear convolution of N1xN1 image with N2xN2 convolving function (e.g. smoothing filter) requires in the image domain of order

N12N2

2 operations.

Instead using DFT, multiplication, inverse DFT one needs of order

4N2Log2N operations.

Here N is the smallest 2n number greater or equal to N1+N2-1.

Rule of thumb: Use Image convolution for small convolving functions, and DFT for large convolving functions.

Convolution example

What artifacts do you see in the convolved image?

Convolution: Wrap-around errors

● Why? DFT assumes periodic functions/data.● Avoid by using zero padding! How much needed?● Consider two NxM images. If image 1 is nonzero

over region N1xM1 and image 2 is nonzero over region N2xM2 then we will not get any wrap-around errors if

If the above is not true we need to zero-pad the images to make the condition true!

Convolution: Wrap-around errors

N should be at least 256+256-1=511

DFT lecture 1

● What is 1D continuous FT?● The 2D Discrete Fourier Transform● Important things in a discrete world:

– Freq. Smoothing and leakage

– Aliasing

– Centering

– Edge effects

– Convolution

● Coming up: Matlab exercises