EEM 463 Introduction to Image Processing
Chapter 5Image Restoration and
Reconstruction
Fall 2017
Assist. Prof. Cihan Topal
Anadolu University, Dept. EEE
Slides Credit: Frank (Qingzhong) Liu
11/24/2017 2
Image Restoration
► Image restoration: recover an image that has beendegraded by using a prior knowledge of the degradationphenomenon.
► Model the degradation and applying the inverse process inorder to recover the original image.
► The principal goal of restoration techniques is to improvean image in some predefined sense.
► Although there are areas of overlap, image enhancement islargely a subjective process, while restoration is for themost part an objective process.
11/24/2017 3
A Model of Image Degradation/Restoration Process
►Degradation
Degradation function H
Degraded image is in the spatial domain.
Additive noise ),( yx
11/24/2017 4
A Model of Image Degradation/Restoration Process
If is a process, then
the degraded image is given in the spatial domain by
( , ) ( , ) ( , ) ( , )
H linear, position-invariant
g x y h x y f x y x y
11/24/2017 5
A Model of Image Degradation/Restoration Process
The model of the degraded image is given in
the frequency domain by
( , ) ( , ) ( , ) ( , )G u v H u v F u v N u v
11/24/2017 6
Basics of Filtering of the Frequency (Fourier) Domain
11/24/2017 7
11/24/2017 8
11/24/2017 9
Example: Phase Angles
11/24/2017 10
Example: Phase Angles and The Reconstructed
11/24/2017 11
2-D Convolution Theorem
1
0
1-D convolution
( ) ( ) ( ) ( )M
m
f x h x f m h x m
1 1
0 0
2-D convolution
( , ) ( , ) ( , ) ( , )M N
m n
f x y h x y f m n h x m y n
0,1,2,..., 1; 0,1,2,..., 1.x M y N
( , ) ( , ) ( , ) ( , )f x y h x y F u v H u v
( , ) ( , ) ( , ) ( , )f x y h x y F u v H u v
11/24/2017 12
The Basic Filtering in the Frequency Domain
Why is the spectrum at almost ±45 degree stronger than the spectrum at other directions?
11/24/2017 13
The Basic Filtering in the Frequency Domain
► Modifying the Fourier transform of an image
► Computing the inverse transform to obtain the processed result
1( , ) { ( , ) ( , )}
( , ) is the DFT of the input image
( , ) is a filter function.
g x y H u v F u v
F u v
H u v
11/24/2017 14
The Basic Filtering in the Frequency Domain
► In a filter H(u,v) that is 0 at the center of the transform and 1 elsewhere, what’s the output image?
11/24/2017 15
The Basic Filtering in the Frequency Domain
11/24/2017 16
Summary: Steps for Filtering in the Frequency Domain
1. Given an input image f(x,y) of size MxN, obtain the padding parameters P and Q. Typically, P = 2M and Q = 2N.
2. Form a padded image, fp(x,y) of size PxQ by appending the necessary number of zeros to f(x,y)
3. Multiply fp(x,y) by (-1)x+y to center its transform
4. Compute the DFT, F(u,v) of the image from step 3
5. Generate a real, symmetric filter function*, H(u,v), of size PxQ with center at coordinates (P/2, Q/2)
*generate from a given spatial filter, we pad the spatial filter, multiply the expaddedarray by (-1)x+y, and compute the DFT of the result to obtain a centered H(u,v).
11/24/2017 17
Summary: Steps for Filtering in the Frequency Domain
6. Form the product G(u,v) = H(u,v)F(u,v) using array multiplication
7. Obtain the processed image
8. Obtain the final processed result, g(x,y), by extracting the MxN region from the top, left quadrant of gp(x,y)
1( , ) ( , ) ( 1)x y
pg x y real G u v
An Example: Steps for Filtering in the Frequency Domain
11/24/2017 18
11/24/2017 19
Noise Sources
► The principal sources of noise in digital images arise during image acquisition and/or transmission
Image acquisition
e.g., light levels, sensor temperature, etc.
Transmission
e.g., lightning or other atmospheric disturbance in wireless network
11/24/2017 20
Noise Models (1)
►White noise
The Fourier spectrum of noise is constant
► With the exception of spatially periodic noise, we assume
Noise is independent of spatial coordinates
Noise is uncorrelated with respect to the image itself
11/24/2017 21
Noise Models (2)
Gaussian noiseElectronic circuit noise, sensor noise due to poor illumination and/or high temperature
Rayleigh noiseRange imaging
11/24/2017 22
Range Imaging (1)
Short for High Dynamic Range Imaging. HDRI is an imaging technique that allows for a greater dynamic range of exposure than would be obtained through any normal imaging process.
It is now popularly used to refer to the process of tone mapping* together with bracketed** exposures of normal digital images, giving the end result a high, often exaggerated dynamic range
* Tone mapping is a technique used in image processing and computer graphics to map a set of colours to another; often to approximate the appearance of HDRI in media with a more limited dynamic range** bracketing is the general technique of taking several shots of the same subject using different or the same camera settings
http://en.wikipedia.org/wiki/High_dynamic_range_imaginghttp://www.webopedia.com/TERM/H/High_Dynamic_Range_Imaging.html
11/24/2017 23
Range Imaging (2)
The intention of HDRI is to accurately represent the wide range of intensity levels found in real scenes ranging from direct sunlight to shadows
HDRI, also called HDR (High Dynamic Range) is a feature commonly found in high-end graphics and imaging software
http://en.wikipedia.org/wiki/High_dynamic_range_imaginghttp://www.webopedia.com/TERM/H/High_Dynamic_Range_Imaging.html
11/24/2017 24
Range Imaging: Examples (1)
Tower Bridge in Sacramento,
CA
http://en.wikipedia.org/wiki/High_dynamic_range_imaging
11/24/2017 25
Range Imaging: Examples (2)
Sydney Harbour Bridge HDRi produces greater detail and fewer shadowshttp://en.wikipedia.org/wiki/High_dynamic_range_imaging
11/24/2017 26Old Saint Paul’s Wellinton, New Zealand
11/24/2017 27
http://en.wikipedia.org/wiki/Image:HDRI-Example.jpg
11/24/2017 28
Noise Models (3)
Erlang (gamma) noise: Laser imaging
Exponential noise: Laser imaging
Uniform noise: Least descriptive; Basis for numerous random number generators
Impulse noise: quick transients,
such as faulty switching
11/24/2017 29
Gaussian Noise (1)
2 2( ) /2
The PDF of Gaussian random variable, z, is given by
1 ( )
2
z zp z e
where, represents intensity
is the mean (average) value of z
is the standard deviation
z
z
11/24/2017 30
Gaussian Noise (2)
2 2( ) /2
The PDF of Gaussian random variable, z, is given by
1 ( )
2
z zp z e
70% of its values will be in the range
95% of its values will be in the range
)(),(
)2(),2(
11/24/2017 31
Rayleigh Noise
2( ) /
The PDF of Rayleigh noise is given by
2( ) for
( )
0 for
z a bz a e z ap z b
z a
2
The mean and variance of this density are given by
/ 4
(4 )
4
z a b
b
11/24/2017 32
Erlang (Gamma) Noise
1
The PDF of Erlang noise is given by
for 0 ( ) ( 1)!
0 for
b baza z
e zp z b
z a
2 2
The mean and variance of this density are given by
/
/
z b a
b a
11/24/2017 33
Exponential Noise
The PDF of exponential noise is given by
for 0 ( )
0 for
azae zp z
z a
2 2
The mean and variance of this density are given by
1/
1/
z a
a
11/24/2017 34
Uniform Noise
The PDF of uniform noise is given by
1 for a
( )
0 otherwise
z bp z b a
2 2
The mean and variance of this density are given by
( ) / 2
( ) /12
z a b
b a
11/24/2017 35
Impulse (Salt-and-Pepper) Noise
The PDF of (bipolar) impulse noise is given by
for
( ) for
0 otherwise
a
b
P z a
p z P z b
If either or is zero, the impulse noise is calleda bP P
unipolar
if , gray-level will appear as a light dot,
while level will appear like a dark dot.
b a b
a
11/24/2017 36
11/24/2017 37
Examples of Noise: Original Image
11/24/2017 38
Examples of Noise: Noisy Images(1)
11/24/2017 39
Examples of Noise: Noisy Images(2)
11/24/2017 40
Periodic Noise
► Periodic noise in an image arises typically from electrical or electromechanical interference during image acquisition.
► It is a type of spatially dependent noise
► Periodic noise can be reduced significantly via frequency domain filtering
11/24/2017 41
An Examples of Periodic Noise
11/24/2017 42
Examples: Notch
Filters (1)
0
A Butterworth notch
reject filter D =3
and n=4 for all
notch pairs
11/24/2017 43
Examples: Notch Filters
(2)
11/24/2017 44
11/24/2017 45
Estimation of Noise Parameters (1)
The shape of the histogram identifies the closest PDF match
11/24/2017 46
Estimation of Noise Parameters (2)
Consider a subimage denoted by , and let ( ), 0, 1, ..., -1,
denote the probability estimates of the intensities of the pixels in .
The mean and variance of the pixels in :
s iS p z i L
S
S
1
0
12 2
0
( )
and ( ) ( )
L
i s i
i
L
i s i
i
z z p z
z z p z
11/24/2017 47
Restoration in the Presence of Noise Only
Spatial Filtering
Noise model without degradation
( , ) ( , ) ( , )
and
( , ) ( , ) ( , )
g x y f x y x y
G u v F u v N u v
11/24/2017 48
Spatial Filtering: Mean Filters (1)
Let represent the set of coordinates in a rectangle
subimage window of size , centered at ( , ).
xyS
m n x y
( , )
Arithmetic mean filter
1 ( , ) ( , )
xys t S
f x y g s tmn
11/24/2017 49
Spatial Filtering: Mean Filters (2)
1
( , )
Geometric mean filter
( , ) ( , )xy
mn
s t S
f x y g s t
Generally, a geometric mean filter achieves smoothing comparable to the arithmetic mean filter, but it tends to lose less image detail in the process
11/24/2017 50
Spatial Filtering: Mean Filters (3)
( , )
Harmonic mean filter
( , )1
( , )xys t S
mnf x y
g s t
It works well for salt noise, but fails for pepper noise.It does well also with other types of noise like Gaussian noise.
11/24/2017 51
Spatial Filtering: Mean Filters (4)
1
( , )
( , )
Contraharmonic mean filter
( , )
( , )( , )
xy
xy
Q
s t S
Q
s t S
g s t
f x yg s t
Q is the order of the filter.
It is well suited for reducing the effects of salt-and-pepper noise. Q>0 for pepper noise and Q<0 for salt noise.
11/24/2017 52
Spatial Filtering: Example (1)
11/24/2017 53
Spatial Filtering: Example (2)
11/24/2017 54
Spatial Filtering: Example (3)
11/24/2017 55
Spatial Filtering: Order-Statistic Filters (1)
( , )
Max filter
( , ) max ( , )xys t S
f x y g s t
( , )
Median filter
( , ) ( , )xys t S
f x y median g s t
( , )
Min filter
( , ) min ( , )xys t S
f x y g s t
11/24/2017 56
Spatial Filtering: Order-Statistic Filters (2)
( , )( , )
Midpoint filter
1 ( , ) max ( , ) min ( , )
2 xyxy s t Ss t Sf x y g s t g s t
11/24/2017 57
Spatial Filtering: Order-Statistic Filters (3)
( , )
Alpha-trimmed mean filter
1 ( , ) ( , )
xy
r
s t S
f x y g s tmn d
We delete the / 2 lowest and the / 2 highest intensity values of
( , ) in the neighborhood . Let ( , ) represent the remaining
- pixels.
xy r
d d
g s t S g s t
mn d
11/24/2017 58
11/24/2017 59
11/24/2017 60
11/24/2017 61
Spatial Filtering: Adaptive Filters (1)
Adaptive filters
The behavior changes based on statistical characteristics of the image inside the filter region defined by the mхn rectangular window.
The performance is superior to that of the filters discussed
11/24/2017 62
Adaptive Filters:
Adaptive, Local Noise Reduction Filters (1)
2
: local region
The response of the filter at the center point (x,y) of
is based on four quantities:
(a) ( , ), the value of the noisy image at ( , );
(b) , the variance of the noise corrupti
xy
xy
S
S
g x y x y
2
ng ( , )
to form ( , );
(c) , the local mean of the pixels in ;
(d) , the local variance of the pixels in .
L xy
L xy
f x y
g x y
m S
S
11/24/2017 63
Adaptive Filters:
Adaptive, Local Noise Reduction Filters (2)
2
2
The behavior of the filter:
(a) if is zero, the filter should return simply the value
of ( , ).
(b) if the local variance is high relative to , the filter
should return a value cl
g x y
ose to ( , );
(c) if the two variances are equal, the filter returns the
arithmetic mean value of the pixels in .xy
g x y
S
11/24/2017 64
Adaptive Filters:
Adaptive, Local Noise Reduction Filters (3)
2
2
An adaptive expression for obtaining ( , )
based on the assumptions:
( , ) ( , ) ( , ) L
L
f x y
f x y g x y g x y m
11/24/2017 65
11/24/2017 66
Adaptive Filters:
Adaptive Median Filters (1)
min
max
med
max
The notation:
minimum intensity value in
maximum intensity value in
median intensity value in
intensity value at coordinates ( , )
maximum all
xy
xy
xy
xy
z S
z S
z S
z x y
S
owed size of xyS
11/24/2017 67
Adaptive Filters:
Adaptive Median Filters (2)
med min med max
max
The adaptive median-filtering works in two stages:
Stage A:
A1 = ; A2 =
if A1>0 and A2<0, go to stage B
Else increase the window size
if window size , re
z z z z
S
med
min max
med
peat stage A; Else output
Stage B:
B1 = ; B2 =
if B1>0 and B2<0, output ; Else output
xy xy
xy
z
z z z z
z z
11/24/2017 68
Adaptive Filters:
Adaptive Median Filters (2)
med min med max
max
The adaptive median-filtering works in two stages:
Stage A:
A1 = ; A2 =
if A1>0 and A2<0, go to stage B
Else increase the window size
if window size , re
z z z z
S
med
min max
med
peat stage A; Else output
Stage B:
B1 = ; B2 =
if B1>0 and B2<0, output ; Else output
xy xy
xy
z
z z z z
z z
The median filter output is an impulse
or not
The processed point is an impulse or not
11/24/2017 69
Example:Adaptive Median Filters
Bilateral Filtering – 1 / 3
Bilateral Filtering – 2 / 3
Bilateral Filtering – 3 / 3
Bilateral Filtering - Example
11/24/2017 73
Input Gauss Filtered Bilateral Filtered
11/24/2017 74
Periodic Noise Reduction by Frequency
Domain Filtering
The basic idea
Periodic noise appears as concentrated bursts of energy in the Fourier transform, at locations corresponding to the frequencies of the periodic interference
Approach
A selective filter is used to isolate the noise
11/24/2017 75
Perspective Plots of Bandreject Filters
11/24/2017 76
A Butterworth bandreject filter of order 4, with the appropriate radius and
width to enclose completely the noise
impulses
11/24/2017 77
Perspective Plots of Notch Filters
11/24/2017 78
11/24/2017 79
Several interference components are present, the methods discussed in the preceding sections are not always acceptable because they remove much image information The components tend to have broad skirts that carry information about the interference pattern and the skirts are not always easily detectable.
11/24/2017 80
Optimum Notch Filtering
It minimizes local variances of the restored estimated
Procedure for restoration tasks in multiple periodic interference
Isolate the principal contributions of the interference pattern
Subtract a variable, weighted portion of the pattern from the corrupted image
( , )f x y
11/24/2017 81
Optimum Notch Filtering: Step 1
Extract the principal frequency components of the interference pattern
Place a notch pass filter at the location of each spike.
( , ) ( , ) ( , )NPN u v H u v G u v
1( , ) ( , ) ( , )NPx y H u v G u v
11/24/2017 82
Optimum Notch Filtering: Step 2 (1)
Filtering procedure usually yields only an approximation of the
true pattern. The effect of components not present in the estimate
of ( , ) can be minimized instead by subtracting from ( , )
a weighte
x y g x y
d portion of ( , ) to obtain an estimate of ( , ):
( , ) ( , ) ( , ) ( , )
x y f x y
f x y g x y w x y x y
One approach is to select ( , ) so that the variance of the estimate ( , )
is minimized over a specified neighborhood of every point ( , ).
w x y f x y
x y
11/24/2017 83
Optimum Notch Filtering: Step 2 (2)
2
2
The local variance of ( , ):
1( , ) ( , ) ( , )
(2 1)(2 1)
a b
s a t b
f x y
x y f x s y t f x ya b
11/24/2017 84
Optimum Notch Filtering: Step (3)
2
2
2
The local variance of ( , ):
1( , ) ( , ) ( , )
(2 1)(2 1)
( , ) ( , ) ( , )1
(2 1)(2 1) ( , ) ( , ) ( , )
(1
(2 1)(2 1)
a b
s a t b
a b
s a t b
f x y
x y f x s y t f x ya b
g x s y t w x s y t x s y s
a b g x y w x y x y
g
a b
2
, ) ( , ) ( , )
( , ) ( , ) ( , )
a b
s a t b
x s y t w x y x s y s
g x y w x y x y
Assume that w(x,y) remains essentially constant over the
neighborhood gives the approximation
w(x+s, y+t) = w(x,y)
11/24/2017 85
Optimum Notch Filtering: Step (4)
2
2
The local variance of ( , ):
( , ) ( , ) ( , )1( , )
(2 1)(2 1) ( , ) ( , ) ( , )
a b
s a t b
f x y
g x s y t w x y x s y sx y
a b g x y w x y x y
22
22
( , )To minimize ( , ) , 0
( , )
for ( , ), the result is
( , ) ( , ) ( , ) ( , ) ( , )
( , ) ( , )
x yx y
w x y
w x y
g x y x y g x y x yw x y
x y x y
11/24/2017 86
Optimum Notch Filtering: Example
11/24/2017 87
Optimum Notch Filtering: Example
11/24/2017 88
Optimum Notch Filtering: Example
11/24/2017 89
Optimum Notch Filtering: Example