EE465: Introduction to Digital Image Processing 1
Limitation of Imaging Technology Two plagues in image acquisition
Noise interference Blur (motion, out-of-focus, hazy
weather) Difficult to obtain high-quality
images as imaging goes Beyond visible spectrum Micro-scale (microscopic imaging) Macro-scale (astronomical imaging)
What is Noise? Wiki definition: noise means any
unwanted signal One person’s signal is another
one’s noise Noise is not always random and
randomness is an artificial term Noise is not always bad (see
stochastic resonance example in the next slide)EE465: Introduction to Digital
Image Processing 2
Stochastic Resonance
EE465: Introduction to Digital Image Processing 3
no noise heavy noiselight noise
EE465: Introduction to Digital Image Processing 4
Image Denoising Where does noise come from?
Sensor (e.g., thermal or electrical interference)
Environmental conditions (rain, snow etc.)
Why do we want to denoise? Visually unpleasant Bad for compression Bad for analysis
EE465: Introduction to Digital Image Processing 5
electrical interference
Noisy Image Examples
thermal imaging
ultrasound imagingphysical interference
EE465: Introduction to Digital Image Processing 6
(Ad-hoc) Noise Modeling Simplified assumptions
Noise is independent of signal Noise types
Independent of spatial location Impulse noise Additive white Gaussian noise
Spatially dependent Periodic noise
EE465: Introduction to Digital Image Processing 7
Noise Removal Techniques Linear filtering Nonlinear filtering
Recall
Linear system
EE465: Introduction to Digital Image Processing 8
Image Denoising Introduction Impulse noise removal
Median filtering Additive white Gaussian noise
removal 2D convolution and DFT
Periodic noise removal Band-rejection and Notch filter
EE465: Introduction to Digital Image Processing 9
Impulse Noise (salt-pepper Noise)
Definition
Each pixel in an image has the probability of p/2 (0<p<1) being contaminated by either a white dot (salt) or a black dot (pepper)
WjHi
jiXjiY
1,1
),(0
255),(
with probability of p/2with probability of p/2with probability of 1-p
noisy pixels
clean pixels
X: noise-free image, Y: noisy image
Note: in some applications, noisy pixels are not simply black or white, which makes the impulse noise removal problem more difficult
EE465: Introduction to Digital Image Processing 10
Numerical ExampleP=0.1
128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128
X
128 128 255 0 128 128 128 128 128 128 128 128 128 128 0 128 128 128 128 0 128 128 128 128 128 128 128 128 128 128 128 128 0 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 0 128 128 128 128 255 128 128 128 128 128 128 128 128 128 128 128 128 128 255 128 128 128 128 128 128 128 255 128 128
Y
Noise level p=0.1 means that approximately 10% of pixels are contaminated bysalt or pepper noise (highlighted by red color)
EE465: Introduction to Digital Image Processing 11
MATLAB Command>Y = IMNOISE(X,'salt & pepper',p)
Notes:
The intensity of input images is assumed to be normalized to [0,1].If X is double, you need to do normalization first, i.e., X=X/255;If X is uint8, MATLAB would do the normalization automatically The default value of p is 0.05 (i.e., 5 percent of pixels are contaminated) imnoise function can produce other types of noise as well (you need to change the noise type ‘salt & pepper’)
EE465: Introduction to Digital Image Processing 12
Impulse Noise Removal Problem
Noisy image Y
filteringalgorithm
Can we make the denoised image X as closeto the noise-free image X as possible?
^
X̂denoisedimage
EE465: Introduction to Digital Image Processing 13
Median Operator Given a sequence of numbers {y1,
…,yN} Mean: average of N numbers Min: minimum of N numbers Max: maximum of N numbers Median: half-way of N numbersExample ]56,55,54,255,52,0,50[y
54)( ymedian
]255,56,55,54,52,50,0[y
sorted
EE465: Introduction to Digital Image Processing 14
y(n)
W=2T+1
)](),...,(),...,([)(ˆ TnynyTnymediannx
1D Median Filtering
… …
Note: median operator is nonlinear
MATLAB command: x=median(y(n-T:n+T));
EE465: Introduction to Digital Image Processing 15
Numerical Example]56,55,54,255,52,0,50[yT=1:
],56,55,54,255,52,0,50,[ 5650y
]55,55,54,54,52,50,50[ˆ x
BoundaryPadding
EE465: Introduction to Digital Image Processing 16
x(m,n)
W: (2T+1)-by-(2T+1) window
2D Median Filtering
)],(),...,,(),...,,(),...,,(),...,,([),(ˆ
TnTmyTnTmynmyTnTmyTnTmymediannmx
MATLAB command: x=medfilt2(y,[2*T+1,2*T+1]);
EE465: Introduction to Digital Image Processing 17
Numerical Example 225 225 225 226 226 226 226 226 225 225 255 226 226 226 225 226 226 226 225 226 0 226 226 255 255 226 225 0 226 226 226 226 225 255 0 225 226 226 226 255 255 225 224 226 226 0 225 226 226 225 225 226 255 226 226 228 226 226 225 226 226 226 226 226
0 225 225 226 226 226 226 226 225 225 226 226 226 226 226 226 225 226 226 226 226 226 226 226 226 226 225 225 226 226 226 226 225 225 225 225 226 226 226 226 225 225 225 226 226 226 226 226 225 225 225 226 226 226 226 226 226 226 226 226 226 226 226 226
Y X̂
Sorted: [0, 0, 0, 225, 225, 225, 226, 226, 226]
EE465: Introduction to Digital Image Processing 18
Image ExampleP=0.1
Noisy image Y X̂denoisedimage
3-by-3 window
EE465: Introduction to Digital Image Processing 19
Image Example (Con’t)
3-by-3 window 5-by-5 window
clean noisy(p=0.2)
EE465: Introduction to Digital Image Processing 20
Reflections What is good about median
operation? Since we know impulse noise appears
as black (minimum) or white (maximum) dots, taking median effectively suppresses the noise
What is bad about median operation? It affects clean pixels as well Noticeable edge blurring after median
filtering
EE465: Introduction to Digital Image Processing 21
Idea of Improving Median Filtering Can we get rid of impulse noise
without affecting clean pixels? Yes, if we know where the clean pixels
areor equivalently where the noisy pixels
are How to detect noisy pixels?
They are black or white dots
EE465: Introduction to Digital Image Processing 22
Median Filtering with Noise Detection
Noisy image Y
x=medfilt2(y,[2*T+1,2*T+1]);Median filtering
Noise detectionC=(y==0)|(y==255);
xx=c.*x+(1-c).*y;Obtain filtering results
EE465: Introduction to Digital Image Processing 23
Image Example
clean noisy(p=0.2)
w/onoisedetection
withnoisedetection
EE465: Introduction to Digital Image Processing 24
Image Denoising Introduction Impulse noise removal
Median filtering Additive white Gaussian noise
removal 2D convolution and DFT
Periodic noise removal Band-rejection and Notch filter
EE465: Introduction to Digital Image Processing 25
Additive White Gaussian Noise
DefinitionEach pixel in an image is disturbed by a Gaussian random variableWith zero mean and variance 2
WjHiNjiN
jiNjiXjiY
1,1),,0(~),(
),,(),(),(2
X: noise-free image, Y: noisy image
Note: unlike impulse noise situation, every pixel in the image contaminatedby AWGN is noisy
EE465: Introduction to Digital Image Processing 26
Numerical Example2 =1
X Y
128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128 128
128 128 129 127 129 126 126 128 126 128 128 129 129 128 128 127 128 128 128 129 129 127 127 128 128 129 127 126 129 129 129 128 127 127 128 127 129 127 129 128 129 130 127 129 127 129 130 128 129 128 129 128 128 128 129 129 128 128 130 129 128 127 127 126
EE465: Introduction to Digital Image Processing 27
MATLAB Command
>Y = IMNOISE(X,’gaussian',m,v)
>Y = X+m+randn(size(X))*v;or
Note:rand() generates random numbers uniformly distributed over [0,1]randn() generates random numbers observing Gaussian distributionN(0,1)
EE465: Introduction to Digital Image Processing 28
Image Denoising
Noisy image Y
filteringalgorithm
Question: Why not use median filtering?Hint: the noise type has changed.
X̂denoisedimage
EE465: Introduction to Digital Image Processing 29
f(n) h(n) g(n)
)()()()()()()( nhnfnfnhknfkhngk
- Linearity
- Time-invariant property
)()()()( 22112211 nganganfanfa
)()( 00 nngnnf
Linear convolution
1D Linear Filtering
See review section
EE465: Introduction to Digital Image Processing 30
jwnenfwF )()(
dwewFnf jwn)(
21)(
forward
inverse
Note that the input signal is a discrete sequence while its FT is a continuous function
)()( nhnf time-domain convolution
)()( wHwFfrequency-domain multiplication
Fourier Series
EE465: Introduction to Digital Image Processing 31
Filter Examples
0 0.5 1 1.5 2 2.5 3 3.50
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
LP HP
w
|H(w)|Low-pass (LP)
h(n)=[1,1]
|h(w)|=2cos(w/2)
High-pass (LP)
h(n)=[1,-1]
|h(w)|=2sin(w/2)
EE465: Introduction to Digital Image Processing 32
forward transform inverse transform
• Properties- periodic
- conjugate symmetric
)()( kyNky
)()( * kykNy
1D Discrete Fourier Transform
Proof:
1
0
)()(N
n
knNWnxky
1
0
)()(N
n
knNWkynx
1
0
1
0
)( )()(N
n
knNn
N
n
nNkNn kyWxWxNky
1
0
*1
0
)( )()(N
n
knNn
N
n
nkNNn kyWxWxkNyProof:
}2exp{NjWN
EE465: Introduction to Digital Image Processing 33
Matrix Representation of 1D DFT
FA
NNN
N
NN
aa
aa
..............................
......
1
111
1,
,1
2
NN
Nj
N
klNkl
WeW
WN
a
Re
Im
NWklN
N
llk Wx
Ny
1
1
N
llklk xay
1
DFT:
EE465: Introduction to Digital Image Processing 34
Fast Fourier Transform (FFT)*
Invented by Tukey and Cooley in 1965 Basic idea: divide-and-conquer
Reduce the complexity of N-point DFT from O(N2) to O(Nlog2N)
122/12
22/2
12
)12(12
2
22
1
0
mn
kmNm
k
mn
kmNm
mn
mkNm
mn
mkNm
N
n
knNnk
WxWWx
WxWxWxy
N
N/2-point DFT N/2-point DFT
N-point DFT
EE465: Introduction to Digital Image Processing 35
Filtering in the Frequency Domain
)()()( nhnfng
convolution in the time domain is equivalent to multiplication in the frequency domain
)()()( kHkFkG
f(n) h(n) g(n) F(k) H(k) G(k)
DFT
EE465: Introduction to Digital Image Processing 36
2D Linear Filtering
f(m,n) h(m,n) g(m,n)
),(),(),(),(),(,
nmfnmhlnkmflkhnmglk
2D convolution
MATLAB function: C = CONV2(A, B)
EE465: Introduction to Digital Image Processing 37
2D Filtering=Two Sequential 1D Filtering
Just as we have observed with 2D transform, 2D (separable) filtering can be viewed as two sequential 1D filtering operations: one along row direction and the other along column direction
The order of filtering does not matter )()()()(),( 1111 mhnhnhmhnmh
h1 : 1D filter
EE465: Introduction to Digital Image Processing 38
Numerical Exampleh1(m)=[1,1], h1(n)=[1,-1]1D filter
1111
)()( 11 nhmh
1111
)()( 11 mhnh
MATLAB command: >h1=[1,1];h2=[1,-1];>conv2(h1,h2)>conv2(h2,h1)
EE465: Introduction to Digital Image Processing 39
Fourier Series (2D case)
m n
nwmwjenmfwwF )(21
21),(),(
Note that the input signal is discrete while its FT is a continuous function
),(),( nmhnmf spatial-domain convolution
),(),( 2121 wwHwwFfrequency-domain multiplication
EE465: Introduction to Digital Image Processing 40
Filter Examples Low-pass (LP)
h1(n)=[1,1]
|h1(w)|=2cos(w/2)1D
h(n)=[1,1;1,1]
|h(w1,w2)|=4cos(w1/2)cos(w2/2)2D
w1
w2
|h(w1,w2)|
EE465: Introduction to Digital Image Processing 41
Image DFT Example
Original ray image X choice 1: Y=fft2(X)
EE465: Introduction to Digital Image Processing 42
Image DFT Example (Con’t)
choice 1: Y=fft2(X) choice 2: Y=fftshift(fft2(X))Low-frequency at the centerLow-frequency at four corners
FFTSHIFT Shift zero-frequency component to center of spectrum.
EE465: Introduction to Digital Image Processing 43
Gaussian Filter
)2
exp(),( 2
22
21
21 wwwwH
)2
exp(),( 2
22
nm
nmh
FT
>h=fspecial(‘gaussian’, HSIZE,SIGMA);MATLAB code:
EE465: Introduction to Digital Image Processing 44
(=1)PSNR=24.4dB
Image Example
PSNR=20.2dB
noisy
(=25)
denoised denoised
(=1.5)PSNR=22.8dB
Matlab functions: imfilter, filter2
45
Gaussian Filter=Heat Diffusion
2
2
2
2 ),,(),,(),,(),,(y
tyxIx
tyxItyxIttyxI
Linear Heat Flow Equation:
)()0,,(),,( tGyxItyxI
scale A Gaussian filterwith zero mean and variance of t
Isotropic diffusion:
46
Basic Idea of Nonlinear Diffusion*
x
y I(x,y)
image I
image I viewed as a 3D surface (x,y,I(x,y))Diffusion should be anisotropicinstead of isotropic
Experimental Results
EE465: Introduction to Digital Image Processing 47
(Gaussian filtering)PSNR=24.4dBPSNR=20.2dB
noisy
(=25)
linear diffusion
(TV filtering)PSNR=27.5dB
nonlinear diffusion
Hammer-Nail Analogy
EE465: Introduction to Digital Image Processing 48
Gaussian filter
median filter
salt-pepper/impulse noise
Gaussian noise
periodic noise???
EE465: Introduction to Digital Image Processing 49
Image Denoising Introduction Impulse noise removal
Median filtering Additive white Gaussian noise
removal 2D convolution and DFT
Periodic noise removal Band-rejection and Notch filter
EE465: Introduction to Digital Image Processing 50
Periodic Noise Source: electrical or electromechanical
interference during image acquistion Characteristics
Spatially dependent Periodic – easy to observe in frequency
domain Processing method
Suppressing noise component in frequency domain
EE465: Introduction to Digital Image Processing 51
Image Example
spatial
Frequency (note the four pairs of bright dots)
EE465: Introduction to Digital Image Processing 52
Band Rejection Filter
otherwise
WDwwWDwwH1
220),(
22
21
21
w1
w2
EE465: Introduction to Digital Image Processing 53
Image Example
Before filtering After filtering
Advanced Denoising Techniques*
EE465: Introduction to Digital Image Processing 54
IN
Is
IEIW
jijiN III ,,1
jijiS III ,,1
jijiE III ,1,
jijiW III ,1,
][,1
, IcIcIcIcII WWEESSNNtji
tji
WESNdIgc dd ,,,||),(||
Basic idea: from linear diffusion (equivalent to Gaussian filtering)to nonlinear diffusion (with implicit edge-stopping criterion)