EECE\CS 253 Image Processing
Richard Alan Peters II
Department of Electrical Engineering and Computer Science
Fall Semester 2011
Lecture Notes
. .
Lecture Notes: Reduction of Uncorrelated Noise
This work is licensed under the Creative Commons Attribution-Noncommercial 2.5 License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc/2.5/ or send a letter to Creative Commons, 543 Howard Street, 5th Floor, San Francisco, California, 94105, USA.
. .
Friday, April 21, 2023 2Friday, April 21, 2023 2 1999-2011 by Richard Alan Peters II
Noise in Images All images created through optical projection onto a sensor array are noisy.
Uncorrelated noise– Quantum noise in CCD arrays– Silver halide grains in film photography– Neuronal noise in a retina– Quantization noise in digital photographs
Correlated noise– Due to electrical interference– Due to source / sensor interference– Halftone distortion / moiré patterns
Friday, April 21, 2023 3Friday, April 21, 2023 3 1999-2011 by Richard Alan Peters II
Image with Additive Noise
( ) ( ) ( ), , , .r c r c r c= +J I N
undegraded image
undegraded image
noisy imagenoisy image
additive noise
additive noise
undegraded image
undegraded image
noisy imagenoisy image
additive noise
additive noise
( ) ( ) ( ), , , .v u v u v u= +J I N
spatial domain frequency domain
Friday, April 21, 2023 4Friday, April 21, 2023 4 1999-2011 by Richard Alan Peters II
Uncorrelated Noise
Intensity distributions – normalized histograms
Intensity distributions – normalized histograms
Each pixel’s value has probability of occurrence given by the associated distribution.
Friday, April 21, 2023 5Friday, April 21, 2023 5 1999-2011 by Richard Alan Peters II
Uncorrelated Noise
Each pixel’s value has probability of occurrence given by the associated distribution.
… black pixels occur 75% of the time and
… black pixels occur 75% of the time and
white pixels occur 25% of the time.
white pixels occur 25% of the time.
All values occur with equal probability.
All values occur with equal probability.
This is sparse noise: Only 12.5% of the pixels contain noise. Of those 12.5% …
This is sparse noise: Only 12.5% of the pixels contain noise. Of those 12.5% …
The most likely value is 128 with an average difference of 25 from 128 (std. dev.).
The most likely value is 128 with an average difference of 25 from 128 (std. dev.).
Intensity distributions – normalized histograms
Intensity distributions – normalized histograms
Friday, April 21, 2023 6Friday, April 21, 2023 6 1999-2011 by Richard Alan Peters II
Uncorrelated Color Noise: Gaussian
Friday, April 21, 2023 7Friday, April 21, 2023 7 1999-2011 by Richard Alan Peters II
Uncorrelated Color Noise: Uniform
Friday, April 21, 2023 8Friday, April 21, 2023 8 1999-2011 by Richard Alan Peters II
Gaussian IID Noise Field
= 128 = 32 = 128 = 32
IID: Independent, Identically Distributed
IID no spatial correlation
IID no spatial correlation
Friday, April 21, 2023 9
IID no spatial correlation
IID no spatial correlation
Friday, April 21, 2023 9 1999-2011 by Richard Alan Peters II
Gaussian IID Noise Field
… is this number divided by the number of pixels in the image.
… is this number divided by the number of pixels in the image.
= 128 = 32 = 128 = 32
IID: Independent, Identically Distributed
22
The probability of any one pixel having this value …
The probability of any one pixel having this value …
11
Friday, April 21, 2023 10Friday, April 21, 2023 10 1999-2011 by Richard Alan Peters II
Autocorrelation of an Image
( ) ( ) ( ) ( )( )
( ) ( ) ( )
( )( )( )
1
1 1
1
1 1
ρ, χ ψ ρ; ,ψ χ ;
where
ρ, χ ρ, χ
and
mod , if 0ψ ;
mod , if 0
R C
RCr c
R C
RCr c
r,c r R c C
r,c
x N xx N
x N N x
= =
= =
= + +
= -
ì ³ïï=íï + <ïî
å å
å å
IA I I
I I I
% %
%
Let the support of I be a torus. (I is defined on a torus a la the Fourier transform.)Let be I minus the mean value of I. Make a copy of . Shift the copy by () on the torus. Pixel-wise multiply the shifted version by the original and sum the products.
I I
AI(ρ,χ), the autocorrelation of I at offset (ρ,χ), is a measure of the similarity of I to itself when shifted by (ρ,χ).
AI(ρ,χ), the autocorrelation of I at offset (ρ,χ), is a measure of the similarity of I to itself when shifted by (ρ,χ).
Friday, April 21, 2023 11Friday, April 21, 2023 11 1999-2011 by Richard Alan Peters II
Power Spectrum & Autocorrelation of IID Noise
( ) ( ) 2PS = II F
Power SpectrumPower Spectrum AutocorrelationAutocorrelation
0 C/2-C/2
= 0
The autocorrelation is the inverse FT of the power spectrum.
The autocorrelation is the inverse FT of the power spectrum.
( ) ( ){ }21, Rer c -é ù= ê úë ûIA IF F
Friday, April 21, 2023 12Friday, April 21, 2023 12 1999-2011 by Richard Alan Peters II
Power Spectrum & Autocorrelation of IID Noise
Power SpectrumPower Spectrum AutocorrelationAutocorrelation
0 C/2-C/2
= 0
The autocorrelation of an IID noise image is ≈ δ(ρ,χ). That implies the PS is ≈ constant
The autocorrelation of an IID noise image is ≈ δ(ρ,χ). That implies the PS is ≈ constant
( ) ( ) 2PS = II F ( ) ( ){ }21, Rer c -é ù= ê úë ûIA IF F
Friday, April 21, 2023 13Friday, April 21, 2023 13 1999-2011 by Richard Alan Peters II
Gaussian noise fieldimage
Noise-Free Image and Uncorrelated Noise Field
Friday, April 21, 2023 14Friday, April 21, 2023 14 1999-2011 by Richard Alan Peters II
noise field center row log power spectrumimage center row log power spectrum
IID noise spectrum is flat.
IID noise spectrum is flat.
Image spectrum falls off.
Image spectrum falls off.
Spectra of Noise-Free Image and Uncorr. Noise Field
Friday, April 21, 2023 15Friday, April 21, 2023 15 1999-2011 by Richard Alan Peters II
image + noise field image + noise field center row log PS
Noise energy exceeds image energy beyond a certain freq.
Noise energy exceeds image energy beyond a certain freq.
( ) ( ) 22, ,v u v u>I N( ) ( ) 22
, ,v u v u<I N ( ) ( ) 22, ,v u v u<I N
Sum of Noise-Free Image and Uncorrelated Noise Field
Friday, April 21, 2023 16Friday, April 21, 2023 16 1999-2011 by Richard Alan Peters II
Power Spectra of Noise-Free Image and Noise Field
noise imageoriginal image
Friday, April 21, 2023 17Friday, April 21, 2023 17 1999-2011 by Richard Alan Peters II
original image noisy image
Power Spectra of Sum of Image and Noise Field
Friday, April 21, 2023 18Friday, April 21, 2023 18 1999-2011 by Richard Alan Peters II
blue indicates noise > imageoriginal image
Power Spectra of Sum of Image and Noise Field
Friday, April 21, 2023 19Friday, April 21, 2023 19 1999-2011 by Richard Alan Peters II
red indicates image > noisenoise image
Power Spectra of Sum of Image and Noise Field
Friday, April 21, 2023 20Friday, April 21, 2023 20 1999-2011 by Richard Alan Peters II
image & noisenoisy image
Power Spectra of Sum of Image and Noise Field
Friday, April 21, 2023 21Friday, April 21, 2023 21 1999-2011 by Richard Alan Peters II
Additive Noise: Another Example
original image noise image image+noise
Friday, April 21, 2023 22Friday, April 21, 2023 22 1999-2011 by Richard Alan Peters II
image PS noise PS image+noise PS
displayed: displayed:
( ){ }2log 1+IFAdditive Noise: Another Example
Friday, April 21, 2023 23Friday, April 21, 2023 23 1999-2011 by Richard Alan Peters II
image PS image PS > noise PSimage+noise PS
Additive Noise: Another Example displayed: displayed:
( ){ }2log 1+IF
Friday, April 21, 2023 24Friday, April 21, 2023 24 1999-2011 by Richard Alan Peters II
red indicates image > noise image PS > noise PS
Additive Noise: Reduce Through Blurring?
f0f0f0
At some frequency, f0, there are more components where the noise power is greater than the image power.
At some frequency, f0, there are more components where the noise power is greater than the image power.
Friday, April 21, 2023 25Friday, April 21, 2023 25 1999-2011 by Richard Alan Peters II
passpass
rejectreject
pass
reject
red indicates image > noise image PS > noise PS
Additive Noise: Reduce Through Blurring?
Thus, it makes sense to apply a LPF with cutoff f0, (a blurring filter) to the images and see what happens.
Thus, it makes sense to apply a LPF with cutoff f0, (a blurring filter) to the images and see what happens.
Friday, April 21, 2023 26Friday, April 21, 2023 26 1999-2011 by Richard Alan Peters II
PS of Gaussian blurred image Gaussian Blurred Image
Additive Noise: Reduction Through Blurring.
The result is actually no better. There’s less noise but the blurring looks worse.
The result is actually no better. There’s less noise but the blurring looks worse.
Friday, April 21, 2023 27Friday, April 21, 2023 27 1999-2011 by Richard Alan Peters II
PS of Gaussian blurred image Gaussian Blurred Image
Additive Noise: Reduction Through Blurring.
The result is actually no better. There’s less noise but the blurring looks worse.
The result is actually no better. There’s less noise but the blurring looks worse.
Friday, April 21, 2023 28Friday, April 21, 2023 28 1999-2011 by Richard Alan Peters II
red: image > noise blue: image < noise
power spec. of noisy image
Noise Masking
If the freq. comps. for which noise-power > image-power are known1…
If the freq. comps. for which noise-power > image-power are known1…
1of course they almost never are.1of course they almost never are.
Friday, April 21, 2023 29Friday, April 21, 2023 29 1999-2011 by Richard Alan Peters II
Noise Masking
… mask those out of the FT and invert the result to get…
… mask those out of the FT and invert the result to get…
image < noise masked outpower spec. of noisy image
Friday, April 21, 2023 30Friday, April 21, 2023 30 1999-2011 by Richard Alan Peters II
noisy image noise-masked mage
Noise Masking
… this:… this:
Friday, April 21, 2023 31Friday, April 21, 2023 31 1999-2011 by Richard Alan Peters II
blurred noisy image noise-masked mage
Noise MaskingAlthough the noise-masked image looks better than the blurred one, it is still noisy. Moreover, this example is unrealistic because we know the exact noise power spectrum. In any real case we will at most know its statistics.
Friday, April 21, 2023 32Friday, April 21, 2023 32 1999-2011 by Richard Alan Peters II
Image Degradation Model
( ) ( ) ( ) ( ), , , , .r c r c r c r c= * +J I H N
undegraded image
undegraded image
additive noise
additive noise
degraded image
degraded image
pointspread functionpointspread function
So far, we have considered only additive noise. Before going further it will be useful to consider a more general model of image degradation, one that includes convolution with a pointspread1 function, H, as well as additive noise.
1H is also referred to as the optical transfer function.
Friday, April 21, 2023 33Friday, April 21, 2023 33 1999-2011 by Richard Alan Peters II
Pointspread OperatorsA pointspread operator is a linear model of the distortion acquired during the imaging process. Since it is a linear model, it is a convolution operator. One example of this is aperture distortion, an unavoidable consequence of making an image with a camera that has an opening larger than a point.
Friday, April 21, 2023 34Friday, April 21, 2023 34 1999-2011 by Richard Alan Peters II
Pointspread Operators
pinhole camera aperture camera
A pinhole camera maps one object point to one image point; it is one-to-one.
An aperture camera maps one object point to many image points; it spreads the points.
Friday, April 21, 2023 35Friday, April 21, 2023 35 1999-2011 by Richard Alan Peters II
Pointspread Operators and Convolution , , ,r c r c r cJ I H
,r cH
,r cI
Recall how a convolution works through multiply, shift, and add (See Lect. 7 p. 25ff). That is precisely the effect of imaging through an aperture. It results in a blurry image.
Friday, April 21, 2023 36Friday, April 21, 2023 36 1999-2011 by Richard Alan Peters II
Lenses A properly designed lens will focus the light emanating from a point and thereby reduce the blurring. But no lens can do this perfectly. In fact, the lens adds its own distortion. The result is an optical transfer function, H(r,c), that is convolved with the image.
Friday, April 21, 2023 37Friday, April 21, 2023 37 1999-2011 by Richard Alan Peters II
Image Degradation Model
( ) ( ) ( ) ( )J , I , H , N , .r c r c r c r c= * +
undegraded image
undegraded image
additive noise
additive noise
degraded image
degraded image
pointspread functionpointspread function
Note: The term pointspread operator refers to convolution by the pointspread function.
Friday, April 21, 2023 38Friday, April 21, 2023 38 1999-2011 by Richard Alan Peters II
Image Degradation Model
( )I ,r c
( )J ,r c
( )N ,r c
( ) ( )I , H ,r c r c*
( )H ,r c J , I , H , N ,r c r c r c r c
Friday, April 21, 2023 39Friday, April 21, 2023 39 1999-2011 by Richard Alan Peters II
Image Degradation Model (Frequency Domain)
( ) ( ) ( ) ( ), , , , .v u v u v u v u= × +J I H N
undegraded image
undegraded image
additive noise
additive noise
degraded image
degraded image
pointspread operatorpointspread operator
Friday, April 21, 2023 40Friday, April 21, 2023 40 1999-2011 by Richard Alan Peters II
Image Degradation Model (Frequency Domain)
( ),v uI
( ),v uJ
( ),v uN
( ) ( ), ,v u v u×I H
( ),v uIImages shown are log magnitude.
( ) ( ) ( ) ( ), , , , .v u v u v u v u= × +J I H N
Friday, April 21, 2023 41Friday, April 21, 2023 41 1999-2011 by Richard Alan Peters II
Let I be a perfect image and let K be the image convolved with a pointspread function, H. Then in the frequency domain:
Image Restoration
( ) ( ) ( ), , , .v u v u v u=K W J%
( ) ( ) ( ), , , .u v u v u v= +J K N
( ) ( ) ( ), , , .u v u v u v=K I H
( )( )( )
( ) ( )( )
, , ,, .
, ,
u v u v u vu v
u v u v= =K W J
IH H
%%
If the process of imaging adds noise then we get J = K + N, or in freq.:
We want a filter, W, to remove as much of the noise from J as possible:
Then an estimate of I would be the inverse Fourier transform of
We want to find the filter, W, that results in the closest possible estimate of I i.e. the W that minimizes the energy of the difference between the estimate and I.That is we want to find W such that22 dudve = -òò I I%
is as small as possible. This is called least mean squared (LMS) minimization.
Friday, April 21, 2023 42Friday, April 21, 2023 42 1999-2011 by Richard Alan Peters II
Image RestorationThere are a number of ways to solve for the minimum squared error. All make use of the assumption that the image and the noise are uncorrelated. Depending on how that fact is used, slightly different solutions are found. The most common one used in image processing is the Wiener filter:
2*
22 2.=
+
H IW
H I N
2 *
22
2
.+
=
+
H I H NWJ
NH
I
For frequencies (u,v) where noise power is smaller than the image power acts like an inverse filter since (u,v)/(u,v) < 1 and
( ) ( ) ( )2
2, , , ,u v u v u v» =H
WJ I IH
Then, with a little bit of algebra, we get
and at frequencies where the noise power dominates, (u,v)/(u,v) > 1 and
( ) ( )2 *
22 2, , ,u v u v=
+
I HWJ N
I H N
the fraction is small so the noise power is diminished.
Friday, April 21, 2023 43
Image Restoration
Friday, April 21, 2023 43 1999-2011 by Richard Alan Peters II
This is one of the possible derivations of the Wiener filter
This is one of the possible derivations of the Wiener filter
( )
( )
( )[ ] ( )[ ]
( ) ( ) ( )
( ) ( ){ }( ) ( )
22
2
22
22
2
222
222
222 2
1
1 1
1 1 1
1 2Re 1
1 2Re 1
dudv
dudv
dudv
dudv
dudv
dudv
dudv
dudv d
e
-
-
-
-
-
- -
= -
= -
= - +
= - +
= - + - +
é ù= - + - + - +ê úë ûé ù= - + - +ê úë û
é ù= - + + -ê úë û
òò
òò
òòòòòòòòòò
òò
I I
K WJH H
H K W K N
H K W WN
H K W WN K W WN
H K W K W WN WN K W WN
H K W K W WN WN
H K W WN H W WKN
%
udvòò
Friday, April 21, 2023 44
Image Restoration
Friday, April 21, 2023 44 1999-2011 by Richard Alan Peters II
( ) ( )222 22 1 2Re 1 .dudv dudve - -é ù= - + + -ê úë ûòò òòH K W WN H W WKN
( )12Re 1 ,dudv- -òòH W W I N
1
0.dudv =òòI N
( )2222 1 .dudve - é ù= - +ê úë ûòòH K W WN
From the previous page, the squared error is
The second term should be small compared to the first since it can be written
and the image and the noise are assumed to be uncorrelated1. Thus the error can be approximated by
The mean squared error, ε2, is minimized when W is given by,2*
22 2.=
+
H IW
H I N
Friday, April 21, 2023 45Friday, April 21, 2023 45 1999-2011 by Richard Alan Peters II
Gaussian noise fieldimage
Noise Reduction Through LMS Filtering1
1Here the PSF is the identity.
Friday, April 21, 2023 46Friday, April 21, 2023 46 1999-2011 by Richard Alan Peters II
noisy imageimage
Noise Reduction Through LMS Filtering1
1Here the PSF is the identity.
Friday, April 21, 2023 47Friday, April 21, 2023 47 1999-2011 by Richard Alan Peters II
Additive Noise (Power Spectra)
noisy imageoriginal image
Friday, April 21, 2023 48Friday, April 21, 2023 48 1999-2011 by Richard Alan Peters II
Additive Noise (Power Spectra)
Wiener filternoisy image
In this example we knew the exact image and noise power spectra and the PSF was the identity because the image is synthetic. In a real example, none of that is true.
In this example we knew the exact image and noise power spectra and the PSF was the identity because the image is synthetic. In a real example, none of that is true.
Friday, April 21, 2023 49Friday, April 21, 2023 49 1999-2011 by Richard Alan Peters II
Additive Noise (Power Spectra)
Wiener filternoisy image
Friday, April 21, 2023 50Friday, April 21, 2023 50 1999-2011 by Richard Alan Peters II
Additive Noise (Power Spectra)
original imageWiener filtered image
Friday, April 21, 2023 51Friday, April 21, 2023 51 1999-2011 by Richard Alan Peters II
Additive Noise
Wiener filtered imagenoisy image
Friday, April 21, 2023 52Friday, April 21, 2023 52 1999-2011 by Richard Alan Peters II
Wiener filtered image
Additive Noise
original image
Friday, April 21, 2023 53Friday, April 21, 2023 53 1999-2011 by Richard Alan Peters II
Gaussian blurred image
Additive Noise
original image
Friday, April 21, 2023 54Friday, April 21, 2023 54 1999-2011 by Richard Alan Peters II
noisy image J = I*h + Nimage
Noise Reduction Through LMS Filtering1
1Least Mean Squared. PSF, h, is Gaussian =0, =2.
Friday, April 21, 2023 55Friday, April 21, 2023 55 1999-2011 by Richard Alan Peters II
Image*PSF + Noise (Power Spectra)
original image noisy image J = I*h + N
Friday, April 21, 2023 56Friday, April 21, 2023 56 1999-2011 by Richard Alan Peters II
Wiener filterWiener filtered image
In this example we knew the exact image and noise power spectra and the PSF was Gaussian w/ μ=0, σ=2. In a real example, none of that is true.
In this example we knew the exact image and noise power spectra and the PSF was Gaussian w/ μ=0, σ=2. In a real example, none of that is true.
Image*PSF + Noise (Power Spectra)
Friday, April 21, 2023 57Friday, April 21, 2023 57 1999-2011 by Richard Alan Peters II
Image*PSF + Noise (Power Spectra)
Wiener filterWiener filtered image
Friday, April 21, 2023 58Friday, April 21, 2023 58 1999-2011 by Richard Alan Peters II
Image*PSF + Noise (Power Spectra)
original imageWiener filtered image
Friday, April 21, 2023 59Friday, April 21, 2023 59 1999-2011 by Richard Alan Peters II
Image*PSF + Noise
Wiener filtered imagenoisy image J = I*h + N
Friday, April 21, 2023 60Friday, April 21, 2023 60 1999-2011 by Richard Alan Peters II
Wiener filtered image
Image*PSF + Noise
original image
Friday, April 21, 2023 61Friday, April 21, 2023 61 1999-2011 by Richard Alan Peters II
LMS Image Restoration (Real Example)
For this real example we need to estimate the image power spectrum, the pointspread function and the noise power spectrum.
For this real example we need to estimate the image power spectrum, the pointspread function and the noise power spectrum.
Friday, April 21, 2023 62Friday, April 21, 2023 62 1999-2011 by Richard Alan Peters II
LMS Image Restoration (Real Example)
To estimate the noise power spectrum, analyze a constant area from the image.
To estimate the noise power spectrum, analyze a constant area from the image.
Friday, April 21, 2023 63Friday, April 21, 2023 63 1999-2011 by Richard Alan Peters II
Noise Estimation
+ -+
original blurred w/ Gaussian σ=5
differenceσR = 5.0981 σG = 4.0672σB = 6.9212
Find the std. deviations of each band:
Friday, April 21, 2023 64Friday, April 21, 2023 64 1999-2011 by Richard Alan Peters II
Pointspread Function Estimation
To estimate the PSF, find the image of a point and construct a convo-lution mask from it.
To estimate the PSF, find the image of a point and construct a convo-lution mask from it.
Friday, April 21, 2023 65Friday, April 21, 2023 65 1999-2011 by Richard Alan Peters II
Wiener Filter Estimation
W
2N
2I
2H
Friday, April 21, 2023 66Friday, April 21, 2023 66 1999-2011 by Richard Alan Peters II
LMS Image Restoration (original)
Friday, April 21, 2023 67Friday, April 21, 2023 67 1999-2011 by Richard Alan Peters II
LMS Image Restoration (filtered)
Friday, April 21, 2023 68Friday, April 21, 2023 68 1999-2011 by Richard Alan Peters II
Detail of Results
filtered imageoriginal image matlab’s wiener2
The contrast of these has been increased to make the differences more visible.
The contrast of these has been increased to make the differences more visible.