1
Vladimir Lukin 31/08/2009
Processing of Images Based on Blind Evaluationof Noise Type and Characteristics
Processing of ImagesBased on Blind Evaluation
of Noise Type and CharacteristicsVladimir V. Lukina, Sergey K. Abramova, Nikolay N. Ponomarenkoa,
Mikhail L. Ussa,b, Benoit Vozelb, Kacem Chehdib,Jaakko Astolac
a National Aerospace University, 61070, Kharkov, Ukraine;b University of Rennes I, 22 305 Lannion cedex, BP 80518, France; c Tampere University of Technology, Institute of Signal Processing,
P.O. Box-553, FIN-33101, Tampere, Finland
2
Vladimir Lukin 31/08/2009
Contents
Contents
• Introduction• A priori Knowledge on Noise Statistics, Its Use in Image Processing• Examples of False Preliminary Assumptions on Noise Type• General Aspects of Image Processing• Methods for Blind Determination of Noise Type• Methods for Blind Determination of Noise Statistical Characteristics• Edge Detection Example• Requirements to Methods of Blind Determination of Noise Statistical Characteristics• Image Filtering Examples• Noisy Image Compression• Automatic Procedure for Lossy Compression• Accuracy of Blind Estimation Methods• Conclusions
3
Vladimir Lukin 31/08/2009
Introduction
Applications: Systems for single- or multi-channel remote sensing (RS) data processing
Goal: Carrying out analysis of mixed noise influence for three particular operations of image processing: edge detection, filtering and compression of noisy images
Additivenoise
Influence on: edge detection, filtering and compression
Reasons: transmitting and encoding/decoding
errors
Multiplicative noise
Reason: circuit and atmospheric
noise
Blind noise parameters evaluation→ inaccurate estimates may occur
Impulse noise
Reasons: insufficient observation interval,
speckle effects
Image processing
Blind noise type determination→ false decisions may occur
RS image
4
Vladimir Lukin 31/08/2009
A priori Knowledge on Noise Statistics, Its Use in Image Processing
Characteristics of noise:
1. Noise type (additive, signal-dependent, multiplicative, impulse, mixed, etc.);
2. PDF, variance, relative variance, probability of impulse noise, etc.;
3. Is noise i.i.d. or spatially correlated; 2D ACF or spectrum in the latter case.
Examples:
1. Noise PDF is needed to design optimal edge detectors and to set thresholds for most known edge detectors;
2. Noise statistics is used in image filtering (wavelets, DCT, sigma, Lee, Frost, Kuan) and reconstruction (Katsaggellos et al);
3. Lossy compression of noisy images.
Peculiarities:
There are no universal methods for coping well with any type and level of noise.
Specialized methods that exploit available information on noise type and its characteristics in full extent as a rule provide the most efficient processing.
5
Vladimir Lukin 31/08/2009
A priori Knowledge on Noise Statistics, Its Use in Image Processing
Statement:Information on noise type and statistics is to be available or it should be
retrieved for data (image) at hand.
Otherwise, a used method has to exhibit certain robustness, i.e. ability to perform well enough in conditions of some unknown parameters and characteristics of noise.
Fortunately, some methods for blind noise model identification and estimation of its parameters exist and they are rapidly developing*.
Goal:Carrying out analysis of these methods for three particular operations of image
processing, namely, edge detection, filtering and compression of noisy images.
* B. Vozel, S. Abramov, K. Chehdi, V. Lukin, N. Ponomarenko, M. Uss, Blind methods for noise evaluation in multi-component images, book chapter in “Multivariate image processing”, ISTE Ltd, France, 2009 (in press).
6
Vladimir Lukin 31/08/2009
Examples of False Preliminary Assumptions on Noise Type: Optical Images
The noisy (dominant Poisson) test image (left) and this image after Gamma-correction (γ = 0.6) (right)
Scatter-plots of local estimates of variance for original noisy (left) and corrected images (right)
2ˆloc
l̂ocI
2ˆloc
l̂ocI
7
Vladimir Lukin 31/08/2009
Examples of False Preliminary Assumptions on Noise Type: Radar Images
A real life K-band SLAR image
Scatter-plot of local estimation pairs (variance and squared mean) with two approximation
lines fitted by expert (green line) and automatically (red line)
Other examples: a model of fully developed speckle, noise in ultrasound images, noise in sub-band
images of hyperspectral remote sensing data.
2ˆloc
2l̂ocI
8
Vladimir Lukin 31/08/2009
General Aspects of Image Processing
1. Noise is a phenomenon inherent for any imaging and RS system;
2. Noise presence is often a dominant factor that prevents solving final tasks for which obtained data are intended for;
3. These tasks could be: object detection and localization, sensed terrain classification, segmentation, edge detection, image filtering, segmentation, compression, classification, etc.
4. There are some methods of image processing that do not exploit information on noise type and characteristics (like standard median filter) but methods that use a priori information on noise type and characteristics are able to perform better;
5. There is a lot of different types of noise and their combinations.
9
Vladimir Lukin 31/08/2009
Methods for Blind Determination of Noise Type
An automatic system to identify the nature of the degradation affecting an image and select the appropriate algorithm of its processing has been designed*.
The system is intended for identifying four sources of degradation made of three different noises: additive, multiplicative and impulse noise and a defocusing blur.
Thirteen observation models have been deduced from these four sources: each noise type separately and different possible combinations.
The system has been tested for different simulated data and provided appropriately high probability of correct identification of noise/distortion type.
* Carton-Vandecandelaere, M.-P., Vozel, B., Klaine, L., Chehdi, K., Application to Multispectral Images of a Blind Identification System for Blur, Additive, Multiplicative and Impulse Noises. Proceedings of EUSIPCO, III, 2002, pp. 283-286.
10
Vladimir Lukin 31/08/2009
Methods for Blind Determination of Noise Statistical Characteristics
There are methods for blind estimation of: additive noise variance*;
mixed noise parameters***.
* V. Lukin, S. Abramov, B. Vozel, K. Chehdi, J. Astola, Segmentation-based method for blind evaluation of noise variance in images, SPIE Journal on Applied Remote Sensing, Vol. 2, Aug. 2008, 15 p.
** Lukin V.V., Abramov S.K., Ponomarenko N.N., Vozel B., Chehdi K., Methods for blind evaluation of noise variance in multichannel optical and radar images, Telecommunications and Radio Engineering, Vol. 65 (6), 2006, pp. 509-537.
*** A. Foi, Pointwise Shape-Adaptive DCT Image Filtering and Signal-Dependent Noise Estimation: Thesis for the degree of Doctor of Technology, Tampere University of Technology, Tampere, Finland, Dec. 2007.
They are based on different principles: Curve fitting to scatter-plots;
Analysis of local estimate histograms;
Data analysis in spectral domain;
Bit-plane analysis, etc.
multiplicative noise variance**;
11
Vladimir Lukin 31/08/2009
Edge Detection Example
Noise free (a) and noisy (mixed multiplicative and impulse noise, ) (b) test images; edge maps obtained by Sobel (c) and normalized quasirange (d) edge detectors
Principle:For most of edge detectors, a local parameter is calculated for each pixel
using image values in a given pixel and its neighbors. Then this local parameter is compared to a preset threshold that takes into account noise type and statistics.
2 0.003, 0.01P а b c d
Example:
12
Vladimir Lukin 31/08/2009
Edge Detection
Requirement to accuracy:It is desirable to provide within the limits from till .
Procedure: to estimate local mean; to calculate ; to determine the corresponding local threshold; to obtain a detector output; to compare it to the local threshold; to undertake a decision.
2ˆn 20.9 n 21.1 n
Signal-dependent noise case:
If a dependence of local variance on local mean is known in advance or pre-estimated, then a threshold for an edge detector should be set locally as
or
2 ( )loc locf I
1loc locT 22loc locT
2 ˆˆ ( )loc locf I
13
Vladimir Lukin 31/08/2009
Requirements to Methods of Blind Determination of Noise Statistical Characteristics
1. To produce unbiased estimates;
2. To provide appropriately accurate estimates (characterized by small variance of noise parameter estimation);
3. To be applicable to images of different content and to noise of different level and with different spatial characteristics (a wide range of noise parameters’ variation can be observed in practice);
4. To be fast enough.
Note: if an image is multichannel, processing is component-wise and it can be parallel.
A question is what are appropriately accurate estimates?
14
Vladimir Lukin 31/08/2009
Image Filtering Examples:Test Images and Quantitative Criteria
Goal:Consider how filtering efficiency depends upon accuracy of additive noise
variance estimation.
Test images: Quantitative criteria: An error of additive noise variance
estimation:
Variation of filtering efficiency
2 2 2 2ˆ ˆ( ) /n n n
2 2 2ˆ( ( ) ( )) / ( )n n nMSE MSE MSE
Where is MSE provided under condition that is set as the parameter of a used filter.
2( )nMSE 2n
Test01 Test02
15
Vladimir Lukin 31/08/2009
Image Filtering Examples:Modified Sigma Filter
The plots vs for the test images for 100 and 2252 n
Preliminary conclusion:For the image Test01, smaller output MSE can be reached if when better
edge/detail/texture preservation is provided.
The opposite situation is observed for the image Test02. If , smaller output MSE is produced. Thus, does not necessarily lead to the best efficiency of noise suppression (according to output MSE criterion).
If is within the limits from 0.8 to 1.2, then it is almost guaranteed that one is in the neighborhood of minimal output MSE. This is observed for other filters as well as for multiplicative noise.
2 2ˆn n
2̂
2 2ˆ n n 2 2ˆ n n
2 2ˆn n
2̂
The noise variance was estimated in automatic manner robust to the presence of impulses. The resulting estimate is 24.93.
16
Vladimir Lukin 31/08/2009
Image Filtering Examples:Blind Noise Type and Variance Retrieval
The original NOAA image (a), filtered image (b), and the obtained impulse noise map (c)
2ˆ n
Real-life image:
a
b
c
Identification result:
Image is identified as corrupted by mixed additive and impulse noise.
Impulse noise is specific, spatially correlated, where several neighboring image pixels in rows are corrupted.
Estimation result:
17
Vladimir Lukin 31/08/2009
Image Filtering Examples:Blind Noise ACF Retrieval
Original SAR image (left) and the obtained filtered image (right)
* V. Lukin, N. Ponomarenko, K. Egiazarian, J. Astola, Adaptive DCT-based filtering of images corrupted by spatially correlated noise, Proc. SPIE Conference Image Processing: Algorithms and Systems VI, Vol. 6812, 12 p., 2008.
If fluctuative noise type is determined correctly and its variance is estimated with high enough accuracy, it becomes possible to blindly evaluate spatial correlation properties of noise as well*.
These estimates can be further exploited in DCT based filtering. Taking into account noise spatial spectrum leads to PSNR increasing by 1...3 dB.
18
Vladimir Lukin 31/08/2009
Image Filtering Examples:Fully Automatic Procedure
Stages:1. Blind determination of noise type, additive or multiplicative (if there is no a
priori information);
2. Estimation of its statistical characteristics (variance);
3. Estimation of spatial correlation properties of noise;
4. Image filtering with taking into account the obtained estimates of noise characteristics.
Features:This procedure is also applicable if both additive and multiplicative noise
components have been identified in a given image.
Note that DCT based filters can be easily modified to mixed additive and multiplicative or other signal-dependent noise.
Similar modifications are not a problem for the family of sigma filters and hard-switching locally adaptive filters.
19
Vladimir Lukin 31/08/2009
Image Filtering Examples:3D Processing of Multichannel Images
a b (PSNR= 22.87 dB)
c (PSNR= 25.17 dB) d (PSNR= 30.01 dB)
Automatic procedures based on blind estimation described above can be used for both component-wise and 3D processing of multichannel images.
Improvement of PSNR due to 3D processing is 0.6 dB in comparison to component-wise DCT based processing.
The noise-free test image Goldhill (a), noisy image ( ) (b), intermediate output image (c), resulting output image (d)
Preliminary conclusion:
2 225, 0.01 n P
Noisy Image Compression:Quantitative Criteria
20
Vladimir Lukin 31/08/2009
The standard measures to characterize a compressed image quality
- , where is the decompressed image; 2
1 1
( ) /( 1)I J
d nor ij ij
i j
MSE G G IJ
dijG
It is more reasonable to characterize a compressed image quality by quantitative measures calculated with respect to the corresponding noise-free image (MSEnf, PSNRnf) rather than to the original noisy one (MSEor, PSNRor).
- - for 8 bits image representation.21010log (255 / )or orPSNR MSE
Alternative measures to characterize a compressed image quality
- , where is the noise free image; 2
1 1
( ) /( 1)I J
dnf ij ij
i j
MSE G G IJ
ijG
- .21010log (255 / )nf nfPSNR MSE
Noisy Image Compression:Optimal Operation Point
21
Vladimir Lukin 31/08/2009
0
50
100
150
200
250
2 3 4 5 6 7 8 9 10 11
QSn
MS
En
f
Dependences MSEnf (QSn) for the noisy test gray-scale image Lena for different additive
noise levels
Optimal operation point (OOP): The argument of the curves MSEnf (CR), MSEnf (bpp) or MSEnf (QS) for which these
curves reach theirs minima have been called optimal operation point (OOP): CROOP , bppOOP or QSOOP..
OOP is observed and commonly occurs to be more “obvious” for less complex content images and/or for rather intensive noise.
Main idea:It is worth compressing a noisy image in
the neighborhood of OOP.
Main problem:In practice, noise-free image is not at
disposal.
σ2n = 400 σ2
n = 100 σ2n = 50
n nQS QS n OOPQS
Noisy Image Compression:Peculiarities
22
Vladimir Lukin 31/08/2009
Why lossy (not lossless) compression?
1) Lossy compression is able to provide considerably larger CRs (compared to lossless coding) without degrading image resolution and introducing disturbing artefacts;
2) A positive effect of image filtering can be observed due to lossy compression if introduced losses mainly relate to noise removal and useful image content is preserved.
The RS (Helsinki region) image corrupted by additive Gaussian noise with σ2
n = 100The decoded lossy compressed image
(bpp = 0.75)
Noisy Image Compression:Real-life Test Images
23
Vladimir Lukin 31/08/2009
The compressed image (HBC strategy,AGU coder with QSA=35)
The automatic procedure for attaining OOP have been also designed for pure multiplicative and Poisson noise.
The corresponding homomorphic transform is to be applied (for making noise additive) before compression.
Inverse homomorphic transform is to be carried out after decompression.
The noisy real-life test image Frisco
24
Vladimir Lukin 31/08/2009
Automatic Procedure for Lossy Compression
Stages:1. Blind determination of noise type: additive, Poisson or multiplicative;
2. Estimation of noise statistical characteristics;
3. Carrying out proper forward homomorphic transform if noise is Poisson or multiplicative; calculation or estimation of variance after homomorphic transform (if needed);
4. Carrying out lossy compression with providing OOP by either setting QS according to above recommendations or by finding proper bpp (for JPEG2000 or SPIHT coders, see details in the paper*).
* N. Ponomarenko, V. Lukin, M. Zriakhov, K. Egiazarian, J. Astola, Lossy compression of images with additive noise, Proceedings of International Conference on Advanced Concepts for Intelligent Vision Systems, Antwerpen, Belgium, September, 2005, pp. 381-386.
Note that are to be within the limits from till . .2n 20.8 n 21.2 n
Accuracy of Blind Estimation Methods: TID2008 (Tampere Image Database)
25
Vladimir Lukin 31/08/2009
Test noise-free color images of TID2008 (http://www.ponomarenko.info/tid2008.htm)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 2520
40
60
80
100
120
140
160
180
200
220
Test image index
^ n 2
Blue component
Green componentRed component
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 2510
20
30
40
50
60
70
80
90
100
110
Test image index
^ n 2
Blue component
Green componentRed component
26
Vladimir Lukin 31/08/2009
Accuracy of Blind Estimation Methods:Spatial Domain
Noise variance estimates obtained by the methods* (left) and** (right) with nonoverlapping 5x5 pixels block size for red, green and blue components for the test image set corrupted by spatially
uncorrelated additive noise with =65
* Lukin V.V., Abramov S.K., Vozel B., Chehdi K. Improved minimal inter-quantile distance method for blind estimation of noise variance in images, Proceedings of SPIE/EUROPTO Symposium on Satellite Remote Sensing, Florence, Italy, Sept 2007, SPIE Vol. 6748, 12 p.
** V. Lukin, S. Abramov, B. Vozel, K. Chehdi, J. Astola, Segmentation-based method for blind evaluation of noise variance in images, SPIE Journal on Applied Remote Sensing, Vol. 2, Aug. 2008, 15 p. (open access paper)
2n
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 2520
40
60
80
100
120
140
160
Test image index
^ n 2
Blue component
Green componentRed component
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 250
10
20
30
40
50
60
70
80
Test image index
^ n 2
Blue component
Green componentRed component
27
Vladimir Lukin 31/08/2009
Accuracy of Blind Estimation Methods:Wavelet Domain
Noise variance estimates obtained by the method* for red, green and blue components for the test image set corrupted by spatially uncorrelated (left) and spatially correlated (right) additive noise with
= 65
* L. Sendur, I.W. Selesnick, Bivariate shrinkage with local variance estimation, IEEE Signal Processing Letters, Vol. 9, No 12, 2002, pp. 438-441 (see also http://taco.polv.edu/WaveletSoftware/index.html)
2n
Be careful with spatially correlated noise !!!
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 250
50
100
150
200
250
Test image index
^ n 2
Blue component
Green componentRed component
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 250
20
40
60
80
100
120
140
160
180
200
Test image index
^ n 2
Blue component
Green componentRed component
28
Vladimir Lukin 31/08/2009
Accuracy of Blind Estimation Methods:Spatial Domain & Spatially Correletad Noise
Noise variance estimates obtained by the method* with nonoverlapping 5x5 (left) and 7x7 (right) pixels block size for red, green and blue components for the test image set corrupted by spatially
correlated additive noise with =65
* Lukin V.V., Abramov S.K., Vozel B., Chehdi K. Improved minimal inter-quantile distance method for blind estimation of noise variance in images, Proceedings of SPIE/EUROPTO Symposium on Satellite Remote Sensing, Florence, Italy, Sept 2007, SPIE Vol. 6748, 12 p.
2n
Summary: It is better using 7x7 blocks to have less problems with (possibly) spatiall correlated noise.
29
Vladimir Lukin 31/08/2009
Conclusions
1. It is possible to design multi-stage automatic procedures of image processing based on blind determination of noise type and estimation of its characteristics.
2. The accuracy of such estimates is commonly enough for carrying out efficient edge detection, filtering, and lossy compression in automatic manner.
3. Currently only quite simple situations like identified pure additive or pure multiplicative noise are studied well enough; design and performance analysis for other noise environments are far from completeness.
4. Special attention should be paid to multichannel and hyperspectral images for which, on one hand, necessity to make processing automatic is the most actual and, on the other hand, the situation with such design is the most complex.