L. Yaroslavsky. Course 0510.7211 “Digital Image Processing: Applications” Lecture 11. Image restoration and enhancement: Linear filters

Signalling and imaging devices, signal distortions and the inverse problem.

Least squares restoration: AVERR AV AV a aim sys im par kid

krst

k

N

= −⎛⎝⎜

⎞⎠⎟=

−

∑. .

2

1

1

.

Linear filters: ( )A Arst inp= LF .

Filtering in transform domain. Scalar filters: ( )A T H T Arstd

inp= −1 ; inprr

rstr αηα = ;⇒

( )( ) ⎟⎠⎞⎜

⎝⎛=

∗ 2inprpar.imsis.im

inpr

idrpar.imsis.im

optr AVAV/AVAV αααη

Scalar filters for suppressing additive

noise: A A Ninp id= + ⇒2

.

2

.

2

.

rsysimidrparim

idrparim

optr

AVAV

AV

νγα

αη

+⎟⎠⎞⎜

⎝⎛

⎟⎠⎞⎜

⎝⎛

=

Empirical Wiener filters

( )( ) ⎥

⎥

⎦

⎤

⎢⎢

⎣

⎡ −≈ 2

22

0inprparim

rsysiminprparimopt

rAV

AVAV

α

ναη

.

..,max .

Signal power spectra estimation problem

Rejecting filters (transform shrinkage):ηα

αr

im par rinp

im par rinp

AV thr

AV thr=

⎛⎝⎜

⎞⎠⎟ ≥

⎛⎝⎜

⎞⎠⎟ <

⎧

⎨⎪

⎩⎪

1

0

2

2

,

,

.

.

. Soft and hard thresholding.

Image deblurring: A LA N Linp id= + , where isa linear operator α λ α νrinp

r rid

r= +

( )

( )ηλ

λ α

λ α ν λropt

r

im par r rid

im par r rid

im sys r r

r

r

AV

AV AV

SNRSNR

=+

=+

1 11

2 2

2 2 2

.

. .

Applicability and drawbacks of Wiener-type filtering. Application examples: filtering periodic noise; filtering stripe-noise. Wavelet (sub band decomposition) shrinkage. Multi component signal restoration Least squares approach { } ;K,...,k;NALA k

idkk

inpk 1=+=

( );;

.

,.

,,,

∑∑

=

•

= +== K

mm

l

idlparim

idl

idkparim

l

optlk

K

l

inplk

rstk

SNR

SNR

AV

AVS

AHA

1

21 1

1

α

ααη

( )( ) ⎪⎭

⎪⎬⎫

⎪⎩

⎪⎨⎧

= 2

22

lsysim

idlpariml

lAV

aAVSNR

ν

λ

.

.

Potential restoration quality: ∑=

=K

llSNR/

energysignalerrornrestoratio.rms

11

Super resolution from multiple images. Correlational averaging (see also Lect. 9). Local adaptive linear filters

Local criteria: ( ) ( ) ( )AVLOSS k l AV LOC m n k l LOSS a astat m n m nm n

, , / , ∃ ,, ,,

=⎧⎨⎩

⎫⎬⎭

∑

Spatial neighbourhood. Moving window linear local adaptive filtering in transform domain. Transform selection. Recursive spectral analysis and filtering in DFT, DCT and Haar domains and its computational complexity. 3-D Local adaptive filtering for restoration of multi component images. Problems for self-testing: 1. Formulate the least squares signal restoration problem. Describe scalar filtering in transform

domain and its advantages and limitations.. 2. Describe local adaptive filtering methods and explain their computer implementation. Why DCT is

advantageous to other transforms in this application? 3. Explain approaches to multi component signal restoration and super resolution. 4. Let a set of images of the same object is available. How this can help to improve image quality?

A canonical block-diagram of imaging systems

MSE-OPTIMAL (WIENER) FILTERING: FILTERING NARROW-BAND NOISE

Input image

Filtered image

Filtering periodic noise

Noisy image (scanning atomic force microscope image)

Filtered image

Filtering “banding” noise

20 40 60 80 100 120

6

8

10

12

14Column wise averaged power spectrum along rows

20 40 60 80 100 120

6

8

10

12

14"Filtered" power spectrum; thr=0.01

50 100 150 200 250 90

110

130

150

Average along rows versus row number

50 100 150 200 250

50

100

150

200

250Filtered averaged rows

Linear transformation

Point-wise nonlinear

transformation ( )xa ( )xb

Stochastic transformation

Wiener filtering for noise suppression

Noisy image (Wiener.m, RD=1000; SNR=2)

Noise, std=127.8706

Ideal Wiener filter

Restoration error, std=55.3361

Empirical Wiener filter, 1=γ

Restoration error, std=68.3752

Image processing: ”local vs global” Justification of local processing and adaptation:

It is well known that, when viewing image, human eye’s optical axis permanently hops chaotically over the field of view(A. Yarbus, Eye Movements and Vision, Plenum Press, New-York, 1967. Translated from the Russian edition (Moscow, 1965)) and that the human visual acuity is very non-uniform over the field of view. The field of view of a man is about 30°. Resolving power of man's vision is about 1′. However such a relatively high resolving power is concentrated only within a small fraction of the field of view that has size of about 2° (M.D. Levine, Vision in Man and Mashine, McGraw-Hill, 1985, pp. 110-130]). Therefore, area of the acute vision is about 1/15-th of the field of view.

Test image (left) and results of recording eye fixation when observing this image (right)

Image frame

Image global DCT-spectrum

Image blocks

DCT spectra of blocks

Fragmentation of the image into blocks demonstrates spatial inhomogeneity of the image and the fact that the structure of individual blocks is much simpler than that of the whole image. Adaptive filter design assumes empirical evaluation of signal statistical parameters such as spectra (for local adaptive linear filters). In global image statistics, parameter variations due to image non-homogeneity are hidden and are difficult if not impossible to detect. Therefore in global statistical analysis image local information will be neglected in favour of global one, which usually contradicts processing goals.

LOCAL ADAPTIVE FILTERING IN TRANSFORM DOMAIN

Local adaptive filtering in DCT domain: suppressing additive noise in electrocardiogram

5 0 0 1 0 0 0 1 5 0 0 2 0 0 01 0 0 0

1 5 0 0

2 0 0 0

2 5 0 0

In i t i a l s i g n a l

L o c a l D C T s p e c t r u m ;S z W = 2 5

5 0 0 1 0 0 0 1 5 0 0 2 0 0 0

51 01 52 0

2 5M o d i f i e d lo c a l s p e c t r u m ; S z W = 2 5 ; T h r = 0 . 0 0 1

5 0 0 1 0 0 0 1 5 0 0 2 0 0 0

51 01 52 0

2 5

5 0 0 1 0 0 0 1 5 0 0 2 0 0 0

1 5 0 0

2 0 0 0

2 5 0 0

R e s t o r e d s i g n a l

Transform

Scanning direction

Transform domain

Filter formation

Filter

Point-wise multiplication

by the filter mask

Modified DCT

spectrum

Inverse transform

Output pixel

Output image

Input image

Local adaptive filtering in DCT domain: image denoising

Initial (upper left), noisy (upper right), filtered (bottom left) images and filter

“transparance” map (bottom right)

Noisy image (frgm+40*(rand(256)-0.5));

Filtered image (out = lcdct_trsh(9,9,150,frgm_n));

Filter “transparance” map

Local adaptive filtering: 3D (space-colour) denoising colour images

Local adaptive filtering: blind image restoration (initial and restored images)

Filtering speckle noise in Ultra Sonic images: original (top) and filtered I(bottom) images

Test images

Noisy images

2-D DCT domain de-noising in window 5x5

3-D DCT domain de-noising in window 5x5x5

Local adaptive 2-D and 3-D DCT domain filtering for image de-noising

Frames of a movie acquired by a thermal camera: initial image (top) and filtered image, 3-D DCT domain 5x5x5 window, (bottom)

Local adaptive filtering in DCT domain: Image sharpening and local contrast enhancement

Initial and enhanced air-photograph (lcdct2(aero512,ones(9),0.75,2.5,1.5));

Filtering in transform domain: wavelet shrinkage (D.L. Donoho and I.M. Johnstone, Ideal spatial adaptation by wavelet shrinkage,

Biometrica, 81(3): 425-455, 1994)

Low pass filtering and

downsampling Interpolation

-+ + Soft/hard

thresholding

Interpolation

+ + +

+ + +

Input

Output

Interpolation

-

+

+

Interpolation

Low pass filtering and

downsampling

Low pass filtering and

downsampling

Interpolation

Soft/hard thresholding

Transform domain hybrid (wavelet/sliding window DCT) filtering (B.Z. Shaick, L. Ridel, L. Yaroslavsky, A hybrid transform method for image denoising, Submitted to EUSIPCO2000, Tampere, Finland, Sept. 5-8, 2000)

Low pass filtering and

downsampling Interpolation

-+ +

Sliding window DCT

domain filtering

Interpolation

+ + +

+ + +

Input

Output

Interpolation

-

+

+

Interpolation

Low pass filtering and

downsampling

Low pass filtering and

downsampling

Interpolation

Sliding window DCT

domain filtering

Interpolation

Sliding window DCT

domain filtering

Wavelet shrinkage, DCT domain sliding window and hybrid processing: image de-noising capability