Adaptive DenoisingAdaptive Denoisingfor Video Compressionfor Video Compression
Eren SoyakEECS 463
Winter 2006
Northwestern University
Video Compression and You
Demand for video where no video has gone before
Source You
Source
Source
Video Compression and You
Source You
Source
Source
Encode Medium
Decode
Demand for video where no video has gone before
Video Compression and You
Demand for video where no video has gone before
Source You
Source
Source
Encode Channel
Decode
Pos t proces sing
Preproces sing
Video and Compression Video compression works by identifying and
exploiting redundancy in source video The more information there is in the source,
the more difficult it is to compress into a smaller form
Foreman
Foreman.264
Noise and Compression Noise is usually present in source video due to
various reasons (capture, film grain, quantization, transmission errors etc)
Wide spectrum noise is very difficult to compress
The ever-popular AWGN-type noise
Deprecated old analog-type noise
Dealing with Noise Pre/post filtering methods very useful Simple denoising method: averaging filter
3 pels 5 pels 7 pels
Good, Bad and Ugly Denoising Denoising must distinguish between original signal
and noise, filter out only the noise. Prediction of the noise and/or the original video is usually required for this.
Smoothing, edge loss and blurring are all undesirable
Despeckle
“Smart” blur
10 pel average
Case Study: AWGN Additive White Gaussian Noise (AWGN) can be
introduced by capture devices, especially due to poor lighting and sometimes weather.
AWGN breaks most compression algorithms. Consider signal independent AWGN.
Foreman + AWGN
Advanced Denoising (Wiener) The Wiener filter is commonly used by the
ambitious for generic denoising. Requires little information about noise. Few “catastrophic” corner cases.
Wiener(Foreman + AWGN)
Global Denoising Issues The visibility (and usually compression hindrance)
of noise is a function of the source even if the severity of the noise itself is not – noise is more visible on smooth regions as opposed to texture.
It would be highly desirable to filter noise such that the final video retains local shape/texture characteristics as well.
Adaptive methods begin to suggest themselves.
LMMSE Filtering Linear Minimum Mean Squared Error filter (IIR)
1 2i i212121 = ˆ -i)-i, n)g(n, ih(i), n(ns
Noisy image
Impulse response
LMMSE estimate of ideal image s(n1, n2)
(1)
The Unrealizable Wiener Filter The principle of orthogonality states that the
estimation error s(n1, n2)- (n1, n2) should be orthogonal to every sample of the observed image.
s
0
ˆ =
, ˆ
212121
212121
), kg(k), n(ns), ns(nE
), kg(k), n(ns), ns(n(2)
The Impossible Wiener IR Substituting (1) into (2) and simplifying we can
express the the impulse response of the filter as a 2D convolution
Is impossible to realize since infinite time is required before an output sample is computed.
), n(nR), n(nR), nh(n sggg 212121 =**
“Discrete Wiener-Hopf equation”
autocorrelation of observations
cross correlation between ideal and observed image
Adaptive LMMSE Kuan et al. proposed in ’85:
), n(n)-μ, ng(nσ), n(nσ
) , n(nσ) + , n(n) = μ, n(ns g
vs
sg 21212
212
212
2121ˆ
), n(ns 21ˆ
), n(nμg 21
) , n(nσs 212
), ng(n 21
2vσ
= observation
= filtered output = local variance
= estimated noise variance
= local mean
Adaptive LMMSE Performance Given its adaptive nature to local image properties
the filter is better at preserving edges/texture while removing noise.
It is very process-intensive and sensitive to misestimation of noise variance.
Adaptive LMMSE(Foreman + AWGN)
Comparing Filter Outputs
Comparing Filter Outputs
Adaptive LMMSEWiener
Comparing Compressed Video Compressed at 512 kbps at H.264 Main Profile
Wiener
Adaptive LMMSE
Weighed Adaptive LMMSE Directionally weighed variance matrix
May better account for edges due to 2D direction component
Choice of weight matrix could be optimized
1 2 1
2 3 2
1 2 1
Weighed Adaptive LMMSE Prone to blurring if matrix weights poorly chosen..
Poorly Weighed Adaptive LMMSE(Foreman + AWGN)
Bibliography A. Murat Tekalp, Digital Video Processing, ‘95 J. S. Lim, Two Dimensional Signal and Image
Processing, ‘90 D.T. Kuan, A.A. Sawchuk, T.C. Strand, P. Chavel,
Adaptive noise smoothing filter for images with signal-dependent noise, ‘85