Date post: | 28-Dec-2015 |
Category: |
Documents |
Upload: | giles-norris |
View: | 222 times |
Download: | 1 times |
M. Wu: ENEE631 Digital Image Processing (Spring'09)
Wavelet Coding and Related IssuesWavelet Coding and Related Issues
Spring ’09 Instructor: Min Wu
Electrical and Computer Engineering Department,
University of Maryland, College Park
bb.eng.umd.edu (select ENEE631 S’09) [email protected]
ENEE631 Spring’09ENEE631 Spring’09Lecture 13 (3/11/2009)Lecture 13 (3/11/2009)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [2]
Overview and LogisticsOverview and Logistics
Last Time:
– JPEG Compression– Subband and Wavelet based compression
Subband decomposition; basic ideas of EZW encoding
Today– Continue on Subband and Wavelet based compression
Exploit the structures between coefficients for removing redundancy
More on wavelet transform, and consideration on filters
Assignment#3 is Due Friday noon March 13– JPEG part may be turned in after spring break
Project#1 Posted. Due Monday April 6
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
04
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [3]
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
04
)
Review: 2-D Subband/DWT Review: 2-D Subband/DWT DecompositionDecomposition
From Usevitch’s article in IEEE Sig.Proc. Mag. 9/01
Separable transform by successively applying 1-D DWT to rows and columns
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [4]
Review: Key Concepts in EZW Review: Key Concepts in EZW
Exploit multi-resolution and self-similarity
Parent-children relation among coeff.– Each parent coeff at level k spatially correlates with
4 coeff at level (k-1) of same orientation – A coeff at lowest band correlates with 3 coeff.
Coding significance map via zero-tree– Encode “insignificance map” w/ zero-trees
Avoid large overhead by coding only significant coefficients
Successive approximation quantization– Send most-significant-bits first and gradually
refine coefficient value– “Embedded” nature of coded bit-stream
get higher fidelity image by adding extra refining bits
=> Quality Scalability with fine granularityFrom Usevitch (IEEE Sig.Proc. Mag. 9/01)
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [5]
Recall: EZW Algorithm and ExampleRecall: EZW Algorithm and Example
Initial threshold ~ 2 ^ floor(log2 xmax)– Put all coeff. in dominant list
Dominant Pass (“zig-zag” across bands)
– Assign symbol to each coeff. and entropy encode symbols ps – positive significance ns – negative significance iz – isolated zero ztr – zero-tree root
– Significant coefficients Move to subordinate list Set them to zero in dominant list
Subordinate Pass– Output one bit for subordinate list
According to position in up/low half of quantization interval
Repeat with half threshold, until bit budget achieved – E.g. first look for coeff. above 32; then focus on coeff. between 16
and 32
Fig. From Usevitch (IEEE Sig. Proc. Mag. 9/01)
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [6]
After 1After 1stst Pass PassFrom Usevitch (IEEE Sig.Proc. Mag. 9/01)
Divide quantization interval of [32,64) by half: [32,48) => 40 and [48, 64) => 56
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [7]
After 2After 2ndnd Pass Pass
From Usevitch (IEEE Sig.Proc. Mag. 9/01)
[16,24) => 20, [24, 32) => 28; [32,40) => 36, [40,48) => 44; [48, 56) => 52, [56, 64) => 60
2nd Subordinate Pass:
orig. 53 symbol 0 reconstruct 52 34 0 36
-22 0 -20
21 0 20
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [8]
EZW and BeyondEZW and Beyond
Can apply DWT to entire images or larger blocks than 8x8
Symbol sequence can be entropy encoded – e.g. arithmetic coding
Cons of EZW– Poor error resilience; Difficult for selective spatial decoding
SPIHT (Set Partitioning in Hierarchal Trees)– Further improvement over EZW to remove redundancy– Achieve equal/better performance than EZW w/o arithmetic coding
EBCOT (Embedded Block Coding with Optimal Truncation)– Used in JPEG 2000– Address the shortcomings of EZW (random access, error resilience, …)– Embedded wavelet coding in each block + bit-allocations among blocks
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
/20
04
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [9]
A Close Look at Wavelet TransformA Close Look at Wavelet Transform
Haar Transform – unitaryHaar Transform – unitary
Orthonormal Wavelet FiltersOrthonormal Wavelet Filters
Biorthogonal Wavelet FiltersBiorthogonal Wavelet Filters
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
04
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [10]
Construction of Haar FunctionsConstruction of Haar Functions
Unique decomposition of integer k (p, q)– k = 0, …, N-1 with N = 2n, 0 <= p <= n-1– q = 0, 1 (for p=0); 1 <= q <= 2p (for p>0)
e.g., k=0 k=1 k=2 k=3 k=4 … (0,0) (0,1) (1,1) (1,2) (2,1) …
hk(x) = h p,q(x) for x [0,1]
k = 2p + q – 1“remainder”
power of 2
]1,0[other for 0
22for 2
1
22
1for 2
1
)()(
]1,0[for 1
)()(
21
2/
21
2/
,
0,00
x
qx
q-
N
q-x
q-
N
xhxh
xN
xhxh
ppp
ppp
qpk
1
x
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
/20
04
)
(0,0)
(0,1)
(1,1)
(1,2)
(2,1)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [11]
Haar TransformHaar Transform
Haar transform H
– Sample hk(x) at {m/N}for m = 0, …, N-1
– Real and orthogonal– Transition at each scale p is
localized according to q
Basis images of 2-D (separable) Haar transform– Outer product of two basis vectors
2200
0022
0000
0000
0000
0000
2 2 00
0 0 2 22222
0000
1111
1111
0000
2222
1111
1111
8
1
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [12]
Compare Basis Images of DCT and HaarCompare Basis Images of DCT and Haar
See also: Jain’s Fig.5.2 pp136UMCP ENEE631 Slides (created by M.Wu © 2001)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [13]
Time-Freq (or Space-Freq) InterpretationsTime-Freq (or Space-Freq) Interpretations
– Standard Basis (for data samples); Fourier Basis; Wavelet Basis
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [14]
Recap on Haar TransformRecap on Haar Transform Two major sub-operations
– Scaling captures info. at different frequencies– Translation captures info. at different locations
Can be represented by filtering and downsampling
Relatively poor energy compaction– Equiv. filter response doesn’t have good cutoff & stopband attenuation
1
x
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [15]
Orthonormal FiltersOrthonormal Filters
Equiv. to projecting input signal to orthonormal basis
Energy preservation property– Convenient for quantizer design
MSE by transform domain quantizer is same as reconstruction MSE in image domain
Shortcomings: “coefficient expansion”– Linear filtering with N-element input & M-element filter
(N+M-1)-element output (N+M)/2 after downsample
– Length of output per stage grows ~ undesirable for compression
Solutions to coefficient expansion– Symmetrically extended input (circular convolution) &
Symmetric filter
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [16]
Solutions to Coefficient ExpansionSolutions to Coefficient Expansion Circular convolution in place of linear convolution
– Periodic extension of input signal– Problem: artifacts by large discontinuity at borders
Symmetric extension of input– Reduce border artifacts (note the signal length doubled with symmetry)– Problem: output at each stage may not be symmetric From Usevitch (IEEE
Sig.Proc. Mag. 9/01)
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [17]
Solutions to Coefficient Expansion (cont’d)Solutions to Coefficient Expansion (cont’d)
Symmetric extension + symmetric filters
– No coefficient expansion and little artifacts– Symmetric filter (or asymmetric filter) => “linear phase filters”
(no phase distortion except by delays)
Problem
– Only one set of linear phase filters for real FIR orthogonal wavelets Haar filters: (1, 1) & (1,-1) do not give good energy compaction
Ref: review ENEE630 discussions on FIR perfect reconstruction Qudrature Mirror Filters (QMF) for 2-channel filter banks.
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [21]
Biorthogonal WaveletsBiorthogonal Wavelets
“Biorthogonal”– Basis in forward and inverse transf.
are not the same but give overall perfect reconstruction (PR) recall EE630 PR filterbank
– No strict orthogonality for transf. filters so energy is not preserved But could be close to orthogonal
filters’ performance
Advantage– Covers a much broader class of filters
including symmetric filters that eliminate coefficient expansion
Commonly used filters for compression– 9/7 biorthogonal symmetric filter– Efficient implementation: Lifting approach (ref: Swelden’s tutorial)
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
)From Usevitch (IEEE Sig.Proc. Mag. 9/01)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [22]
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
04
)
Smoothness Conditions on Wavelet FilterSmoothness Conditions on Wavelet Filter
– Ensure the low band coefficients obtained by recursive filtering can provide a smooth approximation of the original signal
From M. Vetterli’s wavelet/filter-bank paper
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [25]
JPEG 2000 Image Compression StandardJPEG 2000 Image Compression Standard
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
04
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [26]
From Christopoulos (IEEE Trans. on CE 11/00)
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [27]
JPEG 2000: A Wavelet-Based New StandardJPEG 2000: A Wavelet-Based New Standard
Targets and features
– Excellent low bit rate performance without sacrifice performance at higher bit rate
– Progressive decoding to allow from lossy to lossless– Region-of-interest (ROI) coding– Error resilience
For details
– JPEG2000 Tutorial by Skrodras @ IEEE Sig. Proc Magazine 9/2001 – David Taubman: “High Performance Scalable Image Compression
with EBCOT”, IEEE Trans. On Image Proc, vol.9(7), 7/2000.– Taubman’s book on JPEG 2000 (on library reserve)– Links and tutorials @ http://www.jpeg.org/JPEG2000.htm
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
/20
04
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [28]
ExamplesExamples
JPEG2KJPEG2K vs. vs. JPEGJPEG
From Christopoulos (IEEE Trans. on CE 11/00)
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [29]
DCT vs. Wavelet: Which is Better?DCT vs. Wavelet: Which is Better?
3dB improvement?
– Wavelet compression was claimed to have 3dB improvement over DCT-based compression
– Comparison is done on JPEG Baseline
Improvement not all due to transforms
– Main contribution from better rate allocation, advanced entropy coding, & smarter redundancy reduction via zero-tree
– Difference due to DCT vs. Wavelet transform is less than 1dB– DCT coder can be improved to decrease the gap
[Ref] "A comparative study of DCT- and wavelet-based image coding", Z. Xiong, K. Ramchandran, M. Orchard, Y-Q. Zhang, IEEE Trans. on Circuits and Systems for Video Technology, v.9, no.5, 8/99, pp692-695.
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [30]
Non-Dyadic Decomposition – Wavelet PacketsNon-Dyadic Decomposition – Wavelet Packets
Figures are from slides at Gonzalez/ Woods DIP book 2/e website (Chapter 7)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [31]
Bit Allocation in Image CodingBit Allocation in Image Coding
Focus on MSE-based optimization;Can further adjust based on HVS
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
04
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [32]
1
0
2 )()(
1min
N
kq kvkvE
ND
Here k2: variance of k-th coeff. v(k); rk is # bits allocated for v(k)
f(-): a func. relating bit rate to distortion based on a coeff’s p.d.f.
Bit AllocationBit Allocation
How many bits to be allocated for each coefficient?
– Related to each coeff’s variance (and probability distribution)– More bits for high variance k
2 to keep total MSE small
1
0
)(1 N
kkrf
N
1
0
.bits/coeff 1
subject toN
kk Rr
N
“Reverse Water-filling”– Try to keep same amount of error in each frequency band
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [34]
Rate/Distortion Allocation via Reverse Water-fillingRate/Distortion Allocation via Reverse Water-filling
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
/20
04
) Optimal solution for Gaussian distributed coefficients Idea: try to keep same amount of error in each freq. band; no need to
spend bits to represent coeff. w/ smaller variance than water level
– Results based on R-D function and via Lagrange-multiplier optimization
– Note the convex shape of R-D function for Gaussian => slope becomes milder; require more bits to further reduce distortion
D
RD
2
Given total rate R, determine and then D; or vice versa given D, determine R
Optimal Not Optimal
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [35]
Details on Reverse Water-filling SolutionDetails on Reverse Water-filling Solution
RRRDi
i
n
iii
tosubj. )(min1
N
ii
N
iiiN RRRDRRJ
111 )(),...(
Construct func. using Lagrange multiplier
R and allfor 0 ii
i
i
i
i
RdR
dDi
dR
dD
R
J Necessary condition Keep the same marginal gain
0for 1
2
1 0,ln
2
1max
2
iii
i
i
i
ii
i RDdD
dR
DdD
d
dD
dR
Equal Slope Condition for bit allocation
D
RD
2
Resulting in equal distortions on Gaussian r.v’sas interpreted in “reverse water filling”
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [36]
Key Result in Rate Allocation: Equal R-D SlopeKey Result in Rate Allocation: Equal R-D Slope
Keep the slope in R-D curve the same (not just for Gaussian r.v)
– Otherwise some bits can be applied to other r.v. for better “return” in reducing the overall distortion
If all r.v. are Gaussian, the same slope in R-D curves for these r.v. correspond to an identical amount of distortion.
R(D) = ½ [ log 2 – log D ] => dR/dD = – 1 / (2D)
Use operational/measured rate-distortion information for general r.v.
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
04
) Rate allocation results:=> Equal slope(o.w. exist a better allocation strategy)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [37]
Summary of Today’s LectureSummary of Today’s Lecture Wavelet transform and coding
– Haar basis/transform; considerations for wavelet filters; JPEG 2000
Optimal bit allocation: “equal slope” in R-D curves– Lead to reverse water filling for Gaussian coefficients
Next lecture– Optimal transform
Readings: see wavelet coding references in Project #1– Gonzalez’s 3/e book Section 8.2.10; 7.1, 7.4-7.5; (7.2-7.3 on wavelet)– Transform coding articles in IEEE Sig. Proc. Magazine Sept. 2001
(special issue on transform coding)
– To explore more: Tutorial on rate-distortion by Ortega-Ramchandran in IEEE Sig. Proc. Magazine, Nov. 98, and Jain’s book Section 2.13;“Wavelets and the lifting scheme: A 5 minute tour” by Wim Sweldens [Zeitschrift für Angewandte Mathematik und Mechanik, 76 (2), pp. 41-44, 1996]
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
04
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [38]
Review: Correlation After a Linear TransformReview: Correlation After a Linear Transform
Consider an Nx1 zero-mean random vector x
– Covariance (or autocorrelation) matrix Rx = E[ x xH ] give us ideas of the correlation between elements Rx is a diagonal matrix for if all N r.v.’s are uncorrelated
Apply a linear transform to x: y = A x
What is the autocorrelation matrix for y ?
Ry = E[ y yH ] = E[ (Ax) (Ax)H ] = E[ A x xH AH ]
= A E[ x xH ] AH = A Rx AH
Decorrelation: try to search for A that can produce a decorrelated y (equiv. a diagonal correlation matrix Ry )
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
04
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [40]
Optimal TransformOptimal Transform
Recall: Why use transform in coding/compression?– Decorrelate the correlated data– Pack energy into a small number of coefficients
– Interested in unitary/orthogonal or approximate orthogonal transforms Energy preservation s.t. quantization effects can be better understood
and controlled
Unitary transforms we’ve dealt so far are data independent– Transform basis/filters are not depending on the signals we are processing
Can hard-wire them in encoder and decoder
What unitary transform gives the best energy compaction and decorrelation?– “Optimal” in a statistical sense to allow the codec works well with many
images => Signal statistics would play an important role
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
04
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [41]
Review: Correlation After a Linear TransformReview: Correlation After a Linear Transform
Consider an Nx1 zero-mean random vector x
– Covariance (or autocorrelation) matrix Rx = E[ x xH ] give ideas of correlation between elements Rx is a diagonal matrix for if all N r.v.’s are uncorrelated
Apply a linear transform to x: y = A x
What is the correlation matrix for y ?
Ry = E[ y yH ] = E[ (Ax) (Ax)H ] = E[ A x xH AH ]
= A E[ x xH ] AH = A Rx AH
Decorrelation: try to search for A that can produce a decorrelated y (equiv. a diagonal correlation matrix Ry )
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
04
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [42]
K-L Transform (Principal Component Analysis)K-L Transform (Principal Component Analysis)
Eigen decomposition of Rx: Rx uk = k uk
– Recall the properties of Rx
Hermitian (conjugate symmetric RH = R); Nonnegative definite (real non-negative eigen values)
Karhunen-Loeve Transform (KLT) y = UH x x = U y with U = [ u1, … uN ]
– KLT is a unitary transform with basis vectors in U being the orthonormalized eigenvectors of Rx
– Rx U = [ 1u1, … N uN ] = U diag{1, 2, … , N}
=> UH Rx U = diag{1, 2, … , N} i.e. KLT performs decorrelation Often order {ui} so that 1 2 … N
– Also known as Hotelling transform or Principle Component Analysis (PCA)
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
/20
04
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [44]
Properties of K-L TransformProperties of K-L Transform
Decorrelation
– E[ y yH ]= E[ (UH x) (UH x)H ]= UH E[ x xH ] U = diag{1, 2, … , N}
– Note: Other matrices (unitary or nonunitary) may also decorrelate the transformed sequence [Jain’s e.g.5.7 pp166]
Minimizing MSE under basis restriction
– If only allow to keep m coefficients for any 1 m N, what’s the best way to minimize reconstruction error?
Keep the coefficients w.r.t. the eigenvectors of the first m largest eigen values
Reference: Theorem 5.1 and Proof in Jain’s Book (pp166)
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
/20
04
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [45]
KLT Basis RestrictionKLT Basis Restriction Basis restriction
– Keep only a subset of m transform coefficients and then perform inverse transform (1 m N)
– Basis restriction error: MSE between original & new sequences
Goal: to find the forward and backward transform matrices to minimize the restriction error for each and every m– The minimum is achieved by KLT arranged according to the
decreasing order of the eigenvalues of R
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
04
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [46]
K-L Transform for ImagesK-L Transform for Images
Work with 2-D autocorrelation function
– R(m,n; m’,n’)= E[ x(m, n) x(m’, n’) ] for all 0 m, m’, n, n’ N-1– K-L Basis images is the orthonormalized eigen functions of R
Rewrite images into vector form (N2x1)
– Need solve the eigen problem for N2xN2 matrix! ~ O(N 6)
Reduced computation for separable R
– R(m,n; m’,n’)= r1(m,m’) r2(n,n’)
– Only need solve the eigen problem for two NxN matrices ~ O(N3)– KLT can now be performed separably on rows and columns
Reducing the transform complexity from O(N4) to O(N3)
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [47]
Pros and Cons of K-L TransformPros and Cons of K-L Transform
Optimality
– Decorrelation and MMSE for the same# of partial transf coeff.
Data dependent
– Have to estimate the 2nd-order statistics to determine the transform– Can we get data-independent transform with similar performance?
DCT
Applications
– (non-universal) compression– Pattern recognition: e.g., eigen faces– Analyze the principal (“dominating”) components and reduce
feature dimensions
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
01
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [48]
Energy Compaction of DCT vs. KLTEnergy Compaction of DCT vs. KLT
DCT has excellent energy compaction for highly correlated data
DCT is a good replacement for K-L– Close to optimal for highly correlated data– Not depend on specific data like K-L does– Fast algorithm available
[ref and statistics: Jain’s pp153, 168-175]
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
04
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [49]
Energy Compaction of DCT vs. KLT (cont’d)Energy Compaction of DCT vs. KLT (cont’d)
Preliminaries– The matrices R, R-1, and R-1 share the same set of eigen vectors
– DCT basis vectors are eigenvectors of a symmetric tri-diagonal matrix Qc
– Covariance matrix R of 1st-order stationary Markov sequence with has an inverse in the form of symmetric tri-diagonal matrix
DCT is close to KLT on 1st-order stationary Markov
– For highly correlated sequence, a scaled version of R-1 approx. Qc
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
04
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [53]
More on Rate-Distortion Based More on Rate-Distortion Based
Bit Allocation in Image CodingBit Allocation in Image Coding
UM
CP
EN
EE
63
1 S
lide
s (c
rea
ted
by
M.W
u ©
20
04
)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [57]
Basic Steps in R-D OptimizationBasic Steps in R-D Optimization
Determine source’s statistics– Simplified models: Gaussian, Laplacian, Generalized Gaussian– Obtain from training samples
Obtain operational R-D points/functions/curves– E.g., compute distortion for each candidate quantizer
Determine objective function and constraints
Search for optimal operational R-D points– Lagrange multiplier approach
select only operating points on the convex hull of overall R-D characteristics (Fig.19 of Ortega’s tutorial)
may handle different coding unit independently
– Dynamic programming approach not constrained by convex hull but has larger search space
~ higher complexity => See Ortega’s SPM’98 tutorial for details
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [58]
Bridging the Theory and the Practice Bridging the Theory and the Practice
Operational R-D curves– Directly achievable R-D pairs with practical implementation
Tradeoff! – How close are the R-D operational points to info. theory bounds?
– Practical considerations: cost of memory, computation, delay, etc.
Often narrowing down to an optimization problem:~ optimize an objective function subject to a set of constraints
Problem of model mismatch
– How good are our assumptions on source modeling?– How well does a coding algorithm tolerate model mismatch?
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [59]
Ref: convex hullRef: convex hull
Formally
– Smallest convex set containing all points For R-D problem, we concerned of the lower boundary of this
set that close to the info. theoretical bound (the upper/right end are open)
– Convex set if X1,..,Xn are in the set, then the linear combinations of
them (with non-negative combination factors) are still in the set
Intuitively
– Use a rubber band to wrap the outermost points so that the band encloses all points
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [60]
1
0
2 )()(
1min
N
kq kvkvE
ND
where k2 -- variance of k-th coeff. v(k); nk -- # bits allocated for v(k)
f(-) – quantization distortion func.
Bit AllocationBit Allocation How many bits to be allocated for each coeff.?
– Determined by the variance of coeff.– More bits for high variance k
2 to keep total MSE small
1
0
2 )(1 N
kkk nf
N
1
0
.bits/coeff 1
subject toN
kk Bn
N
“Inverse Water-filling” (from info. theory)
– Try to keep same amount of error in each freq. band
See Jain’s pp501 for detailed bit-allocation algorithm
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [61]
Details on Reverse Water-filling Solution (cont’d)Details on Reverse Water-filling Solution (cont’d)
DDD i
i
in
i
tosubj. ,lnmin2
121
n
ii
i
in
i
DD
DJ1
2
121 ln)( Construct func. using
Lagrange multiplier
D and allfor 0 1
2
1ii
ii
DDiDD
J Necessary condition Keep the same marginal gain
DDD i
n
i i
i
tosubj. ,0,ln2
1maxmin
1
2
DDDD
D
D
Ji
ii
ii
ii
ii
i
s.t.
if ,
if ,
if ,0
if ,022
2
2
2
Necessary conditionfor choosing
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [63]
Lagrangian Opt. for Indep. Budget ConstraintLagrangian Opt. for Indep. Budget Constraint
Previous: fix distortion, minimize total rate
Alternative: fix total rate (bit budget), minimize distortion
(Discrete) Lagrangian optimization on general source
– “Constant slope optimization” (e.g. in Box 6 Fig. 17 of Ortega’s tutorial)– Need to determine the quantizer q(i) for each coding unit i– Lagrangian cost for each coding unit
use a line with slope - to intersect with each operating point (Fig.14)
– For a given operating quality , the minimum can be computed independently for each coding unit
=> Find operating quality satisfying the rate constraint
n
iiqiiqi rd
1)(,)(,min
)(,)(, iqiiqi rd
n
iiqiiqi
n
iiqiiqi rdrd
1)(,)(,
1)(,)(, minmin
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [64]
(from Box 6 Fig. 17 of Ortega’s tutorial)(from Fig. 14 of Ortega’s tutorial)
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [65]
Bridging the Theory and Ad-hoc Practice Bridging the Theory and Ad-hoc Practice
Operational R-D curves– Directly achievable with practical implementation
Tradeoff! Tradeoff! Tradeoff! – Rate vs. Distortion
how close are the operational points to info. theory bounds?– Also tradeoff among practical considerations
Cost of memory, computation, delay, etc.
Often narrowing down to an optimization problem:~ optimize an objective func. subj. to a set of constraints
Model mismatch problem– How good are our assumptions on source modeling?– How well does a coding algorithm tolerate model mismatch?
M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec13 – Wavelet Coding, etc. [66]
Basic Steps in R-D OptimizationBasic Steps in R-D Optimization
Determine source’s statistics– Simplified models: Gaussian, Laplacian, Generalized Gaussian– Obtain from training samples
Obtain operational R-D points/functions/curves– E.g., compute distortion for each candidate quantizer
Determine objective func. and constraints
Search for optimal operational R-D points– Lagrange multiplier approach
select only operating points on the convex hull of overall R-D characteristics (Fig.19 of Ortega’s tutorial)
may handle different coding unit independently
– Dynamic programming approach not constrained by convex hull but has larger search space
(higher complexity)