Image Quantization Lloyd-Max Quantizer Image Transforms
Digital Image ProcessingLectures 7 & 8
M.R. Azimi, Professor
Department of Electrical and Computer EngineeringColorado State University
February 2011
M.R. Azimi Digital Image Processing
Image Quantization Lloyd-Max Quantizer Image Transforms
Quantization Process
A quantizer maps the continuous variable x into a discrete xq whichtakes values from a finite set r1, . . . , rL of numbers. This mapping isgenerally a staircase function as shown below.
Quantization Rule:Define tk, k = 1, . . . , L+ 1 as a set of transition or decision levels with
t1 and tL+1 as the Min. and Max. values of x, respectively. If x lies in
the interval [tk, tk+1), then xq = rk, where rk is called the kth
reconstruction level.
M.R. Azimi Digital Image Processing
Image Quantization Lloyd-Max Quantizer Image Transforms
Properties:
Quantization mapping is irreversible, i.e. for a given quantizeroutput, the input value cannot be determined uniquely.
Quantization incurs distortion. The design is a trade-offbetween simplicity and performance (minimum distortion).
Quantization is a ”zero-memory” operation, i.e. outputdepends on only one input.
Uniform Quantizer:Simplest and most commonly used (e.g., PCM, differential PCM andtransform coding) quantizer. Let the output of an image sensor takesvalues between 0 to A. If samples are quantized uniformly to L (e.g.L = 256) levels, then transition and reconstruction levels are
tk = A(k−1)L k = 1, . . . , L+ 1
rk = tk + A/2L k = 1, . . . , L
The interval q =∆= tk − tk−1 = rk − rk−1 is constant for different values
of k and is called ”quantization interval”.
M.R. Azimi Digital Image Processing
Image Quantization Lloyd-Max Quantizer Image Transforms
Minimum Mean Squared or Lloyd-Max Quantizer
This quantizer minimizes the mean squared error for a given number ofquantization levels. Let x, with 0 ≤ x ≤ A be a real scalar randomvariable with a continuous PDF pX(x). It is desired to find optimum thedecision levels tk and the reconstruction levels rk for an L-level quantizersuch that the mean square error (MSE) (or quantization distortion)
ε = E[(x− xq)2] =
∫ tL+1
t1
(x− xq)2pX(x)dx
is minimized. Note that pX(x) is the PDF of the amplitude of x i.e.∫ A0pX(x)dx = 1. The MSE can be rewritten as
ε =
L∑i=1
∫ ti+1
ti
(x− ri)2pX(x)dx
M.R. Azimi Digital Image Processing
Image Quantization Lloyd-Max Quantizer Image Transforms
To minimize ε w.r.t. tk and rk
∂ε
∂tk= 0 ⇒ (tk − rk−1)2pX(tk)− (tk − rk)2pX(tk) = 0
∂ε
∂rk= 0 ⇒ 2
∫ tk+1
tk
(x− rk)pX(x)dx = 0, k ∈ [1, L]
where the first derivative is obtained using the fact thattk ≤ x < tk+1 ⇒ xq = rk, simplification of the above equationsgives
tk =rk + rk−1
2(1)
rk =
∫ tk+1
tkxpX(x)dx∫ tk+1
tkpX(x)dx
(2)
i.e. the optimum transition levels lie halfway between the optimumreconstruction levels, which in turn lie at the center of mass of thePDF in between the optimum transition levels.
M.R. Azimi Digital Image Processing
Image Quantization Lloyd-Max Quantizer Image Transforms
These non-linear equations must be solved simultaneously giventhe bounding values t1 and tL+1. In practice, these equations canbe solved by an iterative scheme such as the Newton method.
When the No. of quantization levels is large, an approximatesolution can be obtained by modeling the PDF pX(x) as apiecewise constant function i.e.,
pX(x) ' pX(tj), tj∆=tj + tj+1
2, tj ≤ x ≤ tj+1
M.R. Azimi Digital Image Processing
Image Quantization Lloyd-Max Quantizer Image Transforms
Using this approximation and performing the requiredminimization, the equation for optimum transition level becomes
tk+1 ≈(tL+1 − t1)
∫ akt1
[pX(x)]−1/3dx∫ tL+1
t1[pX(x)]−1/3dx
+ t1
with
ak =k(tL+1 − t1)
L+ t1, k ∈ [0, L]
Thus, this together with optimum reconstruction level in (2) i.e.rk =
tk+tk+1
2 are now de-coupled.
The quantizer MS distortion is
ε ≈ 1
12L2∫ tL+1
t1
[pX(x)]13dx3
M.R. Azimi Digital Image Processing
Image Quantization Lloyd-Max Quantizer Image Transforms
Properties of Lloyd-Max Quantizers
1 Unbiased in the mean i.e. E[X] = E[Xq].
Proof: Since Xq is a discrete r.v. we have
E[Xq] =
L∑k=1
rkpXq(xq = rk)
Using the staircase mapping of quantizer, the PMF at rk is
pXq(xq = rk) =
∫ tk+1
tk
pX(x)dx
Thus, using Eq. (2) of Llyod-Max quantizer we get
E[Xq] =
L∑k=1
rk
∫ tk+1
tk
pX(x)dx =
L∑k=1
∫ tk+1
tk
xpX(x)dx
which can be reduced to
E[Xq] =
∫ tL+1
t1
xpX(x)dx = E[X]
M.R. Azimi Digital Image Processing
Image Quantization Lloyd-Max Quantizer Image Transforms
2 Orthogonality of error with output i.e. E[(X −Xq)Xq] = 0.Proof: To prove we show E[XXq] = E[X2
q ]. Using the correlationdefinition
E[XXq] =
L∑k=1
∫ tk+1
tk
xxqpX,Xq(x, xq = rk)dx
When x ∈ Ωk i.e. segment k, joint PDF pX,Xq (x, xq = rk) can beexpressed as pXq
(xq = rk|x ∈ Ωk)pX(x ∈ Ωk) = pX(x ∈ Ωk).Thus,
E[XXq] =
L∑k=1
rk
∫ tk+1
tk
xpX(x)dx
Using Eq. (2) of Llyod-Max quantizer and result in property 1,
E[XXq] =
L∑k=1
rk
∫ tk+1
tk
xpX(x)dx =
L∑k=1
r2kpXq
(xq = rk) = E[X2q ]
Byproduct: Eq. (2) can be expressed as conditional mean estimator
rk =
∫ tk+1
tk
xpXq(xq|x ∈ Ωk)dx = E[Xq|X ∈ Ωk]
M.R. Azimi Digital Image Processing
Image Quantization Lloyd-Max Quantizer Image Transforms
Remarks
1 Two commonly used PDF’s for quantization of image related dataare:
pX(x) =1√
2πσ2x
exp(− (x− µ)2
2σ2x
) Gaussian
pX(x) =α
2exp(−α|x− µ|) Laplacian
µ is the mean, σ2x is the variance (variance of the Laplacian PDF is
σ2x = 2
α ).
2 For uniform distribution, the Lloyd-Max quantizer become a linearquantizer. Let
pX(x) =
1tL+1−t1 t1 ≤ x ≤ tL+1
0 otherwise
Then from (2) we get rk =(t2k+1−t
2k)
2(tk+1−tk) = tk+1+tk2 . Now combining
this result with (1) yields tk = tk+1+tk−1
2 . Thus,
tk − tk−1 = tk+1 − tk = constant∆= q.
M.R. Azimi Digital Image Processing
Image Quantization Lloyd-Max Quantizer Image Transforms
Also
q =tL+1 − t1
L, tk = tk−1 + q, rk = tk + q/2
and we have rk − rk−1 = tk − tk−1 = q.i.e. all transition and reconstruction levels are equally spaced. The
quantization error e∆= x− xq is uniformly distributed over the
interval (−q/2, q/2); hence MSE is
ε =1
q
∫ q/2
−q/2α2dα =
q2
12
The variance σ2x of a uniform random variable (original image)
whose range is A is A2/12. For a uniform quantizer having B bitsL = 2B, we have q = A/2B. This gives
σ2x
ε= 22B ⇒ SNRdB = 10log1022B = 6B dB
i.e. in this special case the SNR achieved is 6dB/bit.M.R. Azimi Digital Image Processing
Image Quantization Lloyd-Max Quantizer Image Transforms
Image Transforms
2-D unitary transforms have found numerous applications in DIP. Thefollowings are some examples.
1 Data compression using transform coding: reduce the bandwidth bydiscarding or grossly quantizing low magnitude transformcoefficients.
2 Feature extraction: extract salient features of a pattern e.g. in FThigh frequency components correspond to the edge information inthe image.
3 Dimensionality reduction to principal components: reduce thedimensionality of the image space by discarding the smallcomponents.
4 Image filtering: Wiener filter and standard 2-D filters.
M.R. Azimi Digital Image Processing
Image Quantization Lloyd-Max Quantizer Image Transforms
Discrete Space Fourier Transform
The discovery of FFT for computing the FT of sampled signals made itpossible for the Fourier analysis of signals to be performed efficientlyusing digital computers. The computational saving made possible by FFTare even more important in the 2-D case when a large amount of data isinvolved. In what follows we start with 2-D DSFT and then show therelation between 2-D DSFT and 2-D DFT.
Spatial Sampling of Band-limited Images and DSFTWe saw that the 2-D FT of a continuous space image x(u, v) is
X(ω1, ω2) =
∫ ∫ ∞−∞
x(u, v)e−jω1ue−jω2vdudv
and the inverse FT is
x(u, v) =1
4π2
∫ ∫ ∞−∞
X(ω1, ω2)ej(ω1u+ω2v)dω1dω2
M.R. Azimi Digital Image Processing
Image Quantization Lloyd-Max Quantizer Image Transforms
For the sampled image i.e. x(m∆u, n∆v) or simply x(m,n) theFT becomes
X(ω1, ω2) =
∞∑m=−∞
∞∑n=−∞
x(m,n)e−jω1m∆ue−jω2n∆v
∆u,∆v are sampling intervals. Alternatively,
X(Ω1,Ω2) =∑ ∞∑
m,n=−∞x(m,n)e−j(Ω1m+Ω2n)
or 2D DSFTx(m,n) where Ω1∆= ω1∆u,Ω2
∆= ω2∆v, are
discrete frequency variables. Note that since ejΩn = ej(Ω+2kπ)n,we have X(Ω1,Ω2) = X(Ω1 + 2kπ,Ω2 + 2lπ), i.e. the 2-D DSFTis doubly periodic with periods 2π owing to spatial sampling atspacing ∆u and ∆v. The inverse DSFT considering this periodicbehavior becomes
x(m,n) =1
4π2
∫ ∫ π
−πX(Ω1,Ω2)ej(Ω1m+Ω2n)dΩ1dΩ2
M.R. Azimi Digital Image Processing
Image Quantization Lloyd-Max Quantizer Image Transforms
Example 1Determine the frequency response of a 2-D system for which the impulseresponse is
h(m,n) =
1 |m| < M, |n| < N0 otherwise
H(Ω1,Ω2) =∑ ∞∑
m,n=−∞h(m,n)e−j(Ω1m+Ω2n)
=
M−1∑m=−M+1
N−1∑n=−N+1
e−jΩ1me−jΩ2n
= (e−jΩ1(−M+1) − e−jΩ1M
1− e−jΩ1)
×(e−jΩ2(−N+1) − e−jΩ2N
1− e−jΩ2)
M.R. Azimi Digital Image Processing
Image Quantization Lloyd-Max Quantizer Image Transforms
Then
= (ejΩ1(M− 1
2) − e−jΩ1(M− 1
2)
ejΩ1/2 − e−jΩ1/2)
×(ejΩ2(N− 1
2) − e−jΩ2(N− 1
2)
ejΩ2/2 − e−jΩ2/2)
=sin(Ω1(M − 1/2))
sin(Ω1/2)× sin(Ω2(N − 1/2))
sin(Ω2/2)
Note that this is the aliased version of the 2-D sinc function due tothe sampling in the spatial domain (see Fig. M = 32 and N = 16).
−2
0
2
−2
0
2
0
500
1000
1500
Ω1
Aliased 2−D Sinc
Ω2
H(Ω
1,Ω
2)
M.R. Azimi Digital Image Processing
Image Quantization Lloyd-Max Quantizer Image Transforms
Example 2Determine the 2D DSFT of the image x(m,n) = amu(m)v(n) whereu(m) is a 1D unit step function and
v(n) =
1 0 ≤ n ≤ N − 10 elsewhere
X(Ω1,Ω2) =∑ ∞∑
m,n=−∞x(m,n)e−j(Ω1m+Ω2n)
=
∞∑m=−∞
amu(m)e−jΩ1m∞∑
n=−∞e−jΩ2nv(n)
=
∞∑m=0
ame−jΩ1mN−1∑n=0
e−jΩ2n
= (1
1− ae−jΩ1)× e−jΩ2( N−1
2 ) sin(NΩ2/2)
sin(Ω2/2)
M.R. Azimi Digital Image Processing