Segmentation-Based Image Compression
以影像切割為基礎的影像壓縮技術
Speaker: Jiun-De HuangAdvisor: Jian-Jiun Ding
Graduate Institute of Communication EngineeringNational Taiwan University, Taipei, Taiwan, ROC
2
Outline
• Introduction to Image Compression• Segmentation-Based Image Compression• Edge Detection• Image Segmentation• Boundary Description and Compression• Proposed Methods for Boundary Description• Internal Texture Compression• Comclution• Future Work
3
Introduction to Image Compression
• Why we need to compress the image?– Save disk space– Save transformation bandwidth
• The common type of image compression– DCT-based method: JPEG– Wavelet-based method: JPEG2000
4
Introduction to Image Compression
Color Component
of an Image
Transform Coding( DCT or Wavelet )
Quantization EntropyCoding
Bit-stream
• Image compression model
Bit-stream Transform DecodingEntropy
Decoding
Color Componentof an image
Encoder
Decoder
5
Segmentation-Based Image Compression
Image segments of DCT:
Object-oriented segments:
6
Segmentation-Based Image Compression
• Segmentation-based image compression model
Arbitrary-ShapedTransform Coding
Quantization &Entropy Coding
Bit-streamImage
Segmentation
Boundary Transform Coding
Quantization &Entropy Coding
Internal texure
Boundary
Coefficients oftransform bases
Boundarydescriptor
An image
7
Segmentation-Based Image Compression
• Advantage– Pixels in the same segment have extremly high correlation, the c
ompression ratio can be higher.– The boundary of a segment is recorded separately, it may make
the image clear in high compression ratio.– Application in image recognize
• Disadvantage– Large time to encode and decode– Hard to find a common way to segment various images.
8
Edge Detection
• First-order derivatives
• Second-order derivatives
• Hilbert transform
• Short time Hilbert transform
9
Edge Detection
0 50 100
-1
0
1
0 50 100
-1
0
1
0 50 100
-1
0
1
0 50 100
-1
0
1
0 50 100
-1
0
1
0 50 100
-1
0
1
(a) (b)
(c) (d)
(e) (f)
0 50 100
-1
0
1
0 50 100
-101
0 50 100
-1
0
1
0 50 100
-101
0 50 100
-1
0
1
0 50 100
-101
(a) (b)
(c) (d)
(e) (f)
Using differentiation Using HLT
Sharp edge
Step edgeWith noise
Ramp edge
10
Edge Detection
• Short Time Hilbert Transform– Impulse responses and their FTs of the SRHLT for different b. W
e can compare them with the impulse response of the differential operation and the original HLT.
-2 -1 0 1 2-1
0
1
-2 -1 0 1 2
-1
0
1(a) (b)
Time domain Frequency domain
Hilbert transformFT
-2 -1 0 1 2-1
0
1
-2 -1 0 1 2-10
0
10(i) (j)differentiation
FT
-2 -1 0 1 2-1
0
1
-2 -1 0 1 2
-1
0
1
-2 -1 0 1 2-1
0
1
-2 -1 0 1 2
-1
0
1
-2 -1 0 1 2-1
0
1
-2 -1 0 1 2
-1
0
1
(c) (d)
(e) (f)
(g) (h)
Time domain Frequency domain
SRHLT, b=0.25
SRHLT, b=1
SRHLT, b=4
FT
FT
FT
11
Edge Detection
• Short Time Hilbert Transform– Using SRHLTs to detect the sharp edges, the step edges with n
oise, and the ramp edges. Here we choose b = 1, 4, 12, and 30.
100-1
0
1
0 50 100-1
0
1
0 50 100-1
0
1
0 50 100-0.5
00.5
0 50 100-0.5
0
0.5
0 50 100-0.5
00.5
(g)
(i)
(k)
(h)
(j)
(l)
b = 12 b = 30
0 50 100
-101
0 50 100
-101
0 50 100
-101
0 50 100-1
0
1
0 50 100
-1
0
1
0 50 100-1
0
1
(a)
(c)
(e)
(b)
(d)
(f)
b = 1 b = 4
12
Edge Detection
(a) Original image (b) Results of differentiation
(c) Results of the HLT (d) Results of the SRHLT, b=8 (c) Results of the HLT (d) Results of the SRHLT, b=8
(a) image+noise, SNR=32 (b) Results of differentiation
• Short Time Hilbert Transform
13
Image Segmentation
• Thresholding
Gray-level histograms that can be partitioned by (a) Single threshold, and (b) multiple thresholds
14
Image Segmentation
• Edge Linking– Hough transform
Two point in the coordinate The coefficient space
15
Image Segmentation
• Edge Linking– Hough transform
Two points in thePolar coordinate
Coefficient space
16
Image Segmentation
• Region Growing
• Region Splitting and Merging
17
Image Segmentation
• Watershed
18
Boundary Description and Compression
• Polygonal approximations– Merging techniques
– Splitting techniques
19
Boundary Description and Compression
• Fourier descriptor– Set the coordinate of the K-point boundary as a series of comple
x number s(k), k=0,1,…,K-1.– The Fourier descriptor is define as the DFT of s(k).
( ) ( ) ( ), 0,1,...,s k x k jy k k K
12 /
0
1( ) ( ) , 0,1,..., 1
Kj uk K
k
a u s k e u KK
The DFT of s(k)
The inverse DFT of a(u)1
2 /
0
( ) ( ) , 0,1,..., 1K
j uk K
u
s k a u e k K
20
Boundary Description and Compression
• Fourier descriptor– If we only use the first P coefficients, the detail of the recover
boundary will be lost. Smaller P becomes, more detail lost.
Original image R=30% R=20% R=10%
12 /
0
( ) ( )ˆP
j uk K
u
s k a u e
Compression rate: R = P/K
21
Proposed Methods for Boundary Description
• Improvement of Fourier descriptor– We segment the boundary with the corner point and only comput
e the Fourier desriptor of the boundary segment– However, if we do not use the whole coefficients, the recovery b
oundary segment will be closed due to the discontinuous of the two end point
u
a(u)
0 P K
Boundarysegment
Fourierdescriptor
Recoverboundary
truncate
22
Proposed Methods for Boundary Description
• Improvement of Fourier descriptor– To solve the non-closed problem, we adapt the following steps:
1. Record the coordinate of the two end of the boundary segment and shift them to the original of coordinate
2. Shift the other boundary points linearly according to its distance with the end point
3. Add a new boundary which is odd symmetry to the original one
Boundarysegment
Shift linearly
Add a new boundary
23
Proposed Methods for Boundary Description
• Improvement of Fourier descriptor4. Compute the Fourier descriptor to the new boundary which is cl
osed and is continuous in the two end points
5. Because the new boundary is odd symmetry, the Fourier descriptor is odd symmetry, too. There is, we only need to record the first K points of the Fourier descriptor.
( ) ( )DFTx n X k
u
a(u)
0 K 2K-2
Fourierdescriptor
useless
24
Proposed Methods for Boundary Description
• Improvement of Fourier descriptor– Simulation
R=20% R=10% R= 7%Originalimage
generalFourier
descriptor
modifiedFourier
descriptor
25
Internal Texture Compression0 1 2 3 4 5 6 7
0 1 2 3 4 5 6 7
v
uThe 8x8 DCT basis
26
Internal Texture Compression0 1 2 3 4 5 6 7
0 1 2 3 4 5 6 7
v
uThe Arbitraryly-shapedDCT basis
27
Internal Texture Compression0 1 2 3 4 5 6 7
0 1 2 3 4 5 6 7
v
uThe Arbitraryly-shapedDCT basis
Use zig-zag order to do Gram-Schmidt orthonormalize
28
Internal Texture Compression
The Arbitraryly-shaped DCT orthnormal basis
1 2 3 4 5 6 7 8
33 34 35 36 37
9 10 11 12 13 14 15 16
17 18 19 20 21 22 23 24
25 26 27 28 29 30 31 32
29
Internal Texture Compression
0 5 10 15 20 25 30 35 40-100
0
100
200
300
400
500
An arbitraryly-shaped image
The 37 AS-DCT coefficients
AS-DCT
Example:
30
Conclusion
• The compression rate depend on the complex of the image content.
• This compression method is better when the image content is simple.
• There are various method in each step, they suit different image respectly.
31
Future Work
• Find a better method of segmentation which is suit to this compression method.
• Automatic analysis the property of the image and choose the fittest method in each step.
• How to apply this compression method to the image recognize technique.