Post on 03-Apr-2018
transcript
7/28/2019 Color Texture Classication by Integrative Co-Occurrence Matrices
1/12
Pattern Recognition 37 (2004) 965976
www.elsevier.com/locate/patcog
Color texture classication by integrativeCo-occurrence matrices
Christoph Palm1
Institute of Medical Informatics, Aachen University of Technology, D-52057 Aachen, Germany
Received 26 November 2002; received in revised form 4 August 2003; accepted 18 September 2003
Abstract
Integrative Co-occurrence matrices are introduced as novel features for color texture classication. The extended
Co-occurrence notation allows the comparison between integrative and parallel color texture concepts. The information prot
of the new matrices is shown quantitatively using the Kolmogorov distance and by extensive classication experiments on
two datasets. Applying them to the RGB and the LUV color space the combined color and intensity textures are studied
and the existence of intensity independent pure color patterns is demonstrated. The results are compared with two baselines:
gray-scale texture analysis and color histogram analysis. The novel features improve the classication results up to 20% and
32% for the rst and second baseline, respectively.
? 2003 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved.
Keywords: Color texture; Co-occurrence matrix; Integrative features; Kolmogorov distance; Image classication
1. Introduction
Texture analysis is one of the main topics in the eld of
digital image processing. Texture is described as a pattern
with some kind of regularity. The approaches of mathemat-
ical modeling are grouped into structural, statistical and sig-
nal theoretic methods [1]. Structural methods are based on a
more or less deterministic arrangement of textural elements
(texels). They are mainly used in industrial quality control,
where artical patterns are regular and dierences betweenmodel and reality indicate failures [2,3]. Statistical meth-
ods dene textures as stochastic processes and characterize
them by a few statistical features. Most relevant statistical
approaches are Co-occurrence matrices [4], Markov random
elds [5] and autocorrelation methods [6]. Signal theoretic
approaches focus on periodic pattern resulting in peaks in
the spatial frequency domain, e.g. Gabor ltering [7,8] and
wavelet decomposition [9].
1 Christoph Palm is now with the Aixplain AG, Monheimsallee
22, D-52062 Aachen, Germany.
E-mail address: christoph.palm@web.de (C. Palm).
Beside texture, color is an important issue not only in hu-
man vision but in digital image processing where its impact
is still rising. In contrast to intensity, coded as scalar gray
values, color is a vectorial feature assigned to each pixel in
a color image. The mathematical dierence between scalars
and vectors for gray and color values, respectively, demands
a careful transfer of methods from the gray-scale to the color
domain. Although the use of color for texture analysis is
shown to be advantageous, the integration of color and tex-
ture is still exceptional. We dierentiate between parallel,sequential and integrative methods (Section 2).
Several studies favor the statistical texture modeling and
in particular Co-occurrence matrices in the gray-scale do-
main [10]. Nevertheless, just few approaches were made
to transfer Co-occurrence matrices to color images [35]. In
this paper, we study existing approaches for color texture
classication by Co-occurrence features and propose novel
matrices and features to allow the exploitation of the color
information.
After the introduction of general color texture con-
cepts (Section 2) and details of the most relevant color
spaces (Section 3), we shortly repeat the concept of
0031-3203/$30.00 ? 2003 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved.
doi:10.1016/j.patcog.2003.09.010
mailto:christoph.palm@web.demailto:christoph.palm@web.de7/28/2019 Color Texture Classication by Integrative Co-Occurrence Matrices
2/12
966 C. Palm / Pattern Recognition 37 (2004) 965 976
Co-occurrence matrices for gray-scale images (Section 4).
This concept is extended to the color domain (Section
5) dening single- and multi-channel matrices, respec-
tively. The description of the experimental setup (Section
6) is followed by the results and a detailed discussion
(Section 7).
2. Combining color and texture
We group the approaches combining color and texture
into parallel, sequential and integrative approaches.
The parallel concept for color texture analysis separates
the processing of both phenomena. Color is measured glob-
ally according to the histogram ignoring local neighboring
pixels. Texture is characterized by the relationship of the in-
tensities of neighboring pixels ignoring their color. The re-
sults of both analyses are combined subsequently to a feature
vector (Fig. 1, top). The parallel approach is advantageous,because the known methods of gray-scale texture analysis
as well as pixel based color analysis can be applied directly.
Up to now, parallel algorithms are most commonly used for
image retrieval applications with color and texture as sep-
arated discriminative features [11]. However, the view on
texture as a pure intensity based structure is very simplied
and disregards, for instance, colored texture primitives with
constant intensity.
Sequential approaches use color analysis as a rst step of
a process chain. Clustering the color histogram, a partition-
ing of the image is obtained. These consecutive numbered
segments contain color information, but they are just repre-sented by their indices and, hence, by scalars. Subsequently,
the index pattern is processed as gray-scale texture (Fig. 1,
middle). The sequential approach is related to the structural
texture model, where the pattern is composed by (color)
primitives. Hauta-Kasari et al. used it for industrial qual-
ity control [12], Song et al. especially for defect detection
in granite images [13], and Kukkonen et al. for inspection
of ceramic tiles [14]. These examples show the usefulness
of the sequential approach. Nevertheless, it is based on the
segmentation procedure, which depends on several parame-
ters and, therefore, gives no reproducible results. Addition-
ally, the assumption of colored texture primitives cannot be
generalized.The third way of combining the analysis of color and
texture is called integrative, because the information de-
pendency between both image features is taken into ac-
count. Integrative methods can be further dierentiated into
single- and multi-channel strategies (Fig. 1, bottom). The
single-channel methods apply the gray-scale texture anal-
ysis on each color channel separately. The color informa-
tion is used indirectly restricting the intensity pattern to the
wavelength interval associated with the color channel. The
advantage of the single channel approach is the easy adap-
tion of known methods based on the gray-scale domain.
Recently, some of these were compared with the paral-
grayscale image
color image
analysis
color histogram
feature
feature
analysis
segmen-
analysistation
color image color histogram label image
feature
color image
featureanalysis
feature
analysis
analysis
feature
feature
analysis
feature
analysis
analysis
feature
single channel
multi channel
color channels
Fig. 1. Top: Illustration of the parallel concept for color texture
analysis. Textural features and histogram features are strictly sep-
arated. Middle: Illustration of the sequential concept for color tex-
ture analysis. After the color segmentation of the histogram the
texture features are determined on basis of the segment indices.
Bottom: Illustration of integrative single- and multi-channel color
texture analysis. Both methods take color and texture into account.
lel concept and showed signicantly improved results [15].Integrative multi-channel texture analysis handles two (or
more) channels simultaneously. These approaches have al-
ready been proposed for well known textural features like
Markov random elds [16], wavelet [17], Gabor lters [18
21], and autocorrelation features [22], showing encouraging
results.
Up to now, Co-occurrence matrices are just adapted for
sequential [12,14] and integrative single-channel color tex-
ture analysis [15]. The idea of multi-channel Co-occurrence
matrices was rstly addressed by Rosenfeld et al. in 1982,
but found to be computationally cumbersome [23]. In this
paper, we rstly extend the Co-occurrence approach to
7/28/2019 Color Texture Classication by Integrative Co-Occurrence Matrices
3/12
C. Palm / Pattern Recognition 37 (2004) 965 976 967
integrative multi-channel color texture features. Addition-
ally, we introduce an information measure to describe the
supplementary information content of the multi-channel in
comparison to the single-channel approach. The extensive
experimental evaluation of the features in a classication
task takes two color spaces into consideration. The results
show the advantages of the multi-channel approach and
demonstrate the existence of non-intensity based pure color
textures.
3. Color spaces
To study the relation of intensity textures, pure color tex-
tures and combinations of both, respectively, the represen-
tation of intensity within the color space is important. The
RGB (red, green, blue) color space is a projection of the
cumulated intensity of reected light within a dened range
of wavelength to the axes of a three-dimensional space.Certainly, the RGB values are not only dependent on the
reectance properties of the observed pattern but on the il-
luminant and the recording characteristics of the camera.
The problems of color constancy and device independency
are studied in detail elsewhere [22,24] and are not topic of
this paper. Because of the wavelength-dependent intensity
cumulation each color band is inherently a combination of
color and intensity. Additionally, a strong correlation be-
tween the color channels is reported [25].
De-correlating the RGB color bands and separating color
and intensity yields the LUV color space [26]:L
U
V
= HLUV
R
G
B
=1
6
2
2
2
2 1 10
3
3
R
G
B
: (1)Tan et al. reported improving results for color histogram
analysis using LUV [26]. Independently from the idea of
mathematical de-correlation, a similar color space is denedmodeling human opponent color vision [27]. Additionally,
the LUV space is strongly related to the complex color space
since the color planes spanned by UV and the complex col-
ors are equal [28]. With RGB and LUV we study color
spaces where intensity and color are combined or separated,
respectively.
4. Gray-scale Co-occurrence matrices
In contrast to histograms, Co-occurrence matrices
(CMs) characterize the relationship between the values of
d
Fig. 2. Illustration of the concept of gray-scale Co-occurrence ma-
trices. The Co-occurrence matrices within one octant are combined
to one mean meatix to be more robust for long distances d.
neighboring pixels [4]. Therefore, they represent a secondorder statistical measurement. We denote the values of a
gray-scale image f as w {0; : : : ; W 1} with f(p) = w.The pixel position is given by p = (m; n) with m; nZ. Thedistance vector d is noted in polar coordinates (d; %) with
discrete length and orientation d; %N, respectively, com-puted from linear coordinates applying the polar transform
and an appropriate truncation. The probability Pr of two val-
ues w and w co-occurring with pixel positions related by d,
denes the cell entry (w1; w2) of the CM Cd:
Cd (w; w)
:= Pr(f(p1) = w f(p2)
= w| |p1 p2| = d): (2)Hence, Cd is a symmetric two-dimensional neighborhood
histogram, which depends on d. To study the eect of
changing d to the classication results, we extend Cd to
long-distance CMs. Fig. 2 visualizes the idea to build the
mean CM for each octant of a discrete circle approxima-
tion. The single CMs within the octant are summed up and
normalized keeping d constant. Because of the symmetric
denition of CM only four mean CMs remain. Neverthe-
less, with (W
W) they are high dimensional. To reduce
the large amount of data represented by CMs, two methods
are applicable:
quantize the image into few values as a preprocessing step extract special features to describe CM data as a
post-processing method
In this work we applied the last approach and used the matrix
features introduced by Haralick et al. [4]. Unfortunately,
some of the 14 features show redundancies, but it is not yet
clear, which of them can be ignored. Therefore, we selected
eight features (homogeneity, contrast, correlation, variance,
inverse dierence moment, entropy, correlation I, correlation
7/28/2019 Color Texture Classication by Integrative Co-Occurrence Matrices
4/12
968 C. Palm / Pattern Recognition 37 (2004) 965 976
II), that build the set union used in [10,15,29]. They are well
distributed over the four feature groups dened in [10].
The Cartesian product for the selected Haralick features
generates a (8 4)-space for the rotationally variant CMfeatures. To be rotationally invariant, the mean and the vari-
ance of the orientation-dependent features are determined
separately. Overall, the gray-scale CM features (GCFs) are
vectors of size 8 2.
5. Integrative color Co-occurrence matrices
According to our categorization (Section 2), in the follow-
ing two integrative extensions of gray-scale CMs to color
CMs are proposed. The single-channel Co-occurrence ma-
trices (SCMs) consist of gray-scale CMs successively ap-
plied to separated color channels. The correlation between
textures of dierent color channels is captured by novel
multi-channel Co-occurrence matrices (MCMs).
5.1. Single-channel Co-occurrence matrices
5.1.1. Background
For the denition of SCMs, we assume a limited
K-dimensional Cartesian space like RGB and LUV with
K = 3. A SCM SCkd corresponds to Cd applied to the kth
color channel fk of the image f = (f1; : : : ; fK)T:
Pr(wk
; wk
)
:= SCkd(w; w)
:= Pr(fk (p1) = w fk(p2) = w| |p1 p2| = d): (3)
The joint probability Pr(wk; wk) indicates the adjacency
of w and w on the same channel k. The corresponding
rotational invariant single-channel Co-occurrence features
(SCFs) consist of K feature vectors SCFk, where SCFk is
analog to GCF according to k: SCF = SCF1 SCFK.Following Section 3, RGB combines intensity as well
as color information. Therefore, RGB-based SCFs contain
combined color and intensity texture information. Recently,
Drimbarean et al. showed RGB-based SCF-like features tooutperform GCFs [15]. In contrast to RGB, LUV separates
intensity and color strictly. Hence, SCF1 represents pure
intensity texture and equals GCF. In contrast, pure color
textures are modeled by SCF2 and SCF3 according to the
channels U and V, respectively.
5.1.2. Information measurement
The quantication of the information prot of CMs in
comparison to common histograms is based on the Kol-
mogorov distance [30], which measures the dierence be-
tween two probability distributions. The one-dimensional
histogram Pr(wk
) for each channel k is derived from the
Image Number
Distance
1
0 30
0.2
0.8
1
Image Number
Distance
0 30
0.2
0.8
Image Number
Distance
1
0 30
0.2
0.8
Image Number
Distance
1
0 30
0.2
0.8
Fig. 3. Top left: Kolmogorov distance D(SCkd ) for single-channel
Co-occurrence matrices with k = R (white), k = G (black),
and k = B (gray). The image numbers follow the ascending
order of D(SCGd ). Bottom left: D(SCkd ) with k = L (black)
k = U (gray), and k = V (white). The image numbers fol-low the ascending order of D(SCLd ). Top right: D(MC
k1 ;k2d )
for multi-channel Co-occurrence matrices with (k1; k2) = (R; G)
(white), (k1; k2) = (R; B) (black), and (k1; k2) = (G; B) (gray). The
image numbers follow the ascending order of D(MCR;Bd ). Bottom
right: D(MCk1 ;k2d ) with (k1; k2) = (U; V) (black) (k1; k2) = (L; U)
(gray), and (k1; k2) = (L; V) (white). The image numbers follow
the ascending order of D(MCU;Vd ).
corresponding SCkd by summing up the columns:
Pr( wk
) =
Wk1
w=0 SCkd(w; w): (4)
In the case of stochastic independency ofd-adjacent values,
the joint probability Pr(wk; wk) is estimated by the multi-plicative combination of the univariate densities Pr(wk) and
Pr( wk):Pr(wk; wk) = Pr(wk) Pr( wk): (5)In contrast, the actually measured density Pr(wk; wk) is re-
lated to the conditional probability Pr(wk| wk):Pr(w
k; w
k) = Pr(w
k| wk) Pr( wk): (6)The Kolmogorov distance D(SCkd) as dierence betweenPr(wk; wk) and Pr(wk; wk) is equal to the dierence betweenrandom adjacency and the actual adjacency of w and w.Therefore, it is a measure of stochastic dependency and,
consequently, a measure of information gain of the CM in
contrast to the univariate histogram:
D(SCkd) =
1
2
Wk1w=0
Wk1w=0
|Pr(wk; wk) Pr(wk; wk)|: (7)With the help of D(SCkd), the general information prot of
CMs specic to the color channels is determined. Fig. 3
(left) shows the Kolmogorov distance values of 30 images
of the VisTex database (see Section 6) corresponding to
RGB and LUV.
7/28/2019 Color Texture Classication by Integrative Co-Occurrence Matrices
5/12
7/28/2019 Color Texture Classication by Integrative Co-Occurrence Matrices
6/12
970 C. Palm / Pattern Recognition 37 (2004) 965 976
Fig. 3 (right) shows D(MCk1 ; k2d ) for the natural color
textures of the VisTex database (see Section 6) for RGB
and LUV, respectively. According to RGB, the values
range from 0.1 to 0.7. Considering the baseline of zero for
stochastic independency of color and texture, this range
indicates clearly the high information prot using the novel
multi-channel Co-occurrence features in contrast to any
parallel method. The similarity of D(MCk1; k2d ) for dierent
color channels regarding to each image separately allows
their interpretation as a global color texture feature rather
than a channel-specic feature.
In contrast to RGB, the curves for dierent channel com-
binations of LUV vary signicantly. Whereas D(MCk1 ;k2d )
for the intensity independent combination (k1; k2) = (U; V)
reaches values in [0:7; 1:0], the distances according to the
combinations of color and intensity channels are mostly less
than 0.3. Therefore, the information prot analyzing inten-
sity independent color textures is very high and the existence
of pure intensity independent color texture obvious.
6. Experimental evaluation
Studying the Kolmogorov distance, we already showed
the existence of intensity independent color textures as well
as the usefulness of the integrative color texture analysis.
These results will be conrmed by experimental evaluation.
6.1. Experimental setup
To allow the generalization of the classication results,
two dierent image sets are used. From the VisTex dataset
[31] of MIT we choose 30 images of size 512512 (Fig. 4),each of which contains one natural colored texture. One tex-
ture class is built splitting one image into 64 non-overlapping
sub-images of size 64 64. Therefore, inter-class variationcorresponds to local variation of the full-size images. This
dataset is here addressed by DS1. The second dataset called
DS2 contains images of natural barks [32], built up in the
context of the PhD thesis of Lakmann [33]. The images con-
sist of one type of bark and background. Each image shows
a dierent tree, which is previously classied into one of six
classes (Fig. 5). Therefore, the inter-class variation reectsthe natural surface aberration of trees of the same kind. Each
class consists of 68 images of size 384 256 yielding acollection of 408 images. Since we apply an image classi-
cation not a texture segmentation, the image border is ex-
cluded dening a Region-of-Interest of xed size 300200located at the image center.
For classication, the leaving-one-out method is used to
guarantee strict separation of test and training set in com-
bination with the maximization of the number of training
images. A 5-Nearest-Neighbor classier using the Euclid-
ian vector distance was employed. To achieve a compara-
ble range of values for dierent features, a normalization
Fig. 4. Images of the VisTex database (leftright, updown): Bark0,
Bark4, Bark6, Bark8, Bark9, Brick1, Brick4, Brick5, Fabric0, Fab-
ric4, Fabric7, Fabric9, Fabric11, Fabric13, Fabric16, Fabric17, Fab-
ric18, Food0, Food2, Food5, Food8, Grass1, Sand0, Stone4, Tile1,
Tile3, Tile7, Water6, Wood1, Wood2 [31].
Fig. 5. Images of the BarkTex database, one example for each
class (leftright, updown): birch, beech, spruce, pine, oak,
robinia [32].
7/28/2019 Color Texture Classication by Integrative Co-Occurrence Matrices
7/12
C. Palm / Pattern Recognition 37 (2004) 965 976 971
Table 1
The notation dened in this table will be needed for understanding the following tables of classication results
Space Feature vector Abbreviation E
gray GCF m(L) 16
RGB SCF m(R), m(G), m(B) 16
RGB MCF m(RG), m(RB), m(GB) 16RGB HCF m(RG0), m(RB0), m(GB0) 8
LUV SCF m(L), m(U), m(V) 16
LUV MCF m(LU), m(LV), m(UV) 16
RGB m(R)m(G)m(B) m(RGBSC) 48
RGB m(RG)m(RB) m(GB) m(RGBMC) 48
RGB m(RGBSC) m(L) m(RGBSC; L) 64
RGB m(RGBMC)m(L) m(RGBMC; L) 64
RGB m(RG0)m(RB0)m(GB0) m(RGB0) 24
RGB m(RGB0)m(L) m(RGB0 ; L) 72
RGB m(RGB0)m(RGBSC) m(RGB0; SC) 72
RGB m(RGB0)m(RGBMC) m(RGB0; MC) 96
RGB m(RGBSC
) m(RGBMC
) m(RGBSC; MC
) 96
RGB m(RGB0)m(RGBSC)m(RGBMC) m(RGB0; SC; MC) 120
LUV m(U)m(V) m(UV) m(UVII) 48
LUV m(LU)m(LV)m(L) m(LUVID) 48
LUV m(UVII) m(LUVID) m(LUVII; ID) 96
The gray-scale Co-occurrence features, single-channel Co-occurrence features, multi-channel Co-occurrence
features and histogram Co-occurrence features are here noted as the feature vectors GCF, SCF, MCF and HCF,
respectively. The abbreviations of the feature vectors m make clear, which color channels are involved. These
vectors are combined by Cartesian product . The indices MC, SC and 0 indicate multi-channel Co-occurrence,
single-channel Co-occurrence and histogram features, respectively, in the RGB color space. For LUV, the
feature vectors are combined to intensity independent (II) and intensity dependent vectors (ID). E symbolizes
the dimension of the feature space.
according to variance e and mean e of each dimension eis done separately exploiting the training data TR:
m
e =me e
e(12)
with
e =1
M
mTR
me; e =
1
M 1
mTR
(me e)2: (13)
The eth element of an E-dimensional feature vector m and
the corresponding normalized element is denoted with meand me, respectively.
The optimal parameterd of a CM depends on the resolu-
tion of the texture. Zucker et al. developed a method to de-termine this optimal distance [34]. However, this optimum
refers to a distinct texture and one single color channel. In
best case, it can be generalized to the entire texture class. In
our classication experiments several classes are involved.
Hence, an optimal distance for all color channels and all
classes cannot be determined. Consequently, we studied the
color textures according to several distances. The follow-
ing experiments are performed with d = 0 (histogram) and
d = 1; 5; 10; 15; 20.
We combined the previously mentioned SCFs and MCFs
and compared them to pure gray-scale texture and pure color
histogram classication results. To give a clear reference to
the feature vectors m used in the experiments, the abbrevi-ations of the single- and combined feature vectors are listed
in Table 1.
Note, that the combinations in LUV are grouped into
intensity independent (II) and intensity dependent (ID)
whereas in RGB they are grouped into SCFs (SC) and
MCFs (MC).
6.2. Results
The classication results for DS1 and DS2 are listed in
Tables 2 and 3, respectively. The CM-representation of the
color histogram allows the direct comparison of the paralleland the integrative color texture concept (Section 2). Rows
13 of both tables show the results of gray-scale texture fea-
tures m(L) and the color histogram, described by its Haral-
ick features m(RGB0). Obviously, the importance of color
and intensity texture is quite dierent for the two datasets.
Whereas the HCFs forDS1 performs with 0:938 better than
the GCFs with 0:855, the texture features show better dis-
crimination eciency with 0:787 compared to 0:654 for
m(RGB0) according to DS2. For both datasets the parallel
concept combining the two single features outperform with
0.977 and 0.838 for DS1 and DS2, respectively, the results
of the single features.
7/28/2019 Color Texture Classication by Integrative Co-Occurrence Matrices
8/12
972 C. Palm / Pattern Recognition 37 (2004) 965 976
Table 2
Classication results for dataset DS1 according to dierent color texture feature combinations
DS1
0 1 5 10 15 20
1 m(L) 0.855 0.652 0.544 0.531 0.5182 m(RGB0) 0.938
3 m(RGB0; L) 0.977 0.945 0.929 0.910 0.928
4 m(R) 0.868 0.771 0.726 0.702 0.691
5 m(G) 0.869 0.779 0.697 0.691 0.677
6 m(B) 0.896 0.819 0.738 0.720 0.704
7 m(RGBSC) 0.951 0.884 0.826 0.811 0.793
8 m(RGBMC) 0.960 0.870 0.862 0.850 0.864
9 m(RGBSC; MC) 0.971 0.903 0.887 0.864 0.869
10 m(RGB0; SC) 0.974 0.951 0.928 0.906 0.913
11 m(RGB0; MC) 0.972 0.933 0.920 0.925 0.926
12 m(RGB0; SC; MC) 0.977 0.946 0.928 0.917 0.920
13 m(RGBSC; L) 0.946 0.868 0.821 0.807 0.804
14 m(RGBMC; L) 0.966 0.894 0.865 0.864 0.882
15 m(RGBSC; MC; L) 0.968 0.902 0.905 0.876 0.879
16 m(U) 0.886 0.754 0.666 0.609 0.595
17 m(V) 0.856 0.719 0.592 0.536 0.532
18 m(UV) 0.891 0.799 0.795 0.794 0.661
19 m(UVII) 0.969 0.910 0.867 0.840 0.843
20 m(LUVID) 0.948 0.895 0.870 0.852 0.863
21 m(LUVII; ID) 0.986 0.949 0.923 0.916 0.922
The dierent results in one row refer to CM distances from zero (color histogram) up to 20. The feature
combinations are divided into semantically interrelated blocks: parallel color texture concept (Rows 13),
integrative color texture concept in the RGB color space (Rows 49), combination of integrative features with
pure color and pure intensity texture features, respectively (Rows 10 15) integrative color texture concept in
the LUV color space (Rows 1621).
The results for SCF of the RGB color space are presented
in Rows 46. The best results are obtained by m(B) with
d = 1 and correctness 0.896 and by m(B) with d = 5 and
correctness 0.794 for DS1 and DS2, respectively. A general
superior performance of the B channel is not plausible and
depends on the image dataset. All single results for bothdatasets are superior to the corresponding pure gray-scale
texture result. These classication rates for m(R), m(G),
m(B) are improved signicantly (unidirectional Binomial
test statistic, p 0:05) by combining them to m(RGBSC)
(Row 7). According to m(RGBSC) we achieved a correct-
ness of 0.951 and 0.828 for DS1 and DS2, respectively. The
corresponding combination of MCFs to m(RGBMC) yield
with 0.960 better and with 0:795 even worth results forDS1and DS2. Combining m(RGBSC) and m(RGBMC) (Row 8),
the dimension of the feature space rises to 96 and, hence,
the complexity of classication task rises, too. Nevertheless,
the results improved to 0.971 and 0.841, respectively.
The results of Rows 1012 in Tables 2 and 3 allow the
evaluation of possible advantages combining RGB-related
integrative color texture features with the pure color his-
togram features. ForDS1 the combination of SCF, MCF and
both with histogram features showed slightly increasing re-
sults, respectively. The overall combination m(RGB0; SC; MC)performed best with a correctness of 0.977. According to
DS2 the combination of SCF with histogram features im-
prove the performance of SCF with a correctness of 0.841.
In contrast, the performance of the overall combination de-
creased to 0.808. However, the feature space dimension and,
therefore, the complexity of the classication problem, rose
to 120 for this combination (see Table 1). All results of tex-
ture feature combination with the histogram features outper-
form the pure histogram features.
Rows 1315 show the classication results combining
RGB-based SCFs and MCFs with GCFs. For both datasets
all color texture combinations performed better than the pure
7/28/2019 Color Texture Classication by Integrative Co-Occurrence Matrices
9/12
C. Palm / Pattern Recognition 37 (2004) 965 976 973
Table 3
Classication results of dataset DS2 according to dierent color texture feature combinations
DS2
0 1 5 10 15 20
1 m(L) 0.721 0.787 0.742 0.694 0.6372 m(RGB0) 0.654
3 m(RGB0; L) 0.831 0.838 0.814 0.765 0.767
4 m(R) 0.723 0.775 0.708 0.664 0.639
5 m(G) 0.745 0.777 0.740 0.689 0.627
6 m(B) 0.740 0.794 0.750 0.703 0.708
7 m(RGBSC) 0.787 0.828 0.784 0.708 0.666
8 m(RGBMC) 0.807 0.795 0.747 0.705 0.738
9 m(RGBSC; MC) 0.809 0.841 0.789 0.767 0.746
10 m(RGB0; SC) 0.826 0.841 0.806 0.794 0.779
11 m(RGB0; MC) 0.812 0.798 0.759 0.764 0.746
12 m(RGB0; SC; MC) 0.824 0.808 0.813 0.803 0.795
13 m(RGBSC; L) 0.777 0.824 0.784 0.703 0.668
14 m(RGBMC; L) 0.809 0.797 0.767 0.794 0.733
15 m(RGBSC; MC; L) 0.804 0.847 0.772 0.762 0.768
16 m(U) 0.658 0.776 0.714 0.646 0.595
17 m(V) 0.653 0.614 0.597 0.585 0.548
18 m(UV) 0.532 0.548 0.484 0.548 0.453
19 m(UVII) 0.756 0.807 0.759 0.778 0.729
20 m(LUVID) 0.786 0.740 0.694 0.729 0.673
21 m(LUVII; ID) 0.863 0.806 0.836 0.856 0.786
The notation corresponds to that of Tables 1 and 2.
gray-scale texture features. The combined features show
similar results as the corresponding single features in Rows
79. Therefore, no tendency can be remarked which indi-
cates the usefulness of these combinations. For both datasets
the best result of this block is achieved by the overall com-
bination m(RGBSC; MC; L). It yields a correctness of 0:968
and 0:847 for DS1 and DS2, respectively.
The results related to the intensity independent single
feature vectors of the LUV color space are listed in Rows
1618. ForDS1 the best performance is achieved with 0:891
by m(UV) and d = 1. On the other hand, for DS2 the bestcorrectness of 0:776 is achieved by m(U) and d = 5. There-
fore, both are below the respective maxima according to the
single RGB features (Rows 46).
For both datasets the results are enhanced by combin-
ing the single features to m(UVII) (Row 19). With 0:969
and 0:807 for DS1 and DS2, respectively, the improvement
is signicant (p 0:05). The intensity independent texture
features are in both cases superior to the intensity depen-
dent features (Row 20). Note, that m(L) is inherently in-
tegrated into m(LUVID) as SCF1 of the LUV color space.
The best results are obtained by the combination of both, in-
tensity independent and intensity dependent features (Row
21). These are superior not only to other combinations of
the RGB color space but to the parallel color texture con-
cept m(RGB0; L) as well. The results 0:986 and 0:863 for
DS1 and DS2, respectively, refer both to d = 1.
Taking d into account, a strong tendency to an opti-
mum distance for each dataset can be stated. All supe-
rior results for each experiment block are found for d = 1
and (except one) for d = 5 according to DS1 and DS2,
respectively.
7. Discussion
In our experiments, gray-scale texture and color histogram
analysis were compared with novel color texture features.
Additionally, two dierent concepts of color texture analy-
sis, parallel and integrative, were evaluated. This was possi-
ble for the rst time, because in the framework of CMs color
histograms and textural features are both well described.
The two image datasets used are rather dierent in content,
color texture appearance as well as discrimination capabil-
ity of color histogram and gray-scale texture. Nevertheless,
the tendencies of the results are quite similar. Therefore,
7/28/2019 Color Texture Classication by Integrative Co-Occurrence Matrices
10/12
974 C. Palm / Pattern Recognition 37 (2004) 965 976
it is possible to generalize them to large application elds
dealing with natural color textures.
It has been shown, that color information improve the re-
sults of gray-scale texture features [14]. Our experiments
support these results. The combination of color histograms
and gray-scale texture features shows a high supplementa-
tion ability. The results of this parallel color texture concept
are just slightly below the best results we achieved by inte-
grative color texture features.
Whereas the parallel concept is most commonly applied
in the color image processing community, this paper intro-
duced novel integrative color texture features to learn more
about the aspects of color texture and develop new meth-
ods to model them. According to SCMs, we realize notice-
able dierences between the single-channel results. This in-
dicates dierences in the textures of dierent color channels
and relativizes their correlation. This observation supports
our evaluation of Kolmogorov distances in Section 5. Addi-
tionally, the channel-specic features supplement each otherresulting in a signicant better performance for the combi-
nation features. This enhancement is noticeable in particu-
lar considering the increase of dimensionality of the feature
space. Because the gray-scale texture results are below all
single-channel results, the transform from color to gray val-
ues is lossy with respect to textural features.
The novel method of MCMs improve the results of the
SCMs taking the correlation between color channels into ac-
count. With the help of the Kolmogorov distance, we demon-
strated that the multi-channel modeling is not equivalently
realized by the single-channel features in combination with
the color histogram. Therefore, MCMs measure new colortextural features. The low redundancy of both integrative
methods is also shown by the experimental evaluation. The
combination ofm(RGBSC) and m(RGBMC) leads to increas-
ing correctness although the feature space dimensionality
rises by a factor of two.
The combination of the SCFs with the histogram features
show rather similar results to the parallel color texture con-
cept. This could be expected since the only dierence is the
already stated wavelength dependency of colored texture.
Unfortunately, the information prot of texture modeling
subsequently on the color channels is over-compensated by
the disproportionate extension of the feature space. Never-
theless, color histogram and single-channel texture analysissupplement each other. Therefore, the classication correct-
ness rises in comparison to both single components. On the
other hand, the combination of m(RGBSC) with the GCFs
yields worse results according to both components. Again,
this shows the high redundancy of GCFs and SCFs. Study-
ing the redundancy of MCFs with HCFs and GCFs, re-
spectively, in both cases better results in comparison with
the single components can be stated. Therefore, a new kind
of texture information is measurable by the multi-channel
analysis.
In addition to RGB, the experimental evaluation take
the LUV color space into account. Although the methods
of single- and multi-channel color texture analysis are the
same, the interpretation of the results has to be adapted.
Because of the two intensity independent channels U and
V we achieve three intensity dependent and three intensity
independent matrices. Obviously, the intensity independent
features model a completely dierent kind of texture. Al-
though the RGB related color texture features consider both
color and texture, the texture is again intensity based, even
though this intensity is measured on a small wavelength in-
terval. In contrast, the MCMs related to U and V charac-
terize the Co-occurrence of pure colors. Because the three
single feature vectors m(U), m(V) and m(UV) represent
dierent parts of the color plane, they supplement each other
and, hence, their combination yields to a signicant better
classication result. Additionally, the results to the intensity
independent features are better than that of the intensity de-
pendent. The novel pure color texture features in compari-
son with pure intensity texture features show the overall best
performance. Again, this proves the fact low redundancybetween these two concepts.
8. Conclusion
In this paper, we presented novel integrative single- and
multi-channel Co-occurrence matrices for color texture anal-
ysis and put intensity independent color textures in perspec-
tive.
The multi-channel approach integrates color histograms
into the Co-occurrence notation and allows the comparison
of parallel and integrative approaches on the same levelfor the rst time. Additionally, the Kolmogorov distance
is recommended to measure quantitatively the information
gain of the multi-channel approach.
The advantages of the integrative methods were demon-
strated by the Kolmogorov distance as well as by the re-
sults of the experimental evaluation. Nevertheless, because
of the disproportional increase of feature space dimension-
ality, the best results according to RGB are achieved by the
parallel concept of separate analysis of color by histogram
and texture by intensity pattern analysis. However, the inte-
grative color features based on the LUV color space show
not only the existence of intensity independent color texture
but its discriminative power. In combination with the com-mon intensity dependent pattern, colored textures are more
adequately described.
Summary
Color image analysis and texture classication are two
basic tasks in pattern recognition. The dependencies of
both are still not clear and, hence, several concepts dealing
with colored textures exist. The objective of this paper is
to strengthen the basis for the discussion in the eld of
color texture recognition. To achieve this goal, we treated
7/28/2019 Color Texture Classication by Integrative Co-Occurrence Matrices
11/12
C. Palm / Pattern Recognition 37 (2004) 965 976 975
ve main elds within the framework of Co-occurrence
matrices:
Categorization of the concepts for color texture featureextraction.
The identication of these concepts enables an overview
over the relevant literature and allows the comparisonbetween these approaches. Two of them, the parallel and
the integrative method, have been compared within the
experimental section.
Extension of Co-occurrence matrices to multi-channelcolor texture characterization.
Since Co-occurrence matrices are one of the most fre-
quently used methods for gray-scale texture analysis,
multi-channel matrices enlarges the toolbox of color
texture analysis methods signicantly.
Application of Kolmogorov distance to the novel single-and multi-channel matrices.
The Kolmogorov distance serves as a quantitative mea-sure to determine the dependency of intensity textures and
colors. Therefore, the discussion about this dependency
in the scientic community can be adjudged objectively
for each image separately.
Application of integrative Co-occurrence matrices to theRGB as well as the LUV color space.
Since the LUV space separates one intensity channel L
and two color channels U and V, Co-occurrence matri-
ces according to U and V provide intensity independent
and, hence, pure color texture features. This features are
used to prove the existence of intensity independent color
textures.
Classication experiments on two large image databases.The large number of experimental data allows the gen-
eralization of the conclusions drawn on the basis of the
results. In particular, the parallel and the integrative con-
cept are compared in the framework of Co-occurrence
matrices.
Two main conclusions have to be emphasized. Firstly, the
parallel concept of intensity texture analysis and separately
color histogram analysis is suitable for color image classi-
cation. Secondly, the existence of pure color textures as well
as their supplementation ability with respect to intensity de-
pendent textures features is proven. Therefore, the integra-tive color texture features introduced in this paper enable
the characterization of intensity independent color textures
for the rst time. Using these novel features in combina-
tion with gray-scale texture and color histogram features the
classication correctness is increased up to 32%.
References
[1] T.R. Reed, J.M.H. Du Buf, A review of recent texture
segmentation and feature extraction techniques, CVGIP
Image Understanding 57 (3) (1993) 359372.
[2] I. Tus, Automated fabric inspection based on a structural
texture analysis method, in: Recent Issues in Pattern Analysis
and Recognition, Springer, Berlin (1989) 377390.
[3] A. Bodnarova, M. Bennamoun, K.K. Kubik, Suitability
analysis of techniques for aw detection in textiles
using texture analysis, Pattern Anal. Appl. 3 (2000)
254266.[4] R. Haralick, K. Shanmugam, I. Dinstein, Texture features
for image classication, IEEE Trans. Syst. Man Cybernet.
3 (1973) 610621.
[5] Lei Wang, Jun Liu, Texture classication using
multi-resolution markov random eld models, Pattern
Recognition Lett. 20 (2) (1999) 171182.
[6] M. Pietikainen, T. Ojala, Z. Xu, Rotation-invariant texture
classication using feature distributions, Pattern Recognition
33 (2000) 4352.
[7] A.K. Jain, F. Farrokhnia, Un-supervised texture segmentation
using Gabor lters, Pattern Recognition 24 (12) (1991)
11671186.
[8] T. Randen, J.H. HusH
y, Multichannel ltering forimage texture segmentation, Opt. Eng. 33 (8) (1994)
26172625.
[9] Jing-Wein Wang, Chin-Hsing Chen, Wei-Ming Chien,
Chih-Ming Tsai, Texture classication using non-separable
two-dimensional wavelets, Pattern Recognition Lett. 19 (13)
(1998) 12251234.
[10] C. Gotlieb, H. Kreyszig, Texture descriptors based on
Co-occurrence matrices, Comput. Vision Graph. Image
Process. 51 (1990) 4752.
[11] K. Messer, J. Kittler, A region-based image database system
using colour and texture, Pattern Recognition Lett. 20 (1999)
13231330.
[12] M. Hauta-Kasari, J. Parkkinen, T. Jaaskelainen, R. Lenz,
Multi-spectral texture segmentation based on the spectral
Co-occurrence matrix, Pattern Anal. Appl. 2 (4) (1999)
275284.
[13] K.Y. Song, J. Kittler, M. Petrou, Defect detection in
random colour textures, Image Vision Comput. 14 (1996)
667683.
[14] S. Kukkonen, H. Kalviainen, J. Parkkinen, Color features for
quality control in ceramic tile industry, Opt. Eng. 40 (2)
(2001) 170177.
[15] A. Drimbarean, P.F. Whelan, Experiments in colour texture
analysis, Pattern Recognition Lett. 22 (2001) 11611167.
[16] P.-H. Suen, G. Healey, Modeling and classifying color
textures using random elds in a random environment, Pattern
Recognition 32 (1999) 10091017.[17] G. Van de Wouwer, P. Scheunders, S. Livens, D. Van
Dyck, Wavelet correlation signatures for color texture
characterization, Pattern Recognition 32 (1999) 443451.
[18] A.K. Jain, G. Healey, A multi-scale representation including
opponent color features for texture recognition, IEEE Trans.
Image Process. 7 (1) (1998) 124128.
[19] C. Palm, D. Keysers, T.M. Lehmann, K. Spitzer, Gabor
ltering of complex hue/saturation images for color texture
classication, Joint Conference on Information Science 2,
Atlantic City, NJ, USA (2000) 4549.
[20] G. Paschos, Perceptually uniform color spaces for color texture
analysis: an empirical evaluation, IEEE Trans. Image Process.
10 (6) (2001) 932937.
7/28/2019 Color Texture Classication by Integrative Co-Occurrence Matrices
12/12
976 C. Palm / Pattern Recognition 37 (2004) 965 976
[21] C. Palm, T.M. Lehmann, Classication of color textures by
Gabor ltering, Mach. Graph. and Vision 11 (2/3) (2002)
195219.
[22] G. Healey, L. Wang, Illumination-invariant recognition of
texture in color images, J. Opt. Soc. Am. A 12 (9) (1995)
18771883.
[23] A. Rosenfeld, C.-Y. Wang, A.Y. Wu, Multispectral texture,IEEE Trans. Syst. Man and Cybernet. SMC-12 (1) (1982)
7984.
[24] T.M. Lehmann, C. Palm, Local line search for illuminant
estimation in real world color scenes, J. Opt. Soc. Am. A
18 (11) (2001) 26792691.
[25] B.A. Wandell, Foundations of Vision Science, Sinauer Ass.,
Sunderland, MA, USA, 1995.
[26] T.S.C. Tan, J. Kittler, Colour texture analysis using colour
histogram, IEE Proc. Vision Image Signal Process. 141 (6)
(1994) 403412.
[27] M. Mirmehdi, M. Petrou, Segmentation of color textures,
IEEETrans.Pattern Anal. Mach. Intell.22 (2)(2000) 142159.
[28] A. McCabe, T. Caelli, G. West, A. Reeves, Theory of
spatiochromatic image encoding and feature extraction, J. Opt.Soc. Am. A 17 (10) (2000) 17441754.
[29] A. Bradley, P. Jackaway, B. Lovell, Classication in
scale-space: applications to texture analysis, 14th International
Conference on Information Processing in Medical Imaging,
Ile de Berder, France (1995) 375376.
[30] A.N. Kolmogorov, Three approaches to the quantitative
denition of information, Int. J. Comput. Math. 2 (1968)
157168.[31] Database VisTex of color textures of MIT, USA:
ftp://whitechapel.media.mit.edu/pub/VisTex.
[32] Database of color textured barks of University of Koblenz,
Germany: ftp://ftphost.uni-koblenz.de/outgoing/vision/
Lakmann/BarkTex.
[33] R. Lakmann, Statistische Modellierung von Farbtexturen,
Ph.D. Thesis, University Koblenz-Landau, Folbach, Koblenz,
1998 (in German).
[34] S.W. Zucker, D. Terzopoulos, Finding structure in
Co-occurrence matrices for texture analysis, Comput. Graph.
Image Process. 12 (1980) 286308.
[35] J.F.R. Ilgner, C. Palm, A.G. Schutz, K. Spitzer, M.
Westhofen, T.M. Lehmann, Colour texture analysis for
quantitative laryngoscopy, Acta Otolaryngol. 123 (2003)730734.
About the AuthorCHRISTOPH PALM studied Computer Science at the Aachen University of Technology (RWTH), Germany, and
received his Diploma degree in 1997. Between 1997 and 2001 he was Ph.D. student at the Institute of Medical Informatics at the University
Hospital of the RWTH. During his Ph.D. research, his research interests covered all aspects of pattern recognition, texture classication
and color image processing. He is co-editor of a proceedings volume and author of several scientic conference and journal papers. He is
peer reviewer of the Optical Society of America. In 2001 Christoph Palm joined the Aixplain AG, Aachen. Beside image processing, his
research focus is now enlarged to information retrieval and statistical machine translation.
ftp://whitechapel.media.mit.edu/pub/VisTexftp://whitechapel.media.mit.edu/pub/VisTexftp://ftphost.uni-koblenz.de/outgoing/vision/Lakmann/BarkTexftp://ftphost.uni-koblenz.de/outgoing/vision/Lakmann/BarkTexftp://ftphost.uni-koblenz.de/outgoing/vision/Lakmann/BarkTexftp://ftphost.uni-koblenz.de/outgoing/vision/Lakmann/BarkTexftp://ftphost.uni-koblenz.de/outgoing/vision/Lakmann/BarkTexftp://whitechapel.media.mit.edu/pub/VisTex