Content-Based Image Retrieval Using Fuzzy Cognition Concepts

Presented by

Tienwei TsaiDepartment of Computer Science and Engineering

Tatung University2005/9/30

Outline

1. Introduction 1. Introduction

2. Problem Formulation 2. Problem Formulation

3. Proposed Image Retrieval System3. Proposed Image Retrieval System

4. Experimental Results4. Experimental Results

5. Conclusions5. Conclusions

1. Introduction

• Two approaches for image retrieval: – query-by-text (QBT): annotation-based image

retrieval (ABIR)– query-by-example (QBE): content-based

image retrieval (CBIR)

• Standard CBIR techniques can find the images exactly matching the user query only.

• In QBE, the retrieval of images basically has been done via the similarity between the query image and all candidates on the image database. – Euclidean distance

• Transform type feature extraction techniques– Wavelet, Walsh, Fourier, 2-D moment, DCT, and

Karhunen-Loeve.

• In our approach, the DCT is used to extract low-level texture features. – the energy compacting property of DCT

2. Problem Formulation

• Let I be the image database with I := {Xn | n = 1, . . ., N} where Xn is an image represented by a set of features: Xn := {xn m | m = 1, . . ., M}. – N and M are the number of images in the image database and

the number of features, respectively.

• To query the database, the dissimilarity (or distance) measure D(Q, Xn) is calculated for each n as

– dm is the distance function or dissimilarity measure for the mth feature and wm R is the weight of the mth feature.

– Query image Q := {qm | m = 1, …, M}.– For each n, holds. By adjusting the weights wm it is possible to

emphasize properties of different features.

.,,1 ,),(.),(1

N...for nxqdwXQDM

mnmmmmn

3. The Proposed Image Retrieval System

Figure 1. The proposed system architecture.

Feature Extraction

• Features are functions of the measurements performed on a class of objects (or patterns) that enable that class to be distinguished from other classes in the same general category.

• Color Space TransformationRGB (Red, Green, and Blue) ->

YUV (Luminance and Chroma channels)

YUV color space

• YUV is based on the CIE Y primary, and also chrominance.– The Y primary was specifically designed to follow the

luminous efficiency function of human eyes. – Chrominance is the difference between a color and a

reference white at the same luminance.

• The following equations are used to convert from RGB to YUV spaces:

– Y(x, y) = 0.299 R(x, y) + 0.587 G(x, y) + 0.114 B(x, y),

– U(x, y) = 0.492 (B(x, y) - Y(x, y)), and

– V(x, y) = 0.877 (R(x, y) - Y(x, y)).

2 Feature Extraction via DCT

• The DCT coefficients F(u, v) of an N×N image represented by f(i, j) can be defined as

where

1

0

1

0

),()()(2

),(N

i

N

j

jifvuN

vuF ),2

)12(cos()

2

)12(cos(

N

vj

N

ui

.1

,021

)(otherwise

wforw

Discrete Cosine Transform

Characteristics of DCT

• the DC coefficient (i.e. F(0, 0)) represents the average energy of the image;

• all the remaining coefficients contain frequency information which produces a different pattern of image variation; and

• the coefficients of some regions represent some directional information.

Similarity Measurement

• Distance measure – the sum of absolute differences (SAD): avoid

multiplications. – the sum of squared differences (SSD): exploit

the energy preservation property of DCT

• The distance between qm and xnm under the low frequency block of size k×k :

1

0

1

0

2),(),(),(k

u

k

vxqnmmm vuFvuFxqd

nmm

Fuzzy Cognition Query

• To benefit from the user-machine interaction, we develop a GUI for fuzzy cognition, allowing users to adjust the weight of each feature more easily according to their preferences.

• Each image is represented by M features. • Three features (i.e., luminance Y,

chrominance U, and chrominance V) are considered for each image.

4. Experimental Results

• 1000 images downloaded from the WBIIS database are used to demonstrate the effectiveness of our system.

• The user can query by an external image or an image from the database.

• In our experiments, we found that the low frequency DCT coefficients of size 5×5 are enough to make a fair quality of retrieval.

Figure 2. Retrieved results using a butterfly as the query image and its luminance as the main feature.

Figure 3. Retrieved results using a butterfly as the query image and emphasizing the weight of its V component.

Figure 5. Retrieved results using a mountain scene as the query image and Its Y component as the main feature:

(a) the query image; (b) the retrieved images.

(a)

(b)

Figure 4. Retrieved results using a mountain scene as the query image and Its U component as the main feature:

(a) the query image; (b) the retrieved images.

(a)

(b)

5. Conclusions

• In this paper, a content-based image retrieval method that exploits fuzzy cognition concepts is proposed.

• To achieve QBE, the system compares the most significant DCT coefficients of the Y, U, and V components of the query image and those of the images in the database and find out good matches by the help of users’ cognition ability.

• Since several features are used simultaneously, it is necessary to integrate similarity scores resulting from the matching processes.

• An important part of our system is the implementation of a set of flexible weighting factors for this reason.

Future Works

• For each type of feature we will continue investigating and improving its ability of describing the image and its performance of similarity measuring.

• A long-term aim is combining the semantic annotations and low-level features to improve the retrieval performance.

• For the analysis of complex scenes, the concept that provide a high amount of content understanding enable highly differentiated queries on abstract information level. The concept is worthy of further study to fulfill the demands of integrating semantics into CBIR.

Thank You !!!