ReQuest(Validating Semantic Searches)

Norman Piedade de Noronha16th July, 2004

2

Limitations in searching for information on the Web

Lack of syntax: Information is unstructured.

Lack of semantics: Machine processes do not understand the meaning of information.

Unable to properly filter information for users, leading to information overload.

3

The Semantic Web (SW)

SW is a proposal for a Web of machine interpretable data.

Purpose: Automate user and computer tasks. Goal: Add structure and semantics to the

existing Web with metadata and ontologies.

4

Metadata

Describes data about a web resource and not the actual content. Ex: Author, Date.

Resource Description Format (RDF) is the SW standard for metadata representation.

5

Ontologies

Ontologies provide a shared and common understanding of concepts that can be communicated between people and heterogeneous distributed systems.

Ontologies are used in the SW as dictionaries, thesauri and vocabularies.

OWL (Web Ontology Language) is the W3C standard for ontologies representation.

6

Semantic Web Searches

Ex: Contacting an author of a certain article in a particular newspaper.

Article Article’s Author Author’s Name Author’s Email

7

Objectives

1. Build a Semantic Web search environment prototype: ReQuest.

2. Evaluate & compare searches against conventional search engines.

8

Hypothesis

“Searches based on ontologies improve user satisfaction and reduce effort by eliminating irrelevant results.”

9

Outline

Motivation & The Semantic WebReQuest– Architecture– Prototype

ReQuest for NewsValidation Conclusions & Future Work

10

The Semantic Web Stack

11

ReQuest & the SW

XML + NS + xmlschema

RDF

Ontology

ReQuest

12

ReQuest: Use Cases

RequestSearchEngine

ReQuest: Architecture

INTERFACE ENGIN

E

INTERFACE ENGIN

E

REPOSIT

ORY

REPOSIT

ORY

METADATA SUBSYSTEM

METADATA SUBSYSTEM

ONTOLOGY SUBSYSTEM

ONTOLOGY SUBSYSTEM

14

Implementation

Ontology Subsystem – Java, RDF Schema

Metadata Subsystem – Java, RDF & RSS

Repository – SQL (PosGreSQL)

Interface Engine – HTML & Java Servlets

15

Outline

Motivation & The Semantic Web

ReQuestArchitecture

Prototype

ReQuest for News

Validation

Conclusions & Future Work

16

Configure new domain in ReQuest

1. Selecting Input Data.

2. Configuring ReQuest.

3. Defining Equivalences.

4. Launching New Domain.

17

News Domain: Person Ontology

18

News Domain: Ontologies

Newspaper– Created RDF Schema File

Document– Imported from Dublin Core

19

News Domain: Metadata Files

Newspaper– Created RDF File

Person– Created RDF File

Document– Imported from RSS News Feeds

20

News Domain: Setup

Configuring ReQuest– Adding Ontology – Metadata Associations.– Defining periodicity of retrieval of metadata.

Defining Equivalences Launching a New Domain

– Retrieve and Process Ontologies– Generate template for interface.

21

ReQuest User Interface

•Person

•Newspaper

•Document

22

ReQuest User Interface

•Person

•Newspaper

•Document

23

ReQuest User InterfaceReQuest User Interface

24

Validation

Survey5 “volunteers”.9 queries on news domain.ReQuest vs. Google.

25

Examples of search queries

(Q1) Find the post office address for the publisher Público.

(Q9) How many distinct articles were published by Público about Futebol between the 5th of January, 2004 and the 7th of January, 2004.

26

Individual Query Survey

How hard was the query to formulate?Did the semantic links help find the information?How long did it take to find the information?How relevant was the obtained information for your need?How many results were not interesting in the first page?Which search system was easier to use?

27

User Feedback

Improve interface.

Rank search results.

Reduce information by providing a reduced version of results.

Search within results.

Search with properties from different contexts.

Domain search preferred to Global search.

28

Hypothesis Validation

“Searches based on ontologies improve user satisfaction and reduce effort by eliminating irrelevant results.”

Measurements:1.Information Need Satisfaction.2.Effort Reduction.3.Irrelevant Results Reduction.

29

Measure 1: Information Need Satisfaction

Only one user achieved greater success with Google.

Google’s results better in only one out of nine queries.

30

Measure 2:Effort Reduction

Majority of users were successfully aided by ReQuest approximately 7 times, while only 20% managed to solve more than half of the tests with less effort with Google.

Some users did not produce a single test query where Google required less effort than ReQuest.

31

Measure 3: Irrelevant Results Reduction

Only first page results were compared. 80% of users found fewer irrelevant results

with ReQuest than with Google. ReQuest was more precise than Google for

48.9% of all questions, while Google was more precise for 24.4%.

32

Outline

Motivation & The Semantic Web

ReQuestArchitecture

Prototype

ReQuest for News

Validation

Conclusions & Future Work

33

Conclusions

Positive Feedback:Searches based on ontologies improved user satisfaction

Reduced effort by eliminating irrelevant results.

Offering users the ability to select the search context is a more exact method for expressing the information need than key words.

34

Conclusions

Results not statistically significant due to:Population

User Profile

Queries

35

Future Work

More Extensive Validation. Domain searches enhanced with ability to

restrict values. Support OWL. Multilingual Searches.

36

References

Presented at Interacção 2004 as

Norman Noronha, Mário J. Silva

Using the Semantic Web for Web Searches

July 2004.

Digital Deposit.

TUMBA – A Portuguese Search Engine.

37

THANKS

Thank you very much for your attention.