of 23
8/8/2019 Meta-search Engine (1)
1/23
Evaluation of Meta-Search engine
Merge Algorithms
Presentation by
8/8/2019 Meta-search Engine (1)
2/23
Concept
The core of Meta search engine is the result integration algorithms.
This presentation focuses on evaluating the integration algorithms
ofmeta-search engine, and compares meta-search to the general
search engine by experiments.
An experiment method to determine the priority of participant of
meta-search engine is also proposed.
The experiment result proved that the meta-search engine could get
quality searching result on average.
8/8/2019 Meta-search Engine (1)
3/23
Existing System:
General search engines like Google, Yahoo, MSN etc.
Drawbacks:
As there are huge no. of documents on the world wide web, it is
very difficult to locate the information that is relevant to the users
interest.
8/8/2019 Meta-search Engine (1)
4/23
Proposed System:
Meta Search engine merge algorithms.
Benefits:
The queries posted by users which are submitted to the metasearch engine.
It sends the query to multiple single search engines intern.
When retrieved items are returned by the underlying search engines
the meta-search engine further processes these items and presentsrelevant items to the user, based on a integration algorithm.
8/8/2019 Meta-search Engine (1)
5/23
Explanation
It is very difficult to locate information that is relevant to the internet
users interest, as there are huge no. of documents on the world
wide web.
There are several approaches to help the users to find informationfrom the web.
One is meta-searchmethod.
Queries are submitted to the meta-search engine which sends the
query to multiple single search engines.
8/8/2019 Meta-search Engine (1)
6/23
When retrieved items are returned by the underlined search
engines further processes these items and presents relevant itemsto the user based on a result integration algorithm.
JP callam proposed 4 typical synthesis algorithms to process
different situations.
Kirsch provided another typical method which required the
participant search engine to return some additional information.
The meta-search engine would use this information to re calculate
the relevance of document on the client side.
Meta crawler imported the concept credibility to determine the
relevance between documents and queries.
8/8/2019 Meta-search Engine (1)
7/23
Profusions algorithm was one of the normalized score method.
Inquires adopted a merge strategy which also re calculated the
relevance on the client side.
I am evaluating and then compare in experiments 4 merge
algorithms through this project
1. Simple merge algorithm
2. abstract merge algorithm
3. Position merge algorithm
4. abstract / position merge algorithm
8/8/2019 Meta-search Engine (1)
8/23
Simple merge algorithm
This approach is similar to the single search engine, and it just to
range the results from participate search engines intern by using a
multi wave merge method.
In practical user are interest in search results of the first three pagesand high probability.
The result after three pages could only enhance the completeness
of the search results but could not improve user experience
anymore.
8/8/2019 Meta-search Engine (1)
9/23
8/8/2019 Meta-search Engine (1)
10/23
Abstract merge algorithm:
The main idea of abstract merge algorithm is to rank search
results with the relevance between query and the abstract
information of search results.
First, we need extract the terms from query, and calculate the
relevance between terms and abstract.
Second, we calculate the relevant between query and each page.
Finally, the results are returned to users according to theirrelevance between query and abstract of one page.
8/8/2019 Meta-search Engine (1)
11/23
8/8/2019 Meta-search Engine (1)
12/23
Position merge algorithm:
1. Description of the algorithm
The basic idea of position merge algorithm is to make a full use of the
original position information from each single search engine.
For the same query, there are some pages which will occur in severalresult list of different participant search engines, but their position in
different result lists may not be the same.
To reconcile this contradiction , we should take the position in different
participants into account.
8/8/2019 Meta-search Engine (1)
13/23
2. Determining the priority
In the position algorithm the priority of search engines need to be
determined in advance.
In this experiment, I used two main search engines Google and Yahoo asthe participants.
8/8/2019 Meta-search Engine (1)
14/23
8/8/2019 Meta-search Engine (1)
15/23
Abstract/ position merge algorithm:
The abstract and the position are very important information.
abstract/ position merge algorithm considered these two factors
synthetically to make the
integrated results to meet the user needs.
Following is the formulae to calculate the relavance bertween the
result and query.
8/8/2019 Meta-search Engine (1)
16/23
8/8/2019 Meta-search Engine (1)
17/23
Valuation and Analysis:
The evaluation of the results from different merge algorithms:In this section, i am going to compare the results of Simple merge
algorithm, abstract merge algorithm, Position merge algorithm and
abstract / position
merge algorithm, then estimates which result is better and morerelevant to the user query intent.
we use to main the measures accuracy, which is the ratio of the
results in the whole results. and they wait, namely the location of
the related results in the whole results.
Firstly, setting up 10 keywords which are more broadly, more
representative and more accordant with the users' queries,
such as
8/8/2019 Meta-search Engine (1)
18/23
Beijing Olympic Games, Beijing University 2007 Bill gates,
Dell Computer, Heilongjiang University, Jackie Chan
Movie, the NBA Yaoming, Prison break introduction, The
DaVinci Code, The Great Wall, Thinking in JAVA.
Secondly, for each query , we take the first top 30 results for
three algorithms as the basis data set for analyzing. Every
result will be judged whether the
results are related to the keywords or not based on the content
of each result for each query by human.
If one result is relevant, we will mark the location of the result
which occur in the 30 result. and then calculate the weight of
the each relevant result with the weight function.
8/8/2019 Meta-search Engine (1)
19/23
Compare meta-search engine with Google and Yahoo:
We established a meta search engine which take Google and yahoo
as the search engines.
We used the above 4 merge algorithms to fusion the results from the
participants.
We did a test, to evaluate whether the meta search engine could
improve the experience of the user.
We choose following 5 queries as the testing set:
"Bill Gates Beijing University 2007", "Heilongjiang University" , " The
great Wall", " Beijing Olympic Games" and "Dell Computer".
8/8/2019 Meta-search Engine (1)
20/23
Firstly, we need fetch the top 30 results fromGoogle and
Yahoo respectively for every query.
Secondly, we use the same way to estimate the relevance of
these results respectively and mark the positions.
Then calculate wait for every result and accumulate the waits.and get the mean value. Then we get accuracy, (the no of
related results) / 30
8/8/2019 Meta-search Engine (1)
21/23
The following are the evaluating results as shown in the tables
The weight and accuracy of Google and Yahoo
Key word
Accuracy Weight
Yahoo
Accuracy Weight
Bill Gates Beijing
University 2007
0.6667 5.4207 0.5333 5.8663
Heilongjiang
University
0.3667 5.4580 0.3333 4.9942
The great Wall 0.6 5.3192 0.5667 5.5166
Beijing Olympic
Games
0.4333 4.6565 0.4667 5.0207
Dell Computer 0.5667 5.2430 0.5667 5.4057
Average 0.52668 5.1295 0.49334 5.3607
8/8/2019 Meta-search Engine (1)
22/23
The comparison of the weight and accuracy between the three ranking
algorithms of Meta-Search engine and Google and Yahoo.
accuracy weight
Google 0.5467 5.2195
Yahoo 0.4933 5.3607
Summary 0.5649 5.3965
Position 0.5567 5.4083
Summary/position 0.5749 5.3966
8/8/2019 Meta-search Engine (1)
23/23
Conclusion:
In this project we design an experiment to evaluate abstract
merge algorithm, position merge algorithm, abstract/position
merge algorithm ofmeta search
engine and compare the meta- search engine with the generalsearch engines such as Google and Yahoo.
From the experiment results it is concluded that the bettermerge
algorithm could improve the quality of searching.