Undefined 1 (2009) 1–5 1 IOS Press A Session-based Ontology Alignment Approach for Aligning Large Ontologies Editor(s): Name Surname, University, Country Solicited review(s): Name Surname, University, Country Open review(s): Name Surname, University, Country Patrick Lambrix a,∗ and Rajaram Kaliyaperumal b a Department of Computer and Information Science and Swedish e-Science Research Centre Linköping University SE-581 83 Linköping, Sweden b Department of Computer and Information Science Linköping University SE-581 83 Linköping, Sweden Abstract. There are a number of challenges that need to be addressed when aligning large ontologies. Previous work has pointed out scal- ability and efficiency of matching techniques, matching with background knowledge, support for matcher selection, combination and tuning, and user involvement as major requirements. In this paper we address these challenges. Our first contribution is an ontology alignment framework that enables solutions to each of the challenges. This is achieved by introducing different kinds of interruptable sessions. The framework allows partial computations for generating mapping suggestions, partial validations of mapping suggestions and use of validation decisions in the (re-)computation of mapping suggestions and the recommendation of alignment strategies to use. Further, we describe an implemented system providing solutions to each of the challenges and show through experiments the advantages of our approach. Keywords: Ontologies, Ontology engineering, Ontology alignment 1. Introduction 1 In recent years many ontologies have been devel- oped and many of those contain overlapping informa- tion. Often we want to use multiple ontologies. For in- stance, companies may want to use community stan- dard ontologies and use them together with company- specific ontologies. Applications may need to use on- tologies from different areas or from different views on one area. In each of these cases it is important to know the relationships between the terms in the dif- * Corresponding author. E-mail: [email protected]. 1 This paper is an extended version of [19]. ferent ontologies. Further, the data in different data sources in the same domain may have been anno- tated with different but similar ontologies. Knowledge of the inter-ontology relationships would in this case lead to improvements in search, integration and anal- ysis of data. It has been realized that this is a ma- jor issue and much research has recently been done on ontology alignment, i.e. finding mappings between terms in different ontologies (e.g. [7]). The research field of ontology alignment is very active with its own yearly workshop as well as a yearly event, the Ontology Alignment Evaluation Initiative (OAEI, e.g. [6]), that focuses on evaluating systems that automat- ically generate mapping suggestions. Many systems 0000-0000/09/$00.00 c 2009 – IOS Press and the authors. All rights reserved
Transcript
Undefined 1 (2009) 1–5 1IOS Press
A Session-based Ontology AlignmentApproach for Aligning Large OntologiesEditor(s): Name Surname, University, CountrySolicited review(s): Name Surname, University, CountryOpen review(s): Name Surname, University, Country
Patrick Lambrixa,∗ and Rajaram Kaliyaperumalb
a Department of Computer and Information Scienceand Swedish e-Science Research CentreLinköping UniversitySE-581 83 Linköping, Swedenb Department of Computer and Information ScienceLinköping UniversitySE-581 83 Linköping, Sweden
Abstract.There are a number of challenges that need to be addressed when aligning large ontologies. Previous work has pointed out scal-
ability and efficiency of matching techniques, matching with background knowledge, support for matcher selection, combinationand tuning, and user involvement as major requirements. In this paper we address these challenges. Our first contribution is anontology alignment framework that enables solutions to each of the challenges. This is achieved by introducing different kindsof interruptable sessions. The framework allows partial computations for generating mapping suggestions, partial validations ofmapping suggestions and use of validation decisions in the (re-)computation of mapping suggestions and the recommendation ofalignment strategies to use. Further, we describe an implemented system providing solutions to each of the challenges and showthrough experiments the advantages of our approach.
In recent years many ontologies have been devel-oped and many of those contain overlapping informa-tion. Often we want to use multiple ontologies. For in-stance, companies may want to use community stan-dard ontologies and use them together with company-specific ontologies. Applications may need to use on-tologies from different areas or from different viewson one area. In each of these cases it is important toknow the relationships between the terms in the dif-
* Corresponding author. E-mail: [email protected] paper is an extended version of [19].
ferent ontologies. Further, the data in different datasources in the same domain may have been anno-tated with different but similar ontologies. Knowledgeof the inter-ontology relationships would in this caselead to improvements in search, integration and anal-ysis of data. It has been realized that this is a ma-jor issue and much research has recently been doneon ontology alignment, i.e. finding mappings betweenterms in different ontologies (e.g. [7]). The researchfield of ontology alignment is very active with itsown yearly workshop as well as a yearly event, theOntology Alignment Evaluation Initiative (OAEI, e.g.[6]), that focuses on evaluating systems that automat-ically generate mapping suggestions. Many systems
2 P. Lambrix and R. Kaliyaperumal / A Session-based Ontology Alignment Approach for Aligning Large Ontologies
have been built and overviews can be found in e.g.[15,31,39,7,23,40] and at the ontology matching website http://www.ontologymatching.org.
The existing frameworks for ontology alignmentsystems (e.g. [5,24]) describe different componentsand steps in the ontology alignment process suchas preprocessing, matching, filtering and combiningmatch results, and user validation of the mapping sug-gestions generated by the ontology alignment system.Systems based on the existing frameworks functionwell when dealing with small ontologies, but thereare a number of limitations when dealing with largerontologies. Some recent work (e.g. [40,14]) has de-fined challenges that need to be addressed when deal-ing with large ontologies. According to [14] interac-tivity, scalability, and reasoning-based error diagno-sis are required to deal with large ontologies. In [40]the following challenges related to aligning large on-tologies are defined. Regarding scalability [40] dis-cusses efficiency of matching techniques. This is im-portant as many participants in the OAEI have perfor-mance problems when dealing with large ontologies.Further, matching with background knowledge shouldbe used (which could include in the [40] interpretationof background knowledge the error diagnosis of [14]).Based on OAEI experience it is also clear that there isa need for support for matcher selection, combinationand tuning. There is also a need for user involvement inthe matching process. First, the user could be involvedduring the mapping generation. Further, as stated bythe OAEI organizers [6], automatic generation of map-pings is only a first step towards a final alignment anda validation by a domain expert is needed. In this pa-per we address these challenges. Ourfirst contributionis an ontology alignment framework that enables scal-ability, user involvement, use of background knowl-edge and matcher selection, combination and tuning(Section 3). This is achieved by introducing differentkinds of interruptable sessions: computation, valida-tion and recommendation sessions. It is the first frame-work that allows partial computations for generatingmapping suggestions. Currently, to our knowledge, nosystem allows to start validating mapping suggestionsbefore every suggestion is computed. It also is the firstframework that allows a domain expert to validate asub-set of the mapping suggestions, and continue lateron. Further, it supports the use of validation results inthe (re)computation of mapping suggestions and therecommendation of alignment strategies to use. Oursecondcontribution is the first implemented systemthat integrates solutions for these challenges in one
system (Section 4). It is based on our session-basedframework. It deals with efficiency of matching tech-niques by, in addition to the sessions, avoiding exhaus-tive pair-wise comparisons between the terms in thedifferent ontologies. It provides solutions to match-ing with background knowledge by using previousdecisions on mapping suggestions as well as usingthesauri and domain-specific corpora. Matcher selec-tion, combination and tuning is achieved by using ap-proaches for recommending matchers, combinationsand filters. Further, user involvement is supported inthe validation phase through user interfaces that havetaken into account earlier experiments with ontologyengineering systems user interfaces. User decisions aretaken into account in the matching and recommenda-tion steps. Ourthird contribution are two kinds of ex-periments (Section 5) that show the advantages of thesession-based approach. The first kind of experimentsshows the use of our framework and system and pointto alignment quality improvements based on the newfunctionality. The second kind of experiments showshow such a system can be used for evaluating strate-gies that could not (easily) be evaluated before.
2. Background
In general, from a knowledge representation pointof view, ontologies may contain concepts, relations,axioms and instances. Concepts and relations are of-ten organized in hierarchies using the is-a (or sub-sumption) relation, denoted by⊑. The task of ontol-ogy alignment is to create an alignment between on-tologies. An alignment is a set of mappings (also calledcorrespondences) between entities from the differentontologies. The most common kinds of mappings areequivalence mappings (≡) as well as mappings usingis-a and its inverse (⊑, ⊒). For instance, for conceptsA from the first ontology and A’ from the second on-tology<A,A’,≡> represents the fact that A and A’ areequivalent.
A large number of ontology alignment systems havebeen developed. Many of these are based on the com-putation of similarity values between terms in differ-ent ontologies and can be described as instantiationsof the general framework in Figure 1. The frameworkconsists of two parts. The first part (I in Figure 1) com-putes mapping suggestions. The second part (II ) inter-acts with the user to decide on the final alignment.
An alignment algorithm receives as input two sourceontologies. Part I typically contains different compo-
P. Lambrix and R. Kaliyaperumal / A Session-based Ontology Alignment Approach for Aligning Large Ontologies 3
alignment
ontologies
generaldictionary
instancecorpus
domainthesaurus
matchermatcher
matcher
Preprocessing
checkerconflict
user
II
I
accepted and
suggestionsrejected
filter
combination
suggestionsmapping
Fig. 1. An existing framework (extension of the framework in [24]).
nents. A preprocessing component can be used to mod-ify the original ontologies, e.g. to extract specific fea-tures of the concepts in the ontologies, or to parti-tion the ontologies into mappable parts thereby reduc-ing the search space for finding mapping suggestions.The algorithm can include several matchers that calcu-late similarities between the terms from the differentsource ontologies or mappable parts of the ontologies.They often implement strategies based on linguisticmatching, structure-based strategies, constraint-basedapproaches, instance-based strategies, strategies thatuse auxiliary information or a combination of these.Each matcher utilizes knowledge from one or multi-ple sources. Mapping suggestions are then determinedby combining and filtering the results generated byone or more matchers. Common combination strate-gies are the weighted-sum and the maximum-basedstrategies. The most common filtering strategy is the(single) threshold filtering. By using different prepro-cessing, matching, combining and filtering techniques,we obtain different alignment strategies. The result ofpart I is a set of mapping suggestions.2
In part II the mapping suggestions are then pre-sented to the user, a domain expert, who accepts or re-jects them. The accepted mapping suggestions are part
2Traditionally, in the OAEI it is this result (and thus part I)that isevaluated. In 2013, for the first time there was a track for evaluatinginteraction and thus also some issues related to part II.
of the final alignment. The acceptance and rejectionof suggestions may also influence further suggestions.Further, a conflict checker could be used to avoid con-flicts introduced by the mapping suggestions.3
There can be several iterations of parts I and II. Theoutput of the alignment algorithm is a set of mappingsbetween terms from the source ontologies. All systemsimplement part I while some also implement part IIand allow iterations.
In the next section we propose a framework that in-cludes the existing framework in some of its compo-nents.
3. Framework
Our new framework is presented in Figure 2. The in-put are the ontologies that need to be aligned. The out-put is an alignment between the ontologies which con-sists of a set of mappings that are accepted after vali-dation. The framework defines three kinds of sessions:computation, validation and recommendation sessions.When starting an alignment process the user starts acomputation session. When a user returns to an align-
3During the recent years some systems allow not only for conflictchecking but also for repairing of mappings or mapping suggestions,e.g., [27,14,29,33,13].
4 P. Lambrix and R. Kaliyaperumal / A Session-based Ontology Alignment Approach for Aligning Large Ontologies
Fig. 2. Framework.
ment process, she can choose to start or continue acomputation session or a validation session.
During thecomputation sessionsmapping sugges-tions are computed. The computation may involve pre-processing of the ontologies, matching, and combina-tion and filtering of matching results (as in part I ofthe old framework). Auxiliary resources such as do-main knowledge and dictionaries may be used. A rea-soner may be used to check consistency of the pro-posed mapping suggestions in connection with the on-tologies as well as among each other (as in part II in theold framework). Users may be involved in the choiceof algorithms. This is similar to what most ontologyalignment systems do. However, in this case the algo-rithms may also take into account the results of previ-ous validation and recommendation sessions. Further,we allow that computation sessions can be interruptedand partial results can be delivered. It is therefore pos-sible for a domain expert to start validation of resultsbefore all mapping suggestions are computed. The out-put of a computation session is a set of mapping sug-gestions.
During the validation sessionsthe domain expertvalidates the mapping suggestions generated by thecomputation sessions. A reasoner may be used tocheck consistency of the validations. The output of avalidation session is a set of mapping decisions (ac-cepted and rejected mapping suggestions). The ac-cepted mapping suggestions form a partial alignment(PA) and are part of the final alignment. The mappingdecisions (regarding acceptance as well as rejection ofmapping suggestions) can be used in future computa-tion sessions as well as in recommendation sessions.
Validation sessions can be interrupted and resumed atany time. It is therefore not neccesary for a domain ex-pert to validateall mapping suggestions in one session.The user may also decide not to resume the validationbut start a new computation session, possibly based onthe results of a recommendation session.
The input for therecommendation sessionscon-sists of a database of algorithms for the preprocessing,matching, combination and filtering in the computa-tion sessions. During the recommendation sessions thesystem computes recommendations for which (combi-nation) of those algorithms may perform best for align-ing the given ontologies. When validation results areavailable these may be used to evaluate the different al-gorithms, otherwise an oracle may be used. The outputof this session is a recommendation for the settings ofa future computation session. These sessions are nor-mally run when a user is not validating and results aregiven when the user logs in into the system again.
Most existing systems can be seen as an instanti-ation of the framework with one or more computa-tion sessions. Some systems also include one valida-tion session.
4. Implemented System
We have implemented a prototype based on theframework described above. We have used and ex-tended some components from the SAMBO system
P. Lambrix and R. Kaliyaperumal / A Session-based Ontology Alignment Approach for Aligning Large Ontologies 5
Fig. 3. Screenshot: start session.
Fig. 4. Screenshot: start computation session.
[24] and developed and implemented several new com-ponents.4
4.1. Support for sessions
When starting an alignment process for the firsttime, the user starts a computation session. However,if the user has previously stored sessions, then a screenas in Figure 3 is shown and the user can start a newsession or resume a previous session. The informationabout sessions is stored in the session managementdatabase. This includes information about the user, theontologies, the list of already validated mapping sug-gestions, the list of not yet validated mapping sugges-tions, and last access date. In the current implemen-tation only validation sessions can be saved. When acomputation session is interrupted, a new validation
4In the text we explicitly mention which components are takenfrom or further developed from previous work. When nothing ismentioned, it means we have developed new algorithms.
session is created and this can be stored. When a userends or interrupts a session, the user can ask the sys-tem to, using the obtained validation decisions, filterthe non-validated mapping suggestions, preprocess thedata for a future session or compute a recommendationfor the settings of a new computation sesssion.
4.2. Computation sessions
4.2.1. Settings selectionFigure 4 shows a screenshot of the system at the
start of a computation session. It allows for the settingof the session parameters. During thesettings selec-tion the user selects algorithms for the matching, com-bining and filtering steps as well as whether prepro-cessed data should be used. An experienced user maychoose her own settings. Otherwise, the suggestion ofa recommendation session (by clicking the ’Use rec-ommendations from predefined strategies’ button) or adefault setting may be used. It is also possible to in-spect a list of predefined strategies as well as a list of
6 P. Lambrix and R. Kaliyaperumal / A Session-based Ontology Alignment Approach for Aligning Large Ontologies
the top recommended strategies with their recommen-dation scores and select a strategy from these lists. Thesettings selection is stored in the session informationdatabase. The computation session is started using the’Start Computation’ button.
4.2.2. PreprocessingWhen a PA is available (e.g. after an (interrupted)
validation session - in this case this step can be initi-ated after the end or interruption of a previous valida-tion session), thepreprocessingstep partitions the on-tologies into corresponding mappable parts accordingto the method we developed in [20]. This method com-putes corresponding mappable parts that make sensewith respect to the structure of the ontologies. Themethod retrieves a sub-set of the equivalence mappingsin the PA (called consistent group) such that each con-cept occurs at most once as first argument in a map-ping, at most once as second argument in a mappingand for each pair of selected equivalence mappings<A,A’,≡> and<B,B’,≡> where A and B are con-cepts in the first ontology and A’ and B’ are conceptsin the second ontology, we require that A⊑ B iff A’⊑ B’. This sub-set respects the is-a hierarchy in thetwo ontologies and is then used to partition the twoontologies. Each element (which is a set of concepts)in the partition of the first ontology has a correspond-ing element (which is a set of concepts) in the par-tition of the second ontology and only mappings be-tween concepts in corresponding elements respect thestructure of the ontologies. Therefore, the matcherswill not compute similarity values between all pairsof concepts, but only between concepts in mappableparts, thereby considerably reducing the search space.The user may choose to use this preprocessing step bychecking the ’use preprocessed data’ check box (Fig-ure 4).
As an example, consider the two ontologies in Fig-ure 5 where the nodes represent concepts and theedges inverses of is-a relations (e.g., the concept rep-resented by node 2 is a sub-concept of the conceptrepresented by node 1). Then the set of mappings{<2,B,≡>, <3,F,≡>, <6,D,≡>, (5,C,≡>} is not aconsistent group as the concept represented by 5 isa sub-concept of the concept represented by 2, butthe concept represented by C is not a sub-concept ofthe concept represented by B. However, {<2,B,≡>,<3,F,≡>, <6,D,≡>} is a consistent group. Using aconsistent group we can partition the two ontologies.A mapping<A,A’,≡> divides the two ontologies intothree parts. The first ontology is divided into (i) the
descendants of A, (ii) A and (iii) the rest. The secondontology is divided into (i) the descendants of A’, (ii)A’ and (iii) the rest. We use each mapping in the con-sistent group in this way. By taking the intersectionsof all these parts, we obtain different pieces in the firstontology that have a corresponding piece in the secondontology. For instance, consider the ontologies in Fig-ure 5 and the consistent group {<2,B,≡>, <3,F,≡>,<6,D,≡>}. In this case the first ontology has the fol-lowing pieces: descendants of node 6 (empty), node 6,descendants of node 2 that are not node 6 or descen-dants of node 6 (node 5), node 2, descendants of node3 (empty), node 3, the rest (nodes 1, 4, 7, 8). The sec-ond ontology is divided into: descendants of node D(empty), node D, descendants of node B that are notnode D or descendants of node D (node E), node B, de-scendants of node F (empty), node F, the rest (nodes A,C). The corresponding mappable parts between the on-tologies are: ({5}, {E}) and ({1,4,7,8}, {A,C}) (Figure6).
4.2.3. MatchersMatcherscompute similarity values between terms
in different ontologies. Whenever a similarity valuefor a term pair using a matcher is computed, it isstored in the similarity values database. This can bedone during the computation sessions, but also dur-ing the recommendation sessions. In the current im-plementation we have used string matching for match-ing relations. Regarding concepts, the matchers com-pute similarity values between pairs of concepts as re-ceived from the preprocessing step (all pairs or pairsof concepts in mappable parts). We use the linguistic,WordNet-based, UMLS-based and instance-based al-gorithms from the SAMBO system [24]. The matchern-gram computes a similarity based on 3-grams. Ann-gram is a set of n consecutive characters extractedfrom a string. Similar strings have a high proportionof n-grams in common. The matcherTermBasicuses acombination of n-gram, edit distance and an algorithmthat compares the lists of words of which the terms arecomposed.5 A Porter stemming algorithm is employedto each word. The matcherTermWNextends TermBa-sic by using WordNet [47] for looking up is-a relations.The matcherUMLSM uses the domain knowledge in
5This is similar to a combination of n-gram, edit distance and Jac-card. According to [1] this should give good results for the f-measurefor standard ontologies. Also according to [1], for biomedical on-tologies edit distance gives good precision while Jaccard gives goodrecall and f-measure.
P. Lambrix and R. Kaliyaperumal / A Session-based Ontology Alignment Approach for Aligning Large Ontologies 7
5 6
2 3
7 8
4
1
B
D E
A
C
F
Fig. 5. Ontologies.
5 6
2 3
7 8
4
1
B
D E
A
C
F
Fig. 6. Partitions.
the Unified Medical Language System (UMLS, [42])to obtain mapping suggestions. Finally, the instance-based matcherNaiveBayesmakes use of research liter-ature that is related to the concepts in the ontologies. Itis based on the intuition that a similarity measure be-tween concepts can be defined based on the probabilitythat documents about one concept are also about theother concept and vice versa [43]. For this matcher foreach ontology that we want to align we generate a cor-pus of documents. Then for each ontology a documentclassifier is generated using its corpus. This classifierreturns for a given document the concept that is mostclosely related. Documents of one ontology are thenclassified by the document classifier of the other ontol-ogy and vice versa and a similarity measure betweenconcepts in the different ontologies is computed basedon the number of documents related to one concept be-ing classified to the second concept and vice versa.
The user can define which matchers to use in thecomputation session by checking the check boxes infront of the matchers’ names (Figure 4). To guaranteepartial results as soon as possible the similarity valuesfor all currently used matchers are computed for onepair of terms at a time and stored in the similarity val-ues database. When the similarity values for each cur-rently used matcher for a pair of terms are computed,they can be combined and filtered (see below) imme-diately. As ontology alignment is an iterative process,
it may be the case that the similarity values for somepairs and some matchers were computed in a previ-ous round. In this case these values are already in thesimilarity values database and do not need to be re-computed.
4.2.4. CombiningResults from different matchers can becombined.
In our system we allow the choice of the two mostcommon approaches: a weighted-sum approach and amaximum-based approach. In the first approach eachmatcher is given a weight and the final similarity valuebetween a pair of terms is the weighted sum of thesimilarity values divided by the sum of the weights ofthe used matchers. The maximum-based approach re-turns as final similarity value between a pair of terms,the maximum of the similarity values from differentmatchers. The user can choose which combinationstrategy to use by checking radio buttons, and weightscan be added in front of the matchers’ names (Figure4).
4.2.5. FilteringMost systems use a thresholdfilter on the similarity
values to decide which pairs of terms become mappingsuggestions. In this case a pair of terms is a mappingsuggestion if the similarity value is equal to or higherthan a given threshold value. Another approach that weimplemented is the double threshold filtering approach
8 P. Lambrix and R. Kaliyaperumal / A Session-based Ontology Alignment Approach for Aligning Large Ontologies
that we developed in [2]. In this approach two thresh-olds are introduced. Pairs with similarity values equalto or higher than the upper threshold are retained asmapping suggestions. These pairs are also used to par-tition the ontologies in a similar way as in the prepro-cessing step. The pairs with similarity values betweenthe lower and upper thresholds are filtered using thepartitions. Only pairs of which the elements belong tocorresponding elements in the partitions are retained assuggestions. Pairs with similarity values lower than thelower threshold are rejected as mapping suggestions.When a PA is available, a variant of double thresholdfiltering can be used, where the PA is used for parti-tioning the ontologies [20]. The user can choose singleor double threshold filtering and define the thresholds(Figure 4). Further, to obtain higher quality mappings,we always remove mapping suggestions that conflictwith already validated correct mappings [20].
4.2.6. Ending and interruptingThe session can be interrupted using the ’Interrupt
Computation’ button. The user may also specify be-forehand a number of concept pairs to be processedand when this number is reached, the computation ses-sion is interrupted and validation can start. This settingis done using the ’interrupt at’ field (Figure 4). Theoutput of the computation session is a set of mappingsuggestions where the computation is based on the set-tings of the session. Additionally, similarity values arestored in the similarity values database that can be usedin future computation sessions as well as in recommen-dation sessions. In case the user decides to interrupt acomputation session, partial results are available, andthe session may be resumed later on. The ’Finish Com-putation’ button allows a user to finalize the alignmentprocess. (A similar button is available in validation ses-sions.)
4.3. Validation sessions
The validation sessions allow a domain expert tovalidate mapping suggestions. The mapping sugges-tions can come from a computation session (completeor partial results) or be the remaining part of the map-ping suggestions of a previous validation session. Forthe validation we extended the user interface of a sys-tem previously developed by our group, SAMBO [24],which took into account lessons learned from experi-ments [16,17] with ontology engineering systems’ userinterfaces. As stated in [8] our user interface evalua-tions are one of the few existing evaluations and our
system is one of the few systems based on such evalu-ation. Through the interface, the system presents map-ping suggestions (Figure 7) with available informationabout the terms in the mapping suggestions. When aterm appears in multiple mapping suggestions, thesewill be shown at the same time. The user can accepta mapping suggestion as an≡, ⊑ or ⊒ mapping, orreject the mapping suggestion by clicking the appro-priate buttons. Further, the user can give a preferredname to equivalent terms as well as annotate the deci-sions. The user can also review the previous decisions(’History’) as well as receive a summary of the map-ping suggestions still to validate (’Remaining Sugges-tions’). After validation a reasoner is used to detectconflicts in the decisions and the user is notified if anysuch occur.
The mapping decisions are stored in the mappingdecisions database. The accepted mapping suggestionsconstitute a PA and are partial results for the final out-put of the ontology alignment system. The mappingdecisions (both accepted and rejected) can also be usedin future computation and recommendation sessions.Validation sessions can be stopped at any time and re-sumed later on (or if so desired - the user may also starta new computation session).
4.4. Recommendation sessions
We implemented several recommendation strate-gies. The first approach (an extension of our work in[44]) requires the user or an oracle to validate all pairsin small segments of the ontologies. To generate thesesegments we first use a string-based approach to de-tect concepts in the different ontologies with similarnames. In the implementation we used exact matching.The sub-graphs of the two ontologies with the matchedconcepts as roots are then candidate segments. Amongthe candidate segments a number of elements (15) ofsmall enough size (maximally 60 concepts) are re-tained as segments. As a domain expert or oracle hasvalidated all pairs in the segments, full knowledge isavailable for these small parts of the ontologies. Therecommendation algorithm then proposes a particu-lar setting for which matchers to use, which combina-tion strategy and which thresholds, based on the per-formance of the strategies on the validated segments.The advantage of the approach is that it is based onfull knowledge of the mappings of parts of the ontolo-gies. An objection may be that good performance onparts of the ontologies may not lead to good perfor-mance on the whole ontologies. The disadvantage of
P. Lambrix and R. Kaliyaperumal / A Session-based Ontology Alignment Approach for Aligning Large Ontologies 9
Fig. 7. Screenshot: mapping suggestion.
the approach is that a domain expert or an oracle needsto provide full knowledge about the mappings of thesegments. The second and third approach can be usedwhen the results of a validation are available. In thesecond approach the recommendation algorithm pro-poses a particular setting based on the performance ofthe alignment strategies on all the already validatedmapping suggestions. In the third approach we use thesegment pairs (as in the first approach) and the resultsof earlier validation to compute a recommendation.The advantages of these approaches are that decisionsfrom different parts of the ontologies can be used, andthat no domain expert or oracle is needed during thecomputation of the recommendation. However, no fullknowledge may be available for any parts of the on-tologies (e.g. for some pairs in the segment pairs, wemay not know whether the mapping is correct or not),and validation decisions need to be available. We notethat in all approaches, when similarity values for con-cepts for certain matchers that are needed for comput-ing the performance, are not yet available, these will becomputed and added to the similarity values database.
To define the performance of the alignment algo-rithms several measures can be used. We define themeasures that are used in our implementation. We as-sume there is a set of pairs of terms for which fullknowledge is available about the correctness of themappings between the terms in the pair. For the firstapproach this set is the set of pairs in the segments.In the other approaches this set is the set of pairs inthe mappings decisions (accepted and rejected). For agiven alignment algorithm, let then A be the number ofpairs that are correct mappings and that are identifiedas mapping suggestions, B the number of pairs that arewrong mappings but were suggested, C the number ofpairs that are correct mappings but that were not sug-
gested, and D the number of pairs that are wrong map-pings and that were not suggested (see Table 1). In A+ D cases the algorithm made a correct decision andin B + C cases the algorithm made a wrong decision.In our system we use then the following measures (seeTable 2). Pc, Rc and Fc are the common measures ofprecision, recall and their harmonic mean f-measure.These focus on correct decisions for correct mappings.Pw, Rw and Fw are counterparts that focus on cor-rect decisions regarding wrong mappings. Sim1 is asimilarity measure that computes the ratio of correctdecisions over the total number of decisions. Sim2 isthe Jaccard-similarity where the case of non-suggestedwrong mappings is not taken into account (assumed tobe a common and non-interesting case).
The results of the recommendation algorithms arestored in the recommendation database. For each of thealignment algorithms (e.g. matchers, combinations,and filters) the recommendation approach and the per-formance measure are stored. A user can use the rec-ommendations when starting or continuing a computa-tion session.
4.5. User interface
In [9] a cognitive support framework for ontologyalignment systems is proposed. The framework wasdeveloped using cognitive support theories, a litera-ture review of ontology alignment tools as well asa small observational case study. Different require-ments for ontology alignment systems were identifiedand divided into four conceptual dimensions: analy-sis and decision making (requirements 1.1-1.4), in-teraction (requirements 2.1-2.5), analysis and gener-ation (requirements 3.1-3.4), and representation (re-
10 P. Lambrix and R. Kaliyaperumal / A Session-based Ontology Alignment Approach for Aligning Large Ontologies
Table 1
Number of correct/wrong mappings that are suggested/notsuggested.
Suggested Not suggested
Correct A C
Wrong B D
Table 2
Performance measures.
Pc = A/(A+B), Rc = A/(A+C), Fc = 2PcRc/(Pc+Rc)
Pw = D/(C+D), Rw = D/(B+D), Fw = 2PwRw/(Pw+Rw)
Sim1 = (A+D)/(A+B+C+D), Sim2 = A/(A+B+C)
quirements 4.1-4.7). In this section we discuss the cog-nitive support of our system using these requirements.
In the analysis and decision making dimension wesupport the following. In addition to the functionalitydescribed earlier, our system has a component for man-ual ontology alignment where the ontologies are rep-resented as indented trees. In this component the usercan select a term from the first ontology and a termfrom the second ontology and manually create a map-ping (1.1). It also supports ontology exploration (1.1).The tool provides means for the user to accept/rejectmapping suggestions (1.2). Further, the user receivesinformation about the definitions of terms (1.3). Someinformation about the context of the ontology termsis available in the mapping suggestions as well as inthe manual alignment component (1.4). In the interac-tion dimension we support exploration (2.1) and search(2.4) of the ontologies via the manual alignment com-ponent. Exploration of potential mappings is supportedthrough the remaining suggestions list (2.2). Further,we support exploration of already verified mappings(2.3) through the history list. The system also supportsadding details on verified mappings through the anno-tation functionality (2.5). In the analysis and genera-tion dimension we support the automatic discovery ofmapping suggestions (3.1). The mapping state can besaved and users are allowed to return to a given state(3.3). Potential conflicts arising from adding mappingsare detected and the user is notified of potential prob-lems (3.4). Regarding the representation dimension weprovide a visual representation of the ontologies usingindented trees (4.1). We also provide some informationregarding the mappings (4.3) via the annotation func-tionality. Through our PA-based algorithms we haveways to compute mappable regions (4.4). We provideprogress feedback through the different tabs, sessionsand the history list (4.6).
There are a number of requirements that are not sup-ported or should be supported in a better way. We donot have a filter strategy for showing, for instance, onlymappings with exact names or only mapped conceptsin the ontologies (2.4). The current system does notdeal with instances and thus does not support the trans-forming of instances from the source ontology to thetarget ontology (3.2). The current system detects po-tential conflicts but does not suggest ways of resolv-ing them (3.4). We have worked on an integrated sys-tem for ontology alignment and debugging [13] where(3.4) is the main focus of the work. Although the sys-tem provides some information regarding the mappingsuggestions (4.2) and mappings (4.3), more informa-tion available in the different databases could be pre-sented as well as in a better way. For instance, we donot show explanations on why mapping suggestionswere suggested (4.7). Although we have algorithmsfor computing mappable regions, we do not have a vi-sual presentation of these (4.4). In general, the visual-ization of the ontologies, mappings and mapping sug-gestions is subject for future work and different tech-niques need to be investigated. For instance, indentedtrees are more organized and familiar to novice users,but a graph visualization may be more controllable andintuitive [10]. Further, we do not identify specific start-ing points (4.5).
5. Experiments
In this section we discuss two kinds of experi-ments. All experiments show the advantages of using asession-based system regarding performance of com-putation of similarity values, filtering or recommenda-tion. Further, the experiments in Sections 5.3-5.4 ad-ditionally show how a session-based system can beused for evaluating PA-based and recommendation al-
P. Lambrix and R. Kaliyaperumal / A Session-based Ontology Alignment Approach for Aligning Large Ontologies 11
gorithms. These kinds of evaluations could not or noteasily be performed before.
5.1. Experiments set-up
We use the OAEI 2011 Anatomy track for our ex-periments which contains the ontologies Adult MouseAnatomy (AMA) and the anatomy part of the NCIThesaurus (NCI-A). (Removing empty nodes in thefiles) AMA contains 2737 concepts and NCI-A con-tains 3298 concepts. This gives 9,026,626 pairs of con-cepts. Further, a reference alignment containing 1516equivalence mappings is available and thus we focuson equivalence mappings in our experiments.
We used the following alignment strategies. Weused matchersn-gram, TermBasic, TermWN, UMLSMandNaiveBayes6 as introduced in Section 4.2. As com-bination strategies we used weighted sum with possi-ble weights 1 and 2 as well as the maximum-based ap-proach. Further, we used the single and double thresh-old strategies with threshold values 0.3, 0.4, 0.5, 0.6,0.7 and 0.8. In total this gives us 4872 alignment strate-gies. For each of these strategies we computed Pc, Rc,Fc, Pw, Rw, Fw, Sim1 and Sim2 based on the OAEIreference alignment. For instance, Table 3 shows thetop 10 strategies with respect to Sim2. All these 10strategies use a weighted-sum combination, doublethreshold filtering and includeUMLSMand at least onestring matching-based matcher. These strategies havealso a high Fw of over 0.99. The top 10 strategies withrespect to Rc all includeUMLSMand at least one ofn-gramor TermWN. All these strategies use a maximum-based combination approach, single threshold filteringand, as expected, a low threshold (0.3). The best strate-gies find 1497 correct mapping suggestions. The high-est Pc for these strategies is, however, less than 0.016.When sorting strategies based on Pc, 528 strategies hadmaximum Pc value of 1. All of these strategies includeNaiveBayes. Six of the strategies are single matcherstrategies (NaiveBayeswith thresholds 0.6, 0.7, 0.8,0.6;07, 0.6;0.8 and 0.7;0.8). No strategy has threshold0.3. Among those strategies the maximum amount ofcorrect mapping suggestions is 259. All 528 strategieshave Rw = 1 and Pw > 0.99. They have high Sim1
6ForNaiveBayeswe generated a corpus of PubMed [37] abstracts.We used a maximum of 100 abstracts per concept. For AMA the to-tal number of documents was 30,854. There were 2413 concepts forwhich no abstract was found. For NCI-A the total number of docu-ments was 40,081. There were 2886 concepts for which no abstractwas found.
values and low Sim2 values. With respect to the othermeasures, i.e. Rw, Pw, Fw and Sim1, the strategies donot show much variation. Therefore, in the remainderof this paper, we mainly discuss results with respectto Fc and Sim2. Fc is a standard measure; Sim2 has ahigh correlation to Fc, but has a higher degree of dif-ferentiation in our experiments.
For the experiments in Sections 5.3 and 5.4 wechose three alignment strategies (Table 4) as a basis fordiscussion. Strategy AS1 uses a weighted sum combi-nation ofTermBasicwith weight 1 andUMLSM withweight 1, and double threshold filtering with thresh-olds 0.4;0.7 (columns 2-4 in Table 4). AS1 gener-ates 1324 mapping suggestions (column 5). AS1 isthe strategy with best Fc (0.86) and Sim2 (0.75) val-ues. AS2 is an average strategy regarding Fc (0.65)and Sim2 (0.48). It uses a weighted sum combinationof TermWNwith weight 2,n-gramwith weight 1 andNaiveBayeswith weight 1, and single threshold fil-tering with threshold 0.5. It generates 1824 mappingsuggestions. AS3 performs poorly for Fc (0.48) andSim2 (0.32), but has a high Rc value (0.89). It uses aweighted sum combination ofn-gramwith weight 1,TermBasicwith weight 1, andUMLSMwith weight 2,and single threshold filtering with threshold 0.3. It gen-erates 4061 mapping suggestions.
5.2. Computation of Similarity Values
In the first experiment we investigate the influenceof using sessions and the similarity values database onthe efficiency of the ontology alignment system. Foreach of the matchers we computed the similarity val-ues for all pairs of concepts. When a similarity value iscomputed it is stored in the similarity values database.Previous approaches could not take advantage of pre-viously stored values. However, computation sessionsin a session-based approach can take advantage of thefact that previous computation and recommendationsessions already stored similarity values. In Table 5 weshow for two of the matchers the computation timesfor when previous values were stored and for when noprevious values were stored. We do this for the com-putation of 10%, 20% (of which 10% stored), 50% (ofwhich 20% stored), 75% (of which 50% stored) and100% (of which 75% stored) of the 9,026,626 pairs.For instance, forn-gram the computation and storageof 902,662 similarity values took 2.59 minutes. Thecomputation and storage of 1,805,324 similarity val-ues from scratch took 5.08 minutes. However, assum-ing 902,662 similarity values are already stored and
12 P. Lambrix and R. Kaliyaperumal / A Session-based Ontology Alignment Approach for Aligning Large Ontologies
checking the database, it will take 3.98 minutes. Usingthe database is advantageous for string matchers, andeven more advantageous for more complex matchersfor which the speed-up may be up to 25%. The session-based approach leads therefore to reduced computationtimes and reduced waiting times for the domain expert.
5.3. Using the Validation Decisions from PreviousSessions for Filtering
There are few approaches that can take into accountalready given mappings. Further, it is not commonthat such a set of pre-existing mappings exists. In asession-based approach, however, every validation ses-sion generates such sets, which can be used to improvethe quality of the mapping suggestions and reduce un-necessary user interaction. Further, the knowledge ofthe domain expert is taken into account at an earlystage. In this experiment we investigate the influenceof sessions and validation decisions for filtering.
5.3.1. Filtering using validated correct mappingsTable 6 shows for the strategies AS1, AS2 and
AS3 the reduction of the number of mapping sugges-tions by using the filter strategy that removes map-ping suggestions that are in conflict with already val-idated correct mappings. It shows the number of re-
moved mapping suggestions after 500, 1000 and 1300processed mapping suggestions. The results show thatAS1 does not produce many mapping suggestions thatwould conflict. The results also suggest that the re-moval should be done as soon as possible. For in-stance, for AS3 when we would process 1000 sug-gestions without removal, the 156 that would be re-moved after 500 processed suggestions may actuallyhave been - unnecessarily - validated by the domainexpert. Therefore, in our system we perform the re-moval after every validation of a correct equivalencemapping and thereby reduce unnecessary user interac-tion. We also remind that the strategies AS1, AS2 andAS3 produce 1365, 1824 and 4061 mapping sugges-tions, respectively. Therefore, having processed 1000mapping suggestions means that 73%, 40% and 25%of the suggestions have been processed for AS1, AS2and AS3, respectively.
5.3.2. Double threshold filtering using validatedcorrect mappings
In our next experiment, once a session is locked, weuse double threshold filtering with thresholds 0.3 (low-est considered threshold) and 0.6 on the remaining un-validated mapping suggestions of that session. Table 7shows for the strategies AS1, AS2 and AS3 the totalnumber of mapping suggestions (columns 2-4) and the
P. Lambrix and R. Kaliyaperumal / A Session-based Ontology Alignment Approach for Aligning Large Ontologies 13
Table 5
Matcher computation time (in mins).
n-gram NaiveBayes
number of pairs without previous with previous without previous with previous
number of correct suggestions (columns 5-7) that areremoved by this operation. There are two values sepa-rated by ’/’. As double threshold filtering heavily relieson the structure of the ontologies and many is-a rela-tions are actually missing in AMA and NCI-A [22], weexperimented with the original ontologies (first value)and the repaired7 ontologies (second value). The re-sults show that this filtering has a positive effect onFc. Further, in most cases more mapping suggestions,but also more correct suggestions are removed in theoriginal ontologies than in the repaired ontologies, andthe quality in terms of Fc is higher for the repaired on-tologies. We also note that the worse the strategy thehigher the effect.
5.4. Recommendation Strategies with and withoutSessions
The experiments in this section show how recom-mendation strategies can be used within the session-based framework. Further, we evaluate different rec-ommendation strategies. For these experiments weused Sim2 as recommendation measure. For some of
7We repaired (or enriched or completed) the ontologies by addingthe missing is-a relations that were detected by logical reasoning onthe ontologies and the reference alignment. If A and B belong to oneontology, A’ and B’ belong to the other ontology, A⊑ B is derivablein the first ontology, A≡ A’, and B ≡ B’, then we should havethat A’ ⊑ B’ is derivable in the second ontology. If this is not thecase then A’⊑ B’ is added to the second ontology. More advancedtechniques could be used for repairing, e.g. [21,18,46].
the experiments we also needed to generate segmentpairs. We used the method as described in Section 4.4.The system generated 94 segment pair candidates ofwhich 15 were randomly chosen as segment pairs. Themaximum number of concepts in a segment is 12 andthe minimum number is 3. The total number of con-cept pairs for all 15 segment pairs together is 424. Ac-cording to the reference alignment of the OAEI, 46 ofthose are correct mappings. The maximum number ofcorrect mappings within a segment pair is 7 and theminimum is 1.
5.4.1. Session-based recommendation usingvalidation decisions only
In this experiment we use the recommendation al-gorithm that computes a performance measure for thealignment strategies based on how the strategies per-form on the already validated mapping suggestions.Tables 8, 9 and 10 show the recommended strategiestogether with their Fc value on the current validationdecisions and their actual Fc value, after having pro-cessed 500/5038, 1000, ..., 4000 suggestions for AS1,AS2 and AS3, respectively. For AS1, AS1 itself doesnot appear among the top 10 recommendations for allthe sessions. The strategies that received the best scorefor 500, 1000 and 1300 processed suggestions haveactual Fc values of 0.18, 0.85 and 0.23, respectively.The results are explained by the consistent group in
8503, because the validation decision for suggestion 500 removesother suggestions.
14 P. Lambrix and R. Kaliyaperumal / A Session-based Ontology Alignment Approach for Aligning Large Ontologies
Table 7
Double threshold filter using validated correct mappings.
the double threshold filtering. For AS2, AS1 does notappear among the top 10 recommendations for all thesessions. The reason for this behavior is the differencebetween the number of correct mapping suggestionsproposed by AS1 and AS2. That is, some of the cor-rect mapping suggestions that are proposed by AS2will not be proposed by AS1. In this experiment, com-pared to AS1, the recommended strategies propose 1,15, 20 and 46 more correct mapping suggestions for500, 1000, 1500 and 1800 processed suggestions, re-spectively. We note that the recommended strategy al-ways has an actual Fc ≥ 0.76 and the strategy which isrecommended after 1800 processed suggestions uses amaximum-based combination approach. For AS3, thestrategy that receives the best score after 1000, 2000and 2500 processed suggestions is also the best strat-egy (AS1) in reality. Otherwise, AS1 is within the top10 recommendations. In these cases AS1 is not recom-mended because it suggests 2, 1, 13, 6 and 48 morewrong mapping suggestions for 503, 1500, 3000, 3500and 4000 processed suggestions, respectively, whichare not suggested by the recommended strategies. Thereason for the better performance of the recommendedstrategy is due to the generated consistent group inthe double threshold filtering. We note that the recom-mended strategy always has an actual Fc ≥ 0.85 (withbest 0.861 for AS1).
In general, when using an ontology alignment sys-tem with session-based recommendation, a user startswith one alignment strategy and can change strategybased on validations during the alignment process.Therefore, we also performed an experiment where theuser starts with AS1, AS2 or AS3, performs sessions inwhich a maximum of 500 suggestions are processed,and where each new session uses the alignment strat-egy that is recommended by the recommendation algo-rithm based on the validation decisions of all previoussessions. The new computation session will only com-pute ’new’ mapping suggestions, i.e. mapping sugges-tions that were not validated before.
Tables 11, 12 and 13 show results for the recommen-dation algorithm that uses validation decisions only inwhich the first sessions were started with the strategiesAS1, AS2, and AS3, respectively. The rows in the ta-bles indicate the recommended strategy after each ses-sion in the experiments. In the case of AS3 the recom-mended strategy always has an actual Fc >0.84 whichis close to the best strategy Fc (0.86). Similar behavioris observed for AS2, the recommended strategy alwayshas an actual Fc >0.83. In the AS1 case, the recom-mendation becomes better after session 4. In session 6,both AS1 and AS3 recommended strategies with simi-lar Fc (0.84). Compared to the recommendation resultsof AS1 shown in Table 8 the quality of recommenda-tion for the AS1 case improves as the number of ses-sions increases. The reason for the better performanceis due to the differences in the oracle. In the previousexperiment, all the recommendations are done with themapping suggestions from AS1 which are mostly cor-rect. On the other hand, the validation decisions used inthis experiment contain more information about wrongsuggestions since the strategy is different for every ses-sion.
As the performance for the AS1 case is not good,particularly in the early sessions, and an importantcause for this is the lack of negative examples, i.e.wrong mappings suggestions, we investigated whetherthe performance could be improved by automaticallygenerating negative examples when these are not avail-able or only few are available. We implemented an ap-proach that generates wrong mappings based on map-pings validated to be correct. The approach swaps con-cepts in correct mappings. For instance, if A≡ A’ andB ≡ B’ are correct mapping suggestions, then we gen-erate the wrong mapping suggestions A≡ B’ and B≡ A’. Then we selected the wrong mapping sugges-tions whose similarity based on edit distance is be-tween 0.45 and 0.65. In this experiment we gener-ated 300 wrong mapping suggestions. Table 14 showsthe results for the recommendation when starting thealignment process with AS1. We note that the recom-
P. Lambrix and R. Kaliyaperumal / A Session-based Ontology Alignment Approach for Aligning Large Ontologies 15
Table 8
Session-based recommendation using validation decisions only - AS1.
Using recommended strategy after each session - session-based recommendation using validation decisions only - AS1 - automatically generatedwrong mapping suggestions.
session matchers weights threshold rec actual
Fc Fc
1 UMLSM 1 0.8 0.980 0.844
2 UMLSM;TermWN 1;1 0.5 0.950 0.850
3 n-gram;UMLSM 1;1 0.5 0.943 0.851
4 n-gram;UMLSM 1;1 0.5 0.938 0.851
on how the strategies perform on the already validatedparts of the segment pairs. Tables 15, 16 and 17 showthe results for AS1, AS2 and AS3, respectively. ForAS1, the recommended strategy after 500, 1000 and1300 processed suggestions has actual Fc = 0.07. Thereason for this result is that AS1 has very high preci-sion so the oracle (validated suggestions) has very littleinformation about wrong mapping suggestions. How-ever, it has much information about correct mappingsuggestions. The strategy that is recommended in thethree sessions is one that has very high recall but thatalso suggests many wrong mappings which the algo-rithm cannot detect. Similar behavior is observed forAS2, but the oracle used in this case has better infor-mation about wrong mapping suggestions than the one
which is used in the AS1 case. The recommended strat-egy for all the sessions has actual Fc = 0.624.
For AS3, the strategies that are recommended after503, 1000, 1500, 2000 and 2500 processed suggestionshave actual Fc = 0.53, after 3000 actual Fc = 0.76, andafter 3500 and 4000 actual Fc = 0.82. This result showsthat as the number of processed suggestions increases,the recommended strategy becomes better. This is be-cause the quality of the oracle increases.
Also for this recommendation stategy we performedan experiment where the user starts with AS1, AS2 orAS3, performs sessions in which a maximum of 500suggestions are processed, and where each new ses-sion uses the alignment strategy that is recommendedby the recommendation algorithm based on the valida-
P. Lambrix and R. Kaliyaperumal / A Session-based Ontology Alignment Approach for Aligning Large Ontologies 17
Table 15
Session-based recommendation using segment pairs and validation decisions - AS1.
processed matchers weights threshold rec actual
suggestions Fc Fc
500 NaiveBayes;n-gram 1;1 0.3;0.8 1 0.070
1000 NaiveBayes;n-gram 1;1 0.3;0.8 1 0.070
1300 NaiveBayes;n-gram 1;1 0.3;0.8 1 0.070
Table 16
Session-based recommendation using segment pairs and validation decisions - AS2.
processed matchers weights threshold rec actual
suggestions Fc Fc
500 NaiveBayes;n-gram 1;1 0.3;0.5 1 0.624
1000 NaiveBayes;n-gram 1;1 0.3;0.5 1 0.624
1300 NaiveBayes;n-gram 1;1 0.3;0.5 1 0.624
1800 NaiveBayes;n-gram 1;1 0.3;0.5 1 0.624
Table 17
Session-based recommendation using segment pairs and validation decisions - AS3.
tion decisions of all previous sessions. Similarly as be-fore, the new computation session will only compute’new’ mapping suggestions. Tables 18, 19 and 20 showthe results for the recommendation algorithm that usessegment pairs and validation decisions. For AS1 andAS3 the algorithm proposed the same strategy for ses-sions 2 to 4. After session 7, both cases recommendedstrategies with similar Fc (0.82) values. Even thoughthe best strategy overall (AS1) is not recommended,for the selected segment pairs these are the best strate-gies and thus the best that the algorithm can propose.We also note that the quality of recommendation forthe AS1 case is improved compared with the recom-mendation results of AS1 in Table 15. For AS2 the al-gorithm proposed the same strategy for sessions 1 to 3(Fc = 0.62) and sessions 4 to 8 (Fc = 0.76), the recom-
mendation did not change after session 4. The reasonfor this behavior is lack of information about correctmappings in the oracle. That is, the segment pairs usedin this experiment have a reference alignment of 46mappings, but the oracle used by the AS2 case has in-formation about only 35 mappings. On the other hand,the oracles used by the AS1 and AS3 cases have infor-mation about 42 and 45 mappings, respectively.
5.4.3. Session-independent recommendation usingsegment pairs and oracle
In this experiment we use the recommendation al-gorithm that uses segment pairs and computes a per-formance measure for the alignment strategies basedon how the strategies perform on the segment pairs.This requires an oracle that has full knowledge aboutthe mappings in the segment pairs and for this we use
18 P. Lambrix and R. Kaliyaperumal / A Session-based Ontology Alignment Approach for Aligning Large Ontologies
Table 18
Using recommended strategy after each session - session-based recommendation using segment pairs and validation decisions- AS1.
the reference alignment as provided by the OAEI. Asthis recommendation strategy is independent from theactual validation decisions, the recommendation doesnot change during the alignment process. It can there-fore be performed in the beginning. Based on the per-formance on the 15 small segments pairs (with a refer-ence alignment of only 46 mappings), the recommen-dation algorithm gives Sim2 = 0.87 and Fc = 0.93 for
AS1, Sim2 = 0.52 and Fc = 0.68 for AS2, and Sim2 =0.47 and Fc = 0.64 for AS3 (see Table 21).
However, there are also 145 strategies that have ahigher Sim2 value than AS1. The top 8 recommendedstrategies all use double threshold filtering and haveSim2 = 0.98 and Fc = 0.99 for the segment pairs, andan actual Fc between 0.8 and 0.84. They suggest 45correct mappings and 0 wrong mappings, whereas AS1
P. Lambrix and R. Kaliyaperumal / A Session-based Ontology Alignment Approach for Aligning Large Ontologies 19
suggests 42 correct mappings and 2 wrong mappings.We also note that that there are 81 strategies whichhave Sim2 >0.9 and Fc >0.95 on the segment pairs.
5.5. Summary of lessions learned
5.5.1. Use of the session-based approach and systemWe showed the usefulness of the system and its
components through experiments with many align-ment strategies on the OAEI 2011 Anatomy track on-tologies.
We showed that using the session-based approachleads to alignment quality improvements. As the ap-proach allows for the partial computation and the par-tial validation of mappings suggestions, validation de-cisions can be taken into account during the follow-ing sessions. The validation decisions represent do-main expert knowledge and can be used earlier in thealignment process than in former frameworks. Duringcomputation sessions a PA can be used for reducingthe search space, which according to the experimentsin [20] often leads to an improvement of Fc. As shownin this paper, the use of validation decisions from pre-vious sessions for different kinds of filtering also hasa positive effect on Fc. These approaches also reduceunnecessary user interaction.
The session-based approach also supports the rec-ommendation of alignment strategies. As, in general,we do not know which alignment strategies performwell for a particular pair of ontologies, according toour experiments using the recommendations after eachsession usually leads to better alignments.
Further, during computation and recommendationsessions, computed similarity values are stored in thesimilarity values database. Using this database in fur-ther computation sessions reduces computation timesand waiting times for the domain expert.
5.5.2. Lessons about alignment strategiesWe also learned some lessions about the actual
alignment algorithms. For instance, filtering out sug-gestions that are in conflict with validation decisionsafter the locking of sessions is useful and the worsethe initial strategy, the more useful this is. In our sys-tem we, therefore, perform this kind of removal afterevery validation of a correct equivalence mapping andthereby reduce unnecessary user interaction. Also fil-tering after the locking of a session using the doublethreshold filtering method is useful, and the more com-plete the is-a structure in the ontologies is, the betterthe results.
The recommendation is important, especially whenthe initial strategy is not good. It is also clear that theapproaches using validation decisions (with and with-out segment pairs), become better the more sugges-tions are validated. Further, when using the recom-mended strategy after each session improves the fi-nal result. We also found, that, when too few wrongmapping suggestions are available, we can improvethe performance by automatically generating wrongmapping suggestions. For the approaches using seg-ment pairs, the experiments show that the choice ofthe segment pairs influences the recommendation re-sults (which is different from the conclusions of ex-periments in [44]). Therefore, strategies for choosingsegment pairs need to be investigated. In our experi-ments among the strategies with validation decisions,the strategy with validation decisions only performedbest, but the strategy with validation decisions and seg-ment pairs may be improved with better segment se-lection strategies.
6. Related Work
To our knowledge there is no other framework orsystem that deals with all the challenges for align-ing large ontologies that our approach deals with. Ear-lier frameworks (e.g. [5,24]) and the systems builtaccording to these frameworks have focused on thegeneration of mappings suggestions, similar to non-interruptable computation sessions in which validateddata usually is not taken into account. Some systemsalso allow to validate data, similar to non-interruptablevalidation sessions. As there is no similar frameworkor system, we briefly address related work regardingthe different components and used techniques.
The computation of mapping suggestions includespreprocessing, matching, combining and filtering.There are some approaches that reduce the searchspace by segmenting or partitioning the ontologies andusing anchors (concept pairs with high similarity) toconnect mappable segments [12] or segment similar-ity [3]. Some approaches use the locality of anchors toreduce the search space [38,45]. In [45] anchors canalso be pairs with low similarity values. The main dif-ference with our approach is that we use validationdecisions to partition the ontologies.
For the matching many algorithms have been pro-posed (e.g. many papers on http://ontologymatching.org/).As mentioned before, they often implement strategiesbased on linguistic matching, structure-based strate-
20 P. Lambrix and R. Kaliyaperumal / A Session-based Ontology Alignment Approach for Aligning Large Ontologies
Table 21
Session-independent recommendation using segment pairs and oracle.
strategy Fc Sim2
AS1 0.93 0.87
AS2 0.68 0.52
AS3 0.64 0.47
gies, constraint-based approaches, instance-based strate-gies, strategies that use auxiliary information or acombination of these. The results from OAEI andevaluation studies such as in [24,1,30] provide someknowledge on the performance of the matchers. In oursystem we used linguistic matching, instance-basedstrategies, and strategies that use auxiliary informa-tion.
The most commonly used combination strategiesare the weighted-sum and the maximum-based ap-proaches. Our system supports these. There are somemore advanced combination strategies such as in theschema metamatching framework of [4] and the agent-based method in [41].
Regarding filtering, most systems use single thresh-old filtering, while we additionally support doublethreshold filtering. In contrast to most systems, oursystem can also take into account PAs or validation de-cisions.
There are some systems that allow validation ofmappings such as SAMBO [24], AlViz [25], COGZ[9] for PROMPT, COMA++ [3] and AML [34]. Noneof these systems allow, however, interruptable ses-sions. LogMap2 [14] allows user interaction althoughit does not have graphical user interfaces yet. Interrupt-ing user interaction in this case means using heuristicsto deal with remaining mapping suggestions. There areapproaches that try to minimize user interaction. Forinstance, in [11] minimal mappings are computed forlight-weight ontologies and these are presented for val-idation. Further, recently, in [32] evaluation measuresfor user interaction were proposed for which the eval-uation can be fully automated.
There are very few recommendation approaches. In[28] it is argued that finding appropriate alignmentstrategies should be based on knowledge about thestrategies and their previous use. As a first step a num-ber of factors (related to input, output, approach, us-age, cost and documentation) were identified that arerelevant when selecting an alignment strategy. The rel-evant data is collected by questionnaires. The Ana-lytic Hierarchy Process is used to detect suitable align-ment approaches. The results from OAEI and evalu-
ation studies such as in [24,30] could provide usefulinput data for this approach. In [5], APFEL, a ma-chine learning approach to optimize alignment strate-gies is proposed. In APFEL a set of feature parame-ters are declared for the source ontologies, the simi-larity assessment, and the different matchers, combi-nation and filter algorithms. To generate training data,an existing parametrization is used and mapping sug-gestions are generated. These suggestions need to bevalidated by the user. A machine learning approach isthen used to learn an optimal parametrization. Thereare some approaches for tuning the parameters in theontology alignment systems. The RiMOM [26] andUFOme [36] systems use textual and structural char-acteristics of the ontologies for the selection of match-ers, combinations and filters. The system in [35] usessuch characteristics to configure itself in an adaptiveway. Falcon-OA [12] includes an approach to tune thethresholds for the matchers.
7. Conclusion
In this paper we presented to our knowledge the firstframework and implemented system that allow a userto interrupt and resume the different stages of the on-tology alignment task. Our work addressed several ofthe challenges in ontology alignment presented in [40].Further, we showed the usefulness of the system andits components through experiments with many align-ment strategies on the OAEI 2011 Anatomy track on-tologies. We also showed that the session-based frame-work enabled experimentation and evaluation of newalignment approaches (both in computation and rec-ommendation) that are based on validation decisions.These evaluations were not possible or cumbersomebefore.
In future work we will continue to develop andevaluate computation strategies and recommendationstrategies. Especially interesting are strategies thatreuse validation results to e.g. reduce the search spaceor guide the computation. Further, we will investi-gate new strategies for recommendations using valida-
P. Lambrix and R. Kaliyaperumal / A Session-based Ontology Alignment Approach for Aligning Large Ontologies 21
tion decisions, including segment selection strategies.A further interesting track is to integrate debuggingstrategies into the alignment process as in [13]. In asession-based approach debugging can be performedearly and thereby increase the quality of the alignment.Finally, we are interested in investigating different vi-sualization techniques for the representation of ontolo-gies, mapping suggestions and mappings.
Acknowledgments
We acknowledge the financial support of the Swedishe-Science Research Centre (SeRC). We thank QiangLiu, Muzammil Zareen Khan and Shahab Qadeer fortheir implementation work on earlier versions of thesystem.
References
[1] M Cheatham and P Hitzler. String similarity metrics for on-tology alignment. In12th International Semantic Web Confer-ence - Part II, LNCS 8219, pages 294–309, 2013.
[2] B Chen, P Lambrix, and H Tan. Structure-based filtering forontology alignment. InIEEE WETICE Workshop on SemanticTechnologies in Collaborative Applications, pages 364–369,2006.
[3] H-H Do and E Rahm. Matching large schemas: approachesand evaluation.Information Systems, 32:857–885, 2007.
[4] C Domshlak, A Gal, and H Roitman. Rank aggregation forautomatic schema matching.IEEE Transactions on Knowledgeand Data Engineering, 19(4):538–553, 2007.
[5] M Ehrig, S Staab, and Y Sure. Bootstrapping ontology align-ment methods with APFEL. In4th International Semantic WebConference, LNCS 3729, pages 186–200, 2005.
[6] J Euzenat, C Meilicke, H Stuckenschmidt, P Shvaiko, andC Trojahn. Ontology alignment evaluation initiative: Six yearsof experience.Journal on Data Semantics, XV:158–192, 2011.
[7] J Euzenat and P Shvaiko.Ontology Matching. Springer, 2007.[8] S Falconer and N Noy. Interactive techniques to support on-
tology matching. In Bellahsene, Bonifati, and Rahm, edi-tors,Schema Matching and Mapping, chapter 2, pages 29–51.Springer, 2011.
[9] S Falconer and M-A Storey. A cognitive support frameworkfor ontology mapping. In6th International Semantic Web Con-ference and 2nd Asian Semantic Web Conference, LNCS 4825,pages 114–127, 2007.
[10] B Fu, N Noy, and M-A Storey. Indented tree or graph? A us-ability study of ontology visualization techniques in the con-text of class mapping evaluation. In12th International Seman-tic Web Conference - Part I, LNCS 8218, pages 117–134, 2013.
[11] F Giunchiglia, V Maltese, and A Autayeu. Computing min-imal mappings. In4th International Workshop on OntologyMatching, CEUR-WS 551, pages 37–48, 2009.
[12] W Hu and Y Qu. Falcon-AO: a practical ontology matchingsystem.Journal of Web semantics, 6(3):237–239, 2008.
[13] V Ivanova and P Lambrix. A unified approach for aligningtaxonomies and debugging taxonomies and their alignments.In 10th Extended Semantic Web Conference, LNCS 7882, pages1–15, 2013.
[14] E Jimenez-Ruiz, B Cuenca-Grau, Y Zhou, and I Horrocks.Large-scale interactive ontology matching: Algorithms and im-plementation. In20th European Conference on Artificial Intel-ligence, pages 444–449, 2012.
[15] Y Kalfoglou and M Schorlemmer. Ontology mapping: the stateof the art. The Knowledge Engineering Review, 18(1):1–31,2003.
[16] P Lambrix and A Edberg. Evaluation of ontology mergingtools in bioinformatics. InPacific Symposium on Biocomput-ing, pages 589–600, 2003.
[17] P Lambrix, M Habbouche, and M Perez. Evaluation of on-tology development tools for bioinformatics.Bioinformatics,19(12):1564–1571, 2003.
[18] P Lambrix and V Ivanova. A unified approach for debuggingis-a structure and mappings in networked taxonomies.Journalof Biomedical Semantics, 4:10, 2013.
[19] P Lambrix and R Kaliyaperumal. A session-based approachfor aligning large ontologies. In10th Extended Semantic WebConference, LNCS 7882, pages 46–60, 2013.
[20] P Lambrix and Q Liu. Using partial reference alignments toalign ontologies. In6th European Semantic Web Conference,LNCS 5554, pages 188–202, 2009.
[21] P Lambrix and Q Liu. Debugging the missing is-a structurewithin taxonomies networked by partial reference alignments.Data & Knowledge Engineering, 86:179–205, 2013.
[22] P Lambrix, Q Liu, and H Tan. Repairing the missing is-a struc-ture of ontologies. In4th Asian Semantic Web Conference,LNCS 5926, pages 76–90, 2009.
[23] P Lambrix, L Strömbäck, and H Tan. Information Integrationin Bioinformatics with Ontologies and Standards. In Bry andMaluszynski, editors,Semantic Techniques for the Web: TheREWERSE perspective, chapter 8, pages 343–376. Springer,2009.
[24] P Lambrix and H Tan. SAMBO - a system for aligning andmerging biomedical ontologies.Journal of Web Semantics,4(3):196–206, 2006.
[25] M Lanzenberger and J Sampson. AlViz - a tool for visual on-tology alignment. In10th International Conference on Infor-mation Visualization, page 430âAS440, 2006.
[26] J Li, J Tang, Y Li, and Q Luo. RiMOM: a dynamic multi-strategy ontology alignment framework.IEEE Transactions onKnowledge and Data Engineering, 21(8):1218–1232, 2009.
[27] C Meilicke. Alignment Incoherence in Ontology Matching.PhD thesis, University of Mannheim, 2011.
[28] M Mochol, A Jentzsch, and J Euzenat. Applying an analyticmethod for matching approach selection. In1st InternationalWorkshop on Ontology Matching, CEUR-WS 225, 2006.
[29] D Ngo and Z Bellahsene. Yam++ Âav results for oaei 2013. In8th International Workshop on Ontology Matching, CEUR-WS1111, pages 211–218, 2013.
[30] D Ngo, Z Bellahsene, and K Todorov. Opening the black boxof ontology matching. In10th Extended Semantic Web Confer-ence, LNCS 7882, pages 16–30, 2013.
[31] NF Noy. Semantic integration: A survey of ontology-basedapproaches.Sigmod Record, 33(4):65–70, 2004.
[32] H Paulheim, S Hertling, and D Ritze. Towards evaluating in-teractive ontology matching tools. In10th Extended Semantic
22 P. Lambrix and R. Kaliyaperumal / A Session-based Ontology Alignment Approach for Aligning Large Ontologies
Web Conference, LNCS 7882, pages 31–45, 2013.[33] C Pesquita, D Faria, E Santos, and FM Couto. To repair or
not to repair: reconciling correctness and coherence in ontol-ogy reference alignments. In8th International Workshop onOntology Matching, CEUR-WS 1111, pages 13–24, 2013.
[34] C Pesquita, D Faria, E Santos, J-M Neefs, and FM Couto. To-wards visualizing the alignment of large biomedical ontolo-gies. In10th International Conference on Data Integration inthe Life Sciences, LNBI 8574, pages 104–111, 2014.
[35] E Peuckert, J Eberius, and E Rahm. A self-configuring schemamatching system. In28th International Conference on DataEngineering, pages 306–317, 2012.
[36] G Pirro and D Talia. UFOme: An ontology mapping systemwith strategy prediction capabilitues.Data & Knowledge En-gineering, 69:444–471, 2010.
[37] PubMed. http://www.ncbi.nlm.nih.gov/pubmed/.[38] M Hanif Seddiqui and M Aono. An efficient and scalable algo-
rithm for segmented alignment of ontologies of arbitrary size.Journal of Web Semantics, 7(4):344–356, 2009.
[39] P Shvaiko and J Euzenat. A survey of schema-based matchingapproaches.Journal on Data Semantics, IV:146–171, 2005.
[40] P Shvaiko and J Euzenat. Ontology matching: state of the artand future challenges.IEEE Transactions on Knowledge andData Engineering, 25(1):158–176, 2013.
[41] V Spiliopoulos and G Vouros. Synthesizing ontology align-ment methods using the max-sum algorithm.IEEE Trans-actions on Knowledge and Data Engineering, 24(5):940–951,2012.
[42] Unified Medical Language System.http://www.nlm.nih.gov/research/umls/about_umls.html.
[43] H Tan, V Jakoniene, P Lambrix, J Aberg, and N Shahmehri.Alignment of biomedical ontologies using life science litera-ture. In International Workshop on Knowledge Discovery inLife Science Literature, LNBI 3886, pages 1–17, 2006.
[44] H Tan and P Lambrix. A method for recommending ontologyalignment strategies. In6th International Semantic Web Con-ference and 2nd Asian Semantic Web Conference, LNCS 4825,pages 494–507, 2007.
[45] P Wang, Y Zhou, and B Xu. Matching large ontologies basedon reduction anchors. In22nd International Joint Conferenceon Artificial Intelligence, pages 2243–2348, 2011.
[46] F Wei-Kleiner, Z Dragisic, and P Lambrix. Abduction frame-work for repairing incomplete EL ontologies: Complexity re-sults and algorithms. In8th AAAI Conference on Artificial In-telligence, pages 1120–1127, 2014.
