Summarization from medical documents: a surveycgi.di.uoa.gr/~takis/aim.pdf · Summarization from...

Artificial Intelligence in Medicine (2005) 33, 157—177

http://www.intl.elsevierhealth.com/journals/aiim

Summarization from medical documents: a survey

Stergos Afantenosa,*, Vangelis Karkaletsisa, Panagiotis Stamatopoulosb,1

aSoftware and Knowledge Engineering Laboratory, Institute of Informaticsand Telecommunications, National Centre for Scientific Research (NCSR)‘‘Demokritos’’, 15310 Aghia Paraskevi Attikis, Athens, GreecebDepartment of Informatics, University of Athens, TYPA Buildings,Panepistimiopolis, GR-15771 Athens, Greece

Received 16 December 2002; received in revised form 21 July 2004; accepted 21 July 2004

KEYWORDSSummarization from

medical documents;Single-document

summarization;Multi-document

summarization;Multi-media

summarization;Extractive

summarization;Abstractive

summarization;Cognitive

summarization

* Corresponding author. Tel.: +30 210 6503149.E-mail addresses: [email protected] (S. Afantenos),

[email protected] (V. Karkaletsis),[email protected] (P. Stamatopoulos).

1 Tel.: +30 210 7752222.

0933-3657/$ — see front matter # 2004 Elsevier B.V. All rights resedoi:10.1016/j.artmed.2004.07.017

1. Introduction

New technologies, such as high-speed networksand inexpensive massive storage, along with theremarkable growth of the Internet, have led to an

Summary

Objective: The aim of this paper is to survey the recent work in medical documentssummarization.Background: During the last decade, documents summarization got increasing atten-tion by the AI research community. More recently it also attracted the interest of themedical research community as well, due to the enormous growth of information thatis available to the physicians and researchers in medicine, through the large andgrowing number of published journals, conference proceedings, medical sites andportals on the World Wide Web, electronic medical records, etc.Methodology: This survey gives first a general background on documents summariza-tion, presenting the factors that summarization depends upon, discussing evaluationissues and describing briefly the various types of summarization techniques. It thenexamines the characteristics of the medical domain through the different types ofmedical documents. Finally, it presents and discusses the summarization techniquesused so far in the medical domain, referring to the corresponding systems and theircharacteristics.Discussion and conclusions: The paper discusses thoroughly the promising paths forfuture research in medical documents summarization. It mainly focuses on the issue ofscaling to large collections of documents in various languages and from differentmedia, on personalization issues, on portability to new sub-domains, and on theintegration of summarization technology in practical applications.# 2004 Elsevier B.V. All rights reserved.

rved.

158 S. Afantenos et al.

enormous increase in the amount and availability ofon-line documents. This is also the case for medicalinformation, which is now available from a variety ofsources. However, information is only valuable tothe extent that it is accessible, easily retrieved andconcerns the personal interests of the user. Thegrowing volume of data, the lack of structuredinformation, and the information diversity havemade information and knowledge management areal challenge towards the effort to support themedical society. It has been realized that addedvalue is not gained merely through larger quantitiesof data, but through easier access to the requiredinformation at the right time and in the most sui-table form. Thus, there is a strong need forimproved means of facilitating information access.

The medical domain suffers particularly from theproblem of information overload since it is crucialfor physicians and researchers in medicine and biol-ogy to have quick and efficient access to up-to-dateinformation according to their interests and needs.Considering, for instance, the scientific medicalarticles [1, p. 38] state that: ‘‘. . .there are fivejournals which publish papers in the narrow speci-alty for cardiac anesthesiology, but 35 differentanesthesia journals in general; approximately 100journals in the closely related fields of cardiology(60) and cardiothoracic surgery (40); and over 1000journals in the more general field of internal med-icine.’’ The situation becomes much worse if oneconsiders relevant journals or newsletters in otherlanguages, Web sites with relevant information,medical reports, etc.

Given the number and diversity of medical infor-mation sources, methods must be found that willenable users to quickly assimilate and determine thecontent of a document. Summarization is one suchapproach that can help users to quickly determinethe main points of a document. Radev et al. [2]provide the following definition for a summary: ‘‘Asummary can be loosely defined as a text that isproduced from one or more texts, that conveysimportant information in the original text(s), andthat is no longer than half of the original text(s) andusually significantly less than that. Text here is usedrather loosely and can refer to speech, multimediadocuments, hypertext, etc. The main goal of asummary is to present the main ideas in a documentin less space. If all sentences in a text documentwere of equal importance, producing a summarywould not be very effective, as any reduction in thesize of a document would carry a proportionaldecrease in its informativeness. Luckily, informa-tion content in a document appears in bursts, andone can therefore distinguish between more andless informative segments. Identifying the informa-

tive segments at the expense of the rest is the mainchallenge in summarization.’’

Although initial work on summarization datesback to the late 1950s and 1960s (e.g. [3,4]), fol-lowed by some sparse publications (e.g. [5,6]), mostresearch in the field has been carried out during thelast decade. During these last few years, research-ers examined a great variety of techniques andapplied them in different domains and genres ofdocuments, in order to see which are the ones thatyield the most practical results for each domain andgenre.

This survey presents the potential of summariza-tion technology in the medical domain, based on theexamination of the state of the art, as well as ofexisting medical document types and summariza-tion applications. An important aspect of this surveyis that it is not restricted to a mere examination ofthe various summarization techniques, but it exam-ines the issues that arise in the use of these tech-niques taking into account the characteristics of themedical domain.

The structure of the survey is as follows. Secondsection presents a roadmap of summarization com-prising the factors that have to be taken intoaccount and the main techniques considered sofar in the summarization literature. Third sectionpresents different types of medical documents andthe requirements that they introduce to the sum-marization process. Fourth section examines thetechniques used so far for summarization in medicaldocuments. Finally, fifth section summarizes themost interesting remarks of this survey and presentspromising paths for future research, while last sec-tion concludes the paper.

2. Summarization roadmap

A summarization system in order to achieve its tasktakes into account several factors. These factorsconcern mainly the type of input documents, thepurpose that the final summary should serve, andthe possible ways of presenting a summary. Sum-mary evaluation is also an important issue. Thesefactors are examined in the following sections.Various techniques that have been used so far fordocument summarization are also presented. Thispresentation is necessary for the examination ofexisting approaches to summarization from medicaldocuments.

2.1. Summarization factors

A detailed presentation of the factors that have tobe taken into account for the development of a

Summarization from medical documents: a survey 159

summarization system has been given in [7]. How-ever, as it is noted there, all these factors are hardto define and therefore it is very difficult to capturethem precisely enough to guide summarization invarious applications. The following presentation offactors adopts the main categorization presented in[7]: input, purpose and output. For each of thesecategories, the factors considered as the mostimportant ones are presented.

2.1.1. Input factorsThe main factors in this category are the following.

2.1.1.1. Single-document versus multi-docu-ment. This is the unit input parameter or the spanparameter, as Sparck-Jones [7] and Mani [8] respec-tively call it, which in simple words is the number ofdocuments that the system has to summarize. Insingle-document summarization the system pro-cesses just one document at a time, whereas inmulti-document summarization more than onedocument are processed by the system.

2.1.1.2. Language. Another input factor is thenumber of languages in which the input documentsare written. So, a system can bemonolingual,multi-lingual or cross-lingual. In the first case, the outputlanguage is the same as the input language. In thecase of multilingual summarization systems, theoutput language is the same as the input language,but the system can handle a certain number oflanguages. In the final case of cross-lingual summar-ization, the system can accept a source text in aspecific language and deliver the summary inanother language, not necessarily the same as theinput one.

2.1.1.3. Text versus multimedia summaries. An-other important factor is the medium used to repre-sent the content of the input document(s), as well asthe output summary. Thus, we have text, or multi-media (e.g. images, speech, video apart from tex-tual content) summarization. The most studied caseis, of course, text summarization. However, thereare also summarization systems that deal, for exam-ple, with the summarization of broadcast news [9]and of diagrams [10].

2.1.2. Purpose factorsThese factors concern the possible uses of the sum-mary, the potential readers of the summary, as wellas the domain(s) that must be covered by the sys-tem.

2.1.2.1. Informative versus indicative summarie-s. According to the function that the summary is

supposed to serve when presented to its reader, itcan either be indicative or informative. An indica-tive summary does not claim any role of substitutingthe source document(s). Its purpose is merely toalert its reader in relation to the contents of theoriginal document(s), so that the reader can choosewhich of the original documents to read further. Thepurpose of an informative summary, on the otherhand, is to substitute the original document(s) as faras coverage of information is concerned. Apart fromthe indicative and informative summaries, there arealso critical summaries [7,8], but, as far as we know,no actual summarization system creates criticalsummaries.

2.1.2.2. Generic versus user-oriented summar-ies. This factor concerns the information a systemneeds to locate in order to produce a summary.Generic systems create a summary of a documentor a set of documents taking into account all theinformation found in the documents. On the otherhand, user-oriented systems try to create a sum-mary of the information found in the document(s)which is relevant to a user query. In a sense, we cansay that the query-oriented summarization systemsare user-focused, adapting each time to the verballyexpressed needs of the users, as viewed through thequery they make or through their model (persona-lized summaries).

2.1.2.3. General purpose versus domain-specific.General-purpose systems can be easily ported to adifferent domain (e.g. financial, medical). This canbe done, for instance, by changing the resourcesthat characterize the domain (e.g. keywords, adomain-specific ontology), or by tuning specificparameters which concern the selection of the mostappropriate techniques for the domain. On theother hand, domain-specific systems are able toprocess documents belonging to a specific domain.

2.1.3. Output factorsThese factors are related to the criteria that areused to judge the quality of the resulting summaryas well as with the type of summary in terms ofwhether this is an extract from the original docu-ment(s) or an abstraction.

2.1.3.1. Output quality. The developer of a sum-marization system has to specify certain qualitativeor quantitative criteria, which are related to thespecific summarization task and the evaluationmethod that will be used (see Evaluation). Suchcriteria may be among others the completeness,the accuracy, the coherence of the resulting sum-mary, etc. If accuracy is crucial for a specific task,


Table

1Representative

systemsemployingextractivetech

niques

Input

Purpose

Output

Method

Eva

luation

[3]

Single-docu

ment,

English,text

Generic,

domain-specific

(tech

nical

pap

ers)

Sentence

sStatistics

(Edmundsonianparad

igm),

norevision

[4]

Single-docu

ment,

English,text

Generic,

domain-specific

(scientificarticlesonspecifictopics)

Sentence

sStatistics

(Edmundsonianparad

igm),

use

ofthematic

keyw

ords,

norevision

Intrinsic

[13]

Single-docu

ment,

multilingu

al,text

Generic,

domain-specific(news)

Sentence

sStatistics

(Edmundsonianparad

igm),

norevision

Intrinsic

[14]

Single-docu

ment,

multilingu

al,text

User-oriented,domain-specific

(scientifican

dtech

nical

texts)

Sentence

sStatistics

(Edmundsonianparad

igm),

use

ofthesauri,revision

[15]

Multi-docu

ment,

multilingu

al(English,

Chinese),

text

Generic,

domain-specific(news)

Sentence

sLa

ngu

ageproce

ssing(toidentify

keyw

ords)

Extrinsic

[16]

Single-docu

ment,

English,text

Generic,

generalpurpose

Paragrap

hs

Graph-based,statistics

(cosinesimilarity,

vectorspac

emodel)

Intrinsic

[17]

Multi-docu

ment,

English,text

User-oriented,Generalpurpose

Text

regions(sentence

s,parag

raphs,

sections)

Graph-based,co

hesionrelations,

langu

ageproce

ssing

Intrinsic,

extrinsic

[18,19

]Single-docu

ment,

English,text

Generic,

domain-specific

(scientificarticles)

Sentence

sTree-based,langu

ageproce

ssing

(toidentify

theRST

relationsmarke

rs)

Intrinsic,

extrinsic
then the system developer must tune its system
accordingly, i.e. producing accurate summaries eventhough these do not contain all the relevant results.

2.1.3.2. Extracts versus abstracts. Considering therelation that the summary has to the source docu-ment(s), a summary can either be an extract or anabstract. An extract involves the selection and ver-batim inclusion of ‘‘material’’ from the source docu-ment(s) in the summary; this ‘‘material’’ is usuallysentences, paragraphs or even phrases. Theexcerpted textual units can be included in thesummary verbatim, or they can be processed furtherin order to smooth the text flow. An abstract, on theother hand, involves the identification of the mostsalient concepts prevalent in the source docu-ment(s), the fusion and the appropriate presenta-tion of them, usually through Natural LanguageGeneration.

2.2. Evaluation

Although the summarization community considersevaluation as a critical issue, it still remains unclearwhat the evaluation criteria should be. This ismainly related to the subjective aspect of summar-ization, in terms of whether or not a summary is of‘‘good’’ quality. Existing evaluation techniques canbe split into two categories, intrinsic and extrinsicones (see Section 5 in [11]). An intrinsic methodevaluates the outcome of a summarization systemindependently of the purpose that the summary issupposed to serve. An extrinsic evaluation, on theother hand, evaluates the produced summary interms of a specific task.

An intrinsic method can measure the quality ofthe summary using criteria such as the integrity ofits sentences, the existence of anaphors withouttheir referents (for an extract), the summary read-ability (for an abstract), the fidelity of the summarycompared to the source document(s). Another wayto perform an intrinsic evaluation is to have humansubjects create a ‘‘gold’’ summary, i.e. an ideal one,which will be compared to the summary created bythe system. In this case evaluation can be morequantitative and measure things such as precisionor recall. The problemwith this approach is that it isusually difficult to make people agree on whatconstitutes a ‘‘gold’’ summary. One way to sidestepthis problem is to employ a utility-basedmeasure, inwhich a sentence is not assigned a Boolean value(belonging or not to the summary), but instead avalue in a scale, according to the opinion of eachjudge. Those values are then averaged and the topsentences can be considered as forming the ‘‘gold’’summary.


In an extrinsic method, the summary is evaluatedin relation to the particular task it is supposed toserve. Thus, such an evaluation can greatly varyfrom system to system. Relevance assessment is anextrinsic method and is usually performed by show-ing judges a document (summary or source) and atopic and asking them to determine whether thatdocument is relevant to the topic. If, on average,the choices for the source document and the sum-mary are the same, then the summary scores high inthe evaluation. Of course, instead of providing asingle topic, a list of topics can be provided askingthe judges to choose one of them. Another exampleis the evaluation of reading comprehension. In thiscase, judges are given a document (either the sum-mary or the source document) and are asked a set ofquestions. Their answers to these questions deter-mine their comprehension of the text. If the answersto the summary and the corresponding source docu-ment(s) are similar, then the summary is positivelyevaluated.

2.3. Summarization techniques

Summarization techniques can be classified accord-ing to the factors presented in Summarization fac-tors. For example, they can be classified accordingto the number of input documents (single-documentversus multi-document), to the type of these docu-ments (textual versus multimedia), to the outputtypes (extractive versus abstractive), etc. In thissection, the various summarization techniques arepresented under the following classification:

� e
xtractive; � a bstractive; � m ulti-document; � m ultimedia.
An extra category is added to include a techniquethat although it presents similarities with techni-ques in the other categories, it has the special ch-aracteristic of approaching summarization from acognitive perspective aiming at simulating the hu-man summarizers’ tasks.

2.3.1. Extractive techniquesTwo representative categories of extractive techni-ques implemented by existing systems (see Table 1)are presented below.

The first one concerns statistical techniquesbased on what [8, pp. 47—53] calls the Edmundso-nian paradigm. In this paradigm, each sentenceshould be ranked in relation to the other sentences,so that the n highest ranked sentences could beextracted and form the summary. The ranking is

normally based on a formula, which assigns a weightto each sentence based on various factors. Forexample, the cue phrases or keywords that thesentence contains, its location in the document,the fact that it may contain some non-trivial wordsthat are also found in the sections’ headings of thedocument. The problem is that most of the times theproduced summary is suffering from incoherencies(semantic gaps, anaphora problems). Some of thesystems falling under this category post-process theproduced summary using revision techniques inorder to resolve such problems.

The second category of extractive techniquesconcerns the creation of a graph (or tree) repre-sentation of the document(s) to be summarizedexploiting machine learning and/or language pro-cessing techniques. Several different representa-tions can be used:

� T
he nodes of the graph are the paragraphs of thedocument to be summarized and the edges repre-sent the similarity between the paragraphs theyconnect. The paragraphs corresponding to nodeswithmany edges can be extracted in order to formthe summary of the document.
� T
he nodes of the graph are text items such aswords, phrases, proper names and the edges arecohesion relationships between these items, suchas coreference, hypernymy. Once the graph forthe document is created, the most salient nodesin the graph can be located, based for example ona user query. The set of salient nodes can then beused to extract the corresponding sentences,paragraphs, or even sections that will form thesummary.
� A
tree representation can be created exploitingrelations from the rhetorical structure theory(RST) [12]. The tree nodes are sentences, whichare connected using RSTrelations such as elabora-tion, antithesis, etc. Then in order to get themostsalient sentences, the tree must be traversed inorder to build a partial ordering of the sentencesin terms of their importance. According to thetarget compression rate, the top n sentences canbe extracted and presented as a summary.
Table 1 presents representative systems employ-ing the extractive techniques presented above, tak-ing into account the factors specified inSummarization factors. The input field concerns t-he number of input documents, their language, andwhether they contain only text. The purposefield concerns whether the resulting summary isindicative or informative (in most of the cases itis difficult to judge this), generic or user-oriented,as well as whether the technique is a general


Method

Eva

luation

Scriptac

tiva

tion,ca

nnedge

neration

sInform

ationextraction,NLG

Eva

luationofsystem

components

Syntactic

proce

ssingofrepresentative

sentence

s,NLG

Intrinsic

-based

tation

Syntactic

proce

ssingofrepresentative

sentence

s,ontology-basedan

notation,NLG

Extrinsic

al tationin

UNL

Statistics

(forscoringeac

hUNLsentence

),removingredundan

twords,

combiningsentence

s

purpose or a domain-specific one. The output fieldconcerns the ‘‘material’’ used to create the sum-mary (sentences, paragraphs, sections). The tablealso includes a field about the specificmethods used(e.g. statistics, language processing, use of revi-sions, etc.), as well as a field on the evaluation a-pproach. The lack of the corresponding informationin some field values denotes that a definite answercannot be given.

2.3.2. Abstractive techniquesThe most straightforward way of creating abstractsis to identify in a way the most important informa-tion in the document(s), appropriately encode it andthen feed it to a natural language generation (NLG)system [20,21], which generates the summary. Tworepresentative categories of abstractive techni-ques, implemented by existing systems (see Table2), are presented below.

In the first category the process of identifying andencoding themost important information in thedocu-ment(s) can be performed using prior knowledgeabout the structure of this information. This knowl-edge is represented through cognitive schemas suchas frames, scripts, templates. Thus, in such cases, thesummary produced is not a generic one, but a ratheruser-orientedone since the schemacanbeconsideredas a sort of user query. Different approaches in thiscategory may be the following:

tput

ipts plate

sters

tology

resen

ceptu

resen

� U

semployingab

stractivetech

niques

Purpose

Ou

Inform

ative,user-oriented,

domain-specific

Scr

Inform


domain-specific

Tem

Generic,

domain-specific

(newsarticles)

Clu

Inform


domain-specific

On

rep

Con

rep

tiohtind

se of a script, i.e. a sort of a simple-structuredtemplate with slots identifying common impor-tant events over a domain. There is a separatescript for each domain. When a document isprocessed, the corresponding script is activatedand its slots filled with information from thedocument. The activation can be performedthrough the appearance of certain words, or bythe activation of another script. After the scripthas been activated and filled, the summary can begenerated using in most cases simple techniques(canned text generation versus the more sophis-ticated NLG techniques) due to the rather simplestructure of scripts.

em

, , , , t
� U
epresentative

syst

Input

Single-docu

ment

English,text

Multi-docu

ment,

English,text

Single-docu

ment

English,text

Single-docu

ment

English,text

Single-docu

ment

multilingu

al,tex

se of a MUC1-like domain-specific template, asort of a relational database, having a morecomplex structure compared to scripts. The tem-plate can be filled from a document using infor-mation extraction techniques. The filledtemplates can then be processed in order totransform them in an appropriate form for the

Table

2R

[22]

[23,24

]

[25]

[26]

[27]

1 MUC (Message Understanding Conferences) were evalua-n conferences for information extraction systems; seetp://www.itl.nist.gov/iaui/894.02/related_projects/muc/ex.html.

http://www.lehmam.freesurf.fr/automatic_summarization.htm



NLG system. Processing is done using varioussemantic operators, such as Change of Per-spective, Contradiction, Addition/Elaboration,Refinement, Agreement, etc.

The second category involves techniques that donot use prior knowledge about the structure of theimportant information to be used in the summary,but instead they produce a semantic representationof the document(s), which is then fed to the NLGsystem. Different approaches in this category maybe the following:

� T
he documents are linguistically processed in orderto identify noun phrases, verb phrases that can belinked to the concepts, attributes and relations of adomain-specific ontology. Ontology-based annota-tions can then be used to select the importantdocument regions (sentences, paragraphs). Theseregions are then converted into some semanticrepresentation using the results of the linguisticprocessing and the ontology-based annotations.This representation is then fed to an NLG systemthat produces the abstract.
� T
he summarization system can identify informa-tional equivalent paragraphs in the input docu-ment(s) using clustering techniques. From eachtheme some representative sentences areextracted, which can be analyzed syntacticallyand then fed to a sentence generator in order toproduce the abstract.
Table 2 presents representative systems employ-ing abstractive techniques. Table 2 fields are thesame with the fields in Table 1. There are differ-ences between the two tables concerning the out-put and the method fields, which are filled withdifferent values. The output field in the abstractivetechniques concerns the ‘‘semantic representa-tion’’ used to create the summary (scripts, tem-plates, ontology-based representation, clusters ofinformational equivalent document regions). Themethod field is filled with the specific methods used(script activation, information extraction, syntacticprocessing, NLG, etc.).

2.3.3. Multi-document summarizationtechniquesRadev et al. [2] define multi-document summariza-tion as the process of producing a single summary ofa set of related source documents. As they note, thisis a relatively new field where three major problemsare introduced: (1) recognizing and copingwith redundancy, (2) identifying important differ-ences among documents, and (3) ensuring summarycoherence.

While a technique extracting textual units — suchas sentences — from a single document, may copewith redundancy and preserve the coherence of theoriginal document, extracting textual units frommultiple documents increases the redundanciesand incoherencies, since textual units are not pre-viously connected across documents. Thus, abstrac-tive techniques seem more appropriate for multi-document summarization. However, extractivetechniques can also be used followed by a post-processing stage in order to ensure summary coher-ence and cope with redundancy. In both cases, thefirst step is to identify those documents talkingabout a specific topic. This can be done, usingclassification techniques in terms of relevance toa query or existing topic models, or using clusteringtechniques. The next step is to identify the impor-tant information to be added in the summary in thegroup of the topic-specific documents.

In the extractive-based category, a system canapply firstly extractive techniques to each docu-ment separately in order to locate and rank themost important document regions (phrases, sen-tences, paragraphs). The highest ranked regionsfrom all the documents can then be combined andre-ranked using similarity measures, in order tominimize redundancy. Some cohesion rules can thenbe applied to the final set of document regions inorder to produce the summary. Another approachwould be to work with all the documents from thebeginning. For instance, the topic model producedfrom the group of input documents can be comparedto sentence vectors from all the documents in orderto get the, most similar to that topic, sentences.

The abstractive-based category can also involveextractive techniques at a pre-processing stage. Insuch a case, the extracted document regions arelinguistically processed in order to be converted intosome representation, which can be used by an NLGsystem, which will then produce the abstract.Another approach involves the establishment anduse of a set of intra- and inter-document relation-ships, which could hold between various textualunits of the documents, such as words, sentences,paragraphs or whole documents, and which couldguide the identification of the most salient informa-tion across the multiple documents. Such relation-ships concern not only the similarities acrossdocuments, but also their differences (e.g. equiva-lence, contradiction, elaboration, etc.).

To cope with the inherent problems of multi-document summarization (redundancy, incoheren-cies), a different output representation can be usedinstead of producing a summary containing, forinstance, the most salient sentences across docu-ments or even their abstraction. Ando et al. [28] use


Table

3Representative

systemsemployingmulti-docu

mentsummarizationtech

niques

Input

Purpose

Output

Method

Eva

luation

[29]

Multi-docu

ment,

English,text

User-oriented,ge

neralpurpose

Extracts

Statistica

l(m

ulti-docu

mentmax

imal

marginal

releva

nce

—MD-M

RD),

revision

Intrinsic

[30]

Multi-docu

ment,

English,text

Generic,

generalpurpose

Extracts

Statistica

l(support

vectormac

hine,MD-M

RD)

Intrinsic

[28]

Multi-docu

ment,

English,text

Generic,

generalpurpose

Ascatterplotofthe

most

salientsentence

sStatistics

(vectorspac

emodel),langu

ageproce

ssing

[31]

Multi-docu

ment,

English,text

Generic,

generalpurpose

Intra-

andinter-docu

mentrelationships

a scatter plot per topic presenting the extractedsentences visually to the user.

Table 3 includes representative examples ofmulti-document summarization techniques. Com-pared to Tables 1 and 2, there are differencesconcerning the values in the output and the methodfields. The output field concerns whether the outputis an extract or abstract or even another represen-tation format. The method field is filled with thespecific methods used (statistical, language proces-sing, revision stage, use of intra- and inter-docu-ment relationships).

2.3.4. Multimedia summarization techniquesIn this section, work onmultimedia summarization isdiscussed. This involves such diverse fields as dialogsummarization, summarization of diagrams, andvideo summarization. Due to the limited numberof relevant works and their field-specific features,only one approach per field is presented.

2.3.4.1. Dialogue summarization. Zechner [32,33]works on dialogue summarization in unrestricteddomains and genres. Zechner’s works with tran-scripts of human dialogs, which are generated eithermanually or automatically (in this case Zechner alsoaddresses the issue of speech recognition errors).Themajor steps in Zechner’s work are the following.In the first step, input tokenization, all noise isremoved and the transcript is tokenized. Disfluencydetection follows in which false starts, restarts orrepairs and filled pauses are annotated and cor-rected. In this step the boundaries of the speakers’sentences are also detected. In the next step, cross-speaker information linking, pairs of question—answer among the speakers are identified and anno-tated. Those pairs are extracted together laterwhen producing the summary, thus making theresulting summary more coherent and informative.The following step is topic segmentation wheresegments of the discussion on a certain topic areidentified and a list of keywords, for each topic, isextracted. The final step, sentence ranking andselection, uses a version of the maximal marginalrelevance (MMR) algorithm [34] to create a summaryof extracted sentences, for each topic.

2.3.4.2. Diagram summarization. Futrelle [10]presented a preliminary and only partially imple-mented work on diagram summarization, yet quiteinnovative. His aim was to present a summary ofthe diagrams in a scientific paper, either byselecting (i.e. extracting) one or more diagramsfrom the paper or by distilling (i.e. simplifying) adiagram, or even by merging several diagrams.Futrelle assumes that a structural description of


the diagrams can be obtained as metadata from theauthor or by parsing the diagrams. In order toachieve his goals he takes into account not onlythe structural description of each diagram, butthe text in its caption or in the diagram itself.

2.3.4.3. Video summarization. Merlino and Mark[9] extract information from various media in orderto provide a summary of a broadcast news story. Theyuse MITRE’s broadcast news navigator (BNN [35])which helps searching and browsing of news stories,not from just one channel, but from a variety ofsources. They use silence and speaker change fromthe audio, anchor and logo detection from the videoand closed-captioned text in order to segment thestream into news stories. For the purpose of present-ing a summary, they experiment with a diversity ofpresentation methods, which include mixed-media.The first one is key-frames, i.e. important singleframes and shots i.e. an important sequence offrames. The secondone is extracted single sentences,based onweights according to the presence of namedentities (organizations, persons and places). The finalone is named entities keywords.

2.3.5. Summarization from a cognitive scienceperspectiveThe techniques presented so far pay little attentionto the ways used by humans to create the summariesthemselves. Endres-Niggemeyer et al. [36—39] onthe other hand, try to simulate the human cognitiveprocess of professional summarizers. They aim atdeveloping an empirical model for summarization,based on professional summarizers, and to imple-ment this model into a system, which will ‘‘imitate’’human process of summarizing. For this purpose,they recruited six professional summarizers whoworked on nine summarization processes. Thewholeprocess included the division of the tasks of theprofessional summarizers into sub-tasks, the inter-pretation of each sub-task into amore formal frame-work (i.e. giving a name, functional definition,etc.), and the hierarchical organization of theresulting strategies according to their function.Despite the diversity of the technical backgroundand cognitive profile of each professional summar-izer and their idiosyncrasies on creating summaries,the results are quite stunning. Quoting from [36, p.129]: ‘‘83 strategies are used by all experts of thegroup, 60 strategies are shared by five experts,another 62 strategies are common knowledge offour summarizing experts, 79 strategies belong tothe repertory of three summarization experts, 101strategies are used by two experts, and 167 strate-gies are individual.’’ From all these strategies, 79agents, which simulate them, were finally imple-

mented in the SimSum system [36]. SimSum is imple-mented as an object-oriented blackboard, whichinvolves 79 object-oriented agents, each one per-forming a relatively simple task. For instance, theContext agent checks whether the context condi-tions of the query aremet by an input document, theTexttoProposition agent transforms input sentencesinto propositions known to the domain ontology, theRedundancy agent checks if a proposition hasalready been introduced. All the agents cooperatein order to deliver the summary. They can access acommon knowledge database, which contains thetext and ontology concepts. Furthermore severalRST relations have been implemented for discourselevel structures of the text.

3. The medical domain

Medical Informatics represents the core theories,concepts and techniques of information applicationsinmedicine. It involves four different levels depend-ing on the focus from the cell to the population [40]:

� B
ioinformatics concerns molecular and cellularprocesses, such as gene sequences;
� Im
aging informatics concerns tissues and organs,such as radiology imaging systems;
� C
linical informatics concerns clinicians andpatients, involving applications of various clinicalspecialties;
� P
ublic health informatics concerns populationsinvolving applications such as the disease surveil-lance systems.
Medical information distributed through all theabove levels concerns various document types: sci-entific articles, electronic medical records, semi-structured databases, web documents, e-mailed r-eports, X-rays images, videos. The characteristics ofeach document type have to be taken into accountin the development of a summarization system. S-cientific articles aremainly composed of text and, inseveral cases, they have a sectioning that can beexploited by a summarization system. Electronicmedical records contain structured data, apart fromfree text. Web documents may appear in healthdirectories and catalogs, which need to be searchedfirst in order to locate the interesting web pages.The web pages layout is also another factor thatneeds to be taken into account. E-mailed reports aremainly free text without any other structure. X-raysimages, videos such as echocardiograms, representa completely different document type, where textmay not be included at all or may be a part of animage.


4 See http://host.ariadnegenomics.com/downloads/.5 See KDD Cup 2002 Task1: Information Extraction from

Compared to other domains, medical documentshave certain unique characteristics that make verychallenging and attractive their analysis. Unique-ness of medical documents is due to their volume,their heterogeneity, as well as due to the fact thatthey are the most rewarding documents to analyze,especially those concerning human medical infor-mation due to the expected social benefits (see[41]).

3.1. Scientific articles

The number of scientific journals in the fields ofhealth and biomedicine is unmanageably large, evenfor a single specialty, making very difficult for phy-sicians and researchers to stay informed of the newresults reported in their fields. Scientific articlesmay contain apart from text, structured data (e.g.tables), graphs, or images. Therefore, depending onthe summarization task, the system may have toprocess various types of data. It may be the case, forinstance, that important information (e.g. experi-mental results) is found in a table and needs to belocated and added to the summary. The articlelayout could also be exploited, since a large numberof articles reporting on experimental results havean almost standard sectioning, the order of thesections being Introduction, Methods, StatisticalAnalysis, Results, Discussion, Previous Work, Lim-itations of the Study, and Conclusions. A study of thetypes of scientific articles on the various fields ofmedicine is necessary for their processing either forsummarization or for other language processingtasks.

3.2. Databases of abstracts

Despite the plethora of medical journals, most ofthem are not freely accessible over the Web, due tocopyright reasons. Luckily, there are other onlinedatabases, which contain the abstracts and citationinformation of most articles on the general field ofMedicine. One such online database is MEDLINE,2

which contains abstracts from more than 3500 jour-nals. MEDLINE provides keyword searches andreturns abstracts that contain the keywords. Theabstracts are indexed according to the MedicalSubject Headings (MeSH)3 thesaurus. Apart fromthe access to the abstracts, MEDLINE also providesfull citations of the articles along with links to thearticles themselves, in case they are online. Hershet al. [40] have created a corpus from Medline,which consists of titles and abstracts from articles

2 See http://www.ncbi.nlm.nih.gov/entrez/query.fcgi.3 See http://www.nlm.nih.gov/mesh/meshhome.html.

of 270 Medical Journals over a period of 5 years. Thiscorpus is annotated with information such as theMedline identifier, MeSH terms assigned by humans,title, abstract, publication type, source andauthors. The corpus was created for the experi-ments described in [40]. Abstracts represent a dif-ferent type of document, since they are onlycomposed of text and metadata (such as the MeSHannotations in MEDLINE). A summarization systemmust be able to exploit these metadata in severalways. For instance, in multi-document summariza-tion, the system must be able to locate first thoseabstracts that discuss a specific topic, and topicinformation is found on the metadata of theabstracts. On the other hand, a sub-language ana-lysis of the abstracts may be necessary in order toidentify certain characteristics that may affect theperformance of the language processing applica-tion. As it is noted in Ariadne Genomics NLP whitepaper,4 their sub-language analysis indicated that‘‘MEDLINE sentences often include idiosyncratic lin-guistic constructs not necessarily reflected in gen-eralized English grammar’’. This explains, as theyclaim, why existing syntactic parsers with generalgrammar are not suitable for dealing with this typeof text.

3.3. Semi-structured databases

A number of databases have been built to provideaccess to biomedical information, such as informa-tion about protein function and cellular pathways.More than 280 semi-structured databases exist cur-rently.5 Some examples are the following:

� F

Biokd

lyBase: fruit fly genes and proteins;
� M ouse Genome Database (MGB); � P rotein Information Resource (PIR); � D IP: The database of interacting proteins.6
Let us take DIP, for example, which provides anintegrated set of tools for browsing and extractinginformation about protein interaction networks. Asit is reported in [42], DIP database is implementedas a relational database composed of four tables:protein table, interaction table, method table,and reference table. The reference table lists allthe references to different articles that demon-strate protein interactions and link them toMEDLINE database. Therefore, a summarization

medical Articles (http://www.biostat.wisc.edu/�craven/dcup/).6 See http://dip.doe-mbi.ucla.edu.









system that exploits information from DIP shouldbe able to employ DIP tools for browsing the data-base and even access relevant MEDLINE abstracts(see previous discussion) through the referencetable.

3.4. Web documents

During the last few years, a number of specializedhealth directories and catalogs (portals) have beencreated, such as CliniWeb,7 HON,8 CISMeF,9 MedicalMatrix, Yahoo Health, HealthFinder. Some of thesecatalogs, including the first three of the above, areadditionally indexed with the MeSH Thesaurus. Thisallows complex queries to be stated, which exploitthe hierarchical structure of the MeSH. CliniWeb, forinstance, provides clinically oriented information,including:

1. C

7

8

9

ataloging content per-page basis;
2. I ncluding only pages that have clinical content,
i.e., excluding individual and institutional homepages, advertisements, and lists of links;

3. I
ndexing with a higher level of specificity, usingMeSH as opposed to broad subject categoriessuch as Orthopedics or Cancer. CliniWeb providesaccess to Web pages manually indexed by a largesubset (trees A—G) of MeSH, including the majortrees, Diseases, Anatomy, and Chemicals andDrugs.
Another example of a health resources portal isCISMeF, which aims to describe and index the mainFrench-language health resources to assist healthprofessionals and consumers in their search forelectronic information available on the Internet.In April 2002, the number of indexed resourcestotaled over 9600 with a mean of 50 new sites eachweek. CISMeF uses two standard tools for organizinginformation: the Medline bibliographic databaseMeSH thesaurus and several metadata element sets,including the Dublin Core. To index resources, CIS-MeF uses four different concepts: ‘‘meta-term’’,keyword, subheading, and resource type. CISMeFcontains a thematic index, including medical spe-cialities and an alphabetic index.

Web pages included in catalogs, such as the aboveones, form a different type of medical documents.These are .html pages, which may contain informa-tion from other media apart from text (e.g. images,videos). Even in the case that they contain text only,there will be links pointing to other relevant pages

See http://www.ohsu.edu/cliniweb/.See http://www.chu-rouen.fr/cismef/.See http://www.chu-rouen.fr/cismef.

with interesting information for the summarizationtask, there may be interesting information stored ina table, etc. Web page layout should also be takeninto account in order to locate interesting informa-tion inside the web page, especially in the case thatthe catalog pages are generated dynamically from adatabase. In addition, the identification of webpages that are relevant to the summarization task,demands the use of web spidering techniques.Therefore, as it is the case of web informationretrieval and extraction tasks in other domains(see the results of the CROSSMARC project10 in[43]), summarizing from web documents needs totake into account the web catalog and web pagesstructure and features. It must also be able toexploit metadata information (e.g. MeSH annota-tions) already used by some of the existing webcatalogs. However, even in the cases where a webcatalog is not indexed, the summarization systemmust be able to employ existing medical ontologies,thesauri or lexica. This is essential for a scientificdomain with rich terminology, where the informa-tion to be extracted or summarized needs to be asprecise as possible.

3.5. E-mailed reports

With the advent of the Web, several web-basedservices, whose purpose is the exchange of opinionsand news, have emerged. For example, ProMED-mail11 is a public free service, which promotesthe exchange of news concerning bursts of epi-demics. Other non-free services, such as MDLinx,12

provide physicians and researchers with the oppor-tunity to subscribe and receive alerts concerningnew findings on their specialty fields, described injournal articles. The use of e-mailed reports for afast dissemination of epidemiological informationby the Internet shows an increasing success formonitoring epidemiological events [44]. Thedescriptive possibilities of these reports and theirability to deal with unattended situations makethem competitive for reporting emerging infectiousdisease outbreaks and unusual disease patterns,including biological threats. However, as [45] note,analysts cannot feasibly acquire, manage, anddigest the vast amount of information availablethrough emailed reports or other informationsources 24 h a day, 7 days a week. In addition, accessto foreign language documents as well as the localnews of other countries is generally limited. Evenwhen foreign language news is available, it is usually

10 See http://www.iit.demokritos.gr/skel/crossmarc.11 See http://www.promedmail.org.12 See http://www.mdlinx.com/.








no longer current by the time it gets translated andreaches the hands of an analyst. This very realproblem raises an urgent need for the developmentof automated support for global tracking of infec-tious disease outbreaks and emerging biologicalthreats.

ProMED-mail is a service monitoring news oninfection disease outbreaks around the world, 7 daysa week. By providing early warning of outbreaks ofemerging and re-emerging diseases, ProMED aims atenabling public health precautions at all levels in atimely manner to prevent epidemic transmissionand to save lives. ProMED sources of informationinclude — among others — media reports, officialreports, online summaries, local observers. Reportsare also contributed by ProMED-mail subscribers. Ateam of expert moderators investigate reportsbefore posting them to the network. Reports aredistributed by e-mail to subscribers and posted onthe ProMED-mail web site. ProMED-mail currentlyreaches over 30,000 subscribers in 150 countries.ProMED-mail is also available, apart from English, inPortuguese and in Spanish. Both of these lists coverdisease news and topics relevant to Portuguese- andSpanish-speaking countries, respectively.

E-mailed reports for monitoring infectious dis-ease outbreaks and emerging biological threatsrepresent a different type of medical documents.Such reports may contain apart from raw text,various types of information in attached files. Thefact that these reports may be in several languagesor may point to other sources, such as local news,makes the summarization task evenmore difficult. Asub-language analysis may also be necessary forthese types of documents since they often followa specific writing style and structure. Medical ter-minology should also be taken into account, as it isthe case in the other document types, exploitingexisting medical resources for the specific diseasesor biological threats.

3.6. Electronic medical records

Most hospitals keep a record for each of theirpatients. Usually the records contain data ofpatients in a standard structured form, with pre-defined fields or tabular representations, as well asfree text fields containing unstructured informa-tion, usually doctors’ reports about their patients(either written reports or the result of dictation). AsMckeown et al. [46] note, a patient record for anysingle patient consists of many individual reports,collected during a visit to hospital. For somepatients, this can be up to several hundred reports.

A system summarizing information from medicalrecords needs to take several factors into account. It

must be able to process the free text reports, whichmay be problematic due to the specific sub-languageused by the clinicians or due to the fact that thereport is the result of dictation. Information fromwritten reports may also be combined with informa-tion existing in structured data (tables, graphs) oreven information existing in other media (e.g. X-rays images, videos). The situation, however, maybecome even more complex for a summarizationsystem that aims to summarize information for aclinician, which is collected not only from thepatient record but also from other records (similarcases to the specific patient), from relevant scien-tific articles or abstracts from journals or databasesrespectively. If a summarization system is to beintegrated into the busy clinical workflow, it mustprovide the clinician with such facilities.

3.7. Multimedia documents

Apart fromdocuments in textual form, physicians andresearchers produce and use several other docu-ments, which are multimedia in nature. Such docu-ments can be graphs, such as cardiograms, imagessuch as X-Rays, etc., videos, such as the variousechograms, e.g. echocardiograms, echoencephalo-grams, etc., or the medical videos used mainly foreducational purposes, e.g. videos of clinical opera-tions or videos of dialogs between the doctor and thepatient.Most of thesedocuments arenowtranscribedand stored in digital form, even connected to thespecific patient record, giving the users the ability tosearch and access themmuch faster than in the past.This is a completely different type of medical docu-ments, which contain very interesting informationthat must be added to a summary. Techniques fromareas other than language processing, such as imageprocessing and video analysis must also be employedin order to locate the information to be included inthe summary. In addition, several of thesemultimediadocuments are also linked with free text reports,which must also be used by the summarization sys-tem. As noted in the discussion on electronic medicalrecords, a summarization system integrated in theclinical workflow must be able to handle such docu-ments. Concluding, in the medical domain, proces-sing of multimedia documents is crucial forsummarization and in general for information retrie-val and extraction applications.

4. Summarization techniques in themedical domain

Most of the researchers extend to the medicaldomain the techniques already used in other


domains. Based on the categorization given earlierin Summarization techniques, the techniques usedin the medical domain are classified under thefollowing categories:

� e

1

tre1

1

xtractive single-document summarization;
� a bstractive single-document summarization; � e xtractive multi-document summarization; � a bstractive multi-document summarization; � m ultimedia summarization; � c ognitive model based summarization.
In the following sections, various summarizationprojects/systems are presented based on this cate-gorization.

4.1. Extractive single-documentsummarization

One of the projects belonging in this category isMiTAP [45]. The aim of MiTAP (MITRE Text and AudioProcessing)13 is to monitor infectious disease out-breaks or other biological threats by monitoringmultiple information sources such as epidemiologi-cal reports, newswire feeds, email, online news,television news and radio news in multiple lan-guages. All the captured information is filteredand the resulting information is normalized. Eachnormalized article is passed through a zoner thatuses human-created rules to identify the source,date, and other fields such as the article title andbody. The zoned messages are processed to identifyparagraph, sentence and word boundaries as well aspart-of-speech tags. The processed messages arethen fed into a named entity recognizer, whichidentifies person, organization and location namesas well as dates, diseases, and victim descriptionsusing human-created rules. Finally, the document isprocessed by WebSumm [47], which generates asummary out of modified versions of extracted sen-tences. For non-English sources, a machine transla-tion system is used to translate the messagesautomatically into English. In addition to single-document summarization, MiTAP has recently incor-porated two types of multi-document summariza-tion: Newsblaster [46]14 automatically clustersarticles and generates summaries based on eachcluster, Alias I15 produces summaries on particularentities and generates daily top 10 lists of diseasesin the news.

3 For more information on MiTAP, visit http://tides2000.mi-.org.4 See http://www.cs.columbia.edu/nlp/newsblaster.5 See http://www.alias-i.com/.

Another project is MUSI [48]. MUSI stands for‘‘MUltilingual Summarization for the Internet’’and it is a cross-lingual summarization system,which uses articles from The Journal of Anaesthe-siology as input. The journal is freely accessibleonline16 and its articles are written in Italian andEnglish. MUSI takes those articles and creates sum-maries from them in French and German. The sys-tem is query-based and it extracts sentences fromthe input article according to the following criteria:cue phrases, position of the sentences, query wordsand compression rate. That is, MUSI follows theEdmundsonian paradigm for the selection of thesentences. Once the sentences have beenextracted, two approaches can be followed: eitherthey are used as they are to form the extractivesummary or they are converted into a semanticrepresentation to produce an abstractive summary.

A third project exploiting extractive techniques ispresented in [49] The most important aspect of thisapproach is that it ranks the extracted sentencesaccording to the so-called cluster signature of thedocument. More specifically, their prototype systemtakes medical documents (result of a query using asearch engine) as input and clusters them intogroups. These groups are then analyzed for featureswith high support, called key features, forming acluster signature that best characterizes each docu-ment group. The summary is generated by matchingthe cluster signature to each sentence of the docu-ment to be summarized. Both the sentence and thecluster signature are represented using a vectorspace model. The ranked sentences are thenselected and presented to the user as a summary.Johnson et al. [49] used for their experimentsabstracts and full texts from the Journal of theAmerican Medical Association.

4.2. Abstractive single-documentsummarization

MUSI [48] is a system generating either extractivesummaries (see the previous section) or abstractiveones. In the case of abstractive summarization,after the system has selected the sentences, itconverts them into a predicate-argument structurerepresentation, instead of simply presenting themto the user. The steps in achieving that representa-tion are: tokenization, morphological analysis,shallow syntactic parsing, chunking, dependencyanalysis and mapping to the internal representa-tion. After the representation has been achieved,they create the summaries of those extracted

16 You can access this online journal at http://anestit.unipa.it/esiait/esiaing/esianuming.htm.








sentences using the natural language generation(NLG) system Lexigen [50] for the French languageand TG/2 [51] for German. The generationsystems produce indicative summaries of thedocument content. Summaries include both trans-lated portions of the extracted sentences, and‘‘meta-statements’’ about the original document.The latter provide the user with additionaloptional information about the content and struc-ture of the source text, the relevance of theextracted pieces of information as well as of thewhole document with respect to the query, etc.Users can customize the summary length, as wellas some other aspects concerning style and presen-tation.

TRESTLE (Text Retrieval Extraction and Summar-ization Technologies for Large Enterprises) is asystem, which produces single sentence summariesof Scrip17 pharmaceutical newsletters [52]. Theirsystem is in essence an Information Extraction sys-tem, which relies heavily on Named Entity (NE)recognition. For this system, also drug names anddiseases are named entities, apart from the classicalones, such as organization, person and location.TRESTLE allows users to navigate through the Scriparticles, and thus find the information that they areinterested in, using the named entities that thesystem has extracted, which are links to the originalarticles from which the NEs have been extracted.Apart from this, TRESTLE also creates single sen-tence summaries for each newsletter from the tem-plate that was filled by the Information Extractionprocess. A link is also provided to the original news-letter.

4.3. Extractive multi-documentsummarization

Although the production of summaries frommultipledocuments is usually done with abstractive techni-ques, Kan et al. [53,54] follow a different approach.They argue that different types of summaries, suchas indicative or informative, serve different infor-mational purposes and both can be useful, and thatextracting sentences for the creation of an informa-tive multi-document summary ‘‘is well acceptedsince it is simple, fast and easy to evaluate’’. Theirsystem, Centrifuser, which is the summarizationengine of the PERSIVAL (PErsonilized Retrieval andSummarization of Image, Video and Language) pro-ject,18 produces both indicative and informativemulti-document summaries, with the aim of high-

17 See http://www.pjpub.co.uk for more information on Scripnewsletters.18 See http://persival.cs.columbia.edu/.

lighting the similarities and differences among thedocuments.

The input to the Centrifuser is articles, retrievedby the search engine of the PERSIVAL system accord-ing to the patient record and the user query. Foreach article they create a topic tree which depictsthe sectioning of each article. A composite topictree is then created by merging together all thetopic trees and adding details to each node such asrelative typicality (i.e. how typical is that topiccompared to the rest of the topics), position withinthe article, and various lexical forms in which it maybe expressed [53]. In the next step they try to matchthe nodes of the topic trees with the query. Thematched nodes do not contain any text, but insteadthey point to sections in the original documents,from which the most representative sentencesshould be extracted. Since the compression rateposed will not always allow for each topic to receivea sentence, the first step is to choose which topicsare going to receive a sentence. In the next stepthey choose for each topic the representative sen-tences. The final step for the creation of the sum-mary involves the ordering of those sentences,which is achieved by first ordering the topics accord-ing to each topic’s typicality, and then ordering thesentences themselves inside every topic, accordingto the physical position of every sentence.

4.4. Abstractive multi-documentsummarization

Apart from informative extractive multi-documentsummaries, Centrifuser creates indicative abstrac-tive multi-document summaries, as well, which areused by the PERSIVAL users for searching papers. Asnoted in Extractive multi-document summarization,the approach of Kan et al. [53,54] leads to nodes inthe topic trees which match with the query of theuser. This, they argue, can be the first phase forNatural Language Generation (NLG). In the next stepof NLG, which they call planning, they try to figureout which nodes of the topic trees they will sum-marize. To achieve this, they determine whichnodes are relevant, irrelevant and intricate, basedon how deep the nodes are, compared with thequery node, i.e. the node that matches the userquery. Thus, nodes that are descendants of thequery node and are below depth k are consideredintricate, above depth k relevant and all the othernodes (i.e. the ones that are not descendants of thequery node) are irrelevant. In the final NLG step,realization, the ordered information is converted totext. For a more thorough treatment of Centrifuser,see [55].




Apart from the abstractive indicative summariesthat Centrifuser produces, PERSIVAL producesanother type of abstractive summary [1,56]. Thesesummaries are not concerned with highlighting thesimilarities and differences among several medicalarticles, but with the creation of an informativeabstractive summary. That summary is tailoredaccording to the preferences of two different typesof users: the physician, the patient or her relatives.

The system should identify in the documents andextract tuples of the form: (Parameter(s), Finding,Relation). The relations can be any of the followingsix types: association, prediction, risk, absence ofassociation, absence of prediction and absence ofrisk. They call those tuples results. Elhadad andMckeown [56] using empirical methods, i.e. inter-views with the physicians, concluded that a sum-mary should fulfill the following qualitative criteria,in relation to the results:

� C
ompleteness and accuracy. The results should becomplete and accurate, in the sense that all therelevant results and only them should beincluded.
� R
epetitions and contradictions. The systemshould identify repetitions and contradictionsamong the results. In order to do so, Elhadadand Mckeown [56] have created a representationof the results, which allows them to identifyrelations such as subsumption and contradictionamong the results.
� C
oherence and cohesion. Coherence for [56] isestablished by ‘‘accurate aggregation and order-ing of the related results’’. Cohesion is defined asfollows: ‘‘two sentences are part of the sameparagraph, if and only if they are related.’’Related are the sentences that present eitherthe same finding, or the same parameter(s).
The system described in [56] takes input fromthree different sources:

� P
atient record. In general, the patient recordconsists of structured documents, usually in tab-ular form, and unstructured documents, andsometimes it can be very large.
� J
ournal medical articles. Their system takesas input a vast amount of online articles frommedical journals on the field of cardiology. Infact, the articles that are input to the systemare the ones that globally match the patient, i.e.the ones that contain information relevant to thepatient.
� T
he user query. Although the physician’s query isposed in natural language, the system does not tryto fully understand the question and give an
answer, but instead gives as much information aspossible about the question using some of thequery keywords.

The input articles are first classified automati-cally into three categories: prognosis, treatment,diagnosis. The next step involves the identificationand extraction of the results, i.e. the tuples men-tioned above. For this purpose, the authors areexploiting the ‘‘rigid’’, as they call it, structureof the medical articles. This means that they try tolocate the Results section and select the sentencesthat are relevant to the patient. The selected se-ntences are then passed to the extraction module,that extracts in a template form, the following i-nformation: the finding(s), the parameters, therelation, the degree of dependence of the para-meters, the article and the sentence it has beenextracted from and various other minor informa-tion. The templates are filled with the aid of hand-crafted patterns.

The next step involves the determination ofwhich portions, if any, of the extracted parametersare relevant with the patient record. After that, theresulting templates are merged and ordered. Toachieve this, the templates are rendered into aninternal ‘‘semantic’’ representation, in the form ofa graph. From this graph, they are able to identifyrepetitions and contradictions. A repetition occursif two nodes are connected bymore than one vertex,and the vertices have ‘‘similar’’ types. What issimilar and what is not has been ‘‘established’’ ininterviews with physicians. A contradiction occurs inthe same situation, but now the vertices have dif-ferent types. Repetitions and contradictions areused in order to create a more coherent summary.With this method they manage to perform the mer-ging of the templates. For the ordering, they use thefollowing criteria:

� Q
uery based: a relation that answers the userquery is weighted higher.
� S
alience based: recitations and contradictions areweighted higher.
� D
omain based: studies with physicians show thatsome relation types are more interesting thanothers. For instance a risk relation is weightedhigher than an association relation.
� S
ource based: dependent relations from the sametemplate are presented together.
The final step involves the creation of the sum-mary, through NLG techniques. In the finalsummary, all the medical terms are hyperlinkedto their definitions. This is achieved by conne-cting the system of Elhadad and Mckeown [56]

172S.

Afan

tenosetal.

Table 4 Summarization systems from medical documents

Input Purpose Output Method Evaluation

[45] Single-document(also multi-document), mutilingual,multimedia (text, audio, video)

Indicative, user-oriented,domain-specific

Sentences (extracts) Language processing (named entityrecognition, machine translation),machine learning

Extrinsic

[48] Single-document, multilingual, text Indicative, user-oriented,domain-specific

Sentences (extracts),abstracts

Statistics (sentences extraction),language processing (semanticrepresentation for abstraction)

Intrinsic, extrinsic

[49] Single-document, monolingual, text Indicative, generic,domain-specific

Sentences (extracts) Statistics (vector space model)

[52] Single-document, monolingual, text Indicative, generic,domain-specific

Abstracts Language processing(information extraction)

[55] Multi-document, monolingual, text Indicative-informative,generic, domain-specific

Extracts, abstracts Statistics (clustering using similaritymeasures), language processing

Extrinsic

[56] Multi-document, monolingual, text Informative, user-oriented,domain-specific

Abstracts Language processing (informationextraction, NLG)

[58] Single-document, video(echocardiograms)

Generic, domain-specific Video sequences(extracts)

Image and video processing

[59] Single-document, video(clinical operations,dialogues, presentations)

Generic, domain-specific Video sequences(extracts)

Image and video processing

[61,62] Multi-document, monolingual, text Informative, user-oriented,domain-specific

Abstracts Agents simulating summarization tasks,language processing


with DEFINDER, a text mining tool for extrac-ting definitions of terms from medical articles(see [57]).

4.5. Multimedia summarization

Ebadollahi et al. [58] and Xingquan et al. [59] pre-sent systems performing summarization of docu-ments which have multimedia content,echocardiograms and medical videos respectively.

The work presented in [49] is part of the PERSIVALproject mentioned above. In their study they areconcerned with echocardiograms (ECGs). ECGs areusually videotaped for archival purposes andrecently they have started to be transcribed intoa digital format, which helps clinicians, and facil-itates the task of summarizing them. Summarizingan ECG, and video in general as seen in the work ofMerlino and Mark [9], involves extracting the mostinteresting video frames, which are called key-frames, which enable the user to easily navigatethrough the ECGs and view their essential parts. For[58] summarizing an ECG involves two things: par-sing the ECG and selecting the key-frames. The aimof the parser is to temporally segment thesequences of the video into smaller units, whichare called shots. A shot is a sequence of frames inwhich the camera is uninterrupted. In the context ofECG videos, a shot corresponds to a single positionand angle of the ultrasound transducer. The methodthey use for the parsing is a special case of thealgorithm presented in [60]. The next step is thekey-frame selection, which extracts the most infor-mative (important) frames in the sequence of thevideo. After mentioning several methods forextracting key-frames, they conclude that for thecontext of ECGs key-frames are ‘‘the local extremaof the cardiac periodic expansive—contractivemotion’’, since ‘‘the time at which the cardiacmotion changes from expansive to contractive cor-responds to the end-diastole and the time at whichthe motion changes from contractive to expansivecorresponds to end-systole’’. Having performed theabove two tasks, they create two summaries whichthey call static and dynamic.

� S
tatic summary. This summary, in essence, isconstituted from the selection of the extractedkey-frames, and it is useful for browsing the con-tent of the echo video.
� D
ynamic summary. This summary, also called clin-ical summary among the clinicians, is a concate-nation of the small extracted sequences of thevideo. They chose to extract one (or more, basedon the needs of the clinicians) cycle of the heartmotion, known also as R—R cycle. By joining those
segments of videos they create the dynamic sum-mary.

Xingquan et al. [59], follow a similar approach to[58] in order to parse the video stream into physicalunits. Then video group detection, scene detectionand clustering strategies are used to mine the videocontent structure. Various visual and audio featureprocessing techniques are utilized to detect somesemantic cues, such as slides, face and speaker c-hanges, etc. within the video, and these detectionresults are joined together to mine three types ofevents from the detected video scenes (presenta-tions by doctors or experts on video topics, clinicaloperations to present details of diseases, and dia-logs between doctors and patients). Based on minedvideo content structure and event information, ascalable video skimming and summarization tool,ClassMiner, has been constructed to visualize thevideo overview and help users access video content.Their system utilizes a four-layer video skimming,where levels 4 through 1 consist of representativeshots of clustered scenes, all scenes, all groups, andall shots of the video, respectively.

4.6. Cognitive model based summarization

Based on the cognitive model used in the SimSumsystem (see Summarization from a cognitive scienceperspective), Endres-Niggemeyer [61,62] presentedits extension, the SummIt-BMT, which is concernedwith the summarization of MEDLINE abstracts andarticles for bone marrow transplantation, a specia-lized field of internal medicine. SummIt-BMT is aquery-based summarization system. In general, thesummarization process is the following:

1. A
user forms a search scenario using conceptsfrom the domain ontology.
2. T
his scenario is mapped to a MEDLINE query. Ifthe outcome of the query points to journal arti-cles, they are included in the results.
3. A
text retrieval component identifies the inter-esting pieces of text in the results.
4. T
hose pieces are summarized in relation to thequery scenario. Links to the original articles arealso given.
Although SummIt-BMT is based on SimSum it dif-fers from it in several ways. It is not a presentationalmodel anymore but a functional one. Thus, agentssimulating lower level cognitive processes have b-een replaced by functional ones. Text productionagents have been removed since SummIt-BMT doesnot produce smooth text, but organized text clipsthat are linked to their source positions. As the


application field is bone marrow transplantation(BMT), a BMT ontology was set up. Although severalmedical ontologies existed which are loosely relatedto the BMT field, a BMT-specific ontology was cre-ated due to the fact that the existing ontologies didnot contain enough deep BMT knowledge for textknowledge processing. The ontology they created isvery important for Summit-BMT, since it is beingused in almost all the stages of the summarizationprocess.

A scenario interface reflecting everyday situa-tions of BMT physicians [63] helps users to statetheir queries. Users fill in ontology concepts, whichare for their convenience equipped with definitionsand explanations assembled from various sources onthe web. From scenario forms and user-selectedontology terms the system obtains structuredqueries in the predicate-logic form that it can‘‘understand’’. Queries are given to the searchengines, which return a set of documents, abstractsand maybe journal articles, from MEDLINE. Theretrieved documents are checked for possible rele-vance by a text passage retrieval component. Irre-levant documents are discarded. From the finalset of documents, the summarization agents takethe positive passages from text passage retrieval,represent their phrases and sentences in a predi-cate-logic form, and examine them with human-style criteria: whether they are related to the userquery, whether they are redundant, and so on. Theagents remove items that do not meet their rele-vance criteria.

Table 4 summarizes the main features of theprojects/systems presented in Summarization tech-niques in the medical domain.

5. Promising paths for future research

Although initial work on Summarization dates backto the late 1950s and 1960s (e.g. [3,4]), mostresearch on the field has been performed duringthe last few years. The result is that the researchfield has not yet achieved a mature state, and avariety of challenges still need to be overcome. Thescaling to large size collections of documents, theuse of more sophisticated natural language proces-sing techniques for generating abstracts, the avail-ability of annotated summarization corpora fortraining and testing purposes, are some of thesechallenges.

This is also the case for the domain of medicaldocuments. The study of existing summarizationtechniques in other domains, the examination ofdifferent types of medical documents and the studyof techniques reported so far in the literature for

medical summarization lead to certain interestingremarks concerning the promising paths for futureresearch. These remarks are presented below interms of the summarization factors.

In terms of the input medium, almost all methodsconcern summarization from text, although thespecific domain can provide a lot of useful inputin other media as well (e.g. speech, images, videos).Summarizing information from different media (e.g.spoken transcriptions and textual reports related tothe specific echo-videos) is an important issue forpractical applications, representing a promisingpath for future research and development.

Concerning the number of the input documents,both categories of techniques (single and multi-document) have been examined. As it is the casein other domains, apart frommedicine, single-docu-ment summarization methods are mainly usingextractive techniques, whereas almost all of themulti-document summarizers are based on abstrac-tive techniques. However, the selection betweenthe simpler extractive techniques and the morecomplex abstractive ones should not only be basedon the number of input documents, but also on theavailable resources, tools and the summary purposeand output factors.

Concerning the language of the input docu-ment(s), most of the existing systems are monolin-gual (English in almost all the cases). There are twocases (MiTAP, MUSI), where the multilingual aspectwas taken into account. In the MiTAP case, this wasdue to the domain (monitoring disease outbreaks)where the information sources are in various lan-guages. On the other hand, MUSI summarizes thearticles of a bilingual Journal. In themedical domainthere is an enormous amount of documents in var-ious categories (e.g. patient records) in other lan-guages apart from English. There are resources andtools in several other languages that can beexploited in building summarizers for handling morethan one language using either shallow or deeperapproaches for language processing.

In relation to purpose factors, the existing meth-ods mainly concern indicative summarization. Thepurpose of such summaries is to navigate the readerto the required information, which seems to besufficient for most practical applications in medi-cine as long as no better solution is available. Theproduction of indicative summaries seems to ‘‘indi-cate’’ that the shallow summarizing strategies usedso far are not enough for producing informative oreven critical summaries. Deeper language proces-sing techniques [64] and their combination withshallow processing ones seems to be a promisingpath for future research in NLP in general and morespecifically in summarization.


There is a trend towards user-oriented summa-ries, which is reasonable since summarization sys-tems in the medical domain aim to cover theinformation needs of different user types (clini-cians, researchers, patients) and specific users. Userinvolvement does not concern only the submission ofa query to the system, but also the summary cus-tomization and presentation according to the user’smodel. The PERSIVAL system [56] maintains infor-mation about the users’ preferences taking intoaccount their expertise in the domain, as well asthe users’ access tasks. The summary presentationcan also be affected by the user’s model (e.g.production of a summary in the form of hypertext,combination of text and images or video, etc.).Personalized access to medical information is acrucial issue and needs to be further investigated.There is a lot of expertise from the application ofuser modeling techniques in other domains whichcan also be exploited in the medical domain (see[65,66]).

Domain customization is another significantissue. Most of the existing medical summarizationsystems are able to process documents belonging inspecific sub-domains of medicine. Emphasis mustbe given to the development of technology that canbe easily ported to new sub-domains. The devel-opment of open architecture systemswith reusableand trainable components and resources is impera-tive in summarization technology. This is directlyrelated to the ability of exploiting pre-existingmedical knowledge resources. There are currentlyvarious knowledge repositories such as the UnifiedMedical Language System (UMLS),19 and MeSH,which can be exploited in several ways by summar-ization engines. For instance, they can be used tolocate interesting document(s), and interestingsentences inside those documents. They can evenbe used to create conceptual representations ofthe selected sentences in order to produce abstrac-tive summaries in the same or in a different lan-guage. Such approaches are presented in theliterature and can be further investigated. Thedevelopment of customizable summarization tech-nologies requires also in-depth study of themedicaldocument types and medical sub-language. A gen-eral-purpose system must be able to exploit thevarious characteristics of the medical documents.For instance, the sectioning of scientific articles,the specialized language used in e-mailed reportsor in patient records are important features thatcan significantly affect the performance of theinvolved language processing tools. In general,the research community must cooperate towards

19 See http://www.nlm.nih.gov/research/umls/.

the development of portable summarization tech-nology and the medical domain can provide thenecessary application areas.

Concerning the output factors, the quality of thesummarization output is strongly related to thesummarization task. Therefore, qualitative andquantitative criteria need to be established follow-ing a study of the domain and the users’ interests. Interms of the decision between extractive andabstractive techniques, as noted above, this hasto take into account several factors related to theinput documents, the purpose of the summary, thequalitative criteria established as well as the avail-able resources and tools.

6. Conclusions

This survey presented the potential of summariza-tion technology in the medical domain, based on theexamination of the state of the art, as well as ofexisting medical document types and summariza-tion applications.

The challenges that the summarization researchhas to overcome need to be viewed under the prismof the requirements of the specific field. The scalingto large collections of documents in various lan-guages and from different media, the generationof informative summaries using more sophisticatedlanguage and knowledge engineering techniques,the generation of personalized summaries, the port-ability to new sub-domains, the design of evaluationscenarios which model real-world situations, theintegration of summarization technology in practi-cal applications such as the clinical workflow, areamong the issues that the summarization commu-nity needs to focus on.

Acknowledgements

The authors would like to thank the anonymousreviewers, as well as Dr. Constantine D. Spyropoulosand Dr. George Paliouras, for their helpful and con-structive comments. Many thanks also to Ms. EleniKapelou and Ms. Irene Doura for checking the use ofEnglish.

References

[1] McKeown KR, Jordan DA, Hatzivassiloglou V. Generatingpatient-specific summaries of online literature. In: HovyE, Radev D, editors. Intelligent text summarization papersfrom the 1998 AAAI symposium, vols. 34—43. Satabford, CA,USA: AAAI Press; 1998.



[2] Radev D, Hovy E, McKeown K. Introduction to the specialissue on text summarization. Comput Linguist 2002;28(4).

[3] Luhn HP. The automatic creation of literature abstracts. IBMJ Res Dev 1958;2(2):159—65.

[4] Edmundson HP. New methods in automatic extracting. JAssoc Comput Mach 1969;16(2):264—85.

[5] Paice CD. The automatic generation of literature abstracts:an approach based on the identification of self-indicatingphrases. In: Oddy RN, Robertson SE, van Rijsbergen CJ,Williams PW, editors. Information retrieval research. Lon-don: Butterworth; 1981. p. 172—91.

[6] Paice CD. Constructing literature abstracts by computer.Inform Process Manage 1990;26(1):171—86.

[7] Sparck-Jones K. Automatic summarizing: factors anddirections. In: Mani I, Maybury MT, editors. Advances inautomatic text summarization. 1999. p. 10—12 [chapter 1].

[8] Mani I. Automatic summarization. Volume 3 of Naturallanguage processing. Amsterdam/Philadelphia: John Benja-mins Publishing Company; 2001.

[9] Merlino A, Mark M. An empirical study of the optimal pre-sentation of multimedia summaries of broadcast news. In:Mani I, Maybury MT, editors. Advances in automatic textsummarization. 1999. p. 3910—401 [chapter 25].

[10] Futrelle RP. Summarization of diagrams in documents. In:Mani I, Maybury MT, editors. Advances in automatic textsummarization. 1999. p. 4030—421 [chapter 26].

[11] Mani I, Maybury MT, editors. Advances in automatic textsummarization. The MIT Press; 1999.

[12] Mann WC, Thompson SA. Rhetorical structure theory:towards a functional theory of text organization. Text1988;8(3):243—81.

[13] Dalianis H, Hassel M, de Smedt K, Liseth A, Lech TC, Wede-kind J. Porting and evaluation of automatic summarization.In: Holmboe H, editor. Nordisk Sprogteknologi. 1988.

[14] Abderrafih L. Multilingual alert agent with automatic textsummarization. http://www.lehmam.freesurf.fr/automa-tic_summarization.htm.

[15] Hsin-Hsi C, Lin C-J. Multilingual news summarizer. In: Pro-ceedings of the18th International Conference on Computa-tional Linguistics, July 31—August 4 2000. University ofSaarlandes, p. 159—65.

[16] Salton G, Singhal A, Mandar M, Buckley C. Automatic textstructuring and summarization. Inform Process Manage1997;33(2):193—207.

[17] Mani I, Bloedorn E. Summarizing similarities and differencesamong related documents. Inform Retrieval 1999;1(1):1—23.

[18] Marcu D. The Rhetorical parsing of natural language texts.In: Proceedings of the 35th Annual Meeting of the Associationfor Computational Linguistics. New Brunswick, New Jersey:Association for Computational Linguistics; 1997, p. 96—103.

[19] Marcu D. The theory and practice of discourse parsing andsummarization. The MIT Press; 2000.

[20] Reiter E, Dale R. Building applied natural language genera-tion systems. Nat Language Eng 1997;3(1):57—87.

[21] Reiter E, Dale R. Building natural language generation sys-tems. In: Studies in natural language sing. Cambridge Uni-versity Press; 2000.

[22] DeJong G. An overview of the FRUMP system. In: LehnertWG, Ringle MH, editors. Strategies for natural languageprocessing. New Jersey: Erlbaum: Hillsdale; 1982. p. 149—76.

[23] Radev DR, McKeown KR. Generating natural language sum-maries from multiple on-line sources. Comput Linguist1998;24(3):469—500.

[24] Radev DR. Generating natural language summaries frommultiple on-line sources: language reuse and regeneration.PhD thesis, Columbia University; 1999.

[25] Barzilay R, Elhadad M. Using lexical chains for textsummarization. In: Mani I, Maybury MT, editors. Advancesin automatic text summarization. 1999. p. 1110—121.

[26] Saggion H, Lapalme G. Generating indicative-onformativesummaries with SumUM. Comput Linguist 2002;28(4):497—526.

[27] Virach S, Potipiti T, Charoenporn T. UNL document summar-ization. In: Proceedings of the First International Workshopon Multimedia Annotation (MMA2001), Tokyo, Japan, Janu-ary 2001.

[28] Ando R, Boguraev B, Byrd R, Neff M. Multi-document sum-marization by visualizing topical content. In: Proceedings ofthe ANLP/NAACL 2000 Workshop on Automatic Summariza-tion, Seattle, WA, April 2000.

[29] Goldstein J, Mittal V, Carbonell J, Callan J. Creating andevaluating multi-document sentence extract summaries. In:Proceedings of the 2000 ACM CIKM International Conferenceon Information and Knowledge Management, McLean, VA,USA, November 2000, p. 165—72.

[30] Tsutomu H, Suzuki J, Isozaki H, Maeda E. NTT’s multipledocument summarization system for DUC 2003. In: Proceed-ings of the Workshop on Text Summarization, at the HumanLanguage Technology Conference 2003, Edmonton, Canada,May 31—June 1, 2003.

[31] Radev DR. A common theory of information fusionfrom multiple text sources, step one: cross-documentstructure. In: Proceedings of the First ACL SIGDIALWorkshop on Discourse and Dialogue, Hong Kong, October2000.

[32] Zechner K. Automatic summarization of spoken dialogues inunrestricted domains. PhD thesis, Carnegie Mellon Univer-sity, School of Computer Science, Language TechnologiesInstitute, November 2001.

[33] Zechner K. Automatic summarization of open-domain multi-party dialogues in diverse genres. Computat Linguist2002;28(4):447—85.

[34] Carbonell J, Goldstein J. The use Of MMR, diversity basedreranking for reordering documents and producing summa-ries. In: Proceedings of the 21st Annual International ACMSIGIR Conference on Research and Development in Informa-tion Retrieval, Melbourne, Australia. Poster Session, Augusts1998.

[35] Merlino A, Morey D, Maybury M. Broadcast news navigationusing story segments. In: Proceedings of the ACM Multi-media; 1997. p. 381—91.

[36] Endres-Niggemeyer B. Summarizing information. Berlin:Springer-Verlag; 1998.

[37] Endres-Niggemeyer B. SimSum: an empirically foundedsimulation of summarizing. Inform Process Manage2000;36(4):659—82.

[38] Endres-Niggemeyer B, Maier E, Sigel A. How to implement anaturalistic model of abstracting: four core working steps ofan expert abstractor. Inform Process Manage 1995;31(5):631—74.

[39] Endres-Niggemeyer B, Neugebauer E. Professional summar-izing: no cognitive simulation without observation. J Am SocInform Sci 1998;49(6):486—506.

[40] Hersh W, Buckley C, Leone TJ, Hickam D. OHSUMED: aninteractive retrieval evaluation and new large test collec-tion for research. In: Proceedings of the 17th Annual ACMSIGIR Conference; 1994. p. 192—201.

[41] Cios Krzysztof J, William Moore G. Uniqueness of medicaldata mining. Artif Intell Med 2002;26:1—24.

[42] Xenarios I, Salwinski L, Duan XJ, Higney P, Kim S, EisenbergD. DIP: the database of interacting proteins a research toolfor studying cellular networks of protein interactions. NuclAcids Res 2002;30:303—5.




[43] Karkaletsis V, Spyropoulos CD. Cross-lingual informationmanagement from web pages. In: Proceedings of theNinth Panhellenic Conference in Informatics (PCI-2003);2003.

[44] Woodall J. Official versus unofficial outbreak reportingthrough the internet International. J Med Inform1997;47:31—4.

[45] Damianos L, Day D, Hirschman L, Kozierok R, Mardis S,McEntee T, et al. Real Users, Real Data, Real Problems:The MiTAP System for Monitoring Bio Events. Proceedingsof the Conference on Unified Science & Technology forReducing Biological Threats & Countering Terrorism (BTR2002); 2002. 167—77.

[46] McKeown K, Elhadad N, Hatzivassiloglou V. Leveraging acommon representation for personalized search and sum-marization in a medical digital library. In: Proceedings of theJoint Conference on Digital Libraries; 2003.

[47] Mani I, Bloedorn E. Summarizing similarities and differencesamong related documents. Inform Retrieval 1999;1(1):1—23.

[48] Lenci A, Bartolini R, Calzolari N, Agua A, Busemann S, CartierE, et al. Multilingual summarization by integrating linguisticresources in the MLIS-MUSI project. In: Proceedings of theThird International Conference on Language Resources andEvaluation (LREC’02); 2002.

[49] Johnson DB, Zou Q, Dionisio JD, Liu VZ, Chu WW. Modelingmedical content for automated summarization. Ann NYAcadSci 2002;980:247—58.

[50] Coch J, Chevreau K. Interactive multilingual generation. In:Gelbuckh A, editor. Computational linguistics and intelligenttext processing. Lecture notes in computer science, vol.2004. Berlin: Springer-Verlag; 2001.

[51] Busemann S. Best-first surface realization. In: Scott D,editor. Eighth International Natural Language GenerationWorkshop Proceedings; 1996 p. 101—10.

[52] Gaizauskas R, Herring P, Oakes M, Beaulieu M, Willett P,Fowkes H, et al. Intelligent access to text: integratinginformation extraction technology into text browsers. In:Proceedings of the Human Language Technology Conference(HLT 2001); 2001. p. 189—93.

[53] Kan M-Y, McKeown KR, Klavans JL. Applying natural languagegeneration to indicative summarization. In: Proceedings ofthe Eighth European Workshop on Natural Language Gen-eration; 2001.

[54] Kan M-Y, McKeown KR, Klavans JL. Domain-specific informa-tive and indicative summarization for information retrieval.In: Workshop on text summarization (DUC 2001); 2001.

[55] Kan, M-Y. Automatic text summarization as applied to infor-mation retrieval: using indicative and informative summa-ries. PhD dissertation, Columbia University, New York, USA,February; 2003.

[56] Elhadad N, McKeown KR. Towards generating patient specificsummaries of medical articles. In: Proceedings of the Auto-matic Summarization Workshop (NAACL 2001); 2001.

[57] Klavans JL, Muresan S. DEFINDER: rule-based methods forthe extraction of medical terminology and their associateddefinitions from on-line text. In: Proceedings of the Amer-ican Medical Informatics Association Annual Symposium,AMIA 2000; 2000.

[58] Ebadollahi S, Chang S-F, Wu H, Takoma S. Echocardiogramvideo summarization. In: Proceedings of the SPIE MI 2001;2001.

[59] Xingquan Z, Fan J, Hacid M-S, Elmagarmid AK. Classminer:mining medical video for scalable skimming and summariza-tion. In: Proceedings of the 10th ACM International Confer-ence on Multimedia (Demonstration); 2002. p. 79—80.

[60] Zabih R, Miller J, Mai K. A feature-based algorithm fordetecting and classifying scene breaks. In: Proceedings ofthe ACM Multimedia; 1993. p. 189—200.

[61] Endres-Niggemeyer B. Empirical methods for ontology engi-neering in bone marrow transplantation. In: InternationalWorkshop on Ontological Engineering on the Global Informa-tion Infrastructure; 1999.

[62] Endres-Niggemeyer B. Human-style; 2001. www summa-rization. http://www.ik.fh-hannover.de/ik/person/ben/human-stylesummanew.pdf.

[63] Becher M, Endres-Niggemeyer B, Fichtner G. Scenario formsfor web information seeking and summarizing in bone mar-row transplantation. In: COLING 2002: Workshop on Multi-lingual Summarization and Question Answering; 2002.

[64] Oepen S, Flickinger D, Uszkoreit H, Tsuji J. Introduction tothe special issue on recent achievements in the domain ofHPSG-based parsing. J Nat Language Eng 2000;6(1):1—14.

[65] Alfred K. Generic user modeling systems. User Model User-adapted Interaction 2001;11(12):49—63.

[66] Pierrakos D, Paliouras G, Papatheodorou C, Spyropoulos CD.Web usage mining as a tool for personalization: a survey.User Model User-adapted Interaction 2003;13(4):311—72.

http://www.ik.fh-hannover.de/ik/person/ben/human-stylesummanew.pdf

http://www.ik.fh-hannover.de/ik/person/ben/human-stylesummanew.pdf

Date post:	18-May-2020
Category:	Documents
Upload:	others
View:	9 times
Download:	0 times

Summarization from medical documents: a surveycgi.di.uoa.gr/~takis/aim.pdf · Summarization from...

Documents