Chapter 1 FROM TREEBANK RESOURCES TO LFG F-STRUCTURES Automatic F-Structure Annotation of Treebank Trees and CFGs extracted from Treebanks Anette Frank Xerox Research Centre Europe [email protected]Louisa Sadler University of Essex [email protected]Josef van Genabith Dublin City University [email protected]Andy Way Dublin City University [email protected]Abstract We present two methods for automatically annotating treebank resources with functional structures. Both methods define systematic patterns of correspondence between partial PS configurations and functional structures. These are applied to PS rules extracted from treebanks, or directly to constraint set encodings of treebank PS trees. Keywords: Automatic annotation, higher-level syntax, corpus linguistics, robustness 1
In this contributionwe addresstwo importantconcerns:automaticannota-tion of treebanksandCFGsextractedfrom suchtreebankswith LFG f(eature)-structures(KaplanandBresnan1982),(Dalrympleetal. 1995).
Treebankswhichencodehigher-level functionalstructure,in additiontopurephrasestructureinformation,arerequiredastrainingresourcesfor probabilisticunificationgrammarsanddata-drivenparsingapproaches,e.g. (Bod andKa-plan1998).Manualconstructionof treebankswithfeaturestructureannotationsis very labourandcostintensive. Sois thedevelopmentof new or thescaling-upof existingunificationgrammarswhichcanbeusedto analysetext corpora.What is more,even if a large-coverageunificationgrammaris available,typi-cally, for eachinputstringit wouldgeneratehundredsor thousandsof candidate(constituentandfeaturestructure)analysesfrom whichahighly trainedexperthasto select. Although proposalshave beenmadefor filtering andrankingparsingambiguities(e.g. (Charniak1993),(Abney 1997),(Franketal.2000)),to datenoneis guaranteedto uniquelydeterminethebestanalysis.In ordernotto compromisethequalityof thecorpusunderconstruction,a linguisticexpertis requiredto find thebestamonga largenumberof candidateanalyses.
In anumberof papersvanGenabithetal. (1999a,b,c)presentedanew corpusbasedmethod. The basicidea is the following: take an existing treebank,readoff the CFG following (Charniak1996), manuallyannotateit with f-structureannotations,providemacrosfor thelexical entriesandthen“reparse”thetreebanktrees(not thestrings)deterministicallyfollowing theoriginal treestructureannotationsassignedin thetreebank.Duringthis“reparsing”process,the f-structureannotationsareresolved, andan f-structureis produced.Theentire processis deterministicif the featurestructureannotationsare, andto a considerableextent costly manual inspectionof candidateanalysesisavoided. The methodis an improvementbut still involves a large labourintensivecomponent,namelymanualannotationof thegrammarrules.
Treebankgrammars(CFGsextractedfromtreebanks)arelargeandgrow withthe sizeof the treebank(Charniak1996),(Krotov et al. 1998). They featureratherflat rules,many of whichshareand/orrepeatsignificantportionsof theirRHSs. This causesproblemsfor manualrule annotationapproachessuchastheonedescribedin (vanGenabithet al. 1999a,b,c).Manualrule annotationis labourintensive,errorprone,repetitiveandrisksmissinggeneralisations.
In thispaperweshow how f-structureannotationof bothgrammarrulesandtreefragmentscan(to a largeextent)beautomated.
The observation that constituentand higher-level featurestructurerepre-sentationsstandin a systematicrelationshipinformstheoreticalwork in LFG(KaplanandBresnan1982),(Dalrympleet al. 1995)andHPSG(PollardandSag1994). In LFG c(onstituent)-structureand f-structureare independentlevelsof representationwhich arerelatedin termsof a correspondencefunc-tion
Whatis new in ourapproachis that(i) weemploypartial andunderspecifiedannotationprinciplesin aprinciplebasedc- to f-structureinterfacefor theLFGarchitecture;(ii) we usetheseto automatefunctionalstructureassignmenttoflat and“noisy” treebanktreesandCFGsextractedfrom them;(iii) we reuseexisting linguistic resources.In contrastto moretheoreticallyinformedworkin LFG andHPSG,treebanksdonot tendto follow highly abstractandgeneralX � architecturaldesignprinciples.Thechallengein ourapproachis to developgrammarsandannotationrulesfor realtext.
Thepotentialbenefitsof automationareconsiderable:substantialreductionin developmenteffort, hencesavings in time andcostfor treebankannotationandgrammardevelopment;the ability to tacklelarger fragmentsin a shortertime,aconsiderableamountof flexibility for switchingbetweendifferenttree-bankannotationschemes,anda naturalapproachto robustness.Our methodscan alsobe viewed as a new corpus-anddata-driven approachto grammardevelopment,anapproachthatasmuchaspossiblerecyclesexistingresources.
In our work to datewe have developedtwo relatedbut interestinglydiffer-ent methods.Both methodsdefineassociationprinciplesascorrespondencesbetweenpartial andunderspecifiedc- andf-structureconfigurations.In oneapproachwereadoff aCFGtreebankgrammarfollowing themethodof Char-niak(1996)andthencompileannotationprinciplesoverthetreebankgrammar.In our secondapproachwe operatedirectly on constraintsetencodingsof PStreebanktreesandrewrite or annotatethemdirectlywith f-structures.
Both methodsarepartial in the following further sense:the first requiresmanualinspection,completionand correctionof the output (setsof anno-tated grammarrules) producedby the automaticannotationprocess. Thesecondmethodis fully automaticandrobust,andyields partial,unconnectedf-structuresin thecaseof missingannotationrules.
Wedescribetwoexperiments,onefor eachmethod.In ordertoexploresomeof the possiblearchitectures,for the first experimentwe developeda regular
4
expressionbasedannotationprinciple interpreterwhich operateson grammarruleswith order independentandmonotonicinterpretationof principles. Forthe secondexperimentwe employeda term rewriting systemwhich operateson constraintsetdescriptionsof LFG structures.The term rewriting systemallows us to exploit both order dependent, cascadedandorder independentformulationsof annotationprinciples. In our first experimentwe usedthefirst 100 treesof the AP treebank(LeechandGarside1991), in the second,experiment166treesof theSusannetreebank(Sampson1993).
The paperis structuredasfollows: in Section2 we motivateanddescribeour methodsin moredetail. In Sections3 we reporton our two experiments.For eachexperimentwe explain thedesign,describethedataandevaluatetheresults.In Section4wecomparethetwomethodsandoutlineongoingresearch.Section5 concludes.
2. METHODS FOR AUTOMATIC F-STRUCTUREANNOTATION
In LFG the correspondencebetweenfunctional and phrasalstructureisdefinedin termsof functionalannotationsof theRHScategoriesin CFGrules.
S F GIHJLKNM'OQPSR�TVUXW Y HKNUXW VP F YKNUXW GIHJLKNZ[PSR�TVUXWAnnotationfollows universalandlanguagespecificprinciples. We define
annotationprinciplesasinvolving partial andunderspecifiedphrasestructureconfigurationsandapply themto CFG rulesor tree fragmentsthat meettherelevantpartial configuration.To illustratethe idea: a headprincipleassigns\
= ] to the X daughterin all XP ^ _�_8_ X _8_�_ configurations,irrespective ofthesurroundingcategorial context. For theexampleat hand,thechallengeinour approachis to provide annotationprinciplesthat identify headsin theflattreebanktreeandruleconfigurationswhichgenerallydeviatesignficantlyfromX � designprinciples.Annotationprinciplescapturegeneralisationsandcanbeusedto automaticallyannotatePSconfigurationswith functionalstructuresina highly generalandeconomicalway. Both annotationmethodsarebuilt on
FromTreebankResourcestoLFG f-Structures 5
this insight: in the first, correspondencesareappliedto CFG rulesextractedfrom treebankswhile in the secondcorrespondencesare applieddirectly toconstraintsetencodingsof treebanktreesandtreefragments.
2.1 F-STRUCTURE ANNOTATION OF CFGSEXTRACTED FROM TREEBANKS
Annotation Principle Interpr eter. Our CFGrule annotationprinciplesareof the form `baQcVd[e . ` and c areregular expressions(under)specifyingLHSsandRHSsof CFGrulesin termsof categorial andconfigurationalconstraints.The regular expressionsprovided includeKleeneandpositive Kleene( f , g ),optionality( h ), disjunction( i ) andalimited form of complement( j ). e is asetof attribute-valuestructureannotations(rule decorations).Givena grammarruleof theform kla[m (expandingamothercategory k intoasequenceof daughtercategoriesm ) andanannotationprinciple `na[cld[e , if ` matchesk and c matchesm , then kVaQm is annotatedwith e . A singlegrammarrule canmatchmultipleprinciplesandasingleprinciplemaymatchagivengrammarrule in morethanoneway. Theannotationsresultingfrom all possiblematchesarecollectedandthegrammarrule is annotatedaccordingly. More formally, let thedenotationo oApnq q
of aregularexpressionp
bethesetof stringsdenotedbyp.GivenaCFGrulekla[m andasetof annotationprinciplesebr of theform `na[cld[e , kla[m is annotated
with thesetof featurestructureannotationss :kVaQmVdQs if f sutwv8e/xAybr{z|ebr with rut}`ba[cld[e and k~z o o ` q q and m{z o o c q q�Annotationis monotonicandorderindependent.
Example Principles. In our Prolog implementation,CFG grammarrulesextractedfrom thetreebankarerepresentedas���%�~���Q�Q�%�Q�Q�u�����u�������%�����
The next principle statesthat in non-conjunctive contexts·¸�l� sequences,possiblyseparatedby adverbials¹bº[� , form opencomplementsequenceswherethesubjectof thepreceding�V� controlsthatof thefollowing:
construction.In �V�ÉÆ��V� sequencesthesecond�V� providesanopencomplement���Q�S�b� to thefirst with the subjectof the secondcontrolledby the subjectofthefirst. The �b� in thefirst rule is analysedastheobjectof therightmost �V� ,while the �b� in thesecondrule is eitheranadjunctor anobliqueargumenttothe �[� . The last two examplerulesshow conjunctivestructures.Note that inthefinal rule the �n� is analysedasobliqueor asanadjunctto therightmost�[� .Hereour currentannotationprinciplesmissa possibleattachmentof the �n� tothemother�[� .
FromTreebankResourcestoLFG f-Structures 7
2.2 F-STRUCTURE ANNOTATION OF TREEBANKTREES
In oursecondapproachwebuild onapurecorrespondenceview of theLFGarchitecture,wherethemappingfromc- to f-structureisencodedby theprojec-tion function
(n1)= ô � ó(n2) = ô & ( ô � M�O[P´R )= ô & , ( ô &�õ�öQ÷�ø )= ’temperature’ó
(n3)= ô 9 ó(n4) = ô ñ ó
(n1) =ó
(n3) =ó
(n4) ( ô ñ õ�öQ÷�ø = ’rise’ . . .
Modular projectionprinciples for f-structure annotation of treefragments.To illustratethekey ideaof partialf-structureannotationprinciples,below wedisplaythe representationof a complex NP. This complex configurationcanbebrokendown into modular, piece-wisecorrespondencesof partial c- andf-structures,abstractingawayfromirrelevantmaterialin thesurroundingcontext.
Thefunctionalcontributionof theprenominaldeterminertheis independentof the presenceof AP or PP, and is capturedby the partial correspondenceconstraintsstatedontheright handside.
NP:n1
Det:n2 . . . N:n4
the
���Xð�'&Ið � � ��� ú � > ; ��0 � ó(n1) = ô � ó
(n1)=ó
(n2)ó(n2) = ô & spec(ô � , the)
An AP daughterof NP is analysedasan � ����������� of the nominalhead,unlesstheN headis omitted.Theformergeneralisationis capturedbelow.
8
NP:n1
AP:n3 . . . N:n4
���Xð�� < � �é� 2 ú >�� �'9 ð�������� ó(n1)= ô � adjunctx( ô � , ô 9 )ó(n3)= ô 9
�-projectionandf-structurein a modular, declarative way. By
dint of abstractingaway from immaterialc-structurecontext, the principlesgeneraliseover specifictreeconfigurations,andthereforeapply to fragmentsof unseentrees.
In the correspondence-basedapproachannotationprinciplescan apply tonon-local tree fragments. This allows us to associatepartial f-structureswith complex c-structurefragments. For example,by specifyingnon-localc-structurefragmentsin binary branchingVPs, we capturetenseand ac-tive/passive distinctionsof the verbal complex in a naturalway. This is il-lustratedfor thecharacteristicconstructionindicativeof presentperfecttense.
BDCCETheideaof modularannotationprinciplesis muchin thespirit of projection
principlesasproposedby(Dalrymple2000)and(Bresnan2000),andprovidesaprinciple-basedc- to f-structureinterfacein theLFGarchitecture.% Applicationof annotationprinciplesto c-structuretreesfollowsthedescription-by-analysis(DBA) approachof (HalvorsenandKaplan1995)in thec-to-f-structureinter-face. While in the classicalDBA approachcompletePS rules are matchedagainstthe c-structure,in our approachpartial (non-local)c-structure frag-mentsarematchedagainstthec-structuretrees.
A term rewriting systemfor f-structure annotation. To defineandpro-cessannotationprincipleswe make use of an existing term rewriting sys-tem,originally designedfor transfer-basedMachineTranslation(Kay 1999),(Frank1999).
The systemtakesas input an unorderedset of n-ary terms � Æ'& , and anorderedsetof rewrite rules �)( & . * If theLHS terms� matchtheinput, the
FromTreebankResourcestoLFG f-Structures 9
matchingterms� areeliminatedfrom theinputset,andtheterms & areaddedto theoutputset. A rule appliesto each instantiationof theLHS termsin theinput. Besidesterms � thatareto beeliminatedfrom the input, theLHS maystatepositive g� andnegative +S� terms. A rule with positive term g� onlyappliesif � matchessometermin theinput. Positive termsarenot eliminatedfrom the input set. A rule with negative term +� only appliesif � doesnotmatchany termin theinput. Theorderin which therulesarestatedis crucial:Eachrule appliesto thecurrent inputset,andyieldsanoutputset.Theoutputsetof a ruleconstitutestheinputsetfor thenext rule.
A term representationof the LFG architecture We encodetheLFG pro-jectionarchitecturein a termrepresentationlanguageasfollows:
via thepredicate@A&ÎÔ×¹CB�D"sb�ÇÆ�sFEHG . The rule below definesthe externalNP asthe I���J�� of K � , the f-structureprojectedfrom the S node. The predicate�CLC@n� �MD5N�Æá��G is defined(by useof macros)asa finitely constrainedtransitiveclosureovertheprecedencerelation�OLP@b� . It canbeusedtounderspecifyprece-denceconstraintsholdingbetweennodesQ�R and QOS , allowing for anarbitraryor elsea restrictedsequenceof interveningcategories.
Order independencein a cascadedrewrite system Although annotationrulesoperatein a cascaded,orderdependentway, orderindependencecanbeobtainedby requiringthatno annotationrule refersto f-structureinformationintroducedby other rules, and no rule consumes(or adds)any c-structureinformationreferredto by otherrules.Theseconstraintsensurethatannotationruleshave accessto thefull initial input structure,andno morethanthis, andtherebyguaranteeorderindependenceof annotation,irrespectiveof theorderinwhichtherulesarestatedandapplied.Theeffectof orderindependencecanbeobservedby invertingtheapplicationorderof thesubjectandhead-projectionrulesabove: while the intermediatetermsetwill bedifferent,thefinal outputsetwill beidentical.
avoid applicationof differentannotationrulesto thesametreefragment,lead-ing to inconsistencies.Referenceto f-structureinformationcanalsobeusedto generaliseannotationrules. If several PSconfigurationsare indicative ofe.g.asubject,or passivevoice,suchdiverseconfigurationscanbecapturedbyreferringto themoreabstractf-structureinformationtofurtherguidef-structureconstruction.Theorderof annotationrulesmustthenensurethat therequiredf-structureinformationis introducedby previousannotationrules.
An annotation grammar consists,just like an ordinaryLFG grammar, ofdifferenttypesof annotationrules: lexical, morphosyntactic,andphrasal.
Lexical and morphosyntactic rules Morphosyntacticrulesintroducemor-phological(andsomesemantic)informationencodedin lexical category labelsinto the f-structurespace. The examplegiven below illustrateshow highlyspecificcategory distinctionsin treebankencodingscanbeneutralised:once����\ ber is encodedin f-structure,basedon the �b� Ä vs. �n��È distinction,thedistinctioncanbeneutralisedby mappingboth lexical category labelsto thegeneralisedlabel �b� (see(vanGenabithet al. 1999b)for a similar approach).Suchgeneralisationsareessentialfor compactrule definition. For example,below the instantiationof the ]P^H_P� -value of nounsis capturedin a singlelexical rulewhichappliesto all “generalised”�b� -daughters.
arc(A,ML,B,nn1)==
=num(B,sg),ntype(B,common),arc(A,ML,B,nn).
arc(A,ML,B,nn2)==
=num(B,pl),ntype(B,common),arc(A,ML,B,nn).
+arc(A,n,B,nn),+lex(B,Lex) ==
=equal(A,B),pred(B,Lex), pers(B,’3’).
Tenseinformationaswell astheactive/passive distinctioncanbecapturedby statingconstraintson thepartialc-structurecontext of verbs,asillustratedbelow for presentperfecttensein a flat VP, as it is assignedin the Susannecorpus. For binarybranchingVPs (asassignedin thePenn-IITreebank),wecandefinecomplex tenseinformationin similarways,by extendingannotationrulesto non-localtreefragments(seeabove and(Frank2000)).
BEPartial phrasalrulesandunderspecification Annotationrulesaredesignedto apply to modular, partial c-structureconfigurations,to definetheir corre-
12
spondingfunctionalprojections.Eventhoughtreebanksdonot tendto followclassicalX � syntax,specifictypesof treebranchescorrespondto functionalde-pendenciesin f-structure.Annotationrulesapply, in thegeneralcase,to singletreebranches,with somecontextual constraints,andgeneraliseto unseentreeconfigurations.Below, that-clauses(category � ) areassociatedwith a func-tion �Hab\;] in f-structureby referringto a singlebranch( ¹TL×� ) in c-structure,abstractingaway from irrelevantco-occurrencesin thec-structurecontext.
The examplealso illustratesthe effect of underspecification.that-clausescanappearin differentsyntacticcontexts. By referringto an underspecified(variable)mothernodelabel kV` , we generaliseover variouspossiblemotherlabels(e.g. (in)finite,modal,nominalor adjectivephrases).
+arc(A,ML,B,f), +compform(B,that) ` comp(A,B).
Finer categorial restrictionscan be capturedby definingclassesof categorylabelsin disjunctive templates.c Below, the disjunctive template�b� �Q¹XdeDZfYGdefinesa classof category labels(� ÆnºÒÆS� ). Thetemplateis called(by logical“and” &&) in theannotationrule for PPs(� ) to definethis restrictedclassofalternativeNP-typesascomplements(i.e., a J�� ) of prepositionsin asinglerule.
Grammatical function assignment In languageslike English,grammaticalfunctionassignmentreliesheavily onc-structureconfigurations,while still notbeingfully deterministic.In casemarkinglanguages,morphologicalmarkingwill be usedto constraingrammaticalfunction assignment.Below we givean examplefor the assignmentof abJO� vs. a J���i functionsfor transitive andditransitive verbs in English, which is determinedby surfaceorder. Long-distancephenomenaarecapturedby pathexpressions(see(Frank2000)).
+arc(A,vp,C,np),+arc(A,vp,D,np),+precx(C,D) ==
=obj2(A,D). %
ZQPSRZjof ditransitives
+arc(A,vp,C,np),+arc(A,vp,D,np),+precx(C,D) ==
=obj(A,C). %
ZQPSRof ditransitives
+arc(A,vp,C,np),-arc(A,vp,D,np),g D k == Ch ==
=obj(A,C).l %
ZQPSRof transitives
Subcategorisationassignment Weinducesubcategorisationframes(these-manticforms)bycollectinggrammaticalfunctionsassignedbyannotationrulesinto the predicate’s semanticform, following themethodof (vanGenabithetal. 1999a).
Obviously, pure c-structureinformation doesnot allow us to distinguishbetweenNP, PP, or infiniti val argumentsvs. adjuncts. Similarly, lackinglexical information,raisingandcontrolconstructionscanonly berepresentedasinvolving anaphoriccontrol. In (Frank2000)we show how to extendthis
Partial annotationandrobustness Oursecondf-structureannotationmethodembodiesan importantaspectof robustness.In the caseof missingannota-tion rulesthesystemdoesnot fail, but partial treesareleft without f-structureannotation.Weobtain(typically large)partial,unconnectedf-structures.
Moving treebanks Our framework can also be usedto adjust particulartreebankencodings,by “moving” treebanksto a differentstructuralencoding,therebyfacilitating principle-basedf-structureinduction. In our treatmentoftheSusannecorpus,wedefinedasetof c-structurerewriting rulesto transformtheencodingof coordinationandflat modalVP structuresinto morestandardPSanalyses,which lendthemselvesto principle-drivenf-structureannotation.
3. TWO EXPERIMENTS
3.1 EXPERIMENT I
Experiment Design. Our first experimentinvolvesthefirst 100treesof theAP treebank(LeechandGarside1991). We refer to this subsectionasAP01.Wepreprocessthetreebankusingthestructurepreservinggrammarcompactionmethodreportedin (vanGenabithet al. 1999b)preservingasmuchcategorialfine-grainednessas is requiredto guide annotation. From this we extract atreebankgrammarfollowing (Charniak1996). We develop a setof featurestructureannotationprinciples. The regular expressionbasedinterpreterde-scribedin section2.1 compilestheprinciplesover therulesextratedfrom theAP01 treebankfragment. The resultsobtainedarecomparedagainsta man-ually annotated“gold standard”referencegrammarandprecisionandrecallmeasuresarereported.mData. The AP treebankannotationschemaemploys183 lexical tag typesand53 non-terminalcategory types,with treestructureencodedin termsoflabelledbracketing.Thecorpusis ‘skeletallyparsed’,thatis, it containssomeunlabelledbrackets.We remove thesein an automaticpre-editingstep. Thesentencesin the AP01 fragmentrangefrom 4 to 50 leaf tokens(includingpunctuationsymbols). The AP01 sectionof the corpusattests94 of the 183lexical tag typesand 25 of the 53 phrasaltag types. The large numberofhighly discriminatingterminalandnon-terminalcategoriesresultsin a largenumberof flat andoftenveryspecificrules.To facilitateannotationweusethestructurepreservinggrammarcompactionmethodpresentedin (vanGenabithetal.1999b)tocompactthegrammarintoamoregeneralonethatstill preservesimportantcategorial informationto drive automaticannotation.Compaction
T1 sentences average phrasal lexical CFGrulelengh types types types
AP01 100 20 25 94 511AP01c 100 20 12 28 330
Manually AnnotatedReferenceGrammar. In ordertoevaluateExperimentI wemanuallyconstructeda“gold standard”referencegrammarfollowing(vanGenabithetal.1999a,b,c).Thegrammarfeatures1143annotations,onaverage3.46annotationsperrule.
Automatic Annotation and Evaluation. For theexperimentweconstructed119 annotationprinciples, this against330 CFG rules resulting in a tem-plate/ruleratioof 0.36.Weexpecttheratioto skew morein favourof templatesaswe proceedto larger fragments.Automaticannotationgenerates1029an-notations,on average3.12annotationsper rule. ExperimentI is evaluatedintermsof precisionandrecallmeasures:
Thenumbersareconservative: precisionandrecallarecomputedautomat-ically for a first passencodingof annotationprinciplesasregularexpressions.The resultsare encouragingand indicatethat automaticannotationis moreoftenpartialthanincorrect.
3.2 EXPERIMENT II
Our methodfor f-structureannotationof treesin section2.2 is evaluatedinExperimentII, this time basedontheSusannecorpus(Sampson1993).
FromTreebankResourcestoLFG f-Structures 15
Data TheSusannetreebankencodeslabelledbracketedstructureswith sur-faceformandlemmatisedlexical entries.Functionalcategorylabels(subj,obj)andtracesindicatingcontrolor long-distancedependenciesareeliminatedinpreprocessing,to guaranteeanon-biasedevaluationwith conventionalPStreesasinput. In preprocessingwe alsocollapseoverspecificphrasalcategories.
Somedecisionson PS assignmentin the Susannecorpusare debatable.We defineda setof c-structurerewriting rulesthat transformtheencodingofcoordinationandflat modalVP structuresinto a standardPSanalysis.
Experiment Design We chosetwo sectionsof theSusannecorpus,J01andJ02(text typeJ: learnedwriting). On these,werananexperimentin 3 steps:
First,basedonthefirst66sentencesof J01,wedevelopf-structureannotationrulestocover50sentences.In step2weapplytheresultingannotationgrammarAG1 to the first 50 sentencesof J02, and measurethe annotationresults.GrammarAG1is thenupgradedto AG2,whichcoversthese50sentences.Werecordthe numberof rulesthat wereaddedor modified. In step3, AG2 isappliedto theremaining46 sentencesof J02. Again, we measuretheresults.In this experimentwe applied an order dependentannotationschemethatconsumesc-structuretermswhilebuildingupthef-structure(cf. (Frank2000)).We establisheda naturalorderfor thedifferenttypesof annotationprinciplesdiscussedin section2.2.1.
Evaluation and Results Table3 providesbasicdataon thesesubsections:the numberof sentencesandaveragesentencelength; the numberof phrasaland lexical categoriesand the numberof distinct PSrulesand PSbranchesencodedby thecorpustrees.Notethatthepercentageof new (unseen)PSrulesin J02-1andJ02-2is considerablyhigherthanfor new (unseen)treebranches.This is not surprising,andsupportsour annotationscheme,whereannotationinvolvesunderspecified,partialtrees(oftensinglebranches).
Although small scale,we considertheseresultsaspromising. Upgradingto larger fragmentstakeslittle effort due to the generalisationcapacityofannotationprinciples. This is alsobroughtout by the increasingpercentageof correct f-structureassignmentsto unseentrees,and the fact that partialf-structureassignmentsgenerallyconsistof largepiecesof partialf-structures.
4. DISCUSSIONAND CURRENT RESEARCH
We have presentedtwo automaticf-structureannotationmethodsfor tree-banksandgrammars.Both methodsandthe experimentsshow considerableoverlapandseveralinterestingdifferences.
AnnotationprinciplescanapplytoextractedPSrulesor toPStreefragmentsencodedasconstraintsets.OursecondmethodcanbespecialisedtoPSrulesbyrestrictingtreestodepthone.Thefirstmethodgeneratesanannotatedgrammar,whichcanbeusedto reparsetreebanktreesor serveasabasisfor developingastand-aloneLFGresource.In thesecondapproachanf-structureisbuilt duringtheannotationprocess.In orderto parsefreetext, this methodcanbeappliedto the outputof (P)CFGparsing. The samearchitecturecanbe implementedusingtheprinciplesdesignedin thefirst approach.Our secondapproachcanbemodifiedto annotate(non-local)treefragmentswith f-descriptionsfor thereparsingscenarioappliedin the first method. Both our methodsusecom-pactiontechniquesfor generalisingoverspecificcategorisation. In the firstexperimentthe structureof treebankentriesremainsunchangedwhile in thesecondcertainstructuresaretransformedto conventionalPSanalysesto sup-portprinciple-basedannotation.For ourfirstmethod,weimplementedanorderindependentandmonotonicannotationprinciple interpreter. For the second,a moregeneralterm rewriting systemwasused. The term rewriting systemallowsusto exploit anorderdependent,cascadedstatementandprocessingofannotationprinciples.Alternatively, thetermrewriting systemcanimplementorderindependentannotationwithoutconsumptionof inputconstraints.Orderindependencecansometimeseasemaintenanceof annotationprinciples,but
FromTreebankResourcestoLFG f-Structures 17
requiresmorecomplex andverboseconstraintsin orderto avoid inconsistentannotations.By contrast,orderdependentcascadedrewriting allowsfor acom-pactrepresentationof annotationrules.Theextrapowerof anorderdependentsystemcanbe usefulin category generalisationandsubcategorisationinduc-tion duringtheannotationprocess.ExperimentI usesa manuallyconstructed“gold standard”referencegrammarfor evaluation,experimentII is evaluatedwith respectto how it performsonextendingthetreebankfragment.For largerfragments,clearlythis is theonly possibleevaluationmethod.
Robustnessis an inherentpropertyof the approachespresentedhere. Itresidesin anumberof levels: First,ourprinciplesarepartialandunderspecifiedandwill matchnew, asyet unseenconfigurations.Second,theprinciplesareconditional. If a certaincontext (a regular expressionor a constraintset) ismet, a principle applies. Even if only few principlesapply, the systemwillnot fail but deliver partialannotations.Third, theconstraintsolver employedin our secondmethodcancopewith partial, unconnectedor evenconflictinginformation. A constraintsolver of this type can alsobe importedinto theprocessingof rulesannotatedby ourfirst method.
Both approachesfactor out information spreadover CFG grammarrulesinto modularandgeneralprinciples.To a first approximation,thereasonwhyour principlesallow a compactrepresentationof grammaticalknowledgeisthefollowing: by andlargetheannotationprinciplescapturestatementsaboutsingle mother– daughterrelationshipsin CFG rulesor local treesof depthone. This meansthat the principlesareessentiallyaboutsinglebranchesinlocal configurations.Given a treebank(grammar)with Q distinct categoriestheworstcasenumberof distinctbranchesis Q+· . Contrastthis with theworstcasenumberof possiblegrammarrules:r ^ts � ut Q ·r ^ts � s · ut Q ã_�_8_ _8_�_ _8_�_r ^ts � _8_�_Ws�v ut Q v�w �
Clearly, givenagrammarwith Q categoriesandaRHSrulelengthof atmostx , theworstcasenumberof differentgrammarrulesvy z � � Qz w �|{ Q ·
for x~}o� ismuchhigherthantheworstcasenumberQ+· of distinctbranches.
18
In our currentresearchwe areworkingwith thePenn-IItreebankresource.Comparedto ourAP andSusanneexperimentsweareapplyingourmethodstoa treebankfragmentlargerby anorderof magnitude.
In order to develop stand-aloneLFG grammarswe needsemanticforms(subcategorisationlists) to enforcesubcategorisationrequirements. We arecurrentlyexploring a numberof waysof semi-automaticallycompiling thesefrom machinereadabledictionariesand the f-structureannotatedcorpusre-sourcesproduced.
We expect that our approachcanalsofeed into grammardevelopmentef-forts. To besure,becausetreebankgrammarsarelargeandflat, automaticallyannotatedtreebankgrammarsare lessmaintainablethan the morecompact,linguisticallydesignedgrammarswhichfollow X � designprinciples.However,aspointedout above, our approachesallow for a novel grammardesignandprocessingarchitecture:givenatreebank,aPCFGcompiledfrom thetreebankparsesnew text. For eachinput string, the (possibly Q -) bestparsetreesarepassedon to the annotationinterpreterswhich annotateor rewrite the parsetreesandinducef-structures.We considerthis a promisingnew approachtolargescaleandcorpusbasedgrammardevelopementwith applicationsin areassuchasinformationretrieval.
5. SUMMARY
We have presentedtwo automaticf-structureannotationmethodsfor tree-banksandgrammars.Theapproachesmakeuseof acorpus-basedstrategy thattakesdisambiguatedtreestructuresasinput,andcombinethemwith traditionalrulebasedtechniquesin theform of (linguisticallymotivated)annotationprin-ciples. Theprinciplesareusedto automaticallyenrichtreebanksor extractedtreebankgrammarswith higher-level functional informationnot presenttheoriginalcorpora.Automaticannotationholdsconsiderablepotentialin curtail-ing developmentcostsandopensupthepossibilityof tacklinglargefragments.To date,our experimentsarerelatively small-scale.Still, we have presentedangrammardevelopmentandtreebankannotationmethodologywhichis data-driven, semi-automatic,reusesexisting resourcesand covers real text. WefoundtheLFG framework very conducive to ourexperiments.We dobelieve,however, that the methodscanbe generalised,andwe intendto apply themin an HPSGscenarioandto semanticrepresentationbasedannotations.Oursecondmethodcouldbeappliedto work in aTAGscenario(seealsothecloselyrelatedwork in Neumann(1998,2000)andNeumannandFlickinger(1999)).Inour work to date,contrary to an often perceived view, we have found thattreebanksencodehighly usefullinguisticinformation,albeitoftenin ratherflatrepresentations.
FromTreebankResourcestoLFG f-Structures 19
Our methodsencouragework in thebestlinguistic traditionas(i) they areconcernedwith reallanguageand(ii) they enforcegeneralisationsin theformof annotationprinciples.Ourmethodsfactorout informationspreadoverCFGrules into modularand generalprinciples. What is new in our approachisthat(i) theprinciplesstatepartialandunderspecifiedcorrespondencesbetweenc- and f-structureconfigurationsand (ii) they are appliedto flat and noisytreebankrepresentationsthatdo not follow generalX � designprinciples. Ourexperimentsshow how theoreticalwork andideason principlescantranslateinto grammardevelopmentfor real texts. In this sensethemethodsmayhelpto bridge the often perceived gap betweentheoreticallymotivatedviews ofgrammarasasetof principlesvs. grammarsfor “real” text.
Acknowledgements
Theauthorswish to thankthemembersof thePargramgroup,in particularRonKaplan,Mary DalrympleandJohnMaxwell aswell asJoanBresnan,forhelpfuldiscussionsandfeedback.
Figure 1.A.1 F–structurefor: “Observationsof the radio emissionof a planetwhich hasanextensive atmospherewill probethe atmosphereto a greaterextent than thoseusing shorterwave lengthsand should in somecasesgive otherwiseunobtainableinformation about thecharacteristicsof thesolidsurface.”
91.6%,respectively. Theseresultswereachievedwith ourpreviousProloglist constraintbasedformulationof annotationprinciplesandthecorrespondinginterpreter. In movingto thenew regularexpressionbasedformat presentedin this paperwe have not yet beenable to undertakethe all importantfine tuning ofprinciplesrequiredto achieve precisionandrecall resultsabove90%. Fine tuning is currentlyunderwayandwe hopeto report the final results(ratherthan the preliminary first passresults)beforethe presentvolumegoesto print.
M. Dalrymple,R.M Kaplan,J.T. Maxwell III, andA. Zaenen,editors.1995.Formal Issuesin Lexical-FunctionalGrammar. CSLI LectureNotes,No.47.CSLI Publications.
M. Dalrymple.2000.Lexical-functionalgrammar. Manuscript,XeroxPARC.A. Frank.1999.FromParallelGrammarDevelopmenttowardsMachineTrans-
lation.A ProjectOverview. In Proceedingsof MachineTranslationSummitVII ”MT in theGreatTranslationEra” , pages134–142.
A. Frank. 2000. Automatic F-StructureAnnotation of TreebankTrees. in:M. ButtandT.H.King editors,Proceedingsof theLFG00Conference,19- 20July 2000,Universityof Californiaat Berkeley, CSLI OnlinePublications,Stanford,CA, [�dFdQ� �5�T�> T F P+V�bÓTBV�b¡"Ó>dl¹Î�V�l�FLnº7¡W@QºSÔY� �bÔnÕYBl�[�Q¹Xd+�´�Î�+ÓT� .
22
A. Frank,T. King, J. Kuhn and J. Maxwell. 2000.Optimality TheoryStyleConstraintRankingin Large-scaleLFG Grammars(revised andextendedversion).In Sells,P., editor,OptimalityTheoreticSyntax.CSLI Publications,Stanford,CA.
M. Kay. 1999.ChartTranslation.In Proceedingsof MachineTranslationSum-mit VII ”MT in theGreatTranslationEra” , pages9–14.
A. Krotov,M. Hepple,R.Gaizauskas,andY.Wilks. 1998.CompactingthePennTreebankGrammar. In Proceedingsof COLING/ACL’98, pages699–703.
G. LeechandR. Garside,1991.Runninga GrammarFactory:On theCompi-lation of ParsedCorpora,or ‘Treebanks’, pages15–32.MoutondeGruyter,Berlin.
G.Neumann.2000.AutomaticExtractionof StochasticLexicalizedTreeGram-mars from Treebanks.In A. Abeille (ed) Treebanks:building and usingsyntacticallyannotatedcorpora.Kluwer
G. NeumannandD. Flickinger. 1998.LearningStochasticLexicalizedTreeGrammarsfrom HPSG.DFKI TechnicalReport,Saarbrücken,1999.
G.Neumann.2000.AutomaticExtractionof StochasticLexicalizedTreeGram-marsfrom Treebanks.In Proceedingsof the4thworkshopontree-adjoininggrammarsandrelatedframeworks.Philadelphia,PA, USA, August,1998.
C. Pollard and I. Sag.1994.Head-DrivenPhraseStructure Grammar. TheUniversityof ChicagoPress,Chicago,Illinois.
G. Sampson,1993.TheSusanneCorpus. Release2.J. van Genabith,L. Sadler, and A. Way. 1999a.Data-driven compilationof
LFG semanticforms. In EACL’99 Workshopon Linguistically InterpretedCorpora (LINC-99),Bergen,Norway, June12th, pages69–76.
J. vanGenabith,L. Sadler, andA. Way. 1999b. StructurePreservingCF-PSGCompaction,LFG andTreebanks.In ProceedingsATALAWorkshop- Tree-banks, JourneesATALA, Corpusannotespour la syntaxe,UniversiteParis7, France,18-19Juin1999,pages107–114.
