Relational Entity Linking with Cross Document Coreference

transcript

Xiao Cheng, Bingling Chen, Rajhans Samdani,Kai-Wei Chang, Zhiye Fei and Dan Roth

University of Illinois at Urbana-Champaign (UI_CCG)

Talk Outline

Introduction Architecture

Entity Linking Approach Preprocessing Wikification

Formulation Relational Analysis

Cross Document Coreference Reconciliation

Evaluation

Entity Linking Specification

Output

Wikipedia

Arti-cles

500000

1000000

1500000

2000000

2500000

3000000

3500000

4000000

4500000

5000000

TAC KBNon-TAC KB

<query id="EL13_ENG_0015"> <docid>bolt-eng-DF-170-181137-9030298</docid> <name>Lightning Bolts</name> <beg>15959</beg> <end>15973</end></query>

query_id link_idEL13_ENG_0015 NIL0006EL13_ENG_0016 E0273299…EL13_ENG_0821 NIL0006

Entity Linking using Wikification and Cross-Doc Coref

Wikipedia

Arti-cles

500000

1000000

1500000

2000000

2500000

3000000

3500000

4000000

4500000

5000000

TAC KBNon-TAC KB

query_id link_idEL13_ENG_0015

NIL0006

EL13_ENG_0016

E0273299

…EL13_ENG_0821

NIL0006

…EL13_ENG_0937

NIL0288

…EL13_ENG_1914

NIL0288

Cross Document

Coreference

Wikification

Blumenthal (D) is a candidate for the U.S. Senate seat now held by Christopher Dodd (D), and he has held a commanding lead in the race since he entered it. But the Times report has the potential to fundamentally reshape the contest in the Nutmeg State.

Ambiguity

Concepts outside of KB (NIL) Blumenthal ?

Variability

Scale Millions of labels

Wikification Challenges

ConnecticutCT

The Nutmeg StateTimesThe New York TimesThe Times

Key Innovation

Improved Wikification for Structured EL Relational Inference for Linking (Cheng and Roth, EMNLP’13) No retraining

Non-trivial cross-document clustering Best Latent Left-Linking approach (Samdani et al. ’12)

Talk Outline

Cross Document Coreference Evaluation

Entity Linking Architecture

Linking

Wikification

Cross-Doc Coreference

Supervise

LinkingProblem

TAC Query

Preprocessing

Query Normalization Document Transformation

Purposeful Coreference

ReconcileLinking Clusters

Query Mapping

Wikipedia Entities

TAC KB Entities

Partial NIL Clusters

Talk Outline

Cross Document Coreference Evaluation

Preprocessing

Query normalization Handling spelling mistakes and slangs – one of the reasons we did not

achieve expected performance

In document coreference – some coreferent mentions are easier to link than the query mention

Obomber, Obamadinejad, Osama Obama, Nobama, Obambi, Obamination, ObaMao, Owe Bama, 0bama, O-balm-a, O-bomb-a

Preprocessing

Document transformation Document can be as long as 100k characters for a single query Need to truncate documents but minimize the loss of critical contexts

Original

Opening

Query Context

Coreferent Context

Talk Outline

Evaluation

State-of-the-art Wikification systems (Ratinov et al. 2011) can achieve the above with local and global statistical features Reaches bottleneck around 70%~ 85% F1 on non-wiki datasets What is missing?

Wikification Bottleneck

, the of deposed , …

Motivating Example

Mubarak wife Egyptian President Hosni Mubarak

What are we missing with Bag of Words (BOW) models? Who is Mubarak?

Constraining interaction between concepts (Mubarak, wife, Hosni Mubarak)

Mubarak, the wife of deposed Egyptian President Hosni Mubarak, …

Relational Inference for Wikification

Our contribution Identify key textual relations for Wikification A global inference framework to incorporate relational knowledge

Significant improvement over state-of-the-art Wikification systems

(Mubarak, wife, Hosni Mubarak)

Mention Segmentation

Candidate Generation

Candidate Ranking

NIL Linking

Traditional Wikification Pipeline

Mention Segmentatio

Candidate Generation

Candidate Ranking

Determine NILs

Traditional Wikification 1 - Mention Segmentation

...ousted long time Yugoslav President Slobodan Milošević in October. Mr. Milošević's Socialist Party…

Sub noun phrase chunks NER Capitalized phrases

Traditional Wikification 1 - Mention Segmentation

Obtains nested mentions

Traditional Wikification 2 - Candidate Generation

1 Socialist_Party_(France)

2 Socialist_Party_(Portugal)

3 Socialist_Party_of_America

4 Socialist_Party_(Argentina)

1 Slobodan_Milošević

2 Milošević_(surname)

3 Boki_Milošević

4 Alexander_Milošević

Approach Collect known mappings from Wikipedia page titles, hyperlinks… Limit to top-K candidates based on frequency of links (Ratinov et al.

Traditional Wikification 3 - Candidate Ranking

k ek4 sk

1 Socialist_Party_(France) 0.23

2 Socialist_Party_(Portugal) 0.16

3 Socialist_Party_of_America 0.07

4 Socialist_Party_(Argentina) 0.06

k ek3 sk

1 Slobodan_Milošević 0.7

2 Milošević_(surname) 0.1

3 Boki_Milošević 0.1

4 Alexander_Milošević 0.05

Local and global statistical features

Traditional Wikification 4 – Determine NILs

k ek4 sk

k ek3 sk

Is the top candidate really what the text referred to? Binary classifier

This answer is wrong We did not generate the

correct candidate based on top-K prior

Talk Outline

Evaluation

Formulation (0)

Intuition Promote pairs of candidate concepts coherent with textual relations

(Mubarak, wife, Hosni Mubarak)

Formulate as an Integer Linear Program (ILP):

If no relation exists, collapse to the unstructured decision

Formulation (1)

Whether a relation exists between and

weight of a relation

Whether to output th candidate of the th mentionweight to output

Formulation (2)

k ek4 sk

k ek3 sk

r(1,2)34

eki: whether a concept is chosen

ski : score of a concept

r(k,l)ij: whether a relation is present

w(k,l)ij : score of a relation

r(4,3)34

Talk Outline

Evaluation

Overall Approach

Relational WikificationCandidate Generation

Candidate Ranking

Determine NILs

Relation Analysis

Relation Identification

Relation Retrieval

Relational Inference

ACE style in-document coreference (Chang et al. ‘13) Extract named entity-only coreference relations with high precision

Syntactico-Semantic relations (Chan & Roth ‘10)

Easy to extract with high precision Aim for high recall, as false-positives will be filtered Sparse, but covers ~80% relation instances in ACE2004

Type Example

Premodifier Iranian Ministry of Defense

Possessive NYC’s stock exchange

Formulaic Chicago, Illinois

Preposition President of the US

Argument 1 Relation Type Argument 2

Yugoslav President apposition Slobodan Milošević

Slobodan Milošević coreference Milošević

Milošević possessive Socialist Party

Overall Approach

Candidate Ranking

Determine NILs

Relation Analysis

Relation Retrieval

What concepts can “Socialist Party” refer to? More robust candidate generation

Identified relations are verified against a knowledge base (DBPedia)

Query Pruning Only 2 queries per pair necessary due to strong baseline.

Relation Retrieval

q1=(Socialist Party of France,?, *Milošević*)q2=(Slobodan Milošević,?,*Socialist Party*)

k ek4 sk

k ek3 sk

Relation Retrieval

Argument 1 Relation Type Argument 2

Milošević possessive Socialist Party

Relation Retrieval

k ek4 sk

21 Socialist_Party_of_Serbia 0.0

k ek3 sk

𝑟34(1,21)=1

Overall Approach

Candidate Ranking

Determine NILs

Relation Analysis

Relation Retrieval

Relational Inference - coreference

k ek2 sk

k ek3 sk

𝑟23(1,1)=1

Determine unknown concepts (NILs)

How to capture the fact: “Dorothy Byrne” does not refer to any concept in Wikipedia

Identify coreferent nominal mention relations Generate better features for NIL classifier

Dorothy Byrne, a state coordinator for the Florida Green Party,…

k ek2 sk

1 Green_Party_of_Florida 1.0

k ek1 sk

1 Dorothy_Byrne_(British_Journalist)

2 Dorothy_Byrne_(mezzo-soprano)

nominal mention

Determine unknown concepts (NILs)

Create NIL candidate for structured inference e.g. corrects other coreferent “Dorothy” later in the document

Dorothy Byrne, a state coordinator for the Florida Green Party,…

k ek2 sk

1 Green_Party_of_Florida 1.0

k ek1 sk

0 NIL 1.0

1 Dorothy_Byrne_(British_Journalist)

2 Dorothy_Byrne_(mezzo-soprano)

nominal mention

Talk Outline

Evaluation

Cross Document Coreference

NILs can be viewed as KB entries with partial information A uniform model for entity representation Shared features with Entity Linking system Can be supervised using existing EL systems

Cross document coreference cluster example:Naomi Campbell to give evidence at Charles Taylor trial: spokeswoman.

Supermodel Campbell says 'nothing to gain' from Taylor trial testimony.

Cross Document Coreference Approach

Run document-level coreference Aggregate all features in a document-level coreferent cluster Use both mention-level features and document-level features

String similarity features (NESim, Do et al. ‘09) Context TF-IDF similarity features Document-level cluster features

Training: using both TAC data and Wikifier generated data

100%75%

50%25%

00.10.20.30.40.50.60.70.80.9

Clustering B³ F1 on TAC Entity Linking 2012

Trivial L³M No str feat. L³MAllLink SumLink

100%75%

50%25%

00.10.20.30.40.50.60.70.80.9

Clustering B³ F1 on Wikifier generated data

Trivial L³M No str feat. L³MAllLink SumLink

Talk Outline

Evaluation

Query mapping reconciliation

• Max• {0.8,0.7,0.2} = Seattle Seahawks

• Sum• {0.8,0.7+0.2} = Seattle

• No Threshold• NIL classifier always outputs “non-NIL”• Same as Max otherwise

Seattle (0.7)

Seattle Seahawks (0.8)

Seattle(0.2)

[Seattle] has won…

[Seattle] Seahawks ended the game…

… cheered for [Seattle]…

Talk Outline

Evaluation

Evaluation – TAC KBP 2011 Entity Linking

Run Relational Inference (RI) Wikifier “as-is”: No retraining using TAC data

CogComp

MS_MLI

NUShim

*Median

TAC KBP 2011 Entity Linking Performance

Micro AverageB³F1

System Names*Median of top 14 systems

Evaluation – TAC 2012 Entity Linking Error Analysis

Max Sum Perfect Ranking Exact Max0

B³3 F1

Official 2013 Performance

Official 2013 Performance Break-down: Link Type

Official 2013 Performance Break-down: Doc domain

Official 2013 Performance Break-down: NER type

Conclusion

Importance of linguistic and world knowledge Identification of relational information benefits Wikification

and Entity Linking Future work

Robust preprocessing on noisy input/adapt to EL task requirement “Self-supervision” on NIL clustering Unified NIL and KB entity representation Joint entity typing, coreference and disambiguation Incorporate more relations

Demo: http://cogcomp.cs.illinois.edu/demo/wikify Download: http://cogcomp.cs.illinois.edu/page/download_view/Wikifier

Thank you!

BACK UP SLIDESBack up slides

Applications

Knowledge Acquisition via Grounding Coreference Resolution

Learning-based multi-sieve co-reference resolution with knowledge (Ratinov et al. 2012)

Information Extraction Unsupervised relation discovery with sense disambiguation (Yao et al.

2012) Automatic Event Extraction with Structured Preference Modeling (Lu

and Roth, 2012 ) Text Classification

Gabrilovich and Markovitch, 2007; Chang et al., 2008

Wikification Performance Result

ACE MSNBC AQUAINT Wikipedia60

F1 Performance on Wikification datasets

Milne&WittenRatinov&RothRelational Inference

Relational Entity Linking with Cross Document Coreference

Documents