EXCITEMENT: EXploring Customer
Interactions through Textual EntailMENT
Short project introduction
FP7-ICT-2011-7 (Objective 4.2b, STREP)
Partners
Academic Partners:
Bar-Ilan University, Ramat Gan, Israel (I. Dagan)
DFKI, Saarbrücken, Germany (G. Neumann)
Fondazione Bruno Kessler, Povo, Italy (B. Magnini)
University of Heidelberg, Germany (S. Pado)
Industrial Partners:
NICE, Ra'anana, Israel (English analytics
provider, coordinator)
German company (OMQ, German IT support company)
AlmaViva, Roma, Italy (Italian analytics provider)
EXCITEMENT: EXploring Customer Interactions through Textual EntailMENT
Genesis of the Proposal
Collaboration between BIU and NICE developed in
2009/10
Topic: Semantic inference
Smaller scale, national Israeli funding
Idea for a larger-scale project, early 2010
Consortium formed at ACL 2010 conference (July)
Weekly status calls; meeting in Brussels Nov 2010
Previous collaborations
BIU
NICE
DFKI
FB
K
HEI
ALM
A
OMQ
joint Israeli
project
joint work on entailment;
book chapter
joint work on EC
QALL-ME project
joint project
joint project
joint work on
entailment
Two goals
Scientific goal (Goal A) : Develop and advance a
“MOSES-style” open platform for multi-lingual
textual inference
Industrial goal (Goal B) : Inference-based
processing of customer interactions across
languages and interaction channels
Textual
Inference
Customer
Interactions
Open Platform for
Multi-lingual TI
Inference-based
Interaction
Analysis
Goal A: Multi-lingual Textual Inference
Identification of semantic inference relations between texts Question: Who acquired Overture?
Text: Yahoo’s buyout of Overture…
Anwer: Who = Yahoo
Framework: Textual inference No analysis to specific linguistic formalism
Instead: Establish mapping between texts at textual level Good mapping: valid inference; bad mapping: invalid inference
Free choice of algorithms, mapping levels, … Robustness, efficiency
Textual Inference: Subgoals (1/3)
Challenges to address:
Majority of work for English – reusability for other
languages problematic
No sharing of algorithms or resources across individual
systems
Wheel must be “reinvented” for each new study
Goal A1: A Generic Multilingual Architecture
Develop modular APIs for inference platforms
Language-independent specifications of
modules, algorithms, resources
Language-independent preprocessing
Generic Entailment Platform
EXCITEMENT: EXploring Customer Interactions through Textual EntailMENT
Textual Inference: Subgoals (2/3)
Challenges to address:
Large-scale knowledge about valid inference patterns only
available for English
Knowledge is never complete
Methods for approximate matching necessary
Goal A2: Algorithmic Progress in Textual
Inference
Multi-lingual and cross-lingual knowledge induction
methods
Frameworks for probabilistic matching
Textual Inference: Subgoals (3/3)
Challenges to address:
NLP application builders interested in inference must
integrate specialized external systems or develop own
system
High threshold
No practical re-use of existing
modules, algorithms, resources
Goal A3: An open-source multi-lingual textual
inference platform
Open-source reference implementation of a textual
inference platform for practical exploitation of Goal A1
Similar to MOSES in MT community
Fostering of an open-source community around platform
Goal B: Customer Interaction Analytics
Customer interaction analytics is large and growing
Multiple channels: Call centers, email contacts, web
forums
Substantial added value for companies:
(dis)advantages of products, typical customer
problems, typical problems in customer handling, …
Analysts must be supported by automatic systems
that represent interactions in a compact, expressive
form
State-of-the-art: Keyword-based analysis
Our proposal: Replace by textual inferenceEXCITEMENT: EXploring Customer Interactions through Textual EntailMENT
Interaction Analytics: Subgoals (1/3)
Goal B1: Interaction exploration and processing
schemes at the statement level
Development of concrete algorithms for phrasing
interaction analytics queries in terms of entailment
questions
Focus: Statement level (as opposed to keyword level)
Use case 1: Creation of entailment graphs
Entailment for visualization of customer information
Use case 2: Searching customer information
Entailment for query expansion
EXCITEMENT: EXploring Customer Interactions through Textual EntailMENT
Use case 1: Creation of entailment graphs
Use case 2: Entailment-driven query expansion
Interaction Analytics: Subgoals (2/3)
A priori, inference technology groups statements
only according to their inference relations
May not correspond exactly to analysts’ needs
Goal B2: Integrating textual inference and
domain knowledge
Examples: Predefined categories, process ontologies
Develop methods to integrate such extralinguistic
knowledge into the textual inference computation process
EXCITEMENT: EXploring Customer Interactions through Textual EntailMENT
Interaction Analytics: Subgoals (3/3)
Goal B3: Implemented industrial application
Harvesting benefits of goals B1 and B2 in industrial
context
Identification of sufficiently robust and efficient entailment
modules for practical deployment
Integration of entailment platforms into existing industrial
systems
On-field evaluation for different language-channel
combinations
EXCITEMENT: EXploring Customer Interactions through Textual EntailMENT
Language-Channel Matrix
English German Italian
Email NICE OMQ
Speech NICE ALMA
Social
Media
HEI ALMA
EXCITEMENT: EXploring Customer Interactions through Textual EntailMENT
English German Italian
Email OMQ
Speech NICE
Social
Media
ALMA
Evaluation on Domain Benchmarks (Open Platform)
On-Field Evaluation (Industrial Systems)
Work program sketch WP 1. User requirements (NICE) M 1-27
WP 2. Multilingual data collection (ALMA) M 1-27
WP 3. Specification of architecture (HEI) M 1-27
WP 4. TE Open Platform (FBK) M 1-36
WP 5. Multilingual knowledge acquisition (BIU) M 1-36
WP 6. Textual Inference components development (DFKI) M 6-36
WP 7. Text exploration: integration and evaluation (OMQ) M 12-36
WP 8. Open platform evaluation and distribution (BIU) M 10-36
WP 9. Dissemination and exploitation (FBK) M 1-36
WP 10. Scientific Coordination (FBK) M 1-36
WP 11. Management (NICE) M 1-36
EXCITEMENT: EXploring Customer Interactions through Textual EntailMENT
Pert diagram
Development cycles
Cycle 1: Month 1-12 First cycle through all WPs
End result: Two workable products First release of open source platform
Cycle 2: Month 12/24 Second cycle through all WPs
End result: Two robust, feature-complete products Second release of open source platform
Second release of industrial applications
Cycle 3: Month 24/36 Third cycle through all WPs
End result: Two robust, feature-complete products Second release of open source platform
Second release of industrial applications
Milestones
EXCITEMENT: EXploring Customer Interactions through Textual EntailMENT
MS1 (month 12): First Textual Entailment Platform
(internal use) – Cycle I. The milestone is planned in
correspondence with the first year review meeting.
MS2 (month 18): Open platform - first release. This
milestone aims at showing that the functionalities of
the first version of the Textual Entailment Open
Platform, developed in WP4 and WP5, are
successfully achieved, and that the open source
distribution has properly started.
Milestones
EXCITEMENT: EXploring Customer Interactions through Textual EntailMENT
MS3 (month 24): Open platform and text exploration applications –
Cycle II. This milestone will show the progresses in the distribution of
the open platform (WP8) as well as the text exploration applications
at the end of the second development/evaluation cycle.
Correspondence with both the first EXCITEMENT workshop and the
expected second year review meeting.
MS4 (month 36): Open platform and text exploration applications –
Cycle III. This milestone aims at showing the final results of the
project at the end of the third development/evaluation cycle, both in
terms of the open platform (WP8) and of the impact of the text
exploration components on the companies applications (WP7). final
EXCITEMENT workshop and the expected third year review
meeting.