+ All Categories
Home > Engineering > Multimedia Answer Generation for Community Question Answering

Multimedia Answer Generation for Community Question Answering

Date post: 17-Aug-2015
Category:
Upload: swami06
View: 23 times
Download: 5 times
Share this document with a friend
Popular Tags:
17
Multimedia Answer Generation for Community Question Answering
Transcript
Page 1: Multimedia Answer Generation for Community Question Answering

Multimedia Answer Generation for Community Question Answering

Page 2: Multimedia Answer Generation for Community Question Answering

Problem Statement

• Textual Answers• Multimedia Answers

Page 3: Multimedia Answer Generation for Community Question Answering

Literature SurveySr. No Author Year Contribution

1. Trec: Text Retrieval Conference[http://trec.nist.gov ]

1990 Text based QA

2. S.A. Quarteroni, S. Manandhar 2008 Text based QA based on type of questionsOpen Domain QA

3. D. Molla & J.L Vicedo 2007 Restricted Domain QA

4. H. Cui, M.Y. Kan 2007 Definitional QA

5. R.C.Wang, W.W. Cohen, E. Nyberg

2008 List QA

6. H. Yang, T S chua, S. Wang 2003 Video QA

7. J.Cao, Y-C- Wu, Y-S Lee 2004-2009

Video QA using OCR & ASR

8. Lot of authors 2003-2013

Content Based Retrieval

Page 4: Multimedia Answer Generation for Community Question Answering

System Decomposition

• Answer medium selection,• Query generation • Multimedia data selection and

presentation.

Page 5: Multimedia Answer Generation for Community Question Answering

Pre-requisites

• Datasets• Image & Video Mining API– Flickr, Picasaweb, Youtube, etc.

Page 6: Multimedia Answer Generation for Community Question Answering

Process Flow

Dataset collection

Classification(Conversational & Informational)

The answer medium selection and query selection components

Query Generation for Multimedia Retrieval

Multimedia Data Duplicate Removal

Result Re-ranking

Page 7: Multimedia Answer Generation for Community Question Answering

Answer Medium Selection

• Classification– only text,– Text + image– text + video– text + image + video

• Approach:– Question Based Classification– Answer Based Classification– Media Resource Analysis

Page 8: Multimedia Answer Generation for Community Question Answering

Query Extraction

For each QA pair, we generate three queries.

1. Convert the question to a query,

2. Identify several key concepts from verbose answer which will have the major impact on effectiveness.

3. Finally, we combine the two queries that are generated from the question and the answer respectively.

Page 9: Multimedia Answer Generation for Community Question Answering

Query Generation for Multimedia Search

• Query Extraction• The second step is query selection.

Page 10: Multimedia Answer Generation for Community Question Answering

Query Selection

• Three-class classification task, since we need to choose one from the three queries

• We adopt the following features:– POS Histogram.

• For the queries that contain a lot of complex verbs it will be difficult to retrieve meaningful multimedia results.

• We use POS tagger to assign part-of-speech to each word of both question and answer.

• Here we employ the Stanford Log-linear Part-Of-Speech Tagger and 36 POS are identified.

• We then generate a 36-dimensional histogram, in which each bin counts the number of words belonging to the corresponding category of part-of-speech.

Page 11: Multimedia Answer Generation for Community Question Answering

(2) Search performance prediction.– Clarity score for each query based on the KL divergence

between the query and collection language models. – We can generate 6-dimensional search performance

prediction features in all (there are three queries and search is performed on both image and video search engines).

• Therefore, for each QA pair, we can generate 42-dimensional features.

• Based on the extracted features, we train an SVM classifier with a labeled training set for classification

• i.e., selecting one from the three queries.

Page 12: Multimedia Answer Generation for Community Question Answering

Clarity function:

Page 13: Multimedia Answer Generation for Community Question Answering

Multimedia Data Selection & Prediction

• We perform search using the generated queries to collect image and video data with Google image and video search engines respectively.

• Most of the current commercial search engines are built upon text-based indexing and usually return a lot of irrelevant results.

• Therefore,– Re-ranking by exploring visual information is essential to

reorder the initial text-based search results. – Here we adopt the graph-based re-ranking method.

Page 14: Multimedia Answer Generation for Community Question Answering

Graph Based Re-Ranking & Duplicate Removal

Page 15: Multimedia Answer Generation for Community Question Answering

List of Algorithms• Core sentence extraction from question• Stemming & stop-words removal on answers• Question Type based on Answer Medium ( Naïve Bayes)• Head word extraction • Media Resource Analysis

– Clarity score based on KL Divergence• Query Generation• Query Selection

– POS Feature Extraction– Search Performance Prediction

• Multimedia Data selection & presentation– Graph based ranking– Face Detection Algorithms– Feature Extraction from images– Key frame identification & extraction

Page 16: Multimedia Answer Generation for Community Question Answering

References

1. M. Surdeanu, M. Ciaramita, and H. Zaragoza, “Learning to rank answers on large online QA collections,” in Proc. Association for Computational Linguistics, 2008

2. S. Cronen-Townsend, Y. Zhou, andW. B. Croft, “Predicting query performance,” in Proc. ACM Int. SIGIR Conf., 2002.

3. Liqiang Nie, Meng Wang, Yue Gao, Zheng-Jun Zha, and Tat-Seng Chua “Beyond Text QA: Multimedia Answer Generation by Harvesting Web Information” IEEE Multimedia Transaction 2013

Page 17: Multimedia Answer Generation for Community Question Answering

Thank You….


Recommended