Download - Multimedia Answer Generation for Community Question Answering

7/18/2019 Multimedia Answer Generation for Community Question Answering

http://slidepdf.com/reader/full/multimedia-answer-generation-for-community-question-answering-56d4ed306f291 1/17

Multimedia Answer Generation for

Community Question Answering



Problem Statement

• Textual Answers

• Multimedia Answers



Literature Survey

Sr. No Author Year Contribution

1. Tre! Text "etrievalConferene#$tt%!&&tre.nist.gov '

1(() Text based QA

*. S.A. Quarteroni+ S.Manand$ar

*)), Text based QA based onty%e of -uestions%en /omain QA

0. /. Molla 2.L 3iedo *))4 "estrited /omain QA

5. 6. Cui+ M.7. 8an *))4 /e9nitional QA

:. ".C.;ang+ ;.;. Co$en+<. =yberg

*)), List QA

>. 6. 7ang+ T S $ua+ S.;ang

*))0 3ideo QA

4. 2.Cao+ 7?C? ;u+ 7?S Lee *))5?

*))(

3ideo QA using C"

AS"

?



System /eom%osition

• Answer medium seletion+

• Query generation

• Multimedia data seletion and%resentation.



Pre?re-uisites

• /atasets

• mage 3ideo Mining AP – Blir+ Piasaweb+ 7outube+ et.



Proess Blow

/atasetolletion

Classi9ationDConversational

nformationalE

T$e answer mediumseletion and -uery

seletion om%onents

Query Generation forMultimedia "etrieval

Multimedia /ata

/u%liate "emoval

"esult "e?raning



Answer Medium Seletion

• Classi9ation – only text+

– Text F image

– text F video

– text F image F video

• A%%roa$! – Question @ased Classi9ation – Answer @ased Classi9ation

– Media "esoure Analysis



Query <xtration

For each QA pair, we generate threequeries.

1. Convert t$e -uestion to a -uery+

*. dentify several ey one%ts from verbose

answer w$i$ will $ave t$e maor im%at oneHetiveness.

0. Binally+ we ombine t$e two -ueries t$at are

generated from t$e -uestion and t$e answerres%etively.



Query Generation for Multimedia

Sear$

• Query <xtration

• T$e seond ste% is -uery seletion.



Query Seletion

• T$ree?lass lassi9ation tas+ sine we need to$oose one from t$e t$ree -ueries

• ;e ado%t t$e following features!

–

PS 6istogram.• Bor t$e -ueries t$at ontain a lot of om%lex verbs it will

be diIult to retrieve meaningful multimedia results.

• ;e use PS tagger to assign %art?of?s%ee$ to ea$ wordof bot$ -uestion and answer.

• 6ere we em%loy t$e Stanford Log?linear Part?f?S%ee$ Tagger and 0> PS are identi9ed.

• ;e t$en generate a 0>?dimensional $istogram+ in w$i$ea$ bin ounts t$e number of words belonging to t$eorres%onding ategory of %art?of?s%ee$.



D*E Sear$ %erformane %redition.

– Clarity sore for ea$ -uery based on t$e 8L divergene between t$e -uery and olletion language models.

– ;e an generate >?dimensional sear$ %erformane%redition features in all Dt$ere are t$ree -ueries andsear$ is %erformed on bot$ image and video sear$enginesE.

• T$erefore+ for ea$ QA %air+ we an generate 5*?dimensional features.

• @ased on t$e extrated features+ we train an S3M

lassi9er wit$ a labeled training set for lassi9ation

• i.e.+ seleting one from t$e t$ree -ueries.



Clarity funtion!



Multimedia /ata Seletion

Predition

• ;e %erform sear$ using t$e generated -ueriesto ollet image and video data wit$ Googleimage and video sear$ engines res%etively.

• Most of t$e urrent ommerial sear$ enginesare built u%on text?based indexing and usuallyreturn a lot of irrelevant results.

• T$erefore+ – "e?raning by ex%loring visual information is essential

to reorder t$e initial text?based sear$ results.

– 6ere we ado%t t$e gra%$?based re?raning met$od.



Gra%$ @ased "e?"aning

/u%liate "emoval



List of Algorit$ms

• Core sentene extration from -uestion

• Stemming sto%?words removal on answers

• Question Ty%e based on Answer Medium D =aJve @ayesE

• 6ead word extration

• Media "esoure Analysis – Clarity sore based on 8L /ivergene

• Query Generation

• Query Seletion – PS Beature <xtration

– Sear$ Performane Predition

•Multimedia /ata seletion %resentation – Gra%$ based raning

– Bae /etetion Algorit$ms

– Beature <xtration from images

– 8ey frame identi9ation extration



"eferenes

1. M. Surdeanu+ M. Ciaramita+ and 6. Karagoa+Learning to ran answers on large online QAolletions+N in Proc. Association forComputational Linguistics+ *)),

*. S. Cronen?Townsend+ 7. K$ou+ and;. @. Croft+Prediting -uery %erformane+N in Proc. ACM Int.SIGIR Conf.+ *))*.

0. Li-iang =ie+ Meng ;ang+ 7ue Gao+ K$eng?2un K$a+

and Tat?Seng C$ua @eyond Text QA! MultimediaAnswer Generation by 6arvesting ;ebnformationN <<< Multimedia Transation *)10



T$an 7ouO.