D5.7 –Robot Semantic Sentiment Analysis - mario-project.eu · Managing Active & Healthy Ageing...

ManagingActive&HealthyAgeingwithServiceRobots

D5.7– RobotSemanticSentimentAnalysis

ProjectAcronym: MARIO

ProjectTitle: Managingactiveandhealthyagingwithuseofcaringservicerobots

ProjectNumber: 643808

Call: H2020-PHC-2014-single-stage

Topic: PHC-19-2014

TypeofAction: RIA

Ref. Ares(2017)3839883 - 31/07/2017


D5.7– RobotSemanticSentimentAnalysis_FinalversionWorkPackage WP5

DueDate M30

SubmissionDate 31/07/2017

StartDateoftheProject 01/02/2015

DurationoftheProject 36Months

Organisation ResponsibleofDeliverable CNR

Version 2.1

Status Final

Authorname(s) ValentinaPresutti (CNR)AldoGangemi (CNR)DiegoReforgiato Recupero (R2M)Domenico Pisanelli (CNR)PaoloCiancarini (CNR)AndreaGiovanniNuzzolese (CNR)LuigiAsprino (CNR)AlessandroRusso(CNR)

Reviewer(s) ChristosKouroupetroglou (CNET),André Freitas(R2M)

Nature ☐ R– Report☐ P– Prototype☐ D– Demonstrator☒ O– Other

DisseminationLevel ☒ PU– Public☐ CO– Confidential,onlyformembersoftheConsortium(includingtheCommission)☐ RE– RestrictedtoagroupspecifiedbytheConsortium(includingtheCommissionServices)

2


RevisionHistoryVersion Date Modifiedby Comments

0.1 16/06/2017 ValentinaPresutti (CNR) Firstdraft startingfromD5.3

0.2 19/06/2017 AlessandroRusso(CNR) Revisionandchangesonthedocumentstructure

0.3 22/06/2017 AlessandroRusso(CNR) Addedcontentonpolaritydetection

0.4 26/06/2017 AndreaGiovanniNuzzolese (CNR) Revisionandextensionofsemanticsentimentanalysiscontent

0.4.1 27/06/2017 AndreaGiovanniNuzzolese (CNR) Unifiednotationforfigureswithknowledgegraphs

0.5 05/07/2017 AlessandroRusso(CNR) Initialcontentonusecasescenario

0.5.1 06/07/2017 AlessandroRusso(CNR) ContentandfiguresforMyMemoriesscenario

0.5.2 10/07/2017 AndreaNuzzolese (CNR) Addedframe-basedexampleinusecasescenario

0.6 13/07/2017 AlessandroRusso(CNR) Revisionofdocumentoutlineandoverviewsections

1.0 16/07/2017 AlessandroRusso(CNR)Overallrevisionandminoreditstoimprovereadability;draftversionforinternalreview

continues

3


RevisionHistory(cont.)Version Date Modifiedby Comments

1.0 19/07/2017 AdamSantorelli (NUIG) Commentsandfeedbackonv1.0

1.0 21/07/2017 ChristosKouroupetroglou (CNET)AndréFreitas(PASSAU) Reviewer'sfeedbackonv1.0

1.1 25/07/2017 AlessandroRusso(CNR) Revisionandupdatesaccordingtoreviewer'sfeedback

2.0 27/07/2017 AlessandroRusso,LuigiAsprino,AndreaGiovanniNuzzolese (CNR)

Revisionoftheoveralldocumentwrtreviewers’comments;addedabbreviationslist;minorformattingedits;versionforcoordinator

2.0 28/07/2017 AislingDolan(NUIG)AlexandrosGkiokas (ORTELIO) Commentsandfeedbackonv2.0

2.1 28/07/2017 AlessandroRusso(CNR) Revisionwithadditionaledits;finalversion

4


Copyright © 2016, MARIO Consortium

The MARIO Consortium (http://www.mario-project.eu/) grants third partiesthe right to use and distribute all or parts of this document, provided that theMARIO project and the document are properly referenced.

THIS DOCUMENT IS PROVIDED BY THE COPYRIGHT HOLDERS ANDCONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OFMERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE AREDISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER ORCONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITEDTO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA,OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANYTHEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THEUSE OF THIS DOCUMENT, EVEN IF ADVISED OF THE POSSIBILITY OF SUCHDAMAGE.

5


ExecutiveSummary

Deliverable 5.7 describes the software components that were designed anddeveloped in the context of Task 5.3 (WP5) to provide the MARIO systemarchitecture and software framework with Sentiment Analysis capabilities.This version represents the final, consolidated version of Deliverable 5.3.

Specifically, the components presented here constitute MARIO’s SentimentAnalysis subsystem. The main component aims at providing semanticsentiment analysis capabilities, through a formal representation andevaluation of the sentiment expressed in text sentences. It is built as anextension of FRED (a machine reading component introduced in Deliverable5.6) and relies on novel resources developed in the context of this task.

6


TableofContentsExecutiveSummary 6ListofAcronymsandAbbreviations 8Introduction 9

WorkPackage5objectives 10Purposeandtargetgroupofthedeliverable 11Relationstootheractivitiesintheproject 12DocumentOutline 13AboutMARIO 14

OverviewonSentimentAnalysis 15MARIOSentimentAnalysisSubsystem 20

Polaritydetection 25Semanticsentimentanalysis 29

Opinionmodelontology 33Sentimentlexicalresources 39Frame-basedsentimentanalysis 42

RepresentativeUseCaseScenario 53SentimentanalysisintheMyMemoriesapp 54

Concludingremarks 67References 69

7


AcronymsandAbbreviations

8

API ApplicationProgrammingInterface

CGA ComprehensiveGeriatric Assessment

JSON JavaScriptObjectNotation

KB KnowledgeBase

MPI Multidimentional PrognosticIndex

MON MARIOOntologyNetwork

NLP NaturalLanguage Processing

NLU NaturalLanguage Understanding

OWL WebOntologyLanguage

PWD Person/PeoplewithDementia

RDF ResourceDescriptionFramework

REST Representationalstatetransfer

S2T SpeechtoText


Introduction

Naturallanguageistheprimarymeansthroughwhichpeoplewithdementia(PWD)caninteractwithMARIOrobots.Task5.3dealswithMARIO’scapabilitytoextractsentimentinformationfromnaturallanguagesentencesexpressedbyapersonwithdementia(PWD).TheabilitytoautomaticallyextractandcategorisesentimentinformationfromthetextualrepresentationofnaturallanguagesentencesenablesMARIOtoadjustandadaptitsbehaviouraccordingtothesentimentoropinionpotentiallyexpressedbyPWD.

Thisdeliverablepresentstheapproaches,algorithmsandsoftwarecomponentsthatenableMARIOtoperformsentimentanalysisoverthetextualrepresentationofspokennaturallanguageinput.Inparticular,MARIOisequippedwithsemanticsentimentanalysiscapabilities,whichexploitaframe-baseformalrepresentationofthetextualinputbyaugmentingtheNaturalLanguageProcessing(NLP)capabilitiesdiscussedinDeliverable5.6.

9


WorkPackage5Objectives

WP5aimsatdevelopingtheframeworkandtoolsthatallowMARIOrobotstointeractwithhumansandunderstandtheirneedsexpressedthroughspokennaturallanguage.Asfundamentalbuildingblocks,understandingcapabilitiesexploitmachinereading/listeningcomponentsandRDF/OWLontologiestofirstproduceandthenprocessaformalencodingofthetextualrepresentationofnaturallanguage.Assuch,themainobjectivesofWP5are:• todesignanddeveloptheMarioOntologyNetwork(MON)andKnowledgeBase;

• toprovideMARIOwiththeabilityoftransformingnaturallanguageintoaformalrepresentation,toenablereadingandlisteningcapabilitiesonthebasisofFRED;

• toprovideMARIOwiththecapabilityofrecognising,storingandreusingsentimentinformation,onthebasisofsemanticsentimentanalysistechniques.

10


PurposeandTargetGroupoftheDeliverable

ThisdeliverableaimsatdescribingthesoftwarecomponentsdesignedanddevelopedinTask5.3.Thesecomponentsprovidetherobotwithsentimentanalysiscapabilitiescomplementingtheabilitytoprocessandunderstandspokennaturallanguage.Theroleofsentimentanalysisinhuman-robotinteraction,withafocusonPWD,isconsidered.

Duetoitstechnicalnature,thedeliverableismainlytargetingresearches,practitionersanddevelopersinterestedinsentimentanalysistechniquesandalgorithms,andinparticularsemanticframe-basedtechniquesandtheirapplicationforservicecompanionrobots.Inadditiontothetechnicalaspects,concreteusecasesareconsidered,withtheaimofprovidinghealthexpertswithanunderstandingonhowthesetechniquescanimprovetheinteractionbetweenPWDandcompanionrobots.

11


RelationstootherActivitiesintheProject

ThisdeliverabledirectlyreliesontheresultsofTask5.1(seeD5.1)asfarasthebackgroundknowledgeandknowledgemodelsthatitusesareconcerned(aspartoftheMON– MARIOOntologyNetwork),anditisstronglyrelatedtotheactivitiescarriedoutinTask5.2concerningNaturalLanguageUnderstanding(seeD5.6).

TheoverallWP5receivesasinputtheuserandfunctionalrequirementsandthesystemarchitecturefromWP1,whileWP2providestheKompai robotandplatformwherethesoftwarecomponentsaredeployed.ThesecomponentsprovidesentimentanalysisservicestotheapplicationsandmodulesdevelopedinWPs3,4and6withspecificfocusoncapturingandrepresentingsentimentknowledge,inlinewiththeintegrationproceduresdefinedinWP7.

ValidationactivitiesinWP8providefeedbacktotheiterativedesignanddevelopmentprocessofthesoftwarecomponents,andcontributetotheirevolutionandrefinement.

12


DocumentOutline

Therestofthisdeliverableisorganisedinthreemainsections.Specifically:

• thefirstsectionprovidesanoverviewofsentimentanalysisanditsroleinsupportingtheinteractionwithPWD;

• thesecondsectionprovidesadescriptionofMARIO’sSentimentAnalysissubsystem,withafocusonthemodels,resourcesandalgorithmsthatsupportandenablesemanticsentimentanalysiscapabilities;

• thethirdsectionpresentsarepresentativeusecasescenariosfortheSentimentAnalysissubsystem,byoutlininghowitscapabilitiesareusedinthecontextoftheMyMemoriesapplicationforreminiscence.

13


AboutMARIO

MARIO addresses the difficult challenges of loneliness, isolation and dementia in olderpersons through innovative and multi-faceted inventions delivered by service robots.The effects of these conditions are severe and life-limiting. They burden individualsand societal support systems. Human intervention is costly but the severity can beprevented and/or mitigated by simple changes in self-perception and brain stimulationmediated by robots.

From this unique combination, clear advances are made in the use of semantic dataanalytics, personal interaction, and unique applications tailored to better connectolder persons to their care providers, community, own social circle and also to theirpersonal interests. Each objective is developed with a focus on loneliness, isolationand dementia. The impact centres on deep progress toward EU scientific and marketleadership in service robots and a user driven solution for this major societalchallenge. The competitive advantage is the ability to treat tough challengesappropriately. In addition, a clear path has been developed on how to bring MARIOsolutions to the end users through market deployment.

14


SentimentAnalysis


SentimentandEmotionswhenInteractingwithPWD

Theroleandimportanceofrecognisingthesentiment oremotionalstate ofPWDwheninteractingwiththemthroughaconversationalapproachislargelyrecognised.

IdentifytheemotionalstateoftheresponseHowisthispersonfeeling?Iftheyhavebeenabletospeak,whatdothewordsconvey?

http://www.scie.org.uk/dementia/after-diagnosis/communication/conversation.asp

Sometimestheemotionsbeingexpressedaremoreimportantthanwhatisbeingsaid.

http://www.alz.org/care/dementia-communication-tips.asp

16


SentimentAnalysis– Overview(1/2)

Sentimentanalysisconcernsthestudyofintelligentalgorithmscapableofautomaticallymining(i.e.,identifyingandcategorising)opinionsfromnaturallanguagecontent.

InthecontextofMARIO,weareinterestedinapplyingsentimentanalysistosentencesinnaturallanguagespokenbyPWD,especiallywhentheyareexpectedtoexpressfeedbackaboutsomerecentactivityortoconverseaboutevents,peopleandplacesthatcharacterisetheirlifehistoryandpastexperiences.

ThisinformationcanthenbestoredandusedinordertocontributetoapersonaliseduserexperienceandinfluenceMARIO’sdecisionmakingwheninteractingwiththeusers.

17



TheinitialworkingandresearchhypothesiswasbasedontheassumptionthatsentimentanalysiscapabilitiescouldbemainlyusedtosupporttheComprehensiveGeriatricAssessment(CGA)andMultidimensionalPrognosticIndex(MPI)roboticmodules(investigatedinWP4).

AsdetailedinDeliverables4.3and5.6,thehuman-robotinteractionprocessrequiredtosupportCGAhastobebasedonaconversationalapproachdrivenbystandardisedclinicalquestionnaires.

Sincetheinitialvalidationactivitiesandon-the-fieldobservationsrelatedtothedevelopmentoftheCGAmodule,itemergedthatPWD’srepliestothequestionsdefinedintheassessmentquestionnairesdonotcarrysentimentinformation (mostofthequestionsassumeayes/noanswerorarebasedonamultiple-choicestructure),preventingsentimentanalysistechniquestobeeffectivelyusedandcontributetotheMPI.

18



Asdetailedlaterinthisdeliverable,animmediateapplicationofsentimentanalysiscapabilitiesistotheMyMemories appforreminiscence(presentedinD3.3),whichenablesMARIOtointeractwithPWDtostimulateconversationandelicitmemories,bypromptingthemandaskingthemtoexpresstheirmood/feelings/opinion(e.g.,aboutpersonaleventsorpeople)afterlookingatapicturewithfewwordsorasentence,etc.

ThisinformationexpressedthroughspeechiscapturedbytheUnderstandingSubsystem andprocessedbytheSentimentAnalysismodule,soastoproducedataexpressingsentimentandemotionalinformationassociatedtospecificentitiesorevents.

AtalaterstageMARIOcanusethisinformationinordertodecidewhetherto,e.g.,showapictureagainortalkaboutaspecificpersonorpastevent,accordingtospecificgoalsandconversationalstrategiesitisimplementing.

19


MARIOSentimentAnalysisSubsystem

Components,servicesandcapabilities

ManagingActive&HealthyAgeingwithServiceRobots21

ArchitecturalReference

ReferringtoMARIOarchitecture,thistaskcontributestothe NaturalLanguageUnderstandingsubsystem(seeD5.6),byintroducingsentimentanalysismodulesandservices.


Approach andContributions

InordertoachievethegoalofthistaskandprovideMARIOwithmultilingualsentimentanalysiscapabilities,we:• adoptedamodular andservice-orienteddesign approach,tosupportmultipleapproacheshavingdifferentcapabilities;

• reusedanontologyforrepresentingopinions;• integratedsentimentlexicalresources intoMARIO’sbackgroundknowledge(relyingontheFramester resource[13,14],describedinDeliverable5.1);

• developedasentence-polarity evaluationmodule;• extendedandimplementedasemanticframe-basedsentimentanalysis algorithm.

22


SentimentAnalysisComponents

TheSentimentAnalysisSubsystemcomplements(andrelieson)thecapabilitiesprovidedbytheNaturalLanguageUnderstanding(NLU)Subsystem(describedinD5.6).

Itimplementstwomainstrategies,detailedinthenextslides,forcomputingsentiment andemotionalscores:1. simplesentimentpolarityanalysis

– itoperatesatasentencelevel,toidentifytheoverallpolarity (Negative-Neutral-Positive)ofagivensentence;

2. advancedsemanticsentimentanalysis– itoperatesonasemanticframe-basedrepresentation ofasentence,toidentifythe

sentimentexpressedbyanopinionholder onacertainentity ortopic;– itreliesontheMARIOKnowledgeBase(accessedviatheRESTAPIprovidedbyLizard;see

Deliverable5.1) andotherresourcesalreadyexploitedbytheNLUsubsystem,withtheadditionofsentimentlexicalresources;

– thiscapabilityisprovidedbyasoftwarecomponentaccessiblethroughaRESTservice1.

231 http://wit.istc.cnr.it/stlab-tools/sentilo/service


ComponentsOverview

24

• ThisfigurehighlightsthemaindependenciesandtheflowamongthecomponentsdevelopedinthistaskandothercomponentsintheMARIOarchitecture.

• TheSentimentAnalysissubsystemdirectlyinteractswithMARIOKnowledgeBaseviatheRESTAPIprovidedbyLizard (seeDeliverable5.1)forretrievingandstoringdata.

• Theothercomponentsaredevelopedinotherproject’stasksandWPs(T5.1,T5.2,WP3,WP4andWP6).


PolarityDetection


Sentence-basedPolarityDetection

Aimsatidentifyingtheoverall tonality/sentiment (reflectingspeaker’sattitude)expressedinasentence.• Itclassifiesagivensentence/textaccordingtoapolarityscale (typically,Negative-

Neutral-Positive),potentiallywitharankingscore orconfidencevalue.• Itmapstoaclassificationproblem,addressedwithMachineLearningtechniques;

– e.g.,NaïveBayes,MaximumEntropy(MaxEnt),SupportVectorMachines(SVM)orDeepNeuralNetworkclassifiers,exploitingsyntacticfeaturesextractedfromthesentences.

• ThepolarityclassificationstepisprecededbyNaturalLanguageProcessing(NLP)tasks:– basicNLPpipelineincludestokenization,sentencesplittingandsyntactic

parsing;– part-of-speech(POS)taggingandlemmatisationstepscanbeaddedtothe

pipeline.

26


PolarityDetectionModule

MARIO’spolaritydetectionmodule reliesontheSentimentAnalysiscapabilitiesoftheStanfordCoreNLP framework[1],which:• usesadeeplearningapproachwithRecursiveNeuralNetworks,trainedonthe

StanfordSentimentTreebankdataset[2];• isbasedonanunderlyingmodelthataimsatcapturingtheeffectsofcontrastive

conjunctionsaswellasnegation;• classifiesandscoresinputsentencesaccordingto5classesofsentiment:very

negative(-2),negative(-1),neutral(0),positive(1),andverypositive(2);• reliesonaNLPpre-processingpipelinewithtexttokenization,sentencesplitting

andsyntacticparsing.

27

Tokenization SentenceSplitting

SyntacticParsing

SentimentClassification

Polaritydetectionmodule

2verypositive

1positive

0neutral

-1negative

-2verynegative

inputtext

foreachsentence


PolarityDetection

Pros:+ pre-trainedalgorithmsandmodelsareavailableforthedifferentstepsof

theprocessingpipeline;+ goodforsentencesexpressingasingle,generalopinionorsentiment;+ simpletouseandinterprettheanalysisresults.

Limitations:- unabletoclearlydistinguishmultipleopinionsexpressedinasingle

sentenceoverdifferententities/topics• e.g.,thesentence“Thefoodwasgreat,buttheweatherwassobad!” isclassifiedasa

wholeasNegative,withnodistinctionaboutthedifferentopinionsexpressedaboutthefoodandtheweather

- doesnotexploitsemanticfeaturesoftheinput.

Beyondpolaritydetection:abilitytoidentifythesentiment expressedbyanopinionholder onacertainentity ortopic.

28


SemanticSentimentAnalysis


What’sanopinion

Anopinion canbedefinedasanintentionalstatementbysomebody(holder)onsomefact(topic)thatisexpressedwithapossiblesentiment.

ASentimentAnalysissystemshouldbeabletoextractandcharacteriseopinions byrecognisingtheattitude (positive,negativeorobjective)ofanopinionholder onacertaintopic,orbyevaluatingtheoveralltonality ofasentence.

30


Formaldefinition

ThegoalofSentimentAnalysisistodetectquintuples(ej,ajk,soijkl,hi,tl)fromunstructuredtext,whereanopinionisaquintuple[3,4]:

(ej,ajk,soijkl,hi,tl)

where:• ej isatargetentity;• ajk isanaspect/feature oftheentityej;• soijkl isthesentimentvalue oftheopinionfromopinionholderhi onaspectajk ofentityej attimetl.soijkl ispositive,negativeorneutral,orarating;

• hi isanopinionholder;• tl isthetime whentheopinionisexpressed.

31


SemanticsintoSentimentAnalysis

Traditionalapproacheshardlycopewithsubtlelinguisticforms,combinedandconcurrentpositive/negativeopinions,andimplicitjudgements.

Theliteratureshowsevidencethattheinclusionofsemanticfeatures insentimentanalysisalgorithmsimprovestheiroverallperformance,e.g.[5].

Linkeddata,ontologies,controlledvocabularies,andlexicalresourceshelpaggregatingtheconceptualandaffectiveinformationassociatedwithnaturallanguageopinions.

32


OpinionModelOntology


Opinionmodelontology (1/5)

Ouropinionmodel definesthemainconceptscharacterisinganopinion,andisusedforannotating thesemanticrepresentationofasentence,inordertoidentifyitsopinionholderandtopic.

34


Opinionmodelontology(2/5)

35

AsshowninthisFigure,wedefinethesemanticrepresentationofanopinionsentenceashavingtwoparts:(i) opiniontriggercontextand(ii) opinionatedcontext.

Theopiniontriggercontext isoptionalandidentifiesconceptsthatindicatethepresenceofanopinionexpressedinthesentence,anditsholder.Itiscomposedoftwoparts:theentitiesthatallowtheidentificationoftheopinionholders (i.e.holders),andtheeventse.g.,think,say,support,etc.,thatactastriggersofanopinion(i.e.opiniontriggers).

Theopinionatedcontextidentifiesconceptsthatexpressanopinion(possiblyincludingsentiments).Itiscomposedoftwoparts:theentitiesthatidentifytheopinion topics (i.e.topics),andthoseexpressingtheopinionanditspossibleassociatedsentiments(i.e.,theopinionfeatures).

Insomecases,termsthatactivateanopinioncanalsoconveytheopinionitselforapossiblesentiment,e.g.“approve”or“deny”.Whenthishappens,suchtermsplaytheroleofopiniontriggersaswellasthatofopinionfeatures.



36

Asfortopics,wedistinguishbetweenmaintopicsandsub-topics.

Infact,atopiccanbeacomplexstructure.e.g.aneventorasituation,includingothercomplexstructures.ForthepurposeofSentimentAnalysis,itisimportanttodistinguishallmaintopics,thatarethedirecttargetsofanopinion,fromsub-topics,whichcouldbeindirecttargetsofanopinion.

Forexample,themaintopicoftheopinionsentence:“Annasaystheweatherwillbecomebeautiful”istheeventbecome,whileweather isasub-topic.



37

Thefollowingaxiomsformalisetheconceptsofmaintopicandsub-topicbyusingastandarddescriptionlogicsyntax,directlytranslatableintoOWL(thelanguageusedforrepresentingallMARIOontologies; seeDeliverable5.1).

(MainTopic ⊔ SubTopic) ⊑ Topic

Topics(ofopinionsentences)canbeeithermaintopics(directtargetsoftheopinion)orsub-topics(indirecttargetsofanopinion).

(Topic ⊓ (∃involvedIn(dul:Situation ⊓ MainTopic))) ⊑ SubTopic

Whenamaintopicisasituation,itsinvolvedentitiesaresubtopics.

(Topic ⊓ (∃dependsOn(dul:Event ⊓ MainTopic))) ⊑ SubTopic

Whenamaintopicisanevent,entitiesthathaveadependencyrelationwithitaresub-topics.Examplesofdependencyrelationsare:participationintheevent(e.g.,havingaroleinit),

causalityrelations,temporalrelationse.g.,follows,precedes),etc.



Followingtheopinionmodel,thesemanticrepresentationofthesentence“Annasaystheweatherwillbecomebeautiful”isdepictedintheFigureabove.

38


SentimentLexicalresources


Sentimentlexicalresources (1/2)

Lexicalresourcesarekeytoannotate naturallanguagesentenceatthe“lexicallayer”withsentimentinformation.

Theyprovidetheterms (andpossiblyascore forthem)thatactastriggers ofanopinionorasopinionfeatures.

MARIOSemanticSentimentAnalysismodulereliesondifferentlexicalresources,listedinthenextslide.

40


Sentimentlexicalresources (2/2)

• Sentic-Net [6]:apubliclyavailablesemanticandaaffectiveresourceforconcept-levelopinionandsentimentanalysis.SenticNetisbuiltbymeansofsenticcomputing,aparadigmthatexploitsbothAIandSemanticWebtechniquestobetterrecognise,interpret,andprocessnaturallanguageopinionsovertheWeb.WehavetransformedSentic-NettotheRDFformatandaligneditwithFramester(see D5.1)soastoexploititintheSentimentAnalysismodule.

• SentiWord-Net [7]:alexicalresourceforopinionmining.SentiWordNetassignstoeachsynsetofWordNetthreesentimentscores:positivity,negativity,objectivity.ThisresourceisintegratedwithinFramesterwiththesameapproachandaimastheintegrationofSenticNet.

• DepecheMood [8]:anemotionlexiconbuiltby harvestingcrowdsourcedaffectiveannotation fromasocialnewsnetwork.

• (arevisionof)theLevin’sclassificationofverbs [9]:inthisrevisionwehaveclassifiedfourclassesofopinionverbsthatimplythepresenceofanholder;wecallthemopiniontriggerverbs.TheyarealsoincludedinMARIObackgroundknowledgethroughalignmentwithFramester.

• Sentilo-Net [10]:aresourcethatannotatessemanticrolesofframessoastoenable theselectionofsubtopicsthatareindirectlyaffectedbyopinionsexpressedinasentence,aswellastheevaluationoftheirpolarity.Thisresourceiskeytoevaluatesentimentexpressedincomplexstructuressuchaseventsanddistinguishthedifferentsentimentsassociatedwiththeirparticipants.

41


Frame-basedsentimentanalysis


Frame-basedsentimentanalysis

ThismoduleisimplementedasanextensionofFRED,henceitexploitsadeepparsinganalysis ofasentenceanditsframe-basedrepresentation.

FREDaswellastheuseofBabelNet1 aspartofFramester (seeD5.1)guaranteetheapplicabilityofourmethodtobothItalianandEnglish,whicharethetargetlanguagesforMARIO.

Theframe-basedrepresentationofthesentence isfurtherannotatedwiththeopinionmodelontology(seeslides32-37)

Thecoreofthemoduleisasentimentpropagationalgorithm thatreliesontheSentilo-Netresource(seeslide40).Thealgorithmcomputesthesentimentscoreassociatedwitheachspecificidentifiedtopic inthesentencerepresentation,accordingtotherolethattheyplayintheirparticipatingframe.

Inthefollowingslidesweshowthroughanexamplehowthealgorithmworksandhowthedifferentlexicalsentimentresourcesareinvolved.

431 http://babelnet.org/

ManagingActive&HealthyAgeingwithServiceRobots 44

Thegraphaboverepresentsthesentence:“PeoplehopethatthePresidentwillbecondemnedbythejudges”

ThesentenceisfirstprocessedbyFRED (see D5.6), whichprovidesasoutputitsframe-basedgraphrepresentation.


Thegraphisthenpassedtotheframe-basedsentimentanalysismodule,thatannotates theresultinggraphwiththeopinionmodelontology.

Thisannotationisperformedbyidentifyingtriggeringverbs,opinionholders andopiniontopicsaccordingtorulesassociatedwiththerevisedLevin’sverbclassification[9].

Furthermore,thesemanticrolesidentifiedinthesentenceareannotated accordingtotheSentilo-Netresource,inordertoindicateforeachsub-topicwhetherandhowitisindirectlyaffectedbytheevent(e.g.,sentilo:playSentisitveRole).


Finally,basedontheSentilo-Net-relatedannotations,thescoresarepropagatedthroughthegraph inordertoassignthemtotheacutalentitiestheyarereferredtoandwiththecorrectsign.

Allsentimentfeatures (termscarryingasentimentpolarity)areidentifiedandtheirscoresrepresentedinthemodel.


Sentimentpropagationalgorithm (1/4)

Itreliesonasub-algorithmnamedCombinedScorewhichassignsanindividualsentimentscore(ifapplicable)toeachelementinanopinionsentencegraph. Tothisaim,itreliesonSentiWordNetandSenticNet.

Thealgorithmassignsascoretoadjectivesandadverbsthatareidentifiedbydul:hasQuality relationvalues,andtoinstancesofdul:Event thatarerecognisedastriggerevents,i.e., identifiedbysentilo:hasOpinionTrigger relationvalues.

Wehaveinvestigatedandimplementedtwoalternativeapproachesforscoreselection:• thefirstapproachassignsascoreretrievedbyqueryingthepolarityattributeofaconceptinSenticNet;

• thesecondonecombinesSenticNetandSentiWordNetscores.

47


Sentimentpropagationalgorithm (2/4)

FromasetofempiricalobservationsonusingthePropagationalgorithmwithdifferentapproaches,wenoticedthatthemethodthatcombinesthescoresfromthetworesourcesshowstobemorereliable.

Thisconfirmedourexpectationsbasedonthefollowingrationale:sometimes,SenticNetmissesascorevalueforarequiredconcept.Moreover,itprovidesonescoreperconceptwithoutdistinguishingitspossibledifferentnuances.Hence,SenticNetscoreapproximatesanaveragevalueforthescoresofallpossiblesenses,orpossiblyindicatesthemostprobableone.

Forthisreason,combiningtheSentiWordNetscoresofmostfrequentsensesandtheSenticNetscorecanprovideanappropriatebalancedvalue.Wehavedevisedasimpleheuristics forcomputingthiscombinedscore.

InthenextslidetheCombinedScorealgorithmissketched.

48


Sentimentpropagationalgorithm(3/4)

49

CombinedScore Algorithm


Sentimentpropagationalgorithm(4/4)

Givenanentity,identifiedasatopicofanopinion (eitheramainorsubtopic),wecomputeitssentimentscore bycombiningthescoresofallitsassociatedopinionfeatures (i.e.,valuesofdul:hasQualityrelations),whichareextractedfromtheRDFgraphrepresentingtheopinionatedsentence.

Ifatopicparticipatesinanevent orasituationoccurrence,wepropagate theirsentimentscorestoit,accordingtothesemanticsexpressedbytheframe-basedthematicrole(e.g.,vn.role:Agent)thatitplays,itssensitivenessandfactualimpactattributevalues.

50


Where:• sc(x) isthescoreofanentityx asprovidedbytheCombinedScorealgorithm;• qi(t) isanobjectvalueofatripletdul:hasQualityqi.Suchtriplesrepresentdirectopinion

features,i.e.adjectivesandadverbs,associatedwithentitiescomposingtheopinionsentence;• typej(t) isatypeoftexpressedintheRDFgraphbymeansofrdf:type triples;• cxtk(t) isacontextoft,ifany.Itcanbeeitherasituationoranevent,whichtparticipatesin;• truth(t) isatruthvalueassociatedwitht,wheretistypicallyaneventorsituationoccurrence,

oraquality.Ifitsvalueisfalse,itmeansthattheentityisnegated– forexample,inasentencesuchas‘‘Johnisnotagoodguy’’,aRDFtriplesituation_1

boxing:hasTruthValueboxing:False wouldbeincludedinthegraph,anditseffectwouldbetochangethesignofthesentimentscoreassignedtothefeaturegood;

• mod(t) isamarkedmodalityofatopict,ifany– forexample,inasentencesuchasIwouldlikeadog,anRDFrelationshipfred:like_1

boxing:hasModalityboxing:Necessarywouldbeincluded.Atthistime,thepropagationalgorithmdoesnotyetusethisinformation,butitsabstractmodel,thef function,includesit;

• trig(sent) isanopiniontriggerexpressioninthesentencecontainingt.51

ThesentimentscoreSCsentiment ofatopict canbedefinedasafunctionf definedas:

Sentimentscore


Sentimentpropagationalgorithmflowchart

52


RepresentativeUseCaseScenario

SentimentAnalysisintheMyMemoriesApplication


SentimentanalysisintheMyMemoriesApp

TheMyMemoriesapp isarepresentativeexampleofaMARIOapplicationthatcombinesthecapabilitiesofNLPmodulesandservices withthecapabilitiesofSentimentAnalysismodulesandservices,aspartofitslogicandinteraction/dialoguemanagementstrategy.• Apprequirements,characteristicsanddesignaredetailedinDeliverable3.3.• AppinteractionwithMARIO’sNLPmodulesanddialoguemanagementarepresentedinDeliverable5.6.

Inthefollowing,wedescribethepeculiaritiesofthisappintermsofSentimentAnalysisandtheirimpactontheapp’sdialoguemanagementapproach• forthesakeofself-containedness andreadabilityofthisDeliverable,themaincharacteristicsoftheappreportedinD5.6aresummarisedagain.

Researchoutcomesrelatedtothisappshavebeenpresentedatthe1stInternationalWorkshoponApplicationofSemanticWebtechnologiesinRobotics(AnSWeR),co-locatedwiththe14thExtendedSemanticWebConference(ESWC2017)inPortoroz,Slovenia[11].

54


MyMemoriesApplication

Enablestherobottoundertakeinteractiveandpersonalisedreminiscence sessionsthroughaconversationalapproachbasedonuser-specificknowledgeandmaterials.• KnowledgeBase(KB)support: userprofiles,family/social

relationships,lifeevents,taggedmediaobjects(e.g.,photographs).• interactionpatterns: prompttheuserwithfocusedmemory

triggers(verbalprompt+photograph).

55

User-specificKnowledge Base

Interaction Patterns, NLPand Dialogue Management


ArchitecturalReference

56

Managesreminiscencesessionsanddialoguestateandflow• interactionpatternandpromptselection• NLPservicesinvocationsandinterpretation

Parametricinteractionpatternsprovidedialogueblueprint• question/promptformulation• NLPinterpretationstrategy


InteractionPatterns

Theinteractionwiththeuserduringareminiscencesessioncanfollowtwodifferentconversationalapproaches,bothbasedonsystem-initiated dialoguefragmentsdefinedintheformofinteractionpatterns.1. Question-based:MARIOaskstheuserfocusedclosed-ended questions

relatedtotheimagecontextuallyshownasmemorytrigger– detailedinD5.6

2. Prompt-based:MARIOpromptstheuserwithopen-ended promptsorquestionsrelatedtotheimage:– promptsaimatstimulatingreminiscenceaboutpeople(e.g.,“Whatwasyou

sisterlikeasachild?Tellmemoreabouther!”),places(e.g.,“WhatwasitliketogrewupinLondon?”)andlifeevents(e.g.,“Tellmemoreaboutyourweddingday!”)

– sentimentanalysis capabilitiesenableMARIOtoidentifythepolarityorsentimentexpressedbythePwD andtoreplyandactaccordingly

57


Prompt-basedInteractionPatterns

58

Patternstructureandelements(JSON-encoded)

precondition

prompt

ifPositiveSay

ifNeutralSay

ifNegativeSay

ConstraintmappingtoqueriesovertheKB,definingunderwhichconditionsthepromptingquestioncanbeused.

MultilingualparametricprompttemplatestobeinstantiatedwithKBdata(entitiesandtheirpropertyvalues).

Multilingualparametricsystemutterancetemplatetobeinstantiatedifuser’sanswerhaspositive sentiment.

Multilingualparametricsystemutterancetemplatetobeinstantiatedifuser’sanswerhasneutral sentiment.

Multilingualparametricsystemutterancetemplatetobeinstantiatedifuser’sanswerhasnegative sentiment.


Prompt-basedInteractionProcess(1/2)

Foraninteractionpattern whoseapplicabilitypreconditions aresatisfied(wrttheKnowledgeBase,andinparticularforaspecificphotograph),thedialogueismanagedaccordingtothefollowingmainsteps:• thecorrespondingprompttemplate isinstantiatedandthepromptisissuedtothePWD;

• thetextualrepresentationofPWD’svocalinputisprocessedrelyingonthecapabilitiesoftheSentimentAnalysissubsystem;

• dependingonthesentimentandscoreidentifiedforPWD’sanswerinthesentimentanalysisstep,thecorrespondingutteranceisissuedbyMario,asdefinedintheinteractionpattern.

Theoverallprocessisgraphicallysummarisedinthefollowingslideandthenillustratedwithaconcreteexample.

59


Prompt-basedInteractionProcess(2/2)

60


Example

61

USER ANSWER PROCESSINGTask: sentimentanalysis (polaritydetection)overuser’sutterance

Possible outcomes• Positive ⟶ encouragetotellmore(“Thatsoundsnice!Tellmemoreaboutyourmarriage!”),

showotherpicsof sameevent• Neutral ⟶ proposal(“Tellmemoreifyoulike,oraskmetoshowyouanotherphoto!”)• Negative⟶ movetonextphoto


Example– Polaritydetection(1/2)

62

“Irememberitwasagreatday!”

http://nlp.stanford.edu:8080/sentiment/rntnDemo.html


Polaritydetection(1/2)

SimplepolaritydetectionenablesMARIOtoidentifyuser’sreactiontoaspecificpromptandimage.

Polarityinfoprovidedbythesentimentanalysismodulecanbe• usedtodrivehowMARIOproceedsinthereminiscencesession

– usedifferentutterancesandstrategiesfordifferentpolarities(asoutlinedinthepreviousslides:showempathyandasktotellmoreforapositivereaction,proposetomovetonextphotoforanegativereaction,etc.);

– selectnextinteractionpatterndependingonpolarity,forexamplebychoosingoravoidingphotos/promptsrelatedtothecurrentsubject(lifeevent,person,etc.);

• storedinrelationtotheimageandsubjectoftheprompt– reusedintheimage/patternselectionprocessinsubsequentreminiscencesessions(e.g.,

favouringmemorytriggersgeneratingpositivefeelings).

However,noinfocanbeextractedonspecifictopic(s)orentitiesaboutwhichthesentimentisexpressed(e.g.,theweddingdayitself,thewife,thepicture,etc.)• finer-grainedsentimentpolarityscoresareprovidedbyframe-basedsentimentanalysis.

63


Polaritydetection(2/2)

ThepolaritydetectionprocessisalsoinfluencedbythecharacteristicsofPWD’sutterancesandtheirprocessingbytheSpeechtoText(S2T)component(describedinD5.6).• IfanutterancecomingfromtheS2Tisrecognisedasasinglesentencebythesentencesplitstepin

theNLPpipeline,thepolarityofthesentenceisanalysedandevaluated,andMARIOreactsasoutlinedbefore(seeslides58-59).– Foragivenprompt,thePWDmayexpressdifferentopinionsisdifferentutterances,eachcorrespondingtoa

singlesentenceandfollowedbyMARIO’sreaction.– Eachsentenceisprocessedandanalysedindependently(e.g.,apositivereactionwillfirstleadMARIOto

encouragethePWDtotellmoreaboutthecurrentperson/event/place;asubsequentsentencewithnegativepolaritywillthenleadMARIOtosuggesttomovetoanotheritem).

• IfanutterancecomingfromtheS2TisrecognisedasmultiplesentencesbythesentencesplitstepintheNLPpipeline,thepolarityofeachsentenceisanalysedandevaluated.– Theoverallpolarityoftheutterancecanbeassessedbycombiningthepolarityscoresofthesentences(e.g.,

bycomputinganaveragesentimentscoreoverthenumberofsentences).– Again,noinfocanbeextractedonspecifictopic(s)orentitiesaboutwhich(potentiallydifferent)sentiments

areexpressed.– Forexample,anutterancewithapositivesentenceandanegativeonewithsimilarscoreswouldbe

consideredasneutral,regardlessofthementionedtopicsorentities;noinfoiscapturedonthepositiveandnegativeattitudesandthecorrespondingtopicsorentitiesthatcharacteriseeachsentence.

64


Example– Frame-basedanalysis(1/2)MARIOcanmodelthequestionbyusingtheframe-basedapproachenabledbyFRED.

65

Whatwasyourweddingdaylike?


Example– Frame-basedanalysis(2/2)Frame-basedrepresentationofthe user’s opinionsentence

66

Thereminiscenceassociatedwith thePatient’sweddingday

isrecognisedasapositivesituation(0.313polarityvalue)

“Irememberitwasagreatday!”

Apositivescore(0.564)is alsospecifically associated withtheentity representing theday


ConcludingRemarks(1/2)

• ThesoftwarecomponentspresentedinthisdocumentarepartoftheoverallMARIOframeworkdeployedon6Kompai-2robots usedforevaluationandvalidationactivitiesinthe3projectpilotsites:– GeriatricUnitatthehospitalCasaSollievo della Sofferenza,Italy;– NursingCentres attheNationalUniversityofIreland,Galway;– CommunitiesinStockport.

• Atthetimeofwriting,evaluationandvalidationactivitiesareundergoing(accordingtothetime-linedefinedinWP8).

• Thedataandfeedbackthatwillbecollectedduringthefinalvalidation period (Aug-Nov2017)willserveasabasisforassessing,indifferentsettings,thecapabilitiesoftheSentimentAnalysissubsystem

67


ConcludingRemarks(2/2)

• Whilesimplepolarity-basedsentimentanalysistechniquesarecurrentlyusedinmultipledomains,theframe-basedsemanticsentimentanalysisapproachinvestigatedinTask5.3representsamajorcontribution,inparticularforthetargeteddomain

• Ongoingresearchactivities,whosetimeframegoesbeyondWP5,aimtoinvestigatehowtheapproachcanbeextendedfor:– aspect-based sentimentanalysis,torecognisethesentiment

expressedonspecificaspects/featuresrelatedtoatopicorentity– emotionanalysis,todetectspecificemotionsoremotionalstates

beyondpolarity

68


References(1/2)

69

[1] StanfordCoreNLP Sentiment.Availableat:http://nlp.stanford.edu/sentiment/code.html[2] R.Socher,A.Perelygin,J.Wu,J.Chuang,C.Manning,A.Ng,andC.Potts."RecursiveDeepModels

forSemanticCompositionalityOveraSentimentTreebank".In:Proc.oftheConferenceonEmpiricalMethodsinNaturalLanguageProcessing(EMNLP2013),2013

[3] BingLiu.“SentimentAnalysisandSubjectivity”.HandbookofNaturalLanguageProcessing,2010[4] BingLiu.“SentimentAnalysisandOpinionMining”.Morgan&ClaypoolPublishers.May2012[5] H.Saif,Y.He,andH.Alani.“SemanticSentimentAnalysisofTwitter”.Boston,UA,pp.508–524,

Springer,2012[6] E.Cambria,C.Havasi,andA.Hussain,“SenticNet 2:Asemanticandaffectiveresourceforopinion

miningandsentimentanalysis”. In:Proc.FLAIRSConf.,2012.[7] A.Esuli,S.Baccianella,andF.Sebastiani,“SentiWordNet 3.0:Anenhancedlexical resourcefor

sentimentanalysisandopinionmining”. In:Proc.7thconf.Int.LanguageResourcesEvaluation,2010.

[8] DepecheMood.Availableat:https://github.com/marcoguerini/DepecheMood/releases[9] Levinopinion.Availableat:http://www.stlab.istc.cnr.it/documents/sentilo/levin-opinion.zip[10]Sentilo-Net.Availableat:http://www.stlab.istc.cnr.it/documents/sentilo/sentilonet.zip


References(2/2)

70

[11] L.Asprino,A.Gangemi,A.G.Nuzzolese,V.Presutti,A.Russo:“Knowledge-drivenSupportforReminiscenceonCompanionRobots”.In:Proceedingsof1stInternationalWorkshoponApplicationofSemanticWebtechnologiesinRobotics(AnSWeR 2017),Portoroz,Slovenia,2017

[12]A.Gangemi,V.Presutti,D.Reforgiato Recupero,A.G.Nuzzolese,F.Draicchio,M.Mongiovì:“SemanticWebMachineReadingwithFRED”.SemanticWebJournal,8(6),2017

[13]A.Gangemi,M.Alam,L.Asprino,V.Presutti,D.Reforgiato Recupero.“Framester:AWideCoverageLinguisticLinkedDataHub”.In:Proceedingsofthe20thInternationalConferenceonKnowledgeEngineeringandKnowledgeManagement(EKAW2016),pp.19-23,2016

[14]Framester - Agianthuboflinguisticlinkedopenresources.https://lipn.univ-paris13.fr/framester/

Date post:	05-Oct-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times

D5.7 –Robot Semantic Sentiment Analysis - mario-project.eu · Managing Active & Healthy Ageing...

Documents