Date post: | 11-Feb-2017 |
Category: |
Data & Analytics |
Upload: | ana-canhoto |
View: | 3,613 times |
Download: | 4 times |
Studying sentiment on social media
Ana Isabel Canhoto - Oxford Brookes Universitywww.anacanhoto.com
Canhoto 2015 1
2
Emotions impact on:•Information retrieval•Information processing•Information retention•Decision-making•Behaviour•Assessment of consumption experiences
Why study sentiment?
Canhoto 2015
Image source: http://images.flatworldknowledge.com/sirgy/sirgy-fig06_x003.jpg
3
What are we talking about when we talk about sentiment analysis?
More: http://www.mxmindia.com/2012/03/tweets-take-wing-in-airline-social-media-study/
Canhoto 2015
4
Traditional approaches - Experiments
More: http://www.psych.nyu.edu/amodiolab/Publications_files/Social_Psychological_Methods_of_Emotion_Elicitation.pdf
Canhoto 2015
5
Traditional approaches – Interviews
Canhoto 2015
Real time Unprompted No need to recall past behaviour Non-intrusive Cost-effective…
6
The Social Media Promise
Canhoto 2015
7Canhoto 2015Source: http://cs-wordpress.s3.amazonaws.com/crowdsource-v4/uploads/2013/11/sentiment-analysis-ui.png
Canhoto 2015 8
Pratik Thakar, Head of creative content for Coca-Cola Asia-Pacific:
“Every office has a listening centre listening to what people are saying about our brands, good and bad, 24 hours a day. We look at what’s trending and how we can respond [to discussions about Coca-Cola] and to anything happening in the world. (…) I believe that social media is a big focus group. It’s a good way to identify trends and what people are talking about”
Source: http://www.campaignasia.com/Article/402239,Dont+believe+everything+you+hear+Cokes+Pratik+Thakar.aspx
9
Many turning to third parties for automated tracking and analysis of SM conversations…
Canhoto 2015
44% of businesses engaged in sentiment analysisHilpern, K. 'In it to win it?' The Marketer, July-August 2012, pp.34-37
Estimated cumulative revenues cc $2bn in 2014
Source: http://breakthroughanalysis.com/2013/12/30/aw-re-aw-text-analytics-industry-study_start-ups-and-aquisition-activities_max-breitsprecher/
How accurate are these tools?
Promotional literature: accuracy rates of 70% - 80% (Davis & O’Flaherty, 2012)– Not clear how the coefficients were
calculated– Not possible to independently verify these
claims 10Canhoto 2015
11Canhoto 2015
Open access
12
Sources of vulnerability
Canhoto 2015
• Accuracy: extent to which different researchers agree on the classification of a particular data object (Gwet, 2012)– System vs human coders– System A vs System B…
13Canhoto 2015
Conversations about coffee •Food and beverages = most widely discussed topic on social media (Forsyth, 2011)•‘Charged with a wide range of cultural meanings’ (Grinshpun, 2014)•Subject of many (netnographic) studies - e.g., Kozinets, 2002
14Canhoto 2015
• Sample of 200 tweets• Search terms: ‘coffee’ + variants ‘latte’,
‘mocha’, ‘cappuccino’, ‘espresso’ and ‘Americano’, as well as the terms ‘flavour’, ‘aroma’ and ‘caffeine’.
• Multiple users– Exclude manufacturers and retailers.
15Canhoto 2015
Analysis - Stage 1: Polarity of emotion•Positive vs. negative– As per Koppel & Schler (2006): comments that did
not express an emotion, were given the code ‘neutral’.
•Manual + 2 automated tools
16Canhoto 2015
17Canhoto 2015
Analysis - Stage 2: Type of emotion• As per Plutchik (2001)•Manual + 3 Automated tools
18Canhoto 2015
19Canhoto 2015
20Canhoto 2015
Messages where all types of coders agreed
Examples:“Found a euro cent on my walk and have a great cup of coffee in hand. Monday is already off to a good start”
“Feeling much more alive this morning now that I’ve had my coffee. Thank you #Nespresso”.
Clearly positive!21Canhoto 2015
Messages where automated tools agreed (but different from manual coding)
Example:“In uni. I think without this cup of coffee I would hulk out”
Very short segments
22Canhoto 2015
The rest
Example:“The early shift sucks. Oh well at least my latte is yummy :) “
23
Multiple objects
Multiple emotions
Canhoto 2015
Example:“100 copies of Ghosts sold overnight means a definite Starbucks run this morning. Possibly coffee out twice this week! Maybe even sushi!!”
Lack of emotionally charged words
24Canhoto 2015
Example:“How the heck am I supposed to be able to sleep well without coffee in my system? fucking snow”
Subtlety - Negative sentiment due to absence of product
25Canhoto 2015
Example:“Having coffee with my grandma before work right now. QT”
Syntax and style, specially abbreviations and slang
26Canhoto 2015
Example:“This coffee shop needs to change there music up every once and a while. Or maybe I should go home”
Target of emotion is not coffee!
27Canhoto 2015
28Canhoto 2015
29Canhoto 2015
30
Compounded by:• Very short segments of text• Rich in abbreviations and slang• Typos or grammatical errors• Specific culture and netiquette of the media• Skills of data analyst
Canhoto 2015
As a consequence:•Inaccurate representation of the overall sentiment [towards coffee]– Both sentiment polarity and emotional state
•Segments that should have been excluded from the analysis were retained in the corpus of data– Might skew results
•Concerns with the quality of the insights and subsequent decisions
31Canhoto 2015
To improve accuracy [1/2]:•Take into consideration the social context of the conversation– E.g., Tweets before or after the one being coded; wide
patterns (e.g., Mondays); cultural connotations (e.g., Japan vs. UK)
– But what about sarcasm and highly contextualised uses of language? (e.g., Sick)
32Canhoto 2015
Pratik Thakar:“When people say good things, you don’t just take it as it is. Someone might be asking them to say it; there might be some design mechanism working. But when people are unhappy, they go super-loud, and they are genuine at that time. ”Source: http://www.campaignasia.com/Article/402239,Dont+believe+everything+you+hear+Cokes+Pratik+Thakar.aspx
To improve accuracy [2/2]:•Develop dictionaries that reflect the specific syntax and style
•Software solutions that “translate” commonly used abbreviations and typos– E.g., BRB – be right back– Changing norms – e.g., LOL
•Familiarise with software
33Canhoto 2015
Studying sentiment on social media
Ana Isabel Canhoto - Oxford Brookes Universitywww.anacanhoto.com
Canhoto 2015 34