Post on 01-Nov-2014
description
transcript
Mat Morrison@mediaczar
Tuesday, 5 July 2011
Tuesday, 5 July 2011
Tuesday, 5 July 2011
Elephant in the roomTuesday, 5 July 2011
or Emperor’s ClothesTuesday, 5 July 2011
Tuesday, 5 July 2011
Volume
Tuesday, 5 July 2011
VolumeRelevanceAUTHORITYSentiment
qualified by
Tuesday, 5 July 2011
quick Question:
How Accurate
is Sentiment analysis?
Tuesday, 5 July 2011
The
stinks
Tuesday, 5 July 2011
Robot Human
supervi
sed
Machin
eLearnin
g
Tuesday, 5 July 2011
Robots first...
Tuesday, 5 July 2011
I’m unqualified to
make judgements
about the science...
Tuesday, 5 July 2011
I’m unqualified to
make judgements
about the science...
...and So are you.
Tuesday, 5 July 2011
Because the Science Is hidden in a box
Tuesday, 5 July 2011
Because the Science Is hidden in a box
But scientific Results are Reproducable!!!
Tuesday, 5 July 2011
What’s the buzz around “Green Lantern”?
Tuesday, 5 July 2011
Positive Negative Net
Sysomos (Blogs) 40 17 23
TweetSentiment 29 11 18
WDTIT? 56.8 42 14.8
SocialMention 18 5 13
SM2 20 16 4
Sysomos (Tweets) 6.4 7.6 -1.2
Rotten Tomatoes 27 73 -46
A quick poll of a few sentiment engines’ analysis of the buzz surrounding the film “Green Lantern” demonstrates no correspondence.
Tuesday, 5 July 2011
Tuesday, 5 July 2011
Tuesday, 5 July 2011
RT @BangsandaBun: According to
some, I wasn't deserving/grateful/
cool enough to be there, but I
blogged about my Glastonbury
experience http://t.co/eEInrnu
Turk THis!
Tuesday, 5 July 2011
RT @BangsandaBun: According to
some, I wasn't deserving/grateful/
cool enough to be there, but I
blogged about my Glastonbury
experience http://t.co/eEInrnu
Turk THis!
Robot says Negative
Tuesday, 5 July 2011
I know the bands for Glastonbury
were great this year, but I still
don't think you could have paid me to
be part of that mental crowd! You?
Turk THis!
Tuesday, 5 July 2011
I know the bands for Glastonbury
were great this year, but I still
don't think you could have paid me to
be part of that mental crowd! You?
Turk THis!
Robot says positive
Tuesday, 5 July 2011
SENTIMENT scoring using Amazon Mechanical Turk disagreed with 16/40 (40%) robot judgments.
Only 13/40 (32.5%) of robotic judgments were passed by all three human Judges.
Raw Data available at http://mczar.me/mkue1p
Tuesday, 5 July 2011
But Are Humans Reliable?
Emotional State/Mood
Priming
Cultural bias
Cognitive Bias
Famously not.
???
Tuesday, 5 July 2011
It Gets Worse...
The Same Human Scorer,
when presented with the
same statement on
multiple occasions, may
score it differently each
time.cf. A Guerilla Experiment in Sentiment Analysis http://mczar.me/lrp7Yi
Tuesday, 5 July 2011
Tuesday, 5 July 2011
The
stinks
Tuesday, 5 July 2011
Academic studiesgenerally focus on
Movies & the STOCK Market
Why is this?
Tuesday, 5 July 2011
Social Media data are full of irrelevant...
Tuesday, 5 July 2011
Social Media data are full of irrelevant...
Tuesday, 5 July 2011
Tuesday, 5 July 2011
VolumeRelevanceAuthoritySentiment
Tuesday, 5 July 2011
VolumeRelevanceAuthoritySentiment
All four are needed to make sense of data
But I’m only talking sentiment
Tuesday, 5 July 2011
The
stinks
Tuesday, 5 July 2011
To qualify social media buzz
Why might marketers want to perform Sentiment analysis?
To guide customer service interactions
As a Performance indicator
To identify fans
Tuesday, 5 July 2011
Is sentiment actionable?
Product Query
Product Complaint
Product endorsement
Product review
Product COmparison
...I need to know more
Is it a
...
???
Tuesday, 5 July 2011
!"#$
#$
"#$
%##$
%"#$
&##$
&"#$
'##$
()*+,-+.$ /+0+,-+.$ 12342.5$ 6+-.42.5$ 72.08$ 9:.;<$ 725$
!"#"$%&'#
=)*;>$?)@2<$
A2.-4.@)3>$?)@2<$
B;3C>,;<<$?)@2<$
Massive (and unrepresentative) spike in Net Positive
Sentiment caused by specific campaign activity
THE KPI Problem
Should we be allowed to mark our own homework?
Tuesday, 5 July 2011
Tuesday, 5 July 2011
Tuesday, 5 July 2011
Tuesday, 5 July 2011