Date post: | 20-Jan-2016 |
Category: |
Documents |
Upload: | philip-peters |
View: | 216 times |
Download: | 1 times |
Sentiment and Opinion
Jan 18, 2011
Analysis of Social Media Seminar
William Cohen
First assignment: due Friday
• Go to http://malt/mw• Create an account for yourself
– use andrew id
• Go to your user page– Your real name & a link to your home page– Preferably a picture– Who you are and what you hope to get out of the class (Let me
know if you’re just auditing)– Any special skills you have, research interests that you have,
related projects you have been or might be working on, etc.
Outline
• Announcements• Recap
– With a little more on word senses
• More discussion: what exactly is subjectivity, sentiment and polarity?– Annotating a corpus for subjectivity– Fine-grained sentiment for reviews
• More distinctions:– Agreement and discourse
3
In our previous episode…
4
5
Motivations: sentiment common…
Analysis : modeling & learning
Communication,Language
People
Networks
SocialMedia
6
…and important…
• Product review mining: What features of the ThinkPad T43 do customers like and which do they dislike?
• Review classification: Is a review positive or negative toward the movie?
• Tracking sentiments toward topics over time: Is anger ratcheting up or cooling down?
• Etc.
[These are all ways to summarize one sort of content that is common on blogs, bboards, newsgroups, etc. –W]
…and non-trivial
7
What units do we attach sentiment to?
• Individual words (“nice”, “comfortable”)
• Phrases (“slow service”)
• Sentences?
• Documents?
• … ?
8
9ICWSM 2008 9
Hatzivassiloglou & McKeown 1997
Build a graph of adjectives linked by the same or different semantic orientation (determined by
conjunctions)…
nice
handsome
terrible
comfortable
painful
expensive
fun
scenic
10ICWSM 2008 10
Hatzivassiloglou & McKeown 1997
…and a clustering algorithm partitions the adjectives into two subsets
nice
handsome
terrible
comfortable
painful
expensive
fun
scenicslow
+
Jan - ICWSM 2008 11
Word senses
Senses
Jan - ICWSM 2008 12
Senses
Is this polar?
Jan - ICWSM 2008 13
Non-subjective senses of brilliant
1. Method for identifying brilliant material in paint - US Patent 7035464
2. In a classic pasodoble, an opening section in the minor mode features a brilliant trumpet melody, while the second section in the relative major begins with the violins.
ICWSM 2008 14
Subjective Sense Examples
• His alarm grew Alarm, dismay, consternation – (fear resulting form the
awareness of danger)– Fear, fearfulness, fright – (an emotion experiences in
anticipation of some specific pain or danger (usually accompanied by a desire to flee or fight))
• He was boiling with anger Seethe, boil – (be in an agitated emotional state; “The
customer was seething with anger”)– Be – (have the quality of being; (copula, used with an adjective
or a predicate noun); “John is rich”; “This is not a good answer”)
SN
SN
ICWSM 2008 15
Objective Sense Examples
• The alarm went off Alarm, warning device, alarm system – (a device that
signals the occurrence of some undesirable event)– Device – (an instrumentality invented for a particular purpose;
“the device is small enough to wear on your wrist”; “a device intended to conserve water”
• The water boiled Boil – (come to the boiling point and change from a
liquid to vapor; “Water boils at 100 degrees Celsius”)– Change state, turn – (undergo a transformation or a change of
position or action)
ICWSM 2008 16
Objective Senses: Observation
• We don’t necessarily expect phrases/sentences containing objective senses to be objective– Will someone shut that darn alarm off?– Can’t you even boil water?
• Subjective, but not due to alarm and boil
ICWSM 2008 17
Objective Sense Definition
• When the sense is used in a text or conversation, we don’t expect it to express subjectivity and, if the phrase/sentence containing it is subjective, the subjectivity is due to something else.
18
• Later/related work:
– LIWC, General Inquirer, other hand-built lexicons– Turney & Littman, TOIS 2003: Similar performance with 100M
word corpus and PMI – higher accuracy better if you allow abstention on 25% of the “hard” cases.
– Kamps et al, LREC 04: Determine orientation by graph analysis of Wordnet (distance to “good”, “bad” in graph determined by synonymy relation)
– SentiWordNet, Esuli and Sebastiani, LREC 06: Similar to Kamps et al, also using a BOW classifier and WordNet glosses (definitions).
Hatzivassiloglou & McKeown 1997
What units do we attach sentiment to?
• Individual words (“nice”, “comfortable”)
• Phrases (“slow service”)
• Sentences?
• Documents?
• … ?
19
20
Turney 2002
• Goal: classify reviews as “positive” or “negative”.– Epinions “[not] recommended” as given by authors.
• Method:– Find (possibly) meaningful phrases from review (e.g.,
“bright display”, “inspiring lecture”, …), • based on POS patterns, like ADJ NOUN
– Estimate “semantic orientation” of each candidate phrase• Based on pointwise mutual information: Altavista counts of
phrase’s cooccurrence with “excellent”, “poor”
– Assign overall orentation of review by averaging orentation of the phrases in the review
21
22
Pang et al EMNLP 2002
23
Pang & Lee EMNLP 2004
24
Methods: 2002
• Movie review classification as pos/neg.• Method one: count human-provided polar words
(sort of like Turney):– Eg, “love, wonderful, best, great, superb, still,
beautiful” vs “bad, worst, stupid, waste, boring, ?, !” gives 69% accuracy on 700+/700- movie reviews
• Method two: plain ‘ol text classification– Eg, Naïve Bayes bag of words: 78.7; SVM-lite “set of
words”: 82.9 was best result– Adding bigrams and/or POS tags doesn’t change
things much.
25
Pang & Lee EMNLP 2004
• Can you capture the discourse in the document?– Expect longish runs of subjective text and longish
runs of objective text.– Can you tell which is which?
• Idea: – Classify sentences as subjective/objective, based on
two corpora: short biased reviews, and IMDB plot summaries.
– Smooth classifications to promote longish homogeneous sections.
– Classify polarity based on the K “most subjective” sentences
What units do we attach sentiment to?
• Individual words (“nice”, “comfortable”)
• Phrases (“slow service”)
• Sentences?
• Documents?
• … ?
26
Outline
• Announcements• Recap
– With a little more on word senses
• More discussion: what exactly is subjectivity, sentiment and polarity?– Annotating a corpus for subjectivity– Fine-grained sentiment for reviews
• More distinctions:– Agreement and discourse
27
Manual and Automatic Subjectivity and Sentiment
Analysis Jan Wiebe
Josef RuppenhoferSwapna SomasundaranUniversity of Pittsburgh
29
Everyone knows that dragons don't exist. But while this simplistic formulation may satisfy the layman, it does not suffice for the scientific mind. The School of Higher Neantical Nillity is in fact wholly unconcerned with what does exist. Indeed, the banality of existence has been so amply demonstrated, there is no need for us to discuss it any further here. The brilliant Cerebron, attacking the problem analytically, discovered three distinct kinds of dragon: the mythical, the chimerical, and the purely hypothetical. They were all, one might say, nonexistent, but each nonexisted in an entirely different way...
- Stanislaw Lem, “The Cyberiad”
30
Preliminaries
• What do we mean by subjectivity?• The linguistic expression of somebody’s
emotions, sentiments, evaluations, opinions, beliefs, speculations, etc.– Wow, this is my 4th Olympus camera. – Staley declared it to be “one hell of a
collection”. – Most voters believe that he's not going to
raise their taxes
31
Corpus AnnotationWiebe, Wilson, Cardie 2005
Annotating Expressions of Opinions and Emotions in Language
Leaving aside what’s possible, what sort of inferences about sentiment, opinion, etc would we like to be able to make?
32
Overview
• Fine-grained: expression-level rather than sentence or document level– The photo quality was the best that I have seen in a
camera. – The photo quality was the best that I have seen in a
camera.
• Annotate – expressions of opinions, evaluations, emotions – material attributed to a source, but presented
objectively
33
Overview
• Fine-grained: expression-level rather than sentence or document level– The photo quality was the best that I have seen in a
camera. – The photo quality was the best that I have seen in a
camera.
• Annotate – expressions of opinions, evaluations, emotions,
beliefs – material attributed to a source, but presented
objectively
34
Overview
• Opinions, evaluations, emotions, speculations are private states.
• They are expressed in language by subjective expressions.
Private state: state that is not open to objective observation or verification.
Quirk, Greenbaum, Leech, Svartvik (1985). A Comprehensive Grammar of the English Language.
35
Overview
• Focus on three ways private states are expressed in language
– Direct subjective expressions– Expressive subjective elements– Objective speech events
36
Direct Subjective Expressions
• Direct mentions of private states
The United States fears a spill-over from the anti-terrorist campaign.
• Private states expressed in speech events
“I fear electoral fraud,” Tsvangirai said.
Fear is a private state
Fear is a private state but not of the author
37
Direct Subjective Expressions
• Direct mentions of private states
The United States fears a spill-over from the anti-terrorist campaign.
• Private states expressed in speech events
“We foresaw electoral fraud but not daylight robbery,” Tsvangirai said.
This implies a private state, so it’s not direct..
Fear is a private state
38
Expressive Subjective Elements [Banfield 1982]
• “We foresaw electoral fraud but not daylight robbery,” Tsvangirai said
• The part of the US human rights report about China is full of absurdities and fabrications
Compare:
•“We foresaw difficulties with the electoral process but not to this extent”, Tsvangirai said.
•The part of the US human rights report about China contains many statements that we were unable to verify.
Understood as implying certain mental state
39
Objective Speech Events
• Material attributed to a source, but presented as objective fact
The government, it added, has amended the Pakistan Citizenship Act 10 of 1951 to enable women of Pakistani descent to claim Pakistani nationality for their children born to foreign husbands.
[What does this have to do with opinion? You need it to sort out who has opinions about what… -W]
An example…
40
41
Nested Sources
“The report is full of absurdities,’’ Xirao-Nima said the next day.
(Writer)
42
Nested Sources
“The report is full of absurdities,’’ Xirao-Nima said the next day.
(Writer, Xirao-Nima)
43
Nested Sources
“The report is full of absurdities,’’ Xirao-Nima said the next day.
(Writer Xirao-Nima)(Writer Xirao-Nima)
44
“The report is full of absurdities,” Xirao-Nima said the next day.
Objective speech event anchor: the entire sentence source: <writer> implicit: true
Direct subjective anchor: said source: <writer, Xirao-Nima> intensity: high expression intensity: neutral attitude type: negative target: report
Expressive subjective element anchor: full of absurdities source: <writer, Xirao-Nima> intensity: high attitude type: negative
Attributes:
The anchor is the linguistic expression—the stretch of text—that tells us that there is a private state. [Where to ‘hang’ the annotation’ -W]
The source is the person to whom the private state is attributed. Note that this can be a chain of people.
The target is the content of the private state or what the private state is about.
Attitude type: If not specified, it is to be understood as neutral but can be set to positive or negative as required.
Intensity records the intensity of “the private state as a whole.”
Another example…
45
ICWSM 2008 46
“The US fears a spill-over’’, said Xirao-Nima, a
professor of foreign affairs at the Central University
for Nationalities.
ICWSM 2008 47
“The US fears a spill-over’’, said Xirao-Nima, a
professor of foreign affairs at the Central University
for Nationalities.
(Writer)
ICWSM 2008 48
“The US fears a spill-over’’, said Xirao-Nima, a
professor of foreign affairs at the Central University
for Nationalities.
(writer, Xirao-Nima)
ICWSM 2008 49
“The US fears a spill-over’’, said Xirao-Nima, a
professor of foreign affairs at the Central University
for Nationalities.
(writer, Xirao-Nima, US)
ICWSM 2008 50
“The US fears a spill-over’’, said Xirao-Nima, a
professor of foreign affairs at the Central University
for Nationalities.
(writer, Xirao-Nima, US) (writer, Xirao-Nima)(Writer)
ICWSM 2008 51
Objective speech event anchor: the entire sentence source: <writer> implicit: true
Objective speech event anchor: said source: <writer, Xirao-Nima>
Direct subjective anchor: fears source: <writer, Xirao-Nima, US> intensity: medium expression intensity: medium
…
“The US fears a spill-over’’, said Xirao-Nima, a
professor of foreign affairs at the Central University
for Nationalities.
52
Corpus• www.cs.pitt.edu/mqpa/databaserelease (version 2)• English language versions of articles from the world
press (187 news sources)• Themes of the instructions:
– No rules about how particular words should be annotated. – Don’t take expressions out of context and think about what
they could mean, but judge them as they are used in that sentence.
• Kappa around 0.7 – 0.8.
53
Reasons for fine-grain annotation and analysis
• Turney, Pang et al: document D is about a known product PD, sentiment refers to PD. Life is more complicated:– “The part of the US human rights report about China is full
of absurdities and fabrications”:• What is “absurd & fabricated”? The part, the US, the report,
or China?
• For sentiment about products we want to know what is good or bad: there are usually tradeoffs– Huge screen very heavy– Very fast really expensive
Outline
• Announcements• Recap
– With a little more on word senses
• More discussion: what exactly is subjectivity, sentiment and polarity?– Annotating a corpus for subjectivity– Fine-grained sentiment for reviews
• More distinctions:– Agreement and discourse
54
55
ICWSM 2008 56
Hu & Liu 2004Mining Opinion Features in Customer Reviews
• Here: explicit product features only, expressed as nouns or compound nouns
• Use association rule mining technique rather than symbolic or statistical approach to terminology
• Extract associated items (item-sets) based on support (>1%)
I think this technique basically amounts to taking frequent ngrams, after they do the pruning - W
Sample- one of many papers
ICWSM 2008 57
Hu & Liu 2004
• Feature pruning – compactness
• “I had searched for a digital camera for 3 months.”• “This is the best digital camera on the market”• “The camera does not have a digital zoom”
– redundancy/overlap• manual ; manual mode; manual setting
• Feature expansion– For sentences with opinion words and no
features, add NP closest to each opinion word
ICWSM 2008 58
Hu & Liu 2004
• For sentences with frequent feature, extract nearby adjective as “effective opinion” for
• Based on opinion words, gather infrequent features (N, NP nearest to an opinion adjective)
– The salesman was easy going and let me try all the models on display.
ICWSM 2008 59
Hu & Liu 2004
• Semantic orientation of words– Propogate labels for a set of 30 seeds through
WordNet using synonymy and antonymy
• Opinion sentences: opinion word + feature• Semantic orientation of sentences
– Flip word polarity if there are nearby negations– Go with the majority of opinion words– Break ties with majority of words that are part of
“effective opinions”• i.e., adjective closest to a feature
ICWSM 2008 60
Hu & Liu 2004
• Summary:– Feature identification: 72-80% recall/precision
• on 500 reviews from five domains.
– Opinion sentence extraction (opinion word + feature): 60-80% recall/precision
– Sentence-level orientation accuracy: 73-95%
Comment: 80% on each step does not mean you’re done… -W
61
Outline
• Announcements• Recap
– With a little more on word senses
• More discussion: what exactly is subjectivity, sentiment and polarity?– Annotating a corpus for subjectivity– Fine-grained sentiment for reviews
• More distinctions:– Agreement and discourse
62
63
Everyone knows that dragons don't exist. But...
- Stanislaw Lem, “The Cyberiad”
64
(General) Subjectivity Types[Wilson 2008]
Other (including cognitive)Note: similar ideas:polarity, semantic orientation, sentiment
ICWSM 2008 65
PDTB
[In that suit, the SEC accused Mr. Antar of engaging in a "massive financial fraud" to overstate the earnings of Crazy Eddie, Edison, N.J., over a three-year period. ARG1]IMPLICIT_CONTRAST [ Through his lawyers, Mr. Antar has denied allegations in the SEC suit and in civil suits previously filed by shareholders against Mr. Antar and others. ARG2]
Contrast between the SEC accusing Mr. Antar of something, and his denying the accusation
66
Subjectivity
In that suit, the SEC [[accused SENTIMENT-NEG] Mr. Antar of engaging in a "massive financial fraud" to overstate the earnings of Crazy Eddie, Edison, N.J. ARGUING-POS], over a three-year period.
Through his lawyers, Mr. Antar [has denied AGREE-NEG] allegations in the SEC suit and in civil suits previously filed by shareholders against Mr. Antar and others.
Two attitudes combined into one large disagreement between two parties
ICWSM 2008 67
Subjectivity
In that suit, the SEC [[accused SENTIMENT-NEG] Mr. Antar of engaging in a "massive financial fraud" to overstate the earnings of Crazy Eddie, Edison, N.J. ARGUING-POS], over a three-year period.
Through his lawyers, Mr. Antar [has denied AGREE-NEG] allegations in the SEC suit and in civil suits previously filed by shareholders against Mr. Antar and others.
Subjectivity: arguing-pos and agree-neg with different sources; Hypothesis: common with contrast. Help recognize the implicit contrast.
68
69
George Orwell
70
71
72
73
Where do we look for…?
Sentiment/Subjectivity
• Individual words (“nice”, “comfortable”)
• Phrases (“slow service”)• Sentences• Documents• Genres
– RottenTomatos vs IMBD plot summaries
Coherence
• Between words– Cooccurence– Relations in WordNet
• Between sentences– Proximity– Discourse structure
• Between documents– Hyperlinks, references to
entities– Agreement/disagreement
74