QUOting the
The structure of political media coverageas revealed by quoting patterns:
POTUS*Vlad Niculae (me)
*Caroline Suen*Justine Zhang
Cristian Danescu-Niculescu-Mizil Jure Leskovec
Cornell (MPI-SWS) Stanford Cornell (Stanford, MPI-SWS) Cornell (MPI-SWS)Stanford
1
equal contribution{
The press claims to befair and balanced.
Tea Party Anti-Tax March, Sep 12, 2010. Capital Mall, DC
Photo Credit: Bob Simmons license CC-BY-NC-ND 2.0
America’s trust in media is at
an all time low.(Source: Gallup poll, Sept. 17 2014)
2
“Media Bias” is subjective
Source: Gallup poll, Sept. 17 2014
Liberals
Conservatives
0% 25% 50% 75% 100%
Too liberal Too conservative
(Out of the ones who say it’s biased.)
3
“Media Bias” is subjectivemultidimensional
Predetermined dimension Proxy to slant
[Dalton et al, 1998][Peake, 2007]
Democrat/Republican Endorsement
[Schiffer, 2006] Democrat/Republican Congress
[Gentzkow et al, 2010][Groseclose et al, 2005]
[Lin et al, 2011]Democrat/Republican Congress
[Ho et al, 2008] Liberal/Conservative Supreme Court Justices
& other theoretically possible dimensions… 4
hard to quantify
Large Scale,
Unsupervised,
Without fixing a bias dimension.
5
We want a way to address media bias:
Insight: Quoting patterns.
Photo Credit: David Suter
Selection BiasNo space to report everything.
Outlets must makeconscious choices.
Fair world:” are the choices based only on newsworthiness?
Or is there something else behind?6
“
7
Photo Credit: DonkeyHotey license CC-BY 2.0 8
With or without this Congress, I will keep taking actions that help the economy grow. But I can do a whole lot more with your help. Because when we act together, there's nothing the United States of America can't achieve.
Barack Obama, 2012 State Of The Union address.9
—Right Wing News
With or without this Congress, I will keep taking actions that help the economy grow. But I can do a whole lot more with your help. Because when we act together, there's nothing the United States of America can't achieve.
10
—CNN, Huffington Post
With or without this Congress, I will keep taking actions that help the economy grow. But I can do a whole lot more with your help. Because when we act together, there's nothing the United States of America can't achieve.
11
Are quoting patterns systematic?
Do they correspond to intuition?
Can language characterize the bias?
12
Are quoting patterns systematic?
Do they correspond to intuition?
Can language characterize the bias?
13
Quoting graph
275 online media outlets.(newspapers, blogs, …)
54K Obama quotes in 6 years.
Edge = outlet reported quote.230K edges.
)The Adjacency
Matrix(X=
14
Are quoting patterns systematic?
Are quoting patterns systematic?
Can we predict quoting?If we can predict quoting choices, it means there is a pattern.
15
Are quoting patterns systematic?
Can we predict quoting?Leave out 250K entries of X.
16
Are quoting patterns systematic?
Can we predict quoting?Leave out 250K entries of X.
17
Matthews Correlation Coefficient
0% 14% 28%
Are quoting patterns systematic?
Can we predict quoting?Leave out 250K entries of X.
18
Matthews Correlation Coefficient
0% 14% 28%
Reca
ll
0%
17%
33%
50%
Precision
0% 12,5% 25% 37,5% 50%
Are quoting patterns systematic?
Can we predict quoting?
19
Matthews Correlation Coefficient
0% 14% 28%
Reca
ll
0%
17%
33%
50%
Precision
0% 12,5% 25% 37,5% 50%
Leave out 250K entries of X.
In a “fair world” without bias: - popular quotes get cited more,
Are quoting patterns systematic?
Can we predict quoting?Leave out 250K entries of X.
In a “fair world” without bias: - popular quotes get cited more,
20
Quote Popularity
Matthews Correlation Coefficient
0% 14% 28%
Reca
ll
0%
17%
33%
50%
Precision
0% 12,5% 25% 37,5% 50%
Are quoting patterns systematic?
Can we predict quoting?Leave out 250K entries of X.
In a “fair world” without bias: - popular quotes get cited more, - and big outlets quote more.
21
Quote Popularity
Matthews Correlation Coefficient
0% 14% 28%
Reca
ll
0%
17%
33%
50%
Precision
0% 12,5% 25% 37,5% 50%
Are quoting patterns systematic?
Can we predict quoting?
22
Quote Popularity
+ Outlet Size
Matthews Correlation Coefficient
0% 14% 28%
Reca
ll
0%
17%
33%
50%
Precision
0% 12,5% 25% 37,5% 50%
Leave out 250K entries of X.
In a “fair world” without bias: - popular quotes get cited more, - and big outlets quote more.
Are quoting patterns systematic?
Can we predict quoting?
23
Quote Popularity
+ Outlet Size
Matthews Correlation Coefficient
0% 14% 28%
Reca
ll
0%
17%
33%
50%
Precision
0% 12,5% 25% 37,5% 50%
Matrix completion by low-rank factorization
Are quoting patterns systematic?
Can we predict quoting?
24
Quote Popularity
+ Outlet Size
Matthews Correlation Coefficient
0% 14% 28%
Reca
ll
0%
17%
33%
50%
Precision
0% 12,5% 25% 37,5% 50%
Matrix completion by low-rank factorization
X ≈ X̂ = U ⋅ VT
Are quoting patterns systematic?
Can we predict quoting?
25
Quote Popularity
+ Outlet Size
Matthews Correlation Coefficient
0% 14% 28%
Reca
ll
0%
17%
33%
50%
Precision
0% 12,5% 25% 37,5% 50%
Matrix completion by low-rank factorization
)( )( )(≈
outlet space quote space
X ≈ X̂ = U ⋅ VT
Are quoting patterns systematic?
Can we predict quoting?
26
Matrix completion by low-rank factorization
)( )( )(≈
outlet space quote space
X ≈ X̂ = U ⋅ VT
Quote Popularity
+ Outlet Size
MatrixCompletion
Matthews Correlation Coefficient
0% 14% 28%
Reca
ll
0%
17%
33%
50%
Precision
0% 12,5% 25% 37,5% 50%
Are quoting patterns systematic?
Do they correspond to intuition?
Can language characterize the bias?
27
Are quoting patterns systematic?
Do they correspond to intuition?
Can language characterize the bias?
28
Yes! There is systematicity
beyond newsworthiness.
Are quoting patterns systematic? Yes!
Do they correspond to intuition?
Can language characterize the bias?
29
)( )( )(≈
outlet space quote space
X ≈ X̂ = U ⋅ VT
30
Do they correspond to intuition?
)( )( )(≈
outlet space quote space
X ≈ X̂ = U ⋅ VT
31
Do they correspond to intuition?
-0,16
-0,08
0
0,08
0,16
0,00 0,03 0,07 0,10 0,13
The first two latent dimensions:
32
)(outlet space
X̂ = ⋅VT
Do they correspond to intuition?
-0,16
-0,08
0
0,08
0,16
0,00 0,03 0,07 0,10 0,13
The first two latent dimensions:
33
)(outlet space
X̂ = ⋅VT
Do they correspond to intuition?
-0,16
-0,08
0
0,08
0,16
0,00 0,03 0,07 0,10 0,13
The first two latent dimensions:
National ReviewThe BlazeDaily Caller
PJ MediaTown Hall
Real ClearReasonWeekly Standard
34
)(outlet space
X̂ = ⋅VT
Do they correspond to intuition?
-0,16
-0,08
0
0,08
0,16
0,00 0,03 0,07 0,10 0,13
The first two latent dimensions:
National ReviewThe BlazeDaily Caller
PJ MediaTown Hall
Real ClearReasonWeekly Standard
The Guardian
The BBC
Al Jazeera
The Hindu
35
)(outlet space
X̂ = ⋅VT
Do they correspond to intuition?
-0,16
-0,08
0
0,08
0,16
0,00 0,03 0,07 0,10 0,13
The first two latent dimensions:
BarackObama.com
National ReviewThe BlazeDaily Caller
PJ MediaTown Hall
Real ClearReasonWeekly Standard
Mother Jones
The Nation
Washington MonthlyCrooks & Liars
Daily Kos
The Guardian
The BBC
Al Jazeera
The Hindu
36
)(outlet space
X̂ = ⋅VT
Do they correspond to intuition?
Does it correspond to intuition?
At the top of dimension 2:
Free RepublicLonely ConservativeRight Wing News…
37
Does it correspond to intuition?
At the top of dimension 2:
Free RepublicLonely ConservativeRight Wing News…
38
Does it correspond to intuition?
At the top of dimension 2:
Free RepublicLonely ConservativeRight Wing News…
Around zero on dimension 2:NY PostThe EconomistLA Times…
39
Does it correspond to intuition?
At the top of dimension 2:
Free RepublicLonely ConservativeRight Wing News…
Around zero on dimension 2:NY PostThe EconomistLA Times…
40
Does it correspond to intuition?
At the top of dimension 2:
Free RepublicLonely ConservativeRight Wing News…
Around zero on dimension 2:
Around the bottom of dimension 2:Sydney Morning HeraldOttawa CitizenThe Globe And Mail…
NY PostThe EconomistLA Times…
41
Does it correspond to intuition?
At the top of dimension 2:
Free RepublicLonely ConservativeRight Wing News…
Around zero on dimension 2:
Around the bottom of dimension 2:Sydney Morning HeraldOttawa CitizenThe Globe And Mail…
NY PostThe EconomistLA Times…
42
Are quoting patterns systematic? Yes!
Do they correspond to intuition?
Can language characterize the bias?
43
Are quoting patterns systematic? Yes!
Do they correspond to intuition?
Can language characterize the bias?
44
Kind of. It’s not only
liberal/conservative.
Are quoting patterns systematic? Yes!
Do they correspond to intuition? Kind of.
Can language characterize the bias?
45
Can language characterize the bias?
)( )( )(≈
outlet space quote space
X ≈ X̂ = U ⋅ VT
46
Can language characterize the bias?
)( )( )(≈
outlet space quote space
X ≈ X̂ = U ⋅ VT
47
Can language characterize the bias?
-0,16
-0,08
0
0,08
0,16
0,00 0,03 0,07 0,10 0,1348
(The same axes!)
)(quote space
X̂ = U ⋅
Can language characterize the bias?
-0,16
-0,08
0
0,08
0,16
0,00 0,03 0,07 0,10 0,13
If we don’t work even harder than we did in 2008, then we’re going to have a government that tells the American people, “you’re on your own.”
49
)(quote space
X̂ = U ⋅
Can language characterize the bias?
-0,16
-0,08
0
0,08
0,16
0,00 0,03 0,07 0,10 0,13
The truth is, you could figure out on the back of an envelope how to get this done. The question is one of political will.
If we don’t work even harder than we did in 2008, then we’re going to have a government that tells the American people, “you’re on your own.”
50
)(quote space
X̂ = U ⋅
Can language characterize the bias?
-0,16
-0,08
0
0,08
0,16
0,00 0,03 0,07 0,10 0,13
If we don’t work even harder than we did in 2008, then we’re going to have a government that tells the American people, “you’re on your own.”
The truth is, you could figure out on the back of an envelope how to get this done. The question is one of political will.
By the end of the next year, all U.S. troops will be out of Iraq.
51
)(quote space
X̂ = U ⋅
Can language characterize the bias?
Average sentiment of quotes:
52
(The same axes!)
Can language characterize the bias?
Average sentiment of quotes:
53
(The same axes!)
Can language characterize the bias?
Average sentiment of quotes: Amount of negation in quotes:
54
(The same axes!)
Conclusion
55
Online visualization & data:http://snap.stanford.edu/quotus/
Quoting patterns are systematic:We can predict them.
Conclusion
-0,16
-0,08
0
0,08
0,16
0,00 0,03 0,07 0,10 0,13
56
Online visualization & data:http://snap.stanford.edu/quotus/
Quoting patterns are systematic:We can predict them.
Selection bias is multidimensional.
Language plays a part:Sentiment and negation show alignment.