Data Visualization at Twitter

transcript

Krist Wongsuphasawat /@kristw

visualizationat Twitter

Bangkok, Thailand

Computer EngineerBangkok, Thailand

Chulalongkorn University

Programming + Soccer

M.S. in Computer ScienceUniv. of Maryland

PhD in Computer ScienceUniv. of MarylandInformation Visualization

IBMMicrosoft

Sr. Data Visualization ScientistTwitter

at Twitterdata visualization

visualizationat Twitterdata

visat Twitterdata

at Twitterdata

“Tweets”

at Twitterdata

“Tweets”

#eventsWorld Cup ElectionOscars

TV Shows New Year

Breaking news

EarthquakeSuper Bowl

Protest

at Twitterdata

“Tweets”

TV Shows New Year

Breaking news

Protest

#curiositySleep patternHuman behaviorLanguage …

at Twitterdata

“Tweets”

TV Shows New Year

Breaking news

Protest

What could we learn from the Tweets?

#curiositySleep patternHuman behaviorLanguage …

visat Twitterdata

“Tweets”

Tell stories about an event, Pursue curiosity or inspiration

visat Twitterdata

“Tweets”

Tell stories about an event, Pursue curiosity or inspiration

(with deadline)

Challenge accepted

visat Twitterdata

“Tweets”

Get data

Having all TweetsHow people think I feel.

How people think I feel. How I really feel.

Having all Tweets

• Too much data

• Want only relevant Tweets

• hashtag: #BRA

• keywords: “goal”

• Need to aggregate & reduce size

• Long processing time (hours)

Challenges

Hadoop ClusterVertica

Pig / Scalding (slow) SQL

Data Storage

Workflow

Data Storage

Workflow

Data Storage

Smaller datasetYour laptop

Workflow

Data Storage

Final dataset

Tool node.js / python / excel (fast)

Your laptop

Workflow

Smaller dataset

visat Twitterdata

“Tweets”

Get data

Visualize

• Peek into data

• Check data & test ideas

• Decide how to visualize

• Guided by data type

• Choose tools

• Start building

Visualize

• Peek into data

• Check data & test ideas

• Decide how to visualize

• Guided by data type

• Choose tools

• Start building

Visualize

Tableau Yeoman

(+ media)photos, videos

Where? When?

GEO TIME

Where? When?

GEO TIME

Visualize Data

Where? When?

GEO TIME

Visualize Data

Time Tweets/second

Time Tweets/second + Annotation

http://www.flickr.com/photos/twitteroffice/5681263084/

Where? When?

GEO TIME

Visualize Data

GeoHeatmap

Low density

High density

GeoSan Francisco

flickr.com/photos/twitteroffice/8798020541

Low density

High density

GeoSan Francisco

Rebuild the world based on

tweet volumes

twitter.github.io/interactive/andes/

Where? When?

GEO TIME

Visualize Data

www.wordle.net

Some experiments during World Cup

www.wordle.net

Word cloud of Tweets right after the 1st goal

Text Word cloud of Tweets right after the 1st goal

www.wordle.net

It was an “own” goal.

Text WordTree [Wattenberg & Viégas 2008]www.jasondavies.com/wordtree

www.jasondavies.com/wordtree

Where? When?

GEO TIME

Visualize Data

Time + Geo

blog.twitter.com/2011/global-pulseyoutu.be/SybWjN9pKQk

Japan Earthquake 2011

Time + Geo Tweet pattern [Rios & Lin 2012]

Late night

Daytime

Late night

Daytime

Late night

Daytime

Late night

Daytime

Late night

Daytime

Late night

Daytime

Late night

Daytime

Late night

Daytime

Where? When?

GEO TIME

Visualize Data

Geo + Text Real-time Tweet map

most frequent

Gmail was down Jan 24, 2014

Nelson Mandela passed away Dec 5, 2013

Where? When?

GEO TIME

Visualize Data

Time + Text

UEFA Champions League

Biggest tournament for European soccer clubs

Many Tweets during the matches

Dortmund Bayern MunichTeam 1 Team 2

Time + Text

Dortmund Bayern Munich

Count Tweets mentioning the teams every minute

Team 1 Team 2

Time + Text

Time + Text UEFA Champions League

+ “goal” count + context

+ “offside”

+ players

A B C D

Competition Tree

A B C D

Competition Tree

A B C D

Competition Tree

vs + =

Where? When?

GEO TIME

Visualize Data

Time + Text + Geo State of the Union

twitter.github.io/interactive/sotu2014

1) timeline + topic from Tweets

4) Density map of Tweets about selected topic

3) Volume of Tweets by topics

during selected part of the SOTU

2) context (speech)

twitter.github.io/interactive/sotu2014

Time + Text + Geo State of the Union

World Cup 2014Time + Text

Time + Text + Geo World Cup 2014

Where? When?

GEO TIME

Visualize Data

Where? When?

GEO TIME

Visualize Data

+Non-Twitter data

CONTEXT

Time + Text New Year 2014

Time + Text + Geo (c) New Year 2014

twitter.github.io/interactive/newyear2014/

visat Twitterdata

“Tweets”

Get data

Visualize

visat Twitterdata

“Tweets”

Get data

Visualize

Evaluate

visat Twitterdata

“Tweets”

Get data

Visualize

Evaluate

Iterate!

• Self

• Peer feedback

• Non team members / Potential audience

Evaluation

visat Twitterdata

Get data

Visualize

Evaluate

visat Twitterdata

Get data

Visualize

Evaluate

big data => small data

visat Twitterdata

Get data

Visualize

Evaluate

big data => small data

What? Where? When?

visat Twitterdata

Get data

Visualize

Evaluate

big data => small data self, peer, external

What? Where? When?

visat Twitterdata

“Tweets”

Get data

Visualize

Evaluate

What? Where? When?

visat Twitterdata

“Tweets”

Get data

Visualize

Evaluate

What? Where? When?• users • followers graph • logs • etc. !

• derived data: language, sentiment

visat Twitterdata

“Tweets”

Get data

Visualize

Evaluate

What? Where? When?Who? …• users

• followers graph • logs • etc. !

visat Twitterdata

“Tweets”

Get data

Visualize

Evaluate

(with deadline)

visat Twitterdata

“Tweets”

Get data

Visualize

Evaluate

• derived data: language, sentiment @kristw / https://interactive.twitter.com

(with deadline)

visat Twitterdata

“Tweets”

Get data

Visualize

Evaluate

• derived data: language, sentiment @kristw / https://interactive.twitter.com

(with deadline)

+ visualizations by @philogb, @miguelrios & @trebor

Questions?

visat Twitterdata

“Tweets”

Get data

Visualize

Evaluate

• followers graph • logs • etc.

@kristw / https://interactive.twitter.com

(with deadline)

+ visualizations by @philogb, @miguelrios & @trebor

Thank you

Data Visualization at Twitter

Data & Analytics