Post on 17-Jun-2020
transcript
Since 2013-02-20 / Copyright 2013 NTT Communication Science Laboratories. All Rights Reserved.
Social curation as corpora for large-
scale multimedia content analysis
Akisato Kimura
NTT Communication Science Laboratories
akisato@ieee.org @_akisato
Acknowledgment
Social curation as corpora for large-scale multimdia content analysis 2
• I really appreciate the following collaborators
for their great commitments for this work.
Prof. Kevin Duh
@ NAIST
Dr. Katsuhiko
Ishiguro
(石黒勝彦)
Mr. Koh
Takeuchi
(竹内孝)
Ms. Kaori
Kataoka
(片岡香織)
Summary
Social curation as corpora for large-scale multimdia content analysis 3
• Social curation has a great potentialfor analyzing large-scale multimedia contents.
1. Curated contents share a specific context, unlike common social media contents.
2. Quite a large amount of collection is
freely available.
Agenda
Social curation as corpora for large-scale multimdia content analysis 4
1. What is “Social Curation”?
2. Why is “Social Curation” emerging?
3. How does “Social Curation” work?
4. What is the next?
Since 2013-02-20 / Copyright 2013 NTT Communication Science Laboratories. All Rights Reserved.
What is “Social Curation”?
What is “curation”?
Social curation as corpora for large-scale multimdia content analysis 6
(From Wikipedia)
• Traditionally, a curator of a cultural heritage
institution (e.g., gallery, museum, library or
archive) is a content specialist responsible for an
institution's collections and involved with the
interpretation of heritage material.
Standard flow of social media
Social curation as corpora for large-scale multimdia content analysis 7
Creators generate and
upload contents to social
services
Consumers enjoy
created contents
Process of social curation
Social curation as corpora for large-scale multimdia content analysis 8
Creators generate and
upload contents to social
services
Curator picks up
contents and summarizes
in a list
Consumers enjoy
summarized lists
1. What is “Social Curation”?
Curators : the significance
Social curation as corpora for large-scale multimdia content analysis 9
Curator picks up
contents and summarizes
in a list
1. What is “Social Curation”?
Curator makes significant differences
from existing social media contents.
Statistics
Social curation as corpora for large-scale multimdia content analysis 10
“Pew Internet : Social Networking (full detail)“
– 46% of adult internet users post original photos or videos
online that they themselves have created. We call them creators.
– 41% of adult internet users take photos or videos that they
have found online and repost them on sites designed for sharing images
with many people. We call them curators.
– Overall, 56% of internet users do at least one of the creating or curating
activities we studied and 32% of internet users do both
creating and curating activities.
(Cf. http://pewinternet.org/Commentary/2012/March/Pew-Internet-Social-Networking-full-detail.aspx)
Example 1 (in Japan)
Social curation as corpora for large-scale multimdia content analysis 11
(cf. http://matome.naver.jp/odai/2134491562701539701)
• Specialized to Twitter messages
• One of the most popular
curation services in Japan
1. What is “Social Curation”?
Example 2 (in Japan)
Social curation as corpora for large-scale multimdia content analysis 12
(cf. http://matome.naver.jp/odai/2134491562701539701)
• One of the most popular curation services in Japan
• Can deal with various multimedia contents
(microblogs, news, images, music, videos…)
• Provide incentive compensation plans for curators
1. What is “Social Curation”?
Example 3
Social curation as corpora for large-scale multimdia content analysis 13
(cf. http://storify.com/weatherchannel/meteor-slams-earth-injuring-hundreds)
• Quite similar to Naver Matome,
but more sophisticated
• Can deal with
various multimedia contents
• One of the most popular
curation
services in the world
1. What is “Social Curation”?
Example 4
Social curation as corpora for large-scale multimdia content analysis 14
• Specialized to images
• One of the most emerging curation
services in the world
1. What is “Social Curation”?
(cf. http://pinterest.com/akisatoo/)
Example 5
Social curation as corpora for large-scale multimdia content analysis 15
(cf. http://www.pearltrees.com/#/N-u=1_334641&N-p=23176992&N-s=1_3109322&N-fa=3109322&N-f=1_3109322)
1. What is “Social Curation”?
Lots of curation services available
Social curation as corpora for large-scale multimdia content analysis 16
(Cf. http://www.youbrandinc.com/ultimate-lists/ultimate-list-content-curation-tools-platform/)
1. What is “Social Curation”?
Social Curation Summit
Social curation as corpora for large-scale multimdia content analysis 17
1. What is “Social Curation”?
(Cf. http://www.mediabistro.com/socialcurationsummit/)
Since 2013-02-20 / Copyright 2013 NTT Communication Science Laboratories. All Rights Reserved.
Why is “Social Curation” emerging?
Win-win-win relationships
Social curation as corpora for large-scale multimdia content analysis 19
[Curators]
• Convenient as a scrap board
• Useful for self-branding and
(stealth) marketing
Win-win-win relationships
Social curation as corpora for large-scale multimdia content analysis 20
[Consumers]
• Avoid being drown in a sea of
social media
• Enjoy more focused stories
Win-win-win relationships
Social curation as corpora for large-scale multimdia content analysis 21
[Service providers]
• Probe individual preferences
& “hidden voices” of users
Service providers
Social curation as a second media
Social curation as corpora for large-scale multimdia content analysis 22
• Social curation
= social media as sensors + focused storytelling
Social mediaSocial curation
Mass media
Automatic social curation
Social curation as corpora for large-scale multimdia content analysis 23
(Cf. http://paper.li/g9ine/1307853228)
Automatic social curation (in Japan)
Social curation as corpora for large-scale multimdia content analysis 24
(Cf. http://gunosy.com)
Uniqueness of Japanese culture
Social curation as corpora for large-scale multimdia content analysis 25
• Japanese social media users are used to be
familiar to social curation tools as “まとめサイト”.
• But we haven’t noticed they are social curation
services, we regarded them as a kind of SNSs.
Social media as corpora
Social curation as corpora for large-scale multimdia content analysis 26
• Feasible to exploit them as sensors
for real-world events and internet memes
2. Why is “Social Curation” emerging?
Sadilek+ “Modeling Spread of
Disease from Social Interactions,”
Proc. ICWSM12
(Best Paper Award candidates)
(Cf. http://www.youtube.com/watch?v=3S2rq2SKTSw)
Social curation as corpora
Social curation as corpora for large-scale multimdia content analysis 27
1. Curated contents are manually selected.
– Only “credible” contents survive in the process.
2. Why is “Social Curation” emerging?
Social curation as corpora
Social curation as corpora for large-scale multimdia content analysis 28
2. Curated contents share the same context.
– Effectively present curator’s interest or perspective.
2. Why is “Social Curation” emerging?
(Cf. http://storify.com/cbccommunity/how-has-texting-changed-your-life)
Since 2013-02-20 / Copyright 2013 NTT Communication Science Laboratories. All Rights Reserved.
How does “Social Curation” work?
Content analysis with social curation
Social curation as corpora for large-scale multimdia content analysis 30
• Despite the excitement of social curation,
few researches have been performed so far.
– Most previous work dealing with social curation
comes from sociology, not computer science.
– Most previous work from CS communities handled
social media contents, not socially curated contents.
• Social curation has so much room to grow
as a promising research topic!!
3. How does “Social Curation” work?
3 aspects for handling social curation
Social curation as corpora for large-scale multimdia content analysis 31
Data resourcesKnowledge &
findingsMethods
The most significant aspect
among all the aspects.Difference resources yield
difference analysis results.
Any sophisticated methods
aren’t required, since data
is clean and well organized.
2 policies for social curation analysis
Social curation as corpora for large-scale multimdia content analysis 32
• Wisdom or diversity of crowdsVegetables
Face
Green Fruits
FunnyStrange
Our contributions
Social curation as corpora for large-scale multimdia content analysis 33
Togetter Pinterest
Wisdom of
crowds
Finding topic-wise
power users[Takeuchi+ IBIS12]
Estimating
image interestingness [Ishiguro+ ICDM12]
Image context discovery[Kimura+ to appear]
Diversity,
personalization
Assisting social curation [Duh+ ICWSM12]
User preference analysis[Kataoka+ PRMU13]
Policies Data resources
Assisting social curation
Social curation as corpora for large-scale multimdia content analysis 34
3. How does “Social Curation” work?
Demo video
Social curation as corpora for large-scale multimdia content analysis 35
Contributions
Social curation as corpora for large-scale multimdia content analysis 36
• Data
– Togetter
• Finding
– Curated contents are supervised corpora.
– This can be acquired through large corpus analysis.
• Method
– Learning to rank : popular IR technique
Corpus analysis of Togetter
Social curation as corpora for large-scale multimdia content analysis 37
3. How does “Social Curation” work?
# curated contents in a listsufficiently long for analysis
# users involved in a listInvolves several users,
only social curation could capture it.
Median40 tweets
90%250 tweets
Median6 users
90%60 users
Corpus analysis of Togetter
Social curation as corpora for large-scale multimdia content analysis 38
3. How does “Social Curation” work?
Occupied by curators’ tweets
Curators gathers lots of tweets
by other people
Findings from the stats
Social curation as corpora for large-scale multimdia content analysis 39
• Lists are very diverse
– in terms of size, topic, and purpose.
• Lists are rather large and elaborate
– by curators’ manual efforts.
• Lists are probably suitable as corpora
– for analyzing microblog messages.
3. How does “Social Curation” work?
Assisting social curation
Social curation as corpora for large-scale multimdia content analysis 40
• Content discovery is time-consuming
• Let curators focus on filtering & presentation
3. How does “Social Curation” work?
retrieve
tweet
tweet
Partially Curated List(queries)
more?
aggregate
Rankingmanual
inspection
tweet
tweet
tweet
tweet
Twitter Timelines
suggest new contents
Finding topic-wise power users
Social curation as corpora for large-scale multimdia content analysis 41
3. How does “Social Curation” work?
Demo (snapshot)
Social curation as corpora for large-scale multimdia content analysis 42
Contributions
Social curation as corpora for large-scale multimdia content analysis 43
• Data
– Togetter
• Findings
– Lists are diverse, but each list has a specific topic.
– Only informative tweets would be gathered into a list.
– Topically credible users frequently appear in lists.
• Method
– Novel extension of non-negative matrix factorization
Method in detail
Social curation as corpora for large-scale multimdia content analysis 44
• Formulated as a factorization of
multiple matrices correlated to each other
Factorization
X
Y
Z
lists
users
words
popularity
0.0017% non-zero
1.23% non-zero
1.82M
160K
230K 2
W
A
BH
lists
users
words
Power users and lists
about a topic “space”
Power users and lists
about a topic “Hadoop”
Qualitative analysis
Social curation as corpora for large-scale multimdia content analysis 45
Authority Profile #follow
hadukino Space and astronomy n China 1,321
ohnuki_tsuyoshi Space developer 5,477
5thstar Space pilot finalist 2,386
ShinyaMatsuura Writer on space development 18,609
sctracker Spacecraft tracker 847
Topical words
宇宙(space) 衛星(satellite) 地球(earth) ロケット(rocket) 軌道(orbit) JAXA はやぶ
さ(Hayabusa) 探査(exploration) 打ち上げ(launch) 飛行(flight) 移籍(transfer) あかつ
き(Akatsuki)
Topic 1 “Space”
Authority Profile #follow
understeer Solutions Architect 2,368
yutuki_r Cloud, Hadoop, NoSQL 800
kimtea NoSQL, Google AppEngine 2,742
repeatedly D, Ruby, Python, Opera 4,081
siena Web techniques and design 1,065
Topical words
使う(use) Hadoop クラウド(cloud) さん(Mr.) AWS ない(not) 話(story)データ(data) これ
(this)いい(good) 開発(development) セッション(session) PHP処理(processing) DB
サーバ(server)
Topic 2 “Hadoop”
Qualitative analysis
Social curation as corpora for large-scale multimdia content analysis 46
Estimating interestingness of images
Social curation as corpora for large-scale multimdia content analysis 47
Contributions
Social curation as corpora for large-scale multimdia content analysis 48
• Data
– Togetter
• Domain knowledge & findings
– A lot of lists contained an image content.
– View counts of images can be regarded as a
quantitative and measurable proxy of interestingness.
• Methods
– Dispense with expensive CV/PR techniques
Formulation
Social curation as corpora for large-scale multimdia content analysis 49
• View count prediction by curation information
3. How does “Social Curation” work?
Results
Social curation as corpora for large-scale multimdia content analysis 50
• Easily available social curation features
performed greatly.
3. How does “Social Curation” work?
Individual preference analysis
Social curation as corpora for large-scale multimdia content analysis 51
3. How does “Social Curation” work?
Contributions
Social curation as corpora for large-scale multimdia content analysis 52
• Data
• Domain knowledge & findings
– Every curated image collection (board) reflects
individual preference of a user.
• Methods
– Saliency-based object detection
– Image clustering based on a topic model (LDA)
Pinterest vs. Flickr
Social curation as corpora for large-scale multimdia content analysis 53
3. How does “Social Curation” work?
Service style (1) Content sharing
(2) Social network
(1) Social curation
(2) Social network
Network structure User-centric network Content-centric network
Resources Uploading original
contents is a major style.
Most contents come from
outside, including Flickr.
Popularity Still a mainstream, but in
a falling trend.
Emerging, becoming a
mainstream.
Research progress Frequently used in CS
researches.
Few researches have
been performed.
Framework
Social curation as corpora for large-scale multimdia content analysis 54
• Corpora for personalized image retrieval
3. How does “Social Curation” work?
Qualitative results
Social curation as corpora for large-scale multimdia content analysis 55
• Confirm many boards reflect user preferences
3. How does “Social Curation” work?
Images in a board (probably reflecting user preferences)
Proposed method
(MAP estimation)
Feature-based NN
Random sampling
Application examples
Social curation as corpora for large-scale multimdia content analysis 56
• Query = “Christmas”Storage (board) of User 1
Retrieval results for User 1
Storage (board) of User 2
Retrieval results for User 2
Since 2013-02-20 / Copyright 2013 NTT Communication Science Laboratories. All Rights Reserved.
What is the next?
Future of social curation services
Social curation as corpora for large-scale multimdia content analysis 58
• We are now at the dawn of social curation
– But, it’s a “social curation” bubble, coming from
over-expectation for a new trend.
– A selection process will start soon.
• The key would be inter-service partnerships.
– Remember, why service providers do it :
Probe user preferences & “hidden voices”
– Fundamental limitation of a single service
4. What is the next?
Future of social curation services
Social curation as corpora for large-scale multimdia content analysis 59
• Movements of web giants cannot be ignored
– Information stored in social curation services would
be really demanding for portal sites, SNSs and ECs.
• Will become a segment-market
1. Marketing : Collaborating with ads and E-commerce.
2. Portal : Entertaining users
4. What is the next?
Future of social curation as corpora
Social curation as corpora for large-scale multimdia content analysis 60
• At the predawn in CS communities
– Finding a promising corpus yields 3 papers :-)
(Analysis, method, application)
• There is a high bar to approach
– Requires various knowledge and techniques in a wide
range of research areas
• NLP, IP, SP, IR, ML,
network analysis, human activity etc.
4. What is the next?
Future of social curation as corpora
Social curation as corpora for large-scale multimdia content analysis 61
• The fastest way to learn domain knowledge isto be a user.
• Multi-platform analysis would be promising.
– Fundamental limitation of a single platform.
– The point is the way of identifying users.
• Which do you want, wisdom or diversity?
– Wisdom : An alternative to crowdsourcing
– Diversity : Analyzing contents from multiple views
4. What is the next?
Summary
Social curation as corpora for large-scale multimdia content analysis 62
• Social curation has a great potentialfor analyzing large-scale multimedia contents.
1. Curated contents share a specific context, unlike common social media contents.
2. Quite a large amount of collection is
freely available.
Social curation as corpora for large-scale multimdia content analysis 63
• The presentation slides will be uploaded soon.
http://bit.ly/prmu2013FebCuration
Akisato Kimura, Ph.D
NTT Communication Science Laboratories
akisato@ieee.org @_akisato
Social curation as corpora for large-scale multimdia content analysis 64
Since 2013-02-20 / Copyright 2013 NTT Communication Science Laboratories. All Rights Reserved.
Appendix
参考文献
Social curation as corpora for large-scale multimdia content analysis 66
• Duh+ “Creating stories: Social curation of Twitter messages,” Proc. ICWSM2012.
• Takeuchi+ “Stacked non-negative matrix factorization for social media analysis,” Proc. IBIS2012 (In Jap).
• Ishiguro+ “Towards automatic image understanding and mining via social curation,” Proc. ICDM2012.
• Kataoka+ (Yesterday’s talk, In Jap).