use cases with semantics & user

interaction in TV & cultural heritage

collections

Lora Aroyo@laroyo

HUMANS & MACHINES TOGETHER

Friday, May 17, 13

Time-consuming:5 time the video duration

Coarse grained: refer to entire video

Professional vocabulary:experts use a specific vocabulary

that is unknown to general audiences

OBSERVATION I:VIDEO ANNOTATIONS

Friday, May 17, 13

People request fragments:broadcast (33%), stories (17%),

fragments (49%)

Finding fragments longer:stories (2x), fragment (3x)

Not finding:35% of clicked results

not found by title or term

OBSERVATION II:SEARCH BEHAVIOR

Friday, May 17, 13

Improve support for fragment search

time-based annotations

bridging vocabulary gap between searcher

& cataloguer

crowdsourcing?

OBSERVATION III:CHANGE NEEDED

Friday, May 17, 13

CULTURAL HERITAGE WITH INVOLVED CONSUMERS &

PROVIDERS

data: open, shared & accessible

infrastructures: interoperable

smart services: relevant content in right context

PART I

Friday, May 17, 13

http://chip-project.org

http://agora.cs.vu.nl

Rijksmuseum Amsterdam

Friday, May 17, 13

http://openimages.eu

http://academia.nl

MUNCH project

CHOICE project

Netherlands Institute for Sound & Vision

Friday, May 17, 13

EuropeanaCultural Search

http://e-culture.multimedian.nl

http://europeana.eu/portal/thoughtlab.html

http://europeana.eu

Friday, May 17, 13

How to use the Crowd

Friday, May 17, 13

http://waisda.nlhttp://www.prestoprime.org/

@waisda

Friday, May 17, 13

Friday, May 17, 13

Friday, May 17, 13

Friday, May 17, 13

Friday, May 17, 13

Two PilotsPilot I Pilot II

months 8 2

videos 612 1544

players 2000 438

tags 420000 172000

Friday, May 17, 13

On the Role of User-Generated Metadata in A/V CollectionsRiste Gligorov et al. KCAP Int. Conference on Knowledge Capture 2011

Friday, May 17, 13

time-basedbernhard

just “tags”

On the Role of User-Generated Metadata in A/V CollectionsRiste Gligorov et al. KCAP Int. Conference on Knowledge Capture 2011

Friday, May 17, 13

user vocabulary8% in professional vocabulary23% in Dutch lexicon89% found on Google

objects (57%)

westminsterabbeyabbeypriestergeestelijken

hekpaardentochtaankomst

koetskroningmensenmassaparadekroon regen

On the Role of User-Generated Metadata in A/V CollectionsRiste Gligorov et al. KCAP Int. Conference on Knowledge Capture 2011

Friday, May 17, 13

user vocabulary8% in professional vocabulary23% in Dutch lexicon89% found on Google

persons (31%)

bernhard

juliana

objects (57%)

On the Role of User-Generated Metadata in A/V CollectionsRiste Gligorov et al. KCAP Int. Conference on Knowledge Capture 2011

Friday, May 17, 13

user vocabulary8% in professional vocabulary23% in Dutch lexicon89% found on Google

locations (7%)

engeland persons (31%)

objects (57%)

On the Role of User-Generated Metadata in A/V CollectionsRiste Gligorov et al. KCAP Int. Conference on Knowledge Capture 2011

Friday, May 17, 13

user vocabulary8% in professional vocabulary23% in Dutch lexicon89% found on Google

locations (7%)

persons (31%)

objects (57%) no events

no scenes

On the Role of User-Generated Metadata in A/V CollectionsRiste Gligorov et al. KCAP Int. Conference on Knowledge Capture 2011

Friday, May 17, 13

just “tags”

On the Role of User-Generated Metadata in A/V CollectionsRiste Gligorov et al. KCAP Int. Conference on Knowledge Capture 2011

Friday, May 17, 13

hye park just “tags”

typos

On the Role of User-Generated Metadata in A/V CollectionsRiste Gligorov et al. KCAP Int. Conference on Knowledge Capture 2011

Friday, May 17, 13

just “tags”

typos

no unique references

bernhard

On the Role of User-Generated Metadata in A/V CollectionsRiste Gligorov et al. KCAP Int. Conference on Knowledge Capture 2011

Friday, May 17, 13

just “tags”

typos

no unique references

beefeater

no synonyms

On the Role of User-Generated Metadata in A/V CollectionsRiste Gligorov et al. KCAP Int. Conference on Knowledge Capture 2011

Friday, May 17, 13

THE QUALITY OF TAGS?

Friday, May 17, 13

Tag gardeningannotations for video, fragments & scenes

adding types & roles to user tags

mapping tags to concepts, disambiguationspelling corrections

single & batch tags/video editing

Friday, May 17, 13

Linking user-generated video annotations to the web of dataMichiel Hildebrand et al. International Conference on Multimedia Modeling 2012

Friday, May 17, 13

Linking user-generated video annotations to the web of dataMichiel Hildebrand et al. International Conference on Multimedia Modeling 2012

group by type (faceted)

Friday, May 17, 13

Linking user-generated video annotations to the web of dataMichiel Hildebrand et al. International Conference on Multimedia Modeling 2012

Friday, May 17, 13

Linking user-generated video annotations to the web of dataMichiel Hildebrand et al. International Conference on Multimedia Modeling 2012

concepts uniquely identified

Friday, May 17, 13

Linking user-generated video annotations to the web of dataMichiel Hildebrand et al. International Conference on Multimedia Modeling 2012

Friday, May 17, 13

Linking user-generated video annotations to the web of dataMichiel Hildebrand et al. International Conference on Multimedia Modeling 2012

background info

Friday, May 17, 13

Friday, May 17, 13

Friday, May 17, 13

Friday, May 17, 13

Install Your Own Waisda?

Friday, May 17, 13

TV WITH INVOLVED CONSUMERS & PROVIDERS

42% UK adults discuss/comment on TV programs while watching (YouGov Deloitte report, Aug 2010)

49% of UK Twitter users use Twitter regularly when watching TV (BBC Audience Research Twitter survey, Aug 2010)

PART II

Friday, May 17, 13

but ...

© VickyBuser, BBC

Friday, May 17, 13

lost in space© RedBee Slides at MIPCube2012

Friday, May 17, 13

demand for new experienceFriday, May 17, 13

http://notube.tv http://www.slideshare.net/NoTubeProject

@notubeproject

© Dan Brickley

Friday, May 17, 13

NoTube’s N-Screen

N-Screendrag & drop, sharing & TV control

drag & drop, shuffle, sharing & TV control

Friday, May 17, 13

NoTube’s User Activitiesaggregate

Friday, May 17, 13

NoTube’s User Interestsprofile

Friday, May 17, 13

NoTube’s Beancounteraggregate, analyze & profile

Friday, May 17, 13

Recommendations in NoTube

• surface interesting, new & relevant programs to individual and group users• combine in a complementary way different statistical & semantic approaches• define metrics for, e.g. serendipity, diversity, relevance

© Dan Brickley

Friday, May 17, 13

• access to basic metadata about programs • links are the basic currency of social media

• URIs for the things you watch • links to related Web entities

• even a small amount of metadata enables interesting apps

Open Web Standards & Data

Friday, May 17, 13

• The move to IP - opportunity to discover what people watch in greater breadth and depth• gather consumers’ viewing behavior & video streams• combine them with enhanced EPG as input for a holistic live-stream data mining analysis

• generate a high-quality linked open dataset (LOD) describing live TV programming• market research on viewing behavior

Friday, May 17, 13

Michiel Hildebrand

Lotte Belice BaltussenJacco van Ossenbruggen

Marteen Brinkerink

Johan OomenGuus Schreiber

Riste Gligorov

Lora Aroyo

THE TEAM

@lottebelice

@johanoomen

@mbrinkerink

@laroyo

@jrvosse

@GuusSchreiber

@McHildebrand

Friday, May 17, 13

THE TEAM

Friday, May 17, 13

• http://waida.nl• http://prestoprime.org• http://notube.tv• http://vista-tv.eu• http://lora-aroyo.org

• @waisda• @prestocenter• @notubeproject• @vistatv• @laroyo

Questions

Friday, May 17, 13