Melt Barcelona Variazioni Vuorikari Final

transcript

http://lre.eun.org

Collaborative Content Annotation in

Multilingual Europe

Riina Vuorikari, Frans Van AsscheEuropean Schoolnet

1st VARIAZIONI WorkshopBarcelona Nov 29 2007

http://lre.eun.org

Multilingual Europe* motivation and context* federated architecture

* co-existence of LOM and

unstructured metadata* envisaged enrichment services

http://lre.eun.org

Multilingual Europe

* what does multi-linguality mean for tags and how do

users deal with it?

http://lre.eun.org

Motivation of this work

European education, especially that of K-12 education,

is inherently multi-lingual and multi-cultural.

http://lre.eun.org

http://lre.eun.org European Schoolnet

Established a network among European Educational Authorities

in 1996

Currently 27 members from more than EU

http://lre.eun.org

...times are changing

http://lre.eun.org

Proliferation of web-based servicesfor sharing digital items

http://lre.eun.org

Proliferation of web-based servicesfor sharing digital items

http://lre.eun.org

New challenges for repositories

European teachers have access to multiple repositories of digital learning resources by – Educational Authorities, – publishers, – other teachers,.

Users become more demanding and expect services that are seen elsewhere (rss, personalised feed-selections, bookmarks, community rankings ..)

http://lre.eun.org

Since 1999 EUN's goal:

to facilitate the access to multi-lingual

learning resources repositories

http://lre.eun.org

EUN + Federation of LORs

Learning Resources Exchange

http://lre.eun.org

Long process in semantic interoperability since 1999

– First DC based Application Profile in 2001

Semantic interoperability for K-12

http://lre.eun.org

LOM based Application Profile v 3.0LOM based Application Profile v 3.0http://insight.eun.org/intern/shared/data/insight/lre/AppProfilev3p0.pdf

http://lre.eun.org

Multi-lingual thesaurus in more than 15 languages

http://lre.eun.org

Challenge for users

End-users (e.g. teachers) have difficulties to discover and find resources from educational repositories

– Metadata does not always match search terms

Locating content across linguistic and national borders within Europe has proven hard

– Despite the use of a multilingual Thesaurus and controlled vocabularies

http://lre.eun.org

New Mission Critical

MetadataEgology for Learning Technologies

=> co-existence of structured and unstructured metadata

=> by expert indexers and end-users

http://lre.eun.org

LRE Architecture

http://lre.eun.org

Architecture

http://lre.eun.org

Enrichment: by experts

By expert indexers:

“toolbox will be extended with tools for the development of taxonomies, and with tools for automatic translation and vocabulary management, as well as for collecting attention metadata to track what users actually do with the MELT infrastructure”

Original Metadata

Samgi generated Metadata

instance

Edited by expert indexer

Merged Metadata

http://lre.eun.org

Multi-lingual enrichment

http://lre.eun.org

Collaborative content enrichment in multilingual Europe

Addition to the traditional LOM

By users interacting with the portal, resources and other users

Four main tools:– social bookmarking and tags in multiple

languages – rating of usefulness – pedagogical annotations (used in learning events)– levels of user engagement when interacting with

the system (what is viewed, how many times,..)

http://lre.eun.org

Social bookmarking

To store, organise, share and search bookmarks of web pages. Keep found things found!

Bookmarks are usually public and shared

Users organise bookmarks with informal tags (instead of the traditional folders)

Find like-minded users with similar interest.

http://lre.eun.org

What is a tag?

Metadata externally applied to an item

Can be used for sorting or managing

A hook for aggregating

Provides identifier and/or description

Personal marker by Thomas Vander Wal

http://lre.eun.org

Bookmark is a triple (user, resource, {tags})

http://lre.eun.org

A great navigational aid to discover resources!

http://lre.eun.org

Follow those digital

traces!

http://lre.eun.org

More ways for social navigation

Discover resources through tags in

multiple languages

http://lre.eun.org

by Stiphy

“Social”

trails

visible..

http://lre.eun.org

to flock

http://lre.eun.org

http://lre.eun.org Can tags and bookmarks be used to

connect users cross national and

language borders?

http://lre.eun.org

Social bookmarking in multi-lingual environment

http://lre.eun.org

.. allows new ways to discover both resources and people!

http://lre.eun.org

How do users bookmark?

January 1 to October 31 2007

A post : a triple of (user, item, {tag}) 1022 posts to favourites 142 users 682 different learning resources (items) 1029 multilingual tags recorded to the system,

some of which were reused by users

http://lre.eun.org

How do users bookmark?

0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150

Distribution of posts and inverse Power Law

Column JColumn K

http://lre.eun.org

How do users tag?

Most posts consisted of only one tag

More than half of tags were used just once (54%)

Only about 10% of tags were reused more than twise

No guided tagging in this pilot! This may change.

Percent of tags/postingOnetag 79%2 tags 15%3 tags 4%4 tags 1%5 tags 0.39%6 tags 0.23%7 tags 0.08%

http://lre.eun.org

Are tags found useful?

LO Type Lang Keyword % of votes1 Tag En healthy meal 1 0 0%

1 Thesaurus En health education 7 7 %

2 Tag En EU 7 7 %

2 Tag Pl Europa 62 %

3 Thesaurus En physics 85%

3 Thesaurus En mathematics 92 %

4 Tag En -GeoGebra program 62 %

5 Thesaurus En Geography 92 %

5 Thesaurus En Island 69%

Vuorikari, R., Ochoa, X., Duval, E. Analysis of User Behavior on MultilingualTagging of Learning Resources. In Workshop proceedings of the EC-TEL conference: SIRTEL07 (EC-TEL ’07) (Crete, Greece, September 17-20, 2007)

http://lre.eun.org

Are tags found useful?

LO Type Lang Keyword % of votes1 Tag En healthy meal 1 0 0%

1 Thesaurus En health education 7 7 %

2 Tag En EU 7 7 %

2 Tag Pl Europa 62 %

3 Thesaurus En physics 85%

4 Tag En -GeoGebra program 62 %

5 Thesaurus En Geography 92 %

5 Thesaurus En Island 69%

Tags, produced with no outlay, show an encouraging and

potential gain in overall usefulness!

Vuorikari, R., Ochoa, X., Duval, E. Analysis of User Behavior on MultilingualTagging of Learning Resources. In Workshop proceedings of the EC-TEL conference: SIRTEL07 (EC-TEL ’07) (Crete, Greece, September 17-20, 2007)

http://lre.eun.org

Does the language matter?

Need for better ways to identify the language– Give rules (if the user first preferred languages is..,

then..)– Automate the recognition of languages– Out-source it to users

http://lre.eun.org

“Travel well” tags

About 15% of tags contain a general term, a name, place, etc. that is easily understood without translation

e.g. AIDS, software, EU, Euroopa, Europa, europe, Evropa, geograafia, Pythagoras, etc.

http://lre.eun.org

What's the point of travel well tags?

If those tags need no translation or language filtering to be understood, and

..if they can be identified:

We can be sure to show at least some tags to users – whose language preferences we don't know, and – in whose language there are no tags or keywords

available.

http://lre.eun.org

User engaging with resources

Steps taken:- views page- views metadata- bookmarks and

tags- rates

- what about the actual use?

http://lre.eun.org

Semantic analysis of tags

Factual tags 63%(Golder: item topics, kinds of item, category refinements)

Subjective tags 29%( Golder: item qualities)

Personal tags 3% (Golder: item ownership, self-reference, tasks organisation)

5% other

Sen et al. (2006).

http://lre.eun.org

Why tag categories? In Sen et al. (2006) it

was found that tags of different categories can be useful for different tasks

In our case it is too early to say anything, but ...we'll have an eye on it!

http://lre.eun.org

social bookmarks Metadata LOM tags folksonomy social tagging

multi-linguality social classificationthanks! for your attention

learning resources user communities

discover resources and itemsquestions?

teachers social navigationsocial tracespaths, trails

http://www.cs.kuleuven.ac.be/~hmdb/infovis/calibrate/calibrate.htmlflock

Melt Barcelona Variazioni Vuorikari Final

Technology