March 23rd 2012

Mitg

lied

de

r H

elm

ho

ltz-G

em

ein

sch

aft

Social Bookmarks and Tags

in Journal Evaluation

| Stefanie Haustein

Overview

social

bookmarking

social

tagging

Journal Perception and Usage

measuring problematic despite standards

only available for usage at own institutions

publishers shut global statistics away

alternative needed to measure worldwide journal usage

bookmarking articles on social bookmarking platforms

indicates journal usage

Introduction: Social Bookmarking in STM

modeled on delicious

allow academic users to

store references online

indexing resources with tags

sharing literature with

other users

finding new content

through other users

Introduction: Social Bookmarking in STM

Retrieval of Bookmarking Data

45 journals

168,109 articles

2004-2008 bookmarks bibliographic data

Retrieval of Bookmarking Data

download was checked and limited to bookmarks to

documents published in 45 journals from 2004 to 2008

13,760 correct bookmarks retrieved

940

10778

2042

BibSonomy

CiteULike

Connotea

Number of bookmarks retrieved

Retrieval of Bookmarking Data

completeness / correctness of bookmarking data

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

journal title DOI ISSN journal abbreviation

Perc

enta

ge o

f bookm

ark

s fo

und b

y

searc

h s

trate

gy

Search stragey

all services BibSonomy CiteULike Connotea

Retrieval of Bookmarking Data

missing DOIs were replaced by CrossRef and manually

bookmarks were matched to the bibliographic entries from

Web of Science via DOI

98.9% of bookmarks matched

2.8% doubles in

user names

between services

results were

combined

145 users

2054 users

313 users

802 articles

8082 articles

1995 articles

BibSonomy

CiteULike

Connotea

Number of unique users and articles

Retrieval of Bookmarking Data

13,608 bookmarks were matched to 10,280 articles

2,441 unique users

0

50

100

150

200

250

300

350

satoshi (322 posts)

bronckobuster (238 posts)rice (234 posts)

Nu

mb

er

of

bo

ok

mark

s p

er

use

r

Users from BibSonomy, CiteULike and Connotea

1,179 users posted one article

75% of content is created by 21%

of users

8,511 articles were only

bookmarked once

0

500

1000

1500

2000

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Re

v M

od

Ph

ys

Re

p P

rog

Ph

ys

Ph

ys R

ep

Ne

w J

Ph

ys

Ph

ys R

ev

E

Soft

Mat

ter

J Sta

t M

ech

Ph

ys R

ev

A

Eur

Ph

ys J

E

EPL

Act

Cry

st A

Ap

pl P

hys

Le

t

Ph

ys F

luid

s

Eur

Ph

ys J

B

J Sta

t P

hys

An

n P

hys

J Ph

ys C

on

de

ns …

Nan

ote

chn

ol

J Ap

pl P

hys

J Ph

ys A

J PH

YS D

IEEE

Nan

ote

chn

ol

Co

mp

ut

Mat

er

Sci

Ap

pl P

hys

A

J Lo

w T

em

p P

hys

Ph

ys S

tat

Sol B

Nu

cl In

stru

m M

eth

A

Ph

ys L

ett

A

Ph

ys S

tat

Sol A

Sup

erc

on

d S

ci …

Ph

ys S

cr

Nu

cl In

stu

m M

eth

B

Ph

ys T

od

ay

JETP

Le

tt

Int

J Th

erm

op

hys

Act

Cry

st B

J Vac

Sci

Te

chn

ol A

J Mag

n M

agn

Mat

er

Ph

ysic

a C

J Rh

eo

l

Ph

ysic

a B

Pra

man

a

Hyp

erf

ine

Inte

ract

Solid

Sta

te I

on

Ph

ys S

olid

Sta

te

Nu

mb

er p

ub

licat

ion

s bo

okm

arke

d

Usa

ge r

atio

publications bookmarked

usage ratio

Usage Ratio

Pb : number of publications bookmarked

Pj : number of publications in journal j

0

500

1000

1500

2000

2500

3000

0

150

300

450

600

750

900

Ph

ys R

ev

E

Ap

pl P

hys

Le

t

J Ap

pl P

hys

Ph

ys R

ev

A

J Ph

ys C

on

de

ns …

Re

v M

od

Ph

ys

Ne

w J

Ph

ys

EPL

J Ph

ys A

Nan

ote

chn

ol

Ph

ys R

ep

Eur

Ph

ys J

B

J Sta

t M

ech

Re

p P

rog

Ph

ys

J PH

YS D

Ph

ys F

luid

s

Ph

ys L

ett

A

Nu

cl In

stru

m M

eth

A

Ap

pl P

hys

A

J Mag

n M

agn

Mat

er

Eur

Ph

ys J

E

Ph

ys S

tat

Sol B

J Sta

t P

hys

Nu

cl In

stu

m M

eth

B

Soft

Mat

ter

Ph

ys S

tat

Sol A

Ph

ysic

a B

Ph

ys S

cr

Ph

ys T

od

ay

Co

mp

ut

Mat

er

Sci

Sup

erc

on

d S

ci …

Ph

ysic

a C

Act

Cry

st A

JETP

Le

tt

J Lo

w T

em

p P

hys

An

n P

hys

J Vac

Sci

Te

chn

ol A

IEEE

Nan

ote

chn

ol

Pra

man

a

Solid

Sta

te I

on

Int

J Th

erm

op

hys

Act

Cry

st B

Hyp

erf

ine

Inte

ract

Ph

ys S

olid

Sta

te

J Rh

eo

l

Nu

mb

er o

f bo

okm

arks

Usa

ge d

iffu

sio

n

bookmarks

usage diffusion

Usage Diffusion

Rj : number of users of journal j

0.0

1.0

2.0

3.0

4.0

5.0

6.0

Ph

ys R

ev

A

Ap

pl P

hys

Le

t

Ph

ys R

ev

E

J Ap

pl P

hys

Nu

cl In

stru

m …

Ph

ys F

luid

s

Nu

cl In

stu

m M

eth

B

J Ph

ys C

on

de

ns …

Ph

ysic

a C

Soft

Mat

ter

J Lo

w T

em

p P

hys

Int

J Th

erm

op

hys

IEEE

Nan

ote

chn

ol

Act

Cry

st A

Eur

Ph

ys J

E

EPL

Ne

w J

Ph

ys

Re

v M

od

Ph

ys

Nan

ote

chn

ol

J Mag

n M

agn

Mat

er

J Rh

eo

l

J Vac

Sci

Te

chn

ol A

J Sta

t M

ech

Ph

ysic

a B

J Ph

ys A

Ph

ys L

ett

A

J PH

YS D

Co

mp

ut

Mat

er

Sci

J Sta

t P

hys

Ph

ys S

cr

Hyp

erf

ine

Inte

ract

Eur

Ph

ys J

B

JETP

Le

tt

Ph

ys S

tat

Sol A

Sup

erc

on

d S

ci …

Ph

ys S

tat

Sol B

Re

p P

rog

Ph

ys

Ph

ys R

ep

Solid

Sta

te I

on

Act

Cry

st B

Ap

pl P

hys

A

An

n P

hys

Ph

ys T

od

ay

Pra

man

a

Ph

ys S

olid

Sta

te

Usa

ge in

ten

sity

Article and Journal Usage Intensity

Bj : number of bookmarks to journal j

Pb : number of publications bookmarked

Rj : number of users of journal j

0.0

1.0

2.0

3.0

4.0

5.0

6.0

Ph

ys R

ev

A

Ap

pl P

hys

Le

t

Ph

ys R

ev

E

J Ap

pl P

hys

Nu

cl In

stru

m M

eth

A

Ph

ys F

luid

s

Nu

cl In

stu

m M

eth

B

J Ph

ys C

on

de

ns …

Ph

ysic

a C

Soft

Mat

ter

J Lo

w T

em

p P

hys

Int

J Th

erm

op

hys

IEEE

Nan

ote

chn

ol

Act

Cry

st A

Eur

Ph

ys J

E

EPL

Ne

w J

Ph

ys

Re

v M

od

Ph

ys

Nan

ote

chn

ol

J Mag

n M

agn

Mat

er

J Rh

eo

l

J Vac

Sci

Te

chn

ol A

J Sta

t M

ech

Ph

ysic

a B

J Ph

ys A

Ph

ys L

ett

A

J PH

YS D

Co

mp

ut

Mat

er

Sci

J Sta

t P

hys

Ph

ys S

cr

Hyp

erf

ine

Inte

ract

Eur

Ph

ys J

B

JETP

Le

tt

Ph

ys S

tat

Sol A

Sup

erc

on

d S

ci …

Ph

ys S

tat

Sol B

Re

p P

rog

Ph

ys

Ph

ys R

ep

Solid

Sta

te I

on

Act

Cry

st B

Ap

pl P

hys

A

An

n P

hys

Ph

ys T

od

ay

Pra

man

a

Ph

ys S

olid

Sta

te

Usa

ge in

ten

sity

article usage intensity journal usage intensity

0.0

1.0

2.0

3.0

4.0

5.0

6.0

Ph

ys R

ev

A

Ap

pl P

hys

Le

t

Ph

ys R

ev

E

J Ap

pl P

hys

Nu

cl In

stru

m M

eth

A

Ph

ys F

luid

s

Nu

cl In

stu

m M

eth

B

J Ph

ys C

on

de

ns …

Ph

ysic

a C

Soft

Mat

ter

J Lo

w T

em

p P

hys

Int

J Th

erm

op

hys

IEEE

Nan

ote

chn

ol

Act

Cry

st A

Eur

Ph

ys J

E

EPL

Ne

w J

Ph

ys

Re

v M

od

Ph

ys

Nan

ote

chn

ol

J Mag

n M

agn

Mat

er

J Rh

eo

l

J Vac

Sci

Te

chn

ol A

J Sta

t M

ech

Ph

ysic

a B

J Ph

ys A

Ph

ys L

ett

A

J PH

YS D

Co

mp

ut

Mat

er

Sci

J Sta

t P

hys

Ph

ys S

cr

Hyp

erf

ine

Inte

ract

Eur

Ph

ys J

B

JETP

Le

tt

Ph

ys S

tat

Sol A

Sup

erc

on

d S

ci …

Ph

ys S

tat

Sol B

Re

p P

rog

Ph

ys

Ph

ys R

ep

Solid

Sta

te I

on

Act

Cry

st B

Ap

pl P

hys

A

An

n P

hys

Ph

ys T

od

ay

Pra

man

a

Ph

ys S

olid

Sta

te

Usa

ge in

ten

sity

article usage intensity journal usage intensity

Social tagging

title

abstract

full text

classification

systems thesauri

author keywords

subject

headings

social

tagging

Actors in subject indexing

authors

professional indexers

readers

Social tagging

Comparison with conventional index terms

automatic term unification

removal of special characters

unifying BE and AE suffixes

stemming (Porter 2)

lexical term comparison on document level

Mean of 724 article-

based similarity values

Mean

overlap-tag ratio

Mean

overlap-analyzed term

ratio

Mean

cosine similarity

Title terms 0.365 0.245 0.279

Abstract terms 0.503 0.048 0.143

Author keywords 0.118 0.104 0.103

WoS KeyWords Plus 0.029 0.030 0.026

Inspec headings 0.133 0.034 0.062

Social tagging

A reader-specific perspective on journal content

tags (uncleaned) assigned to articles published

in J Phys Condens Matter in 2004

tags (uncleaned) assigned to articles published

in J Phys Condens Matter in 2008

Conclusions

data from social bookmarking services can be used in

journal evaluation

number of bookmarks can be used alternatively to measure

worldwide usage of journals

tags reveal the users’ perspective on journal content

poor quality of bookmarking data makes retrieval and

analysis complex

social bookmarking in STM is still in its infancy and services

are competing for a critical mass

Outlook: Mendeley