1 This work is licensed under a Creative Commons License Attribution Non-commercial ShareAlike 2.0...

transcript

1This work is licensed under a Creative Commons License

Attribution Non-commercial ShareAlike 2.0 Germany

Towards International Repositories Infrastructures

Workshop 16/17 March, 2009Norbert Lossau,

Director Göttingen State and University Library

& Scientific Coordinator DRIVER

Attribution Non-commercial ShareAlike 2.0 Germany 2This work is licensed under a Creative Commons License

Topics

Objectives of the Workshop

Visions - use cases – infrastructure and components

International Repositories Infrastructure(s) – Where do we stand today?

Challenges

Global Data Network: a model for the International Repositories Infrastructure?

How do we proceed: our next two days (and beyond)

1. Identify and establish relationships with key thought leaders, major projects/activities and services, and leading practitioners from around the world

2. Suggest commonalities between infrastructures, points of possible collaboration and pathways that might take the collaboration forward

3. To come to a shared vision of an international repositories infrastructure or, at least, the infrastructure components that might best be developed internationally

4. To identify the essential components of an international repositories infrastructure

5. To review the approaches to sustainability, scalability and interoperability being taken by these components, bearing in mind the wider research infrastructure

6. To consider ways in which the progress might be coordinated and reviewed over time

7. Focus the agenda to achieve tangible outcomes

Topics

Challenges

High-level Vision …

Free and unrestricted access

to sciences and human knowledge representation

worldwide,

incl. cultural heritage

Berlin Declaration, October 2003

International Repositories or Knowledge Infrastructure Vision …

Discovery & Access

Management

Usage & manipulati

Collaboration &

Sharing

Dissemination &

Publishing

To support the… complete research cycle, working with scientific information

High-level use cases & possible infrastructure components

Preservation actionsfile format registries

validation tools

representation information registries

IngestSWORD

shared metadata services

name / factual authority services

automatic metadata creation services

High-level use cases & possible infrastructure components

Accesswidespread OAI-ORE implementation

common text-mining API?

Online Reputation and reportingeffective, real-time, automatic forward and backward citation mechanisms

factual authority (common tagging of objects with funder / grant number metadata)

Infrastructure components non-technical factors

Discovery & AccessEstablished search & browsing behaviours and pathways

Online Reputation and reportingEstablished evaluation mechanisms (impact factor)

Preservation actionsAdditional (manual) effort on the author side required?

IngestAdditional (manual) effort on the author side required?

Use cases - behind the scene“ infrastructures – essential components

A ‘component’ might be:

A service (eg sherpa-romeo, connotea, BASE, funder repository, institutional repository)

A service environment (eg, Amazon S3, Microsoft Azure, Facebook)

A technical success factor (eg consistent use of DC to point from a metadata record to the ‘full text’, use of OAI-ORE, the DRIVER Guidelines), or

a non-technical success factor (e.g. filling repositories through OA-agreements with publishers).

These components will form the focus for the workshop and the action plans that will be its principal output.

Challenges

Topics

International Repositories Infrastructure – Where do we stand today? „Briefings“

Author identification

Copyright and licensing

Global harvesters (other than search engines)

Harvesters – subject or discipline based

Ingest – selected issues

Institution identifiers

Peer review

Persistent identifiers

Preservation

International Repositories Infrastructure – Where do we stand today?

Prestige and profiling services

Registries

Repository software

Repository support organisations

Storage

Usage reporting and metrics

User services

Validation and certification of repositories

Versioning

Development status of components - for discussion

Advanced?

Information on the repository landscape

Global harvesters

Preservation of research papers

Repository software

Storage

Validation & certification of repositories

A brief insight into some components =>…

International Repositories Infrastructure – Where do we stand? Repositories• Informal survey carried out by SURF earlier in

2005• DRIVER Inventory Study 2007

1. Produced 7 studies in 3 publications

• Inventory study into the present type and level of OAI compliant Digital Repository activities in the EU

• A DRIVER's Guide to European Repositories• The Investigative Study of Standards for Digital

Repositories and Related Services 2. Disseminated through the DRIVER website (in Open Access) +

as 3 books (Amsterdam University Press)

International Repositories Infrastructure – Where do we stand? Repositories

• “Research Repositories in Europe: the 2008 DRIVER Inventory study”, Maurits van der Graaf (on behalf of SURF, DRIVER)

Research Repositories in Europe

TopicsGrowth, total number and situation

Contents, coverage and depositing

Technical issues and standards

Services on top of repositories

Steady increase of number of Digital Repositories

Total of 280, yearly increase by 25-30

Large part of universities in half of European countries

Conclusions content, coverage and depositing

More flexibility in access formsTrend to more OAVersion of deposited full text articles

Trend towards depositing postprint stage

Work processes for depositingNo harmonisation

Growing (partly) mandatory depositing32% in 2008, while 25% in 2006

(Still) Coverage of a third33% of Researchers delivering in repositories 35% of Research output of an institution deposited

Technical issues

Various technical issues 2008 2006persistent identifier 84% 75%

long-term availability secured 52% 73%

statistical data on access and usage 72% 70%

some form of subject indexing 86% 93%

author identifier 31% 33%

locally developed

CDSware

Digitool

DSpace

Fedora

GNU EPrint

MyCoRe

VITAL other

Technical issues

Is your repository technically prepared for Enhanced Publications?

Metadata standards on the rise: DIDL, MODS and OAI-ORE

46.1% 32.6%

YESNo, but

no, but we have plans to prepare our repository

no, no plans

International Repositories Infrastructure – Where do we stand? Repositories

• OpenDOAR – a comprehensive register of digital repositories worldwide- More than 1300 repositories listed

OpenDOAR

Repository Type=>1072 institutional and 177 disciplinary

Content Type=>815 hold journal articles, 318 Multimedia, audiovisual….69 datasets, 27 software etc.…

OpenDOAR

=>Languages:1133 English155 German96 Spanish86 French73 Japanese…3 Africaans…2 Pashto, Pushto…1 Bulgarian1 Romanian

OpenDOAR

=>Disciplines763 Multidisciplinary86 Science General…99 Health and medicine…98 History and Archaeology…75 Social Sciences General

OpenDOAR

Where do we stand? Repository platforms & their (international) communities

EPrints*, DSpace*, Fedora Commons*, OPUS (GE), DiVA (SE), CDS Invenio (CERN),

Where do we stand? Country organisa-tions and/or repository infrastructures

Australia, Belgium, Brazil, Canada, France, Germany, Hungary, Ireland, Italy, Japan, The Netherlands, Nordic countries, Portugal, Spain, UK, ???

Where do we stand? Global Harvesters

OAIster, BASE, Scientific Commons

Where do we stand? Cross-Country organisations & repository infrastructures

DRIVER

Where do we stand? Repository Infrastructure Architectures

DRIVER

Existing Solutions: Repository Aggregation Systems (RAS)

RAS aggregate content from OAI-PMH Repositories, form an Information Space and provide community-specific functionalities via Web User Interfaces

Well known examplesBASE (DE)

DAREnet (NE)

OAIster (USA)

Others…

OAI-PMH

Institution Site

OAI-PMH

Institution Site

OAI-PMH

Institution Site

Aggregator

Information Space

Search

Service Open Infrastructures (SOI), DRIVER

Inspired by component-oriented systemsComponents provide specific functionality in isolation

Components can be provided by different Service Providers and be shared between applications

Applications are formed by combining independent components under the control of System Managers

Service Open InfrastructureComponents are distributed services running on the network at different sites

Open to instance and types of services: instances or new functionality can be added/removed any time

Infrastructure architecture

Functionality LayerUser Interface

ServiceRecomm.Service

CommunityService

UserService

SearchService

Repositories

Data Layer

OAI-PMHService

IndexService

BrowseService

StoreService

AggregatorService

UserService

ValidatorService

Text EngineService

EnablingLayer

CollectionService

ReuseFunctionality sharing

OAI-PMH

Aggregator

Search

OAI-PMH

Institution Site

…OAI-PMH

Institution Site

OAI-PMH

Institution Site

Search

Aggregator

User Profiling

Others

Aggregator

Search

FunctionalityServices

Institution Site

Dynamic, distributedRun-time Infrastructure

ContentResources

Where do we stand? Repository Infrastructure Interoperability

DINI Certificate

DRAMBORA

TRAC project

DRIVER Guidelines, Maurice Vanderfeesten, SURF + DRIVER partners (some of the following slides have been presented by Maurice on the 29 August 2007, TICER Digital Libraries a-la-Carte)

- Guidelines

- Validate

- Workflow

Interoperability pragmatics

Guidelines

- Chapter 1: Use of OAI-PMH - Chapter 2: Use of Metadata OAI_DC - Chapter 3: Use of Best Practices for OAI_DC- Chapter 4: Use of Compound Object Wrapping - Chapter 5: Use of Vocabularies and Semantics - Chapter 6: Use of Quality labels - Chapter 7: Use of Persistent Identifiers- Chapter 8: Use of Usage Statistics Exchange- Chapter 9: Use of Intellectual Property Rights

DRIVER Guidelines in various languages

Attribution Non-commercial ShareAlike 2.0 GermanyThis is as optional footer

Guidelines

From the inventory study:

72.5% knows DRIVER Guidelines; 54.5% tries to follow them

Does your repository follow the DRIVER guidelines? n %

We do not know about the DRIVER guidelines 49 27.5

We know about the DRIVER guidelines, but do not follow them 32 18.0

We know about the DRIVER guidelines and (make every effort) to follow them 97 54.5

Validator

- Detects interoperability failures

- Goes deep into the metadata content

- Provides explanation about guideline principals per

interoperability feature.

- Offers recommendations on how to correctly modify

your repository to interoperable standards

- Creates a report for future reference

=>Developed at the University of Athens

Where do we stand? Technology Components – 2008 Study

Slides from the „Technology Watch“, Karen Van Godtsenhoven, University of Gent (+ DRIVER partners)

Structure of Technology Watch Report

ChaptersDRIVER-GRID interaction

Interoperability

Long Term Preservation (LTP)

DRIVER-CRIS interaction (added later)

Result: two main partsNew communities and technologies (GRID, CRIS, LTP)

Interoperability of EP’s (5 types)

Structure of each chapterTheory - Case studies - Outcomes for DRIVER

Interoperability Enhanced Publications

Interoperability in DRIVER context: Exchange and dissemination of EP’s as complex, compound objects, based on textual publication

Focus on five types of representing and publishing enhanced publications (relationship of files within objects)

Envelope models or packaging formatsOverlays, maps, feedsEmbedding formatsNew/Old publishing formatsWeb services

NOT focus on ingest or descriptive metadata (Russell, Vanderfeesten, Hochstenbach, Van Godtsenhoven)

Envelopes

Access to metadata, structural data, identifiers, and binary streams of publications all in one package (= envelope)

MPEG 21-DIDL in DARE context

IMS – CP

ODF packages

OOXML/ Package convention

Open e-book packageComparison: table with all features in doc

Overlays, maps and feeds

SWAP, ORE and POWDER all qualify as good formats / models for the dissemination of EP’s

SWAP uptake by community is very low (high complexity)

OAI-ORE very popular in community and used in DRIVER demonstrator for EPs

POWDER: recent W3C standard, viable alternative to ORE (when the aggregations are of a very dynamic nature or cannot be simply enumerated)

New Publishing Formats

ODF (ISO 26300:2006) versus OOXML (ISO 29 500-1:2008)File format ISO standards for saving & exchanging office documents (alternative to proprietary formats e.g. doc, ppt)Open up access to structured content which can be reused by other services e.g. DRIVERGuarantee long term accessibilityControversy surrounding development of OOXML: DRIVER should adopt approach that is capable of using both ODF and OOXML Plus: many disciplinary xml types, structured and crawlable data

Where do we stand? Infrastructure Technology Components in Practice

Automating and monitoring harvesting, data processing and indexing processes

DRIVER Repository Map

DRIVER Admin (Internal) Control Panel I

Monitor repository landscape I

DRIVER Admin (Internal) Control Panel II

Monitor repository landscape II

DRIVER Admin (Internal) Control Panel III

Monitor &

Process

Repository D

DRIVER Admin (Internal) Control Panel V

Check repository

index profile updates

Where do we stand? Linking publications to datasets (Enhanced Publications)

DRIVER – Enhanced Publications

Technology

The demonstrator aggregates scientific web resources via OAI-ORE v0.9 and RDF. XSLT is used to transform these into XHTML. CSS and Javascript do the rest of the presentation. A Java applet is used to dynamically display the relations between resources. Although these relations can be fed to the applet as parameters, they are not yet automatically interpreted from the RDF-serialisation

Driver II Definition Enhanced Publication

An Enhanced Publication (EP) is:a textual publication enhanced with:

research data (evidence of the research) and/or

extra materials (to illustrate or to clarify) and/or

post-publication data (commentaries, ranking)

So: ever developing

Challenges

Topics

Challenges towards an International Repositories Infrastructure

Complex matrix, addresses

(min.) five main dimensions

Countries (political, finance, organisational, legal issuesetc.)

Academic disciplines

Content access & usage

Multiple content resource types

Data Models & Technology

Countries

Disciplines

Content type

Content access& usage

Essential: keep all stakeholders and their perspectives in mind

Researchers/disciplines

Research managers

Library Managers

Repository Managers (technical & content)

Computer Scientists

Publishers & further content providers

Service & Infrastructure providers

Funders

The Diversity & Wealth of academic disciplines

EC, Framework 6 Programme: 46 pages, c. 40 entries each

Countries

Disciplines

Content type

Content access& usage

Discipline Schema (Keywords): European Commission

7 main areas

Text, Manuscript

Drawing

Painting

Radio, TV Broadcasts

Papyri

Cuneiform tablets

Artefacts

Buildings

Language audio recordings

Content type

Discipline Schema: Deutsche Forschungsgemeinschaft, Germany

4 main domains

Life Sciences

Natural Sciences

Engineering

14 subdomains

Disciplines and their International Repositories Infrastructures

ExamplesArXiV – Physics, Mathematics, Informatics

PubMed - Life Sciences

CLARIN, OLAC – Linguistics, language archives (datasets, international)

CESSDA – Social Sciences (datasets, international)

DARIAH – Humanities (datasets, international)

RePEc; NEEO – Economics (pre-/postprint publications, international)

METAFOR – Meteorology, Climate research (publications + datasets, international)

MACE - Architecture

IVOA - Astronomy

Issues to be addressed on the way towards an international repositories infrastructure, e.g.

Each discipline – one infrastructure?

Each information type – one infrastructure?

Same data models, technology, same services – different implementation

Same data models and syntax- different semantics

Project specific goals & funding – external liaision & collaboration

Focus on a specific community, a country, a region – cross community, cross-country initiatives

Further issues

Content! Filling repositoriesPublications: business models publishers

Research data: culture of sharing data

Challenges

Topics

Global Data Networks vs. Global Repository Infrastructure?

Data networks are „neutral carriers“ of information - Digital repositories contain the actual information

Content resources – multiple semantics and formats

Data networks are generic – knowledge infrastructures are disciplin-specific

Cultural issues for disciplines: „You share the network – but not your research data“

Financing: hardware vs. service, business cases

Further, architectural models for infrastructures?

JISC-fundedcontent providers

institutionalcontent providers

externalcontent providers

brokers aggregators catalogues indexes

institutionalportals

subjectportals

learning managementsystems

media-specificportals

end-userdesktop/browser pr

fusion

OpenURLlink servers

shared infrastructure

authentication/authorisation (Athens)

institutional profilingservices

terminology services

service registries

identifier services

metadata schema registries

This work is licensed under a Creative Commons LicenseAttribution-ShareAlike 2.0

JISC Information Environment architecture

Ingenieur-Wissenschaft

Weitere Wissenschaft

Geistes- und Sozial-

Wissenschaften

Natur- und Lebens-

Wissenschaften

Dokum.Server

DigitalisierteSammlungen

Fernsehen/Radio

Forschungs-daten …Mail

1 This work is licensed under a Creative Commons License Attribution Non-commercial ShareAlike 2.0...

Documents