Date post: | 27-Dec-2015 |
Category: |
Documents |
Upload: | georgiana-fowler |
View: | 220 times |
Download: | 1 times |
1
The NSDL:A Case Study in Interoperability
William Y. ArmsCornell University
2
The NSDL is a program of the National Science Foundation's Directorate for Education and Human Resources, Division of Undergraduate Education.
The NSDL Core Integration is a collaboration between the University Center for Atmospheric Research (Dave Fulker), Columbia University (Kate Wittenberg) and Cornell University (Bill Arms).
The ideas discussed in this talk do not represent the official views of the NSF or the Core Integration team.
Acknowledgement and Disclaimer
3
Research Funding: Europe and USA
Europe
Grant is awarded to carry out the research plan specified in proposal
USA
Grant is awarded to carry out research in the area described in the proposal, but is not expected to follow the precise plan.
4
New Initiatives during a Grant
Program Activity University
Gigabit testbed Mosaic Illinois
CSTR Lycos Carnegie Mellon
DLI-1 Google PageRank Stanford
DLI-2 Open Archives Initiative Cornell
Examples of significant partial funding that was not envisaged in the proposal.
5
NSF-funded Research Programs
NSF
Solicitation
Proposals
Research
New ideas
New ideas
6
The NSDL Program
NSF's objective
Build a comprehensive digital library for all aspects of science education
NSF's approach
Solicitation encouraged wide diversity of proposals divided into general categories
Best 60+ proposals funded -- more to follow
Grants allow projects flexibility
Result
A splendid set of projects
A challenge in interoperability!
7
NSDL Collections Funded by the NSF (a) Focused collections
8
9
10
11
NSDL Collections Funded by the NSF (b) Aggregates and federations
12
13
14
15
NSDL Service Projects Funded by the NSF
16
17
18
19
NSDL Core Integration Team Funded by the NSF
20
Responsibility without Authority
Core Integration
Budget $4-6 million
Staff 25 - 30
Management Diffuse How can a small team, without direct management control, create a very large-scale digital library?
21
All branches of science, all levels of education, very broadly defined:
Five year targets
1,000,000 different users
10,000,000 digital objects
10,000 to 100,000 independent sites
How Big might the NSDL be?
22
Collections The NSDL program funds only a fraction of the relevant collections.
23
Every Collection is Different
24
... to provide a coherent set of services across great diversity.
The Core Integration Task ...
25
A Spectrum of Interoperability
26
Approaches to interoperability
The conventional approach
Wise people develop standards: protocols, formats, etc.
Everybody implements the standards.
This creates an integrated, distributed system.
Unfortunately ...
Standards are expensive to adopt.
Concepts are continually changing.
Systems are continually changing.
Different people have different ideas
27
Interoperability is about agreements
Technical agreements cover formats, protocols, security systems so that messages can be exchanged, etc. Content agreements cover the data and metadata, and include semantic agreements on the interpretation of the messages. Organizational agreements cover the ground rules for access, for changing collections and services, payment, authentication, etc.
The challenge is to create incentives for independent digital libraries to adopt agreements
28
Function versus cost of acceptance
Function
Cost of acceptance
Many adopters
Few adopters
29
Example: textual mark-up
Function
Cost of acceptance
SGML
ASCII
HTML
XML
30
Example: security
Function
Cost of acceptance
Public key infrastructure
IP address
Login ID and password
31
Levels of interoperability
Level Agreements Example
Federation Strict use of standards AACR, MARC(syntax, semantic, Z 39.50and business)
Harvesting Digital libraries expose Open Archivesmetadata; simple metadata harvesting
protocol and registry
Gathering Digital libraries do not Web crawlerscooperate; services must and search enginesseek out information
32
Metadata StrategyMetadata is expensiveThe NSDL cannot afford to create it manually
33
Metadata Strategy
• Support eight standard formats
• Collect all existing metadata in these formats
• Provide crosswalks to Dublin Core
• Expose records in the metadata repository for others to harvest
• Concentrate on collection-level metadata
• Use automatic generation to augment item-level metadata
34
Users
Collections
Metadata repository
The Metadata Repository
Services
The metadata repository is a resource for service providers.
It holds information about every collection and item known to the NSDL.
35
Services Strategy
36
The Metadata Repository as a Resource
Records are exposed through Open Archives Initiative harvesting protocol.
Core Integration team will provide some services based on the metadata repository.
The architecture encourages others to build services.
37
Example: Search Service
Portal
Portal
Portal
Search andDiscoveryServices Collections
SDLIP OAI
http
Metadata repository
James Allan, Bruce Croft (University of Massachusetts, Amherst)
38
Research Challenges:
Extending the Architecture to Support Greater Riches
Federations with rich sets of agreements (e.g., MARC, Z39.50)
Rich object models (e.g., interactive, dynamic, continuous time)
Language tools (e.g, thesaurus, gazetteer)
... and Lesser Riches
Web crawling
Automated quality control