Building Tools and Services on the NDR

Dean Krafft, Cornell Universitydean@cs.cornell.edu

Building Tools and Services on the Fedora-based NSDL Data RepositoryStructure of the talk:

NSDL 2.0 – the vision The Fedora-based NSDL Data Repository

(NDR) Inspiring Contribution and Collaboration:

ExpertVoices, OurNSDL, MyNSDL Other NSDL 2.0 Services and Tools Discussion: Using NDR Collaborative Tools

to Build Pathways-focused Communities

The NSDL 2.0 Vision Create an NSDL that guides not just

resource discovery (with a one-way data flow: NSDL → Users), but instead Supports creating “context” to guide resource

selection and use Presents resources in context: in a lesson plan;

with ratings; correlated with education standards Enables community tools for structuring,

evaluation, annotation, contribution, and collaboration around the resources in the library

Goal: Create a dynamic, living library with a two-way data flow: NSDL ↔ users

Fedora: the NDR middleware A Flexible, Extensible Digital Object

Repository Architecture (http://www.fedora.info)

Open source project with $2.2 million in Mellon funding 2002-2007

Collaboration of Cornell and Univ. of Virginia Key funded users include:

eSciDoc project (collaboration of the Max Planck Society and FIZ Karlsruhe)

VTLS Corp., Harris Corp., Library of Congress Australian Research Repositories Online to the

World (ARROW) Royal Library Denmark, National Library, and DTU

What is Fedora? An architecture, toolkit, and implementation:

middleware, not a vertical application DSpace in contrast: a vertical application

with a fixed workflow targeted at users Stores arbitrary internal and external digital

objects, disseminations (transformations and combinations), relationships among objects

Entirely SOAP/REST based, disseminations are URLs

XML data store; RDBMS cache; RDF triplestore supports relationship queries

Implementing the NDR with Fedora Multiple Object Types:

Resources (with local or remote content) Metadata Aggregations (collections) Metadata Providers (branding) Agents

Relationships with arbitrary graph queries: Structural (part of) Equivalence Annotation

NSDL RecommenderService

ExampleCollection

NSDL BigBang

NSDL Agent1000

MDP 3000

Aggr2002 M

4002

NSDL Collections

1002

Aggr2005

M4005

NSDLRecom-mended

1005

NSDL RSAgent 1004

MDP 3004

ExampleAgent 10010

MDP 10011

Aggr10012

Aggr2004

M10005

Example.org

10006

pBy

pBy

asWith

asWithpBy

mOf

m4

m4

m4

agg4

mdp4

agg4mdp4

agg4

agg4

1st mOf

asWith

Types of Objects

Agents

Aggregators

Metadata Providers

Resources

Metadata

Types of Relationships

associatedWith (asWith)metadataProviderFor (mdp4)aggregatorFor (agg4)providedBy (pBy)metadataFor (m4)memberOf (mOf)· 1st. A recommended resource· 2nd. Makes it a “blessed” NSDL Collection

2nd mOf

M10007

m4

pBy

mdp4

NSDL FEDORA-BASED REPOSITORY

Draft NDR API Characteristics Uses REST calls for all interactions; uses handles

(DOIs) for all external references Ensures external applications can’t violate the

NDR model constraints Disseminations allow combining metadata from

multiple sources, or related content Authentication: Requests signed with private key

associated with an agent Authorization: Agent can become a metadata

provider or aggregator; can create resources Documentation being developed at

http://ndr.comm.nsdl.org

NDR Architecture

An Information Network Overlay Think of the NDR as a lens for viewing

science content on the net Content can be:

Local: stored directly in the NDR Remote: accessed through a URL Computed: derived from a database or

web service Archived: an older version stored at SDSC

It all has a repository-based URL

Network Overlay ViewUser View

API/UI

Repository View with Relations & Annotations

Resources on the Web

Status of the NDR Two NDR instances up and running

Full production load (approx. 1.8 million resources)

Test server for NDR API external testing (small) Test search interface to production load at

nsdlib.org Currently running MR and NDR ingest in

parallel with MR as primary service Planned switch to NDR as production

service on October 1, 2006 Continue running MR in parallel (as

backup) through December 2006

How should we use the NDR? The NDR provides powerful capabilities

for: Creating context around resources Enabling the NSDL community to directly

contribute resources and context Representing a web of relationships among

science resources and information about those resources

How do we use it? Here’s one specific example …

ExpertVoices

What is Expert Voices? A system using blogging technology to:

Support STEM conversations among scientists, teachers and students

Tie NSDL resources to real-world science news Create context for resources to enhance

discovery, selection and use Enable NSDL community members to become

NSDL contributors: of resources, questions, reviews, annotations, and metadata

Expert Voices ≠ LiveJournal Contributors are carefully selected, contributions

are about science, the process of science, and education

Expert Voices As An Educational Tool Topic-based discussion (e.g. tsunamis)

with pointers to related resources Research outreach (Criterion 2) –

explaining and documenting NSF-funded research

Experts can add resources with topical context to the NSDL

Resources can be reviewed and annotated Question/answer and discussion forum:

scientist ↔ teacher ↔ student ↔ librarian

Broadening Participation: An Expert Voices Learning Scenario

“Hurricane Season Blog” run by a National Weather Service hurricane expert, an Earth Science teacher, and a school media specialist familiar with NSDL resources

Expert creates an entry for Hurricane Gertrude “On track to hit Ft. Lauderdale in 72 hours” “Currently undergoing eyewall replacement cycle” “Expecting 15 foot storm surge”

Media specialist adds links to NSDL resources: Hurricane Hunters site, latest satellite photos, and USGS flooding and flood plain site (storm surge context)

Teacher makes connections to relevant standards and appropriate pedagogy for use by other teachers

Students experience engaging real-time, real-world applications of science lessons

Broadening Participation: An Expert Voices Outreach Scenario

NSF grantee: Bioluminescence researcher wants to make research K-12 accessible

Creates an Expert Voices conversation Enables his students and researchers to

document process and results – how science really works

Writes about publications and educational resources (e.g. www.photobiology.info) Adds these to the NSDL, creating audience-level

metadata Entries serve as annotations that create K-12

context for the college-level research

Expert VoicesImplementation Initial blog system is multi-user WordPress WordPress plug-ins provide NDR

integration and Shibboleth authentication Publication of blog entry creates:

Content, as a new resource with simple metadata

New NDR resources New metadata for any referenced resources in

content Graph of relationships between entry and all

referenced resources Blog available as independent RSS feed

NDR Entry for Expert Voices

Blog Entry

NewMetadata

NewAudience

MD

ReferencedNew

Resource 1

ReferencedExisting

Resource 2

Annotates

Metadata forMetadata for

Member ofMetadataProvider

MetadataProvider

ExistingCollectionTopic-

basedBlog

Member of

Inferred relationshipbetween resources

But Expert Voices is just the beginning…

OurNSDL: NDR-Integrated Wiki Community of approved contributors

(e.g. teachers, librarians, scientists) are granted edit access on OurNSDL wiki

New resources and metadata are created as wiki pages and reflected into the NDR

Non-wiki-based NDR resources and metadata are displayed as read-only wiki pages, subject to comment and linking

User and project pages organize NDR resources

Planned implementation in MediaWiki

MyNSDL: NDR-integrated tagging, bookmarking, and recommendation Based on Connotea open-source

folksonomic tagging/bookmarking system Tags and bookmarking structure are

reflected back into the NDR Authorized users can “automatically”

recommend new NSDL resources simply by tagging them

Gives user a personal view of NSDL resources

NDR Application: OnRamp NDR-integrated multi-user, multi-project

content management system Supports NSDL single sign-on and group

management Decentralized workflow for the creation

and distribution of both simple and complex content

Disseminates content in multiple publication and online forms

Delivery estimated 3Q06

NDR Application: Content Assignment Tool Developed by Anne Diekema, Elizabeth Liddy,

et al. at the Syracuse University Center for Natural Language Processing

Uses text analysis and machine learning to suggest Educational Standards alignment for resources

Content expert assigns standard, and system learns from the assignment

Standalone tool available now; standards associated with resources in the NDR by 3Q06

Other applications in development Automated grade-level assignment based

on vocabulary analysis (San Diego Supercomputer Center)

iVia-based Expert-Guided crawl: Tool for Pathways and others to turn websites into resource collections (UC Riverside)

Automated subject assignment (UC Riverside)

Instructional Architect: Lesson plan development for K12 teachers (Utah State)

Moodle Course Management System (proposed)

…

NSDL 2.0 Ecosystem

Protocol:OAI-PMHHTTPRESTNDR API

STEMCollections

SearchServiceArchive

Service

Fedora-basedNDR

Using Collaborative Tools to Build Pathways-focused Communities We plan to implement general collaborative

tools on the NDR: WordPress blog, OnRamp, MediaWiki, Connotea. Can you use these?

We can implement more specialized tools on the NDR (e.g. Moodle, CASAA, Instructional Architect). What would you use?

To achieve the NSDL 2.0 vision, we need to build sets of communities in each Pathway area. How can this happen for your Pathway? What can CI do to help?

Contact Information

Dean B. KrafftCornell Information Science301 College Ave.Ithaca, NY 14850USAdean@cs.cornell.edu

This work is licensed under the Creative Commons Attribution-NoDerivs 2.5 License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nd/2.5/ or send a letter to Creative Commons, 543 Howard Street, 5th Floor, San Francisco, California, 94105, USA.