+ All Categories
Home > Technology > DSpace & DuraCloud Integrations

DSpace & DuraCloud Integrations

Date post: 18-May-2015
Category:
Upload: tim-donohue
View: 1,308 times
Download: 0 times
Share this document with a friend
Description:
DSpace & DuraCloud Integrations talk, as presented as part of the DuraCloud Workshop at Open Repositories 2011 on June 6, 2011. More Information on work presented in these slides can be found at: * https://wiki.duraspace.org/display/DSPACE/ReplicationTaskSuite
Popular Tags:
33
Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0) To request other use: [email protected] DSpace + DuraCloud Integrations Tim Donohue DuraSpace
Transcript
Page 1: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

DSpace + DuraCloud IntegrationsTim Donohue

DuraSpace

Page 2: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

1. DSpace AIP Backup & Restore (1.7 +)(Initial DuraCloud use case: Backup & Restore)

2. DSpace Curation Task System (1.7 +)

3. DSpace Replication Task Suite (1.8)

Basis for DSpace Integration

Page 3: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

• Primary Use Cases– Backup & Restore of DSpace Content

• All content or just partial (Community/Collection/Item)

– Migration/Export of DSpace Content• All content or just partial (Community/Collection/Item)

– DuraCloud Integration

Intro to Archival Info Pkgs (1.7+)

Page 4: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

How to Backup DSpace (pre-1.7)

Full Database Backup

FolderBackup

Database Assetstore Folder

Page 5: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

How to Restore All (pre-1.7)

Full Database Backup

FolderBackup

Database Assetstore Folder

Page 6: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

How to Restore a Collection (pre-1.7)

Full Database Backup

FolderBackup

Database Assetstore Folder

Temporary Database

Temporary Folder?

Page 7: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

How to Restore a Collection (pre-1.7)

Full Database Backup

FolderBackup

Database Assetstore Folder

Temporary Database

Temporary Folder?

Page 8: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

Backup via Archival Info Pkgs

Package for each Community,

Collection & Item

AIP backup

Page 9: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

Restore All via Archival Info Pkgs

AIP backup

Package for each Community,

Collection & Item

Page 10: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

Restore a Collection via AIPs

AIP backup

Collection AIP

Items in Collection

1 2

Page 11: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

What’s in an AIP?

METS(DIM / MODS / PREMIS /

METSRights)

LicenseContent Files

or Logos

*Also a BagIt version in works

Other Filesin Bundles

(optional)

Archival Information Package (AIP)

Page 12: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

What’s in an AIP?

Descriptive Metadata: DIM & MODS Tech/Preservation Metadata: PREMIS

Related Object AIPs

METS(DIM / MODS / PREMIS /

METSRights)

LicenseContent Files

or Logos Other Filesin Bundles

(optional)

Rights Metadata: METSRights

Page 13: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

The “Site” AIP

Top-Level Community

AIPsMETS

(DIM / MODS / PREMIS / METSRights)

Special AIP for site-wide info/metadata:(e.g. Group Memberships, EPeople)

Page 14: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

Restore All In-Archive Content (Files + Metadata)Restore All People & GroupsRestore All Permissions / Access RightsRestore Community / Collection Logos, Metadata,

Rights & Item TemplatesRestore Community / Collection / Item HierarchyRestore In-Process / Incomplete ItemsRestore Collection OAI-PMH/ORE Harvest SettingsRestore all configuration files (dspace.cfg, etc.)

What can AIPs restore?

Page 15: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

Migrate a Collection

Items in Collection

One DSpace Install Another DSpace Install

Collection AIP

12

Page 16: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

Migrate Content

12

(Future work)One DSpace Install

Collection AIP

Items in Collection

Page 17: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

Package for each Community,

Collection & Item

DuraCloud Integration (1.7.x)

Local “Watch” Folder

1

./dspace packager -d

java -jar synctool.jar

[1]

[2]

Page 18: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

DuraCloud Integration (1.7.x)

Package for each Community,

Collection & Item

Local Folder

2

java -jar retrievaltool.jar

./dspace packager -r

[1]

[2]

Page 19: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

New: DSpace Replication Suite in 1.8

Page 20: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

• Enables a basic ‘microservices’ approach to curating DSpace objects

• Anyone can build a task & share it.• Currently tasks must be written in Java

– Working on JRuby & Jython integration (1.8?)

• “Frees” admin tasks from Command Line– Can now run from Admin UI or CLI

DSpace Curation System (1.7+)

Page 21: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

• A set of curation tasks geared towards ‘replicating’ (backup/restore/audit) content

• “Wraps” 1.7 DSpace AIP Backup & Restore

DSpace Replication Suite (1.8)

Backup content to AIP (filesystem or DuraCloud)Restore/Replace from AIPAudit AIP (compare to DSpace content)Basic IO Tracking of AIP Upload/DownloadsAll replication tasks can be run via Admin UI

Page 22: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

Package for each Community,

Collection & Item

DuraCloud Integration (1.8.x)

Local Temp Folder(Cache)

ORCommand line Curation Tools

1

“Replication Task Suite”:• Suite of Curation Tasks• One step Sync process• Via UI or CLI

1

Page 23: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

Package for each Community,

Collection & Item

DuraCloud Integration (1.8.x)

Local Temp Folder(Cache)

ORCommand line Curation Tools

“Replication Task Suite”:• One step Retrieve process• Via UI or CLI• Also ‘auditing’ tools

1

1

Page 24: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

DSpace Replication Suite Demo

Page 25: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

Page 26: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

Page 27: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

Page 28: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

Page 29: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

Cannot yet take advantage of DuraCloud streaming capabilities (AIPs are zip files)

Cannot yet take advantage of DuraCloud transformation services (AIPs are zip files)

Known Limitations

Page 30: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

• Working towards “unzipped” AIPs (1.8?)– METS file & Content files stored in an AIP ‘folder’

but NOT zipped up– Support for DuraCloud streaming, etc.

• DSpace UI Streaming Integration (@mire)• ‘Auto-Sync’ options

– Updates in DSpace -> DuraCloud (queued?)– Updates via DuraCloud services -> DSpace?

Next Steps

Page 31: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

• MIT : Richard Rodgers & Wendy Bossons – Developed Curation Task Framework– Developed initial Replication Suite tasks

• @mire : Mark Diggory– Look for @mire’s “Integrating DuraCloud Services

in DSpace” talk on Friday at 3:30pm

In Large Thanks to…

Page 32: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

• Replication Task Suite:– https://wiki.duraspace.org/display/DSPACE/Replic

ationTaskSuite

• AIP Backup & Restore:– https://wiki.duraspace.org/display/DSDOC/AIP+Ba

ckup+and+Restore

• Curation Task System:– https://wiki.duraspace.org/display/DSDOC/Curati

on+System

For More Information

Page 33: DSpace & DuraCloud Integrations

Licensed under Creative Commons Attribution-Share Alike 3.0 Unported License (CC BY-SA 3.0)To request other use: [email protected]

• Package : http://www.flickr.com/photos/halfbisqued/2353845688/• Harddrive & Terminal icons: http://tango.freedesktop.org/Tango_Desktop_Project• Folder icon: http://www.openclipart.org/detail/13740• Database icon: http://www.openclipart.org/detail/68413• Zip Pkg icon: http://veryicon.com/icons/system/capital-icon-suite-mac/zip-10.html• File icons: http://veryicon.com/icons/system/rhor-v2-part-3/• Checkmark & Delete icons: http://veryicon.com/icons/system/on-stage/• Tools Icon: http://veryicon.com/icons/system/azullustre/

Photo/Icon Acknowledgments


Recommended