RDFMETS

Metadata Interoperability

Metadata Encoding & Transmission Standard (METS)

• METS is an XML schema designed for creating XML document instances that express the hierarchical structure of digital library objects, the names and locations of the files that comprise those objects, and the associated metadata.

• METS can be understood as a binder that unites metadata about a particular resource

• A METS record includes six parts:– Header– Descriptive metadata– Administrative metadata– File groups– Structural map– Behavior section

Metadata Encoding & Transmission Standard (METS)

• A system for packaging metadata necessary for both the management of digital library objects within a repository and the exchange of such objects between repositories, or between repositories and their users

• Used for: Digital collection repositories• Developed by the Digital Library Federation

(DLF) and Library of Congress (LC)

What is METS used for?

• To package metadata with digital objects in XML syntax

• For retrieving, storing, preserving, and delivering resources

• For interchange of digital objects with their associated metadata

• As an information package in a digital repository

What are some characteristics of METS?

• METS is:– an open standard– non-proprietary– developed by library community– (relatively) simple– extensible– modular

Seven Sections of a METS Document

• <mets>• <metsHdr/> - METS header (document talks about

itself)• <dmdSec/> - Descriptive metadata (MODS, etc.)• <amdSec/> - Administrative metadata (copyright info.,

etc.)• <fileSec/> - File section (names and locations of files)• <structMap/> - Structural map (relationships of the

parts) <structLink/> - Linking information

• <behaviorSec/> - Binding executables/actions to object

• </mets>

Structural Map: structMap

• Object modeled as tree structure (e.g., book with chapters with subchapters….)

• Every node in tree can be associated with descriptive/administrative metadata

• Can describe relationships among:– Integral files, whether individual or multiple– Parts of files – Sets of files or parts of files played in sequence– Sets of files or parts of files in parallel– Other METS documents (nested)– Other structural divisions in non-hierarchical

(hypertext) fashion

File Inventory: fileSec

• Records file specific technical metadata (file size, creation date/time) as well as identifying component files

• Files are arranged into groups, which can be arranged hierarchically

• Files may be referenced (using Xlink) or contained within the METS document

MD sections: <dmdSec> & <amdSec>

• Assumes different communities need to develop own element sets

• Provides wrapper for externally defined descriptive / admin element sets that can be:– internal (XML) or – external (referenced by XLink) to METS

document

METS Header: metsHdr

• Metadata regarding METS document– CREATEDATE, MODIFYDATE, etc.– Agents (Creator, Editor, etc.)

• Alternative Record ID values

Structural Linking Section: structLink

• Multiple links allowed within any METS document

• Useful for mapping between structural maps, e.g., for web sites

METS Extension Schema

• Descriptive Metadata (e.g., DC, MARC, MODS)

• Administrative Metadata: 4 buckets– Technical (image, text, audio, video formats)– IP Rights (XrML, ODRL, metsRights)– Digital Provenance (persistent file & DO

information, capture/migration, e.g., PREMIS)– Source (analog or digital)

The Descriptive Metadata Section with mdWrap

<mets> <dmdSec> <mdWrap> <xmlData> <!-- insert data from different namespace here --> </xmlData> </mdWrap> </dmdSec> <fileSec></fileSec> <structMap></structMap></mets>

Use <mdWrap> to embed descriptive metadata within a METS document

• <mets>• …• <dmdSec>• <mdWrap>• <xmlData>• <!-- insert metadata from different namespace here

-->• </xmlData>• </mdWrap>• </dmdSec>• …• </mets>

Metadata wrap section acts as “socket” to hold metadata from

other XML schemas or “vocabularies”

<dmdSec> with MODS Extension Schema

• <mets:mets>• …• <mets:dmdSec>• <mets:mdWrap>• <mets:xmlData>• <mods:mods></mods:mods>• </mets:xmlData>• </mets:mdWrap>• </mets:dmdSec>• …

</mets:mets>

Descriptive metadata section

MODS data contained inside the metadata

wrap section

Use of prefixes before element names to identify schema

The Descriptive Metadata Section with MODS and relatedItem elements

<mets:mets> <mets:dmdSec> <mets:mdWrap> <mets:xmlData> <mods:mods> <mods:relatedItem type=“constituent”> <mods:relatedItem type=“constituent”></mods:relatedItem> </mods:relatedItem> </mods:mods> </mets:xmlData> </mets:mdWrap> </mets:dmdSec> <mets:fileSec></mets:fileSec> <mets:structMap></mets:structMap></mets:mets>

METS document with two hierarchies (logical and physical

<mets:mets> <mets:dmdSec> <mets:mdWrap> <mets:xmlData> <mods:mods> <mods:relatedItem> <mods:relatedItem></mods:relatedItem> </mods:relatedItem> </mods:mods> </mets:xmlData> </mets:mdWrap> </mets:dmdSec> <mets:fileSec></mets:fileSec> <mets:structMap> <mets:div> <mets:div></mets:div> </mets:div> </mets:structMap></mets:mets>

<mods:mods> <mods:titleInfo> <mods:title>Bernstein conducts Beethoven </mods:title> </mods:titleInfo> <mods:name> <mods:namePart>Bernstein, Leonard</mods:namePart> </mods:name> <mods:relatedItem type="constituent"> <mods:titleInfo>

<mods:title>Symphony No. 5</mods:title> </mods:titleInfo> <mods:name>

<mods:namePart>Beethoven, Ludwig van</mods:namePart> </mods:name> <mods:relatedItem type="constituent">

<mods:titleInfo> <mods:partName>Allegro con moto</mods:partName> </mods:titleInfo> </mods:relatedItem> <mods:relatedItem type="constituent"> <mods:titleInfo> <mods:partName>Adagio</mods:partName> </mods:titleInfo> </mods:relatedItem>

</mods:relatedItem> </mods:mods>

MODS relatedItem type=“constituent”

• Child element to MODS• relatedItem element uses MODS content model

– titleInfo, name, subject, physicalDescription, note, etc.

• Makes it possible to create rich analytics for contained works within a MODS record

• Repeatable and nestable recursively – Making it possible to build a hierarchical tree structure

• Makes it possible to associate descriptive data with any structural element

<mets:mets> <mets:dmdSec> <mets:mdWrap> <mets:xmlData> <mods:mods> <mods:relatedItem> <mods:relatedItem></mods:relatedItem> </mods:relatedItem> </mods:mods> </mets:xmlData> </mets:mdWrap> </mets:dmdSec> <mets:fileSec></mets:fileSec> <mets:structMap> <mets:div> <mets:div></mets:div> </mets:div> </mets:structMap></mets:mets>

<mets:mets> <mets:dmdSec> <mets:mdWrap> <mets:xmlData> <mods:mods> <mods:relatedItem> <mods:relatedItem></mods:relatedItem> </mods:relatedItem> </mods:mods> </mets:xmlData> </mets:mdWrap> </mets:dmdSec> <mets:fileSec></mets:fileSec> <mets:structMap> <mets:div> <mets:div></mets:div> </mets:div> </mets:structMap></mets:mets>

Hierarchy to represent “logical” structure (nested

relatedItems)

Hierarchy to represent “physical” structure (nested

div elements)

METS Example<mets:dmdSec ID="beelos00d"><mets:mdWrap MDTYPE="MODS"><mets:xmlData><mods:mods version="3.0"><mods:titleInfo xlink:type="simple"><mods:nonSort>The </mods:nonSort><mods:title>loss of the SS. Titanic</mods:title><mods:subTitle>its story and its lessons</mods:subTitle></mods:titleInfo><!-- rest of MODS descriptive metadata here --></mods:mods></mets:xmlData></mets:mdWrap></mets:dmdSec>

MODS relatedItem type=“constituent” element

1. Child element to MODS

2. relatedItem element has same content model as mods (titleInfo, name, subject, physicalDescription, note, etc)

3. The relatedItem element makes it possible to create very rich analytic descriptions for contained works within a MODS records

4. relatedItem element is repeatable and it can be nested recursively (thus making it possible to build a hierarchical tree structure)

5. relatedItem elements make it possible to associate descriptive data with any structural element.

<mods:mods> <mods:titleInfo> <mods:title>Bernstein conducts Beethoven and Mozart</mods:title> </mods:titleInfo> <mods:name> <mods:namePart>Bernstein, Leonard</mods:namePart> </mods:name> <mods:relatedItem type="constituent"> <mods:titleInfo> <mods:title>Symphony No. 5</mods:title> </mods:titleInfo> <mods:name> <mods:namePart>Beethoven, Ludwig van</mods:namePart> </mods:name> <mods:relatedItem type="constituent"> <mods:titleInfo> <mods:partName>Allegro con moto</mods:partName> </mods:titleInfo> </mods:relatedItem> <mods:relatedItem type="constituent"> <mods:titleInfo> <mods:partName>Adagio</mods:partName> </mods:titleInfo> </mods:relatedItem> </mods:relatedItem></mods:mods>

textMD (NEW!)

• textMD is a XML Schema that details technical metadata for text-based digital objects.

• It most commonly serves as an extension schema used within the (METS) administrative metadata section.

• It canalso exist as a standalone document.• In the future textMD can be used within the

PREMIS element <additionalTechnicalCharacteristics>, an extension for format-specific metadata within the PREMIS preservation metadata Object XML Schema version 2.0.

textMD schema allows for detailing properties such as

• encoding information (quality, platform, software, agent)

• character information (character set and size, byte order and size, line terminators)

• languages • fonts • markup information • processing and textual notes • technical requirements for printing and viewing • page ordering and sequencing

Is there a problem?• Many metadata schemes and element sets

– Well known & documented– Less known and little public documentation

• Similar/same content described by different metadata schemes and vocabularies– No canonical metadata record for an object

• Varied syntaxes for encoding metadata– No canonical syntax

• A vital and diverse metadata ecology!• No problem, unless….

Metadata in the networked environment

• Interaction between systems that use metadata– Harvesting– Searching

• Integrating different types of metadata for local information management– Technical metadata for digital asset mgmt

• Reusing metadata in local applications– ONIX metadata in library systems

• Interoperability?

Interoperability

• Interoperability allows different computer systems, networks, and software to work together and share information

• Usually achieved by following standards

• Generally, an increase in specialization results in a decrease in interoperability

• Allows different systems to make use of same data

Interoperability

• Advantages– Can increase awareness and use of

collections– Reduces geographic and domain-specific

isolation of collections– Creates new avenues for scholarship– Likely to assist / promote the longevity of data

and collections– Holy Grail = one-stop access to the universe

of online resources

Interoperability

• Disadvantages– Consensus– Compromise– Delays– Loss of independence– Uniformity– Increased implementation difficulties– Loss of specificity and detail

• Worthy goal?

Importance of interoperability

Systems and organizations will interoperate

One should actively be engaged in the ongoing process of ensuring that the systems, procedures and culture of an

organisation are managed in such a way as to maximise opportunities for exchange and re-use of information, whether internally or externally.

Paul Miller, 2000

Metadata interoperability has to be the underlying principle of networked information management.

• Marcia Lei Zeng, 2001

Interoperability

System-oriented definition:– The ability of two or more systems or components

to exchange information and use the exchanged information without special effort on either system

User-oriented definition:– The condition achieved when two or more

technical systems can exchange information directly in a way that is satisfactory to users of the systems (AAP)

Interoperability factors

• In the context of networked information retrieval– Multiple and disparate systems (operating systems,

information retrieval systems, etc.)

– Multiple protocols

– Multiple formats of data

– Multiple metadata schemes

– Multiple vocabularies, ontologies, disciplines

– Multiple languages

– Multiple character sets

So we have …

• Many metadata schemes and element sets• Similar/same content described by different

metadata schemes and vocabularies• Varied syntaxes for encoding metadata• Which reflect:

– Community practices, needs, meaning– Cost barriers to adopting common standards– Lack of knowledge of available standards– Not invented here syndrome

Mechanisms for addressing interoperability

• Crosswalks and mapping

• Application profiles

• Registries

• Resource Description Framework (RDF)

Mapping and crosswalks

• Mapping: Intellectual activity that identifies semantically equivalent elements in different metadata schemes

• Crosswalk: Documentation resulting from mapping showing the equivalencies and conversion specifications

• 1998 NISO White Paper on Crosswalks

Unfortunately, the specification of a crosswalk is a difficult and error-prone task requiring in-depth knowledge and specialized expertise in the associated metadata standards

St. Pierre & LaPlant, 1998

Metadata “Crosswalks”

• Crosswalks result from analyzing and mapping the similarities of different metadata systems.

• Crosswalks provide mechanisms for reconciling differences between automated systems and enabling them to interoperate.

• Crosswalks enable the goal of integrated access to be achieved.

• Crosswalks provide the key to automating “translation” from one system to another.

Mapping Issues

• Semantic, structural, and data conversion• One-way or reversible mappings?• Mapping between any two elements:

– One-to-one– One-to-many (repeatable elements; unique more narrowly defined

elements)– Many-to-one (complete mapping; incomplete mapping)– One-to-zero (no semantically equivalent element)

• Data conversion– From less inclusive to more inclusive format– From uncontrolled to controlled vocabulary

• Correct and efficient mapping of metadata elements among various formats is the essential condition for ensuring metadata interoperability

Zeng & Xiao, 2001

“Absolute Crosswalking”

• It requires exact mapping between the involved elements (say, vra.title dc.title) of a source schema (e.g., VRA Core) and a target schema (e.g., DC).

• Where there is no exact equivalence, there is no crosswalking (e.g., vra.technique [empty space])

Mapping Process

-- Make a list of all elements of a source format and verify whether there is a matching element in the target format.

• If yes, then the name of that element is recorded in the crosswalk table.

• If no, an empty space will be left in the table.

• When all of the elements in the source metadata are checked, the task is finished.

“Relative Crosswalking"

• It requires to map all elements in a source schema to at least one element of a target schema, regardless of whether the two elements are semantically equivalent or not (e.g., vra.technique dc.format).

Mapping Process

• Make a list of all elements of a source format.

• Map every element of the source format to the element(s) of the target format.

• Find the most close element to match, even though it may not match well.

ContributorCoverage

----------Location.CurrentRepository

Format----------Technique

Dublin CoreDublin CoreVAR Core (3.0)

Relative crosswalking

Absolutecrosswalking

ContributorCoverage

----------Location.CurrentRepository

Format----------Technique

Dublin CoreDublin CoreVAR Core (3.0)

Relative crosswalking

Absolutecrosswalking

target

source

Pros and Cons -- Absolute Crosswalking

• ensures the equivalency (or closely-equivalent matches) of elements,

• does not work well for data conversion, • data values in non-mappable space will be

left out, especially when a source schema has a richer structure than that of the target schema. Example: http://php.indiana.edu/%7Efryp/marcmap.html

Pros and Cons -- Relative Crosswalking

• When data conversion is conducted, at least the values in the fields of a source database will find a place to reside in the target database.

• Appears to work better when mapping from complex to simpler schema, e.g., from MARC to DC, but not vice versa. http://www.loc.gov/marc/marc2dc.html

• The problem is that mapped elements are not really equivalent.

But, neither is complete

• Please note that in both cases, the elements of the target format may be incompletely included in the crosswalk table.

• For example, – DC element LANGUAGE is never an element

in the VRA Core.– When VRA is the source format, LANGUAGE

will never be mapped.

Common properties in crosswalks

a semantic definition of each metadata element;whether or not a metadata element is mandatory, optional, or mandatory based on certain conditions;whether or not a metadata element may occur multiple times; constrains due to the organization of metadata elements relative to each other, e.g., hierarchical parent-child relationships;constrains imposed on the value of the element (e.g., free text, numeric range, date, or a controlled vocabulary)optional support for locally defined metadata elements.

The Limits of Crosswalks

• There are one-to-many and many-to-one relationships among the elements being mapped.

• There are also extra elements in a source or target schema.

• The corresponding units may exist at multiple-element, sub-element or qualifier level.

Problems exist in direct mapping when:

• converting from a less inclusive format to a more inclusive format,

• converting from an uncontrolled vocabulary content to a controlled vocabulary content,

• converting to fields where there are indirect impacts on the corresponding fields and values, (e.g., 043 and 6xx #z),

• data contents using different controlled vocabularies,

• data being converted into non-searchable fields.

Switching-across when multiple schemas are involved

Schema ASchema A

Schema BSchema B

Schema CSchema C

Schema DSchema D

SwitchSwitch

Mapping Process

• One of the schemas is used as the switching mechanism among multiple schemas.

• Instead of mapping between every pair in the group, each of the individual metadata schemas is mapped to the switching schema only.

• Example: http://www.getty.edu/research/conducting_research/standards/intrometadata/crosswalks.html

Application profiles

Application profiles consist of data elements drawn from one or more namespace schemas combined together by implementors and optimised for a particular local application.

Heery & Patel, 2000

• Reuse of elements from different sets, but cannot define new elements

• Specify permitted schemes (e.g., date/time formats, controlled vocabulary) for data values

• Can refine standard definitions• By defining application profiles and, most importantly by declaring

them, implementers can start to share information about their schemas in order to inter-work with wider groupings.

Heery & Patel, 2000

RegistriesThe term "registry" covers a broad range of databases, documentation services, or Web-based portals

providing access to schemas. Baker, et al., 2001

• Metadata registry:– An index of metadata terms, official definitions, local variations extensions– Can enable the reuse of existing elements rather than users/communities

reinventing their own• UK Schemas Project: Includes registry of several metadata

element sets • EU Cores Project: Includes registry of core vocabularies and

profiles; a schema creation tool and Web interface to register schemas

• Dublin Core Metadata Registry: Authoritative source for DC; Designed to promote the discovery and reuse of exiting metadata definitions;

• Almost universally, registries are seen as our best hope in the medium term for a scalable solution to the problem of mapping and translating between a diversity of schemas. Baker, et al., 2001

Resource Description Framework (RDF)

• Provides a basic grammar for representing metadata terms, their semantics, relationships, etc.

• Use of Uniform Resource Identifiers (URIs) to identify namespace schemas where terms are declared and defined

Metadata Mapping for IntegratedAccess to Diverse Information Resources

• There is no “one-size-fits-all” metadata standard.

• Metadata mapping can make it possible to provide integrated access to diverse sets of information (library, archive, museum -- e.g., MARC, EAD, CDWA).

Metadata interoperability?

• In this context, interoperability means – Effective reuse and integration of existing

metadata from various sources– Ability to discover and act on authoritative

specifications of metadata vocabularies– Support for interaction with one or more

metadata sets created for other communities

• The diverse metadata ecology presents challenges but reflects real-world needs of diverse information communities

Resource Description Framework (RDF)

• XML = a syntax for specifying document structure

• RDF = a syntax for specifying document semantics– a means to make statements about (i.e., specify the

values of) properties of Web resources• properties = element types• values = element contents

• components:– an XML-based syntax; a graph-based data model– a schema language; a query language

RDF

• RDF model is based on the idea of making structured information statements in the form of subject-predicate-object expressions (RDF triples)

– subject of an RDF triple represents the resource

– predicate represents traits, characteristics or aspects of the resource and expresses a relationship between the subject and the object

RDF Example: "New York has the postal abbreviation NY"

• subject-predicate-object form:– New York (subject)--has the postal abbreviation

(predicate)--NY (object)

• RDF: <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:terms="http://purl.org/dc/terms/">

<rdf:Description rdf:about="urn:x-states:New%20York">

<terms:alternative>NY</terms:alternative>

</rdf:Description>

</rdf:RDF>

RDF

<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:dc="http://purl.org/dc/elements/1.1/"> <rdf:Description rdf:about="http://en.wikipedia.org/wiki/Tony_Benn"> <dc:title>Tony Benn</dc:title> <dc:publisher>Wikipedia</dc:publisher> </rdf:Description></rdf:RDF>

To an English-speaking person, the same information could be represented simply as:

The title of this resource, which is published by Wikipedia, is 'Tony Benn'

RDF

<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:foaf="http://xmlns.com/foaf/0.1/" xmlns:dc="http://purl.org/dc/elements/1.1/"> <rdf:Description rdf:about="http://en.wikipedia.org/wiki/Tony_Benn"> <dc:title>Tony Benn</dc:title> <dc:publisher>Wikipedia</dc:publisher> <foaf:primaryTopic> <foaf:Person> <foaf:name>Tony Benn</foaf:name> </foaf:Person> </foaf:primaryTopic> </rdf:Description></rdf:RDF>

RDF example

Title: Metadata Interoperability and StandardizationCreator: Marcia Lei ZengEmail: mzeng@kent.eduAffiliation: Kent State UniversitySubject: Metadata interoperabilityDate: July 1, 2005Format: text/PPTLanguage: EnglishNote: Metadata Workshop lecture notesURL: http://www.slis.kent.edu/~mzeng/1.ppt

Original record -- 〉 RDF record

http://www.slis.kent.edu/~mzeng/1.ppt

dc:titledc:format dc:date

dc.language

dc:creatordc:description

dc:subject

Metadata Interoperability and Standardization text/ppt

2005-07-01

English

Metadata interoperability

Metadata Workshop lecture notes

Zeng_Leivcard:fn

Marcia Lei Zeng vcard:org

vcard:email

Kent State University

mzeng@kent.edu

<?xml version=”1.0”?><?xml:namespace href="http://www.w3c.org/RDF/" as="RDF"?><?xml:namespace href="http://purl.oclc.org/RDF/DC/" as="DC"?><?xml:namespace href="http://www.imc.org/pdi/#vCard" as="VCARD"?> <RDF:RDF> <rdf:Description rdf:href = “http://www.slis.kent.edu/~mzeng/1.ppt”> <dc:creator rdf:href= “Zeng_Lei”/>

<dc:title> Metadata Interoperability and Standardization </dc:title> <dc:subject> metadata interoperability </dc:subject> <dc:date>2005-07-01</dc:date> <dc:format>text/ppt</dc:format> <dc:language>en</dc:language> <dc:description> Metadata Workshop lecture notes </dc:description></rdf:Description><rdf:Description ID= “Zeng_Lei”>

<vcard:fn>Marcia Lei Zeng </vcard:fn><vcard:email>mzeng@kent.edu </vcard:email>

<vcard:org>Kent State University </vcard:org></rdf:Description></rdf:RDF>

RDF

• The Resource Description Framework– It is a language for representing information

about resources in the World Wide Web.– It is particularly intended for representing

metadata about Web resources, such as the title, author, and modification date of a Web page, copyright and licensing information about a Web document, or the availability schedule for some shared resource.

RDF

• What it does: – RDF is intended for situations in which this

information needs to be processed by applications, rather than being only displayed to people.

– RDF provides a common framework for expressing this information so it can be exchanged between applications without loss of meaning.

– RDF integrates a variety of applications from library catalogs and world-wide directories to syndication and aggregation of news, software, and content to personal collections of music, photos, and events.

RDF

• How it looks:

– XML syntax-based

• How it works:

– The RDF specifications provide a lightweight ontology system to support the exchange of knowledge on the Web.

Example

“John Smith is the creator of the resource identified by http://www.example.org/index.html ”

• the thing the statement describes (the Web page, in this case) • a specific property (creator, in this case) of the thing the statement

describes • the thing the statement says is the value of this property (who the

creator is), for the thing the statement describes

http://www.example.org/index.htmlcreator

John Smith

Triples

The RDF terms for the various parts of the statement are:

• the subject is the URL http://www.example.org/index.html

• the predicate is the word "creator"

• the object is the phrase "John Smith"

Using URIs (Uniform Resource Identifiers)

• RDF uses URIs as the basis of its mechanism for identifying the subjects, predicates, and objects in statements.

• a subject http://www.example.org/index.html • a predicate http://purl.org/dc/elements/1.1/creator • and an object http://www.example.org/staffid/85740

http://www.example.org/index.html has a creator whose value is ID#85740

Groups of statements: Several Statements About the Same Resource

RDF Example

RDF

• RDF is based on a concrete formal model utilizing directed graphs that elude to the semantics of resource description.

• The basic concept is that a Resource is described through a collection of Properties called an RDF Description.

• Each of these Properties has a Property Type and Value.

• Any resource can be described with RDF as long as the resource is identifiable with a URI as shown in the next slide

RDF / Namespaces: an example

<RDF xmlns:DC="http://purl.org/DC"><DESCRIPTION

about="http://scow.gseis.ucla.edu/faculty/jfurner/memos/greg1.html"><DC:Creator>Jonathan Furner</DC:Creator><DC:Date>1999-05-05</DC:Date><DC:Subject>Greg, sci-fi

salutations</DC:Subject></DESCRIPTION></RDF>

RDF Example

• <? xml version="1.0" ?> <RDF xmlns = "http://w3.org/TR/1999/PR-rdf-syntax-19990105#" xmlns:DC = "http://purl.org/DC#" > <Description about = "http://dstc.com.au/report.html" > <DC:Title> The Future of Metadata </DC:Title> <DC:Creator> Jacky Crystal </DC:Creator> <DC:Date> 1998-01-01 </DC:Date> <DC:Subject> Metadata, RDF, Dublin Core </DC:Subject> </Description> </RDF>

RDF Example

• The first line of Example 1 simply indicates that this is an XML document.

• The next line indicates two namespaces - RDF and DC - with RDF being the default namespace.

• All the properties in the description will come from one of these namespaces.

• The main section of Example 1 - between the <Description> tags - shows four Properties that describe the resource pointed to by the URI in the about attribute in the <Description> tag.

• These properties come from the Dublin Core (DC) namespace (DC, 1998).

• In this case the Title, Creator, Date, and Subject properties are indicated for the resource at: <http://dstc.com.au/report.html>.

Digital preservation: advances & remaining challenges

• Groups around the world and conferences continue to make significant progress in raising awareness about digital preservation imperative

• Gradual shift in focus from articulating problem to solving it …– Not so much “Why is digital preservation important” anymore; rather, “What must

be done to achieve preservation objectives?”

• Many practical challenges in implementing reliable, sustainable digital preservation programs

• One key implementation challenge: preservation metadata

Preservation metadata includes• Provenance:

– Who has had custody/ownership of the digital object?

• Authenticity:– Is the digital object what it purports to be?

• Preservation Activity:– What has been done to preserve the digital object?

• Technical Environment:– What is needed to render and use the digital object?

• Rights Management:– What IPR must be observed?

Makes digital objects self-documenting across time

PREMIS: Preservation Metadata: Implementation Strategies

• June 2003: OCLC, RLG sponsored international working group• Objective:

– Define implementable, core preservation metadata, with guidelines/recommendations for management and use

• Membership: – > 30 experts from 5 countries, libraries, museums, archives,

government agencies, private sector• PREMIS Data Dictionary: comprehensive, practical resource for

implementing preservation metadata in digital archiving systems– Comprehensive view of information requirements needed to

support digital preservation– Based on deep pool of institutional experiences in setting up

and managing operational capacity for digital preservation– Builds on previous work

What PREMIS is and is not

• What PREMIS is:– Common data model for organizing/thinking about preservation

metadata– Guidance for local implementations– Standard for exchanging information packages between

repositories

• What PREMIS is not:– Out-of-the-box solution: need to instantiate as metadata

elements in repository system– All needed metadata: excludes business rules, format-specific

technical metadata, descriptive metadata for access, non-core preservation metadata

– Lifecycle management of objects outside repository– Rights management: limited to permissions regarding actions

taken within repository

PREMIS Data Model

Types of information covered in PREMIS (by entity type)

• Object– Object ID– Preservation level– Object characteristics(format, size, etc.)– Storage– Environment– Digital signatures– Relationships– Linking identifiers

• Event– Event ID– Event type– Event date/time– Event outcomes– Linking identifiers

• Agent– Agent ID– Agent name

• Rights – Rights statement– Granting agent– Permission granted

PREMIS: EntitiesIntellectual Entities

• Set of content that is considered a single intellectual unit for purposes of management and description (e.g., a book, a photograph, a map, a database)

• May include other Intellectual Entities (e.g. a website that includes a web page)

• Has one or more digital representations• Not fully described in PREMIS DD, but can be

linked to in metadata describing digital representation

PREMIS: EntitiesObject Entities

• Discrete unit of information in digital form• **Objects are what repository actually preserves**• Three types of Object:

– FILE: named and ordered sequence of bytes that is known by an operating system

– REPRESENTATION: set of files, including structural metadata, that, taken together, constitute a complete rendering of an Intellectual Entity

– BITSTREAM: data within a file with properties relevant for preservation purposes (but needs additional structure or reformatting to be stand-alone file)

• Examples:• chapter1.pdf (a file)• chapter1.pdf + chapter2.pdf + chapter3.pdf (representation of a book with

three chapters)• TIFF file containing header and 2 images (2 bitstreams (images), each with

own set of properties (semantic units):e.g., identifiers, technical metadata, inhibitors, … )

PREMIS: EntitiesEvent Entities

• An action that involves or impacts at least one Object or Agent associated with or known by the preservation repository

• Helps document digital provenance. Can track history of Object through the chain of Events that occur during theObjects lifecycle

• Determining which Events are in scope is up to the repository (e.g., Events which occur before ingest, or after de-accession)

• Determining which Events should be recorded, and at what level of granularity is up to the repository

• Examples:– Validation Event: use JHOVE tool to verify that chapter1.pdf is a valid

PDF file– Ingest Event: transform an OAIS SIP into an AIP (may be one Event or

multiple Events)– Migration Event: create a new version of an Object in an up-to- date

format

PREMIS: EntitiesAgent Entities

• Person, organization, or software program/system associated with an Event or a Right (permission statement)

• Agents are associated only indirectly to Objects through Events or Rights

• Not defined in detail in PREMIS DD; not considered core preservation metadata beyond identification

• Examples:– Markus Enders (a person)– Göttingen State and University Library (an organization)– JHOVE version 1.0 (a software program)

PREMIS: EntitiesRights Entities

• An agreement with a rights holder that grants permission for the repository to undertake an action(s) associated with an Object(s) in the repository.

• Not a full rights expression language; focuses on permissions relevant for preservation.

• Example:– Priscilla Caplan grants FCLA digital archive

permission to copy and migrate the Object “metadata_fundamentals.pdf” for preservation purposes.

Sample Data Dictionary entrySemantic unit size Semantic components

None

Definition The size in bytes of the file or bitstream stored in the repository.

Rationale Size is useful for ensuring the correct number of bytes from storage have been retrieved and that an application has enough room to move or process files. It might also be used when billing for storage.

Data constraint Integer Object category Representation File Bitstream Applicability Not applicable Applicable Applicable Examples 2038927 Repeatability Not repeatable Not repeatable Obligation Optional Optional Creation/ Maintenance notes

Automatically obtained by the repository.

Usage notes Defining this semantic unit as size in bytes makes it unnecessary to record a unit of measurement. However, for the purpose of data exchange the unit of measurement should be stated or understood by both partners.

Why is PREMIS important to catalogers?

• As we take responsibility for more digital materials, we need to ensure that they can be used in the future

• Most preservation metadata will be generated from the object, but catalogers may need to verify its accuracy

• Catalogers may need to play a role in assessing and organizing digital materials– Understanding the structure of complex digital objects– Determining significant properties that need to be

preserved