Rhizomik Semantic Integration and Retrieval of Multimedia Metadata Roberto García and Universitat...

transcript

Rhizomik

Semantic Integration and Retrieval of Multimedia Metadata

Roberto García and Universitat de Lleida, Lleida, Spain

Òscar CelmaUniversitat Pompeu Fabra, Barcelona, Spain

Rhizomik

Contents

Introduction Ontological framework Integration framework Integration & Retrieval Scenario Conclusions Future Work

Rhizomik

Contents

Rhizomik

Introduction

Digital Media requires Multimedia Metadata. Main requirements:

Great amounts of metadata Low-level to user-friendly metadata Interoperability

Proposal: semantic metadata and an ontological framework for integration.

Rhizomik

Contents

Rhizomik

Ontological Framework

Multimedia ontology. Profit from non-semantic initiative MPEG-7. XML Semantic Web:

XSD2OWL: schema to ontology. XML2RDF: XML instance data to RDF instances.

Complete MPEG-7 OWL Ontology

Rhizomik

MPEG-7: Multimedia Content Description Standard content-based description for audiovisual information content searching, filtering and identification Descriptors:

low-level features structure semantic

modelscollectionscreation...

Rhizomik

Other semantic multimedia metadata initiatives: Hunter02: not complete, RDF+DAML. Tsinaraki04: not complete, semantic part of MDS. Troncy03: not complete, from an ontology to MPEG-7.

Direct benefits: XQuery vs. Semantic Query, example next…

Rhizomik

One XQuery for each type

One semantic query for supertype.

SegmentType hierarchy.

Rhizomik

Ontological Framework ReDeFer

XSD2OWLMappings:

owl:intersectionOfowl:unionOf

sequencechoice

owl:maxCardinalityowl:minCardinality

@maxOccurs@minOccurs

rdfs:subClassOfextension@base|restriction@base

owl:RestrictioncomplexType//element

owl:ClasscomplexType|group|attributeGroup

rdfs:rangeelement@type

rdfs:subPropertyOfelement@substitutionGroup

rdf:Propertyowl:DatatypePropertyowl:ObjectProperty

element|attribute

OWLXML Schema

Rhizomik

Contents

Rhizomik

Integration Framework

MPEG-7 OWL Ontology as integration base for: Semantic metadata: align ontologies with MPEG-7

ontology. MusicBrainz (http://musicbrainz.org) Simac (http://www.semanticaudio.org) Music Vocabulary (http://www.kanzaki.com/ns/music) Etc.

XML2RDF: RDF from MPEG-7 XML instance metadata.

Rhizomik

Integration Framework. Architecture

Signal ProcessingAudio

MPEG-7 XML

Content-based metadata

XML2RDF

Another Ontology

RDFContext-based

metadata

MPEG-7 Ontology

Integration

Retrieval

Higher-level metadata

DL Classifier

SWRL Engine

XSD2OWL

XMLSchemas: MPEG-7...

RDFS / OWL: MusicBrainz...

Rhizomik

MusicBrainz to MPEG-7 OWL mappings:

musicbrainz:duration ≡ mpeg7:MediaDuration

musicbrainz:sortName mpeg7:Name

dc:title mpeg7:Title

dc:author mpeg7:Creator

musicbrainz:Track mpeg7:AudioSegmentType

musicbrainz:Album mpeg7:CollectionType

musicbrainz:Artist mpeg7:CreatorType

Rhizomik

Simac to MPEG-7 OWL mappings:

simac:meter ≡ mpeg7:Meter

simac:tempo ≡ mpeg7:Beat

simac:key ≡ mpeg7:Key

simac:mode ≡ mpeg7:Scale

simac:Descriptor ≡ mpeg7:AudioDSType

simac:duration ≡ mpeg7:MediaDuration

simac:title ≡ mpeg7:Title

simac:Track mpeg7:AudioSegmentType simac:name ≡ mpeg7:GivenName

simac:Artist mpeg7:CreatorType

Rhizomik

Music Vocabulary ontology to MPEG-7 OWL mappings:

music:meter ≡ mpeg7:Meter

music:key ≡ mpeg7:Key

music:Artist mpeg7:CreatorType

music:sections ≡ mpeg7:AudioSegment

music:Music_Unit mpeg7:AudioSegmentType

Rhizomik

Integration Framework. Architecture

MPEG-7 XML

XML2RDF

Another Ontology

RDFContext-based

metadata

MPEG-7 Ontology

Integration

Retrieval

DL Classifier

SWRL Engine

XSD2OWL

Rhizomik

XML2RDF: XML tree RDF graph.

Deduce blank node types from XSD2OWL ontologies restrictions.

elem elemelem

elem elem

Empty Text

elemattr

Empty Text Text Text

Blank nodes

rdf:Properties

XML tree model RDF graph model

Rhizomik

• XML2RDF example

mpeg7:DescriptionMetadataType

1.0mpeg7:Confidence

mpeg7:PlaceTypempeg7:CreationLocation

2000-10-10T19:45:00+09:00mpeg7:CreationTime

mpeg7:CreatorTypempeg7:Creator

mpeg7:CreationToolTypempeg7:Instrument

2001-09-20T03:20:25+09:00

mpeg7:LastUpdate

completeDescriptionExample

mpeg7:PrivateIdentifier

098f2470-bae0-11cd-b579-08002b30bfeb

mpeg7:PublicIdentifier

mpeg7:RightsType

mpeg7:Rights

mpeg7:Version

Tokyompeg7:AdministrativeUnit

jpmpeg7:Region

mpeg7:AgentType, mpeg7:PersonTypempeg7:Agent

mpeg7:ControlledTermUseTypempeg7:Role

mpeg7:Setting

mpeg7:TermUseType

mpeg7:Tool

RID#mpeg7:RightsID

mpeg7:Name

mpeg7:Namempeg7:href

mpeg7:name

mpeg7:value

mpeg7:Name

Creator

creatorCS

Shibata Yoshiaki

sensitivity

Wizzo Extracto ver. 2

Rhizomik

Contents

Rhizomik

Integration & Retrieval Scenario

Low-level descriptors from audio track signal:

Track editorial metadata:

<http://example.org/track#1> a simac:Track;simac:tempo “122”;musicbrainz:trmid “e3c41bc1-4fdc-4ccd-a471-243a0596518f”.

<http://example.org/track#2> a musicbrainz:Track;dc:title “Blowin’ in the wind”;dc:author [musicbrainz:sortName “Bob Dylan”];musicbrainz:trmid “e3c41bc1-4fdc-4ccd-a471-243a0596518f”.

Rhizomik

Integration & Retrieval Scenario Generic MPEG-7 based integration rules.

Example: audio track with the same fingerprint are the same individual, their metadata is merged.

mpeg7:AudioType(track1) mpeg7:AudioType(track2) musicbrainz:trmid(track1, trm1) musicbrainz:trmid(track2, trm2) (trm1 = trm2) owl:sameIndividualAs(track1, track2)

Rhizomik

MPEG-7 XML

XML2RDF

Another Ontology

RDFContext-based

metadata

MPEG-7 Ontology

Integration

Retrieval

DL Classifier

SWRL Engine

XSD2OWL

Rhizomik

Integration & Retrieval Scenario Transparent retrieval, MPEG-7 based queries.

RDQL example:

SELECT ?titleWHERE (?track,<rdf:type>,mpg7:AudioSegmentType), (?track,<mpeg7:Title>,?title),

(?track,<mpeg7:Creator>,?author), (?author,<mpeg7:Name>,“Bob Dylan”),(?track,<mpeg7:Beat>,?tempo)

AND (?tempo >= 120)USING mpeg7 FOR

<http://rhizomik.upf.edu/ontologies/2005/03/Mpeg7-2001.owl#>

Rhizomik

Contents

Rhizomik

MPEG-7 XML

XML2RDF

Another Ontology

RDFContext-based

metadata

MPEG-7 Ontology

Integration

Retrieval

DL Classifier

SWRL Engine

XSD2OWL

Rhizomik

Conclusions

MPEG-7 OWL ontology, an upper-level multimedia ontology for: Semantic metadata integration

align ontologies to MPEG-7 OWL. XML metadata integration

XML2RDF. Semantic queries for retrieval

generic queries based on MPEG-7.

Rhizomik

Contents

Rhizomik

Future Work Semantics-based recommendation systems

http://foafing-the-music.iua.upf.edu http://rhizomik.net

Infer user-friendly metadata from low-level descriptors.

mpeg7:AudioType(track) mpeg7:Beat(track, t) (t > 120) mpeg7:Loudness(track, l) (l > 0.9) genres:Rock(track)danceability(track, “high”)

Rhizomik

Thank you for your attention

Contact:

roberto@rhizomik.net

ocelma@iua.upf.edu

Rhizomik Semantic Integration and Retrieval of Multimedia Metadata Roberto García and Universitat...

Documents