PC Chairs Lynda Hardman Alex Hauptmann Nadia Magnenat-Thalman

transcript

<is web> Information Systems & Semantic WebUniversity of Koblenz ▪ Landau, Germany

General ChairsDietrich PaulusSteffen Staab

SAMT 2008 Semantics and Digital Media Technology

Koblenz, December 3-5, 2008http://samt2008.uni-koblenz.de

PC ChairsLynda HardmanAlex Hauptmann

Nadia Magnenat-Thalman

Semantic Web as a Knowledge Infrastructure for Multimedia

Steffen Staab

ISWeb - Information Systems & Semantic Web

Steffen Staabstaab@uni-koblenz.de

MMKM3 of 28

Depict ISWeb People!

Context is missing!

Precision

Recall

12/19 (FTE)

12/33 (all)

MMKM4 of 28

There is a picture of ISWeb in 2006 in the Wiki!

MMKM5 of 28

Some more ISWeb members you find on our Web page!

MMKM6 of 28

Outside of our Web Site you may also discover current members…

MMKM7 of 28

For others, you need to integrate information from the Web and from Word documents of my mailbox

MMKM8 of 28

Knowledge Infrastructure

A Multimedia objects exist in an environment.

This environment (context) can and should be used to find, access and present the objects.

Knowledge about Multimedia Objects exists in an environment, too.

This knowledge infrastructure can and should be used to find, access and present the objects.

Well, known in the multimedia field

Little explored until now

MMKM9 of 28

Structure of the Environment

Knowledge in Portal

Knowledge in File System

Knowledge in Wiki

Knowledge in Outside Service

Knowledge in Mailbox

MMKM10 of 28

Structure of the Environment1. Collect or Access

2. Harvest/Query Knowledge

3. Collect & Organize Knowledge

4. Use Knowledge

MMKM11 of 28

What does the Knowledge Infrastructure need?

1. Ontologies Multimedia Ontologies Context Ontologies Domain Ontologies

2. APIs

3. Querying and reasoning

MMKM12 of 28

How to structure this knowledge?

PersonRegion

MMKM13 of 28

Scenario

The "ISWeb" at Bad Kreuznach (team web page)

Localize a region (person)Annotate the content (interpretation)Carsten Saathoff, aceMedia Project lead, ISWeb, Semantic Multimedia, KAT Annotation ToolLink to knowledge on the Web:Reg1 foaf:depicts kodemaniak:mekodemaniak:me rdfs:label “Carsten Saathoff"kodemaniak:me rdf:type foaf:Person

Summer School Semantic Web-05 (video)

MMKM14 of 28

Problem

The "ISWeb" at Bad Kreuznach

(team web page)Summer School Semantic Web-05 (video)

Multimedia objects are complex Compound information objects, fragment identification

Semantic annotation Subjective interpretation, context dependent

Linked data principle Open to reuse existing knowledge

Multimedia O

Domain O

Context O

MMKM15 of 28

Example: Fragment Identification with COMM

http://isweb.uni-koblenz.de/People/isweb-2007.jpg

core:image-data

http://en.wikipedia.org/wiki/Image:Yalta_Conference.jpg

loc:spatial-mask-role

5 25 10 20 15 15 10 10 5 15"^ x̂sd:string

dns:realized-by

loc:region-locator-descriptor

loc:bounding-box

dns:plays

dns:played-bydns:defines

data:has-rectangle

MMKM16 of 28

Example: Region Annotation with COMM

core:image-data

http://en.wikipedia.org/wiki/Image:Yalta_Conference.jpg

loc:spatial-mask-role

5 25 10 20 15 15 10 10 5 15"^^xsd:string

dns:realized-by

loc:region-locator-descriptor

loc:bounding-box

dns:plays

dns:played-bydns:defines

data:has-rectangle

dns:setting core:semantic-annotation

core:semantic-label-role

dns:defines

dns:played-by

http://en.wikipedia.org/wiki/Churchill

foaf:Person

rdf:type

http://kodemaniak.de/foaf.rdf#me

What is important here, is what you don‘t see!

MMKM17 of 28

Steffen“I apply for this position. Here

is my cover letter and CV ...”

Bercovici.pdf

Mrs. Werger

Bercovici.pdf

“Please prepare papers for his hire,

Mrs. Werger“

“Thanks for your application and here is some information

about the NeOn project…” Colleagues

Another Scenario ...

MMKM18 of 28

… Contextualized Information

Bercovici.pdfNoam

“I apply for this position. Here is my cover letter and CV ...”

Colleagues

Context of multimedia information is lost

MMKM19 of 28

… Communicating

Steffen

Communication Context of multimedia information is lost

MMKM20 of 28

…classifying information…

Classification of multimedia information is lost

MMKM21 of 28

Descriptions and Situations

Descri

sDescriptions and Situations

Descri

X-COSIM Ontology

Sharing, Arguing, Explaining

Filing, Downloading, Modifying,Converting

Structuring,Summarizing,User-defined Ontologies

Creating, Reusing, Copy & Paste

MMKM22 of 28

DOLCE for Developers of E-mail Clients?

Particular

EndurantPerdurantAbstract Quality

DescriptionSituation

X-COSIMA

SPARQLRDF API

RDF Store

X-COSIMO

File Manager, Browser, Mail Client, Task Manager, Text Processor, ...

Dublin Core, Calendar, Foaf, ...

ReferenceModel,Storage

Apps,DomainModels

DataAccess,Mapping

Map between contextualized representations and the reference model

X-COSIM Architecture

MMKM23 of 28

Resulting Meta Data

ClassificationAspect

RealizationAspects

CommunicationAspect

Steffen

MMKM24 of 28

What does the Knowledge Infrastructure need?

1. Ontologies Multimedia Ontologies Context Ontologies Domain Ontologies

2. APIs

3. Querying and reasoning

Jump to end

MMKM25 of 28

Where are these metadata?

Steffen

Server

MMKM26 of 28

Federation of Knowledge

MMKM27 of 28

:NasirFOAF{:NasirFOAF dc:creator :Nasir .:Nasir :depictIn http://nas.jpg.}

:ISWebWiki{kodemaniak:me :member :ISWeb .Kodemaniak:me :depictIn :WikiImg1 .:ISWeb :depictIn http://..wiki1.jpg .}

Networked Graphs

:ISWebFOAF g:definedBy „CONSTRUCT {?X :depictIn...“ . }

:ISWebFOAF{:ISWeb foaf:name „Info Syst…“.:ISWeb :member :Nasir.:ISWeb :depictIn http://..wiki1.jpg.

Images of team members depict part

of the group

:ISWeb :depictIn http://nas.jpg .

Copy images that depict ISWeb

MMKM28 of 28

:NasirFOAF{:NasirFOAF dc:creator :Nasir .:Nasir :depictIn http://nas.jpg.}

:ISWebWiki{kodemaniak:me :member :ISWeb .Kodemaniak:me :depictIn :WikiImg1 .:ISWeb :depictIn http://..wiki1.jpg .}

Meta Knowledge

:ISWeb :depictIn http://nas.jpg .

:ISWeb :depictIn http://..wiki1.jpg.

:ISWebFOAF{:ISWeb foaf:name „Info Syst…“.:ISWeb :member :Nasir.

:ISWebFOAFMeta{:ISWebFOAF provenance :NasirFOAF .:ISWebFOAF provenance

:ISWebWiki . }

MMKM29 of 28

Take home message

Knowledge Structures of Multimedia Knowledge Infrastructure Knowledge CaptureSemantic Web ties them together

Some first steps are done: Standardization of Semantic Weblanguages RDF carried along with some media:

RDF in PDF, RDF-A in Open Document Format

Interesting Issues: Standardization of generic protocols Management of heterogeneity

of data, applications and services Mixed systems of reasoning and low level extraction ….

MMKM30 of 28

Systems & Literature

COMM Ontology & API: http://comm.semanticweb.org/

X-Cosim Ontology & API:http://isweb.uni-koblenz.de/Research/x-cosim

Networked graphs: http://isweb.uni-koblenz.de/Research/NetworkedGraphs

Querying for meta knowledge:http://isweb.uni-koblenz.de/Research/MetaKnowledge

Distributed querying:http://isweb.uni-koblenz.de/Research/DistributedSPARQL

ISWC-2007

K-Cap-2007

WWW-2008a

WWW-2008b

…to be written…

Thank You!

General ChairsDietrich PaulusSteffen Staab

SAMT 2008 Semantics and Digital Media Technology

Koblenz, December 3-5, 2008http://samt2008.uni-koblenz.de

PC ChairsLynda HardmanAlex Hauptmann

Nadia Magnenat-Thalman

PC Chairs Lynda Hardman Alex Hauptmann Nadia Magnenat-Thalman

Documents