Lehrstuhl Informatik V(Informationssysteme)

Prof. Dr. M. Jarke

M. Spaniol,R.Klamma,M. Jarke

I5-SKJ-2911-1/14

Workshop onMM

Semantics

Workshop onMM

Semantics

Semantic processing of multimedia data by MPEG-7 for capacious knowledge management

Marc Spaniol, Ralf Klamma, Matthias JarkeRWTH Aachen

Milovy, 29th of November

Workshop on Multimedia Semantics

Lehrstuhl Informatik V(Informationssysteme)

Prof. Dr. M. Jarke

M. Spaniol,R.Klamma,M. Jarke

I5-SKJ-2911-2/14

Workshop onMM

Semantics

Workshop onMM

Semantics

Agenda• Multimedia knowledge creation

• Framework of community-centred knowledge management– Transcription– Addressing – Localization

• Semantic knowledge management by XML and MPEG-7

• The Virtual Entrepreneurship Lab (VEL)

• Conclusion & Outlook

Lehrstuhl Informatik V(Informationssysteme)

Prof. Dr. M. Jarke

M. Spaniol,R.Klamma,M. Jarke

I5-SKJ-2911-3/14

Workshop onMM

Semantics

Workshop onMM

Semantics

Knowledge creation process• Making implicit knowledge

explicit (Externalization)

• Management of explicit data (Combination)

• Retrieval of explicit knowledge (Internalization)

• Sharing of implicit knowledge (Socialization)

There is no media free knowledge (Multi) media semantics?

Knowledge Management = Multimedia Management + Ontologies

Externalization

Internalization

implicit

explicitimplicit

explicit

CombinationSocialization

The Knowledge-Creating Company [Nonaka, Takeuchi 95]

Lehrstuhl Informatik V(Informationssysteme)

Prof. Dr. M. Jarke

M. Spaniol,R.Klamma,M. Jarke

I5-SKJ-2911-4/14

Workshop onMM

Semantics

Workshop onMM

Semantics

How to transfer media dependent knowledge?

Multimedia Concepts

• Explanation of activities

• Media versatility

• Unspecified information

• Sample creation

• Learning by doing

Ontologies

• Abstraction of information

• Strict classification and categorization of data

• Modeling of data sources and information types

• Creation of domain specific structures

• Predefined data structures

Lehrstuhl Informatik V(Informationssysteme)

Prof. Dr. M. Jarke

M. Spaniol,R.Klamma,M. Jarke

I5-SKJ-2911-5/14

Workshop onMM

Semantics

Workshop onMM

Semantics

Goal: Combining their benefits by simultaneously reducing their deficits

Analysis ofknowledge management concepts

Multimedia Concepts Applications

Ontology based Information Systems

Content adaptability o o

Media versatility + o

Community suitability + -

Interactivity o o

Personalization - +

User interface flexibility o o

Application popularity + o

Lehrstuhl Informatik V(Informationssysteme)

Prof. Dr. M. Jarke

M. Spaniol,R.Klamma,M. Jarke

I5-SKJ-2911-6/14

Workshop onMM

Semantics

Workshop onMM

Semantics

Knowledge creation framework: Transcription - Addressing - Localization

• New computer science approach:Combination of ontologies and multimedia concepts

Supporting Transcription

by (re-) combination of

transcription tools

Stabilization Addressing

Optimizing Localization

• Testing in an interdisciplinary collaborative research center

Externalization

Internalization

implicit

explicitimplicit

explicit

CombinationSocialization

The Knowledge-Creating Company [Nonaka, Takeuchi 95]

Transcribing – Media/Reading [Jäger, Stanitzek 02]

Signals of disturbance [Kümmel, Schüttpelz 02]

MultimediaConcepts

Ontologies

Lehrstuhl Informatik V(Informationssysteme)

Prof. Dr. M. Jarke

M. Spaniol,R.Klamma,M. Jarke

I5-SKJ-2911-7/14

Workshop onMM

Semantics

Workshop onMM

Semantics

Semantics in community multimedia

Lehrstuhl Informatik V(Informationssysteme)

Prof. Dr. M. Jarke

M. Spaniol,R.Klamma,M. Jarke

I5-SKJ-2911-8/14

Workshop onMM

Semantics

Workshop onMM

Semantics

Structural and content based KMwith XML and MPEG-7

Basis

Schema- Links&Media Basic Basic Tools Localization Tools datatypes

XML

MPEG-7

[IEEE 02]

Userinteraction

User preferences

UsageHistory

Navigation & Access

Index

Views

Variations

Content organization Collections Models

Creation & Production

Content Management Media Usage

Content description

Structural aspects Semantic aspects

Lehrstuhl Informatik V(Informationssysteme)

Prof. Dr. M. Jarke

M. Spaniol,R.Klamma,M. Jarke

I5-SKJ-2911-9/14

Workshop onMM

Semantics

Workshop onMM

Semantics

<Mpeg7>...<ContentDescription xsi:type="ContentEntityType">

<MultimediaContent xsi:type="VideoType">...

<VideoSegment id="seg1"><TextAnnotation type="Question">

question</TextAnnotation><TextAnnotation type="Categories">

related categories</TextAnnotation><TextAnnotation type="Phases">

phases of an entrepreneurship</TextAnnotation><TextAnnotation type="Persons">

founder</TextAnnotation><TextAnnotation type="Companies">

company name</TextAnnotation><MediaTime>

starting and ending point of the medium</MediaTime></VideoSegment>

...</MultimediaContent>

</ContentDescription> ...

</Mpeg7>

Structuring of heterogenous Data data XML

3-dimensional Categorization

Multimedia Information

Additional Information

Lehrstuhl Informatik V(Informationssysteme)

Prof. Dr. M. Jarke

M. Spaniol,R.Klamma,M. Jarke

I5-SKJ-2911-10/14

Workshop onMM

Semantics

Workshop onMM

Semantics

<Mpeg7><DescriptionMetadata id="Kais Rolle">

general information about the document</DescriptionMetadata><ContentDescription xsi:type="ContentEntityType">

<MultimediaContent xsi:type="MultimediaCollectionType"><ContentCollection name="Kais Rolle”>

<TextAnnotation>Collection type (public/private)

</TextAnnotation><Content xsi:type="ImageType">

Medium, technical Data, Annotations</Content><Content xsi:type="VideoType">

---</Content>

</ContentCollection></MultimediaContent>

</ContentDescription><ContentManagement xsi:type="UserDescriptionType">

user management</ContentManagement>

</Mpeg7>

Content and contextdescription by MPEG-7

Multimedia Information

Transcription

Addressing

Lehrstuhl Informatik V(Informationssysteme)

Prof. Dr. M. Jarke

M. Spaniol,R.Klamma,M. Jarke

I5-SKJ-2911-11/14

Workshop onMM

Semantics

Workshop onMM

Semantics

Case study:Didactic concept of VEL

• Hypermedia learning environment• Extensive, close to reality media material• Explorative, non-linear learning• Context dependent presentation• Repeated considerations due to context changes• Learning structure:

– Student = explorer– Teacher = consultant, mentor

• Public / private collections• Users‘ annotations

Lehrstuhl Informatik V(Informationssysteme)

Prof. Dr. M. Jarke

M. Spaniol,R.Klamma,M. Jarke

I5-SKJ-2911-12/14

Workshop onMM

Semantics

Workshop onMM

Semantics

Virtual Entrepreneurship Lab (VEL)

Lehrstuhl Informatik V(Informationssysteme)

Prof. Dr. M. Jarke

M. Spaniol,R.Klamma,M. Jarke

I5-SKJ-2911-13/14

Workshop onMM

Semantics

Workshop onMM

Semantics

• MPEG-7 helps to overcome gap betweenMultimedia Concepts and Ontologies

• Combining loose characterization and generic classificationwith domain modeling and categorization

• Research on personalization and adaptive user interfaces

• Current projects at : i5

SOCRATES: Chat/Talk-tool for aphasics self help groupsMARS: Electronic music analysis toolMECCA: Virtual movie archive for cinematic scientists

Covering multimedia semantics by MPEG-7 Personalized multimedia addressing

Conclusion & Outlook

Lehrstuhl Informatik V(Informationssysteme)

Prof. Dr. M. Jarke

M. Spaniol,R.Klamma,M. Jarke

I5-SKJ-2911-14/14

Workshop onMM

Semantics

Workshop onMM

Semantics

Contact

mspaniol@cs.rwth-aachen.de

klamma@cs.rwth-aachen.de

jarke@cs.rwth-achen.de

www.graeculus.de