Date post: | 04-Jan-2016 |
Category: |
Documents |
Upload: | robyn-curtis |
View: | 214 times |
Download: | 1 times |
Using Several Ontologies for Describing Audio-
Visual Documents:A Case Study in the Medical
Domain
Sunday 29th of May, 2005
Antoine Isaac1 & Raphaël Troncy2
Multimedia and the Semantic Web
2005/05/29 A. Isaac & R. Troncy - MSW'2005 2
Describe AV documents
• Various uses / Different granularities– Identification, feature extraction, structural decomposition,
semantic description
• Description deep meaning cannot be accessed and processed by systems– Knowledge is often implicit: labels and comments in natural
language
• Formal semantics should be interesting– Reasoning with AV document descriptions– Interoperability with formal domain-specific ontologies, allowing
to mix AV and domain-related reasoning
Use of Semantic Web technologies to better retrieve, re-use and process AV content
2005/05/29 A. Isaac & R. Troncy - MSW'2005 3
Objectives
• Settle an mini-experiment to show the benefits of using semantic web technologies for annotating multimedia content
• Show that the use of:– formal ontologies and rules,– inference capabilities,– annotation design pattern
… are highly desirable for better accessing AV content
2005/05/29 A. Isaac & R. Troncy - MSW'2005 4
Agenda
• Corpus
• Ontological Resources– AV Ontology– Medical Ontology
• Annotating the Videos
• Querying the Knowledge Base
• Performing Reasoning
• Conclusion
2005/05/29 A. Isaac & R. Troncy - MSW'2005 5
Corpus
• Medicine-related TV documentaries– 30 documents, about 30 hours– 50% deals with heart and heart surgery theme
• Good examples of how AV features are used to popularize scientific notions
• Describe both the form and the content– AV-oriented parts (documentary structure)– Thematic-oriented parts (medicine notions)
2005/05/29 A. Isaac & R. Troncy - MSW'2005 6
Ontological Resources
• Building an Audio-Visual Core Ontology [Isaac & Troncy, 2004]– Characterization of programs and sequences (AV genre)– Decomposition of programs and sequences– Ability to introduce description of the activities that constitute the
context of AV documents (roles of people involved, way production and broadcast are achieved, etc.)
• Legitimacy– Grounding conceptualization by observed purposes and domain
initiatives, study of 30 years of documentary practices– Articulation with an upper-level ontology: DOLCE [Gangemi,
2002]
2005/05/29 A. Isaac & R. Troncy - MSW'2005 7
Ontological Resources
<owl:Class rdf:ID="DialogSequence"> <rdfs:subClassOf rdf:resource="#SpokenSequence"/> <rdfs:subClassOf> <owl:Restriction> <owl:onProperty> <owl:ObjectProperty rdf:about="#hasParticipant"/> </owl:onProperty> <owl:minCardinality rdf:datatype="&xsd;int">2</owl:minCardinality> </owl:Restriction> </rdfs:subClassOf> </owl:Class>
2005/05/29 A. Isaac & R. Troncy - MSW'2005 8
Ontological Resources
• Extension of AV core with specific application notions– Exemplification, demonstration, etc.
• Re-use of Medical Ontologies– Menelas: domain of coronary pathologies
• Concepts dealing with heart surgery
– Alternative choices are possible• Galen (concepts dealing with surgical procedures)
• Articulation between the ontologies– No use of automatic alignment methods or tools– State by hand OWL axioms (equivalentClass)
2005/05/29 A. Isaac & R. Troncy - MSW'2005 9
Description Process
• Segmentation of the AV material– Selection of relevant documentary items
• Knowledge-based AV description– Documentary structure characterization– Segment description
2005/05/29 A. Isaac & R. Troncy - MSW'2005 10
Segmenting the Videos
2005/05/29 A. Isaac & R. Troncy - MSW'2005 11
Describing the Videos
• Annotation Mechanism– The structure is described at the knowledge
level• Concepts and relations from the AV ontology are
manually introduced in the description
– Content description• Link to external world themes and entities
• Documentary patterns– Layered approach [Troncy, 2003]
– AV description language [Troncy & Carrive, 2004]
2005/05/29 A. Isaac & R. Troncy - MSW'2005 12
Describing the Videos
• Relational Indexing Pattern– Help for user: specify how concepts and relations
have to be used– Important for ontology conception and use (with
reasoning knowledge)
• Simple pattern that can lead to complex descriptions– Recursive relational structure
2005/05/29 A. Isaac & R. Troncy - MSW'2005 13
Describing the Videos
2005/05/29 A. Isaac & R. Troncy - MSW'2005 14
Querying the Knowledge Base
• Example:« retrieve the programs that explain a disease and
show one of its causes »
• Need for the following inferences:– Subsumption
– Composition
Disease(x)x)CVDisease(
z),explains(x z),explains(yy)ence(x,hasSubSequ z)shows(x, z)shows(y, y)ence(x,hasSubSequ
2005/05/29 A. Isaac & R. Troncy - MSW'2005 15
Performing reasoning
• A layered complexity approach– RDFS: subsumption– OWL DL: complex definitions + algebraic properties– Rules: horn clauses
• Concrete implementation– RDFS: Sesame Architecture [Broekstra, 2002]
– OWL DL: BOR Reasoner [Simov, 2002] – OWL-DLP [Grosof, 2003] + Rules : Sesame custom
inference module
2005/05/29 A. Isaac & R. Troncy - MSW'2005 16
Examples
• DL definition
• Composition rule
),(),(),( zxrepresentszyrepresentsyxencehasSubSequ
))(
)((
))(
(
nRoleinstitutio
lehospitalRoalRoleprofessionleacademicRoExpertRole
ExpertRolerolesomePersonpanthasParticisome
InterviewrviewExpertInte
2005/05/29 A. Isaac & R. Troncy - MSW'2005 17
Summary
Explicit triples
Inferred triples
All triples
RDF Model 129
AV Ontology 5231 10810 16041
Menelas Ontology
10534 26637 37171
Instances 276 1507 1783
Total 16041 38954 54995
2005/05/29 A. Isaac & R. Troncy - MSW'2005 18
Conclusion
• This experimentation:– Uses Semantic Web languages and tools for describing AV
contents– Uses several ontologies to capture both the structure and the
content of the documents– Uses relational indexing patterns for the annotation
• Future work: thorough evaluation of those techniques involving real users
• A problem that cannot be generally solved: fixing a trade-off between expressivity and tractable computation– Ad hoc, according to the needs of the application targeted