MotivationResearch topics

Conclusion

Interoperable and Unified Multimedia Retrieval inDistributed and Heterogeneous Environments

Florian Stegmaier

Supervisor: Harald Kosch

Co-Supervisor: Mario Doller

Chair of Distributed Information SystemsUniversity of Passau, Germany

October 26th, 2010

Presenter: Florian Stegmaier, University of Passau ACMMM10 – Interoperable and Unified Multimedia Retrieval

MotivationResearch topics

Conclusion

Outline of this talk

1 MotivationMultimedia retrieval in today’s worldDistributed and heterogeneous retrieval use cases

2 Central research questionHow to provide an unified access to distributed andheterogeneous environments?How can query semantic be maintained during metadata andquery language transformation?Can query classification and backend benchmarking establishan efficient / intelligent data fusion process?

3 Conclusion

Presenter: Florian Stegmaier, University of Passau ACMMM10 – Interoperable and Unified Multimedia Retrieval

MotivationResearch topics

Conclusion

Multimedia retrieval in today’s worldDistributed and heterogeneous retrieval use cases

Multimedia retrieval in today’s world

Multimedia data can be produced, published and consumed byeverybody everywhere at any time

Petabytes of multimedia data and metadata stored andoffered in popular systems

Central research topics i am facing in my PhD thesis

Unified multimedia retrieval, metadata interoperability, queryexpansion and data fusion

Presenter: Florian Stegmaier, University of Passau ACMMM10 – Interoperable and Unified Multimedia Retrieval

MotivationResearch topics

Conclusion

Multimedia retrieval in today’s worldDistributed and heterogeneous retrieval use cases

Use cases for distributed and heterogeneous retrieval

Local query processing

Find images which are similar to“aGivenImage.jpg” andannotated with “vacation” andtaken at “Lake Garda”!

Distributed query processing

Medical examination system

Image Retrieval System

(CT Scans)

DICOM(patient data)

RDF triple store(diseases and anatomy)

query query query

Select CT scan, where disease is“lesion”, anatomy “near liver”and the CT scan is similar to“aGivenCTScan.jpg”!

Presenter: Florian Stegmaier, University of Passau ACMMM10 – Interoperable and Unified Multimedia Retrieval

MotivationResearch topics

Conclusion

Unified access to distributed environmentsMetadata interoperability and query transformationEfficient / intelligent data fusion process

Unified access to distributed, heterogeneous environments

Two fundamental decisions:

supported query language:MPEG Query Format (query language & managementcapabilities), JPSearch compliant for still images

architecture: external metasearch engine

Presenter: Florian Stegmaier, University of Passau ACMMM10 – Interoperable and Unified Multimedia Retrieval

MotivationResearch topics

Conclusion

Unified access to distributed environmentsMetadata interoperability and query transformationEfficient / intelligent data fusion process

Syntactic metadata transformation

JPSearch transformation rules1:

XML based

JPSearch Core Schema as pivot metadata schema

registration, identification and management of metadataschemas and transformation rules

1Presented at SAC2010 (Track ASIIS).

Presenter: Florian Stegmaier, University of Passau ACMMM10 – Interoperable and Unified Multimedia Retrieval

MotivationResearch topics

Conclusion

Unified access to distributed environmentsMetadata interoperability and query transformationEfficient / intelligent data fusion process

Semantic and syntactical metadata transformation

W3C Ontology and API for Media Ressource2:

Ontology based

Media Ontology as pivot metadata schema

syntactic mappings by the use of rules, semantic mappingexpressed by e.g. SKOS

2Reference implementation will be presented as Demo at SAMT2011 – online at: http://mawg.joanneum.at/

Presenter: Florian Stegmaier, University of Passau ACMMM10 – Interoperable and Unified Multimedia Retrieval

MotivationResearch topics

Conclusion

Unified access to distributed environmentsMetadata interoperability and query transformationEfficient / intelligent data fusion process

Query reformulation and expansion

Issue during metadata transformation: maintain query semantic

Dynamic query expansion

“Google-like” search

inserting new data sources while query execution(e.g., extracting metadata of a Query-By-Example andforward this to a metadata based retrieval service or use of thelinked data cloud)

Presenter: Florian Stegmaier, University of Passau ACMMM10 – Interoperable and Unified Multimedia Retrieval

MotivationResearch topics

Conclusion

Unified access to distributed environmentsMetadata interoperability and query transformationEfficient / intelligent data fusion process

Knowledge based data fusion process

Knowledge based on:

Benchmarking content based multimedia retrievalClassification of MPEG query format requests

→ estimating retrieval quality of connected data sources to chooseappropriate algrithms3

3Presented at AMR2010.

Presenter: Florian Stegmaier, University of Passau ACMMM10 – Interoperable and Unified Multimedia Retrieval

MotivationResearch topics

Conclusion

State of researchDiscussion

State of research

Presenter: Florian Stegmaier, University of Passau ACMMM10 – Interoperable and Unified Multimedia Retrieval

MotivationResearch topics

Conclusion

State of researchDiscussion

Discussion

Thank you for your attention . . .

. . . any questions?

Contact informations:

Email: florian.stegmaier@uni-passau.de

Twitter: http://www.twitter.com/fstegmai

Presenter: Florian Stegmaier, University of Passau ACMMM10 – Interoperable and Unified Multimedia Retrieval