Lime broker

© Fraunhofer IDMT

Patrick Aichroth, Marcel Sieland, Luca Cuccovillo, Thomas Köllmer

The MICO Broker: An Orchestration Framework for Linked Data Extractors

Fraunhofer IDMT

Thomas Köllmer

[email protected]

© Fraunhofer IDMT 2

The MICO Project

http://www.mico-project.eu

3.4 Million Euro Project

7 Partners: Salzburg Research, Fraunhofer IDMT, Oxford University,

University of Passau, Umeå University, Zaizi, InsideOut10

Duration: 36 Months (end: 10/2016)

Goal of MICO is to develop a platform to analyse “media in context ” …

...by orchestrating different content extraction tools that can work simultaneously or in a sequence and …..

...provide valuable metadata to third party applications.


MICO and the MICO brokerMICO: EU project to develop a platform for distributed analysis of multimedia content, based on Linked Data:

media (textual, visual, audio) extractors

persistence (binary + metadata/annotations)

extractor orchestration (“broker”)

common metadata model (MMM, Anno4J)

cross-media querying (SPARQL/MM)

cross-media reco (using all data sources, content-based + CF-based)


MICO and the MICO broker


An example workflow / pipeline






Initial broker version

implemented on top of RabbitMQ

accessible via MICO Event API (Java and C++)

content item input queue: receiving new content

content item reply-to queue: providing extractor results

registry queue: extractor service registration

Initial RabbitMQ approach proved to be quite stable and robust, but:

orchestration via simple mime-types: unintended processing and loops

Other features desired → new Broker version needed

Broker challenges and requirements (1/4)



Extractor dependencies and parameters

support for extractor configuration

support for extractor “modes”

support for more than one extractor input and output

support for extractor versioning

Workflow creation

support for extractor dependency checking

simple workflow creation


Workflow execution

support for several workflows per MICO instance

support for error handling

support for workflow status tracking and logging

support for Enterprise Integration Pattern (EIP) support, e.g. routing, aggregation, splitting

support for dynamic routing for context-aware processing





Extractor information is provided at different times (during vs. after packaging) and by different people (dev, other dev, admin)

→ dedicated registration REST service

Extractor I/O typing knows different layers:

mime-types, e.g. image/png,

syntactic types, e.g. image region,

semantic types, e.g. face region

All three must match!

New MICO broker: Design Choices (1/2)


New MICO broker: Design Choices (2/2)

Existing Linked Data sources and the MICO metadata model (MMM) to be reused as far as possible; types should be identified via URIs

Apache Camel for workflow execution: EIP and broad protocol support

Support for semi-automatic workflow creation via dedicated GUI/App

Support for admin feedback

extractors performance

pipeline performance

Support for content sets, and assignment of content sets to workflows


New MICO broker: components & overall process


New MICO broker: data model




New MICO broker: semi-automatic workflow creation

Manual workflow creation: complicated (depends on extractor interdependencies and constraints, content, use case etc.)

Now: Semi-automatic approach uses data model (mime types, syntactic and semantic types)

Output: Camel route


Example Camel route


Example Camel route


New MICO broker: status and outlook

Part of the final MICO release (late 2016)

Current broker status:

Mostly completed

Partially integrated into the platform release

Basic functionalities released under Apache OSS

Future functionalities planned as premium features:

Advanced semi-automatic workflow creation

Advanced distributed process management

Advanced security features

© Fraunhofer IDMT



Fraunhofer IDMT

Ενχαριδτώ!Questions?

© Fraunhofer IDMT



Fraunhofer IDMT

Thank you!Questions?









Date post:	20-Jan-2017
Category:	Technology
Upload:	john-pereira
View:	273 times
Download:	0 times

Lime broker

Technology