Post on 13-Jul-2015
transcript
The LoCloud MoRe aggregator
Gavrilis Dimitris Afiontzi Eleni, Makri Dimitra -‐ Nefeli
Digital CuraAon Unit -‐ IMIS, Athena Research Center
LoCloud is funded by the European Commission's ICT Policy Support Programme
• Project info – Best pracAce network FP7 project – 36 months – 33 partners – 4.250.000 €
• Goal – Establish a repository based aggregator and deliver over 5 million
records to Europeana – Take advantage of cloud based technologies in order to minimize
operaAonal and administraAve costs – Provide services and tools to reduce technical, semanAc and skills
barriers – Facilitate the aggregaAon from small insAtuAons with liXle or no
technical experAse and support
Local Content in a Europeana Cloud
LoCloud Architecture
Index Database
LoCloud Core Services Layer
Storage Node
Storage Node
Storage Node
Storage Node
Lightweight Repository
MINT Enrichment Services
Content Providers
LoCloud Infrastructure
Export
Europeana
LoCloud Architecture
Data Access Layer
Storage Node Storage Node Storage Node Storage Node
Enrichment Service ValidaAon Service
Services Layer
LDL
Voc. Match OAI-‐PMH
MINT
Wikimedia
Geo-‐services
Language Ident.
Vocabularies
Structural
Link checking
Rules Based
Schematron
Four cloud environments
• Storage • Monitoring & logging • Core services deployment • Enrichment services deployment
Distributed
• Enrichment services run on: – Austria – Spain – Greece – Lithuania – Slovenia
• Scalability can be facilitated through AIT’s services cloud
• HarvesAng • ValidaAon • IngesAon • TransformaAon • Enrichment • Previewing • Publishing
Core services
• Harvests content from metadata sources – Generic OAI-‐PMH repository – MINT – LoCloud CollecAons – Wikimedia
• MulAple schemas are supported – OAI_DC – CARARE – CARARE 2.0 – LIDO – EAD – EDM – ESE
HarvesAng service
• Validates incoming informaAon packages – Executes validaAon schemes – ValidaAon micro-‐services – Flexible
• How it is used in MoRe: – Pre-‐validaAon – Post-‐validaAon
ValidaAon service
• Structural • Schema validaAon • Link checking • Schematron rule validaAon
ValidaAon micro-‐services
• Ingest content into storage – Uses storage Layer API – Pluggable drivers for aXaching different technologies / repositories • Apache Cassandra • Filesystem-‐based • Fedora-‐commons
– Versioning support – Complex digital object support
Ingest service
• Transforms enAre informaAon packages into EDM (or any other schema) – MulAple transformaAon rouAnes
• Per schema • Per project • Per provider
– User can aXach rights statement
TransformaAon service
• The Generic enrichment services facilitates the execuAon of the enrichment micro-‐services – Hides the complexity from the user by using enrichment plans
– Hides the mulAple schemas complexity from the service providers by using micro-‐schemas
– Provides seamless integraAon with MoRe’s UI
Enrichment service
• ThemaAc – Thesauri collecAons – Vocabulary matching – Background links
• SpaAal – Geo normalizaAon – Geo coding – Reverse geo-‐coding – Historic place names
• Other – Language idenAficaAon
Enrichment micro-‐services
Enrichment Services Framework
Language IdenAficaAon
Enrichm
ent H
armon
izaAo
n
SpaAal IdenAficaAon
Temporal IdenAficaAon
Reverse Geo-‐coding
SpaAal TranslaAon
Thesauri Enrichment
• Orchestrate enrichment micro-‐services into powerful enrichment workflows
Enrichment plans Language
IdenAficaAon
Temporal IdenAficaAon
Thesauri Enrichment
Reverse Geocoding
SpaAal TranslaAon
• Publishes user selected content to Europeana (or any other target)
• Provides the informaAon through OAI-‐PMH 2.0 • User can select which data-‐stream to publish (e.g. EDM, Enriched EDM)
Publishing service
Thank you! d.gavrilis@dcu.gr e.afiontzi@dcu.gr n.makri@dcu.gr
LoCloud is funded by the European Commission's ICT Policy Support Programme
The views and opinions expressed in this
presentaAon are the sole responsibility of the authors and do not necessarily reflect the views of
the European Commission.
Funding