Date post: | 17-Jan-2017 |
Category: |
Technology |
Upload: | blue-bridge |
View: | 191 times |
Download: | 0 times |
BlueBRIDGE receives funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 675680 www.bluebridge-vres.eu
Virtual Research Environments supporting tailor-made data management services
for marine & maritime sector
13 October 2016 - Brest
Pasquale PaganoCNR, [email protected]
Virtual Research Environments for supporting tailor-made data management services 2
Challenges and Opportunities
Data and Services
Hosted by different Organizations
Accessible through different Protocols
Described with different Metadata
Policies
Different approaches for Credits
Different Licenses
Different Terms of Use
Heterogeneity
Support Validation, Curation,
Harmonization
Measure Uncertainty
Trace Provenance
Modern science is increasingly global, multi-disciplinary and networked
Virtual Research Environments for supporting tailor-made data management services 3
Data Analytics
• are multidisciplinary, involve members belonging to diverse organisations • require to access data and services that are spread among many providers
dynamically aggregated to address questions/problems
• cannot rely on pre-organised and costly supporting environments managed by dedicated organizations
build and operate their own supporting environments
wish to effectively inject new approaches in daily tasks
cost and time required to implement this approach largely exceed the available capacities
Not performed by individuals but group of data analysts
Virtual Research Environments for supporting tailor-made data management services 4
Requirements for IT systems
• Support collaborative data analysis and experimentation
• Implement Traceability and Reproducibility-Repeatability-Reusability
• Enable secure and controlled data sharing
• Tackle simplified access to existing data and processes
• Tackle simplified access to existing computing and storage resources
• Ensure low operational and maintenance costs
• Manage heterogeneous data access policies
Virtual Research Environments for supporting tailor-made data management services 5
Virtual Research Environment
An operational environment
• Where set of resources (data, applications, computational, and storage resources)
• are assigned to group of users via interfaces
• for a limited timeframe
• by hiding complexity of hardware setup and software configuration
L. Candela, D. Castelli, P. Pagano (2013) Virtual Research Environments: An Overview and a Research Agenda. Data Science Journal, Vol. 12
Created on demand
Regulated by tailored policies
No cost for the resource providers
Open to host and operate custom software
Virtual Research Environments for supporting tailor-made data management services 6
Application BundlesReady to use technologies
To develop applications interfacing gCube facilities
AppsCubeTo aid modelling and analysing of distribuition data, comparing checklists, and producing maps
BiolCube
To facilitate data publication with appropriate tools including semantic technologies
ConnectCube
To properly access, consume and produce geospatial information
GeosCube
To assist tabular data validation, data enrichment ad efficient analytical tools
StatsCube
To support deployment, operation & mgmt of a data infrastructure
IceCube
Virtual Research Environments for supporting tailor-made data management services 7
VRE Creation
Configuration
ApplicationsMetadata
Data
Simple and effective process to define a new environment
Virtual Research Environments for supporting tailor-made data management services 8
Applications vs Services
Registry
Logi
cal
View
Applications Data
Phys
ical
View
Hardware
Software, Tools, Services
Configuration
Data
VRE Workflow EnablerSPD (BiolCube)ecological and biological data
GeoExplorer (GeosCube)geospatial data
Tabular Data (StatsCube)statistical and reference data
SAI (StatsCube)process importer
(Con
nect
Cube
)
Virtual Research Environments for supporting tailor-made data management services 9
Data Miner (StatsCube)data analytics for interdisciplinary domains
Virtual Research Environments for supporting tailor-made data management services 10
VREs
• Cloud computation• Web interface
available for non experts
• Standard WPS API for easily integration
• 90% processing time reduction
Stock AssessmentEstimates Maximum Sustainable Yield, Biomass, CPUE and catchability from catch statistics, biomass, landings etc.
Virtual Research Environments for supporting tailor-made data management services 11
Performance Evaluation In Aquaculture
Aquafarming assessment tools enacting perform evaluation growth analysis and techno economic investment analysis
Capabilities• Production Planning• Financial Forecast• Skill Building (What-IF)• KPI extraction
• Feed Conversion Rate(FCR)• Growth Per Day(GPD)• Specific Growth Rate (SGR)• Suggested Feeding Rate (SFR)• Mortality Rate (MR)
Virtual Research Environments for supporting tailor-made data management services 12
Biodiversity
Fill knowledge gaps on marine speciesAccount for sampling biasesDefine trends for common species
Plankton regime shift
Herring recovered after the fish ban
LME - MEOW
Virtual Research Environments for supporting tailor-made data management services 13
Fishing Activity
ForecastingTrajectories Analysis
Virtual Research Environments for supporting tailor-made data management services 14
Ecology
Atlantic cod
Coelacanth
Giant squid
AquaMaps
Neural Networks
Neural Networks and MaxEnt
15
Geospatial data processing
Maps comparison
NetCDF
Data extraction Signal processing Periodicity detection
Maps generation
Virtual Research Environments for supporting tailor-made data management services
Virtual Research Environments for supporting tailor-made data management services 16
VREs in operation
Data Infrastructures Computing Infrastructures
Mediator Connector Mediator Connector
Data Curation
Data Preparation
Data Analysis
Data Sharing
Data Publication
Data Provenance
VRE Builder
Security
Monitoring
Marine and MaritimeDigital Humanities
Geothermal
Social Mining
Virtual Research Environments for supporting tailor-made data management services 17
VRE Social NetworkingSocial networking is key to share information in the VRE
It offers a continuously updated list of events / news produced by users and applications
Access VREsDiscuss and
Validate
Share Data, News, Processes
Virtual Research Environments for supporting tailor-made data management services 18
VRE Common WorkspaceA folder-based file system allowing
managing and sharing information objects
Information objects can be
• files, dataset, workflows, experiments, etc.
• organized into folders
Users can
• Share with selected users
• disseminate via persistent public URLs
Virtual Research Environments for supporting tailor-made data management services 19
VRE Software Integration
Download the (python, R, Java, …) script and the user’s data
Execute script
Collect output
Destroy local copies of I/O and script
Save Output on the User’s Workspace, with provenance info
Scientist’s provided script
User’s data
Infrastructure
Virtual Research Environments for supporting tailor-made data management services 20
VRE Collaborative Experiments
WS
Shared online folders
Inputs
Outputs
Results
Computational system
In the e-Infrastructure
Through third party software
Virtual Research Environments for supporting tailor-made data management services 21
VRE Enabling New Workflow
Script provider
Updates the script on his private Workspace
The service downloadsthe script on-the-fly
A user executes an experiment on his/her data
The output, the input and the parameters can be shared with another user
This user can execute the experiment againand share the computation with other users
1
2
3
4
5
6
7
89
10
Virtual Research Environments for supporting tailor-made data management services 22
ConclusionsVRE are defined by users and created on demand• New software can be integrated and used as-a-Service• Invoked via standard interfaces
VRE ensures • Provenance management• Access via an easy-to-use storage system• Collaboration and sharing
VRE enables • Complex workflows• Repeatability, Reproducibility and Reusability
Virtual Research Environments for supporting tailor-made data management services 23
Visit us at www.bluebridge-vres.euTry it at i-marine.d4science.org