DORII reviewRemote instrumentation

communities andapplication support

NA3Roberto Pugliese

roberto.pugliese@elettra.trieste.it

Sincrotrone Trieste SCpA

July 3, 2009, Brussels

Workpackage introduction Workpackage objectives Workpackage deliverables Workpackage overall schedule Workpackage achievements Workpackage future plans Workpackage Summary

Outline

Remote instrumentation communities and application support

Give support for researchers for adaptation and integration of their applications in the extended e-Infrastructure

NA3 Objectives

Connection with other WPs

Subtasks3.1 community support 3.1.1 describe feature of the eInfrastructure3.1.2 document special requirements3.1.3 document estensions to the eInfrastructure3.2 application support deploy of instruments 3.2.1 adaptation of instruments 3.2.2 adaptation of hci components 3.3 application support deployment of services 3.3.1 access portals and tools 3.3.2 adaptation of workflows and collective 3.4 application support deploy evaluation 3.4.1 definition of metrics3.5 application demonstration support3.5.1 prepare demos

Tasks and subtasks

Del. No Deliverable Name WP No

Nature Dissemination Level

Del. Date

DNA3.1 Requirement analysis for pilot and non-pilot applications and evaluation metrics definition

NA3 Report PU M 3

DNA3.2 Pilot applications – early deployment report and evaluation

NA3 Report PU M 15

DNA3.3 Pilot and non-pilot applications – deployment report and evaluation

NA3 Report PU M 22

DNA3.4 Pilot and non-pilot applications – final deployment report and evaluation

NA3 Report PU M 30

Deliverables

QuickTime™ and a decompressor

are needed to see this picture.

Schedule

Partner NA3.1 NA3.2 NA3.3 NA3.4 NA3.5 SUMCSIC 4 6 9 6 4 29CNIT 5 5ELETTRA 4 20 12 6 3 45EUCENTRE 3 24 3 30OGS 4 24 2 30ECOHYDROS 4 18 2 24UC 4 18 2 24Total 23 110 26 12 16 187

NA3.1 NA3.2 NA3.3 NA3.4 NA3.5

23 PMS 110 PMs 26 PMs 12 PMs 16 PMs

Effort distribution

Earthquake communityNetwork-centric seismic simulations

Environment communityOceanographic and coastal observation and

modelling Mediterranean Ocean Observing Network

Oceanographic and coastal observation and modelling using imaging

Experimental science communityOn-line data analysis in experimental science

Communities and pilot apps

Earthquake communityEarthquake early warning system design and

simulationEnvironment community

Monitoring inland waters and reservoirsExperimental science community

On-line data analysis in experimental science (other use cases)

Communities and not pilots

Described to the communities the existing e-Infrastructure

Requirement analysis for pilot and not pilot applications and evaluation metrics definition

Match the applications in the Instrument Infrastructure Architecture (VCR, CE, SE, IE, …)

For each application (focus on pilots)defined a todo list (wbs, …)defined a plan constantly monitor application status and progress

Achievements

QuickTime™ and aBMP decompressor

are needed to see this picture.

Architecture

DORII applications descriptionRequirements

General, instrumentation,Data, storage, computing,Visualisation, interactivity,Networking, collective

Metrics definitionproduction, usabilityPerceived gain (performance, reach, …)

Requirements templateSpecific requirements

Requirements

Status of the VCRStatus of the VCR personalisationStatus of the IEStatus of IE personalisation (IM

implementation)Status of CE (porting of the application code)Status of the SE (connection with the IE)Status of the collective behaviour (workflow

and connection of components)

Application status

We meet regularly every week via EVOMeetings are focused on application short

term and long term needsMeetings try to give answers to specific

application problemsMeetings have an agendaMeetings have minutes with specific action

itemsWe have installed an online survey tool to

collect feedback and perform evaluations

Teamwork achievements

Online survey

Involves instruments computing, storage not yet collectively used, it is using he application manager and mpi

Next developments glider integration, separated VCR for personalisation and better performance, visualisation of NetCDF files

OGS Pilot Application

Involves instruments, computing, storage, not yet collectively used, application manager, example of system level science similar to the NEESGrid

Next developments other instruments integration, separated VCR for personalisation and better performance

EUCENTRE Pilot Application

Involves instruments, computing, storage not yet collectively used, it is using the application manager

Next developments more imaging algorithms, separated VCR for personalisation and better performance

UC Pilot Application

(interactive sci.app: fit2D)

(BL instrument)

(VCR)

(user in ELETTRA)

(user @home)

Online/GRID: CE,SE,etc

ELETTRA SAXS Application

Earthquake early warning system design and simulationInstrument integration in progress

Monitoring inland waters and reservoirsInstrument integration in progress and modelling

almost doneExperimental science community

Syrmep application under development; impressive tomography reconstruction

XRD yet to start, requirement for taverna integration

Status of not pilot apps

It has been developed to demonstrate and DORII (ICT2008, INGRID2009) and for training purposes actually involves only instruments, if we include the remote camera than we can demonstrate also instrument, storage integration

Next developments computing integration and permanent setup

Robot Demo Application

Experimental procedure Focus on end users Access to the DORII infrastructure via VCR For each pilot application we have defined

list of key users (not involved in DORII)a set of scenariosa task list for each scenario

Each key user will first receive a basic training on DORII and the use of the VCR

After the training each key user will execute the scenario

After executing the scenario each user will fill an online questionnaire

Early deployment evaluation

Laboratory studies Subjective assessment General assessment Specific feature assessment Specific task assessment

Regults Overall rating good Security perceived too complex VCR perceived as slow Improvement required for

reliability, application manager,visualisation capabilities, automatisation, metadata support,help and training

Evaluation process ok

Evaluation results

Need for a new VCR Need for a better application manager Need for a Common Library Need for a workflow management system Need to have a customized VCR at least for each

community Need to simplify security (robot certificates) Need for better training material Need for a summer school

Feedback to JRA and SAs

When the first release of the middleware will be available we will adapt pilot applications and port the other applications to production and operational use

We will provide further feedback and finalize thee-Infrastructure

Future plans

Papers, presentations and demonstrations have been included in the dissemination workpackage

VCR for all SRF users (SaaS) MADBAG meteo and air quality data acquisition,

nowcasting and forecasting using modelling, geographical visualisation of results

…

Other archievements

NA3 workpackage is about supporting instrumentation communities and helping them porting their applications to the e-Infrastructure

The involved effort is huge and has to deal not only with technical aspects You have to build trust which is not easy

During this first half of the project we manage to establish a common language, establish a way to work together, collect requirements, deploy pilots, setup an evaluation procedure, provide feedback to the JRA and Sas, complete deliverables

Moreover we prepared good quality demo and involve new communities

We will now focus on pilot application improvements, not pilot application deployment, evaluation and feedback to JRA and SAs to obtain a production quality middleware and infrastructure.

Summary

All the pilot applications are available for demonstration on request

We will demonstrate the DORII e-Infrastructure staus with the SAXS application which is complete, shows integration of instrument, storage, computing resources,applications which can be interactive, online (new to the Grid world) and offline (traditional Grid use), and moreover is in production status and presents a collective use of the resources via application manager and workflow management system

Introducing the demo