Integration of Grid applications in MediGRID

Juergen Falkner, Anette Weisbecker

Cracow Grid Workshop 2006

18. October 2006

Folie 2

The MediGRID Project

- Part of the German e-Science-Initiative „D-Grid“

- Community-Grid for medical sciences and bioinformatics

VIOLA eSciDoc

Astro

Grid

C3

-Grid

HE

P-G

rid

InG

rid

TextG

rid

On

toverse

WIK

ING

ER

WisE

nt

…

Networked Knowledge Management

Generic Middleware and Services

D-Grid Integration Project

Med

iGR

ID

Folie 3

The task

- build and establish a Grid infrastructure for our community (medical sciences and bioinformatics)

- realize access to the applications they need

- enhance the performance of applications by distributing them in the Grid and share resources

- provide application access for new users

- satisfy the customer, i.e. community users, so they will actually accept and use the system

Folie 4

MediGRID Users

Typical MediGRID users (and what they are doing)

- Doctor (looking for data, providing data, processing data)

- Assistant Medical Technician (providing data)

- Researcher doing bioinformatics (processing data)

- Researcher doing clinical studies (processing data)

- Radiologist (providing and processing data – e.g. mammograms -> medical image processing)

- etc.

What they usually have in common:

- not an expert in computer science

- feeling uneasy with complicated applications

- don‘t like command lines

Folie 5

The big picture: Everything is centered around Grid Applications

Tape StorageDisk Storage

Compute Nodes

Sensors and other technical equipment

Databases

Users

Applications

Folie 6

The sense of a Grid – from the MediGRID viewpoint

- help users solve their tasks

- that is: provide applications that solve their tasks

- so the key for the user is:

find the right application

get access to an application interface (preferably a GUI)

- frontend solution: Portals or graphical Clients

- backend (=> Grid): make sure that these applications can be executed efficiently

- therefore the Grid provides simple access to distributed compute resources and data

- that is: automate everything that lies behind the application

gather and keep information on applications and resources and their dependencies

use that information to select optimal resources

provide a workflow execution service to orchestrate application workflows

automatically distribute applications to optimal resources

Folie 7

The morale…

So far:

- Grids are built by technicians

- Usually they are made for the purpose of controlling the technology rather than helping everyday-users

But:

- they should be built for users who benefit from the added-value a Grid can offer

Therefore Grid builders should:

- find out what users want and what they really need

- find out what the benefit of Grid technologies could be for these users

- find out what the capabilities of the users are and what their core jobs are

- usually their business is NOT to set up computers, install software, know the Grid’s resources, pick certain resources, write job scripts etc.!

- try to fit the Grid middleware to the users – not vice versa

Folie 8

Analysis

Evaluation Specification

Production

User Centred Design Process

User-centred Design...

...is the iterative process by which products or systems are developed so that the needs of the users are taken into consideration at all stages of the design process and could also involve user participation.

[Source: ISO 13407]

Folie 9

Methods of requirements engineering

Methods for the processes from the analysis of requirements to implementation and validation

Source: Michael Ehresmann, Oliver Höß, Erich Meier, Thomas Olsson (Editor): Bausteine zur Optimierung von Software-Entwicklungsprozessen, IRB-Verlag, 2006

Management

Surveys Analyses Specifications Verification and Validation

Legend

elicit non-

functional requirements

elicit tasks and business

processes

Set scope

elicit functional

requirements

identify stakeholders and sources

elicit aims

Check feasibility

Formal modelling

Create GUI-model

Create usage model

Create domain model

Date modelling

Create interaction

model

Analize impact of

requirements

Viewpoint-based

documentation

ensure trackability

documentation of

rationale

measurable and

checkabledescription

of requirements

Use of standards

and document structures

documentation of developer requirements

documentation of customer requirements

Basic Practice

define change-process

map roles and responsibilitie

s

Prioritise and negotiate

requirements

Risk evaluation

Estimate costs and

time

Setup Practice

Prepare tests on

requirements

Manage variabillity

Re-use requirements

Formal check of

requirements

Check usability

Context Practice

Enhance requirements

process

Optimisation Practice

Select technologies

Product planning

Prototyping

Review requirements

Folie 10

MediGRID experience

What did we do?

- before we fixed the architecture we conducted surveys with application and resource providers and users

- result: surveys are not enough to get good information from application providers or users

the users are future Grid users – not experienced Grid users

often they don‘t know what you are talking about

to fit the middleware you need to know the workflows as they should be:Better talk to people! Written language is often a mystery!

Therefore:

- we made workshops with all application providers – in our case these are users also

- we had live demonstrations of the not yet gridified applications

- => we understood the workflows!

- => a lot of misunderstandings could be resolved!

Folie 11

Good practice

In General:

- fix the administrational stuff first:

user registration

how do I get my certificates…

security policies (life with resource providers is much easier if they know the rules – any rules!)

- get applications to work – even if they are not yet fully gridified

- first fix the front-end and get the application running to keep users satisfied

- then fix the backend to make things faster, better, more distributed, more parallel, etc.

- think of users as customers

e.g. add help pages to application interfaces

Folie 12

Questions to ask before designing the architecture (excerpts)

- who will be the users?

- what is their educational background?

- shall the users integrate their PCs into the Grid or shall they be free of IT administration (i.e. are portals or Grid clients necessary)?

- which applications shall be available in the Grid?

- what access control is needed for the applications?

- which requirements and dependencies do the applications bring along?

- which types of data shall be used in the Grid?

- what access control is needed for the data?

- are there any laws that regulate the use of certain data in the Grid?

- what data transfer volumes are expected per user?

- how many users will there be per time?

- what hardware and middleware are available and do we have any experience with them?

- ...

Folie 13

Integration of MediGRID-Applications into the Portal Based on the application analyses conducted through surveys and workshops a couple of pilot applications have been identified and integrated into the MediGRID Portal.

- AUGUSTUS: Genome Sequence Analysis

- Gene Ontology:Access to Ontologie-DBvia secured OGSA-DAI

- Medical Image Processing

3D Ultrasound Prostate Imaging

Virtual Vascular Surgery

- Upcoming: analysis of clinical studies in neuroscience, etc.

Folie 14

D-Grid Certificatefor Browser

D-Grid PEMCertificate

User PC

MyProxy Server

Portal

ProxyCertificate

MyProxyUpload

Tool

Lifetime: 2 years

Lifetime : 7 days

Lifetime : 2 hours

D-Grid CAZertifikate

D-Grid CAZertifikate

D-Grid CACertificate

Grid ApplicationPortlets

Grid Resourcesand Applications

MyProxy Upload Tool by Glen Drinkwater: Original: http://tiber.dl.ac.uk:8080/myproxy/ D-Grid-Version: http://tiber.dl.ac.uk:8080/myproxy/sec/myproxy_test.jnlp

PortalCredential

PortalCredential

MyProxy Upload Tool – use of credentials for accessing secured applications when the user is not located inside the Grid

Folie 15

Questions / Discussion

Contact Information:

Juergen Falkner

Fraunhofer IAOBusiness Unit Software TechnologyNobelstr. 1270569 StuttgartGermany

E-Mail: juergen.falkner@iao.fraunhofer.de

Tel.: +49-(0)711-970-2414Fax.: +49-(0)711-970-2401

http://www.swm.iao.fraunhofer.dehttp://www.medigrid.de