The UNICORE GRID Project

Karl Solchenbach

Gesellschaft für Parallele Anwendungen und Systeme mbH

Pallas GmbHHermülheimer Straße 10D-50321 Brühl, Germany

info@pallas.comhttp://www.pallas.comWorkshop CHEP2000, Padua

© Pallas GmbH

UNICORE is a project funded by the German Ministry

of Education and Research (BMFB)

to develop a prototype for seamless, intuitive, and

secure access to computing resources.

Duration 07/1997 - 12/1999

What is UNICORE?

© Pallas GmbH

The Unicore Partners (past)

German BMBF funded for:

– Implementation: Pallas and Genias.

– Users: German Research Centres and Universities (FZ Juelich, RUS, DWD, LRZ, RZU, PC2, ZIB.

Partners/Founders

– fecit

– ECMWF Affiliates

– debis, Inpro

– IBM, NEC, SGI/Cray, Siemens/Fujitsu, Hitachi, SUN, HP

© Pallas GmbH

Different architectures

Different vendors

Different software

Different usage models

Incompatible administration

Incompatible data management

Motivation

© Pallas GmbH

Consequences:

Users work with the computer

– they are familiar with

– the have resources

New systems require

– additional learning

– additional conversions of jobs and data

– high administration overhead

Motivation

© Pallas GmbH

Consequences

Expensive resources are used sub-optimally

Problems are solved inefficiently

Solvable problems are not solved

This holds for research and industry

Motivation

© Pallas GmbH

Seamless batch access to computing resources

Production-ready for existing environments

Based on existing/emerging technologies

Access to distributed data

Interdependent jobs on distributed systems

Goals of UNICORE

© Pallas GmbH

Seamless access to computing resources

Intuitive GUI for batch submission

Same look-and-feel independent of target system: hide the seams

Facilitate distribution of work to most suitable platform or site

Provide full information about resources

Goals of UNICORE

© Pallas GmbH

Production-ready for existing environments

Map abstract UNICORE specifications to site specific functions

Map UNICORE ids (certificates) to local account names (Unix logins)

Site retains full control over resources

No changes to local resource management or accounting

Goals of UNICORE

© Pallas GmbH

Based on existing/emerging technologies

Access via www

Java and Java applets

X.509 certificates

Goals of UNICORE

© Pallas GmbH

Interdependent jobs on distributed systems and access to distributed data

User specifies target system (s)

UNICORE distributes jobs to sites

UNICORE synchronizes execution

UNICORE transfers required data between sites

Goals of UNICORE

© Pallas GmbH

Unicore: Architecture

3 tiers

– User interface: Job Preparation Agent, Job Management and Control

– Site Security: Gateway, authentication

– Job Control: Network Job Supervisor

© Pallas GmbH

Unicore Security and Authentication

Jobs will be submitted to a site over the Internet

– needs a trusted Unicore component to authorise the jobs on any Firewall, the GATEWAY

Sites keep autonomy over who is a user

– each Unicore user must have a local login (different at each site)

– Gateway maps Unicore user to local incarnation HTTPS and X509 certificates

– certificate identifies the Unicore user

– implementation comes as standard with (some) servers and browsers

© Pallas GmbH

The Abstract Job Object

The Abstract Job Object (AJO)

– uniform model of computing actions

– data access as well as computing

– asynchronous

– extensible AJO is the link between instantiations of Unicore agents (and

different implementations)

© Pallas GmbH

UNICORE Environment

User JPA

Browser

Site File StoreSite File Store

AJO Environment

AJO Environment

Site A

Site B

Site C

NJS

NJS

Seamful

Seamless

Import

Export

Transfer

© Pallas GmbH

© Pallas GmbH

© Pallas GmbH

© Pallas GmbH

© Pallas GmbH

© Pallas GmbH

Concurrent use of distributed systems(application-level metacomputing)

Computational steering

Application specific interfaces

Resource brokers

Support for every platform

What is not in the UNICORE prototype?

© Pallas GmbH

UNICORE will establish infrastructure for supercomputer centers in Germany to share resources

Users want UNICORE now

UNCORE has vendor support

Summary

© Pallas GmbH

Create a UNICORE product (Pallas)

Deploy UNICORE at German HPC centers

UNICORE Plus project is funded by German government (01/2000 - 12/2002)

European GRID initiative EUROGRID

Future

© Pallas GmbH

EUROGRID

Idea: European GRID project supported by leading European HPC centers

Experiment for application-specific GRIDs (biology, weather, CAE, …)

Proposal to European Commission

Partners:

– HPC centers: CSAR, IDRIS, ICM, FZ Jülich, Parallab

– Users: MSC, DWD, Aeromatra

– Integration: Pallas, Fecit