April 15 2004eIRG meeting, Dublin, Ireland
11
Authorisation PolicyTowards a European Policy for Resource Sharing
CONTOURS OF A TRANSPARANT GRID ACCESS POLICY
Dr. Patrick AertsDirector of the Netherlands
National Computing Facilities Foundation(NCF)
April 15 2004eIRG meeting, Dublin, Ireland
22
Overview
● The goals● Grid concepts for Europe● The terms, what is involved● Examples, the scope of the problem● Some models presently in place● Complications● Further issues
April 15 2004eIRG meeting, Dublin, Ireland
33
The Goals
● Access to all resources for scientific computing in Europe using the grid
● A “fair share” for all users● Authentication by National Certification
Authorities (CA) using European formats● Authorisation: required, but not not too often● Accounting, using European formats
April 15 2004eIRG meeting, Dublin, Ireland
44
The European grid conceptWhat are we heading for?
● Concept 1: a grid of grids● Grids get formed by and from communities with a
certain common goal● Within these grids things are rather easy:
● Trust, resource sharing, etc.● From these grids a larger (European) grid may arise
● Concept 2: one large grid enabled bunch of resources
● Owners allow their system(s) to be grid enabled and grid aware
● VO’s select their choice from available systems● VO’s seek funding for their project
April 15 2004eIRG meeting, Dublin, Ireland
55
What is involved in Authorisation and
Accounting (1)● Authorisation:● Who is allowed to access a facility● Who provides the financial means
(allocation)• Allocating refers to the mechanism that
determines one’s rights to access an entity● Accounting: refers to the system that keeps
track of the resource units used by a user and the way the associated cost are billed or properly placed at the responsible authority (possibly the user).
April 15 2004eIRG meeting, Dublin, Ireland
66
What is involved in Authorisation and
Accounting (2)• Authorisation determines who has rights for
access,
• Allocation determines to what extent. – Allocation mechanisms may be very different for the
entities within a grid and between grids.
• An authorised person/organisation may have its own funds too
• Whose responsibility is the reliability (trustworthyness) of users: at the authentication level or at the authorisation level?
April 15 2004eIRG meeting, Dublin, Ireland
77
How it works in The Netherlands
a Use Case (1)● Scientific projects are submitted to the
National Science Foundation (NWO)● A selection panel awards the project on
scientific merits, after peer review● NCF/NWO awards the necessary computing
resources for these projects, but also for other qualified projects (also after peer review)
● The national Computer Center, like SARA, then creates an account and installs a budget
● SARA bills NCF at the end of each month for the resources provided in this way
Reality is not much more complicated
April 15 2004eIRG meeting, Dublin, Ireland
88
But also:from biodiversity: bird migration case(2)
● Subgroup in the biology faculty of the Amsterdam University
● University groups may request resources from NCF without going through the NWO selection panel
● In a simulation the migration of one bird is simulated
● Ideally suited for a CPU cluster if one wants to simulate a flock of birds over a longer time
● A VO=bird migration is created and the faculty members request a certificate from the Dutch CA
April 15 2004eIRG meeting, Dublin, Ireland
99
Bird migrati
on
April 15 2004eIRG meeting, Dublin, Ireland
1010
How it (possibly) works in Germany
a Use Case● Scientific projects are submitted to the Fraunhofer Gesellschaft
● A selection panel awards the project on scientific merits
● The Fraunhofer Gesellschaft makes computer resources available through one of its computer centers like Karlsruhe FZK
● FZK then creates an account and a budget● and bills Fraunhofer at the end of the year for the
services providedI assume this is how it works in Germany,
reality may be more complicatedBut that is not relevant for this argument
April 15 2004eIRG meeting, Dublin, Ireland
1111
A Real Examplefrom astrophysics: colliding black holes
● For this sort of calculations one needs a supercomputer
● EU Supercomputer project: DEISA● Let us assume that supercomputers are
also accessible through a grid infrastructure
● A VO=black holes is created and the participating scientists all request a certificate from the German CA
April 15 2004eIRG meeting, Dublin, Ireland
1212
Colliding black holes
April 15 2004eIRG meeting, Dublin, Ireland
1313
Exchange of resources● Assume a bird migration calculation is submitted
to the grid (EGEE) and is send to a cluster of cpu’s at the Karlsruhe computer center
● Assume a colliding black hole simulation is submitted to the grid (DEISA) and is send to the supercomputer at SARA in Amsterdam
● The control of where a job is executed on the grid depends on the available resources at any time
● For this to work SARA and FZK have to accept jobs from the bird migration and black holes VO
● What is the policy for resource providers in Europe to accept/not accept VO’s?
April 15 2004eIRG meeting, Dublin, Ireland
1414
One would hope that ..● The scientists don’t have to worry where
their job migrates to● The scientists don’t have to worry that they
can use resources where their job runs best● The resource providers get the money that
their services cost● A European policy can be defined such that
services can be provided across national borders without cash flow
● In order to fulfill this hope, these issues have to be subjects of the next chapters of the eIRG
April 15 2004eIRG meeting, Dublin, Ireland
1515
International Scientific Collaborations
● The case is much simpler in High Energy Physics:● The Atlas collaborators have already requested
resources from their national funding agencies● The Atlas collaborators are organised in one and the
same Atlas Virtual Organisation VO● Budgets exist for this VO on all major sites with
computer resources in Europe● The fair sharing of those resources is done at the
collaboration level in a Memorandum of Understanding with each of the collaborating institutions
● The collaborating institutions go through the normal procedure for resource assignment at a national level
April 15 2004eIRG meeting, Dublin, Ireland
1616
Smaller National Scientific Projects
● Bird migration simulation was a Dutch initiative from a small university group
● The same in Germany for the colliding black holes study
● Yet resources will be used more efficiently if the computing would not respect national borders
● To achieve this an authorisation policy has to be put in place and nationally created VO’s must be recognised Europe-wide, in some way...
April 15 2004eIRG meeting, Dublin, Ireland
1717
Delegation of RightsA Push Model
● In both cases the Authorisation involves some form of cascading of rights:
● From NCF to SARA to VO to users● Implemented in DataGrid (EDG) in a
push model● GridMapFiles at each site where these rights per
user and VO are described● Push model preferred if AuthZ is
needed globally and instantly (networking)
April 15 2004eIRG meeting, Dublin, Ireland
1818
Delegation of RightsA Pull Model
● It could be implemented the other way● User to SARA to NCF to Project Description
● Depending on the problem this is a better or worse solution
● Shibboleth uses a Pull Model for accessing web resources
April 15 2004eIRG meeting, Dublin, Ireland
1919
Delegation of Rightsan Agent Model
● Virtual Organisations VO’s are used to describe large scientific organisations
● Not all members have the same rights● Authorisation can be further cascaded● Developed in Virtual Organisation
Management Service (VOMS) in DataGrid and DataTag
● Tested now in LHC Grid project LCG
April 15 2004eIRG meeting, Dublin, Ireland
2020
AuthZ Models
AuthZService
Resource
AuthZService
AuthZService
Resource
Resource
Agent
Push Pull
12
33
32
21
14
April 15 2004eIRG meeting, Dublin, Ireland
2121
Acceptable Use Policies
● Use policies are defined at many levels: institutional, national, scientific collaboration, etc.
● National legislation may also impose use policies (security, privacy, etc)
● Often different for different countries● Often different for different resources● These things seem solvable relatively
easy
April 15 2004eIRG meeting, Dublin, Ireland
2222
Complications:
● As long as the resources involved are rather homogeneous and rather simple (like midsize clusters) things are easy
● Once relatively expensive or specialised equipment gets involved things get complicated:
● One has to make a case for renewal and re-investments● Such cases involve accountability, show cases, success
stories● Regional/National pride may be involved, etc.● This is usually a co-responsibility of the authorisation
bodies● So, one does not hand over control over the special
systems in a grid for others to decide on its usage
April 15 2004eIRG meeting, Dublin, Ireland
2323
Complications (2)
● The European grid is best build from the ansatz that there will be many different ad hoc build grids.
● In practice these grids are to a large extend coinciding with the VO’s from other concepts.
● The convergence from this situation to a situation where all relevant systems are grid aware and grid enabled to allow these different grids to glue together has to be guided by the eIRG.
● This means doing things the hard way. But it will keep Europe ahead of developments elsewhere (Teragrid, US), because one of the grid added values has to be sharing diversity rather than sharing homogenity.
April 15 2004eIRG meeting, Dublin, Ireland
2424
Further complications
● If users or VO’s were only to pay in real money:● Wouldn’t that be nice and easy.
● But more often no real money is involved in allocation:
● Either one gets resource units, implicitly meant to be spend on a limited number of dedicated systems, or
● If real money is involved, budgets may cover only a systems running cost, not the integral cost (including re-investments)
● And even then the money is supposed to be spend on a predetermined (number of) systems
● In fact there is no (open) market, but a large number of closed circuits
April 15 2004eIRG meeting, Dublin, Ireland
2525
Success stories
● GEANT● Common basis for all AUP*s defined
● (however: see lecture d. Van dromme)● Big user community: all NRENs in Europe
● DataGrid● New AUP defined● Small user community: relatively easy!
● *AUP= Acceptable Use Policiy
April 15 2004eIRG meeting, Dublin, Ireland
2626
Preferred Solution
● A schema which encompasses all national AUPs without making them all the same
● A schema which separates the “common” basis from differences and accounts for those
● A schema by which AUPs apply for all resources: cpu’s, storage, networking, etc.
● eIRG should stimulate this development
● For the time being: why not have authorisation bodies put a percentage of the systems they govern into a basket for European grid-related usage ( the 5% of Mary Spada, Argonne/SDSC)
April 15 2004eIRG meeting, Dublin, Ireland
2727
Virtual Organisationsa possible model
● In each EU country VOs can easily (through a web form) be created for scientific projects
● When computing resources are assigned to the project the VO is validated
● A validated VO is uploaded with the grid middleware to all sites but is by default “unsupported”
● Each site will “support” all VO’s from countries with which there is an agreed policy for resource sharing (preferably all EU countries)
● Scheduling priorities among VO’s is still a local or national policy
April 15 2004eIRG meeting, Dublin, Ireland
2828
Accounting
● Not all services cost the same:● Supercomputers vs. clusters● What costs archiving or databases● Other non-computer networked facilities
● Each resource provider may have an internationally standardised and man+machine readable SLA per system
● Accounting done per user, billing per VO (or user or AutZ body) by resource provider
● Less a problem for larger international scientific collaborations
April 15 2004eIRG meeting, Dublin, Ireland
2929
Dutch Presidency
● Policy for easy creation of VO’s● Policy for VO support by resource providers● Model for AuthZ
● Common for CPU, storage and network resources● Support for accounting schemes● Respecting anonymity
● Proposals for the %-basket● Possibly linking to the money follows man (M/F)
principle of European research councils● Common Acceptable Use Policy