The OMII-Europe Project and the Impact of European Union Funding
Dr Alistair Dunlop
University of Southampton
EU project: RIO31844-OMII-EUROPE
Talk Overview
• EU Funding– What’s the Motivation– Preparing a bid– Running a project
• OMII-Europe– How has EU funding shaped OMII-Europe?– An overview of the project and its constituent parts
EU project: RIO31844-OMII-EUROPE
Motivation - Why pursue EU funding for OMII-Europe? I
• Preface…– OMII-Soton already exists with goals to
• Reuse, reengineer and integrate grid software components that have already been developed within the UK e-science programme
– EU officials liked OMII model and wanted an equivalent within EU
• Also exists within USA in NMI
– Hence a call was created within FP6 (framework Programme 6) to fund an equivalent entity
EU project: RIO31844-OMII-EUROPE
Motivation - Why pursue EU funding for OMII-Europe? II
• Primary reasons to lead a proposal– Protect “OMII” brand– We were expected to do it– If we didn’t do it someone else would have– Addresses sustainability of institute through diversity
of funding sources• Primary reasons why NOT to LEAD a proposal
– Want money to do our own interesting research
EU project: RIO31844-OMII-EUROPE
Prerequisites before starting to preparing an EU bid
• 1: Find the right EU call (cordis.europa.eu) and read the call, rules, requirements– Requirements include specifying:
• Closing Date• Composition of Consortium (Number of partners and which
member states)• Instruments available (the format needed for the proposal)
• 2: Go to Brussels to talk to the relevant unit head/s to find explore a proposed call– More subtle requirements
• Approximate budget available/suggested• Which institutions to include/exclude• Tone of proposal• Level of competition
EU project: RIO31844-OMII-EUROPE
Before writing a proposal…
• Meet with prospective partners to:– Understand what they want to do – but don’t agree! – Get commitment from them to help and not compete– Describe the level of budget available and their share– Ensure they understand the EU rules as this impacts
what partners can do:• In FP6 research activities are 50% funded. (1:1)• In FP7 this is 75%. (3:1)
• Sketch an outline proposal that conforms to the EU instrument, identifying partners to activities and partner funding
EU project: RIO31844-OMII-EUROPE
Outline proposal…
• The outline proposal for OMII-Europe was not exactly as first thought:– The Instrument dictates certain tasks in the proposal
• I3 (Integrated Infrastructure Initiatives) muct include Network Activities, Service Activities and Joint Research Activities
– Politics dictates inclusion of certain partners– The proposal has to be written to fit partners not the
other way around• => You can shape a proposal but you can’t dictate it
EU project: RIO31844-OMII-EUROPE
The full bid
• Preparing the Bid:– Admin part (takes a lot longer than you would expect)– “Part B” or the “Description of Work”
• Approx 100 – 150 pages of text conforming to template describing what will be done
• All partners must sign this off
• Takes approximately 3 months of pretty much full-time work
• After closing date ~ 2 months to hear if proposal is shortlisted– => proposal is fundable
• Hearings ~ 6 weeks thereafter to rank fundable proposals• Outcome ~ 6 weeks later
EU project: RIO31844-OMII-EUROPE
Before you start…
• Negotiations with commission– Take at least 3 months– Could cut budget and insist on changes to proposal
(not technical)– Need to complete a “consortium agreement” – allow at
least 3 months• Total time from submission to start is approx 8
months to 1 year
EU project: RIO31844-OMII-EUROPE
The good and bad news
• Good News:– Finance department is experienced and very helpful in managing
finances of EU grants– Plenty of other people who can help– Gives you plenty of time to get a good project manager in place!– After you lead one project you make a name for yourself and get
invited to participate in others– Good news for individual researchers employed on project –
travel, salary, etc...• Bad News:
– Large administrative overhead with little resource– The admin costs are lumped with your research costs so to
achieve balance you forfeit some research funding
EU project: RIO31844-OMII-EUROPE
Impact of EU funding on OMII-Europe
• Emphasis changed from “Reuse, Reengineer and Integrate” grid components to “Interoperability and Quality Assurance of grid components” due to partner contributions
• The partner list resulted in grid components and grid middleware distributions being included that were not initially considered
• The Instrument for submitting the proposal required the inclusion of tasks that were not foreseen
EU project: RIO31844-OMII-EUROPE
An Overview of the OMII-Europe project
• EU funded FP6 project (RI)– Starting May 2006, initial 2 year duration– 16 partners (8 European, 4 USA, 4 Chinese)
• Open Middleware Infrastructure Institute for Europe– Complimentary to existing national programmes
(OMII-UK, NMI, C-OMEGA, OMII-China…)• Goal is to provide key software components for
building e-infrastructures• Project will demonstrate “proof of concept” with
expectation for a follow-on project in FP7
EU project: RIO31844-OMII-EUROPE
What will OMII-Europe do?
• Initial focus on providing common interfaces and integration of major Grid software infrastructures
• Common services:– Database Access, Virtual Organisation Management, Portal,
Accounting, Job Submission and Job Monitoring• These represent many of the outputs from the standards
function groups
– Capability to add additional services– Emphasis on porting and re-engineering work, not developing
from scratch• Infrastructure integration
– Initial EGEE/UNICORE/Globus interoperability– Interoperable security framework
EU project: RIO31844-OMII-EUROPE
OMII-Europe guiding principles
• Committed to standards process– Implementing agreed open standards and working with
standards process (GGF/Oasis)• Quality Assurance
– Published methodology and compliance test– All software components have public QA process and audit trail– Working with similar projects and organisations to agree policies
• Impartiality– OMII-Europe is “honest broker” providing impartial
advice/information on e-infrastructures
EU project: RIO31844-OMII-EUROPE
What will OMII-Europe deliver?
• Repository of open-source, quality assured software services for gLite/EGEE, Globus, UNICORE and CROWNgrid– Some services bundled with major grid distributions– Initial integration work with EGEE, UNICORE and
Globus• Public reports on grid infrastructures
– Initial benchmark results– Impartial advice and information
• Evaluation infrastructure to “test” services– User support and training for services
EU project: RIO31844-OMII-EUROPE
Who benefits from OMII-Europe?• E-infrastructure providers
– Choice of grid software to deploy can be determined by selecting the most appropriate system to manage resources.
– Achieved through common interfaces and interoperability of grid systems• Decisions not constrained by membership of a particular VO • Not required to deploy and manage multiple grid distributions
• E-science users– Common methods for accessing grid Infrastructures– Access to resources beyond the immediate e-infrastructure running a
specific grid distribution• Achieved through low level interoperability of Grid distributions
– Users not restricted to a specific, fixed set of resources• E-science application developers
– Applications can be deployed and run on multiple grid environments through adherence to common services
• Not required to develop different solutions for different grids
EU project: RIO31844-OMII-EUROPE
Why Globus, UNICORE, gLite and CROWN?
• Minimal significant set:– gLite is a complete set of middleware developed within EGEE and
is deployed to create a grid containing more than 150 sites and 30 countries
– UNICORE is a major EU and national middleware initiative and is deployed at many supercomputer sites, in particular those available through DEISA
– Globus is the world-leading open-source platform for Grid computing developed within the USA and is used for many research projects world-wide
– All three grid platforms have significant user bodies within Europe– CROWNgrid is the middleware used on the major Chinese grid
infrastructure
EU project: RIO31844-OMII-EUROPE
Database Service
• Implementation of the OGSA-DAI specification from the DAIS-WG within the Data function group of GGF– OGSA-DAI service federates data resources with different
support mechanisms (Relational/XML Databases/flat files) allowing uniform access across these resources
• Number of other data specifications emerging that may be considered later.– transaction management; byte IO; Grid file systems etc
• DAIS implementation already available for Globus 4 – • Work is to port to UNICORE and gLite – Alpha releases
scheduled for May 2007. – Evaluating OGSADai4UnicoreGS
EU project: RIO31844-OMII-EUROPE
Job Submit and Job Monitoring Service
• Implementation of the JSDL (job submission description language) and BES (Basic execution service) specifications from the Compute working group at GGF– Common way to specify and control jobs (abstraction of O/S and
cluster controller)• Other specifications such as scheduling, but above are
essential and well developed with implementations• Work is to make BES and JSDL available on Globus, UNICORE
and gLite– Initial version for UNICORE available in May 2007– JSDL translator (using XSLT) for gLite in testing
EU project: RIO31844-OMII-EUROPE
Virtual Organisation Management Service
• Authorisation service available for Globus and gLite. – Provides information on the user's relationship with
Virtual Organization: groups, roles and capabilities • Work to make VOMS available under UNICORE and
to extend VOMS with SAML support– SAML (Security Authorisation Markup Language) from
OASIS Technical Committee. (standard for XML exchanging authentication and authorisation data between security domains)
– Alpha version for UNICORE with SAML support scheduled for May 2007
EU project: RIO31844-OMII-EUROPE
Accounting Service
• Implementation of the Resource Usage Service (RUS) from the management working group within GGF– Tracks use of resources (accounting in traditional
UNIX sense), but not concerned with payment – Closely related to Usage Record (UG-WG) within GGF
• Specification available for public comment• Alpha version of RUS (or equivalent) available in May
2007 for Globus, gLite and UNICORE
EU project: RIO31844-OMII-EUROPE
Portal Service
• Integration of the Gridsphere portal framework with Globus, UNICORE and gLite and provide portlets for job submit, accounting, etc…– Provide application level portability at a portlet level– Portlets available for main OMII-Europe services
EU project: RIO31844-OMII-EUROPE
Additional Services
• Current solution:
• Chinese partners will make all services available on Chinese CROWNgrid infrastructure
• In May 2007, launch of the second round of service integration
EU project: RIO31844-OMII-EUROPE
OMII-Europe JRA1 re-engineering activities
OGSA DAI BES VOMS RUSGrid
Sphere
Etc. Identified
Components
EGEE
(GLite)
UNICORE
Globus
Etc. OMII-UK, USA, China
EU project: RIO31844-OMII-EUROPE
OMII-Europe Infrastructure Integration
• This activity goes beyond the adoption of common services and focuses on full grid infrastructure integration through employing:– A common security infrastructure
• Much similarity (X509) and differences (handling of proxies, authorisation, anonymity and auditing)
• Intention to define a common security base• Provide a strengthened form of X509 credential management
through using Myproxy
– job migration between Globus/gLite/UNICORE• Builds on Globus/UNICORE Grip project
• Close collaboration with GGF GIN WG
EU project: RIO31844-OMII-EUROPE
Project Structure and Effort Allocation
• Networking activities– Management, Outreach, Training– 8% Person Effort
• Service Activities– Repository, QA, Support– 25% Person Effort
• Joint Research Activities– Re-engineering, new services, integration,
benchmarking– 67% Person Effort
EU project: RIO31844-OMII-EUROPE
Effort (Person Years) per Activity
0
2
4
6
8
10
12
14
NA1: M
anag
e.
NA2: O
utre
ach
NA3: T
raini
ng
SA1: R
epos
it.
SA2: Q
uality
A.
SA3: S
uppo
rt
JRA1:
Dat
abas
e
JRA1:
VOM
S
JRA1:
Acc
ount
.
JRA1:
Job
Sub.
JRA1:
Por
tal
JRA1:
Com
p. E
x.
JRA2:
New
Ser
v.
JRA3:
Sec
urity
JRA3:
Inf.
Inte
g.
JRA4:
Ben
ch.
Funded Effort (Years) Unfunded Effort (Years)
EU project: RIO31844-OMII-EUROPE
OMII-Europe Project Partners
114 person years over 2 years, 5 million Euro, 4 major Grid infrastructures
University of Southampton UK (coordinator) University of Chicago USA
Fujitsu Laboratories Europe UK NCSA, University of Illinois USA
Forschungszentrum Juelich Germany University of Southern California Los Angeles USA
Kungl Tekniska Högskolan Sweden University of Wisconsin-Madison USA
Istituto Nazionale di Fisica Nucleare Italy Beihang University China
Poznan Supercomputing & Networking Center Poland
China Institute of Computing Technology Beijing China
University of Edinburgh UK Computer Network Information Centre Beijing China
CERN, European Organisation for Nuclear Research Switzerland
Tsinghua University China
EU project: RIO31844-OMII-EUROPE
OMII-Europe project summary
• Interoperability is difficult not because of technical issues but because it requires agreement– No one wants to be seen to “lose out” to someone else
• OMII-Europe has support from the major grid Infrastructure providers to deliver interoperability– No point to be trying to solve the problem without vendor support
• OMII-Europe’s emphasis on standards provides a non-biased approach towards interoperability– An open independent process needs to be used to arrive at
technical decisions• Achieving interoperability is a long term goal, don’t try and eat
an elephant in one go!– OMII-Europe will improve overall USABILITY of grid
Infrastructures and improve INTEROPERABILITY of grid infrastructures over the next two years
EU project: RIO31844-OMII-EUROPE
Concluding Comments
• EU research budget is moving towards 2% of total EU GDP.
• Getting a share isn’t that difficult but you need to be politically aware
• It provides a means to support real collaborative research within the EU
• Participating is far easier than being coordinating partner