EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Project OverviewNA1 – Administrative and Technical

Coordination

Alberto Di Meglio (CERN)Project Director

1st EMI Periodic ReviewBrussels, 22 June 2011

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Outline

• What is EMI?• Co-operation with other projects and programs• Project Management

22/06/2011 1st EMI Periodic Review 2

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

What is EMI?

22/06/2011 1st EMI Periodic Review 3

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Early Adoption Production ServicesConsolidation

Evolution

EMI Positioning

22/06/2011 1st EMI Periodic Review 4

2002 2010

Infrastructures

Middleware

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

EMI Mission Statement

22/06/2011 1st EMI Periodic Review 5

The European Middleware Initiative (EMI) project represents a close collaboration of the major European middleware providers - ARC, gLite, UNICORE and dCache - to explore and implement sustainable models to harmonise, evolve and support the distributed computing and data management middleware for deployment in EGI, PRACE and other distributed e-Infrastructures

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Primary Objectives

Simplify the middleware services by delivering a streamlined, coherent, tested and standard compliant distribution meeting the requirements of EGI, PRACE and other distributed computing infrastructures and their user communities.

Increase the interoperability, manageability, usability and efficiency of the services by developing or integrating new functionality following existing and new requirement

Support DCIs operations by reactively and proactively supporting and maintaining the middleware distribution

Strengthen the participation and support for user communities in the definition and evolution of middleware services by promoting the EMI achievements, objectives and plans in collaboration with diverse programs and partners

22/06/2011 1st EMI Periodic Review 6

Consolidate

Evolve

Support

Collaborate

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

EMI Baseline Services

Applications Integrators, System Administrators

EMI Middleware Evolution

22/06/2011 1st EMI Periodic Review 7

Users and Infrastructure Requirements,

Standards,New technologies

3 years

Before EMI After EMI

Specialized services, professional support

and customization

Standard interfaces

Standard interfaces

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

First Year Objectives

Standardize the software development and release management to adopt established best practices from mainstream open source operating system providers

Start the process of interface standardization implementation by implementing what exists and reach wide agreements on what has to be done. Address requirements.

Support EGI and the NGIs without disruption from previous projects, work on an SLA and define agreed retirement calendars for older services

Implement dedicated collaboration programs with DCIs and VRCs, establish the EMI identity and make it known to existing communities, start approaching new communities outside the typical grid users, define exploitation and sustainability strategies

22/06/2011 1st EMI Periodic Review 8

Consolidate

Evolve

Support

Collaborate

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Achievements Highlights Release of the EMI 1 (Kebnekaise):

– For the first time components from 4 providers in a single repository, better compliance with open source operating systems and guidelines

– Common Quality Assurance policies and software engineering tools based on industry standards (ISO/IEC 12207, ISO/IEC 9126, ITIL V3)

Implementation of new functionality following stated user requirements:

– Compliance with standard security protocols (SSL), info system models (GLUE 2), data access protocols (WebDAV, NFS4.1/pNFS), security profiles (SAML)

– First implementation of WMS on SL5 64bit operating system, initial integration of the Argus authorization system with the grid services

– Agreement on Execution Services and Storage Accounting specs

Continuous support for EGI production sites– GGUS as single entry point, dedicated Support Units for all EMI Products– SLA to define service levels and monitor performance

Established EMI identity– Created a growing network of collaboration with concrete MoUs, “Works with

EMI” program for technical partners– Clear name recognition (talks, posters, codename strategy)

22/06/2011 1st EMI Periodic Review 9

Consolidate

Evolve

Support

Collaborate

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Co-operation with otherprojects and programs

22/06/2011 1st EMI Periodic Review 10

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

DCI Roadmap

Collaborations

22/06/2011 1st EMI Periodic Review 11

EMI

EGI, PRACE, WLCG,OSG, etc.

ESFRI,VRCs

StratusLab VENUS-C EDGI

Requirements Releases

RequirementsCollaborations

IGE

SLAs &Support

Standards,Industry

RequirementsCollaborations

EGI-InSPIRE

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Current Relationships

22/06/2011 1st EMI Periodic Review 12

SLA + MoU

MoU

Common initiatives

TheSyrrus, Ltd

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611New Communities and ESFRI

• Requirements from new communities and ESFRI projects have been addressed through participation in common events– EEF workshop in November 2010– Federated Identity Management Workshop in June 2011

• Already addressed:– Consolidation of support channels

• In the work plan:– Simplification of AAI– Persistent storage IDs– Integration with clouds and volunteer computing

22/06/2011 1st EMI Periodic Review 13

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Standardization Collaborations

• EMI has a strong involvement with international standardization bodies and initiatives

• OGF– Several chairs of important working groups– Submission of proposals for new or improved standards

• SIENA– Membership in the Roadmap Editorial Board (REB)– Contribution to the definition of the roadmap document– Participation to common events to present the EMI strategy

22/06/2011 1st EMI Periodic Review 14

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Involvement of commercial partners

• Very challenging activity– The sharing model that makes grid essential for

research is not very suitable for industry– Current business focus is on clouds

• Identify services that can be exploited on their own or integrated within clouds

• Initial contacts with Google, Red Hat, Canonical, TheSyrrus, Ltd– To get advice on market analysis, branding,

identification of services of potential interest for collaboration

22/06/2011 1st EMI Periodic Review 15

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Copyright and Licensing

• Clear assessment of copyright and licensing is required to make sure there are no constraints to exploitation by third parties

• The assessment has started in year 1, but it’s still in progress– Copyright assessment yet to be done, several different models

used by various partners depending on national laws (IP can belong to developers, Institutes, Collaborations)

– Licensing is in better shape• vast majority of the components is licensed under Apache 2.0 or BSD, both

commercially friendly options• some components licensed under or using GPL have to be addressed• dCache has a proprietary license, with some limitations

22/06/2011 1st EMI Periodic Review 16

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Project management

22/06/2011 1st EMI Periodic Review 17

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Partners (26)

22/06/2011 1st EMI Periodic Review 18

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Project Structure

191st EMI Periodic Review22/06/2011

NA1Administrative and Technical Coordination

NA2Outreach and Collaborations

SA1Maintenance and Support

JRA1 - Development, Integration and Evolution

SA2Quality

Assurance

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Project Governance

201st EMI Periodic Review22/06/2011

Project Director

TechnicalDirector

Project Executive Board (PEB) Project Technical Board (PTB)

NA2 SA2 SA1 JRA1

Collaboration Board (CB)

ECB

Tech. Areas

Engineering Management Team (EMT)

PT PT PT PT PT PT PT PT

RM

European Commission

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Project Management

WP Responsibilities

22/06/2011 1st EMI Periodic Review 21

NA1, JRA1

JRA1

JRA1SA1

SA1

SA2

Software & Services

Requirements

Defines

ImplementsCertifies

Release Candidate

Process definition

Process monitoring

NA2NA2

NA1, NA2

Collaborations, exploitation, sustainability

Training

Dissemination & Out-Reach

In-Reach

NA1

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Performance Drivers and KPIs• Main drivers

– “merge” the activities and cultures of 4 different MW providers– increase efficiency and reduce costs, while delivering software and services

to EGI without discontinuity from previous projects

• How to measure “value for money”?– COCOMO and actual costs and effort– Empirical approach due to the lack of any previous benchmark

• What does it mean?– Change very dramatic, too steep. Why?– What part is increase of efficiency and what is people overload?– Differences in salary between industry and academia

22/06/2011 1st EMI Periodic Review 22

EMI P1 COCOMO Pre-EMI

SLOC (M) 2.38 2.38 2.27

Effort (p/y) 64 83 93*

Cost (MEUR) 6.1 11.5 12.5*

*estimated

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611Deliverables and Milestones

• Deliverables:– 36 planned and delivered

• Milestones:– 23 planned and delivered

• Some issues during the year with delays in submitting– A contingency action has been implemented to

enforce better compliance with deadlines– Situations has improved in the second half of the

year

22/06/2011 1st EMI Periodic Review 23

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Budget and Effort Stats

22/06/2011 1st EMI Periodic Review 24

• Budget:– 7,168,079 € or 88.7% of planned budget– 3,482,465 € or 87.6% of the planned requested

contributions

• Effort:– 847 PM or 110.4% of the planned effort

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Budget and effort Stats

22/06/2011 1st EMI Periodic Review 25

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Budget and Effort Stats

22/06/2011 1st EMI Periodic Review 26

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Budget and Effort Distribution

22/06/2011 1st EMI Periodic Review 27

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Conclusions

22/06/2011 1st EMI Periodic Review 28

• EMI had a very intense year focused on merging technologies, processes and people from 4 middleware providers into a coherent project with a common vision

• It has achieved important goals, culminating in the EMI 1 release and its recognition and planned adoption within the DCI community

• A first important step in the consolidation and simplification has been made and represents a solid foundation for the work to be done in the next years

EMI I

NFS

O-R

I-261

611

EMI I

NFS

O-R

I-261

611

Thank you

22/06/2011 291st EMI Periodic Review

EMI is partially funded by the European Commission under Grant Agreement INFSO-RI-261611