page 4

news from the EGI communityISSUE 2MAY 2017

TOP STORIES

page 7

MORE

Advanced Computingfor Research

www.egi.eu

The EGI ISO certificationspage 3

Inspired

Terradue & EGI for Earth Observation

01 Computing centres: LNEC

EGI Services for open science

07 The new Accounting Portal

page 6

GEOSS Data and Computing Challenges

08 OPENCoastS forecast service

09 EOSCpilot: Science Demonstrators

10 AARC is dead - long live AARC!

In the new edition of our newsletter, we focus ourservices and on the contributions of the EGICommunity to the Earth Observation field.

Your feedback and suggestions are always welcome!

Send an email to Sara & Iulia at:

press@egi.eu

Welcome to issue 27!

The LNEC data centre in Lisbon is locatedat the headquarters of the LaboratórioNacional de Engenharia Civil, a researchfacility dedicated to engineeringsciences. The data centre is managed byFCCN and spans an area of 370m². Thecentre hosts a total of 2.500 CPU coresand 1.1 PB storage, from which about700 TB under the Lustre distributedfilesystem serving the HTC and HPCclusters, and about 400 TB under Cephserving the Openstack cloud IaaS.

The data centre is part of the NationalDistributed Computing Infrastructure(INCD) - the Portuguese DigitalInfrastructure. INCD is a partnershipbetween LIP, LNEC and FCCN andrepresents Portugal in the EGIFederation, in IBERGRID (the IberianInfrastructure) and in the Worldwide LHCComputing Grid (WLCG).

Computing centres: LNEC

Mário David shows us the LNEC data centre, part of the Portuguese DistributedComputing Infrastructure (INCD)

More information

INCD - National Distributed ComputingInfrastructurehttp://www.incd.pt

INCD provides computing and storage services toPortuguese-based scientific communities in alldomains and participates in national andinternational projects of strategic relevance.

LNEC - National Laboratory of Civil Engineeringhttp://www.lnec.pt

Your Data Centre

If you work with or at one of the +300data centres federated in the EGI e-infrastructure, we would love to hearfrom you!

Send your pictures to press@egi.eu

Save the date!The upcoming DI4R 2017 will be in Brussels

30 November - 1 December(Credit: Marc Ryckaert/Wikimedia Commons)

1Inspired // Issue #27, May 2017

2

The post delivery on a fineWednesday afternoon back inMarch included a very specialpackage: the official printcertificates sent by TÜV SÜD toconfirm that the EGI Foundationhas been awarded a ISO9001:2015 and a ISO/IEC 20000-1:2011.

This was excellent news. EGI isnow the first European-widepublicly-funded e-infrastructureto be certified against ISOstandards, sign of its maturityand of the competence of itsactors.

The certifications demonstratethat our management systemsregarding the EGI ServiceCatalogue follow ISO 9001:2015and ISO/IEC 20000-1:2011requirements. This includes allactivities of planning, implemen-tation, monitoring and continualimprovement of all processes.

In short, this means that we canoffer better services to the EGIconsumers and more value tothe EGI Council.

The long road to certificationstarted with a strategiccommitment by the EGI Councilto establish efficient processesto manage EGI’s service offer.The first step began in 2015 withthe full implementation of FitSM,a lightweight version of the ISO20000 standard.

Having FitSM in place and all theEGI Foundation staff trained andcertified in service managementaccording to this standardmeant that EGI service

The EGI ISO certifications: new and improved servicedelivery

Yannick Legré writes about what the certificates mean for the EGI Community

management was alreadycomplying with about 80% of theISO standards. The preparationsfor the two rounds of audits inOctober and December 2016ensured that the remaining 20%requirements were also met.

The hard work of the past twoyears paid off: we can now besure that we are operatingimproved, high-quality andreliable services to better serveour user base. And we do so

More information

Yannick Legré is theManaging Director of the theEGI Foundation. @ylegre

Inspired // Issue #27, May 2017

About the certifications

The ISO 9001:2015 certificate covers all business processesincluding administration and finance, human resources, qualitymanagement, risk management, business relationships andcontinuous improvement.

ISO/IEC 20000-1:2011 certification is a specialization of theprevious designed to cover all IT-related services includingcompute, storage and data as well as internal services enablingFederation.

The certifications are an achievement of the EGI-Engage project,co-funded by the European Union (EU) Horizon 2020 programunder grant number 654142.

with consistent, clear,streamlined processes thatassure our stakeholders a betterreturn on investment.

And that, for me, is the definitionof a win-win!

3

State of the artOne of EGI-Engage's keyachievements so far was thepublication of the internal andexternal EGI Services Catalogue.This was an important milestoneto develop a clear valueproposition, increase theaccessibility of available services,and make the providers fromthe EGI federation more visibleto funders and policy makers.

The external catalogue includesall services available forresearchers and is organizedacross three broad categories:Compute, Storage and Data andTraining.

The internal catalogue offersthe services necessary tofederate local, regional andnational infrastructures into theEGI Federation. These servicesare offered to the federateddata centres to provide the“glue” that holds together aharmonized pan-Europeansystem for internationalresearch collaborations.

Scientific applications poweredby EGI services offercommunity-specific capabilitiessuch as datasets, searchcapabilities, data analytics, andscientific software. They are atremendous research-enablinginstrument: the current set ofintegrated scientific applicationscounts thousands of usersworldwide and has providedhigh-impact examples of howindividual researchers canbenefit from EGI services, forinstance:

EGI Services for open science

Tiziana Ferrari writes about current and future EGI service offer

> A team from the University ofMonash used the HADDOCKportal to model the structure ofiron-binding receptors inbacteria. The work waspublished as a paper in NatureCommunications.

> A Swedish scientist used VIP toanalyse the results of a long-term study of the effects ofmultiple sclerosis. The resultswere published in the MultipleSclerosis Journal.

Inspired // Issue #27, May 2017

We are now expanding thescientific application portfoliowith contributions from theNGIs, the EGI-EngageCompetence Centres and anyother partner who is willing toengage with EGI.

To make present and futureservices discoverable andaccessible online, the project hasbeen successfully prototyping aMarketplace, which we areplanning to bring in full

Services by and for the EGI Community

External Services: Compute, Data & Storage, TrainingFor researchers, SMEs & industry, citizens & governments, highereducation. www.egi.eu/services/

Internal Services: Making the federation workFor the members of the EGI Council. www.egi.eu/internal-services/

Scientific applications powered by EGI servicesProvided by and for members and partners of the EGICommunity. Currently available: HADDOCK, Chipster, PowerFit,NBIS toolkit, VIP, Jupyter Notebook, Peachnote, and many more.www.egi.eu/use-cases/scientific-applications-tools/

4Inspired // Issue #27, May 2017

production in August. Themarketplace will give thepossibility to browse services bycategory, choose configurationoptions and place serviceorders.

New ServicesDataHub will increase accessi-bility to third-party research datafor downstream analysis viafederated authentication andauthorization, the bridging ofpreserved data and computingwith the federation of distributeddata repositories and thepossibility of associatingpermanent identifiers to theoutput data that is generated bythe processing workflows of EGI.

The DataHub is currently beingvalidated to address two usecases: the cross-domainfederation of existing storageinfrastructures, and theprovisioning of a distributedplatform for the management ofreplicas of publicly datacollections.

Applications on Demand (AoD)is a service designed to offer ascalable HTC environment. Itdelivers a whole clustercomplete with job scheduler anda library of scientific applications(e.g. Statistical R, JupyterNotebook) and generic utilities(e.g. Docker). The AoD alsoallows researchers to runcustom scientific applications.AoD's compute and storageservices are currently providedby CYFRONET, INFN, BELSPO,MTA SZTAKI, CESGA, UPV, andBIFI. All NGIs interested inmaking the AoD library availableto national user communitiescan simply “plug” their nationalIaaS resources to the service.The service was opened for betatesting at the EGI Conference2017.

The EGI CheckIn will enableaccess to EGI services usingfederated authenticationmechanisms. It is a proxy serviceoperated as a central hub,connecting federated Identity

Providers (IdPs) residing ‘outside’of the EGI ecosystem, andService Providers (SPs) that arepart of EGI. CheckIn wassuccessfully integrated with theELIXIR AAI infrastructure, whichoperates both an IdP andattribute provider service tomanage user accounts andpersonal attributes for everyELIXIR user. CheckIn is in nowproduction and services inaddition to the current ones(GGUS, GOCDB and AppDB) arebeing incrementally added.

More information

Tiziana Ferrari is theTechnical Director of the theEGI Foundation. @tferrariEGI

EGI Serviceshttps://www.egi.eu/services

Applications on Demandhttps://access.egi.eu/start

Save the date!

Digital Infrastructuresfor Research 2017

30 Nov - 1 DecemberBrussels, Belgium

5

Society is facing unprecedentedchallenges for food, water andenergy security. Resilience tonatural hazards is an increasinglyimportant issue as is ecosystemsustainability. Populationgrowth, pandemics and thedevelopment of a sustainableeconomy all have currentlyunknown impacts against abackdrop of climate change,which has the potential toexacerbate all these issues.

Earth observations are necessaryto report and model climatechange and to calculate green-house gas emissions, in line withthe Paris Agreement 2016 onClimate. The intergovernmentalGroup on Earth Observations(GEO) works to connect thedemand for sound and timelyenvironmental information withthe supply of data andinformation about the Earth thatis collected through observingsystems and made available bythe GEO community. GEO isleading a worldwide effort tobuild a Global Earth ObservationSystem of Systems (GEOSS). GEOconvenes providers and users ofopen Earth Observation data,

GEOSS Data and Computing Challenges

Stefano Nativi and Barbara Ryan write about a data platform for earth observation

aiming to highlight best practiceand eliminate duplication ofeffort to harness the DataRevolution for the benefit ofhumanity.

GEOSS is a set of coordinated,independent Earth observation,information and processingsystems that interact andprovide access to diverseinformation for a broad range ofusers in both public and privatesectors. GEOSS links thesesystems to strengthen themonitoring of the state of theEarth. Developed over the lastdecade, GEOSS makes morethan 200,000,000 open EarthObservation data resourcesaccessible for better decisionson a range of areas from foodsecurity, to protection ofbiodiversity, renewable energyand disaster resilience. Withalmost 160 data providers, animportant element of GEOSS isthe brokering framework calledthe GEO DAB (Discovery and

Inspired // Issue #27, May 2017

Access Broker). The GEO DABimplements a brokering patternensuring the necessaryscalability and flexibility.

The GEOSS evolution includesbig data analytics capabilities,particularly data cube functiona-lities, to move from data sharingto information and knowledgegeneration and sharing, inparticular to support the UNAgenda 2030 and the SustainableDevelopment Goals. In keepingwith the System-of-Systemsprinciples and leveraging theimplemented brokering pattern,GEOSS is looking at the EGI as avaluable e-infrastructuresfederation to underpin the newanalytics capabilities requestedby the GEO Community.

More information

Group on EarthObservations (GEO)www.earthobservations.org/

Global Earth ObservationSystem of Systems (GEOSS)www.earthobservations.org/geoss.php

Stefano Nativi is co-chair ofthe "GEOSS Architecture andEvolution" Working Group.

Barbara Ryan is the directorof the GEO Secretariat

GEOSS in numbers

> GEOSS finalizes about 15,000 queries per day.> In 2016 GEOSS handled about 4.5 million of requests.> GEOSS brokers and harmonizes almost 160 GEOSS Providersof data and information.> The GEOSS Providers share about 200 million data granules(i.e. downloadable files), organized in about 42 million datasets.> The GEOSS Portal counts about 3,000 visits per month.

6

Earth observations fromsatellites produce vast amountsof data. In particular, the newCopernicus Sentinel missionsare playing an increasinglyimportant role as a reliable high-quality and free open datasource for scientific, publicsector and commercial activities.

ICT solutions can facilitate thehandling of these large volumesof data and are nowadaysmodifying the expectations thatorganisations have on newservice development and onsupport to Earth Observation(EO) data exploitation. Their goalis more and more to developcapacities to create added value,involving SLAs and accountabilitywith business partners for thedata products and services theybring in this process.

Terradue Cloud Platform isaddressing this topic withsolutions to transfer EOprocessing algorithms to cloudinfrastructures. The platformalso provides services tooptimise the connectivity of thedata centres with moreintegrated discovery andprocessing methods. Forexample, Terradue provides theengineering and operationalsupport for the GeohazardsExploitation Platform (GEP), anESA-funded partnership alsoinvolving private companies(TRE-ALTAMIRA), researchcentres (CNR IREA, CNRS ENS,CNRS EOST and INGV) and spaceagency (DLR EOC).

Terradue & EGI: a partnership for Earth Observation

Pedro Gonçalves on how EGI's Federated Cloud supports Terradue's operations forthe Geohazards Exploitation Platform

GEP and EGIGEP offers a rich set of ready touse EO data processing servicesfor the analysis and monitoringof earthquake, volcanoes andlandslides. The platformfederates the geohazardscommunity by creating aworkplace with cloud-basedmodels of collaboration, wheredata providers, users andtechnology providers join forcesto produce scientific andcommercial exploitable results.

EGI supports Terradue withmatchmaking services betweenICT consumers and theappropriate provider(s) acrossthe EGI Federation and beyond.The computing and storageresources from the ReCaS Bariand BELNET-BEGRID centres areused by Terradue to help theglobal scale systematicproduction of the DLR InSARBrowse Medium-ResolutionService on the GEP. With thisservice, the platform producesinterferograms to show whereearthquakes are most likely toimpact society.

Inspired // Issue #27, May 2017

Currently in a ramp-up phase,which began in September 2016covering 20% of the worldseismic mask, GEP is planned toreach its peak of 50% by the 2ndquarter of 2017 with a produ-ction rate of about 320 Sentinel-1 scenes (160 interferometricpairs) per day. This production isfully ran on EGI Federated Cloudresources.

The Service level agreements(SLAs) established with EGIFoundation enabled Terradue toextend the hybrid cloud infra-structure using a new Open-Nebula OCCI driver, andprovided a reliable cloudinfrastructure for the users ofthe ESA Thematic ExploitationPlatforms.

More information

Terraduehttp://www.terradue.com/

Pedro Gonçalves is Chieftechnical Officer andFounder of Terradue

7

The EGI Accounting Portal is anoperational tool that processes,summarizes and displays theAccounting Repository data,acting as a common interface tothe different accounting recordproviders. Readers can browseviews of the data displayed in auser-friendly way. The portalhelps EGI members and externalparties to understand the use ofresources and how they servethe needs of research.

The Accounting Portal isdeveloped and operated atCESGA and is supported by theaccounting repository infra-structure operated at STFC,where records from the all EGIdata centres are collected.

During the EGI-Engage project,the Accounting Portal wascompletely renewed adoptingnew web tools that offer a betteruser experience and its look andfeel was improved through amodern interface. The portalwas re-organised with anintuitive menu to follow thestructure of the EGI servicecatalogue.

Furthermore, we developed acompletely new view to retrieveaccounting data for the ScientificDisciplines and new types orreports, for example the TOP10/100 resource centres. Newfeatures are also available suchas the dynamic interaction withgraphics, allowing to exposefurther details or zoom certainparts, the contextualised onlinehelp, providing informationabout the meaning of the

The new Accounting Portal

Diego Scardaci introduces the new version of one of EGI's key tools

several metrics and variablesused within the portal. Customviews for research infrastru-ctures can be also easily added,as it was already done for WLCG(see the Research Infrastructureitem in the main menu).

Accounting portal developmentsare continuing and, before theend of EGI-Engage project, morefeatures will be made available.This will include:

> integration with the EGIFederated AAI (the CheckInservice)

> maps with the distribution ofthe accounting data over ageographical area and

More information

The Accounting Portal wasdeveloped by CESGA andaggregates informationcollected in the accountingrepository operated by STFC.

http://accounting.egi.eu

Diego Scardaci is a memberof the EGI Foundation UserCommunity Support Team

Inspired // Issue #27, May 2017

What can we learn fromthe Accounting Portal?

How many HTC jobs were executed by NGIs this year?From 5,600 submitted by the Croatian NGI in January to themore than 19 million sent by CERN in April.

Which data centres were the top 10 HTC resource providers inthe last 12 months (normalised elapsed time)?CERN-PROD, RAL-LCG2 (UK), INFN-T1 (Italy), FZK-LCG2 (Germany),IN2P3-CC (France), GRIF (France), DESY-HH (Gremany), RRC-KI-T1(Russia), Tokyo-LCG2 (Japan), SARA-MATRIX (Netherlands)

How much CPU time was consumed by researchers in the EGIHTC infrastructure in the last 6 months?About 14 billion hours, measured in normalised CPU time(HEPSPEC06)

And if you want specifics... in December 2016 how many CPUhours were consumed by researchers working in the field ofAstronomy?About 330 million (normalised elapsed time).

How many Virtual Machines were created by the researchersin the top 10 Cloud resources providers in the last 12 months?About 190,000.

> the possibility to choose themeasurement unit to be used toshow the accounting data.

8

Seas and oceans are importantdrivers for the Europeaneconomy and they need to bepreserved and developed in asustainable way. In support ofthis view, the EuropeanCommission is implementing along-term strategy called BlueGrowth to boost the progress ofthe European marine sector.

One of the initiatives adhering toBlue Growth is OPENCoastS, orOn-demand Operational CoastalCirculation Forecast Service.

OPENCoastS is a service thatbuilds on-demand circulationforecast systems for differentusers along the North Atlanticcoast. OPENCoastS generatesforecasts of water levels, 2Dvelocities and wave parametersover the region of interest forperiods of 72 hours. The systemis useful in anticipating naturaldisasters and accidents in thecoast, e.g. floods and chemicalspills and can help in search andrescue operations.

The service was developed bythe National Laboratory for CivilEngineering (LNEC) in 2010 asWIFF (Water InformationForecast Framework) and usesthe SCHISM modelling system.

Since 2010, the system has beenproducing 48-hour forecasts ona daily basis for the Portuguesecoast and is running on High-Throughput Compute andstorage resources provided bythe NCG-INGRID-PT data centre,which is part of the Portuguese

Predicting water conditions along the Iberian coasts

Iulia Popescu writes about the OPENCoastS project and its forecast mission

National Distributed ComputingInfrastructure (INCD) and the EGIfederation.

LNEC provides this service toPortuguese-based researchersfree of charge and the regionalforecasts are open to everyonevia the web portal, but LNECresearchers Anabela Oliveiraand Alberto Azevedo areworking on making OPENCoastSa service available at aninternational scale.

The role of distributedcomputing and the future ofOPENCoastSOliveira and Azevedo argue thatthe development of forecastsystems requires a strongknowledge of coastal processesand IT, along with access tosignificant computational andstorage resources. The INCDcomputational and storagecapacity needed to run thewater forecast service every dayuses three typical configurations:

> Reference run: 60h CPU + 3GBstorage / 72h period dailyforecast

> Large run : 600h CPU + 30GB

Inspired // Issue #27, May 2017

storage / 72h period dailyforecast

> Very large reference run:1200h CPU + 60GB storage / 72hperiod daily forecast

These values can amount to upto 2500 cores and 2TB storageper year for a total of 100 sitedeployments.

In the future, the platform coulduse even more computingresources to facilitate the accessto circulation forecasts tobiologists, geologists andbiogeochemists, who havestrong needs in understandingthe impact of water dynamicsand ecology.

More information

OPENCoastS portalhttp://ariel.lnec.pt/

Laboratório Nacional deEngenharia Civil (LNEC)http://www.lnec/pt

Iulia Popescu is aCommunications Officer atthe EGI Foundation

9

EGI is proud to be a partner ofthe European Open ScienceCloud for Research Pilot project(EOSCpilot). The two-year-longproject kicked off in January2017 and lays the foundation forthe European Open ScienceCloud (EOSC), as described bythe European Commission.

Led by STFC and involving 33partners across Europe, theproject will engage withstakeholders to build the skillsand trust necessary for an openapproach to scientific researchand develop a series of sciencedemonstrators with existingcommunities and infrastructure.

Another aim of the EOSCpilotproject is to understand andaddress the barriers that stopEuropean research from fullytapping into the potential ofdata. To improve interoperabilitybetween data infrastructures,the project will engage differentscientific and economicdomains, countries andgovernance models and willdemonstrate how resources canbe shared even when they arevery large and complex.

The science demonstratorsdeveloped by EOSCpilot are keyto the success of the project.They represent the earlyadopters of EOSC and areselected to integrate diverseservices and infrastructure todemonstrate interoperability indifferent scientific domains.

EGI is leading the Service Pilotstask within the project. This taskuses the expertise from existingservices to analyse the

EOSCpilot: Science Demonstrators

Matthew Viljoen on one of the project's key work packages

requirements of sciencedemonstrators and to identifyany technical gaps. EGI isuniquely positioned to do thisthanks to our team’s experiencein service delivery, our diversescientific community and linkswith other e-infrastructures(such as EUDAT and GÉANT).

During the first year of theproject, EOSCpilot has beenworking with the first round ofscientific demonstrators, and iscurrently selecting the nextround.

The Science DemonstratorsThe first five give a good idea ofthe broad range of communitiesand technologies that theproject is working with:

1) Data Preservation for HighEnergy Physics aims to bring theOpen Data Portal, developed byCERN to provide data preser-vation (including software anddocumentation) for the HighEnergy Physics community, tothe wider scientific community.The solution will be deployed ongeneric cloud services andproved to work with samplepreserved High Energy Physicsdata before being prepared fornon-HEP data.

Inspired // Issue #27, May 2017

2) PanCancer has developed theButler genome analysisframework on the EMBL-EBIEmbassy cloud and its EOSCpilotdemonstrator is working todeploy it on generic cloudservices. This will enable theanalysis framework to be usedby more communities.

3) The photon-neutron sciencecommunity demonstrator isworking on making its data andanalysis tools available toscientists all over the worldusing institutional and publiccloud services.

4) TextBridge is a text miningsolution developed for thehumanities and social sciencescommunity at the University ofFlorence. This community isworking with EOSCpilot tomigrate their university serviceto a generic cloud service.

More information

EOSCpilot projecthttp://eoscpilot.eu/

Matthew Viljoen is a SeniorOperations Officer of the EGIFoundation

10

The first Authentication andAuthorisation for Research andCollaboration (AARC) projectconcluded on 30 April 2017 aftertwo years, with many usefuloutputs for e-infrastructures,research infrastructures andlibraries. The second AARCproject, started 1 May, will buildon these achievements andbring a new focus.

AARC has been creating acommon framework for researchand collaboration communities,with one blueprint architecture,one set of policies, and onecollection of training materialsthat should work for everyoneand allow their authenticationand authorisation solutions towork together. AARC has alsobeen working with researchcollaborations to pilot andimprove specific technical andpolicy aspects.

AARC’s approach means thatresearch collaborations canspend less time and less moneyreinventing the authenticationand authorisation wheel, and

AARC is dead - long live AARC!

Laura Durnford writes about the bridge between the old and the new AARC projects

their researchers can focus onresearch. Safe and more reliableaccess for more researchers tomore services, data and software,will allow greater cooperationbetween research collaborationsand open up the possibilities forexciting new research.

What has the AARC projectproduced?> AARC Blueprint Architecture,a set of interoperable buildingblocks for people designing andimplementing access manage-ment solutions for internationalresearch collaborations

> Policies to complement thetechnical research plus recom-mendations and best practicesto implement a scalable andcost-effective framework forintegrated solutions:

>> Snctfi - identifies operationaland policy requirements to helpestablish trust between aninfrastructure and identityproviders. For use by personnelresponsible for management,operation and security of aninfrastructure and those wishingto assess its trustworthiness.

>> Sirtfi – AARC was the mainsponsor for work to create anassurance framework thatallows organisations tocooperate in the coordination ofincident response, in the eventof a federated security incident.

Inspired // Issue #27, May 2017

> Pilots expanding the coverageof federated access, pilotstesting the integration of AARCresults.

What next?While the goals and objectives ofthe second AARC project willlargely remain the same, theproject will take two mainroutes:

> AARC will expand efforts toengage with target communitiesto disseminate information,deliver training, gain feedback,and implement the AARCframework. The number ofpartners has increased from 20to 26, helping to make thisapproach easier.

> AARC will shift the technicalfocus to the question of how toauthorise permitted users toaccess the resources across theboundaries of differentinfrastructures and researchcollaborations.

To maximise the uptake ofexisting materials and any thatare developed in the next phase,a fresh impetus will be given todeveloping and deliveringtraining, and to disseminatingvia ‘how to…’ documents,webinars and so on. This will allbe supported by a change to theproject website so that itbecomes a shop window for theoutputs of the AARC project.

More information

AARC websitehttps://aarc-project.eu/Laura Durnford is a SeniorCommunications Officer atGÉANT.