Distributed Computing Instrastructureas a Tool for e-Science

Jacek Kitowski1,2(B), Kazimierz Wiatr1,3, �Lukasz Dutka1, Maciej Twardy1,Tomasz Szepieniec1, Mariusz Sterzel1, Renata S�lota1,2, and Robert Paj ↪ak1

1 AGH University, ACC Cyfronet AGH, Krakow, Polandkito@agh.edu.pl

2 Department of Computer Science, AGH University, Krakow, Poland3 Department of Electronics, AGH University, Krakow, Poland

Abstract. It is now several years since scientists in Poland can usethe resources of the distributed computing infrastructure – PLGrid. Itis a flexible, large-scale e-infrastructure, which offers a homogeneous,easy to use access to organizationally distributed, heterogeneous hard-ware and software resources. It is built in accordance with good orga-nizational and engineering practices, taking advantage of internationalexperience in this field. Since the scientists need assistance and close col-laboration with service providers, the e-infrastructure is relied on users’requirements and needs coming from different scientific disciplines, beingequipped with specific environments, solutions and services, suitable forvarious disciplines. All these tools help to lowering the barriers that hin-der researchers to use the infrastructure.

Keywords: Distributed infrastructure · IT tools and services ·Computing platforms · Clouds and grids

1 Introduction

The main goal of research is scientific discovery of unknown phenomena. Amongtypical three methodologies making new findings realistic: theoretical approachesby using sophisticated analytical methods, experimental investigations with (usu-ally) big and expensive installations and computational studies, making wideuse of information technology (IT), the last one has resulted in increasing pop-ularity. Due to the complexity of most of the current problems this is a nat-ural way to harness IT approach for both basic research, especially for extremedimension/time scales and for versatile analysis of big data already existing ordescended from experiments. Hence, computing infrastructures have led to ever-increasing contribution to e-Science research, while facing users with demandingtechnological obstacles, due to complicated IT stuff.

In order to prevent the users from the thorny technical problems and to offerthem the most efficient and the most convenient way of making research onfrontiers and challenges of current science – creation of a more flexible and easyto use ecosystem is required.c© Springer International Publishing Switzerland 2016R. Wyrzykowski et al. (Eds.): PPAM 2015, Part I, LNCS 9573, pp. 271–280, 2016.DOI: 10.1007/978-3-319-32149-3 26

272 J. Kitowski et al.

In this paper we present assumptions and foundations of the distributedcomputing e-infrastructure as a tool for e-Science. The presented use case coversits implementation for Polish scientists.

2 Issues for e-Infrastructure Creation

Creation of an e-infrastructure needs synergistic effort in several dimensions:

1. Meeting user demands in the field of grand challenges applications.The activity toward a new e-infrastructure should be supported by a signif-icant group of users with real scientific achievements and wide internationalcollaboration as well as by well-defined requirements.

2. Organizational, which is probably the most important, though the mostdifficult in reality. Two perspectives are significant – horizontal and vertical –equally important and complementing each other.

In the horizontal perspective, a federation of computer centres supportingthe e-infrastructure with different kinds of resources and competences to coverinterests of different groups of users is proposed. Some evident topics are tobe addressed, like policy, collaboration rules, duties and privileges of eachparticipant, for smooth and secure operation. Another feature to be attainedis efficient use of federation resources by evaluation of computational projectsfrom the community in order to grant them the most appropriate softwareand hardware environments.

In the vertical perspective, organizational involvement of computer, com-putational and domain-specific experts into e-infrastructure operations is tobe introduced for development of the most suitable hardware and softwareenvironments for the users, directly dedicated to their needs. Such a kindof collaboration provides the scientific community with necessary expertise,support from the structural, many level helpdesk and training for easy andefficient research using the e-infrastructure. A good example of such organi-zation is Gauss Centre for Supercomputing [1].

3. Technological, which covers several issues including different computinghardware and software supported with scientific libraries, as well as a portfo-lio of middleware environments (e.g. gLite, UNICORE, QCG, generic cloud,like OpenNebula) and user-friendly platforms and portals. On that basis moresophisticated, tailored programming solutions can be developed.

4. Energy awareness, being a relative recent development. The problems facedare optimal scheduling strategies of computing jobs among federation resourcesto minimize energy consumption as a whole. As scale of resources and numberof jobs increase, this problem becomes more critical than ever (e.g. [2]). Energyawareness is also a topic that influences selection of computing hardware.

3 Case Study of e-Infrastructure Conceptualization andImplementation

Due to large funding initiative in Poland and as a response to requirements ofscientists, the Polish Grid Consortium was established in 2007, involving five

Distributed Computing Instrastructure as a Tool for e-Science 273

of the largest Polish supercomputing centres: ACC Cyfronet AGH in Krakow(the coordinator), ICM in Warsaw, PCSS in Poznan, CI TASK in Gdansk andWCSS in Wroc�law. Members of the Consortium agreed to work as a federa-tion to commence and jointly participate in the PLGrid Programme, to createa nationwide e-infrastructure and significantly extend the amount of computingresources provided to the scientific community.

Up-to-date fulfillment of the PLGrid Programme consists of several stagescompleted in subsequent projects.

– PL-Grid Project (2009–2012) aiming at providing the scientific communitywith basic IT platforms and computing services offered by the Consortium,initiating realization of the e-Science model in the various scientific fields. Oneof the measurements of success was ranking of all partners’ resources by theTOP500 list (with total performance of 230 Tflops) in fall 2011, with Zeuscluster in Cyfronet located at 81st position.

– PLGrid Plus Project (2011–2015) focused on users, involving three kinds of con-tractors: computer, computational and domain-specific experts, which resultedin introducing 13 scientific domains with specialized software and hardwaresolutions, together with portals and environments. The total computationalpower offered by the Consortium was increased by additional 500 Tflops.

– PLGrid NG Project (2014–2015) targeting future development of the scien-tific domains by including into the project subsequent 14 scientific areas, dueto rapid increase in demand for services for researchers in other fields. Newdomain-specific services cover a wide range of specialties – including provisionof the specialized software, mechanisms of data storage, modern platforms inte-grating new type of tools and specialized databases – to speed up obtaining sci-entific results as well as streamline and automate the work of research groups.

– PLGridCoreProject (2014–2015) affirmed recognition ofCyfronet as aNationalCentre of Excellence, constituting the next step towards cloud computing andhandling big data calculations. It aims not only at extension of hardware andsoftware portfolio, but also dedicated accompanying facilities. One of them –a new backup Data Centre is on agenda. A new HPC asset has been installed,called Prometheus, with 1.7 Pflops, and put in operation in May 2015 for thecommunity.

It is worth to mention the number of users close to 4000 currently, publishingregularly in highly ranked international journals, often with international collab-orators, and many international projects ongoing with the help of the PLGridinfrastructure, funding by FP6, FP7, RFCS, EDA and other international agen-cies and collaborations. Two books on computing environments, portals, solu-tions and approaches developed during the Programme have been published bySpringer Publisher [3,4].

4 PLGrid Platforms – Selected Examples

The computing infrastructure offered by the PLGrid infrastructure is not limitedonly to high performance computing clusters and large storage resources. A set

274 J. Kitowski et al.

of platforms, tools and services is provided, which hide the complexity of theunderlying IT infrastructure and, at the same time, expose the actual functionsthat are important to perform the research. Within this section, capabilities ofselected tools are described.

4.1 GridSpace – Web-Enabled Platform for Reproducible,Reviewable and Reusable Distributed Scientific Computing

GridSpace2 [5], as built on top of provided computing capabilities, enables sci-entists to easily create and run so-called in silico experiments that are featuredby: (a) reproducibility – ability to effortlessly run the experiment at anothertime, by the other researcher or user, on the other computing capacity or usingthe alternative software, (b) reviewability – ability to effectively examine, verify,assess, test and scrutinize the experiment, (c) reusability – ability to smoothlyapply the experiment to the other case, for the other purpose or in the othercontext.

Fig. 1. GridSpace2 platform layers

GridSpace2 experiments are fully immersed in World Wide Web and struc-tured as workflows composed of code and data items (see Fig. 1). Code itemscan be written in diverse programming languages and are interpreted by, socalled, interpreters, which are implemented as executables and executed throughexecutors on the underlying e-infrastructure. Executables installed on the e-infrastructure carry out computations while executors manage and orchestrate

Distributed Computing Instrastructure as a Tool for e-Science 275

computation and data flow. Data items are simply file system elements that areprocessed, namely read and/or written, when executing code items. In the weblayer, code and data items are embeddable as HTML iframe elements, whichenables to create mash-up web pages that integrate content of various type andsources, including interactive GridSpace2 experiment items.

GridSpace2 is a generic and versatile platform that was applied in exper-iments from various scientific domains such as chemistry, material and urbanengineering, physics and medicine. It was also adopted as a technology for exe-cutable scientific papers, namely, Collage Authoring Environment [6] that wasintegrated with the Elsevier ScienceDirect portal and empowered the first scien-tific journal issue featuring executable papers.

4.2 InSilicoLab – Science Gateway Framework

InSilicoLab [7] is a framework for building application portals, also called ScienceGateways. The goal of the framework development is to create gateways that,on one hand, expose the power of large distributed computing infrastructuresto scientists, and, on the other, allow the users to conduct in silico experimentsin a way that resembles their usual work. The scientists using such an applica-tion portal can treat it as a workspace that organizes their data and allows forcomplex computations in a manner specific to their domain of science.

An InSilicoLab-based portal is designed as a workspace that gathers all thata researcher needs for his/her in silico experiments. This means: (a) capabilityof organizing data that is a subject or a product of an experiment, i.e., facili-tating the process of preparation of input data for computations, possibility ofdescribing and categorizing the input and output data with meaningful metadataas well as searching and browsing through all the data based on the metadata,(b) seamless execution of large-scale, long-lasting data- and computation-intensiveexperiments.

Every gateway based on the InSilicoLab framework is tailored to a specificdomain of science, or even to a class of problems in that domain. The core of theframework provides mechanisms for managing the users’ data – categorizing it,describing with metadata and tracking its origin – as well as for running com-putations on distributed computing infrastructures. Every InSilicoLab gatewayinstance is built based on the core components, but is provided with data mod-els, analysis scenarios and an interface specific to the actual domain it is createdfor (see Fig. 2).

4.3 DataNet – Data and Metadata Management Service

DataNet [8] is a service built on top of the PLGrid high-performance computinginfrastructure to enable lightweight metadata and data management. It allowscreating data-models consisting of files and structured data to be deployed asspecific repositories within seconds.

One of the main goals of DataNet is to make it usable from the largest set oflanguages and platforms possible. That is why the HTTP protocol was used as

276 J. Kitowski et al.

Fig. 2. Architecture of the InSilicoLab framework: domain layer, mediation layer withits core services, and resource layer. In the resource layer, Workers (‘W’) of differentkinds (marked with colors) are shown.

a basis for transferring data between computing nodes and the service, togetherwith the REST methodology applied to structure the messages sent to and fromthe repositories.

DataNet is fully integrated with the PLGrid’s authentication and authoriza-tion system, so existing users can quickly gain access to the service with a fullyautomated registration process.

In order to ensure user data separation, each repository is deployed on a ded-icated PaaS platform, which ensures scaling and database service provisioningfor structured data. For high-throughput scenarios, it is possible to configurethe system to expose several instances of a given repository to increase requestprocessing rate.

Another aspect of using DataNet for data management is collaborative dataacquisition, which is possible, because a given repository is identified by a uniqueURL. The URL can be shared among many computing infrastructures, softwarepackages and different users or groups of users to acquire and process data withina single data model. For some collaboration efforts with large amounts of filesthis introduces structure and means to search the file space.

Figure 3 shows the layered architecture of the service.

4.4 Scalarm – a Platform for Data Farming

Executing a computer simulation many times, each with different input parame-ter values, is a common approach to studying complex phenomena in various sci-ence disciplines. Data farming is a methodology of conducting scientific research,considered as an extension of the task farming approach, combined with Designof Experiment (DoE) methods for parameter space reduction, and output dataexploration techniques [9].

Distributed Computing Instrastructure as a Tool for e-Science 277

Fig. 3. DataNet architecture

A crucial requirement for efficient application of the data farming methodol-ogy is usage of dedicated tools supporting each phase of in silico experiments,following the methodology. Scalarm [10] is a complete platform, supporting theall above-mentioned data farming experiment phases, starting from experimentdesign, through simulation execution, to results analysis. All Scalarm functionsare available to the user via GUI in a web browser (cf. Fig. 4).

To perform data farming experiment in Scalarm, a user prepares a simula-tion scenario, with input parameter types and an application specified. Through

Fig. 4. Basic experiment progress view in Scalarm

278 J. Kitowski et al.

the use of so-called adapters, any application can be run without modification,which allows to use Scalarm in the wide range of scientific disciplines, like metalprocessing technologies [11] or complex multi-agent simulations [12].

In addition to various scheduling systems for Grids, Scalarm supports severalCloud services [13] and user-defined servers. It also supports different resultsanalysis methods with graphical presentation (see Fig. 4) as well as autonomousinput space exploration methods, allowing to change parameter space withoutuser intervention, to satisfy user-defined experiment goal.

4.5 Onedata – Uniform and Efficient Access to Your Data

Grid infrastructures consist of many types of heterogeneous distributed storagesystems, managed locally [14]. Taking into account possible different require-ments of a user, in terms of access to data [15], it is beneficial to provide a varietyof storage systems, which poses challenges for unifying data access. Due to theindependence of the centers in the grids, the management of storage systems(storage services) is decentralized.

The Onedata system [16] provides unified and efficient access to data stored inorganizationally distributed environments, e.g. Grids and Clouds, and it is a com-plete response to the requirements of end-users, developers and administrators.

To offer the required functionalities, Onedata [17] merges and extends: (1)data hosting service, (2) high performance file system, (3) data managementsystem and (4) middleware for developers.

While perceived as a high performance file system, Onedata provides accessto data via a standard file system interface, offering coherent and uniform viewon all data that can be distributed across the infrastructure of a geographicallydistributed organization.

Onedata is a data management system, which allows to manage various stor-age systems in a cost-effective manner without abandoning the uniform viewon data and high performance. Its data management environment consists of:(a) monitoring systems, which gather information about storage utilization,(b) rules definition for automatic data management, (c) event-driven automaticdata management based on the rules.

To provide high performance and scalability, Onedata is implemented inErlang and in C language with noSQL database used. Information about meta-data and the system state is stored in a fault-tolerant, high-performance, dis-tributed noSQL database to avoid performance bottlenecks and guarantee datasecurity.

5 Conclusions

The realization of the PLGrid Programme fits well with the need of developmentof an advanced IT infrastructure designed for the implementation of modern sci-entific research. The well-tailored PLGrid e-infrastructure fulfills researchers’needs for suitable computational resources and services. It also enables Polish

Distributed Computing Instrastructure as a Tool for e-Science 279

scientific units collaboration with international research organizations, becausevast range of services contribute to increase of cooperation between Polish scien-tists and international groups of specialists from twenty-seven different scientificdomains of e-Science.

The essential fact is that anyone who is performing scientific research canbe the user of the infrastructure. Access to the huge computational power, largestorage resources and sophisticated services on a global level is free to Polishresearchers and all those engaged in scientific activities associated with any uni-versity or research unit in Poland. To obtain an account in the PLGrid infrastruc-ture, enabling access to its computing resources, one should only register in thePLGrid Portal [18].

Since 2010, the PLGrid infrastructure has been a part of the European GridInfrastructure (EGI), which aims to integrate the national Grid infrastructuresinto a single, sustainable, production infrastructure. Further strong collaborationand exchange of ideas with EGI is foreseen.

Acknowledgements. This work was made possible thanks to the following projects:PLGrid Plus (POIG.02.03.00-00-096/10), PLGrid NG (POIG.02.03.00-12-138/13) andPLGrid Core (POIG.02.03.00-12-137/13), co-funded by the European Regional Devel-opment Fund as part of the Innovative Economy programme, including the specialpurpose grant from the Polish Ministry of Science and Higher Education.

References

1. Gauss Centre for Supercomputing (GCS) (2015). http://www.gauss-centre.eu/gauss-centre/EN/AboutGCS/aboutGCS node.html

2. Gienger, M.: Towards energy aware scheduling between federated Data Centres. Apresentation given at eChallenges International Conference Bristol (2014)

3. Bubak, M., Szepieniec, T., Wiatr, K. (eds.): Building a National Distributed e-Infrastructure - PL-Grid: Scientific and Technical Achievements. LNCS, vol. 7136.Springer, Heidelberg (2012)

4. Bubak, M., Kitowski, J., Wiatr, K. (eds.): eScience on Distributed ComputingInfrastructure. LNCS, vol. 8500. Springer, Heidelberg (2014)

5. Ciepiela, E., Wilk, B., Har ↪ezlak, D., Kasztelnik, M., Pawlik, M., Bubak, M.:Towards provisioning of reproducible, reviewable and reusable in-silico experi-ments with the GridSpace2 platform. In: Bubak, M., Kitowski, J., Wiatr, K. (eds.)eScience on Distributed Computing Infrastructure. LNCS, vol. 8500, pp. 118–129.Springer, Heidelberg (2014)

6. Collage Authoring Environment (2015). https://collage.elsevier.com/7. Kocot, J., Szepieniec, T., Wojcik, P., Trzeciak, M., Golik, M., Grabarczyk, T.,

Siejkowski, H., Sterzel, M.: A framework for domain-specific science gateways.In: Bubak, M., Kitowski, J., Wiatr, K. (eds.) eScience on Distributed Comput-ing Infrastructure. LNCS, vol. 8500, pp. 130–146. Springer, Heidelberg (2014)

8. Har ↪ezlak, D., Kasztelnik, M., Pawlik, M., Wilk, B., Bubak, M.: A lightweightmethod of metadata and data management with DataNet. In: Bubak, M., Kitowski,J., Wiatr, K. (eds.) eScience on Distributed Computing Infrastructure. LNCS, vol.8500, pp. 164–177. Springer, Heidelberg (2014)

280 J. Kitowski et al.

9. Kryza, B., Krol, D., Wrzeszcz, M., Dutka, �L., Kitowski, J.: Interactive cloud datafarming environment for military mission planning support. Comput. Sci. 13(3),89–99 (2012)

10. Krol, D., Kryza, B., Wrzeszcz, M., Dutka, �L., Kitowski, J.: Elastic infrastructure forinteractive data farming experiment. In: Procedia Computer Science, Proceedingsof ICCS 2012 International Conference on Computer Science, vol. 9 special issue,pp. 206–215. Omaha, Nebraska (2012)

11. Krol, D., S�lota, R., Rauch, �L., Kitowski, J., Pietrzyk, M.: Harnessing HeterogeneousComputational Infrastructures for Studying Metallurgical Rolling Processes. In:Proceedings of eChallenges 2014 Conference, 29–30 October 2014. IIMC, Belfast(2014)

12. Laclavik, M., Dlugolinsky, S., Seleng, M., Kvassay, M., Schneider, B., Bracker, H.,Wrzeszcz, M., Kitowski, J., Hluchy, L.: Agent-based simulation platform evaluationfor human behavior modeling. In: Proceedings of ITMAS/AAMAS 2011, pp. 1–15.Taipei (2012)

13. Krol, D., S�lota, R., Kitowski, J., Dutka, �L., Liput, J.: Data Farming on Hetero-geneous Clouds. In: Proceedings of 7th IEEE International Conference on CloudComputing (IEEE CLOUD 2014). IEEE (2014)

14. S�lota, R., Dutka, �L., Wrzeszcz, M., Kryza, B., Nikolow, D., Krol, D., Kitowski, J.:Storage management systems for organizationally distributed environments PLGridPLUS case study. In: Wyrzykowski, R., Dongarra, J., Karczewski, K., Wasniewski, J.(eds.) PPAM 2013, Part I. LNCS, vol. 8384, pp. 724–733. Springer, Heidelberg (2014)

15. S�lota, R., Niko�low, D., Ska�lkowski, K., Kitowski, J.: Management of data accesswith quality of service in PL-Grid environment. Comput. Inform. 31(2), 463–479(2012)

16. Dutka, �L., S�lota, R., Wrzeszcz, M., Krol, D., Kitowski, J.: Uniform and effi-cient access to data in organizationally distributed environments. In: Bubak, M.,Kitowski, J., Wiatr, K. (eds.) eScience on Distributed Computing Infrastructure.LNCS, vol. 8500, pp. 178–194. Springer, Heidelberg (2014)

17. Wrzeszcz, M., Dutka, �L., S�lota, R., Kitowski, J.: VeilFS - a new face of storageas a service. In: Proceedings of eChallenges 2014 Conference, 29–30 October 2014.IIMC, Belfast (2014)

18. PLGrid Portal (2015). https://portal.plgrid.pl