High PerformanceComputing Cloud -a PaaS Perspective
PratimaDhuldhule, J.Lakshmi, S. K.
Nandy
Introduction
Motivation
Related Work
Methodology
Conclusion
Future Work
High Performance Computing Cloud - aPaaS Perspective
Pratima Dhuldhule, J. Lakshmi, S. K. Nandy
Supercomputer Education and Research CenterIndian Institute of Science, Bangalore
November 2, 2015
High PerformanceComputing Cloud -a PaaS Perspective
PratimaDhuldhule, J.Lakshmi, S. K.
Nandy
Introduction
Motivation
Related Work
Methodology
Conclusion
Future Work
Overview
Introduction
Motivation
Related Work
Methodology
Conclusion
Future Work
High PerformanceComputing Cloud -a PaaS Perspective
PratimaDhuldhule, J.Lakshmi, S. K.
Nandy
Introduction
Motivation
Related Work
Methodology
Conclusion
Future Work
Introduction
I Cloud computing is emerging as a latest computetechnology
I Properties of cloud like availability, elasticity andflexibility have attracted attention of scientists for HPCworkload
I HPC applicationsI dedicated resourcesI timely results
I Cloud platformI shared resourcesI virtualized environment
High PerformanceComputing Cloud -a PaaS Perspective
PratimaDhuldhule, J.Lakshmi, S. K.
Nandy
Introduction
Motivation
Related Work
Methodology
Conclusion
Future Work
Motivation
I HPC applicationsI consume 90% of available resourcesI require low latencyI timely resultsI tight interconnect network
I Disadvantages of HPC in CloudI Shared resources - CPU jitter, I/O overhead, high
latencyI SecurityI Limited type of instancesI No tightly coupled interconnect network
High PerformanceComputing Cloud -a PaaS Perspective
PratimaDhuldhule, J.Lakshmi, S. K.
Nandy
Introduction
Motivation
Related Work
Methodology
Conclusion
Future Work
Motivation for PaaS Model
I Ability to provideon-demand platform
I Guarantees quality ofperformance
I Abstracts and controlsunderlying resources andgives choice of platform
High PerformanceComputing Cloud -a PaaS Perspective
PratimaDhuldhule, J.Lakshmi, S. K.
Nandy
Introduction
Motivation
Related Work
Methodology
Conclusion
Future Work
Related Work
I HPC on cloud has been studied for long time
I [4] - attempts to improve performance of HPCapplication on cloud
I [1] - science clouds for scientific applications
I [3] - discusses strategies to completely removehypervisor layer to overcome virtualization overhead
None of the above work talk about non-virtualized platformsfor HPC applications.
High PerformanceComputing Cloud -a PaaS Perspective
PratimaDhuldhule, J.Lakshmi, S. K.
Nandy
Introduction
Motivation
Related Work
Methodology
Conclusion
Future Work
Methodology
I An architecture to provide a platform with dedicatedresources
I Build resource allocation strategies to meet HPCrequirements
I Bring together PaaS advantages and HPC needsI Design Goals
I Isolated platformI Non-virtualized environment
High PerformanceComputing Cloud -a PaaS Perspective
PratimaDhuldhule, J.Lakshmi, S. K.
Nandy
Introduction
Motivation
Related Work
Methodology
Conclusion
Future Work
HPC Cloud Computing
HPC Cloud Architecture
High PerformanceComputing Cloud -a PaaS Perspective
PratimaDhuldhule, J.Lakshmi, S. K.
Nandy
Introduction
Motivation
Related Work
Methodology
Conclusion
Future Work
Case Study - Machine Failure
Failure can happen in two scenariosI Failure during Netboot
I ESU sends back error messageI Another machine is made available
I Failure after allocationI Monitor detects failureI Scale up request is made
High PerformanceComputing Cloud -a PaaS Perspective
PratimaDhuldhule, J.Lakshmi, S. K.
Nandy
Introduction
Motivation
Related Work
Methodology
Conclusion
Future Work
Sample XML request file
Sample XML request file
High PerformanceComputing Cloud -a PaaS Perspective
PratimaDhuldhule, J.Lakshmi, S. K.
Nandy
Introduction
Motivation
Related Work
Methodology
Conclusion
Future Work
Experimental Setup
I Intel core i7 processor with 8 GB RAM and 1 TBSATA2 disk
I For virtual environment KVM was used as hypervisorI native and virtual machine had following configuration
I single CPUI 1 GB RAM
All VMs were running same workload simultaneously
High PerformanceComputing Cloud -a PaaS Perspective
PratimaDhuldhule, J.Lakshmi, S. K.
Nandy
Introduction
Motivation
Related Work
Methodology
Conclusion
Future Work
Results
Effect of virtualization on CPU
High PerformanceComputing Cloud -a PaaS Perspective
PratimaDhuldhule, J.Lakshmi, S. K.
Nandy
Introduction
Motivation
Related Work
Methodology
Conclusion
Future Work
Results
Effect of virtualization on MemoryBandwidth
High PerformanceComputing Cloud -a PaaS Perspective
PratimaDhuldhule, J.Lakshmi, S. K.
Nandy
Introduction
Motivation
Related Work
Methodology
Conclusion
Future Work
Results
Effect of virtualization on MemoryExecution Time
High PerformanceComputing Cloud -a PaaS Perspective
PratimaDhuldhule, J.Lakshmi, S. K.
Nandy
Introduction
Motivation
Related Work
Methodology
Conclusion
Future Work
Results
Effect of virtualization on Disk IO
High PerformanceComputing Cloud -a PaaS Perspective
PratimaDhuldhule, J.Lakshmi, S. K.
Nandy
Introduction
Motivation
Related Work
Methodology
Conclusion
Future Work
Results
Effect of virtualization on NetworkingBandwidth for UDP
High PerformanceComputing Cloud -a PaaS Perspective
PratimaDhuldhule, J.Lakshmi, S. K.
Nandy
Introduction
Motivation
Related Work
Methodology
Conclusion
Future Work
Results
Jitter for UDP
High PerformanceComputing Cloud -a PaaS Perspective
PratimaDhuldhule, J.Lakshmi, S. K.
Nandy
Introduction
Motivation
Related Work
Methodology
Conclusion
Future Work
Results
Packet Loss for UDP
High PerformanceComputing Cloud -a PaaS Perspective
PratimaDhuldhule, J.Lakshmi, S. K.
Nandy
Introduction
Motivation
Related Work
Methodology
Conclusion
Future Work
Results
Bandwidth for TCP
High PerformanceComputing Cloud -a PaaS Perspective
PratimaDhuldhule, J.Lakshmi, S. K.
Nandy
Introduction
Motivation
Related Work
Methodology
Conclusion
Future Work
Conclusion
I In this work, we have designed an architecture forprovisioning of isolated platform for HPC workload
I We evaluated performance across various physicalresources and studied effect of virtualization on them
I We observed that virtualization causes performancedegradation in HPC applications
High PerformanceComputing Cloud -a PaaS Perspective
PratimaDhuldhule, J.Lakshmi, S. K.
Nandy
Introduction
Motivation
Related Work
Methodology
Conclusion
Future Work
Future Work
I Current architecture have features like starting, scalingand terminating a compute cluster
I Architecture needs further improved features such aspausing/resuming and saving state of compute cluster
I Testing of the architecture for real life HPC workloadscenarios
High PerformanceComputing Cloud -a PaaS Perspective
PratimaDhuldhule, J.Lakshmi, S. K.
Nandy
Introduction
Motivation
Related Work
Methodology
Conclusion
Future Work
References
[1] S. Srirama, O. Batrashev, and E. Vainikki, ”Scicloud:scientific computing on the cloud”, in Proceedings of the2010 10th IEEE/ACM International Conference on Cluster,Cloud and Grid Computing. IEEE Computer Society, 2010,pp. 579-580.[2] S. K. Garg, C. S Yeo, A. Anandasivam, and R. Buyya,”Environment-conscious scheduling of hpc applications ondistributed cloud-oriented data centers”, Journel of Paralleland Distributed Computing, vol. 71, no. 6, pp. 732-749,2011.[3] E. Keller, J. Szefer, J. Rexford, and R. B. Lee, ”Nohype:virtualized cloud infrastructure without the virtualization”, inACM SIGARCH Computer Architecture News, vol. 38, no.3. ACM, 2010, pp. 350-361.[4] A. J. Younge, J.P. Walters, J. Suh, D.-I. D. Kang, Y.Park, S. P. Crago, and G. C. Fox, ”Towards a highperformance virtualized iaas deployment”.
High PerformanceComputing Cloud -a PaaS Perspective
PratimaDhuldhule, J.Lakshmi, S. K.
Nandy
Introduction
Motivation
Related Work
Methodology
Conclusion
Future WorkThank you
