Amazon EC2 performance comparison
How does EC2 compare to traditional supercomputer for scientific applications?
"Performance Analysis of High PerformanceComputing Applications on the Amazon WebServices Cloud", 2010
HPC Benchmarking
NERSC - benchmark frameworkMPI
Head node, worker nodesFile server implemented with EBS
IPM - MPI communication monitorCompared:Amazon EC2 - N node, m1.large instance 4xEC2 compute units 1-1.2 ghz opteron or xeon per unitCarver - 400 node, 2 x intel quad 2.67 nehalem / nodeFranklin - 9660 node cray xt4, quad 2.3 opteron / nodeLawrencium - 198 node 2x intel xeon quad 2.66 / node
NERSC Benchmark SuiteCAM ● Community Atmosphere Model● Stresses processor data movement and MPI interconnect p2p bandwidth
Gamess● General Atomic and molecular electronic structure system● memory access and bandwidth, collective interconnect performance
GTC● Stresses indirect addressing and random access memory
IMPACT-T● Integrated Map and Particle Accelerator Tracking Time● sensitive to memory bandwidth and MPI performance
NERSC Benchmark Suite ContMAESTRO
● Stresses memory performance, latency and global communicationsMILC
● Stresses memory bandwidth, prefetching and processing powerParatec● Parallel Total Energy Code● Stresses global communication bandwidth, processing power
HPCC● 7 synthetic benchmarks● Targets computation, communications
Performance: Application Runtime
Metrics take into account cluster size
Performance Analysis of High Performance Computing Applications on the Amazon Web Services Cloud
Performance: Percentage runtime communicating using IPM
Performance Analysis of High Performance Computing Applications on the Amazon Web Services Cloud
Performance: Sustained Flops
Performance Analysis of High Performance Computing Applications on the Amazon Web Services Cloud
Considerations using EC2Heterogeneous cpus:● Intel Xeon E5430 2.66GHz quad-core processor● AMD Opteron 270 2.0GHz dual-core processor● AMD Opteron 2218 HE 2.6GHz dual-core processor● Cannot optimize code
High performance variability● Sharing hardware with other vms
Slow node communication● Gigabit ethernet
"Transient errors"● Failure to boot, network misconfigurations, virtual machine hangs
Not always able to acquire requested cores● 256+ cores require scheduling/reservation
Cost/Performance compared to Desktop Grid
How does Amazon EC2 compare to Grid Computing?
"Cost-Benefit Analysis of Cloud Computing versus Desktop Grids", 2009
Desktop Grid/Volunteer Computing
Fastest virtual supercomputers (From wikipedia)
Bitcoin network 168.26 PFLOPS
BOINC 5.634 PFLOPS
Folding@Home 5 PFLOPS
MilkyWay@Home 1.6 PFLOPS
SETI@Home 730 TFLOPS
Einstein@Home 210 TFLOPS
Amazon HPC 240 teraflops 17024 cores
Considerations using VC
Slow acquisition of computing resources● 7.8 days to achieve 1000 cloud node
equivalentSlow task deployment● time = (reconnections * # tasks) / # clients● 1000 tasks to 10000 nodes about 45 minSlow completion times● deadlines, priorities, 96+% completion rate● Average 9 days vs < 4 hours on dedicated
Cloud Power attainable given VC Costs
Resources Per Month
*One or the other
Given 12k/Month
Processing Storage
SETI 514 TeraFLOPS
7.7 TB
Amazon 2 TeraFLOPS* 80 TB*
Cloud-VC Hybrid Approach
Host VC Server on Cloud: Cost BreakdownCost-Benefit Analysis of Cloud Computing versus Desktop Grids
Storage vs Bandwidth
Storage vs Bandwidth for a fixed budget
Cost-Benefit Analysis of Cloud Computing versus Desktop Grids
Conclusions
VC outperform clouds on cost for large long term and highly parallel projects● Projects on the order of weeks● VC needs a certain number of volunteer nodes before
cost effectiveness● High startup costs make short term projects not cost
effective1 small EC2 instance is equivalent to 2.83 VC hostsHybrid approach can lower startup and monthly costs of VC● 40% savings on SETI