SAN FRANCISCO, CA, USA
Adaptive Energy-efficient Adaptive Energy-efficient Resource Sharing for Resource Sharing for Multi-threaded Workloads Multi-threaded Workloads in Virtualized Systemsin Virtualized Systems
Can Hankendi Ayse K. Coskun
Boston UniversityElectrical and Computer Engineering
Department
This project has been partially funded by:
Computing in Heterogeneous, Autonomous 'N' Goal-oriented Environments
Energy Efficiency in CEnergy Efficiency in Coomputing Clustersmputing Clusters
• Energy-related costs are among the biggest contributors to the total cost of ownership.
• Consolidating multiple workloads on the same physical node improves energy efficiency.
2
(Source: International Data Corporation (IDC), 2009)
Computing in Heterogeneous, Autonomous 'N' Goal-oriented Environments
Multi-threaded Applications in the Multi-threaded Applications in the CloudCloud
• HPC applications are expected to shift towards cloud resources.
• Resource allocation decisions significantly affect the energy efficiency of server nodes.
• Energy efficiency is a function of application characteristics.
3
Computing in Heterogeneous, Autonomous 'N' Goal-oriented Environments
OutlineOutline
• Background
• Methodology
• Adaptive Resource Sharing
• Results
• Conclusions
4
Computing in Heterogeneous, Autonomous 'N' Goal-oriented Environments
BackgroundBackground
Cluster-level VM Management
- Consolidation policies across server nodes
- VM migration techniques
[Srikantaiah, HotPower’08][Bonvin, CCGrid’11]
Node-level Management
Recent Co-scheduling policies-Co-scheduling contrasting workloads-Balancing performance events across nodes
- Cache misses- IPC- Bus accesses
[Dhiman, ISLPED’09][Bhadauria, ICS’10]
- Co-scheduling based on thread communication
- Identifying best thread mixes to co-schedule
[Frachtenberg, TPDS’05][McGregor, IPDPS’05]
5
Computing in Heterogeneous, Autonomous 'N' Goal-oriented Environments
Virtualized System SetupVirtualized System Setup
• 12-core AMD Magny Cours Server 2x 6-core dies attached side by side in
the same package Private L1 and L2-caches for each core 6 MB shared L3-cache for each 6-core die
6
• Virtualized through VMware vSphere 5 ESXi hypervisor 2 Virtual Machines (VM) with Ubuntu Server Guest OS
Computing in Heterogeneous, Autonomous 'N' Goal-oriented Environments
Methodology: Measurement Methodology: Measurement Setup Setup
• System-level power measurements at 1s sampling rate• Performance counter collection through vmkperf at 1s
sampling rate Counters: CPU cycles, retired instructions, L3-cache
misses• VM-level CPU and memory utilization data collection through
esxtop with 2s sampling rate
System-level power measurement
Logger
esxtop
vmkperf
7
Computing in Heterogeneous, Autonomous 'N' Goal-oriented Environments
Parallel WorkloadsParallel Workloads
• PARSEC 2.1 benchmark suite [Bienia et al., 2008]
Benchmark Application IPC Memory Acc.
blackscholes Financial Analysis Low Low
bodytrack Computer Vision High Medium
canneal VLSI Design Low High
dedup Enterprise Storage Medium Low
ferret Similarity Search Medium Low
freqmine Data Mining High Low
swaptions Financial Analysis High Low
streamcluster Data Mining Low High
vips Media Processing High Low
x264 Media Processing Medium Medium
8
Computing in Heterogeneous, Autonomous 'N' Goal-oriented Environments
Tracking Parallel PhasesTracking Parallel Phases
• consolmgmt• Consolidation management interface• Synchronizes ROI (region-of-interest) of
multiple workloadsconsolmgmt
parsecmgmt
hooks.c
roi-Trigger()start-Logging
Input (Serial) Output (Serial)
Input (Serial) Output (Serial)
Benchmark A
Benchmark B
sleep()
start-Logging() end-Logging()
roi-Trigger()
9
Computing in Heterogeneous, Autonomous 'N' Goal-oriented Environments
Performance Impact of Consolidation Performance Impact of Consolidation • Consolidating multiple workloads can degrade performance
due to resource contention.• Virtualization provides performance isolation by managing
memory and NUMA node affinities.• With native OS, performance variation is 2.5x higher.
10
Average throughput of Streamcluster when co-scheduled with another PARSEC benchmark
Computing in Heterogeneous, Autonomous 'N' Goal-oriented Environments
OutlineOutline
• Background
• Methodology
• Adaptive Resource Sharing
• Results
• Conclusions
11
Computing in Heterogeneous, Autonomous 'N' Goal-oriented Environments
Impact of Application SelectionImpact of Application Selection
• Previous co-scheduling policies focus on application selection to improve energy efficiency.
• Application selection is based on balancing memory operations and CPU usage.
12
A B C D
Computing in Heterogeneous, Autonomous 'N' Goal-oriented Environments
Predicting Power EfficiencyPredicting Power Efficiency
• To improve the energy efficiency, we need to allocate more CPU resources to power-efficient workloads.
• IPC*CPU Utilization metric shows strong correlation with power efficiency.
13
IPC
*CP
U U
tiliz
atio
n
Computing in Heterogeneous, Autonomous 'N' Goal-oriented Environments
• IPC*CPU Utilization metric is used to classify applications according to their power efficiency levels.
• We utilize density based clustering algorithm (DBSCAN) to determine application groups based on their power efficiency classes.
Application ClassificationApplication Classification
14
Computing in Heterogeneous, Autonomous 'N' Goal-oriented Environments
• IPC*CPU Utilization metric is used to classify applications according to their power efficiency levels.
• We utilize density based clustering algorithm (DBSCAN) to determine application groups based on their power efficiency classes.
Application ClassificationApplication Classification
Case 2VM1VM1
ESXi ESXi
VM0VM0
VM1
ESXi ESXi
VM0
Benchmarks
Case 1
VM Configuration
15
Computing in Heterogeneous, Autonomous 'N' Goal-oriented Environments
Reconfiguring Resource AllocationsReconfiguring Resource Allocations• CPU hot-plugging:
Adding/removing vCPUs during runtime Cons: Removing vCPU is not supported in some OSes
• Resource Allocation Adjustment: Allocating/limiting CPU resources for VMs Pros: Fine granularity (resource allocation unit is MHz)
• Both techniques have low overhead, less than 1%.
16
Resource Configuration Comparison
Computing in Heterogeneous, Autonomous 'N' Goal-oriented Environments
Reconfiguration Runtime BehaviorReconfiguration Runtime Behavior
• Resource allocation limits can be dynamically adjusted according to application classes.
• CPU allocation limits can be effectively reconfigured within a second.
17
Computing in Heterogeneous, Autonomous 'N' Goal-oriented Environments
ResultsResults
• Proposed approach improves throughput-per-watt by up to 25% and by 9% on average.
18
Computing in Heterogeneous, Autonomous 'N' Goal-oriented Environments
ResultsResults
• We generate 50 workload sets, each consists of randomly selected 10 PARSEC applications.
19
Set 2
3x canneal3x ferret2x bodytrack1x dedup1x vips
Set 1
4x blackscholes2x vips1x bodytrack1x freqmine1x streamcluster1x swaptions
Computing in Heterogeneous, Autonomous 'N' Goal-oriented Environments
ResultsResults
• We generate 50 workload sets, each consists of randomly selected 10 PARSEC applications.
• Proposed resource sharing technique improves the throughput-per-watt by 12% on average in comparison to application selection based co-scheduling techniques.
20
Computing in Heterogeneous, Autonomous 'N' Goal-oriented Environments
Conclusions & Future WorkConclusions & Future Work
• Consolidation is a powerful technique to improve the energy efficiency on data centers.
• Energy efficiency of parallel workloads varies significantly depending on application characteristics.
• Adaptive VM configuration for parallel workloads improves the energy efficiency by 12% on average over existing co-scheduling algorithms.
• Future research directions include:Investigating the effect of memory allocation decisions on energy efficiency;Utilizing application-level instrumentation to explore power/energy optimization opportunities;Expanding the application space.
21