Multifaceted Resource Management in Virtualized Providers

Íñigo Goiri

PhD DefenseJune 14th, 2011

Advisors:Jordi Guitart and Jordi Torres

2

Motivation

Book Store Internet

Companies offer their servicesover the Internet

3

Service Providers over the Internet

Book Store Internet

Number of usersincreases

Provider requires more resourcesLoad increases

4

Service Providers over the Internet

Book Store Internet

Number of usersdecreases

Provider requires fewer resources

5

Virtualization on Service Providers

Book Store Internet

Database

Web ServerApplication

ServerDatabaseApplication

ServerWeb Server

Database

Web Server

DDBBApplication

Server WebServerApplication

ServerDDBB DDBBWebServer WebServerEncapsulate tasks in

Virtual Machines

VM VM VM VM VM VM VM VM

Load is unbalanced

6

Providing Virtual Machines

Book Store InternetVirtualizedProvider

Desktop

WebServer

HPC

VM VM VM VM VM VM VM VM

VirtualizedProvider

Provider offers its idleInfrastructure as a Service

(IaaS)

We can also offerapplications encapsulated in VMs

7

Managing VirtualizedProvider’s Resources

Balanced loadVMs get enough resources

Energy consumption is high

8

Managing VirtualizedProvider’s Resources

Unbalanced loadSome VMs don’t get enough resources

Energy consumption is lower

9

Managing VirtualizedProvider’s Resources

ChallengeEfficiently Manage Virtualized Provider’s Resources

Maximize Provider’s Profit

Every VM gets enough resourcesEnergy consumption is lower

10

Contents

• Motivation• Multifaceted Scheduling– Cost-Benefit Model– Scheduling Policy– Evaluation

• Multiprovider Scheduling– Capacity Planning– Scheduling– Evaluation

• Conclusion

11

Multifaceted Scheduling

• Cost-benefit model– Multiple facets to consider– Aggregate facets into costs– Consider impact of facets on other facets

• Scheduling policy– Maximize provider profit

• Evaluation

12

Multiple Facets to Consider

1. Service Level Agreement2. Virtualization Management3. Energy Consumption4. Infrastructure Cost

13

1. Service Level Agreement

• Contract between user and provider• User pays for resources– Pay as you go: SLARevenue(t(VM))

• If the provider does not provide the QoS– SLA Penalty: SLAPenalty(VM)

VirtualizedProvider

Revenue

Penalties

14

1. Service Level AgreementSupported Applications

• Batch– Deadlines

• Example– HPC jobs

• Service– Uptime– Performance

• Example– Web Servers

Task 1

Task 3

Time Time

Task 2

Resp

onse

Tim

e

15

1. Service Level AgreementResource Heterogeneity

• Xeon processor– High energy consumption– High performance

• Atom processor– Low energy consumption– Low performance

Task A

Task B

Task C

Task A

Task B

Task C

Max

Wh0 Max

Wh0

16

1. Service Level Agreement

• Estimate SLA Penalties– Actions may imply future violations– SLAPenalty’(Host, VM)

• Factors that might provoke violations– Runtime overhead– Slow host– High host utilization

• Other facets evaluate penalty estimations

17

Time

2. Virtualization Management

• Overhead to manage Virtual Machines– Start VMs– Migrate VMs between nodes

Time

18

Off

2. Virtualization Management

• Overhead to manage Virtual Machines– Start VMs– Migrate VMs between nodes

Time

Off

Mig

ratio

nM

igra

tion

19

2. Virtualization Management

• Overhead implies– Extra time to the VM– Extra load to the Host

• It can imply SLA penalties– Violate deadline– No enough resources to provide performance

• Estimate SLA penalty for every action(Time, Load) → SLAPenalty’(Host, VM) → €

20

3. Energy ConsumptionEnergy vs. SLA

• Low Consolidation– Consume a lot of energy– Fulfill SLA

• High Consolidation– Save energy– Violate SLA

21

3. Energy Consumption vs. SLABatch Type

• Low Consolidation

• High Consolidation

Off

Time

TimeTime

Max

Wh0

Max

Wh0 Max

Wh0

Max

Wh0

22

3. Energy Consumption vs. SLAService Type

• Low Consolidation

• High Consolidation

Off

Time

Time

Time

Max

Wh0

Max

Wh0 Max

Wh0

Max

Wh0

23

3. Energy Consumption• Energy cost

Wh → €• SLA penalties

Host Utilization→ SLAPenalty’(Host, VM) → €

24

4. Infrastructure Cost

• Provider owns infrastructure– Servers– Air conditioners– Racks…

• Capital Expense (CAPEX)• Cost is amortized over time– Provider has already paid for the hardware

€/Period

25

Multiple Facets to ConsiderCalculate profit of VM at Host

1. Service Level Agreement:SLARevenue(VM) +€

2. Virtualization Management: Time: SLAPenalty’(VM) -€

Load: SLAPenalty’(VMs in Host) due to VM -€

3. Energy Consumption: Energy consumed by VM at Host -€

SLA Penalty’(VMs in Host) -€

4. Infrastructure Cost:Cost of Host running VM -€

Total Profit€

26

Scheduling Policy• Decide best VM placement• Maximize provider profit– Hill Climbing (Greedy)

• When to schedule?– System changes– Periodically

• Model Virtualized Provider as a matrix– VM x Host cells– Each is profit of placing VM in Host– Use cost-benefit model

27

OFF

Queue:

Scheduling Policy

1 2

3 4 5

6 7

A

B

C

28

Scheduling Policy

1.5€ 0.2€

-0.5€

1.2€

-0.1€0.2€

1.3€

1.2€0.7€

-1.5€-∞

-∞-∞ -∞

-∞

-0.5€

-0.3€

0.1€ 0.3€

-0.7€ 0.2€

1.5€ 0.2€

-0.5€

1.2€

-0.1€0.7€

0.2€

-0.2€0.7€

1.2€-∞

-∞-∞ -∞

-∞

-0.5€

-0.3€

0.1€ 0.3€

-0.7€ 0.2€

Calculate cost of allocatingEvery VM at every Host

1.5€ 0.2€

-0.5€

1.2€

-0.1€0.2€

1.3€

1.2€0.7€

-1.5€-∞

-∞-∞ -∞

-∞

-0.5€

-0.3€

0.1€ 0.3€

-0.7€ 0.2€

Recalculate cost of allocatingEvery VM at every Host

Schedule VM in maximumProfit placement

When cost is minimizedDispatch VMs

1 2 3 4 5 6 7

A

B

C

After multiple iterations…

29

Scheduling Policy

OFF

Queue:

1 2

3 4 5

6 7

A

B

C

30

EvaluationMultifaceted Scheduling

• One week heterogeneous workload– Batch: Grid5000– Service: SPECWeb2005

• SLA metrics– Batch: Deadline (Added +20% Base Runtime)– Service: Performance (Response Time)

• Provider with 65 nodes– Enough to satisfy workload peaks

31

1. Backfilling + Migration– Backfill VMs– Migrate to consolidate

2. Perfect SLA– Analytical (NP)– Perfect SLA fulfillment– Optimal energy consumption

3. Our proposal– Backfilling + Migration– Aggregate multiple facets– Uses cost-benefit model– Maximize profit

13

2

OFF

1

2

EvaluationScheduling Policies

3

OFF

4

4

32

Evaluation

Energy consumption SLA fulfillment

+ ← Consolidation → - + ← Consolidation → -

33

Evaluation

Energy Saving SLA Fulfillment Profit Increase0

20

40

60

80

100

120Backfilling + MigrationPerfect SLAOur proposal

%

34

Evaluation

• Power consumption over time

35

Multifaceted SchedulingLimitations

1. Service Level Agreement2. Virtualization Management3. Energy Consumption4. Infrastructure Cost– Fixed costs

36

Contents

• Motivation• Multifaceted Scheduling– Cost-Benefit Model– Scheduling Policy– Evaluation

• Multiprovider Scheduling– Capacity Planning– Scheduling– Evaluation

• Conclusion

37

Multiprovider SchedulingOutsourcing

• Infrastructure cost– Capital Expenses (CAPEX)

Solution:CAPEX → OPEX

External Provider

38

Multiprovider SchedulingOutsourcing

• Reduce provider infrastructure (CAPEX)• Multifaceted Scheduling + Outsourcing– Add outsourcing cost (OPEX)– Slower VM creation– Limited VM management

External Provider

39

EvaluationOutsourcing

• Add outsourcing to “Multifaceted Scheduling”• Same environment– Reduce local resources: 65 → 20 → 0 nodes– 20 nodes is enough to provide the average

• External provider: EC2 US– 0.085 €/hour– 5 minutes to start a VM

40

EvaluationOutsourcing

Energy Saving SLA Fulfillment Profit Increase

-60

-40

-20

0

20

40

60

80

10065 Nodes65 Nodes + Outsourcing20 Nodes20 Nodes + Outsourcing0 Nodes + Outsourcing

%

41

Multiprovider SchedulingFederation

• How many local resources?– Optimal number of resources– Change capacity planning

• How to schedule?– New actuators– New trade-offs

• Characterize provider profitability– Cost-benefit model

42

Federated Provider Model

• Multidimensional problem• Evaluate provider profile– Provider capabilities– Expected workload– VM pricing

• Evaluate costs– CAPEX: Infrastructure– OPEX: Energy, Cooling,…

43

Multiprovider SchedulingCharacterize federation

Leverage federated provider model for:Phase 0. Capacity planning– Provider building and setup process– Decide optimal number of nodes

Phase 1. Scheduling– Online process– Decide actions to take

44

Phase 0. Capacity Planning• Load is variable over time• If infrastructure costs are fixed– Underprovision: Cannot support peaks– Overprovision: Underutilized resources

45

Phase 0. Capacity Planning Underprovision

• Solution: Outsourcing• Send peaks to other providers– Reduce provider infrastructure costs– Pay for using external resources

External Provider

46

Phase 0. Capacity Planning Overprovision

• Solution: Insourcing• Offer idle resources to other providers– Cheaper price– Enough resources to support peaks

Offer to other Providers

47

Phase 1. Scheduling

• Analyze provider profitability– Decide best actions to perform

• New actuators to consider– Outsourcing– Insourcing

• Old actuators change– Turn on/off nodes vs. Insourcing

48

Evaluation Characterization

• Provider profitability (darker is better)– Offering 80% of the idle resources– Amazon EC2 pricing

No Insourcing Insourcing

49

Evaluation Phase 0. Capacity Planning

• ISP workload over a week• Get optimal capacity– Leverage provider model– Revenue > Costs

Overprovision

Undeprovision

100 nodes

50

EvaluationPhase 1. Scheduling

51

EvaluationPhase 1. Scheduling

Multifaceted Scheduling

Multifaceted Scheduling + Outsourcing

Multifaceted Scheduling+ Insourcing

Multifaceted Scheduling +

Outsourcing + Insourcing

0

50

100

150

200

250Profit

%

52

Contents

• Motivation• Multifaceted Scheduling– Cost-Benefit Model– Scheduling Policy– Evaluation

• Multiprovider Scheduling– Capacity Planning– Scheduling– Evaluation

• Conclusion

53

Conclusion

• Multifaceted Scheduling– Efficient Management of Virtualized Providers– Consider multiple facets– Handle effect of multiple facets– Every facet has a significant effect

• Multiprovider Scheduling– Add new actuators– New policies required– Tradeoff between Insourcing and Turn on/off

54

Future Work

• Support higher-level SLA metrics• Energy efficient datacenter– Reduce PUE, free-cooling…– Energy-aware scheduling: Green availability– Efficient datacenter placement

• Multiprovider– Apply to real-world scenarios– Enhance provider fault tolerance

55

Summary

• Developed from a multi-level perspective:– Enhancing Virtualization Fabrics– Managing a Virtualized Host– Multifaceted Scheduling– Multiprovider Scheduling

56

Publications• Enhancing Virtualization Fabrics

– [NCA09] VM Creation.– [PDP09] VM Migration.– [NOMS10] VM Checkpointing.

• Managing a Virtualized Host– [CPE09] SLA-driven resource management.– [GECON10] SLA metric for Virtualized Environments.

• Multifaceted Scheduling– [Cluster10] [FGCS] Multifaceted Scheduling.

• Multiprovider Scheduling– [Grid10] Outsourcing into Multifaceted Scheduling.– [Cloud10] Characterizing Federation.

57

Multifaceted Resource Management on Virtualized Providers

Íñigo Goiri

Advisors:Jordi Guitart and Jordi Torres

58

Publications

Enhancing Virtualization Fabrics[NCA09] Í. Goiri, F. Julià, J. Ejarque, M. De Palol, R. M. Badia, J. Guitart, and J.

Torres. Introducing Virtual Execution Environments for Application Lifecycle Management and SLA-Driven Resource Distribution within Service Providers. Proceedings of the 8th IEEE International Symposium on Network Computing and Applications.

[PDP09] Í. Goiri, F. Julià, and J. Guitart. Efficient Data Management Support for Virtualized Service Providers. Proceedings of the 17th Conference on Parallel, Distributed and Network-based Processing.

[NOMS10] Í. Goiri, F. Julià, J. Guitart, and J. Torres. Checkpoint-based Fault-tolerant Infrastructure for Virtualized Service Providers. Proceedings of the 12th IEEE/IFIP Network Operations and Management Symposium.

59

Publications

Managing a Virtualized Host [CPE09] J. Ejarque, M. de Palol, Í. Goiri, F. Julià, J. Guitart, R. Badia, and J.

Torres. Exploiting semantics and virtualization for SLA-driven resource allocation in service providers. Concurrency and Computation: Practice and Experience, 22(5): pages 541–572, 2010.

[GECON10] Í. Goiri, F. Julià, J. O. Fitó, M. Macías and J. Guitart. Resource-level QoS Metric for CPU-based Guarantees in Cloud Providers. In Proceedings of the 7th International Workshop on Economics of Grids, Clouds, Systems, and Services.

60

Publications

Multifaceted Scheduling[Cluster10] Í. Goiri, F. Julià, R. Nou, J. Berral, J. Guitart, and J. Torres. Energy-

aware Scheduling in Virtualized Datacenters. Proceedings of the 12th IEEE International Conference on Cluster Computing.

[FGCS] Í. Goiri, J. Berral, J. O. Fitó, F. Julià, R. Nou, J. Guitart, R. Gavaldà and J. Torres. Energy-efficient and Multifaceted Resource Management for Profit-driven Virtualized Datacenters. Review process in Future Generation of Computer Systems.

61

Publications

Multiprovider Scheduling[Grid10] Í. Goiri, J. O. Fitó, F. Julià, R. Nou, J. Berral, J. Guitart, and J. Torres.

Multifaceted Resource Management for Dealing with Heterogeneous Workloads in Virtualized Datacenters. Proceedings of the 11th ACM/IEEE International Conference on Grid Computing.

[Cloud10] Í. Goiri, J. Guitart, and J. Torres. Characterizing Cloud Federation for Enhancing Providers’ Profit. Proceedings of the 3rd International conference on Cloud Computing.

65

EvaluationOutsourcing

Profit Increase

-60

-40

-20

0

20

40

60

80

100 Backfilling+Migration

65 Nodes

65 Nodes + Outsourcing

33 Nodes

33 Nodes+Outsourcing

20 Nodes

20 Nodes + Outsourcing

0 Nodes + Outsourcing

%