Cloud Services Innovation Platform (CSIP)

Dynamic Scaling for Service Oriented Applications: Implications of Virtual Machine Placement on Infrastructure-as-a-Service CloudsWes Lloyd, Shrideep Pallickara, Olaf David, Mazdak Arabi, Ken Rojas

March 13, 2014

Colorado State University, Fort Collins, Colorado USAIC2E 2014: IEEE International Conference on Cloud Engineering

1OutlineBackgroundResearch ProblemResearch QuestionsExperimental SetupExperimental ResultsConclusions2March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS Clouds2Background3Dynamic Scaling forService Oriented Applications

4WHENHot Spot DetectionVM Launch LatencyFuture Load PredictionPre-provisioning

WHAT

WHEREMarch 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsDynamic Scaling forService Oriented Applications

5WHATSizeVertical ScalingQuantityHorizontal ScalingLoad BalancingVMVMVMVMVMVMVMVMVMVMVMVMVMVMVMVMVM

March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsDynamic Scaling forService Oriented Applications

6WHEREPhysical HostPhysical HostPhysical HostPhysical HostVMVMVMVMVMVMVMVMVMVMVMVMVMVMVMVMVMVMVMVMLaunch RequestsVM SchedulingMarch 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsResearch Problem7IaaS Cloud: VM PlacementIn practice there are two predominant VM placement schemesGreedy fill first: consolidateRound-robin spread first: load balanceCommonly provided byEucalyptus, OpenStack, OpenNebula, Apache CloudStackWhat are the performance implications for dynamic scaling?For SOAs, are they sufficient? 8Physical HostPhysical HostPhysical HostVMVMVMVMVMVMMarch 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsResearch Questions9Research QuestionsWhat performance implications result from VM placement location when dynamically scaling service oriented applications?

How important is VM placement for scaling in response to increasing service demand?

How do resource costs (# of VMs) vary when dynamically scaling service oriented applications as a result of VM placement location?

10RQ1:

RQ2:March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsExperimental Setup11

ModelingServicesUSDA-NRCS - Soil erosion modelsRUSLE2: soil loss from rainfall and runoffWEPS: soil loss from wind12

OMS3RUSLE2/WEPSPOSTGRESQLPOSTGIS~6 million shapes340k XML filesMarch 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsVM-Scaler13VM-ScalerInfrastructure Mgmt RequestsApplication Service RequestProxy ServicesCloud Mgmt ServicesElastic Compute Cloud API Physical HostPhysical HostPhysical HostPhysical HostPhysical HostPhysical HostVMVMVMVMVMVMVMVMVMVMVMVMVMVM

Physical HostPhysical HostPhysical HostPhysical HostPhysical HostPhysical HostVMVMVMVMVMVMVMVMVMVMVMVMVMVM

futureLeast-Busy VM SchedulerMarch 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsLeast-Busy VM PlacementRU-sensors collect Virtual/Physical machine data @ 15s/intervalsBusy-Metric used to calculate aggregate load at each physical machineFlexible metric designObjective not to design perfect metric / VM scheduler

Resource Utilization Data14Network- Network bytes sent (NBR)- Network bytes received (NBS)CPU- Total CPU time weighted 2X

Disk- Disk sector reads (DSR)- Disk sector writes (DSW)Virtualization- Total VM count per hostMarch 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsDynamic Scaling TestsShared cluster load simulation

Modeling workloads

15

Rusle2:7,000 runs/test

WEPS:300 runs/test Initial load before scaling Simulates multi-tenant cloud environments

March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsExperimental Results16Application Performance Improvement vs. Round-Robin VM Placement17

Normalized %Performance ImprovementStatistical significance Average Performance Improvement:

~16.1% RUSLE2~11.6% WEPS_~14% aggregateMarch 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsResource Cost Savingsvs. Round-Robin VM Placement

18Resource Cost Savings% Fewer VMsAverage Savings:

~2.7% fewer VMs~14.7 fewer CPU coresMarch 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsConclusions19ConclusionsLeast-Busy VM placement enabledperformance improvements up to:29% (RUSLE2), 19% (WEPS) When dynamically scaling In the presence of a shared load

These performance improvements were realized using slightly fewer (2.7 %) virtual machines.

20RQ1:

RQ2:

Abstraction of physical hardware using IaaS clouds leads to the simplistic view:(1) Resources are homogeneous (2) Scaling infinitely provides linear performance increases

Our results demonstrate:(1) The importance of fine grained resource management for supporting infrastructure elasticity (2) Where hardware is not infiniteMarch 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsQuestions21

Extra Slides22

Gaps in Related WorkPrior work investigates:WHEN to scale hot spot detectionWHAT to scale size and number of VMsWHERE to scaleTask/job schedulingVM placement/migration across nodesNo studies have investigated implications of VM placement for dynamic scaling application infrastructureMarch 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsAverage Model Execution Time

24March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsLeastBusy VM PlacementLeastBusy VM placementBusy Metric rates resource utilization of physical hosts and virtual machinesResource Identifies Parallel launch: launch N VMs per scaling eventDouble-schedule MAX BusyMetric thresholdOnly if min distance (BusyMetric) to nearest neighbor

March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS Clouds

Eucalyptus 3 Private Cloud(9) Sun X6270 blade serversDual Intel Xeon 4-core 2.8 GHz CPUs24 GB ram, 146 GB 15k rpm HDDsCentOS 5.6 x86_64 (host OS)Ubuntu 9.10 x86_64 (guest OS)Eucalytpus 3.1Amazon EC2 API support8 Nodes (NC), 1 Cloud Controller (CLC, CC, SC)Managed mode networking with private VLANsXEN hypervisor v 4.1, paravirtualization26March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsRUSLE2 ComponentsVirtual MachineDescriptionMModelApache Tomcat 6.0.20, Wine 1.0.1, RUSLE2 Model, Object Modeling System (OMS 3.0)DDatabasePostgresql-8.4, and PostGIS 1.4.0-2. soil data: 1.7 million shapes, 167 million pointsmanagement data: 98 shapes, 489k pointsclimate data: 31k shapes, 3 million points4.6 GB for the state of TNFFile Servernginx http server 0.7.62 57,185 XML files consisting of 305MB.LLoggerCodebeamer 5.5 running 32-bit ApacheTomcat 6.0Custom REST/JSON logging service as wrapper. 27March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS Clouds27SC2M DF LSC4M D

FLSC7LMD

FSC3M D

F LSC5MDF LSC6MD FLSC8MD

F LSC9MD LFSC10M FD LSC11M FDLSC12M LD FSC13M LDFSC14M DLFSC15M LFDSC1M DF L28(15) Tested Component Deployments

Each VM deployed to separate physical machinesAll components installed on composite imageScript enabled/disabled components to achieve configsMarch 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsTested Resource Utilization Variablesc29Network- Network bytes sent (nbr)- Network bytes received (nbs)CPUCPU time

Disk- Disk sector reads (dsr)- Disk sector reads completed (dsreads)March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsAmazon Web Services: White Paper on Application DeploymentAmazon white paper suggests:bundling the logical construct of a component into an Amazon Machine Image so that it can be deployed more often.J. Varia, Architecting for the Cloud: Best Practices, Amazon Web Services White Paper, 2010, https://jineshvaria.s3.amazonaws.com/public/ cloudbestpractices-jvaria.pdfTo support application scaling

30March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsService Isolation Advantages 31tomcat7nginxPostgreSQLMemcacheDBMySQLMongoDBMongoDBMongoDBMongoDBMongoDBMongoDBMongoDBSCALEEnablesHorizontal scalingFault tolerance

March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsService Isolation Overhead32Isolation requiresSeparate operating system instancesMore network traffic

tomcat7nginxPostgreSQLMarch 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsProvisioning VariationProblems & Challenges 33VMPhysical HostPhysical HostPhysical HostPhysical HostPhysical HostPhysical HostVMVMVMAmbiguousMappingVMVMVMVMVMVMVMVMVMVMVMVMVMVMRequest(s) to launch VMsVMs ReservePM Memory BlocksVMs Share PMCPU / Disk / Network

PERFORMANCEMarch 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsApplicationServersLoad BalancerLoad BalancerService RequestsnoSQL data storesrDBMSdistributed cacheInfrastructure ManagementProblems & Challenges 34Scale Services

Tune Application Parameters

Tune Virtualization ParametersMarch 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsApplication Profiling VariablesPredictive Power35

March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsApplication Deployment ChallengesVM image compositionService isolation vs. scalabilityResource contention among componentsProvisioning variation Across physical hardware

36

March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsResource Utilization VariablesStatisticDescription P/VCPU timeCPU time in msP/Vcpu usrCPU time in user mode in msP/Vcpu krnCPU time in kernel mode in msP/Vcpu_idleCPU idle time in msP/VcontextswNumber of context switchesP/Vcpu_io_waitCPU time waiting for I/O to completeP/Vcpu_sint_timeCPU time servicing soft interruptsVdsrDisk sector reads (1 sector = 512 bytes)VdsreadsNumber of completed disk readsVdrmNumber of adjacent disk reads mergedVreadtimeTime in ms spent reading from diskVdswDisk sector writes (1 sector = 512 bytes)VdswritesNumber of completed disk writesVdwmNumber of adjacent disk writes mergedVwritetimeTime in ms spent writing to diskP/VnbrNetwork bytes sentP/VnbsNetwork bytes receivedP/VloadavgAvg # of running processes in last 60 sec37March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsExperimental DataScript captured resource utilization statsVirtual machinesPhysical MachinesTraining data: first complete run20 different ensembles of 100 model runs 15 component configurations30,000 model runs Test data: second complete run30,000 model runs38March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS Cloudsn=# components; k=# components per set

Permutations

Combinations

But neither describes partitions of a set!Application Deployments

39

March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS Clouds

Bells Number 40ModelComponentDeploymentn = #componentsApplication StackVM deployments# of ConfigurationsDatabaseFile ServerLog Server. . .k= #configsconfig 1MDFLconfig 2MFLconfig nMLFD1 VM : 1..n componentsnk4155526203787784,140921,147n. . .DNumber of ways a set of n elements can be partitioned into non-empty subsets

March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsXEN Mbound vs Dbound Performance Same Ensemble41

March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsXEN 10 GB VMs42

March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsKVM Mbound vs Dbound PerformanceSame Ensemble

43March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsKVM 10GB PerformanceSame Ensemble

44March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsKVM 10 GB Performance ChangeSame Ensemble

45March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsKVM Performance ComparisonDifferent Ensembles

46March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsKVM Performance Change From Service Isolation

47March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsService Configuration TestingBig VMsAll application services installed on single VMScripts enable/disable services to achieve configurations for testingEach VM deployed on separate hostProvisioning Variation (PV) TestingKVM used15 total service configurations46 possible deployments48March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsPV: Performance Difference vs. Physical Isolation

49March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsService Configuration Testing - 2Big VMs used in physical isolation were effective at identifying fastest service configurationsFastest configurations isolate L service on separate physical host; and VMsSome provisioning variations fasterOther SC provisioning variations remained slowSC4A-D, SC9C-D Only SCs w/ avg ensemble performance < 30 seconds50March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS Clouds

Can Resource Utilization Statistics

51

Model Application Performance? March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsRQ1 Which are the best predictors? PM Variables52

CPUNetwork I/OMarch 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsRQ2 How should VM resource utilization data be used by performance models?Combination: RUdata=RUM+RUD+RUF+RUL

Used Individually: RUdata={RUM; RUD; RUF; RUL;}

53March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS Clouds

RQ2 How should VM resource utilization data be used by performance models?54D-bound separateD-bound combinedM-bound separateM-bound combinedTreating VM data separately for D-bound was better !RUM or RUMDFLfor M-bound was better !Note the larger RMSEfor D-bound RUMDFL!March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsRQ3 Which modeling techniques were best?Multiple Linear Regression (MLR)Stepwise Multiple Linear Regression (MLR-step)Multivariate Adaptive Regression Splines (MARS)Artificial Neural Network (ANNs)55March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS Clouds

RQ3 Which modeling techniques were best?56MultipleLinearRegressionStepwiseMLRMultivariateAdaptiveRegresionSplinesArtificalNeuralNetworkRUMDFL data used tocompare models.

Had high RMSEtest error for D-Bound (32% avg)Model performance did not vary much

Best vs. Worst

D-BoundM-Bound .11% RMSEtrain.08% .89% RMSEtest.08% .40 rank err.66

March 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS CloudsResource Utilization Statisticsc57CPU- CPU time- CPU time in user mode- CPU time in kernel mode- CPU idle time- # of context switches- CPU time waiting for I/O- CPU time serving soft interrupts- Load average (# proc / 60 secs)

Disk- Disk sector reads- Disk sector reads completed- Merged adjacent disk reads- Time spent reading from disk- Disk sector writes- Disk sector writes completed- Merged adjacent disk writes- Time spent writing to diskNetwork- Network bytes sent- Network bytes receivedPMVMVMPMVMVMVMMarch 13, 2014 IEEE IC2E 2014 Dynamic Scaling for Service Oriented Applications: Implications of VM Placement on IaaS Clouds