Proactive Prediction Models for Web Application Resource Provisioning in the Cloud

_______________________________

Samuel A. Ajila & Bankole A. Akindele

Presentation Outline

Introduction to Problem Area Motivation Goals and Scope Contributions Related work Machine learning algorithms Implementation setup Evaluation metrics Selected results Conclusion

2

Introduction

Cloud computing refers to both the applications delivered as services over the Internet and the hardware and systems software in the data centers that provide those services (SaaS, PaaS and IaaS)

Challenges include: Data security threats, performance unpredictability and prompt (quick) resource scaling

Accurate Virtual machine (VM) resource provisioning: • under and over-provisioning

Current techniques like Control theory, Constraint programming and Machine learning

3

Motivation

VM instantiation (scaling) takes time and ranges from 5 to 12 minutes

Challenges that can result from the instantiation duration:• Possibility of Service Level Agreement (SLA) violations -

Cloud providers• Poor customer’s Quality of Experience (QoE) – Cloud client• Reputation loss – Cloud client/providers

Presently, monitoring metrics made available to clients are limited to CPU, Memory and Network utilization

The motivation is to “predict resource usage so that cloud providers can make adequate provisioning ahead of time”

To extend the monitoring metrics by including Response time and Throughput

4

Goals and Scope

Design and develop a cloud client prediction model for cloud resource provisioning in a Multitier web application environment

The model would be capable of forecasting future resource usage to enable timely VM provisioning

To achieve this goal, SVM, NN and LR learning techniques are analysed using the Java implementation of TPC-W (workload)

The scope of this work is limited to IaaS Prediction model is built around the web server tier. It is

possible to extend to other tiers

5

Contributions

Design and development of a cloud client prediction model that uses historical data to forecast future resource usage

The evaluation of the resource usage prediction capability of SVM, NN and LR using three benchmark workloads from TPC-W

The extension of the prediction model to include Throughput and Response time, thus providing wider and better scaling decision options for cloud clients

The comparison of the prediction capability of SVM, NN and LR models under random and steady traffic patterns

6

Related works

Table 1: Auto-scaling techniques

7

Technique Class CommentsThreshold/Rule based

Reactive Reacts to system changes but do not anticipate them

Control theory Reactive/Proactive Except where used with a proactive approach it suffers reactive issues

Queuing theory Reactive As complexity of system grows, the analytic formulas become difficult

Reinforcement learning

Reactive/Proactive Good but converging at optimal policy can be unfeasibly long (state –action pairs)

Time series analysis Proactive Good and promising

Machine learning algorithms

Linear Regression The linear regression model has the form: (1) is the input data ’s are the unknown parameters estimated from the training data.

Estimation is done using the least squares method. The coefficients are picked to minimize the residual (difference

between the actual and predicted value) sum of squares : is the actual value

8

Machine learning algorithms (cont’d)

Neural Network A two-stage regression or

classification model represented by a network diagram

Derived features are created from linear combinations of the input after which the target is modeled as a linear combination of

Like Linear regression, unknown parameters called weights are sought to make the model fit the training data well

9Figure 1 : Single hidden layer, feed forward neural network

Machine learning algorithms (cont’d)

Support Vector Regression The goal of SVR is to find a function that has at most (the precision by

which the function is to be approximated) deviation from the actual obtained target for all training data

Mathematically, Input data is mapped to higher dimensional feature space via the

kernel function, then a linear regression is performed Goal is to find optimal weights and threshold

10

Architecture

11Figure 2 Implementation architecture

Implementation setup (cont’d)

Time (minutes) 1-7 56-63 154-161 350-357 490-497 504-511

Shopping mix users

84 168 16 180 248 160

Browsing mix users

52 112 36 320 192 160

Ordering mix users

52 108 28 224 268 160

Total user Requests

188 388 80 724 708 480

12

Total length of experiment was about 10 hours Selected experimental workload mix

Evaluation metrics

13

Evaluation is done on the 60% training and 40% held out test dataset

The held out dataset is used to forecast to a maximum interval of 12 minutes (VM instantiation time reported by other authors)

Performance metrics and their calculations

Metric Calculation MAPE (Mean Absolute Percentage Error)

1𝑛 σ | 𝑎𝑖−𝑝𝑖|𝑎𝑖𝑛𝑖=1 where 𝑎𝑖 and 𝑝𝑖 are the actual and predicted values

respectively RMSE (Root Mean Square Error)

√σ ( 𝑎𝑖−𝑝𝑖) 2𝑛𝑖=1 𝑛

MAE (Mean Absolute Error) 1𝑛 σ |𝑝𝑖 − 𝑎𝑖|𝑛𝑖=1

PRED 25 No. of observations with relative error ≤ 25% / No. of observation

CPU Training & Testing Performance

14

CPU Utilization Training and Testing Performance Metric for LR, NN, and SVR

LR (Linear Regression) Model MAPE RMSE MAE PRED(25) LR Training 113.31 14.70 11.11 0.51 LR Testing 36.19 22.13 15.98 0.36 NN (Neural Network) Model MAPE RMSE MAE PRED(25) NN Training 105.63 14.08 9.48 0.59 NN Testing 50.46 31.08 19.82 0.34 SVR(Support Vector Regression) Model MAPE RMSE MAE PRED(25) SVR Training 107.80 15.48 10.09 0.64 SVR Testing 22.84 11.84 8.74 0.64

Selected results (cont’d)

15

CPU utilization test performance metric

Model MAPE RMSE MAE PRED(25)LR 36.19 22.13 15.98 0.36NN 50.46 31.08 19.82 0.34SVR 22.84 11.84 8.74 0.64

Figure 5 CPU Utilization Actual and Predicted test model results

Selected results (cont’d)

16

Throughput test performance metric

Model MAPE RMSE MAE PRED(25)LR 24.62 3.72 2.87 0.63NN 38.90 6.12 4.46 0.47SVR 22.07 3.22 2.41 0.67

Figure 6 Throughput Actual and Predicted test model results

Selected results (cont’d)

17

Response time test performance metric

Model MAPE RMSE MAE PRED(25)LR 12.35 1.39 1.11 0.91NN 17.84 2.02 1.64 0.75SVR 9.92 1.21 0.87 0.93

Figure 7 Response time Actual and Predicted test model results

CPU - Comparison of Prediction Models

18

Throughput – Comparison of Prediction Models

19

Sensitivity Measurement Using Little’s Law

20

Data Consistency Measurement Using Little’s Law

Time (minute) 1-7 56-63 154-161

350-357

490-497

498-503

504-511

Average total user requests

27 (188/7)

55 (388/7)

11 (80/7)

103 (724/7)

101 (708/7)

95 (664/7)

69 (480/7)

Average Throughput (Requests/second)

5.29 10.52 2.47 15.92 10.00 12.23 13.27

Average time spent (seconds)

5.10 5.23 4.45 6.47 10.01 7.77 5.2

Measured time spent 4.66 4.92 9.92 7.53 9.06 9.01 9.45 Time variance (%) 9.44 6.31 55.14 14.08 10.49 13.76 44.97

Conclusion

SVR displayed superior prediction accuracy over both LR and NN in a typically nonlinear, not defined a-priori workload by:

In the CPU utilization prediction model, SVR outperformed LR and NN by 58% and 120% respectively

For the Throughput prediction model, SVR again outperformed LR and NN by 12% and 76% respectively; and finally,

The Response time prediction model saw SVR outperforming LR and NN by 26% and 80% respectively.

Based on this experimental results SVR may be accepted as the best prediction model in a nonlinear, not defined a-priori system

21

Future works

SVR and other machine learning algorithms are good for forecasting, however• training and retraining is a challenge• Parameter selection is still empirical

Combination of SVR and other predicting techniques may mitigate this challenge

Other future direction include• Inclusion of database tier for a more robust scaling

decision• Resource prediction on other non-web application

workload

22

Questions

23

Thank You