DAY 2

Agenda for Today

• Introduce the workload characterization problem.

• Discuss a simple example of characterizing the workload for an intranet.

– What is workload characterization?

– What workload we want to characterize?

• Present a workload characterization methodology.

References

• Thyagaraj Thanalapati, Sivarama P. Dandamudi: An Efficient Adaptive Scheduling Scheme for Distributed Multicomputers. IEEE Trans. Parallel Distrib. Syst. 12(7): 758-768 (2001)

• A. Iosup and D.H.J. Epema, Grid Computing Workloads, IEEE Internet Computing 15(2): 19-26 (2011)

• Feitelson DG. Workload modeling for computer systems perfMemory ormance evaluation. Cambridge University Press; 2015.

Introduction • Workloads appear in many contexts and therefore

have many different types.

• Different workloads have a significant effect on performance.

• Different workloads have different features

• Ensuring that systems meet desired performance goals without excessive (and expensive) over-provisioning is of great importance.

• Therefor, reliable workload models are needed.

What is workload? • The workload of a system can be defined as the set

of all inputs that the system receives from its environment during any given period of time.

A variety of jobs with different characteristics are submitted to the cloud by different users.

A static workload is one in which a certain amount of work is given, and when it is done that is it.

A dynamic workload, in contrast, is one in which work continues to arrive all the time; it is never “done”.

Need to capture

HP Engineering

Sample Cloud Computing Applications

Demand Variability

Demand Volume

Taxes, @Home

Low

Low

High

High

Web Server Space Survey

Comet Detected

Sky Survey

Pharma Research

Social Gaming

Online Gaming

Social Networking

Big Data Analytics

SW Dev/Test

Office Tools

Tsunami Prediction

Epidemic Simulation Exp.

Research

From Iosup

Batch workloads

• Typically tend to process huge volumes of data.

– Example: An application that produce monthly bills for utilities (e.g., power, water, mobile phone bills, etc.)

– Require considerable compute and storage resources.

– Batch workloads are rarely time sensitive and can be scheduled when few real-time tasks are running.

High-performance workloads • High-performance workloads typically require

significant compute capabilities.

– next day weather prediction application

• High-throughput workloads Examples of these applications include

– different patient data in large scale biomedical trials

– different parts of a genome or protein sequence in bioinformatics applications

– different random numbers in a simulations based on Monte Carlo methods

– different model parameters in ensemble simulations or explorations of parameter spaces

Bag-of-Tasks

• Bag-of-tasks refers to parallel jobs with many independent tasks

• The tasks can be executed out of the submission order.

• The following is not bag of task

T1

T2

T3

T5

T4

T7

T8

T6

Example of BoT Applications

• Parameter sweeps – Comprehensive, possibly exhaustive investigation of a model

– Very useful in engineering and simulation-based science

• Monte Carlo simulations – Simulation with random elements: fixed time yet limited inaccuracy

– Very useful in engineering and simulation-based science

• Many other types of batch processing – Periodic computation, Cycle scavenging

– Very useful to automate operations and reduce waste

Workflow Applications

• Workflow is a set of jobs with precedence (think Direct Acyclic Graph)

• Complex applications – Complex filtering of data

– Complex analysis of instrument measurements

Maslina Abdul Aziz, Jemal H. Abawajy, Tutut Herawan: Layered workflow scheduling algorithm. FUZZ-IEEE 2015: 1-7

ADDRESSING BIG DATA ENERGY CONSUMPTION

Collection Station

Emergency

Monitoring Centre

Medical staff

Observation StationData Center

Fog Node

Push

Pull

Push

Pull

Push

Pull

Physical Machines

Sensor

Data

Computational

resources

Virtual (Sensor) Machines

VM Manager

Service Scheduler

Analytics Engine Visualization Enigne

Acc

ess

Co

ntro

l

Dat

a Sh

arin

g P

olic

y

Application 1 Application N

Medical Application Services

E-Health

Data

Middleware

Pu

bli

sh

Alert

PatientIoT+Fog

Jemal H. Abawajy, Mohammad Mehedi Hassan, Federated Internet of Things and Cloud Computing Pervasive Patient Health Monitoring System. IEEE Communications Magazine 55(1): 48-53 (2017)

Sara Ghanavati, Jemal Abawajy and Davood Izadi (2017), The Internet of Things: Foundation for Smart Cities, eHealth, and Ubiquitous Computing, Ricardo Armentano, Robin Singh Bhadoria, Parag Chatterjee, Ganesh Chandra Deka (eds.), CRC Press.

• Integrated Fog computing (FC) and Internet of Everything (IoE)

Workflow Applications • Workflow applications are jobs with precedence (

represented by a directed acyclic task graph). • 75%+ WFs are sized 40 jobs or less, 95% are sized 200

jobs or less (Ostermann et al., 2008)

T1

T2

T3

T5

T4

T7

T8

T6

Structure of divide-and-conquer programs

Maslina Abdul Aziz, Jemal H. Abawajy, Tutut Herawan: Layered workflow scheduling algorithm. FUZZ-IEEE 2015: 1-7

• Example applications – Complex filtering of data

– Complex analysis of instrument measurements

Web Workload

• Example of execution of HTTP requests (sec)

• Characteristics of Web workloads: – burstiness

– heavy-tailed distributions

Request No. CPU time

I/O time

Elapsed time

1 0.0095 0.04 0.071

2 0.0130 0.11 0.145

3 0.0155 0.12 0.156

4 0.0088 0.04 0.065

5 0.0111 0.09 0.114

6 0.0171 0.14 0.163

7 0.2170 1.20 4.380

8 0.0129 0.12 0.151

9 0.0091 0.05 0.063

10 0.0170 0.14 0.189

Average 0.0331 0.205 0.550

Performance Analysis

• There are three common methodologies to evaluate performance

– Use actual workload

– Use traced workload directly to drive a simulation.

– Create a model from the trace and use the model for either analysis or simulation.

Trace Data

• Trace is recorded data as a trace, or log, of workload-related events that happened in a certain system.

• Example

– The Failure Trace Archive (FTA) is a centralized public repository of availability traces of parallel and distributed systems, and tools for their analysis.

– The purpose of this archive is to facilitate the design, validation, and comparison of fault-tolerant models and algorithms.

Trace-data simulation • Trace-driven simulations are common in parallel and

distributed computing

• Shortcomings

– Collecting data may be inconvenient or costly, and instrumentation may introduce overhead.

– Data collection is inherently limited as the conditions under which data is gathered may restrict one to a small sample of the space of interest.

– For example, given a server with 128 nodes, it is not possible to collect data about systems with 64 or 256 nodes.

• We need models to transcend these limitations.

Workload Modeling • Workload modeling is used to generate synthetic

workloads based on real-life job execution observations.

• The goal is typically to be able to create workloads that can be used in performance evaluation studies

Workload Submission Workload

description (may include QoS)

Workload Generation

J1 J1 Jn

Analysis

SLA

Consolidate

result

Negotiation

Framework

Cloud Computing Environment

Performance

Measures Preal

Cloud Computing Environment

Performance

Measures Pmodel

Workload Model

The synthetic workload is supposed to be similar to those that occur in practice on real systems

End User

Real Workload

Synthetic Workload

The result is supposed to align with real application

Replicate in experimental setting cloud environment

Workload Resource Usage Observation

Transaction Classes

Frequency Maximum CPU time (msec)

Maximum I/O time (msec)

Trivial 40% 8 120

Light 30% 20 300

Medium 20% 100 700

Heavy 10% 900 1200

Workload Features

• Workloads are characterized by many attributes that have a great effect on scheduling

– I/O can also have a great effect on scheduling

– memory can lead to swapping.

– for parallel jobs, the number of processors used is an additional parameter, which influences how well jobs pack together.

• The level of detail needed in workload characterization depends on the goal of the evaluation.

Jemal H. Abawajy, Sivarama P. Dandamudi: Scheduling Parallel Jobs with CPU and I/O Resource Requirements in Cluster Computing Systems. MASCOTS 2003: 336-343

Workload Features

• Workloads are characterized by many attributes

– arrival times

– running times

– total memory usage (maybe locality of reference)

– distribution of I/O sizes and how they interleave with the computation

– Communication

– number of processors (VMs)

Adapted from Menascé & Almeida.

52

Selection of characterizing parameters

• Each workload component is characterized by two groups of

information:

• Workload intensity

– arrival rate

– number of clients and think time

– number of processes or threads in execution simultaneously

• Service demands (Di1, Di2, … DiK), where Dij is the service

demand of component i at resource j.

Exponential Distribution

• The Exponential:

– 𝜆 = measures how many things happen per unit

𝑓 𝑥 = 𝜆 ∙ 𝑒−𝜆∙𝑥

0

0.5

1

1.5

2

2.5

1 2 3 4 5 6 7

pro

bab

ility

den

sity

fu

nct

ion

s

X

Lambda 0.5 1 2

Hyper-exponential distribution

• Hyper-exponential distributions is used in service demand modeling

• The hyper-exponential distribution is obtained by selecting from a mixture of several exponential distributions.

• The simplest variant has only two stages:

𝜆1

𝜆2

𝑃

1 − 𝑃 𝜆1 ≠ 𝜆2

𝑓 𝑥 = 𝑝𝑖 ∙ 𝜆𝑖 ∙ 𝑒𝜆𝑖𝑥

𝑛

𝑖=1

0 ≤ 𝑝𝑖 ≤ 1

0 ≤ 𝜆1, 𝜆2𝑖 ≤ 1

Coefficient of Variation

• Coefficient of variation is defined as follows

𝐶𝑣 =𝜎

𝜇

• Where

– 𝜎 is the standard deviation

– 𝜇 is the mean

Long-lived vs short-lived Jobs

• Long-lived processes

• short-lived processes

𝑃 𝑇 > 𝜏 ∝ 𝛼2, 𝛼 ≈ 1

• This means that most processes are short, but a small number are very long.

Selecting VM for Migration

• A VM running on an overloaded physical machine will receive worse service than a VM running on an unloaded machine.

• Load balancing is the activity of migrating a running VM from an overloaded machine to an underloaded one.

Selecting VM for Migration

• It would be most beneficial if we could identify VM that may be expected to continue to run for a long time, and select them for migration.

• But how can we know in advance whether a VM is going to finish executing soon or not?

– The answer is that it depends on the statistics of VM runtimes.

Selecting VM for Migration

• Suppose we have the following as an example VM Start End Now Remain

𝑉𝑀1 0 16 12 4

𝑉𝑀2 0 64 20 44

𝑉𝑀3 0 14 4 10

Time unit

0 2 4 6 8 10 12 14 16 18 20 22 24

𝑉𝑀1 𝑉𝑀2 𝑉𝑀3

64

• The challenge is deciding the VM to migrate.

Selecting VM for Migration

• VM migration involves considerable overhead.

• If the VM terminates soon after being migrated, that overhead has been wasted.

• In addition, the VM cannot run during the time it is being migrated.

• Again, if it terminates soon after the migration, it would have been better off staying in its place.

Performance Measurements

• Classical performance metrics: – Response time,

– throughput,

– scalability,

– resource/cost/energy,

– efficiency,

– Elasticity

– Availability,

– reliability, and

– security

– SLA violation

Energy Consumption

Feature Extraction

Feature Selection

Applications

Resources

Features

Subset Features

Regression Method

Result Evaluation

Makespan

– Traditional: Makespan, Job Slowdown

– Workload Speedup One (SU1)

– Workload Slowdown Infinite (SUinf)

Cost Metrics • Cost Metrics

– Actual Cost (Ca)

– Charged Cost (Cc)

Compound Metrics

• Compound Metrics – Cost Efficiency (Ceff)

– Utility

Adapted from Menascé & Almeida.

66

Summary

• Workload Characterization

• what is it?

• basic concepts

• workload description and modeling

• representativeness of a workload model

• Methodology (1) • Choice of an analysis standpoint

• Identification of the basic component

• Choice of the characterizing parameters

• Data collection

Application

Communication

• Given – 𝑡0 is the start up time or latency,

– n is the number of nodes involved in the communication,

– 𝑟∞ = is the asymptotic bandwidth, which is defined as the maximal bandwidth achievable when the message length approaches infinity,

– m denotes the number of bytes of data to be transmitted.

Workload Parameters • We can use Sevcik's execution time function to

model a wide range of representative applications

– 𝜙: is an indication of the load imbalance in the computation, – W is the cumulative service demand for the job,

– p is the number of processors allocated to the job,

– 𝛼: represents the amount of sequential computation,

– 𝛽:represents the amount of communication and congestion overhead that increases with the number of processors.

– represents the ideal division of an application's basic work across p processors

Maximum Parallelism

Values

Acknowledgement

• The slides have been adapted from

– Menascé & Almeida.