International Journal of Cloud Computing (ISSN 2326-7550) Vol. 2, No. 1, January-March 2014 i IJCC...

International Journal of Cloud Computing (ISSN 2326-7550) Vol. 2, No. 1, January-March 2014

i

http://hipore.com/ijcc

IJCC Editorial Board

Editors-in-Chief Hemant Jain, University of Wisconsin–Milwaukee, USA

Rong Chang, IBM T.J. Watson Research Center, USA

Associate Editor-in-Chief Bing Li, Wuhan University, China

Editorial Board Danilo Ardagna, Politecnico di Milano, Italy

Janaka Balasooriya, Arizona State University, USA

Roger Barga, Microsoft Research, USA

Viraj Bhat, Yahoo, USA

Rajdeep Bhowmik, Cisco Systems, Inc., USA

Jiannong Cao, Hong Kong Polytechnic University, Hong Kong

Buqing Cao, Hunan University of Science and Technology, China

Keke Chen, Wright State University, USA

Haopeng Chen, Shanghai Jiao Tong University, China

Malolan Chetlur, IBM India, India

Alfredo Cuzzocrea, ICAR-CNR & University of Calabria, Italy

Ernesto Damiani, University of Milan, Italy

De Palma, University Joseph Fourier, France

Claude Godart, Nancy University and INRIA, France

Nils Gruschka, University of Applied Sciences, Germany

Paul Hofmann, Saffron Technology, USA

Ching-Hsien Hsu, Chung Hua University, Taiwan

Patrick Hung, University of Ontario Institute of Technology, Canada

Hai Jin, HUST, China

Li Kuang, Central South University, China

Grace Lin, Institute for Information Industry, Taiwan

Xumin Liu, Rochester Institute of Technology, USA

Shiyong Lu, Wayne State University, USA

J.P. Martin-Flatin, EPFL, Switzerland

Vijay Naik, IBM T.J. Watson Research Center, USA

Surya Nepal, Commonwealth Scientific and Industrial Research Organisation, Australia

Norbert Ritter, University of Hamburg, Germany

Josef Schiefer, Vienna University of Technology, Austria

Jun Shen, University of Wollongong, Australia

Weidong Shi, University of Houston, USA

Liuba Shrira, Brandeis University, USA

Kwang Mong Sim, University of Kent, UK

Wei Tan, IBM T.J. Watson Research Center, USA

Tao Tao, IBM T. J. Watson Research Center, USA

Kunal Verma, Accenture Technology Labs, USA

Raymond Wong, University of New South Wales & NICTA, Australia

Qi Yu, Rochester Institute of Technology, USA

Jia Zhang, Carnegie Mellon University – Silicon Valley, USA

Gong Zhang, Oracle Corporation, USA


ii


Call for Articles

International Journal of Cloud Computing

Mission Cloud Computing has become the de facto computing paradigm for Internet-

scale service development, delivery, brokerage, and consumption in the era of

Services Computing, fueling innovative business transformation and connected

human society. 15 billion smart devices would be communicating dynamically

over inter-connected clouds by 2015 as integral components of various

industrial service ecosystems. The technical foundations of this trend include

Service-Oriented Architecture (SOA), business & IT process automation,

software-defined computing resources, elastic programming model &

framework, and big data management and analytics. In terms of the delivered

service capabilities, a cloud service could be, among other as-a-service types, an infrastructure service

(managing compute, storage, and network resources), a platform service (provisioning generic or

industry-specific programming API & runtime), a software application service (offering email-like

ready-to-use application capabilities), a business process service (providing a managed process for,

e.g., card payment), a mobile backend service (facilitating the integration between mobile apps and

backend cloud storage and capabilities) or an Internet-of-things service (connecting smart machines

with enablement capabilities for industrial clouds).

As the first Open Access research journal on Cloud Computing, the International Journal of Cloud

Computing (IJCC) aims to be a valuable resource providing leading technologies, development, ideas,

and trends to an international readership of researchers and engineers in the field of Cloud Computing.

Topics The International Journal of Cloud Computing (IJCC) covers state-of-the-art technologies and best

practices of Cloud Computing, as well as emerging standards and research topics which would define

the future of Cloud Computing. Topics of interest include, but are not limited to, the following:

- ROI Model for Infrastructure, Platform, Application, Business, Social, Mobile, and IoT Clouds

- Cloud Computing Architectures and Cloud Solution Design Patterns

- Self-service Cloud Portal, Business Dashboard, and Operations Management Dashboard

- Autonomic Process and Workflow Management in Clouds

- Cloud Service Registration, Composition, Federation, Bridging, and Bursting

- Cloud Orchestration, Scheduling, Autoprovisioning, and Autoscaling

- Cloud Enablement in Storage, Data, Messaging, Streaming, Search, Analytics, and Visualization

- Software-Defined Resource Virtualization, Composition, and Management for Cloud

- Security, Privacy, Compliance, SLA, and Risk Management for Public, Private, and Hybrid Clouds

- Cloud Quality Monitoring, Service Level Management, and Business Service Management

- Cloud Reliability, Availability, Serviceability, Performance, and Disaster Recovery Management

- Cloud Asset, Configuration, Software Patch, License, and Capacity Management

- Cloud DevOps, Image Lifecycle Management, and Migration

- Cloud Solution Benchmarking, Modeling, and Analytics

- High Performance Computing and Scientific Computing in Cloud

- Cloudlet, Cloud Edge Server, Cloud Gateway, and IoT Cloud Devices

- Cloud Programming Model, Paradigm, and Framework

- Cloud Metering, Rating, and Accounting

- Innovative Cloud Applications and Experiences

- Green Cloud Computing and Cloud Data Center Modularization

- Economic Model and Business Consulting for Cloud Computing


iii


International Journal of

Cloud Computing

January-March 2014, Vol. 2, No.1

Table of Contents

EDITOR-IN-CHIEF PREFACE iv Hemant Jain, University of Wisconsin–Milwaukee, USA

Rong Chang, IBM T.J. Watson Research Center, USA

RESEARCH ARTICLES 1 On The Financification Of Cloud Computing: An Agenda For Pricing And

Service Delivery Mechanism Design Research Robert J. Kauffman, Singapore Management University, Singapore

Dan Ma, Singapore Management University, Singapore

Richard Shang, Singapore Management University, Singapore

Jianhui Huang, The Conference Executive Board Asia Pte. Ltd., Singapore

Yinping Yang, Singapore Management University, Singapore

15 Impacts Of Multi-Class Oversubscription On Revenues And Performance In The

Cloud Rachel A. Householder, Bowling Green State University

Robert C. Green, Bowling Green State University

31 Optimization Of Operational Costs In Hybrid Cooling Data Centers

With Renewable Energy Shaoming Chen, Louisiana State University, US

Yue Hu, Louisiana State University, US

Lu Peng, Louisiana State University, US

45 A Broker Based Consumption Mechanism For Social Clouds Ioan Petri, Cardiff University, UK

Magdalena Punceva, University of Applied Sciences, Western Switzerland

Omer F. Rana, Cardiff University, UK

George Theodorakopoulos, Cardiff University, UK

Yacine Rezgui, Cardiff University, UK

58 Call for Papers: IEEE CLOUD/ICWS/SCC/MS/BigData/SERVICES 2014

Call for Articles: International Journal of Services Computing (IJSC)

Call for Articles: International Journal of Big Data (IJBD)


iv


Editor-in-Chief Preface:

Cloud Management and Assessment

Hemant Jain Rong Chang University of Wisconsin–Milwaukee, USA IBM T.J. Watson Research, USA

Welcome to the inaugural issue of International Journal of Cloud Computing (IJCC), the first open

access on-line journal on cloud computing. The increasing importance of cloud computing is

evidenced from the rapid adoption of this technology in businesses around the globe. The cloud

computing is redefining the business model of various industries from video rental (Netflix is enabled

by cloud) to small start-up companies (companies can be started with very little investment using

cloud infrastructure). The potential of cloud computing is even more promising. The cloud computing

combined with developments like internet of things can significantly change the life as we know today.

However, to deliver on these promises and to prevent clouding computing from becoming a passing

fad significant technical, economic, and business issues need to be addressed. IJCC is designed to be

an important platform for disseminating high quality research on above issues in a timely manner and

provide an ongoing platform for continuous discussion on research published in this journal. We aim

to publish high quality research that addresses important technical challenges, economics of sustaining

this environment, and business issues related to use of this technology including privacy and security

concerns, legal protection, etc. We seek to publish original research articles, expanded version of

papers presented at high quality conferences, key survey articles that summarizes the research done so

far and identify important research issues, and some visionary articles. We will make every effort to

publish articles in a timely manner.

This inaugural issue collects the extended version of five IEEE CLOUD 2013 articles in the general

area of managing Cloud computing environment.

The first article is titled “Cost-Driven Optimization of Cloud Resource Allocation for Elastic

Processes” by Schulte, Schuller, Hoenisch, Lampe, Steinmetz, and Dustdar. They present an approach

to address the cost-driven optimization of cloud-based computational resources, based on automatic

leasing and releasing of resource allocation for Elastic Processes. Empirical study and analysis are

presented.

The second article is titled “Recommending Optimal Cloud Configuration based on Benchmarking in

Black-Box Clouds” by Jung, Sharma, Mukherjee, Goetz, and Bourdaillet. They present a benchmark-

based modeling approach to recommend optimal cloud configuration for deploying user workloads,

based on various non-standardized configuration options offered by cloud providers. A search

algorithm is provided to generate a capability vector consisting of relative performance scores of

resource types. Experimental results are reported.

The third article is titled “Taming the Uncertainty of Public Clouds” by Schnjakin and Meinel. They

present a framework featuring improved availability, confidentiality and reliability of data stored in

the cloud. User data is encrypted together with the RAID technology to manage data distribution

across cloud storage providers. Experiments are conducted to evaluate the performance and cost

effectiveness of the presented approach.

The fourth article is titled “Cloud Standby System and Quality Model” by Lenk and Pallas. The

authors argue that contingency plans, replicate an IT infrastructure to the Cloud, are useful for disaster

preparedness. They propose a cloud standby approach together with a Markov-based model to analyze

and configure cloud standby systems.

The fifth article is titled “Efficient Private Cloud Operation using Proactive Management Service” by

Dong and Herbert. The authors present a distributed service architecture aiming to provide an


v


automated, shared, and off-site operation management service for private clouds. A prototype system

and empirical study are presented.

We would like to thank the authors for their effort in delivering those five quality articles. We would

also like to thank the reviewers as well as the Program Committee of IEEE CLOUD 2013 for their

help with the review process. Finally, we are grateful for the effort Jia Zhang and Liang-Jie Zhang

made to this inaugural issue of International Journal of Cloud Computing (IJCC).

About the Editors-in-Chief

Dr. Hemant Jain is the Interim Director of Biomedical and Health Informatics

Research Institute, Roger L. Fitzsimonds Distinguished Scholar and Professor of

Information Technology Management at University of Wisconsin–Milwaukee. Dr.

Jain specializes in information system agility through web services, service oriented

architecture and component based development. His current interests include

development of systems to support real time enterprises which have situational

awareness, can quickly sense-and-respond to opportunities and threats, and can

track-and-trace important items. He is also working on issues related to providing quick access to

relevant knowledge for cancer treatment and to providing medical services through a virtual world.

Dr. Jain is an expert in architecture design, database management and data warehousing. He teaches

courses in database management, IT infrastructure design and management, and process management

using SAP. Dr. Jain was the Associate Editor-in-Chief of IEEE Transactions on Services Computing

and is Associate Editor of Journal of AIS, the flagship journal of the Association of Information

Systems.

Dr. Rong N. Chang is Manager & Research Staff Member at the IBM T.J. Watson

Research Center. He received his Ph.D. degree in computer science & engineering

from the University of Michigan at Ann Arbor in 1990 and his B.S. degree in

computer engineering with honors from the National Chiao Tung University in

Taiwan in 1982. Before joining IBM in 1993, he was with Bellcore researching on B-

ISDN realization. He is a holder of the ITIL Foundation Certificate in IT Services

Management. His accomplishments at IBM include the completion of a Micro MBA Program, one

IEEE Best Paper Award, and many IBM awards, including four corporate-level Outstanding

Technical Achievement Awards and six division-level accomplishments. He is an Associate Editor of

the IEEE Transactions on Services Computing and the International Journal of Services Computing.

He has chaired many conferences & workshops in cloud computing and Internet-enabled distributed

services and applications. He is an ACM Distinguished Member/Engineer, a Senior Member of IEEE,

and a member of Eta Kappa Nu and Tau Beta Pi honor societies.


1 http://www.hipore.com/ijcc/

ON THE FINANCIFICATION OF CLOUD COMPUTING: AN AGENDA FOR PRICING AND

SERVICE DELIVERY MECHANISM DESIGN RESEARCH Robert J. Kauffman(a), Dan Ma(a), Richard Shang(b), Jianhui Huang(c) and Yinping Yang(a,d)

(a) School of Information Systems, Singapore Management University, Singapore(b) School of Business, Public Admin. and Info. Sciences, Long Island University Brooklyn, NY, USA

(c) The Conference Executive Board Asia Pte. Ltd., Singapore(d) Institute of High Performance Computing (IHPC), A*STAR, Singapore

Email: {rkauffman, madan}@smu.edu.sg, [email protected], [email protected], [email protected]

Abstract Pricing approaches to cloud computing services balance risks and interests between vendor and client, and optimize supply and consumption in terms of cost, uncertainty and economic efficiency. They also leverage the benefits of various services delivery mechanisms for reserved, on-demand, spot-price, and re-sold services in markets that have learned how to transact in full contracts and services instances. This is like a financial market: with services supply and demand, and opportunities to supply and purchase services with spot prices, or to sell or buy contracts for the delivery of future services. Our research suggests that the financification of the cloud computing services market represents a fundamental shift from the traditional model of software sales and large contracts outsourced to services vendors, to short-term contracts and computing capacity provision mechanism designs that are evolving similar to financial markets. We develop this perspective to explain the cloud vendor market, the provision of services, and the ways in which the financification of cloud computing will shape future offerings and the structure of the market. We see these changes in the market in the many ways that vendors offer cloud services of high value to organizations, while making more profitable business models possible.

Keywords: Cloud computing, economics, financification, intermediation, IT services, mechanism design, pricing, research directions, risk management, stakeholders.

___________________________________________________________________________________________________________________________

1. Introduction

Cloud computing is a means of providing commercial

information technology (IT) services to customers and

organizations. In traditional IT markets, CPUs, networks,

data storage and software applications are sold as products.

Customers own a perpetual license after a one-time payment,

but they have to pay for upgrades and other in-house IT

costs. In the past decade, many IT services vendors have

turned to cloud computing: they adjusted their services

provision and pricing schemes to permit “pay-as-you-go”

access so that customers are able to pay for usage or

subscribe to the computing resources they need.

The underlying technologies that empower cloud

computing services are not entirely new. They originated

from the idea of “computation … as a public utility” in the

1960s (Garfinkel 2011). Virtual private network (VPN)

services in the 1990s and grid computing in the early 2000s

were predecessors of today’s cloud computing services.

Amazon played a key role in the development of cloud

computing by providing cloud services to external

customers and launching Amazon Web Services (AWS) on

a utility computing basis in 2006. With the entry of many IT

giants such as Google, Microsoft, Oracle, and IBM, the

cloud services market has become prosperous but more

competitive.

Initially, there were three main types of cloud computing

services in the market: infrastructure-as-a-service (IaaS),

platform-as-a-service (PaaS), and software-as-a-service

(SaaS). As the cloud market matured, more categories

emerged, such as data storage-as-a-service (DSaaS),

hardware-as-a-service (HaaS), desktop-as-a-service (DaaS),

business process-as-a-service (BPaaS), data analytics-as-a-

Service (DAaaS) and others (Rimal et al. 2009).

Some industry reports have suggested the huge market

potential for cloud computing services. The New York-

based 451 Research (2013) forecasted that the cloud

computing market revenue would grow at a compound rate

of 36% to US$20 billion by the end of 2016. Gartner (2012)

reported that annual IT spending on cloud service brokerage

services would have reached US$100 billion by late 2014.

In addition, Transparency Market Research (2011) reported

that the cloud computing services market was valued at

US$79.6 billion in 2011, and would grow at a compound

doi: 10.29268/stcc.2014.2.1.1

http://en.wikipedia.org/wiki/Amazon_Web_Services



annual growth rate (CAGR) of 23.21% and hit US$148.9

billion in 2014, higher than Gartner’s estimate. It also

forecasted that the market value of services production

would reach US$205.4 in 2018, based on a slower CAGR of

8.39%.

There are conflicting viewpoints on costs, performance,

compliance and management, about whether cloud services

are a better alternative to in-house systems, however. Prior

research has pointed out that the monetary cost of running

scientific data-intensive applications using Amazon.com’s

Simple Storage Service (S3) is out of reach for some clients,

because the storage services – including availability,

durability, and access performance – can be expensive and

not altogether necessary (Palankar et al. 2008). This has

been especially true for small and medium enterprises

(SMEs). Deelman et al. (2008) presented other results

though: with storage cost reductions, using cloud services is

cheaper than in-house systems for data-intensive

applications.

Different concerns have been raised regarding the

pricing of cloud computing services (Yeo and Buyya 2007).

Should the services be simple or complex? The menu of

available services now has become quite complex. Instead

of only fixed-price, fixed-menu items, today vendors are

permitting their services to be offered in highly

customizable configurations for customers with different

profiles. The granularity of selectable service components

that are charged separately has become smaller too: from the

size of storage to the number of read/write operations.

Clients who prefer this may know how to configure the

services to achieve customization. But this also increases the

difficulty of cost estimation, because the prices will change

with the configurations. So clients who want simple services

offerings that come with simple fixed-prices may not want

this.

Furthermore, the fact that clients have to put all or part

of their data on the cloud has created concerns about the

control and security of sensitive data that reside there.

Clients will not be able to access their data if the cloud

computing services are down, and they have not backed

them up properly. Sometimes it may be impossible to back

up in a timely way: this is because of the size of the data and

the limitations on the network bandwidth.

Previous studies have recognized the complexity and

importance of appropriate pricing strategy (Demirkan et al.

2008, Durkee 2010, Marston et al. 2011). The changing

cloud services industry invites fuller investigation of pricing

strategies to identify factors that reflect vendor concerns

when they make pricing decisions and evaluate their

services. Some key considerations may be missing in

current industry practice.

This research aims to provide insights to help managers

understand the complex ecosystem of the cloud computing

services market. We also will make meaningful predictions

about future changes in vendor pricing and services

provision. We ask: (1) What factors are driving the

emergence of the new services practices? (2) What

characteristics of pricing and services provision are likely to

persist in the market? (3) And what are the potential

directions for future market services provision and pricing

mechanisms?

In this article, we argue that financification, which has

been discussed in the context of IT outsourcing by Bardhan

et al. (2010a), is also suitable for cloud computing services

and other new kinds of IT services. Financification refers to

the technology-enabled practices associated with financial

market operations, revenue yield management and financial

risk management. With respect to cloud computing services,

it is intended to mean that cloud services vendors and clients

will be increasingly subject to financial market-like

conditions. Services will have bid and ask prices, just like

financial securities assets and derivatives in the stock

market.

Similar to the risks that participants in a financial market

face (securities buyers and sellers, and financial

intermediaries), vendors, clients and cloud services

intermediaries also face multiple sources of risk related to

the provision, use, and management of IT services that are

similar to financial market-like operations. Cloud services

vendors, in particular, face uncertainty in services demand,

and their clients have to balance the benefits of cloud

computing services with the risks related to control and

information security, continuity of services, and integration

with other software applications. On the other hand, cloud

computing services are built on advanced IT, so many

heretofore manual processes can be automated. This makes

it possible for vendors and clients to rapidly adjust and

create value amid the emerging changes and risks. It also

opens up the possibility for cloud intermediaries to add

value in the market.

This article makes three contributions to research on

cloud computing services. (1) Based on financial market

concepts and theory, we provide an analysis of the cloud

computing services market and identify key elements that

are present that make it like a financial market. (2) We

assess the paths for the future development of the cloud

computing services market, based on its increasing

financification. (3) We also offer research directions that

will support the achievement of best practices and good

market design.

2. Financification and the Design ofMarket-Based Economic Exchangein IT Services

We next discuss characteristics of cloud services, the

functioning of the related services market, and the ways in

which it has become increasingly financified – in other

words, more and more like a real-world financial market.

2.1. Cloud Computing Services: Characteristics

doi: 10.29268/stcc.2014.2.1.1



IT services facilitate the processing, manipulation and

managerial use of information with the help of computers,

networks, communication devices, and technologies that

interoperate all these devices and networks. Leavitt and

Whisler (1958) defined IT services in terms of the

technologies used for processing large amounts of

information, applying statistical and mathematical methods

to decision-making, and simulating mental processes with

computers. The definition highlights the role of IT, and IT

services by association, in enhancing the capability of

human beings to rapidly process large amounts of complex

data.

Cloud computing technology enables software

applications to be easily scaled up and down, and allow

companies to gain agility in prototyping, developing, testing,

and deploying new applications and services. They include

technologies that enable data and application portability.

They also comprise virtualization and parallelization

techniques that enable better computing power utilization

(Williams et al. 2012), web-scale resources and data

management technologies (Birman et al. 2009). Finally,

they cover large-scale event notification and multi-tenancy

technologies (Zhou et al. 2010), service-oriented

architecture (Bardhan et al. 2010a) and web services

(Newcomer and Lomow 2004). Cloud computing

technology has created new possibilities for IT services

provision and business operation in ways that are

dramatically different from what was available before.

Cloud computing services have some important

characteristics. First, they are similar to information goods:

the cost of providing an additional unit of services is

negligible. This is true for an additional hour of cloud

services or a new client. Such an approach compares

favorably to the large investment that is required in the

infrastructure to power the services (Varian 1995). Cloud

computing services are also similar to experience goods:

clients have imperfect information about services until they

have tried them out (Shapiro 1983). In addition, cloud

vendors sell services in units that are charged based on the

amount of time a client uses them, making the services

similar to perishable goods, whose value diminishes over

time and cannot be restored (Bardhan et al. 2010a). Finally,

unlike electrical and water utilities, cloud computing

services address multiple purposes. They take various forms,

including storage, computation, networking and applications,

and can be consumed separately and jointly.

2.2. A Profile of the Cloud Computing Market

The cloud computing services market is an ecosystem

that includes different stakeholders that play several

different kinds of roles, including component, service

provision, and infrastructure roles (Adomavicius et al. 2008).

The IT services industry supports the operation and

management of clusters of computers and servers with

specialized code that enables the efficient allocation of

computing resources. These key components make cloud

computing possible. An example of related stakeholders that

play the component provision role in cloud computing is

virtualization solution providers. They have the technical

and technology capabilities to help organizations build their

own data centers and private clouds. The players include

VMware, Citrix, Oracle, and Microsoft Virtual, and others.

A difference between cloud computing and traditional IT

services is that they are delivered via networks, including

the Internet, mobile networks, and private networks.

Network services providers operate local or wide area

networks, mobile networks, satellite networks, or other

types of networks through which clients can transfer data.

Other firms, such as telecom and broadband services

companies, satellite operators, and so on also play the role

of infrastructure providers.

A third role that we observe is application services

providers. Stakeholders in this role are services providers

that directly interact with cloud computing clients. They

mainly are software application providers that deploy

products on cloud platforms enabled by different

infrastructure technologies. Clients subscribe to applications

provided by these stakeholders, and they address clients’

business needs, computing requirements, in multiple

industry settings. The coverage spans online gaming to

scientific computing, and more.

When cloud services providers initially emerged, they

used delivery and pricing mechanisms in a pull-and-lock

mode. Clients locked the resources for however long a

period they needed once they successfully launched a

service instance in the cloud. Later, the services began to be

provided in a pull-and-lose mode. This enabled the vendor

to sell idle computing resources in a cheaper but more

flexible way. A downside was that clients might lose access

to computing resources even though they launched service

instances. The vendor could redirect the resources wherever

they provided a higher return.

On the pricing side, cloud computing services vendors

employ a number of different pricing mechanisms,

including usage-based, subscription-based, and a hybrid mix

of fixed-price and usage-based pricing. Even for a specific

type of pricing though, variations exist in the market. For

example, cloud services subscription plans can differ in the

length of the subscription period (a month, a quarter, or a

year), the limit on the number of user accounts that can

access the cloud within a subscription plan, and the number

of applications that can be hosted.

Prices have been decreasing over the last decade,

suggesting evidence for the growth and maturation of the

cloud computing services business. For example, Amazon

Web Services (AWS), the dominant vendor in the market,

has reduced the prices of its services offerings many times

(Lauchlan 2013). Amazon successfully promoted cloud

services and achieved a 30% market share by 2013, with

more than US$1 billion in revenues in a market with high

estimated annual growth (Nichols 2014). Another sign of

the industry becoming more competitive is the price war



that arose between Amazon and Google, shortly after

Google introduced its own Compute Engine in March 2014

(Jackson 2014).

In this competitive context, clients face the challenge of

being aware of which types of tasks cloud computing

services suit the best. They might be unclear about the total

cost of cloud computing services adoption (Durkee 2010). It

also is not easy for clients to monitor their cloud services

usage, and know the total payments due to the uncertain

demand and complex pricing structures of cloud services

(Weinhardt et al. 2009). These things have slowed down the

adoption of cloud computing services (Perry 2010).

2.3. The Financification of the Cloud Market

Our observations on the cloud computing services

market have focused on new services delivery, pricing

mechanisms, and new stakeholders.

Industry surveys from the Cloud Information Forum

(2011, 2012) have suggested that the primary drivers of the

adoption of cloud services are flexibility, cost savings and

low adoption cost, while the major obstacles are concerns

about data security and privacy, reliability and contractual

liability. So cloud services providers need to meet their

customers’ needs and requirements with a more flexible and

collaborative approach. There is also a need to address the

vendor and client risks better, so it is possible to optimize

supply in the presence of shifting demand, and new pricing

and services delivery mechanisms.

They motivate us to explain how the financification of

the cloud services market is progressing in a fuller way. To

understand this perspective, consider some of the key

features of financial markets: (1) bid and ask prices for

securities; (2) spot, forward and futures prices; (3) liquidity

versus depth; and (4) hedging and risk management. Seeing

these in the cloud computing market will suggest a

progression toward financification.

We have already discussed bid and ask prices, and the

different ways in which cloud computing services prices can

be quoted. Note that there is currently no cloud services

market exchange that handles cloud computing futures

contracts, as are handled by financial market exchanges

when futures contracts for foreign exchange (FX),

derivatives or other financial instruments are involved.

There are some hints that options and forward contracts for

cloud computing may be coming though (Rogers and Cliff

2012b), with recent research on market mechanisms and

demand revelation in IT services (Rogers and Cliff 2010,

2012c; Wu et al. 2008). Related issues have been explored

before for options on IT resources (Clearwater and

Huberman 2005, Yeo and Buyya 2006) and grid computing

(Clearwater and Huberman 2005, Sandholm et al. 2006).

Some of these approaches have been conceptualized similar

to financial options and forward contracts on FX, for

example, which trade in the over-the-counter (OTC) market

between broker-dealers based on bilateral negotiation, but

not as standardized contracts in financial market exchanges.

Other related issues that have been studied include

financial risk management for IT services resources, and the

benefits associated with matching risks between the vendor

and client sides (Benaroch et al. 2010) include, for example,

the technology-enabled practices associated with resource

management, revenue yield management, and risk

management of IT services (Benaroch et al. 2010, Rogers

and Cliff 2010, Kauffman and Sougstad 2008a, 2008b). This

is like portfolio management, the focus on asset pricing, and

the emphasis on financial risk in investments and markets,

only for the IT services market (Bardhan et al. 2010a,

2010b).

In addition, we view the cloud computing services

market as an IT services ecosystem that consists of

interdependent stakeholders that have created the conditions

for the emergence of a near-financial market in this services

arena. There are cloud computing services vendors, cloud

technology services providers, services brokerages,

application service providers, and services clients. Services

vendors provide the major categories of cloud computing

services, including IaaS, PaaS, SaaS, and other “X-as-a-

service” offerings. Cloud technology services providers

further include vendors that offer the technologies that

enable cloud computing. They use virtualization

technologies, application parallelization, large-scale storage

solutions, monitoring and billing technologies, and other

capabilities. Vendors and clients exchange value to achieve

joint economic gain. In addition, market intermediaries and

brokers, similar to those in financial markets, facilitate the

search for and matching of clients with appropriate services

vendors, smooth transaction handling, and offer peripheral

services (Huang 2013, Huang et al. 2013b).

The financification of the cloud computing services

market supports effective practices to enhance market

performance and avoid market failure (Shang et al. 2012,

2013). The mobilization of cloud services resources and

their allocation to productive uses can be coordinated via

vendor pricing that permits clients to discover their

willingness-to-pay, much the same as what occurs in

financial markets and with revenue management (Kauffman

and Ma 2013). In addition, new instruments, similar to

financial instruments in financial markets, can be created to

support the transfer of cloud services resources from one

client to another, a broker to another broker, and so on. Spot

prices for services that change in the market based on

supply and demand, and longer-term lock-in of the services

provide this kind of flexibility (Huang 2013). This form of

economic exchange via trading will help to increase the

liquidity of cloud computing services, creating fuller

utilization and greater market-generated welfare (Huang et

al. 2013a, 2013b). This also provides more flexibility for

when, how, and how many resource units are consumed in

the market, and permits conversion of unutilized resources

into money.

We also are observing new forms of intermediation in

cloud services, similar to other e-commerce markets



(Cartlidge and Clamp 2014, Rogers and Cliff 2012a). Cloud

services brokerages have emerged as digital intermediaries

in the services market, creating value for both cloud clients

and vendors (Gartner 2012). A recent industry article

reported on 35 cloud brokerage firms (Panettieri 2013).

They generate value for clients by supporting cloud services

intermediation, aggregation, and arbitrage. They also offer

value-added customization services, making cloud

computing frictionless and refined. They also facilitate its

integration with a firm’s internal software systems, reducing

the risk of mistaken adoption for clients, similar to what we

observed in the 2000s for electronic markets (Dai and

Kauffman 2004). Cloud computing services also can be

tweaked to meet clients’ needs and still be profitable for the

vendor.

3. State-of-the-Art in CloudComputing Pricing and ServicesProvision

To get a comprehensive view of the cloud services

market, we examined 19 cloud services vendors and 27

services offerings that they provide. They include four

major types of cloud computing services: IaaS, PaaS, SaaS,

and brokered cloud services. IaaS delivers computer

infrastructure based on virtualization technology. PaaS has

an additional layer on which clients can run applications

without knowing how the underlying infrastructure is

implemented. SaaS provides application services that

function as locally-installed software (Vaquero 2008).

Cloud services intermediaries, aggregators, and arbitrageurs

provide brokered cloud services in the market as well

(Gartner 2012). (See Appendix A.)

We reviewed the cloud computing services offerings and

collected pricing information from the major market players.

Our criteria for selecting a vendor were: (1) the vendor must

have made pricing information on all its services available

on its official web site; and (2) the vendor must have been

selected at least once for review in Gartner’s Magic

Quadrant Report (Leong and Chamberlin 2010, 2011;

Leong et al. 2012, 2013). The reports list cloud computing

services vendors that are leaders in the market, in terms of

revenue and market share. This is useful information. It

helps to ensure that we are sampling from an appropriate set

of cloud services vendors in the market, so the right kinds of

firms are represented, which makes our results more

meaningful.

We next offer our reading on the state-of-the-art in the

cloud computing services market, inclusive of current

services provision and pricing mechanisms, and their trends,

based on our observations and analysis.

3.1. Services Delivery and Pricing Mechanisms

Table 1 shows that most PaaS, SaaS, and cloud

brokerage services vendors offer reserved services delivery.

Cloud computing services with reserved resources are pre-

committed resources for clients by the vendor. Clients can

choose from several options for the length of the reservation

period predefined by the services vendor. For example,

Amazon EC2 offers clients with reserved compute instances

for a period of one year or three years. From the vendors’

point of view, reservations benefit them by reducing their

demand uncertainty. Any pre-paid reservation fees can

enhance a vendor’s cash flow, and generate lock-in with

clients.

Associated with the reserved services is reservation-

based pricing, which varies with the type of services offered.

Reservation-based pricing has been popular in the restaurant

and hotel industry; it typically results in increased vendor

revenue (Alexandrov and Lariviere 2008). In the case of

hotel reservations, rooms usually are scarce in popular

attraction areas, so travelers must be willing to pay for

reservations. The same rationale does not hold in cloud

services though. Computing capacity is expandable at a

relatively low cost. So clients have little incentive to reserve

services (Meinl et al. 2010). The situation will change when

the cloud services market becomes more competitive and

new demand emerges due to advances in related

technologies though. These will include sensor technologies

that enable wearable devices connected to cloud-based

health informatics services, and web-based data analytics

that help companies gain insight into their operations,

customers and the marketplace for their products.

Table 1. Services delivery and pricing mechanisms

RESERVAT

ION-BASED

PRICING

USAGE-BASED PRICING TECHNICAL

SUPPORT-

BASED

PRICING FIXED

PRICE

SPOT

PRICE

Reserved

services

delivery

Amazon,

Google,

Microsoft,

Rackspace,

GoGrid,

Joynet, HP

FlexiScale,

Amazon,

Google,

Microsoft,

Rackspace,

GoGrid,

Joynet, HP,

FlexiScale,

On-

demand

services

delivery

Amazon,

Google,

Microsoft,

Rackspace,

GoGrid,

Joynet, HP,

FlexiScale,

Spot-price

services

delivery

Amazon

Brokered

services

delivery

PiCloud,

CloudSigma,

ProfitBricks

On-demand services delivery, which provides reserved

resources based on pay-per-use, also has been widely

adopted in the cloud computing services market. On-

demand services delivery performs in an interesting way.

Once a client launches a job, the vendor will set aside

capacity for the job until the client terminates it. Clients are

charged based on the usage of services, as well as by the

amount of time that the services are used. Usage-based



pricing is optimal for information goods that have negligible

marginal production costs (Maskin and Riley 1984), such as

movie CDs or software apps. Most IaaS vendors in our

survey employ usage-based pricing. The prevalence of

usage-based pricing among IaaS offerings is inconsistent

with the findings in Fishburn et al. (2000) and Sundararajan

(2004), though it matches the findings in an earlier study by

Maskin and Riley (1984). The key difference in these

studies is

whether the transaction costs associated with usage-based

pricing are negligible. IaaS vendors commonly implement

highly automated management systems, which generally

have low transaction costs. So it is reasonable for IaaS

vendors to adopt a pure usage-based pricing scheme.

Prior research suggests that fixed-fee pricing together

with usage-based pricing always outperforms pure usage-

based pricing (Sundararajan 2004). Such two-part tariff

pricing is never worse than any non-linear pricing strategy

(Masuda and Whang 2006, Png and Wang 2010). These

findings are consistent with pricing practices in the cloud

market also. Many PaaS and SaaS vendors have adopted the

two-part tariff pricing model, for example. Clients typically

pay a monthly subscription fee for pre-assigned usage

quotas, and pay an additional price if the usage exceeds

them.

The resource acquisition and allocation of spot-price

services delivery differ from those of reserved resources.

There is no commitment on the part of the vendor to

guarantee access at a given time, other than via the client’s

willingness to pay the spot-market price for services. The

acquisition of resources for spot-price services varies

according to a client’s bid price valuation and the changing

relationship between supply and demand. Clients submit

bids representing the maximum unit prices they are willing

to pay for a predefined type of spot-price service. As soon

as the service price in the spot market goes above the client's

bid price, the vendor will terminate its in-process services.

As a result, computing tasks running as spot-price services

may occasionally be interrupted due to price spikes in the

market. Clients will receive service allocations that are

affected by the interplay between supply and demand, and

bear risks of service termination that are not controllable by

themselves. On the other hand, spot-price services are

cheaper – most of the time, spot-price services represent less

than 25% of all tasks that use reserved resources. The cost

savings from using spot-price resources are attractive for

clients who need cloud services for compute-intensive but

time-insensitive tasks, such as scientific computing, web

crawling, and data analytics.

A variety of resource allocation approaches have

emerged recently, involving predictive analytics, machine

learning, and other models that support value-conscious use

of limited server resources (Das et al. 2011, Mazzucco and

Dumas 2011). Empowered by these resource management

techniques, brokered services delivery is able to provide less

costly and more reliable services to clients. For example,

PiCloud (now owned by DropBox), a computing services

broker that connects clients to Amazon’s cloud services,

emphasizes a positive customer experience. It delivers

results 33% faster and meanwhile saves clients 65% in total

costs compared to spot-price purchases, while still running

85% of jobs as Amazon spot-price services instances (Elliott

2012). Brokered services delivery can be provided in

various ways, such as management services provided as

subscription plans that support day-to-day management of

cloud computing services from various vendors, or value-

added services that are charged by usage. Clients usually do

not have full control of the resources. Instead, services

brokers make it transparent to clients how they acquire,

integrate and manage resources from different services

vendors. They also provide clients with interfaces to

configure and manage their usage.

Most major vendors apply technical support-related

pricing to different technical support plans with different

levels of expertise for client engagement. In general, SaaS

vendors provide greater flexibility in technical support

options, while IaaS and PaaS vendors offer more limited

options for their clients. This may be due to the relative

simplicity of IaaS and PaaS services. For example, IaaS

clients can terminate the services and shift to other vendors

anytime without incurring high costs. In contrast, SaaS

services typically contain functions that are provided only

by a particular vendor. More technical support from the

vendor is needed when problems occur, and a switch to

other SaaS vendors is generally difficult.

3.2. Services Delivery and Pricing Innovations

Amazon has been an innovator in cloud computing

services delivery and pricing mechanisms. It first introduced

its Elastic Compute Cloud (EC2) services in 2006, and used

an on-demand services delivery with a pay-per-use pricing

mechanism. Payments were based on actual usage, charged

by the hour. Since then, Amazon and its competitors

introduced a series pricing innovations in the market.

In 2009, Amazon announced two other new services

delivery mechanisms: EC2 reserved instances and EC2 spot

instances. With the reserved services delivery mechanism, a

client must pay a fixed fee up front to reserve services. The

client still needs to pay for actual usage, but the per-hour

rate will be lower than that in the on-demand pay-per-use

model that Amazon introduced in 2006. Spot-price services

delivery uses a different pricing model that is auction-based.

The major difference between spot-price services and the

other options that Amazon offered was that the spot-price

services were subject to interruption initiated by the vendor.

This pricing mechanism allowed Amazon to ration its idle

computer resources based on client willingness-to-pay.

In spite of its innovative services and pricing design,

Amazon has more or less locked itself into a specific billing

cycle: it always charges clients by the hour. Others are

pricing their services in a more innovative way. For

example, in 2011, CloudSigma, a Zurich-based IaaS vendor,



announced a burst-pricing scheme that had a billing cycle as

short as five minutes. This is similar to the practices that

some telecom services firms used. They initially offered

monthly subscription plans only, and then started to offer

per-second billing. It is likely that the cost associated with

metering and billing in such a short interval is lower now. In

2012, PiCloud offered its clients even more value by

providing a usage-consolidation service. A client could use

1,000 compute instances, each active for one second only,

and then pay the price for using one compute instance for

1,000 seconds. This would have cost the client 1,000

instance hours via Amazon EC2. The emergence of

configurable cloud computing services offerings reflects

advances in managing virtualized computing resources.

Cloud vendors can now give more flexibility to their clients

than ever before.

4. Next-Generation Mechanisms

In the cloud computing services market today, we

observe the pull-and-lose mode of services delivery that

emerged after the pull-and-lock mode, which was the

default in the early days for cloud computing services.

This change in the services delivery mechanisms reflects

the needs of services vendors for more flexibility in selling

and re-allocating their resources, and their desire to

penetrate the IT services market. On one hand, a large

investment in infrastructure puts pressure on cloud services

vendors to recover their investment. On the other hand,

cloud computing services still are underused, despite their

capability to accommodate all kinds of client needs. We can

see this hold-up based on different concerns in adopting

cloud computing expressed by leaders in sectors that rely on

or heavily use IT services, such as financial services and

healthcare.

Cloud computing services are still in the process of

maturing and fast growth. New functionalities are being

added to existing services offerings, and totally new services

are being introduced. In the process, some clients will

naturally be resistant to trying out cloud computing services,

unless there are more options that mitigate both the

operational and financial risks for them. They also will need

more support to transition from legacy systems to the cloud,

a big challenge for many organizations.

We next will offer insights on new services provision

and the kinds of pricing mechanisms needed for the future

development of cloud computing services.

4.1. Quantity Discounted Pricing: Trade Cost Reduction with Demand Uncertainty

It is common in pricing strategy that a services vendor

uses quantity discounts to give buyers incentives to

purchase greater than the usual quantity. Research has

shown that second-degree price discrimination, especially

non-linear pricing strategies such as quantity discounts, is an

effective way for vendors to segment clients, gain market

power and obtain higher profit (Goldman et al. 1984,

Monahan 1984).

In the current cloud market, only storage service

vendors provide quantity discounts in the form of ladder-

shaped tariffs. They offer clients who use the services

bigger discounts on the unit prices. Other than that, quantity

discounts are rarely used in any other categories of cloud

services.

For example, for an Amazon EC2 on-demand standard

instance (small) running on Linux or Unix, the price is fixed

at $0.06 per instance-hour. There is no unit price difference

for a customer who runs 10 instance-hours versus one who

runs 10,000 instance-hours. For information goods, past

research indicates that usage-based pricing with a quantity

discount strategy is optimal when there are no transaction

costs (Maskin and Riley 1984). So it will be an option for

cloud vendors to use quantity discount pricing to incentivize

their clients to consume more services.

4.2. SLA-Based Services Delivery: Flexible Quality Guarantees, Costs and Compensation

Cloud services are experience goods: their tangible

features do not fully reveal their true value. Software

outsourcing contracts have a similar issue due to

information asymmetry (Dey et al. 2010). Enhancing the

completeness of the contract can potentially overcome this

problem, but at a high cost (Hart and Moore 1999). In the

practice of software outsourcing contracting, most vendors

specify the penalties applicable when delivery is delayed

(Whang 1992). Clients also have the right to terminate their

contracts, although this may be explicitly priced in a way

that the vendor can assure it will not be left with idle

capacity that it spent money to create (Benaroch et al. 2010).

In cloud computing, service level agreements (SLA)

serve as incomplete contracts between a client and a

services vendor, similar to other IT and grid computing

services (Li and Gilliam 2009, 2010; Li et al. 2010). Service

uptime guarantees are often stipulated in an SLA, like an

uptime guarantee of 99.9%, and terms specifying service

characteristics and penalties. In current practice, many IaaS

and PaaS vendors include both uptime guarantee and

penalty terms in their SLAs; few SaaS vendors do this

though.

All the vendors we reviewed, except for Salesforce,

provide uptime guarantees. And some IaaS vendors are

offering different uptime guarantees for different types of

services. For example, Amazon provides a 99.9% uptime

guarantee for S3, and a 99.95% uptime guarantee for EC2.

Rackspace provides a 99.9% uptime guarantee for storage

services and a 100% uptime guarantee for network

availability.

Most of the SLAs include uniform penalties that the

vendor must pay to all sorts of clients. Some issues are

ignored by this penalty design approach though. For



example, client attitudes toward the risk of services

downtime differ across applications and periods. Mission-

critical enterprise applications typically carry a much higher

cost for services downtime than non-critical applications

(Hiles 2005). To meet the diverse expectations, the vendor

may wish to consider including customized penalty terms

that are expected to outperform uniform penalties. It may be

mutually beneficial to provide functions for negotiating

penalty terms to satisfy different types of clients. With the

technology affordance of SLA-oriented resource

management of cloud vendors, future services delivery will

differentiate among and satisfy service requests based on the

desired utility of users, balancing risk concerns and service

costs.

4.3. Cloud Computing Services Market Evolution: Toward Financification

Next, we discuss how the cloud computing services

market has been changing,

and how the financification of

the cloud market will reshape

and guide its future evolution.

When usage-based on-

demand services were first

introduced in the market, they

nicely addressed the early

adopters’ uncertainty about

services quality, and to what

extent users needed cloud

resources. With the pay-as-

you-go mechanism, users

were subject to potential risk

of the unavailability of cloud

resources when they needed

them, and the potential for

price increases in the future.

With the financification of the

cloud, we expect an options

and futures contract market

for cloud services to emerge so users of on-demand services

will be able to hedge their risks. In the context of IaaS,

Rogers and Cliffs (2012b) proposed a pricing method that

combines options contracts with on-demand purchasing.

They show that options contracts can

provide clients with flexibility and cost-savings, as well

give the vendor improved server utilization.

Later, with the improvement of service quality and

adoption of cloud, reservation-based services were

introduced to users who wanted to avoid the uncertainty of

availability and price fluctuation. They were subject to the

risk of being locked in and not being fully satisfied with the

services, and they also may have over-estimated their cloud

resources needs. It is conceivable that the financification of

the cloud will also address these additional problems.

Exchange-like markets for cloud services will likely emerge

so users of reserved services can resell unutilized resources.

With the huge investments in cloud computing capacity

that have been made since Salesforce.com emerged, many

vendors now face the spectre of unutilized capacity due to

shifting supply and demand. Spot-price services were

introduced as a way for vendors to monetize their unutilized

capacity. The services were subject to interruption risk

though. So today, the logical next stage of evolution that

will occur in an increasingly financified market is the

further development of cloud brokerage services, which will

provide leverage for more economical use of spot-price

services. An example of this is Amazon’s 2013 launch of its

EC2 Reserved Instance Marketplace, in which users can

resell their unutilized balance of reserve instances to other

clients.

Looking toward the future, we expect to see further

development and evolution of the cloud computing services

market, related to its technical aspects, and the mechanisms

that structure the offering, pricing, purchase, and delivery of

services. See Figure 1 for a summary.

Although the figure may be misinterpreted as

suggesting that the cloud computing services market is an

integrated market, it actually is quite fragmented. There are

variations in how services are provided and consumed, and

how vendors compute what their clients will pay. Thus,

there is potential for a more efficient services market that

subsidizes new clients who have uncertainty about adoption,

use and workload management, and have to deal with

contingent conditions in their day-to-day operations.

In addition, the current marketplace has many

constraints on what resources are available and how they

can be traded between clients. Take Amazon’s EC2



Reserved Instance Marketplace as an example. Users can

resell their unutilized instances to other users. However, in

the current practice, the time available to consume a

reserved instance often is rounded to the nearest month, and

the marketplace will charge a service fee of 12% on the

price. These constraints mitigate the marketplace’s ability to

fluidly facilitate the trading of unutilized resources,

reducing their market liquidity. We believe that a future

financial market exchange-like marketplace will be needed

to optimize resource allocation and re-allocation to

effectively promote the adoption of cloud computing

services.

Finally, because IT services are subject to risk for

quality, cost, delivery, availability, it is likely that

insurance-related products will emerge for them in the

future (Accenture 2010, Cohen 2013). The establishment of

cloud services benchmarks (Yi et al. 2010) and the

maturation of actuarial analysis of cloud services risks will

support this future development, similar to what we have

seen with other IT services (Bardhan et al. 2010a, Gillam et

al. 2013, Kauffman and Sougstad 2008a). An example is

CloudInsure (www.cloudinsure. com), a Rye, New York-

based cloud computing insurance administrator that

specializes in IT services risk transference. Another is MSP

Alliance’s (www.mspalliance.com) managed services

insurance, which offers vendors indemnification against

liabilities from providing cloud services.

5. Research Directions for CloudComputing Mechanism Design

The range of issues that are related to the financification

of cloud computing deserve closer scrutiny. This can be

achieved by laying out a research agenda related to the

fundamental mechanism design issues for cloud computing

services. The issues identified are: the supply and demand

relationship and demand estimation; services offerings and

the structure of market prices; contracting, incentive-making,

and risk mitigation; third-party services and the value of

intermediated cloud services; and future innovations that

have the potential to reshape the entire market.

We begin with the first issue in this research direction on

the supply and demand of cloud computing services:

Research Direction 1 (Supply and Demand of

Cloud Computing Services in the Market).

Understanding the future functioning of the cloud

computing services market requires a basic knowledge

of how supply and demand interact with one another.

An important research direction to pursue involves

developing theoretical models and empirical studies

that will enable senior management to obtain more

knowledge of how supply and demand will play out in

the future, as the conditions, competition and

capabilities in the market change.

Since production and consumption of cloud computing

services are growing globally, it is important for researchers

to assist industry and government observers to establish

measurements and estimates of this area of services in the

economy. For example, Gartner’s estimates on cloud

services were recently expanded to US$180 billion by 2015

(Flood 2013). Seagate estimated that US$79 billion in cloud

computing hardware and equipment will ship by 2018. The

healthcare industry, for example, will use cloud computing

for 600 million images processed each year, and move from

only 15% today to 50% of diagnostic images stored in the

cloud by 2016 (Cox 2013).

These statistics are just the tip of a big iceberg though,

and other issues deserve attention (Woods 2014). Key

considerations are building models for national-level cloud

computing services growth – both in terms of what is

demanded and what is supplied. Some observers view rapid

growth of cloud computing as inevitable (Mason 2013,

Weinman 2009). Within specific industries, there exists the

issue of how different business processes and computing

workloads will be affected by cloud computing services

growth. How much money will be saved? Will downsizing

of organizations occur, shifting cloud computing demand?

And how long will it take to reap business value from cloud

computing, and what can be done to accelerate it?

Research Direction 2 (Services Offerings and the

Structure of Market Prices). There is a need to

pursue new bases for innovation related to cloud

computing services and market mechanism design,

and pricing and quality strategies. This will be a

fruitful research direction because it is necessary and

valuable to develop and test new business models,

pricing approaches, and mechanism design algorithms.

The demand for cloud computing services is driven by

the variety of the client needs and the quality of the services

that are offered, the price structures and price levels at

which they are offered, and the mechanism designs that

meter their delivery. This opens up a broad spectrum of

issues for research. For example, what future business

models are likely to be effective in supporting services that

will create higher demand? Will they be private-label

services with branded performance and unique qualities? Or

will they be more commoditized services whose provision is

driven by the cost leadership of large-volume, high service-

scope vendors? The financification of the cloud computing

services market is likely to be driven toward greater service

commoditization, thinner margins where the services are

provided without recognizable innovations that create value,

and increasing homogeneity in the functionality of the

services that are offered. There are opportunities to conduct

analytical and computational modeling research to assess

the relative performance of different kinds of mechanisms

under different assumptions about future growth and

demand. It will be especially useful to understand the extent

to which prices are dispersed or concentrated across

http://www.cloudinsure.com/

http://www.mspalliance.com/



different vendors who offer similar services.

Also, managers will find it beneficial to understand

more fully how to do effective statistical modeling of spot-

price cloud computing instances in public markets (Javadi et

al. 2011) and how spot prices change in these kinds of

market environments (Javadi et al. 2013). In future markets,

data analytics for different kinds of cloud computing

instances, and how demand and supply interact over time in

the presence of managerial choices on pricing for them, will

become strategic capabilities for cloud computing vendors

and their clients. Cloud computing services should be built

on vendor, client, and market informedness (Li et al. 2014),

so that it is possible to gauge prices under normal operating

conditions, as well as for peak loads in the market (Mattess

et al. 2010).

Research Direction 3 (Incentives, Contracting, and

Risk Mitigation). These issues motivate another

research direction that involves economic analysis of

incentives, assessment of information asymmetries for

SLA contracts, statistical analysis of the sources of

risk, and financial economic modeling of risk and

return.

Incentives, contracts and risk mitigation are different

facets of the same basic problem in cloud computing

services. Vendors need to design services and operate

mechanisms, supported by effective and balanced contracts,

so that it is possible for the client side to believe that there is

fair play with the sharing of the benefits and value arising

from services provision. For example, a vendor may grant

its client a real option involving the right but not the

obligation to switch from fixed-price contracted services to

spot-price instance purchases.

Benaroch et al. (2010, p. 319) argued: “When an IT

services vendor permits a client firm to exercise its

flexibility to switch sourcing modes, it essentially is offering

an opportunity to the client to achieve a marginal cost

advantage per IT service unit at varying demand levels.

There may be a loss of business for the vendor and a value

gain for the client. ... For the client, there will be the

irreversible switching costs of searching for a new vendor,

and whatever re-contracting costs arise in the process ...

Thus, switching between sourcing modes presents … a

trade-off …”

The research on IT services beyond cloud computing

services, especially for outsourcing and fixed services

contracting has explored a number of modeling perspectives

that are likely to be useful in the cloud computing context.

An example is the work of Alvarez and Stenbacka (2007).

They explored how to model IT sourcing and backsourcing

decision-making, so that it is possible to adjust the

contractual acquisition of services when services demand

falls in a flexible way, with a fair price charged by the

vendor. Techopitayakul and Johnson (2001) studied another

research context: application service provider (ASP)

operations. They modeled decision-making under

uncertainty for the value of the software that is used, the

number of users, and the overall usage level. They assessed

a vendor’s offering, including usage-based pricing versus a

flat subscription fee, back-sourcing to in-house computing,

and contract abbreviation. The research is especially

interesting for its inclusion of how learning effects from

service consumption for the vendor and client play into the

valuation of contract terms for IT services.

Research like this offers tremendous motivation to

researchers and managers to port some of these ideas from

statistical analysis, risk management, and financial

economics into cloud computing consulting and services

management practice (Bardhan et al. 2010). Due to the

information asymmetries that are present in cloud

computing, the vendor sees the market as a whole but the

client only knows its own demand (Stantchev and Tamm

2012). There are ample opportunities for process-perfecting

third-party information and data analytics providers to enter

the market, increase vendor and client informedness, and

improve their welfare (Knapper et al. 2011).

Research Direction 4 (Third-Party Services and the

Value of Intermediated Cloud Services). Digital

intermediation in the cloud computing services market

is a key target for mechanism design innovations. The

creation of new knowledge about the industrial

organization and optimization of IT services and cloud

computing intermediation will offer high scientific

payoffs and positive business returns on investment for

research that deals with the hard problems in this

context.

An intermediary’s position in a technology ecosystem

is determined by its viability and sustainability. The

intermediary will demonstrate viability when it creates

economic value for other participants in the ecosystem (e.g.,

buyers and suppliers in supply chain management, or clients

and vendors in cloud computing) in excess of the value

produced in its absence. This value difference must be

sufficient for the intermediary to earn a profit, so it will

maintain its incentive to participate and supply services

(Kauffman et al. 2010). The intermediary will demonstrate

sustainability when it is continuously able to create value

over time through the service transactions it supports to

generate profits that cannot be achieved without similar

market organization.

Beyond these basic observations though, how will we

know which intermediated solutions will work in the market,

and which will not? For example, will it be market structure,

competitive positioning, service pricing strategy, service

quality, or information security that will be the foremost

considerations? What kinds of models and business policies,

and what kinds of empirical evidence and business results

will make it clear what works and what does not? Cloud

computing technology platforms will do well when their

installed base of clients is high, the demand for their

services is relatively stable, and their growth trajectory



looks positive for the future. But investments in cloud

services intermediation, similar to every other facet of

business in a modern economy, will be subject to the

vagaries of competition, vendor strategy errors, sufficient

compatibility, and mistaken services and mechanism

designs. As a result, undertaking research that provides a

deeper understanding of how cloud computing services

firms need to design and operate their businesses also has

the potential to produce useful new knowledge about their

market performance.

Research Direction 5 (Future Innovations in

Technologies, Services and Infrastructures).

Research that identifies the basis for future

innovations in the cloud computing services market

will be of high value, especially if it is possible to

explain and predict how and why, and under what

circumstances, the changes are likely to be observed.

Technology and technology-based services forecasting

are among the most difficult tasks that business and

technology analysts need to undertake in the current

business environment. One perspective on future

innovations and markets for cloud computing services is

that technological innovation will be supply-led, with the

innovations on the vendors’ side, with market demand being

transformed in the process (Adomavicius et al. 2011).

Another related perspective is that cloud computing services

innovation will be demand-led: the more the market

demonstrates its willingness-to-pay for new services, the

harder will vendors work to innovate and drive profit from

the new business. So an important research direction for

cloud services is to study, forecast and analyze how future

innovations will take place and what are their possible

adoption and diffusion paths.

6. ConclusionThis article offers useful contributions for research and

practice. On the research side, it shares a new perspective

for the organization of cloud computing services markets,

supply and demand for services, market mechanisms and

pricing approaches, contracts and incentives, and third-party

intermediation. The cloud computing services market

exhibits key features of financial markets, including: (1) bid

and ask prices for services; (2) spot, forward and futures

prices; (3) services liquidity and services depth; and (4)

opportunities to apply hedging and risk management. We

illustrated this with spot prices and dynamic prices, with

cloud computing insurance, with brokered cloud services,

and other compelling examples.

Our central goal in this article was to demonstrate a

practice-led set of scientific observations that can be

interpreted from the perspective of relevant theory from

financial economics – and its ties to related markets. We are

pleased to offer the financification of the cloud computing

services market contribution to managerial understanding of

a leading example of the dramatic changes made possible

due to a revolution in technology – computing in the cloud –

and the continuing evolution of the IT services practices that

have occurred around it. Through the lens of financification

that we have offered, managers and consultants who are

trying to understand current and future markets for cloud

computing will be empowered to make more confident

predictions and thoughtful explanations for what is to come.

A number of future challenges based on our perspective

are likely. How far will cloud computing services go in

terms of the extent of financification we will see?

Technological, economic, business, and competitive factors

are all likely to play a role in the future. We have not

answered all of the questions that an informed group of

researchers and practitioners are likely to ask. Nevertheless

we have offered a practice-led view of what is likely to

happen in a marketplace that is subject to the inexorable

forces that all financial markets have experienced – as we

have seen in other sectors with perishable services,

including the hospitality, air travel, temporary labor services,

and television and radio entertainment sectors.

7. AcknowledgmentsThe authors acknowledge Singapore’s Agency for

Science Technology and Research (A*STAR) for its

generous support of this research, as well as the following

individuals at the Institute for High Performance Computing

(IHPC) for their input: Terence Hung, Li Xiaorong, Henry

Palit, and Qin Zeng. We benefited from the comments of the

International Journal of Cloud Computing editors, Hemant

Jain of the University of Wisconsin, Milwaukee, and Rong

Chang, at IBM’s T.J. Watson Research Center, and an

anonymous member of the journal’s editorial board. Huang

Jianhui thanks the Ph.D. Program in the School of

Information Systems at Singapore Management University,

and Singapore’s Ministry of Education for doctoral

fellowship funding from 2009 to 2013. In addition, Yang

Yinping benefited from the funding provided by her

Independent Investigator research grant at A*STAR.

8. References451 Research. (2013) Market monitor: cloud computing. White paper, New

York, NY.

Accenture (2010) How cloud computing will transform insurance. White

paper, New York, NY.

Adomavicius, G., Bockstedt, J.C., Gupta, A., Kauffman, R.J. (2008)

Making sense of technology trends in the information technology landscape:

a design science approach. MIS Qtrly., 32(4), 779-809.

Adomavicius, G., Bockstedt, J.C., Gupta, A. (2012) Modeling supply-side

dynamics in IT components, products, and infrastructure: an empirical

analysis using vector autoregression. Info. Sys. Res., 23(2), 397-417.

Alexandrov, A., Lariviere, M. A. (2012) Are reservations recommended?

Mfg. Serv. Ops. Mgmt., 14(2), 218-230.

Alvarez, L.H., Stenbacka, R. (2007) Partial outsourcing: a real options perspective. Intl. J. Ind. Org., 25(1), 91-102.



Bardhan, I.R., Demirkan, H., Kannan, P.K., Kauffman, R.J., Sougstad, R.

(2010a) An interdisciplinary perspective on IT services management and service science. J. Mgmt. Info. Sys., 26(4), 13-64.

Bardhan, I., Kauffman, R.J., Naranpanawe, S. (2010b) IT project portfolio

optimization: a risk management approach to software development governance. IBM J. Res. Dev., 54(2), 1-18.

Benaroch, M., Dai, Q., Kauffman, R.J. (2010) Should we go our own way?

backsourcing flexibility in IT services contracts. J. Mgmt. Info. Sys., 26(4), 317-358.

Birman, K., Chockler, G., van Renesse, R. (2009) Towards a cloud

computing research agenda. ACM SIGACT News, 40(2), 68-80.

Cartlidge, J., Clamp, P. (2014) Correcting a financial brokerage model for

cloud computing: closing the window of opportunity for commercialization.

J. Cloud Comp.: Adv., Sys., Appl., 3(2), 1-20.

Cloud Industry Forum (2011, 2012) UK cloud adoption and trends 2011,

2012. White paper, High Wycombe, UK.

Clearwater, S.H., Huberman, B. (2005) Swing options: a mechanism for

pricing IT peak demand. In Proc. 11th Intl. Conf. Comp. Econ., Wash., DC.

Cohen, R. (2013) New cloud computing insurance attempts to solve cloud

liability concerns for service providers, Forbes, April 24.

Cox, R. (2013) Infographic: cloud’s growing footprint in storage.

SiliconAngle.com, June 27. Available at

siliconangle.com/blog/2013/06/27/ infographic-clouds-growing-footprint-in-storage.

Dai, Q., and Kauffman, R.J. (2004) Partnering for perfection: an economics

perspective on B2B electronic market strategic alliances. In K. Tomak (ed.), Economics, IS and E-Commerce, Idea Group Publishing, Harrisburg, PA,

43-79.

Das, S., Du, A., Gopal, R. and Ramesh, R. (2011) Risk management and optimal pricing in online storage grids. Info. Sys., Res. 22(4), 756-773

Deelman, E., Singh, G., Livny, M., Berriman, B., Good, J. (2008) The cost

of doing science on the cloud: the montage example. In Proc. 2008 ACM/ IEEE Conf. on Supercomp., IEEE Comp. Soc. Press, Washington, DC.

Demirkan, H., Kauffman, R.J., Vayghan, J.A., Fill, H.G., Karagiannis, D.,

Maglio, P. P. (2008) Service-oriented technology and management: perspectives on research and practice for the coming decade. Elec. Comm.

Res. Appl., 7(4), 356-376.

Dey, D., Fan, M., Zhang, C. (2010) Design and analysis of contracts for software outsourcing. Info. Sys. Res., 21(1), 93-114.

Durkee, D. (2010) Why cloud computing will never be free. Comm. ACM,

53(5), 62-69.

Elliott, S. (2012) PiCloud and Princeton Consultants win the first Amazon

EC2 spotathon. Amazon Web Services Blog, December 3, 2012.

Fishburn, P. C., Andrew, M. O., Ryan, C. S. (2000) Fixed fee versus unit pricing for information goods: competition, equilibria, and price wars. In D.

Hurley, B. Kahin, and H. Varian (eds.), Internet Publishing and Beyond:

The Economics of Digital Information and Intellectual Property, 167-189,

MIT Press, Boston, MA.

Flood, G. (2013) Gartner tells outsourcers: embrace cloud or die.

InformationWeek, July 31.

Garfinkel, S. (2011) The cloud imperative. Tech. Rev., October 3.

Gartner. (2012) Predicts 2013: cloud services brokerage. Gartner Research,

Stamford, CT, December 18.

Goldman, M.B., Leland, H.E., Sibley, D.S. (1984) Optimal non-uniform

prices. Rev. Econ. Stud., 51(2), 305-319.

Hart, O., Moore, J. (1999) Foundations of incomplete contracts. Rev. Econ. Stud., 66(1), 115-138.

Hiles, A. (2005) Five nines: chasing the dream. ContinuityCentral.com,

Huddersfield, UK, November 15.

Huang, J. (2013) Pricing strategy for cloud computing services.

Unpublished doctoral thesis, Singapore Mgmt. Univ., Singapore, November.

Huang, J., Ma, D., Kauffman, R.J. (2013a) Damaged services: why

interruptible spot-priced cloud computing services are worthwhile to offer with uninterruptible reserved services. In Proc. 2013 Workshop on IT and

Systems, Milan, Italy, December.

Huang, J., Yang, Y., Shang, R., Kauffman, R.J. (2013b) An empirical study of a hybrid pricing mechanism for brokered cloud services. In Proc. 2013

Workshop on IT and Sys., Milan, Italy, December 2013.

Jackson, J. (2014) Price war! Amazon cuts cloud costs to counter Google. Computerworld, March 28.

Javadi, B., Thulasiram, R., Buyya, R. (2011) Statistical modeling of spot

instances in public cloud environments. In Proc. 4th IEEE/ACM Intl. Conf. on Util. Cloud Comp., IEEE Comp. Soc. Press, Wash/, DC, December.

Javadi, B., Thulasiram, R., Buyya, R. (2013) Characterizing spot price

dynamics in public cloud environments. Fut. Gen. Comp. Sys., 29(4), 988-

999.

Kauffman, R.J., Li, T., van Heck, E. (2010) Business network-based value

creation in electronic commerce. Intl. J. Elec. Comm., 15(1), 113-143.

Kauffman, R.J., Ma, D. (2013) Cost efficiency strategies in the software-

as-a-service market: modeling and related implementation issues. In Proc.

2013 Conf. on Econ. of Grids, Clouds, Sys. and Serv., Zaragosa, Spain, September. Lecture Notes in Computer Science, Springer, New York, NY.

Kauffman, R.J., Sougstad, R. (2008a) Risk management of IT services

portfolios. J. Mgmt. Info. Sys., 25(1), 17–48.

Kauffman, R.J., Sougstad, R. (2008b) Value-at-risk in services-oriented

systems and technology investments: a framework for managing project portfolio uncertainties. Intl. J. Serv. Sci., 1(3), 225–246.

Knapper, R., Blau, B., Conte, T., Sailer, A., Kochut, A., Mohindra, A.

(2011) Efficient contracting in cloud service markets with asymmetric information: a screening approach. In Proc. IEEE 13th Conf. Comm. and

Enterpr. Comp., IEEE Comp. Soc. Press, Washington, DC, 2011, 236-243.

Lauchlan, L. (2013) Gartner: Amazon is still the one to beat in IaaS. Diginomica.com, London, UK.

Leavitt, H.J., Whisler, T.L. 1958. Management in the 1980's. Harvard Bus.

Rev., November, 41-48.

Leong, L., Chamberlin, T. (2010) Magic quadrant for cloud infrastructure

as a service and web hosting. Gartner Research. Stamford , CT.

Leong, L. Chamberlin, T. (2011) Magic quadrant for public cloud infrastructure as a service. Gartner Research. Stamford , CT.

Leong, L., Toombs, D., Gill, B., Petri, G., Haynes, T. (2012) Magic

quadrant for cloud infrastructure as a service. Gartner Research. Stamford , CT.

Leong, L., Toombs, D., Gill, B., Petri, G., Haynes, T. (2013) Magic

quadrant for cloud infrastructure as a service. Gartner Research. Stamford , CT.

Li, B., Gillam, L. (2009) Towards job-specific service level agreements in

the cloud, cloud-based services and applications. In Proc. 5th IEEE E-Science Intl. Conf., Oxford, UK

Li, B., Gillam, L. (2010) Grid service level agreements using financial risk

analysis techniques. In N. Antonopoulos, G. Exarchakos, M. Li and A. Liotta (eds.), Handbook of Research on P2P and Grid Systems for Service-

Oriented Computing: Models, Methodologies and Applications, IGI Global,

New York, NY, 686–710.

Li, B., Gillam, L., O'Loughlin, J. (2010) Towards application-specific

service level agreements: experiments in clouds and grids. In N.

Antonopoulos and L. Gillam (eds.), Cloud Computing: Principles, Systems and Applications, Springer-Verlag, London, UK.

Li, T., Kauffman, R.J., van Heck, E., Vervest, P, Dellaert, B. (2014)

Information strategy and consumer informedness. Info. Sys. Res., in press.

Marston, S., Li, Z., Bandyopadhyay, S., Zhang, J., Ghalsasi, A. (2011)

Cloud computing: the business perspective. Dec. Supp.Sys. 51(1), 176-189.



Maskin, E., Riley, J. (1984) Monopoly with incomplete information. RAND

J. Econ., 15(2), 171-196.

Mason, K. (2013) 3 reasons for the inevitability of cloud computing. Insur.

and Tech., July 19.

Masuda, Y. Whang, S. (2006) On the optimality of fixed-up-to tariff for telecommunications service. Info. Sys. Res., 17(3), 247-253.

Mattess, M., Vecchiola, C., Buyya, R. (2010) Managing peak loads by

leasing cloud infrastructure services from a spot market. In 12th IEEE Intl. Conf. on High Perf. Comp. and Comm., IEEE Comp. Soc. Press,

Washington, DC, 180-188.

Mazzucco, M., Dumas, M. (2011) Achieving performance and availability guarantees with spot instances. In Proc. 13th Intl. Conf. High Perf. Comp.

Comm., IEEE Comp. Soc. Press, Washington, DC, 296-303.

Meinl, T., Anandasivam, A., Tatsubori, M. (2010) Enabling cloud service reservation with derivatives and yield management. In Proc. 12th Intl. Conf.

Comm. Enterprise Comp., IEEE Comp. Soc. Press, Wash., DC, 150-155.

Monahan, J.P. (1984) A quantity discount pricing model to increase vendor

profits. Mgmt. Sci. 30(6), 720-726.

Newcomer, E., Lomow, G. (2004) Understanding SOA with Web Services.

Addison-Wesley Professional, Reading, MA.

Nichols, B. (2014) Can Microsoft or IBM catch Amazon.com in the cloud?

The Motley Fool, May 15.

Palankar, M., Ripeanu, M., Garfinkel, S. (2008) Amazon S3 for science grids: a viable Solution? In Proc. 2008 Intl. Workshop on Data-Aware

Distrib. Comp., ACM Press, New York, NY, 55-64.

Panettieri, J. (2013) Cloud services brokerage list and FAQ. TalkinCloud.com, March 15.

Perry, G. (2010) Shopping the cloud: pricing (or apples and oranges in the cloud). Available at gevaperry.typepad.com.

Png, I. P. L., Wang, H. (2010) Buyer uncertainty and two-part pricing:

Theory and applications. Mgmt. Sci., 56(2), 334-342.

Rimal, B. P., Choi, E., Lumb, I. (2009) A taxonomy and survey of cloud

computing systems. In Proc. 5th

Intl. Joint Conf. on INC, IMS and IDC,

IEEE Comp. Soc. Press, Washington, DC, 44-51.

Rogers, O., Cliff, D (2010) The effects of truthfulness on a computing resource options market. In Proc. 2nd Ann. Intl. Conf. on Adv. Distrib.

Parallel Comp., 2, 330–335.

Rogers, O., Cliff, D. (2012a) A financial brokerage model for cloud computing. J. Cloud Comp.: Adv., Sys., Appl., 1(2), 1-12.

Rogers, O, Cliff, D. (2012b) Options, forwards and provision-point

contracts in improving cloud infrastructure utilization. J. Cloud Comp.: Adv., Sys., Appl.,, 1(21), 1-22.

Rogers, O, Cliff, D. (2012c) Forecasting demand for cloud computing

resources: an agent-based simulation of a two-tiered approach. In Proc. 4th Intl. Conf. Agents Artif. Intell., Vilamoura, Algarve, Portugal, February.

Sandholm, T., Lai, K., Andrade, J., Odeberg, J. (2006) Market-based

resource allocation using price prediction in a high performance computing grid for scientific applications. In Proc. 15th Intl. Symp. on High Perf.

Comp., 1, 132-143.

Shang, R., Huang, J., Yang, Y., Kauffman, R.J. (2012) Exploring spot market users' willingness-to-pay for service-level agreements in cloud

computing. In Proc. 2012 Workshop on E-Bus., Orlando, FL, December.

Shang, R., Huang, J., Yang, Y., Kauffman, R.J. (2013) Analyzing the

impact of cloud services brokers on cloud computing markets. In Proc.

2013 Pacific Asia Conf. on Info. Sys., Jeju, Korea, June.

Shapiro, C. (1983) Optimal pricing of experience goods. Bell J. Econ.,

14(2), 497–507.

Stantchev, V., and Tamm, G. (2012) Reducing information asymmetry in cloud marketplaces. Human Capital and Info. Tech. Prof., 3(4), 1-10.

Sundararajan, A. (2004) Nonlinear pricing of information goods. Mgmt.

Sci., 50(12), 1660-1673.

Techopitayakul, D., Johnson, B. (2001) ASP-based software delivery: a

real options analysis. 5th Ann. Conf. on Real Options Theory Meets

Practice, University of California, Los Angeles, July 13–14.

Transparency Market Research (2011) Cloud computing services market:

global industry size, share, trends, analysis and forecasts, 2012-2018.

Albany, NY, 2011.

Vaquero, L. M., Rodero-Merino, L., Caceres, J., and Lindner, M. (2008) A

break in the clouds: towards a cloud definition. ACM SIGCOMM Comp.

Comm. Rev., 39(1), 50-55.

Varian, H.R. (1995) Pricing information goods. In Proc. Scholarship in the

New Info. Envir. Symp., Harvard Law School, Boston, MA.

Whang, S. (1992) Contracting for software development. Mgmt. Sci., 38(3), 307-324.

Weinhardt, C., Anandasivam, A., Blau, B., Borissov, N., Meinl, T.,

Michalk, W., Stößer, J. (2009) Cloud computing: a classification, business models, and research directions. Bus. Info. Sys. Eng., 1(5), 391-399.

Weinman, J. (2009) Mathematical proof of the inevitability of cloud

computing. Cloudonomics.com, November 9.

Williams, D., Jamjoom, H., Weatherspoon, H. (2012) The xen-blanket:

virtualize once, run everywhere. In Proc.7th ACM Eur. Conf. Comp. Sys.,

ACM Press, New York, NY, 113-126.

Woods, J. (2014) 20 cloud computing statistics every CIO should know.

SiliconAngle.com, January 27.

Wu, F., Zhang, L., Huberman, B.A. (2008) Truth-telling reservations. Algorithmica, 52(1), 65-79.

Yi, S., Kondo, D., Andrzejak, A. (2010) Reducing costs of spot instances

via checkpointing in the Amazon Elastic Compute Cloud. In 3rd IEEE Intl. Conf. Cloud Comp., IEEE Comp. Soc. Press, Washington, DC, 236-243.

Yeo, C.S., Buyya, R. (2006) A taxonomy of market-based resource management systems for utility-driven cluster computing. Softw., Pract.

and Exper., 36(13), 1381-1419.

Yeo, C.S., Buyya, R. (2007) Pricing for utility-drive utility management and allocation in clusters. Intl. J. High Perf. Comp. Appl., 21(4), 405-418.

Zhou, M., Zhang, R., Zeng, D., Qian, W. (2010) Services in the cloud

computing era: a survey. In Proc. 4th Intl. Universal Comm. Symp., IEEE Comp. Soc. Press, Washington, DC, 40-46.

Appendix We reviewed 19 cloud computing services vendors that

offer 27 types of services, including 15 IaaS, 6 PaaS, 7 SaaS,

and 3 cloud services brokerage services. (See Table A1.)

Table A1. Cloud services vendors selected for this study

TYPE NAME VENDOR URL

IaaS

Amazon EC2 On-Demand Instance Amazon goo.gl/fEzlD

Amazon EC2 Reserved Instance Amazon goo.gl/fEzlD Amazon EC2 Spot Instance Amazon goo.gl/fEzlD

Amazon S3 Amazon goo.gl/BcG1n

Infrastructure-as-a-Service Alatum goo.gl/w0B9d

Enterprise VM Hosting nGrid goo.gl/ihEuI

CloudSigma CloudSigma goo.gl/20mev

Cloud Servers GoGrid goo.gl/6Z4bO Joyent Cloud Joyent goo.gl/xkcwA

Rackspace Cloud Servers RackSpace goo.gl/cSZEA

FlexiScale public cloud FlexiScale goo.gl/I9rwE

IaaS ProfitBricks goo.gl/weH6L

Google Compute Engine Google goo.gl/RehH4 HP Cloud HP goo.gl/ZV3Fo

CloudLayer Computing SoftLayer goo.gl/8VKj3

PaaS

Google App Engine Google goo.gl/RLtG8

CloudFare CloudFare goo.gl/Jqt9Q

Force.com Salesforce goo.gl/Lo8jj

Microsoft Windows Azure Microsoft goo.gl/rDwP5 Microsoft SQL Azure Microsoft goo.gl/rDwP5

Amazon Beanstalk Amazon goo.gl/Tpu0E

SaaS Service Cloud Salesforce goo.gl/7sjJf

Sales Cloud Salesforce goo.gl/PkojZ



Chatter Salesforce goo.gl/g7Lqq

Jigsaw Salesforce bit.ly/g6i6Um

Google App for Business Google goo.gl/kxkeZ NetSuite Financial Management NetSuite goo.gl/dtqTH

Office 365 Microsoft goo.gl/Au3tM

Cloud Brokerage

PiCloud Public Cloud PiCloud goo.gl/JGbKT

RightScale Cloud Comm. Edition RightScale goo.gl/PDDwl

Integration Cloud Boomi goo.gl/oO3lz

Note: All information was collected from the vendors’ official websites in October

2013, and updated as of October 2014. Vendors may have changed their website

structures and content related to their services pricing after we completed our update.

Authors

Robert J. Kauffman is Associate Dean

(Research), Deputy Director of the Living

Analytics Research Center, and Professor of IS

at Singapore Management University. His

graduate degrees are from Cornell and Carnegie

Mellon. He is an expert in technology and

strategy, financial IS, IT services, and the

economics of IT.

Dan Ma received her Ph.D. from the Simon

School of Business at the University of

Rochester. She is an Assistant Professor of IS

and Management at the School of Information

Systems, Singapore Management University.

Her expertise is in economics and IS, IT

services, cloud computing, and game theory.

Richard Di Shang is an Assistant Professor of

MIS at the School of Business, Public Admin.

and Info. Sciences, Long Island University

Brooklyn. He received his Ph.D. in Business

(IS) from City University of New York. He

previously was a Scientist at Singapore’s

Agency for Science, Technology and Research

(A*STAR). He applies experiments to test

insights from economics for IT services,

information goods, and e-marketplaces.

Yinping Yang is a Scientist and Capability

Group Manager at the Institute of High

Performance Computing (IHPC), A*STAR,

Singapore. She is affiliated with the School of

Information Systems of Singapore Management

University as an Adjunct Faculty. She received

her Ph.D. in IS from National University of

Singapore. Her research brings design science

and behavorial science to the study of electronic

negotiation systems, social networking sites and

IT services.

Jianhui Huang is a Senior Research Analyst at

the Corporate Executive Board Asia Pte. Ltd.

His Ph.D. degree is from the School of

Information Systems, Singapore Management

University. His research interests include the

economics of IT, business model in IT services,

the impact of cloud computing, and IT value co-

creation.

http://goo.gl/JGbKT



IMPACTS OF MULTI-CLASS OVERSUBSCRIPTION ON REVENUES

AND PERFORMANCE IN THE CLOUD Rachel A. Householder and Robert C. Green

Bowling Green State University [email protected]

Abstract Rising trends in the number of customers turning to the cloud for their computing needs has made effective resource allocation imperative for cloud service providers. In order to maximize profits and reduce waste, providers have started to explore the role of oversubscribing cloud resources. However, the benefits of oversubscription in the cloud are not without inherent risks. This work attempts to unveil the different incentives, risks, and techniques behind oversubscription in a cloud infrastructure. The discrete event simulator CloudSim is used to compare the generated revenue and performance of oversubscribed and non-oversubscribed datacenters. The idea of multi-class service levels used in other overbooked industries is implemented in simulations modeling a priority class of VMs that pay a higher price for better performance. Three simulations are implemented. The first two compare the results of different VM allocation policies without VM migration. The third implements VM migration in an oversubscribed, power-aware datacenter. Results show that oversubscription using multi-class service levels has the potential to increase datacenter revenue, but the benefit comes with the risk of degraded QoS, especially for non-priority customers. Keywords: cloud computing; oversubscription; resource allocation; revenue; CloudSim

__________________________________________________________________________________________________________________

1. INTRODUCTION Utilizing cloud services to meet computing needs is a

concept that is rapidly gaining in popularity as ``in the

cloud'' has become a catchphrase in mainstream society.

According to NIST, ``Cloud computing is a model for

enabling convenient, on-demand access to a shared pool of

configurable computing resources (e.g. networks, servers,

storage applications, and services) that can be rapidly

provisioned and released with minimal management effort

or service provider interaction'' (I. S. Moreno & Xu, 2012).

The resources offered by cloud service providers (known

from here on out as CSPs) can be classified under one of

three service models: Software as a Service (SaaS), Platform

as a Service (PaaS), and Infrastructure as a Service (IaaS).

On the lowest level, IaaS provides access to resources

such as servers, storage, hardware, operating systems, and

networking. Unlike SaaS and PaaS, the customer has the

ability to configure these lower-level resources. IaaS has

become increasingly popular (Wo, Sun, Li, & Hu, 2012) as

it allows customers, especially companies and organizations,

to outsource their IT needs. These companies simply request

the computing resources they desire (Ghosh & Naik, 2012)

and CSPs provide those resources with a high level of

assurance of their reliability and availability. The

outsourcing of computing resources has several benefits for

customers. Services are offered on a pay-as-you-go basis,

allowing customers to pay only for the resources they use.

CSPs handle much of the IT infrastructure management

tasks that customers once had to support themselves.

Additionally, data and services in the cloud are widely

available through the Internet via a variety of devices.

With all of these benefits, the number of customers

looking to migrate to the cloud is on the rise and the ability

of CSPs to efficiently host as many clients as possible on a

fixed set of physical assets will be crucial to the future

success of their business (Williams et al., 2011). Cloud

services are supplied to clients through virtualization

creating the impression that each user has full access to a

seemingly unlimited supply of resources. In reality, a single

physical machine must divide its finite set of resources

amongst multiple virtual machines (VMs). Much research

has been dedicated to developing optimum resource

allocation strategies in a non-overbooked cloud. For

instance, Feng et al. (2012a) uses concepts of Queuing

Theory to maximize revenues and increase resource

utilization levels while adhering to Service Level

Agreement (SLA) constraints and He et al. (2012) employs

a multivariate probabilistic model to optimize resource

allocation. While these strategies have been shown to

improve utilization, a high percentage of resources still sit

idle at any given time (Toms & Tordsson, 2013). As a result,

oversubscription of cloud services has become an appealing

solution to further optimize cloud efficiency. Much work

has been done to investigate cloud oversubscription without

specifically measuring the impact on revenues, and little

work investigates the affects of priority classes on revenues

in an oversubscribed cloud. To address this need, this paper

contributes by using the simulation tool CloudSim to

investigate how adding a priority class of VMs to an

oversubscribed datacenter affects VM debt and QoS. The

rest of the paper is organized as follows: Section 2 provides

a brief literature review; Section 3 provides an overview of

the concept of oversubscription; Section 4 briefly discusses

the economics of cloud computing; Section 5 compares

doi: 10.29268/stcc.2014.2.1.2



cloud computing to other industries that overbook resources;

Section 6 describes the motivations and tools used for the

simulations; Section 7 shows Experiment 1 and its results;

Section 8 discusses Experiment 2 and its results; Section 9

Compares Experiments 1 and 2; Section 10 overviews

Experiment 3 and its results Section 11 reviews some

limitations and future work; finally, Section 12 concludes.

2. LITERATURE REVIEWThis chapter reviews the major research literature in the

area of oversubscribed cloud computing as it pertains to this

study. For organizational purposes, the research in this area

has been generally split into nine different categories:

memory oversubscription, bandwidth oversubscription,

CPU oversubscription, energy efficiency, optimal resource

allocation, multi-class users, maximizing revenue in the

cloud, measuring performance impact, and safe overbooking.

2.1 MEMORY OVERSUBSCRIPTION

Of all the computing resources, the ability to effectively

oversubscribe memory provides a challenging problem for

researchers (Hines et al., 2011). In their overview of

oversubscription, Wang et al. (2012) show that both

quiescing and live migration can be used independently to

remediate overload in a memory oversubscribed cloud. As a

continuation of this work, the authors quiesce VMs with

lower work values first to allow VMs with higher work

values to perform normally and to reduce the number of

migrations needed. Williams et al. (2011) present

Overdriver in an attempt to reduce performance degradation

that comes with memory overload. They use a new method

called cooperative swap for transient overloads and VM

migration for sustained overloads. Hines et al. (2011)

present a framework called Ginkgo that can be used to

automate the redistribution of memory among VMs. Ginkgo

uses an application performance profile along with other

constraints, such as performance level thresholds established

by the SLA, as criteria for memory allocation decisions.

2.2 BANDWIDTH OVERSUBSCRIPTION Bandwidth is another resource that can be

oversubscribed. As the cloud gains in popularity, network

traffic continues to increase which can lead to bottlenecks

and latency (Breitgand & Epstein, 2012). Jain et al. (2012)

focus on remediating overload using VM migration in a tree

topology data center network. They concurrently consider

the load constraints of servers and the traffic capacity

constraints of the tree edges to develop algorithms that

relieve as many hot servers as the network will allow. They

do so by migrating a portion of their VMs to cold servers.

Guo et al. (2013) focus on optimizing the distribution of

network traffic and throughput using a load balancing

technique in a fat-tree data center network.

Breitgand and Epstein (2012) attempt to improve

bandwidth utilization by applying concepts of the Stochastic

Bin Packing problem (SBP). They suggest three algorithms

to allocate VMs belonging to a single SLA class so that the

probability of meeting bandwidth demands is at least the

minimum calculated value.

Wo et al. (2012) use a greedy-based VM placement

algorithm that is traffic-aware to improve the locality of

servers running the same application. They also introduce a

revenue model designed to determine the overbooking ratio

that will maximize profits by reducing the number of SLA

violations.

2.3 CPU OVERSUBSCRIPTION Zhang et al. (2012) consider a cloud that has

overcommitted its processor power. They introduce a VM

migration algorithm called Scattered that focuses on

pinpointing the best VMs for migration based on evaluation

of their workload degree of correlation. Using two

variations of migration, standard migration and VM swap,

Scattered is shown to limit the number of migrations

required to relieve overload and can tolerate larger

overcommit ratios.

2.4 ENERGY EFFICIENCY Moreno and Xu (2012) focus on the value of

overbooking for greener computing. Using a multi-layer

Neural Network, they attempt to predict resource usage

patterns by studying historical data and using the results to

develop optimal allocation algorithms. When overload does

occur a Largest VM First approach is used to select VMs for

migration. Moreno and Xu (2011) attempts to use cloud

oversubscription to promote energy efficiency in real-time

cloud datacenters. Their approach uses customer utilization

patterns to more safely overallocate resources and considers

SLA and energy consumption in the process.

2.5 OPTIMAL RESOURCE ALLOCATION Ghosh and Naik (2012) evaluate the history of CPU

usage of VMs to establish a one-sided tolerance limit that

represents a threshold of risk based on the probability of

overload and SLA violations. They propose that these

analytics can be applied to develop a smart risk-aware

resource allocation tool that can be used to place incoming

requests.

Tomas and Tordsson (2013) propose a new framework

for VM placement. An admission control module

determines if a new job request can be deployed.

Applications are monitored and profiled to help predict their

behavior and predominant type of resource usage (i.e. bursty

CPU or I/O). A smart overbooking scheduler then

determines the best location for the application to be

deployed.

Breitgand et al. (2012) approach optimizing resource

allocation by creating the extended SLA (eSLA) that



provides a probability for successful VM launch in addition

to an availability probability. The eSLA allows for more

aggressive overcommit ratios without increasing the number

of eSLA violations.

2.6 MULTI-CLASS USERS Patouni et. al (2014) briefly discuss introducing a

fairness factor for allocating physical resources in the cloud

amongst multiple classes of users. However, their concept is

not fully developed and they do not consider

oversubscription.

2.7 MAXIMIZING REVENUE IN THE CLOUD Feng et al. (2012b) considers maximizing cloud

revenues by implementing dynamic resource allocation. The

allocation is based on SLA values and considers other

factors such as pricing mechanisms, arrival rates, service

rate, and available resources. However, it does not consider

oversubscription in their model nor does it consider multiple

class levels.

2.8 MEASURING PERFORMANCE IMPACT Hoeflin & Reeser (2012) attempts to quantify the impact

overbooking has on cloud performance. The relationship

between overbooking level and VM utilization is considered

as well monitoring parameters such as CPU utilization and

throughput. These are used to suggest reasonable levels of

oversubscription based on SLA constraints.

2.9 SAFE OVERBOOKING Luis (2014) explores safe overbooking methods that use

the concept of application brownout to help mitigate

performance degradation caused by overload. Tomas &

Tordsson (2014) attempts to implement a risk-aware

overbooking framework. Fuzzy logic is used to measure the

risk of overbooking decisions to help identify the

appropriate level of overbooking.

While all of these works consider topics related to

individual aspects of this research, none of them look at the

impact of overbooking on both performance and revenues in

a multi-class system. This is the primary contribution of this

work.

3. OVERVIEW OF OVERSUBSCRIPTION To oversubscribe a resource means to offer more of that

resource than there is actually capacity for under the

assumption most customers will not actually consume their

entire portion. The goal is to diminish the sum of unutilized

resources and thus increase profits. This section provides a

basic summary of oversubscription in other industries as

well as in the cloud. It also discusses the risks inherent to

employing oversubscription in a cloud computing system.

3.1 OVERSUBSCRIPTION IN OTHER INDUSTRIES

Oversubscribing resources is not a concept unique to

cloud computing. Hotels overbook rooms. When there are

less no-shows than expected, customers must downgrade

rooms or go to another hotel and are compensated for their

denial of services (Noone & Lee, 2011). The healthcare

industry overbooks doctors' time which can lead to

increased waiting time for customers and physician

overtime costs when all patients arrive for their

appointments (Zacharias & Pinedo, 2013). Airlines have

been known to overbook seats (Coughlan, 1999) and cargo

space (Singhaseni, Wu, & Ojiako, 2013). When more

customers show up than predicted, they are typically moved

to another flight which causes delays and other

inconveniences for the customer. To better accommodate

passengers when this occurs, airlines will sometimes form

alliances with their competitors to expand the number of

flights bumped customers can be moved to (Chen & Hao,

2013). The impact of this alliance can be taken into

consideration when developing overbooking policies.

Additionally, the class system can be taken into

consideration with first-class flights typically having lower

levels of overbooking than coach (Coughlan, 1999).

3.2 OVERSUBSCRIPTION IN CLOUD COMPUTING Like the lodging, healthcare, and airline industries, cloud

computing provides ample opportunity for oversubscription.

In recent years, companies have started to notice that they

are only utilizing a small portion of their available resources

(resources being memory, CPU, disk, and bandwidth

capacity). In fact, CSPs on average use only 53% of the

available memory, while CPU utilization is normally only at

40% in most datacenters (Toms & Tordsson, 2013).

Simulations done to study the CPU utilization patterns of

individual VMs have shown that 84% of VMs reach their

maximum utilization levels less than 20% of the time

(Ghosh & Naik, 2012). The underutilization of resources is

a major concern to most CSPs considering the amount of

resources required to run and maintain large datacenters.

Datacenters require a great deal of infrastructure that

consumes large amounts of power (Moreno & Xu, 2012).

Oversubscription helps to maximize resource utilization

which can in-turn help to reduce these costs and increase

profitability.

In the area of cloud computing, a cloud is said to be

oversubscribed when the sum of customers' requests for a

resource exceeds the available capacity. There can be

oversubscription on both the customer's end or the

provider's end (Williams et al., 2011). Oversubscription

stemming from the customer occurs when they do not

reserve enough computing power to meet their needs.

Oversubscription on the provider's end occurs when CSPs

book more requested capacity than they can actually support.

This type of oversubscription is more common than the

former as many customers tend to reserve more resources

than they need (Ghosh & Naik, 2012). Thus, this paper will

focus on overbooking by the CSP.



Figure 1a shows an example of a non-oversubscribed

cloud using memory capacity as an example. Each column

represents a VM sitting on top of the physical machine (PM).

Here 37.5% of the available physical memory is being

utilized. Conversely, Figure 1b shows a corresponding

example for an oversubscribed PM. This model increases

resource utilization to 68.75%.

(a). Non oversubscribed PM

(b). Oversubscribed PM

Figure 1. VM Memory Allocation.

3.3 RISKS OF CLOUD OVERSUBSCRIPTION Though there are valid motivations to oversubscribe

computing resources, the strategy is not without inherent

risks. In order to make oversubscription possible, providers

must make several assumptions about their customers. They

assume that not all customers will use any or all of their

requested resources. They also assume that not all customers

will show up to use their resources at the same time. By

making these assumptions, CSPs flirt with the possibility of

running out of resources that customers have legitimately

paid for. This can have costly consequences for CSPs, one

of which is overload.

Overload occurs when the infrastructure is strained to

meet demands as requests for resources near or exceed the

physical capacity. This can severely degrade the

performance of the cloud and even lead to outages and

failures for some customers (Moreno & Xu, 2012; Williams

et al., 2011; Toms & Tordsson, 2013; Ghosh & Naik, 2012;

Baset et al., 2012; Zhang et al., 2012; Hines et al., 2011;

Jain et al., 2012; Wang, Hosn, & Tang, 2012; Breitgand &

Epstein, 2012; Wo et al., 2012; Guo et al., 2013; Breitgand

et al., 2012). CPU overload can result in an abundance of

processes waiting to run in a VMs CPU queue (Baset et al.,

2012). This can degrade application performance and reduce

the ability of cloud monitoring agents to supervise VMs.

Disk overload, which is not thoroughly discussed in this

paper, can have similar consequences in regards to

performance reduction. If memory becomes overloaded

(usually classified by multiple VMs swapping out to the

disk) it can be devastating because it has the potential to

inhibit any application progress (Williams et al., 2011).

Large overheads and thrashing can seriously impede the

performance of the system (Williams et al., 2011; Baset et

al., 2012). Finally, network overload can cause bottlenecks

that slow progress and can lead to a reduction in resource

utilization and oversubscription gains (Baset et al., 2012;

Jain et al., 2012; Breitgand & Epstein, 2012; Wo et al.,

2012; Guo et al., 2013).

If overload is not managed, CSPs further run the risk of

violating its Service Level Agreements (SLAs). SLAs

provide customers with a sense of security by providing a

level of assurance that their requested resources will be

available and operational when they need them (Ghosh &

Naik, 2012). Some SLAs are legally bonded, which means

that companies could be forced to provide compensations to

customers for SLA violations. Even one SLA violation can

be costly to CSPs and so developing an oversubscription

policy that considers SLA constraints is crucial for effective

implementation.

3.4 OVERLOAD PREVENTION AND MITIGATION In developing a model for oversubscription, CSPs must

take both proactive and reactive steps to reduce overload.

Studying client habits is one predictive measure taken to

determine how best to allocate resources thus optimizing

oversubscription while reducing performance degradation

from overload (Moreno & Xu, 2012; Williams et al., 2011;

Toms & Tordsson, 2013; Ghosh & Naik, 2012; Hines et al.,

2011; Wang et al., 2012; Wo et al., 2012; Breitgand et al.,

2012).

Even with proactive resource allocation models in place,

overload can still occur. When it does, strategies to

effectively detect and manage overload must be employed.

A basic description of some common overload remediation

techniques are discussed by Baset et al. (2012) and are as

follows:

Stealing is the act of taking resources from underloaded

VMs and giving them to overloaded VMs on the same

physical host. Memory ballooning is a common

example of this and it is often a capability installed in

hypervisors that use this as a first line of defense

against overload.

Quiescing is the act of shutting down VMs on an

overloaded PM so that the remaining VMs can function

at normal performance levels (Baset et al., 2012; Wang



et al., 2012). The VMs that are shut down will resume

once overload subsides.

VM Migration is the act of moving a VM from one

physical machine to another (Moreno & Xu, 2012;

Williams et al., 2011; Baset et al., 2012; Zhang et al.,

2012; Jain et al., 2012; Wo et al., 2012). This is

typically done in systems that have VM storage located

on Storage Area Network (SAN) devices as opposed to

local memory. In the former situation, only a memory

footprint of a VM needs to be moved while in the latter,

VM migration to the disk can encumber the datacenter

network.

Streaming disks serves as a solution to reduce the

network costs of migration to the disk. Here, a portion

of the VMs disk is transferred to another PM. When the

transfer is completed, the migrated VM can access disk

space on both the original and new PM. Once the

network traffic is low, the two disks can be reunited.

Network memory can reduce load caused by swapping

on the PM disk by allowing the memory of another PM

to be used for swapping over the network.

4. THE ECONOMICS OF CLOUD

COMPUTING As discussed in Malkowski et al (2010), the profit

model of cloud computing can be described as a trade-off

between two components: the infrastructure cost model and

the provider revenue model. Revenue is calculated as

earnings minus penalties. Overall profit is realized by

calculating the revenue minus the infrastructure cost.

In terms of costs, cloud datacenters require a great deal

of infrastructure that consumes large amounts of power in

operating the hardware and maintaining the environmental

factors. Additionally, datacenters necessitate educated and

skilled employees to manage and maintain their systems.

Cloud revenue is a product of several factors, namely

the pricing model(s) implemented, the number of customers

served, and the number of SLA penalties incurred. Whereas

traditional software business models require a one-time

payment for unlimited use typically for one computer, cloud

pricing models are inspired by those of utility companies

and charge customers on a consumption basis. Cloud

providers can offer a variety of pricing tiers. For instance,

Amazon EC2 offers free, on-demand, reserved, and spot

instances. Within these tiers, variations in VM specifications

allow for further price customization and accommodate

customers with diverse needs such as varying memory,

bandwidth, and CPU requirements.

Some research has considered pricing models that allow

customers to pay for higher priority. In an experiment

performed in Macias et al. (2010), tasks are assigned either

a gold, silver, or bronze priority. The gold tasks have a price

50% higher than the bronze, and the silver tasks have a price

20% higher than the bronze tasks. They proved that

dynamic pricing always generates the highest revenue over

fixed pricing because it adapts to all possible scenarios.

Simulations discussed later in this paper show CSPs can use

fixed pricing to increase revenue, but too many high priority

customers can overwhelm those who have paid for regular

service.

Penalties in the form of SLA violations detract from the

revenue generated by the pricing model. SLA violations are

caused when utilization within the cloud is too high leading

to performance degradation. Some SLAs have a penalty

price that companies pay out of pocket when a violation

occurs. If this is the case, CSPs risk losses in both revenue

and customers.

Overall the total profits generated by the profit schema

of cost-revenues is driven by the number of customers being

served and the amount of resources they consume. If

customer utilization within the cloud is too low, then

infrastructure costs could outweigh revenue (Assuno et al.,

2009). On the other end of the spectrum, if the datacenter

becomes overwhelmed by the load of customers, SLA

penalties will limit profits. These factors must be considered

when searching for the most optimal solution for pricing in

the cloud. A solution is considered to be optimal if it

maximizes the revenue while not violating SLAs or

degrading service. The simulations in future sections begin

to take a look at one part of this equation, namely the

revenues generated from customer utilization of the service.

5. COMPARING CLOUD COMPUTING TO

OTHER INDUSTRIES There are noticeable similarities between overbooking

in other industries and the cloud. Overbooking too many

appointments in a doctor's office can lead to longer wait

times as the doctor tries to see all patients. This is analogous

to how overbooking a resource such as CPU can potentially

lead to longer wait times if overload occurs. Airlines often

have class systems that outline the level of services a

passenger is entitled to. First class is more expensive than

coach, but these passengers also receive a more luxurious

flight. Similarly, some clouds use levels of service to offer

customers flexibility in selecting a price for the computing

experience they need (Baset et al., 2012). In all industries,

waiting times or lack of service availability is a potential

consequence that can lead to customer dissatisfaction.

Researching the application of oversubscription in other

industries raises new questions that can be addressed. Some

of these questions are listed below:

How much should customers be compensated in the

event of an SLA violation to best maintain retention

rates?

Is one type of compensation (i.e. offering vouchers for

computing resources as opposed to monetary

compensation) better able to placate angry customers?

Similar to airline practices, could CSPs benefit from

alliances with competitors by assigning workloads to

them in times of overload? If so, what impact would



that have on overbooking models when compared to

more traditional models?

Can having levels of service similar to airline classes

improve overbooking policies by implementing varying

degrees of overbooking to the different levels?

The answers to these questions may give CSPs an edge

in the highly competitive market that is outsourcing

computing power, yet there are many ways in which cloud-

based oversubscription differs from other industries. Other

industries tend to maximize oversubscription by optimizing

a single primary resource. CSPs can oversubscribe several

resources with hopes to optimize the use of existing

computing infrastructures. When these oversubscription

methods fail, the results may be felt by all subscribers of the

service. Outside of the cloud, the impact of oversubscription

is not always universal. For this reason, proper optimization

of resources within the cloud is extremely important as it

allows providers to offer competitive pricing and also

ensures customer satisfaction.

6. SIMULATING THE EFFECTS OF

OVERSUBSCRIPTIONMuch research on oversubscription focuses on

maximizing resource utilization without exploring the

economic effects in detail. Thus, the simulation tool

CloudSim is used to investigate the difference in revenues

and QoS between oversubscribed and non-oversubscribed

datacenters as a first step towards this goal. Inspired by the

multi-class service model of airlines, additional simulations

are employed to examine these metrics in a priority class

scenario. The simulations attempt to shed light on the

answers to three main research questions: 1) Does

oversubscribing the datacenter CPU have the potential to

increase revenue? 2) Does adding priority to the

oversubscription scheduling policy have the potential to

increase revenue? 3) How does giving priority to some

VMs in an oversubscribed datacenter affect the performance

and debt of both the priority and non-priority VMs?

6.1 CLOUDSIM CloudSim is a discrete event simulator built upon

GridSim that allows for the simulation of virtualized cloud

computing environments. The version of CloudSim used in

these experiments is 3.03.

Experiments 1 and 2 focus on utilizing different VM

scheduling policies within the CloudSim framework,

namely the TimeShared, TimeSharedOverSubscription, and

the TimeSharedOverSubscriptionPriority policies, the last

of which was developed for this research. Experiment 3

focuses on adjusting the power VM allocation policy so that

the TimeSharedOverSubscriptionPriority VM scheduling

policy can be implemented in a power-aware datacenter

with VM migration turned on. Experiment 3 utilizes the

power classes within CloudSim that are introduced in

(Beloglazov & Buyya, 2012).

6.2 CALCULATING DEBT In the simulations, the base price of a regular VM

instance is $.09 per hour. This is the same as the cost of a

standard small, pay-as-you-go VM instance on Windows

Azure.1 For priority VMs, the price is $0.15 per hour. The

price set for priority VMs is intended to allow comparison

of revenues in the priority and non-priority oversubscription

configurations. However, further research may explore the

optimum price for a priority instance given the factors that

affect revenues.

As is typical with most cloud IaaS providers, customers

are only charged for the time that their instances are running.

The simulations follow this precedent by only charging

VMs for the total actual CPU time they use. However, in the

simulations, CPU time is not rounded up to the next hour as

is the case with many IaaS providers.2 3

The calculations

used for computing the total datacenter debt are represented

by the formula below.

𝐷 = 𝑐𝑛 ∑𝑡𝑛

3600𝑣𝑛∈𝑉

+ 𝑐𝑝 ∑𝑡𝑝

3600𝑣𝑝∈𝑉

(1)

where:

𝐷 = Total Datacenter Debt 𝑉 = Set of all VMs in the Datacenter

𝑣𝑛 = Subset of VMs that are non-priority𝑣𝑝 = Subset of VMs that are priority

𝑐𝑛 = Hourly charge for a non-priority VM𝑐𝑝 = Hourly charge for a priority VM

𝑡𝑛 = Cumulative CPU time for a non-priority VM𝑡𝑝 = Cumulative CPU time for a priority VM

7. EXPERIMENT 1: SCALING INITIAL MIPSALLOCATION

The first Experiment was initially presented in

Householder et al. (2014). As an extension, this work

conducts the simulations using 500 hosts. In Experiment 1,

the initial mips allocation is used to scale back non-priority

VMs.

7.1 SIMULATION SETUP In Experiment 1, three main scenarios are simulated and

compared. In the first scenario, the workloads are executed

using a time-shared (TS) VM scheduling policy that does

not allow for oversubscription. All VMs are given 6,000

1 http://www.windowsazure.com/en-

us/pricing/details/virtual-machines/ 2http://www.rackspace.com/cloud/servers/pricing/

3 http://www.rackspace.com/cloud/servers/pricing/



mips per physical element (PE) and are charged $0.09 per

hour. In the second scenario, the time-shared

oversubscription (TSO) VM scheduling policy is employed

in a non-priority setting. All VMs are allotted 6,000 mips

per PE and are charged at a rate of $0.09 per hour. In the

third scenario, the TSO VM scheduling policy is also

employed, but a percentage of the VMs are given priority

over the others so that their jobs run faster than they

typically would in a non-priority configuration.

Table 1. Cost and MIPS of Priority and Non-Priority VMs.

VM Instance Type Price Per Hour ($) MIPS

Non-Priority 0.09 5,000

Priority 0.15 6,000

In order to give a subset of the VMs priority, they are

allotted 6,000 mips per PE while the other non-priority VMs

are scaled back to 5,000 mips per PE. The priority VMs are

charged $0.15 per hour while the non-priority VMs are still

charged $0.09 per hour. The differences between priority

and non-priority VM instances can be seen in Table 1.

Each simulation sets up a single datacenter comprised of

500 physical hosts. Each host is given 10 GB/s of

bandwidth, 16 GB of RAM, and 1024 GB of storage. Each

host is also allocated 48,000 mips distributed across 8 cores,

allowing each core up to 6,000 mips. The hosts are

dynamically carved into VMs using

VMAllocationPolicySimple which assigns a VM to the host

with the fewest cores currently in use (Calheiros et al.,

2011). Each VM is given 100 MB/s of bandwidth, 2 GB of

RAM, and 10 GB of storage. Additionally, each VM is

allocated 2 cores. mips per core are allocated to each VM

based on the VM instance type as is noted in Table 1. The

configurations for the hosts and VMs can be seen in Table 2.

Table 2. Experiment 1 Specifications.

Resources Host VM

Bandwidth (MB/s) 10,240 100

RAM (GB) 16 2

Storage 1,024 10

MIPS 48,000

Cores (PEs) 8 2

Each scenario is tested using three different workload

files obtained from the Parallel Workloads Archive.4 These

files represent the logs of actual workloads ran on real

systems and, in this study, consist of the following three

workloads: Workload 1 (NASA-iPSC-1993-3.1-cln.swf.gz),

Workload 2 (OSC-Clust-2000-3.1-cln.swf.gz), and

Workload 3 (LLNL-Atlas-2006-2.1-cln.swf.gz).

The workloads are used to generate cloudlets which are

jobs assigned to the VMs (Calheiros et al., 2011). Within a

VM, the cloudlets are scheduled using the

CloudletSchedulerTimeShared scheduling policy that

4 http://www.cs.huji.ac.il/labs/parallel/workload/

implements a round robin algorithm. For each workload,

simulations are run for 5 different configurations. These

configurations include time-shared (TS), time-shared

oversubscription non-priority (TSO-NP), time-shared

oversubscription with 10% of the VMs given priority (TSO-

P 10%), time-shared oversubscription with 20% of the VMs

given priority (TSO-P 20%), and time-shared

oversubscription with 40% of the VMs given priority (TSO-

P 40%). For each of these configurations, simulations for

1000, 2000, 3000, 4000, and 5000 VMs are run.

7.2 EXPERIMENT 1 RESULTS Table 3 shows the total datacenter debt for the three

workloads in each of the five configurations for 1000, 2000,

3000, 4000, and 5000 VMs. When the datacenter has 1000

and 2000 VMs, it is not overloaded and so all VMs

successfully execute their jobs in all five configurations. As

a result, the TS and TSO-NP configurations produce the

same debt for 1000 and 2000 VMs since all VMs are

charged the same price and oversubscription has no effect.

Notice that 1000 VMs setting acquires the longest running

time and the most debt. When the workload is spread out

across 1000 VMs, only half of the potential resources in the

datacenter are used, resulting in slower execution times and

greater costs.

Table 3: Exp 1 Total Datacenter Debt

For 3,000-5,000 VMs, the datacenter debt in TSO-NP is

higher than in TS. This is because the datacenter is

overloaded. With each of 500 physical hosts having 48,000

mips, the datacenter can only allocate up to 24,000,000 mips.

In the TS scheduling policy, each of the VMs is given the

full 6,000 mips at all times for both cores. This means that

in a TS configuration, 2000 VMs need 6,000 mips per core

* 2 cores per VM * 2000 VMs = 24,000,000 mips. This is

the maximum physical capacity of the datacenter. Any

VMs over 2000 will fail in a TS scheduling policy. As a

result, 1000 VMs fail in the TS simulation for 3000 VMs,

2000 VMS fail in the TS simulation for 4000 VMs and 3000

VMs fail in the TS simulation for 5000 VMs. Conversely,

all VMs execute successfully in the TSO-NP configurations.



Thus, the TSO-NP allows for more VM instances to be

created and generates more debt as a result. These results

show the limitations of a non-oversubscribed datacenter in

terms of potential revenue.

While the TSO-NP configuration generates more debt,

it comes with a price. Table 4 shows the cumulative

datacenter CPU times for all VMs. Since the datacenter is

not overloaded for 1000 and 2000 VMs, again there is no

difference in the time it takes for all VMs to execute their

jobs between the TS and TSO-NP configurations. However,

when the datacenter is overloaded, TSO-NP continues to

accept VMs which degrades the performance by causing all

jobs to run longer. The more VMs created in the datacenter,

the longer it takes for all VMs to execute successfully.

These results show that oversubscription comes with a cost

to the QoS and must be implemented smartly to prevent a

loss of customers and/or an increase in SLA violations, both

of which would offset the potential economic benefits of

oversubscription.

Table 4: Experiment 1 Total Datacenter Actual CPU Time

(Hrs)

When adding the opportunity for some VMs to pay a

higher price in order for their jobs to run faster, it creates

further opportunities for revenue generation. This is

indicated in Table 3 as TSO-P configurations generate more

total datacenter debt than the TSO-NP configurations.

Further, as the percentage of VMs that are given priority

increases, the datacenter debt also increases, even when the

datacenter is not overloaded at 1000 and 2000 VMs. This is

logical because some VMs are paying more for their

instances.

8. EXPERIMENT 2: A NEW SCALING

MECHANISM The second set of simulations (known from here on out

as Experiment 2) takes a different approach to scaling back

non-priority VM mips. Experiment 1 allots each VM 2 CPU

cores. Upon instantiation, priority VMs are allotted their full

6,000 mips per core and the non-priority VMs are scaled

back to 5,000 mips per core. This means that non-priority

VMs are scaled back regardless of if overload occurs or not.

When overload does occur despite the initial scale-backs,

the VmSchedulerTimeSharedOversubscription class in

CloudSim scales back and redistributes the available mips

for all VMs using the following formula:

𝑆 =𝑇𝐴

𝑇𝑅 (2)

where:

𝑆 = Scaling Factor

𝑇𝐴 = Total available mips on host 𝑇𝑅 = Total required mips by all VMs

As a result of these settings, each VM in Experiment 1

has the potential to be scaled back at some point from its

initial 6,000 mips request. Experiment 2 addresses this issue

by only scaling back VMs that are not priority when an

overload occurs.

8.1 SIMULATION SETUP Experiment 2 has the same physical capacity as

Experiment 1. In the first round of simulations, the

datacenter consists of 500 physical machines. Each PM is

allotted 10,240 MB/s in bandwidth, 20 GB RAM, 1024 GB

of storage, 48,000 mips, and 8 cores. Each VM is allotted

100 MB/s bandwidth, 2 GB of RAM, 10 GB of storage, and

2 cores. In Experiment 2, the mips are only redistributed

when overload occurs. Upon instantiation, the simulation

allots the full 6,000 mips to both priority and non-priority

VMs. When overload occurs, the newly created VM

scheduling policy for oversubscribing with priority takes

over to scale back only the non-priority VMs. This is done

using the following formula:

𝑆 =𝑇𝐴 − 𝑇𝑅𝑝

𝑇𝑅 − 𝑇𝑅𝑝

(3)

where:

𝑆 = Scaling Factor

𝑇𝐴 = Total available mips on host 𝑇𝑅 = Total required mips by all VMs

𝑇𝑅𝑝 = Total required mips by priority VMs

As in Experiment 1, the prices for the VMs are fixed

with non-priority instances costing $0.09 per hour and

priority instances costing $0.15 per hour. Also as in

Experiment 1, the simulations are run on the three

workloads from the Parallel Workloads Archive.

8.2 EXPERIMENT 2 RESULTS Table 5 shows the resulting VM debt for each of the

three workloads while Table 6 shows the resulting CPU

times. Similar to Experiment 1, the TS and TSO-NP

configurations for a given workload produce the same VM

debt and CPU times when the datacenter is not overloaded



(1000 and 2000 VMs). However, unlike Experiment 1, the

CPU times do not change for a given workload in the

underloaded scenarios even as the number of priority VMs

is increased, showing that Experiment 2 does not scale non-

priority VMs unnecessarily. With the consistent CPU times

in underloaded scenarios, the VM debt results for 1000 and

2000 VMs indicate that adding more priority VMs has the

potential to increase VM debt without degrading the QoS.

For the overloaded scenarios of 3000, 4000, and 5000

VMs (with the exception of TSO-P 40% for 5000 VMs),

Experiment 2 results show that the VM debt increases as the

number of priority VMs increases. However, the overall

execution times also increase for the workload as the larger

number of priority VMs further scales back the mips

allocated to non-priority VMs. Thus, the benefits of

increased revenues overall come at the risk of increasingly

degraded QoS and costs for non-priority customers. This

decline in QoS for non-priority customers reaches an

extreme value in the TSO-P 40% simulations. Here, the

priority VMs utilize all of the physical resources which

means that non-priority VMs are scaled back to zero and

their jobs cannot be executed. Thus, Experiment 2 has

limitations on the number of priority VMs it can host.

Table 5: Exp. 2 Total Datacenter Debt

9. COMPARING EXPERIMENTS 1 AND 2Figure 2 shows a comparison of the results for

Experiment 1 and Experiment 2. While both experiments

were run on all three workloads, the overall trends remained

consistent and so Workload 3 is shown as a representation

of the results.

When the datacenter is not overloaded, as is the case for

1000, and 2000 VMs, both experiments are equivalent in the

TSO-NP simulations. Additionally, for the TSO-P

simulations, results for the priority VMs are also equivalent

when the datacenter is not overloaded. This is because they

are never scaled back at any time. However, in the same

TSO-P simulations, Experiment 1 shows degraded

performance leading to increased costs for the non-priority

VMs. These results are attributed to the fact that in

Experiment 1, the non-priority VMs are being scaled back

to 5,000 mips even when no overload has occurred. This

contrasts from Experiment 2 in which no VM is scaled back

unless there is overload. These results can be seen in Figures

2a-2d .

Table 6: Exp. 2 Total Datacenter Actual CPU Time (Hrs)

When the datacenter faces a potential overload scenario,

as is the case with 3000, 4000, and 5000 VMs, Experiment

2 is favorable for the priority VMs by providing lower costs

and faster execution times whereas Experiment 1 is

favorable for the non-priority VMs for the same reasons. In

Experiment 2, priority VMs are never scaled back. Instead,

when overload occurs, the mips remaining after the priority

VMs are allocated are scaled and redistributed to the non-

priority VMs. In Experiment 1, the non-priority VMs are

initially scaled back to 5,000 MIPs. However, if overload

occurs, the mips are scaled back and redistributed amongst

all VMs, even those with priority. Thus, Experiment 1

allows the effects of overload to be felt overall at an earlier

stage. While Experiment 2 does not allow for scaling of

priority VMs, this also presents a new limitation which can

be seen with TSO-P 40% configuration for 5000 VMs.

Without scaling priority VMs, Experiment 2 only allows for

a maximum of 2000 priority VMs to be hosted. Any VMs

beyond 2000 results in failure as not all VMs can be

instantiated. Notice for Experiment 2 that the TSO-P 40%

has a much lower debt and CPU time than the others. This is

because while all 2000 priority VMs are allowed to be

instantiated at their full capacity, it causes

the remaining 3000 non-priority VMs to be scaled back to

zero, and so they cannot execute their jobs. In order to

implement this scheduling policy, CSPs must be careful to

set boundaries to ensure that the number of priority VMs

does not exceed the physical capacity of the datacenter. So,

while Experiment 2 tends to better benefit priority VMs,



(a) VM Debt 1,000 VMs.

(c) VM Debt 2,000 VMs.

(e) VM Debt 3,000 VMs.

(g) VM Debt 4,000 VMs.

(i) VM Debt 5,000 VM.s

(b) CPU Time 1,000 VMs.

(d) CPU Time 2,000 VMs.

(f) CPU Time 3,000 VMs.

(h) CPU Time 4,000 VMs.

(j) CPU Time 5,000 VMs.

Figure 2: Workload 3 Simulation Comparison



Experiment 1 has the potential to host more VMs resulting

in higher potential for oversubscription.

Both Experiments 1 and 2 indicate that allowing some

VMs to pay more for priority allocation has the potential to

increase revenues. Unfortunately these benefits come at the

risk of SLA violations due to decreased performance.

Additionally, some of the increased VM debt generated

comes from charging non-priority VMs for extra CPU time

acquired due to their resources being scaled back. This

raises the question of fairness to non-priority VMs.

If implemented in a real datacenter, the higher costs and

extended execution times may lead to unhappy customers

that will seek other CSPs for further requests. In the next

experiment, some of the concerns regarding fairness to non-

priority VMs and SLA violations are addressed.

10. EXPERIMENT 3: OVERSUBSCRIPTION

WITH VM MIGRATION The third set of simulations (known from here as

Experiment 3) utilizes the VM scheduling policy

implemented in Experiment 2 in conjunction with VM

migration in a power-aware datacenter. With this

combination, non-priority VMs can be scaled back to allow

a PM to host more VMs. When a host becomes over-utilized

past a certain threshold value, VMs can be migrated from

the over-utilized host to an underutilized host. Additionally,

when all VMs from an under-utilized host can be migrated,

the migration is implemented and the newly empty under-

utilized host is turned off to conserve energy.

10.1 SIMULATION SETUP The migration simulations implement the power classes

that can be found in CloudSim 3.0 and were developed in

Beloglazov and Buyya (2012). In these simulations, there

are 250 hosts. Each host is allotted 48,000 mips and 8 cores

which means there are 6,000 mips/core. Hosts are given 18

GB of RAM, 1 Gbit/s of bandwidth, and 1000 GB of

storage. In the datacenter, half of the hosts are HP ProLiant

ML110 G4 servers and the other half are HP ProLiant ML

110 G5 servers. A power model is implemented that

classifies the power consumption for each server type at

different load levels. These consumption values for each

server are based on real data. The Hosts are carved into

homogeneous virtual machines. Each VM is given 2 PEs

and 6,000 mips/PE. Each VM is also allotted 1.7 GB of

RAM and 0.1 Gbit/s of bandwidth.

PlanetLab Workloads. For these simulations,

cloudlets are generated using the dynamic cloudlet

scheduling policy in CloudSim which creates a VM to host

each job in the workload. The workloads used for these

experiments are two PlanetLab workloads. The description

of the workloads can be seen in Table 7. Each workload

represents a workload trace for 1 day from a random server

in PlanetLab (Beloglazov & Buyya, 2012). Inside each

workload file, there are jobs that represent data from virtual

machines on that given server in the given day. These jobs

will become cloudlets and each cloudlet is hosted by a

single VM. Inside of each job file, there are 288 random

samples of values that indicate the CPU utilization

percentage for that VM in 5 minute intervals.

Table 7. Description of Workloads. Date Name Number of

VMs

Mean

Utilization

03/06/2011 20110306 898 16.83%

04/20/2011 20110420 1033 15.21%

Implementing VM Migrations. The method for

implementing VM migration is the same as in Beloglazov

and Buyya (2012) and can be seen in Algorithm 1. The VM

selection policy used to determine which VM should be

migrated from an overloaded host is the Minimum

Migration Time Policy. With this policy the migration time

is estimated using the amount of RAM utilized by the VM

divided by the free network bandwidth available for the

given host. Upon migration, there is an average performance

degradation that can be estimated as 10% of the CPU

utilization. In these simulations, a destination host for a VM

only receives 10% of the migrating VMs mips meaning that

any migration could potentially lead to an SLA violation.

Figure 3: Algorithm taken from Beloglazov and Buyya

(2012).

TSO VM Allocation Policy. A time-shared VM

allocation policy for oversubscribing with priority has been

created for these experiments. It is very similar to the power

static threshold policy used in (Beloglazov & Buyya, 2012).

The primary difference between the two is that upon initial

allocation of a VM to a host, the VM is allowed to be

created and assigned even if there are not enough available

mips on the host machine. As long as the host machine has

not reached its maximum capacity with priority VMs, this

allocation policy can assign a VM to an overloaded host and

then implement the time-shared oversubscription VM

scheduling policy for priority (introduced in Experiment 2)

to scale back the non-priority VMs. Like with the power

static threshold policy, there is a threshold for which a host

is considered overloaded. In these simulations, that

threshold is reached when 90% of the CPU is utilized.

Calculating SLA and Energy Metrics. Calculations

for the SLA violation metric as well as the energy metric are



identical to those utilized in (Beloglazov & Buyya, 2012).

SLA violations are calculated using two metrics. The first is

the percentage of time that a host receives 100% CPU

utilization and the other is the overall performance

degradation that is caused by VM migrations. The product

of these two metrics is used to determine the SLA violation

metric.

As previously described, energy consumption is based

on a power model for two server types. These values are

based on real data for the server that identifies the power

consumption in watts for various loads.

Compensating Non-Priority VMs. The results of the

simulations in Experiments 1 and 2 show that the QoS for

non-priority VMs may become severely degraded due to

increased running times for their jobs. As a result of these

increased running times, the debt calculation in Equation 1

used in the first two experiments may be unfair to non-

priority customers as their longer running jobs lead to

increased VM debt. Thus, in Experiment 3, a compensation

factor is added to make the cost to non-priority customers

more fair.

In order to calculate the compensation factor for non-

priority VMs, the running times for each VM is first

estimated using Equation 4 where 𝑅 is the estimated run

time, 𝐿 is the cloudlet length, ��is the average mips, and 𝐶 is

the number of cores.

𝑅 =𝐿

��𝐶(4)

This estimation is calculated for each non-priority VM

and the results are summed together. Note that since a job in

a PlanetLab workload changes its CPU utilization every 5

minutes, the average mips being utilized for each job is used

to estimate the total requested mips for the VM.

The extra debt accumulated by non-priority VMs due to

degraded service is calculated by taking the difference in

actual and estimated run times and multiplying it by the

price for a non-priority VM using Equation 5 where 𝐷𝐸𝑥𝑡𝑟𝑎

is the extra debt and 𝑇 is the run time.

𝐷𝐸𝑥𝑡𝑟𝑎 = (𝑇𝐴𝑐𝑡𝑢𝑎𝑙 − 𝑇𝐸𝑠𝑡𝑖𝑚𝑎𝑡𝑒𝑑) ∗ 0.09 (5)

Finally, the extra debt is subtracted from the original

debt value calculated to get the adjusted debt value using

Equation 6.

𝐷𝐴𝑑𝑗𝑢𝑠𝑡𝑒𝑑 = 𝐷𝐴𝑐𝑡𝑢𝑎𝑙 − 𝐷𝐸𝑥𝑡𝑟𝑎 (6)

10.2 EXPERIMENT 3 RESULTS Comparing VM Debt and CPU Time. Figures 4 and 5

compare the VM debt and CPU times for the cases when

VM migration is either off or on. When VM migration is on,

both the CPU time and the VM debt tend to be slightly

higher than when VM migration is off. This is likely

because there is some overhead incurred from the VM

migrations. Some minor exceptions to this can be seen.

(a): VM Debt

(b): CPU Time

Figure 4: VM Debt and CPU Time for Workload 20110306

without VM Migration.

(a): VM Debt

(b): CPU Time

Figure 5: VM Debt and CPU Time for Workload

20110306 with VM Migration.



For instance, in Figure 4b the CPU time is slightly lower

when migration is on for 100% priority. Similarly, in Figure

4b the VM debt is slightly lower when VM migration is on

for 0% priority. Further examination of the logs indicate that

in the allotted simulation time given, one less cloudlet

finished its job in both of these scenarios, thus slightly

skewing these factors.

The results in Figures 4 and 5 show that for both

workloads, the overall CPU time remains fairly consistent

even as a higher percentage of priority VMs are allotted.

This indicates that combining migration with the VM

Scheduling policy that scales back non-priority mips could

potentially offset some of the performance degradation

accumulated from adding higher priority percentages.

However, it is unclear from these results whether or not the

priority VMs are receiving any improved performance.

While the CPU times remain fairly consistent as more

priority VMs are added, the VM debt tends to increase even

with the non-priority VMs being compensated for slower

execution times. This suggests there is potential for

increasing revenues using this model.

Non-Priority Compensation. Adding the

compensation factor proves to better balance out the costs

for non-priority VMs. Figure 6 shows the amounts that the

VM debt was adjusted for both migration on and off. Notice

that VMs tend to be compensated slightly more when VM

migration is on. This could be because the VMs are also

being compensated for the overhead incurred due to

migration also.

While this metric attempts to make the datacenter

configuration fairer for non-priority VMs, there is the

potential that non-priority VMs will still pay slightly more

for their units due to slower job execution times caused by

oversubscription. This is the result of using an average mips

value to estimate the run time of a given cloudlet. Future

work should make this estimation more robust so as to more

completely compensate non-priority VMs for the extra costs

resulting from degraded QoS.

Figure 6: VM Debt Adjustments.

Number of VM Migrations. The number of

migrations for the two workloads can be seen in Figure 7.

These results indicate that allowing a large ratio of priority

VMs tends to increase the overall number of VM migrations.

On a much larger scale, this could have dire implications if

the number of VM migrations causes enough performance

degradation to cause SLA violations. While the SLA

violation metric discussed in the next section is low overall,

it does tend to rise as the number of VM priority VMs

increases. Also tracked was the number of priority and non-

priority migrations. Tables 8 and 9 show these values for

each workload and each percentage.

Figure 7: Number of VM Migrations.

Table 8: Number of migrations by priority workload 20110306

Percent Priority Priority

Migrations

Non-Priority

Migrations

0% 0 944

10% 79 862

20% 203 738

30% 333 604

40% 427 510

50% 539 398

60% 619 324

70% 718 225

80% 864 79

90% 944 26

100% 1016 0

SLA. Figure 8 shows the SLA violation metric for both

of the workloads. In the given simulation limit, both

workloads start with slightly higher values as no VMs are

given priority. This value then drops as a small percentage

of priority VMs are allotted. Finally, as nearly all VMs in

the workload are priority, the SLA metric again is higher.

However, the overall values for this metric are considerably

low and so it suggests that the combination of scaling non-

priority VMs and VM migration could potentially be

implemented with low levels of SLA violations. It is worthy

to note that this metric is only captured when VM migration

is turned on. Future work should include a metric that can



compare the SLA dynamic for migration on and migration

off scenarios.

Table 9: Number of migrations by priority workload 2011042.

Percent Priority Priority

Migrations

Non-Priority

Migrations

0% 0 991

10% 87 898

20% 215 770

30% 325 647

40% 450 522

50% 561 418

60% 670 312

70% 816 166

80% 919 74

90% 979 51

100% NA NA

Figure 8: SLA Violation Metric for Migration.

Energy Consumption. Results of this experiment in

terms of energy consumption are shown in Figure 9. Of note

is that when VM migration is used, energy consumption

generally tends to increase along with the percent of priority

VMs. This is sensible as more VMs running while using full

resources should consume a greater amount of energy.

Overall, the use of VM migration as coupled with

oversubscription techniques tends to lead to greater energy

savings.

11. LIMITATIONS AND FUTURE WORKThese simulations are a continuation towards

understanding how adding priority classes to an

oversubscribed datacenter can impact cloud profits and QoS.

However, there are limitations to the current work that need

to be addressed in order to strengthen its value as a well-

rounded economic study that can be applied to real

datacenters. These simulations only consider three

workloads for Experiments 1 and 2 and two workloads for

Experiment 3. Moreover, they are limited to the scope of

CPU oversubscription and on-demand VM instances. While

this work extends previous works by attempting to consider

infrastructure costs such as energy consumption and SLA

violations, more work can be done to further develop the

(a) Energy consumption using VM migration.

(b) Energy consumption without using VM migration.

Figure 9: Energy Consumption.

difference in these metrics in both migration on and

migration off scenarios and how these metrics could impact

overall profits.

Future work seeks to address these limitations by

applying the simulations to more workloads and widening

the scope to consider other pricing tiers (spot, reserved, etc)

as well as considering oversubscription of other resources

such as bandwidth and memory. Future work will also

broaden the scope to consider the impact that other aspects

of the profit schema, such as infrastructure costs and SLA

penalties, have on the overall profits. While this work

addresses some of these, their economic impact has yet to be

explored. Investigations on the impact of additional priority

classes as well as identifying and comparing fair

compensation algorithms for non-priority customers whose

jobs are slowed in lieu of preferred VMs will be conducted.

Finally, future work will explore optimal resource allocation

under SLA constraints for overbooked cloud computing

systems with multiple preference classes.

12. CONCLUSIONSResource allocation models in cloud computing

infrastructures tend to allow large fractions of resources to

sit idle at any given time. The ability to improve resource

utilization and decrease waste can significantly increase the

profits of a CSP. This paper discusses the application of



oversubscription techniques to achieve this goal and

introduces overload as a major risk of oversubscription. The

effects of oversubscription on datacenter revenues and VM

running times are investigated using three experiments. The

first two experiments implement and compare two different

VM Scheduling Policies in a non-power aware datacenter

with no VM migration. The third experiment analyzed the

same factors in a power-aware datacenter with VM

migration turned on and off. Simulation results suggest that

oversubscription does have the ability to allow CSPs to host

more VMs and increase revenues. Adding the opportunity

for some VMs to pay more for a priority instance can

increase revenues further. However as the research suggests,

the results demonstrate that oversubscription in any capacity

comes with the risk of degraded QoS when the datacenter

becomes overloaded. In the priority scenarios, the

degradation in QoS is minimized for priority VMs but

comes at a cost to the debt and running times of the non-

priority instances. Thus, in order for oversubscription to

provide CSPs with economic benefits, it must be

implemented in a smart and balanced manner in order to

limit the degradation to QoS that can lead to a loss of

customers and SLA violations.

13. REFERENCESAssuno, D., Dias, M., Costanzo, A. D., & Buyya, R. (2009). Evaluating the cost-benefit of using cloud computing to extend the capacity of clusters. In

18th ACM International Symposium on High Performance Distributed

Computing (pp. 141–150).

Beloglazov, A., & Buyya, R. (2012). Optimal Online Deterministic

Algorithms and Adaptive Heuristics for Energy and Performance Efficient Dynamic Consolidation of Virtual Machines in Cloud Data Centers.

Concurrency and Computation: Practice and Experience (CCPE), 24,

1397–1420.

Breitgand, D., Dubitzky, Z., Epstein, A., Glikson, A., & Shapira, I. (2012).

SLA-aware resource over-commit in an IaaS cloud. In International Conference on Network and Service Management (pp. 73–81).

Breitgand, D., & Epstein, A. (2012). Improving Consolidation of Virtual

Machines with Risk-aware Bandwidth Oversubscription in Compute

Clouds. In IEEE INFOCOM.

Calheiros, R., Ranjan, R., Beloglazov, A., Rose, C. A. F. D., & Buyya, R.

(2011). CloudSim: A toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning

algorithms. Software Practice and Experience, 41, 23–50.

Chen, X., & Hao, G. (2013). Co-opetition alliance models of parallel flights

for determining optimal overbooking policies. Mathematical and Computer

Modelling, 57(5-6), 1101–1111.

Coughlan, J. (1999). Airline Overbooking in the Multi-Class Case. Journal

of the Operational Research Society, 50, 1098–1103.

Feng, G., Garg, S., Buyya, R., & Li, W. (2012a). Revenue Maximization

Using Adaptive Resource Provisioning in Cloud Computing Environments. In ACM/IEEE 13th International Conference on Grid Computing (pp. 192–

200).

Feng, G., Garg, S., Buyya, R., & Li, W. (2012b). Revenue maximization

using adaptive resource provisioning in cloud computing environments. In

ACM/IEEE 13th International Conference on Grid Computing (GRID) (p.

192-200). doi: 10.1109/Grid.2012.16

Ghosh, R. & Naik, V. K. (2012). Biting Off Safely More Than You Can

Chew: Predictive Analytics for Resource Over-Commit in IaaS Cloud.. In R. Chang (ed.), IEEE CLOUD (p./pp. 25-32), : IEEE. ISBN: 978-1-4673-

2892-0

Guo, Z., Duan, J., & Yang, Y. (2013). Oversubscription Bounded Multicast

Scheduling in Fat-Tree Data Center Networks. In IEEE 27th International

Symposium on Parallel and Distributed Processing (pp. 589–600).

He, S., Guo, L., Ghanem, M., & Guo, Y. (2012). Improving Resource

Utilisation in the Cloud Environment Using Multivariate Probabilistic Models. In IEEE Fifth International Conference on Cloud Computing (pp.

574–581).

Hines, M. R., Gordon, A., Silva, M., Silva, D. D., Ryu, K., & Ben-Yehuda,

M. (2011). Applications Know Best: Performance-Driven Memory

Overcommit with Ginkgo. In IEEE International Conference on Cloud Computing Technology and Science (pp. 130–137).

Hoeflin, D., & Reeser, P. (2012, June). Quantifying the performance impact of overbooking virtualized resources. In IEEE International

Conference On Communications (ICC) (p. 5523-5527).

Householder, R., Arnold, S., & Green, R. (2014). Simulating the Effects of

Cloud-based Oversubscription On Datacenter Revenues and Performance in Single and Multi-class Service Levels. In IEEE Cloud 2014. Anchorage,

Alaska.

Jain, N., Menache, I., Naor, J., & Shepherd, F. (2012). Topology-aware vm

migration in bandwidth over-subscribed datacenter networks. Automata,

languages, and programming (Vol. 7392, pp. 586–597). Springer Berlin Heidelberg.

Luis Tomas, J. T. F. H., Cristian Klein. (2014). The straw that broke the

camel’s back: Safe cloud overbooking with application brownout. In

International Conference on Cloud and Autonomic Computing (CAC) (p.

151-160). doi:10.1109/ICCAC.2014.10.

Macias, M., Fito, O., & Guitart, J. (2010). Rule-based SLA Management

for Revenue Maximisation in Cloud Computing Markets. In Proceedings of the 2010 International Conference on Network and Service Management

(pp. 354–357).

Malkowski, S., Hedwig, M., Jayasinghe, D., Pu, C., & Neumann, D. (2010).

Cloudxplor: A tool for configuration planning in clouds based on empirical

data. In Proceedings of the 2010 ACM Symposium on Applied Computing (pp. 391–398). New York, NY, USA: ACM.

Moreno, I., & Xu, J. (2011, Dec). Customer-aware resource overallocation to improve energy efficiency in realtime cloud computing data centers. In

IEEE International Conference on Service-oriented Computing and

Applications (SOCA) (p. 1-8). doi: 10.1109/SOCA.2011.6166239

Moreno, I. S., & Xu, J. (2012). Neural Network-Based Overallocation for

Improved Energy-Efficiency in Real-Time Cloud Environments. In IEEE 15th International Symposium on Object/Component/Service-Oriented

Real-Time Distributed Computing (pp. 119–126).

Naik, S. A. B., Wang, L., & Tang, C. (2012). Towards understanding of

oversubscription in cloud. In Proceedings of the 2nd USENIX Conference

on Hot Topics in Management of Internet, Cloud, and Enterprise Networks and Services.

Noone, B. M., & Lee, C. H. (2011, August). Hotel Overbooking: The Effect of Overcompensation on Customers Reactions to Denied Service.

Journal of Hospitality and Tourism Research, 35, 334–357.



Patouni, E., Merakos, L., & Verikoukis, C. (2014, Feb). Fair allocation of

cloud amp;amp; physical resources for multi-class users. In IEEE 3rd symposium on Network cloud computing and applications (NCCA)(p. 37-

37). doi: 10.1109/NCCA.2014.14

Singhaseni, C., Wu, Y., & Ojiako, U. (2013). Modeling Overbookings on

Air Cargo Transportation. International Journal of Physical Distribution

and Logistics Management, 43(8), 638–656.

Tomas, L., & Tordsson, J. (2014). An autonomic approach to risk-aware

data center overbooking. IEEE Transactions on Cloud Computing, (2)3, 292–305.

Toms, L., & Tordsson, J. (2013). Improving cloud infrastructure utilization through overbooking. In ACM Cloud and Autonomic Computing

Conference.

Wang, L., Hosn, R. A., & Tang, C. (2012). Remediating Overload in Over-

Subscribed Computing Environments. In IEEE Fifth International

Conference on Cloud Computing (pp. 860–867).

Williams, D., Jamjoom, H., Liu, Y.-H., & Weatherspoon, H. (2011).

Overdriver: Handling Memory Overload in an Oversubscribed Cloud. In ACM SIGPLAN/SIGOPS International Conference on Virtual Execution

Environments.

Wo, T., Sun, Q., Li, B., & Hu, C. (2012). Overbooking-Based Resource

Allocation in Virtualized Data Center. In IEEE 15th International Symposium on Object/Component/Service-Oriented Real-Time Distributed

Computing Workshops (pp. 142–149).

Zacharias, C., & Pinedo, M. (2013). Appointment Scheduling with No-

Shows and Overbooking. Production and Operations Management, (23)5,

788-801.

Zhang, X., Shae, Z.-Y., Zheng, S., & Jamjoom, U. (2012). Virtual machine

migration in an over-committed cloud. In Network Operations and Management Symposium (pp.196–203)

AuthorsRachel Householder has a B.Sc. in

Integrated Mathematics and a M.Sc. in

Computer Science from Bowling Green

State University. She worked as a

Research Assistant for two years at the

Institutional Research Office and as a

high school mathematics teacher for three

years. Her research interests include cloud computing and

business intelligence.

Robert Green received his B.S. in

Computer Science & Applied

Mathematics from Geneva College in

2005, his M.S. from Bowling Green

State University in 2007, and his Ph.D.

from the University of Toledo in 2012.

He currently serves as an Assistant

Professor of Computer Science Professor at Bowling Green

State University. His research interests including Cloud

Computing, High Performance Computing, Population-

based Metaheuristics, Software Development (Mobile

Applications and Web Development), and the modeling,

evaluation, and analysis of complex systems.



OPTIMIZATION OF OPERATIONAL COSTS IN HYBRID COOLING

DATA CENTERS WITH RENEWABLE ENERGYShaoming Chen, Yue Hu, and Lu Peng

Division of Electrical & Computer Engineering Louisiana State University, LA, US {schen26, yhu14, lpeng}@lsu.edu

Abstract

The electricity cost of data centers dominated by server power and cooling power is growing rapidly. To tackle this problem, inlet air with moderate temperature and server consolidation are widely adopted. However, the benefit of these two methods is limited due to conventional air cooling systems ineffectiveness caused by re-circulation and low heat capacity. To address this problem, hybrid air and liquid cooling, as a practical and inexpensive approach, has been introduced. In this paper, we quantitatively analyze the impact of server consolidation and temperature of cooling water on the total electricity and server maintenance costs in hybrid cooling data centers. To minimize the total costs, we proposed to maintain sweet temperature and ASTT (available sleeping time threshold) by which a joint cost optimization can be satisfied. By using real world traces, the potential savings of sweet temperature and ASTT are estimated to be average 18% of the total cost while 99% requests are satisfied compared to a strategy which only reduces electricity cost. The co-optimization is extended to increase the benefit of the renewable energy and its profit grows as the more wind power is supplied Keywords: Data Center, hybrid cooling, cost optimization, renewable energy __________________________________________________________________________________________________________________

1. INTRODUCTION The total cost of ownership (TCO) in data centers

consists of onetime capital costs incurring only at the

beginning or upgrade stage of data centers and monthly

recurring operational costs including electricity cost,

maintenance cost and salaries (Barroso, L. A. and Hölzle, U.

2009). According to recent reports (Ahmad, Faraz and

Vijaykumar, T. N. 2010), the TCO is dominated by the

operational costs, among which salaries are largely not a

technical but an economic factor. Therefore, we focus on

optimization of electricity and maintenance costs in this

work.

The growth of the cost of electricity consisting of server

power and cooling power outpaces expectations. In 2011,

U.S. data centers spent about $7.4 billion in electric power

among which server power and cooling power contribute

significantly to the total (Pelley, S. et al. 2009). Several

studies try to throttle this increase, though few of them

consider the cost of server maintenance.

Prior works employ two methods to reduce energy cost:

increasing server consolidation and increasing inlet air

temperature. Server consolidation is a powerful tool which

has been widely adopted to gain high energy efficiency of

server, which results from keeping active servers in high

utilization by turning off overprovisioned servers (Qouneh,

A, et al. 2011). As an alternative approach to save server

power, Dynamic Voltage and Frequency Scaling (DVFS) is

also used (Elnozahy, M. et al, 2002). However, the benefit

of DVFS is shrinking because the leakage power is

increasing and the voltage of processors is getting very close

to its limit (Meisner, D. et al. 2009). In addition, DVFS only

affects CPU power which amounts to 30% of server power

(Pelley, S. et al. 2009). Server consolidation remains as an

effective and practical method to save server power.

To reduce cooling power, increasing inlet air temper-

ature is a common method since increasing inlet air

temperature by just one degree can reduce cooling energy

consumption by 2 to 5 percent (California Energy

Commission n.d.). However, the room of inlet air

temperature can be raised is very limited due to the

constraint of server temperature below the critical

temperature. To keep the constraint with a low cost, there

are several prior works advocating thermal-aware workloads

placement which distributes workloads according to the

thermal map of data centers (Moore, J. et al. 2005).

Unfortunately, these methods cannot maintain energy

efficiency of traditional air cooling by keeping high inlet air

temperature when data centers are in high utilization

(Qouneh, A, et al. 2011). Therefore, a novel cooling system

is demanded.

As a practical and inexpensive solution of liquid cooling

(Huang, W. et al. 2011), a hybrid cooling system which

combines air and liquid cooling has been proposed and

deployed in data centers such as Aquasar, the first hot water

cooled supercomputer prototype (Zimmermann, Severin, et

doi: 10.29268/stcc.2014.2.1.4



al July 2012). The hybrid cooling system uses water to cool

down high power density components such as processors

and memory which dominate total heat dissipated in servers,

while other auxiliary components show low power density

are still cooled down by air cooling. In this way, hybrid

cooling can remove a mass of heat from datacenter with less

power than conventional air cooling.

In addition to the electricity cost coming from servers

and cooling systems, hardware maintenance cost is also

considerable. According to a typical new multi-megawatt

datacenter in the United States, the cost of server repair and

maintenance is approximately 50% of the costs of server

power and cooling power (Barroso, L. A. and Hölzle, U.

2009). Based on the empirical data of a HPC datacenter,

disks are the most frequently replaced components.

Resulting from server consolidation, frequent turning off

servers or transition between active state and sleeping state

incurs the cost of disk maintenance due to the limited start-

stop cycles for disks (Chen, Y. et al. 2005). Additionally,

higher inlet water temperature increases the cost of CPU and

memory maintenance, since every 10°C increase over 21°C

decreases the lifetime reliability of electronics by 50%

(Patterson, M. K. 2008). Therefore, rather than restricting

chip temperature below a certain threshold, we can balance

the saving of the electricity costs and the increase of the

costs of hardware maintenance through manipulating inlet

water temperature and smoothing the variation of the

number of active servers.

On the other hand, as three years electricity bills of

modern data centers grow over the server equipment cost

(Brill, K. 2007), the sustainability of data centers is

becoming one of top concerns of their owners. Driven by

soaring conventional energy price and the global warming,

the owners swift their power sources to renewable energy

such as wind, solar, and tidal power. We focus on

integrating wind power into our proposed optimization of

electricity and server maintenance costs since wind energy

is cheaper and widely used to power large-scale facilities

(Patel, M. 1999).

The contributions of our work are shown in the

following.

• We set up analytical models for server power, cooling

power and hardware maintenance model in hybrid cooling

data center for the quantitative evaluation. To our best

knowledge, we first build a comprehensive framework

which covers the evaluation of these costs. This framework

provides foundations to optimize the total cost in hybrid

cooling data centers.

• We propose a tradeoff between electricity cost and

maintenance cost. In this work, we show that the typical

optimizations (high inlet water temperature and aggressive

server consolidation) which reduce only electricity cost

could hurt the maintenance costs.

• To minimize the electricity cost and the maintenance

cost, we develop a joint optimization scheme based on

dynamic optimal water inlet temperature and server

consolidation. Our simulation results show that the method

can gain considerable savings of these costs.

• We extend our cost optimization to exploit the benefit

of the wind power. It increases the cost saving of the wind

power and this benefit grows as the more wind power is

supplied based our experiment.

The rest of our paper is organized as follows: we

describe the structure of hybrid cooling in section 3. In

section 4, we build models related to electricity cost and the

cost of server maintenance. We propose cost optimization

methods in section 5. In section 6, we setup a datacenter

model with server performance model and response analysis.

In section 7, we analyze the result of these two methods and

show their potential savings. Finally, we conclude the paper

in section 8.

2. RELATED WORKPrior works of the cost optimization of data centers fall

into two categories: the optimization of electricity cost

(Raghavendra, R. et al.2008) and the optimization of

hardware maintenance cost. To decrease energy consumption

of datacenters, many studies were addressed from server

level (Meisner, D. et al. 2009 ,and Meisner, D. and Wenisch,

T. F. 2012) , rack level (Ranganathan, P. et al. 2006) and

data center (Chen, Y. et al. 2005, Fan, X, et al, 2007 , Lin,

M. et al. April 2011, and Srikantaiah, S. et al. 2008). These

focused on increasing server energy efficiency and reducing

server idle power. On the other hand, Moore et al. (2005)

introduced thermal-aware workloads placement to reduce

cooling power in traditional air cooling data centers. On the

contrary, other researchers employed advanced

infrastructures of cooling systems to solve energy

inefficiency of traditional air cooling (Barroso, L. A. and

Hölzle, U. 2009. Hwang, D. C. et al 2011, and Rubenstein,

B. A. et al 2010). However, all these works just aimed at the

reduction of either cooling power or server power.

To capture an abroad scope of energy savings, several

architects proposed approaches (Ahmad, Faraz and

Vijaykumar, T. N. 2010, Huang, W. et al. 2011, Pelley, S. et

al. 2009, Qouneh, A, et al. 2011) for optimization of cooling

power and server power. For an example, Pelley et al. (2009)

set up a comprehensive framework of total data center power

in data centers to optimize server power and cooling power.

Ahma, F. et al. (2010) proposed a joint optimization of

server power and cooling power with guaranteeing response

time. However, all of these works did not consider the

increment of the costs of hardware maintenance.

On the other hand, several papers discussed the issue

related to hardware maintenance in data centers (Li, S. et al.

2011, Schroeder, B. and Gibson, G. A. 2007, and Srinivasan,

J. et al. 2005). Schroeder et al. (2007) analyzed disk

replacement rate based on the empirical data, which inspired

researchers to study the reliability of hardware in servers.

Unlike the studies focusing on the optimization of

electricity cost or hardware maintenance in data centers, our



approach covered them both. Additionally, though Y. Chen

et al. (2005) minimized the cost of energy and disk

maintenance by combing DVFS and server consolidation,

the author did not discuss cooling cost and maintenance cost

of other components such as processors and memory in

servers.

3. HYBRID COOLING The structure of hybrid cooling in modern data centers is

shown in Figure 1. The closed liquid loop between the

chiller and racks is designed to remove heat dissipation from

the racks. The cool water absorbs heat dissipation from the

racks and returns back to the chiller with heat. In the closed

liquid loop of a rack, the water cooled in the intermediate

Heat Exchanger (HTX) is pumped into servers. In a server,

the water flowing through a liquid cooled plate takes away

power dissipated by processors and memory. Other auxiliary

components such as disks, power supply, and chipsets on

motherboard are still cooled by the air condition as

traditional data centers since these components dissipate less

power and, more importantly, exhibit lower power density

compared with processors and DRAMs.

4. COST MODELS To optimize the electricity cost and the hardware

maintenance cost, we setup the cost models which

quantitatively estimate the impact of server consolidation

and inlet water temperature on the costs when hybrid cooling

is used.

4.1 ELECTRICITY COSTS. The power of a typical data center includes server power,

cooling power and power distribution loss. For power

distribution loss, PDU and UPS draw 10% of load power

(Srinivasan, J. et al. 2005). In the following context, the

models related to server power and cooling power is

addressed.

F

or

serv

er power, Pservers consists of the sum of all active server

power and the sum of sleeping server power. The power for

all servers is written as:

𝑃𝑠𝑒𝑟𝑣𝑒𝑟𝑠 = ∑ 𝑃𝑆𝑒𝑟𝑣𝑒𝑟𝑠(𝑖) + ∑ 𝑃𝑠𝑙𝑒𝑒𝑝

𝑁𝐼𝑆

𝑗=1

(𝑗)

𝑁𝐴𝑆

𝑖=1

(2)

Here, NAS and NIS denotes the number of active servers

and sleeping servers consuming 6 Watts per server (Ahmad,

Faraz and Vijaykumar, T. N. 2010). For an active server, the

total power consists of the power of processors, the power of

memory and the power of other components. The equation is

listed as follows:

𝑃𝑆𝑒𝑟𝑣𝑒𝑟 = ∑ 𝑃𝑃𝑟𝑜𝑐𝑒𝑠𝑠𝑜𝑟

𝑁𝑆

𝑖=1

(𝑖) + ∑ 𝑃𝑀𝑒𝑚𝑜𝑟𝑦(𝑗) +

𝑁𝑀

𝑗 =1

𝑃𝑂𝑡ℎ𝑒𝑟 (3)

where NS and NM are denoted as the number of sockets

and the number of DIMMs in a server. To simplify the

equation, we assume that all servers in data centers have the

same number of sockets and the number of DIMMs.

For the power model of components in a server

(PProcessor, PMemory and POther), we adopt the linear power

model, which is shown as follows:

𝑃 = (𝑃𝑇𝐷𝑃 − 𝑃𝑖𝑑𝑙𝑒) ∗ 𝑈 + 𝑃𝑖𝑑𝑙𝑒 (4)

where PTDP and Pidle indicate the maximum power and idle

power of components while U denotes server utilization.

The configuration of power model in a server is shown in

Table 1. For processors, its idle power amounts to 10% of

the TDP (Chang, J. et al. 2010), while 4 HDD hard disks are

assumed to be installed in the server to fit memory intensive

applications. The specification is derived from a typical

server (Chang, J. et al. 2010).

According to the hybrid cooling structure, the cooling

power can be divided into two parts: the liquid power and air

cooling power:

𝑃𝑐𝑜𝑜𝑙𝑖𝑛𝑔 = 𝑃𝑙𝑖𝑞𝑢𝑖𝑑_𝑐𝑜𝑜𝑙𝑖𝑛𝑔 + 𝑃𝑎𝑖𝑟_𝑐𝑜𝑜𝑙𝑖𝑛𝑔 (5)

To estimate cooling power, E = Q/COP is employed

where E denotes the energy to remove the heat dissipation

Q from data centers and COP (Coefficient of Performance) is

defined as a metric to evaluate the efficiency of cooling

system (Moore, J. et al. 2005). According to prior studies

(Ahmad, Faraz and Vijaykumar, T. N. 2010),

COPair (coefficient of performance) can be derived in the

following equation: COPair = (0.0068 × T^2 + 0.0008 ×T + 0.458) where T is the inlet air temperature.

𝑃𝑡𝑜𝑡𝑎𝑙 = 𝑃𝑠𝑒𝑟𝑣𝑒𝑟𝑠 + 𝑃𝑐𝑜𝑜𝑙𝑖𝑛𝑔

+ 𝑃𝑝𝑜𝑤𝑒𝑟 𝑑𝑖𝑠𝑡𝑟𝑖𝑏𝑢𝑡𝑖𝑜𝑛 𝑙𝑜𝑠𝑠 (1)

Chiller

HTX

HTX:Intermediate Heat Exchanger

Rack

Hot Water

Cool Water

HTX

Hot Water

Cool Water

Rack

Server

Server

Air

conditi

on

Hot Water

Cool Air

Cool Air Processor

s and

DRAMs

Other

Electronic

Compone

nts

Cool Water

Liquid

Cooled PlateHot Air

Hot WaterServer

pump

HTX

Rack

pump

Cool Water

Figure 1: The Structure of Hybrid Cooling



The power of liquid cooling consists of the power of

chiller and the pump power (Hwang, D. C. et al 2011). The

chiller efficiency for a typical chilled water system is also

written as: COPliquid = E/Q (Beitelmal, M. H. and Patel, C.

D. 2006). COPcooled is written in terms of inlet water

temperature: COPliquid = T ∗ 0.18 − 0.4836 based on the

specification of water-cooled screw compressor chiller

(Catalog of Water-Cooled Screw Compressor Chillers. n.d.).

The water pump power is calculated by this equation

(Hwang, D. C. et al 2011):

𝑃𝑝𝑢𝑚𝑝 = 𝑁 ×𝑉𝑤 × 𝛥𝑃𝑤

𝜂𝑝𝑢𝑚𝑝

(6)

where N is the number of servers and Vw is the water volume

flow rate. ΔPw denotes the water side pressure drop based on

the flow resistance. Finally, ηpump indicates the pump

efficiency.

Overall, the cooling power of the data center is

calculated as follows:

𝑃𝑐𝑜𝑜𝑙𝑖𝑛𝑔 = 𝑄𝑙𝑖𝑞𝑢𝑖𝑑 𝑐𝑜𝑜𝑙𝑒𝑑

𝐶𝑂𝑃𝑙𝑖𝑞𝑢𝑖𝑑(𝑇𝑖𝑛𝑙𝑒𝑡_𝑤𝑎𝑡𝑒𝑟) ∗ 𝑡

+𝑄𝑎𝑖𝑟 𝑐𝑜𝑜𝑙𝑒𝑑

𝐶𝑂𝑃𝑎𝑖𝑟(𝑇𝑖𝑛𝑙𝑒𝑡_𝑎𝑖𝑟) ∗ 𝑡+ 𝑃𝑝𝑢𝑚𝑝

(7)

where t is a time interval during which server components

dissipate the heat Qliquid cooled and Qair cooled . The heat

Qliquid cooled removed by liquid cooling, while the heat

Qair cooled consisting of the heat dissipated other components

in active servers and inactive servers. Shown in the Table 1

is the configuration of hybrid cooling derived from (Hwang,

D. C. et al 2011). the pump power of a server is 0.6 watt and

is negligible compared to the chilling power.

Overall, the electricity cost of the data center is written as:

𝐸𝐶 = 𝐾$ 𝑃𝑡𝑜𝑡𝑎𝑙 (8)

Here, Ptotal and K$ respectively denote the power consumed

the data center and commercial KWH Billing Rate which

comes to 9 cents/KWH.

4.2 THE COSTS OF HARDWARE MAINTENANCE. As we have addressed in the introduction, arising

temperature and frequent consolidation could accelerate

server aging processes. Due to high power density of DRAM

and CPU, we focus on their maintenance cost. In addition,

even though hard disks have a low power density, their

limited number of lifetime start-stop cycles is heavily

impacted by frequent server consolidations. Therefore, we

also take the cost of disks maintenance into account.

Thermal model. To investigate the costs of processor

and memory maintenance, we have setup up thermal models.

The CPU temperature TC is calculated as follows from

(Intel® Core™2 Duo Processor E8000¹ and E7000¹ Series

and Intel® Pentium® Processor E5000¹ Series Thermal and

Mechanical Design Guidelines n.d.):

𝑇𝐶 = 𝑇𝑖𝑛𝑙𝑒𝑡 + (𝜃𝐶𝑃 + 𝜃𝑝) ∗ 𝑄𝐶 (9)

Here, Tinlet is the inlet water temperature and QC is the

power dissipated by the CPU. Thermal resistance of the

processor package and TIM (Thermal Interface Material)

layer is denoted by θCP . The value of θCP is derived from

(Hwang, D. C. et al 2011). The thermal resistance of cold

plate which varies with water flow is denoted by θp ,

according to the specification of Lytron CP20 cold plates

(Hwang, D. C. et al. 2011). For the reliability issue of CPU,

there is a threshold temperature for processor chips as 90°C

(Hwang, D. C. et al 2011).

For DRAM, the temperature TM is given as follows:

𝑇𝑀 = 𝑇𝑖𝑛𝑙𝑒𝑡 + (𝜃𝑀𝑃 + 𝜃𝑃) ∗ 𝑄𝑀𝑃 (10)

where QMP is the power dissipated by memory. Thermal

resistance of chip package of DRAM is denoted by θMP

derived from 错误!未找到引用源。 There is a threshold

temperature for DRAM as 85°C (Lin, J. et al 2007). The

characteristics of thermal package of DRAM, CPU and cold

plates are listed in the Table 1.

Thermal Reliability model of electronic devices. After

we have obtained the chip temperature of electronic devices,

we can predict the lifetime of electronic devices based on the

thermal reliability model of electronic devices. The main

factors to determine the lifetime of electronic devices are

power and chip temperature (EPSMA 2005). For memory,

the lifetime prediction model (Li, S. et al. 2011) is adopted.

MTTF (mean time to failure) is widely used to represents the

predicted lifetime of electronic components for

processors: MTTF = 1 λ⁄ . For the prediction of the lifetime

of processor and memory, λ is the number of failures per

million hours and calculated according to Military Handbook

MIL-HDBK-217F (Reliability Prediction of Electronic

Equipment. Military Handbook n.d. ).

𝜆 = (𝐶1𝜋𝑇 + 𝐶2𝜋𝐸)𝜋𝑄𝜋𝐿

(11)

Server Configurations

Part # TDP(w) Idle power(w)

Processor 2 150W 15W

Memory 8 10W 5W

Others - 124W 73.6W

Hybrid Cooling Configurations

Parameter Value

Tinlet_water (°C) 25

Tinlet_air (°C) 25

Vw (GPM) 1

ηpump 70%

ΔPw(psi) 4.2

Thermal Reliability Configurations

𝜃𝐶𝑃 (ºC/W) 0.3

𝜃𝑀𝑃 (ºC/W) 4.75

𝜃𝑝 (ºC/W) 0.03

Maintenance Cost Configurations

Start-stop cycles for disks 40000

CPU maintenance price ($) 300

Disk maintenance price ($) 200

Memory maintenance price ($) 150

Table 1 Configurations of simulated server



𝜋𝑇 = 0.1exp (−𝐸𝑎

8.617 × 10−5(

1

𝑇𝑝 + 273−

1

298))

(12)

Here, Ea is the effective activation energy (Ev) and TP is the

temperature of electronic devices. The parameters

(C1,C2,πE,πL πQ) are derived from (Reliability Prediction of

Electronic Equipment. Military Handbook n.d. ). We have

scaled the lifetime of CPU and memory according to recent

studies (Li, S. et al. 2011). The lifetime of CPU is expected

to be 7 years when chip temperature is 70 ºC (Srinivasan, J.

et al. 2005), while the expected lifetime of 2GB DRAM is 5

years when its temperature is 65 ºC (Li, S. et al. 2011).

4.3 THE COSTS OF HARDWARE MAINTENANCE. After the thermal reliability of electronic devices has

been introduced, we evaluate the costs of processors and

DRAM maintenance based on their thermal reliability is

given as follows:

RC = the cost of hardware maintenances /MTTF . For a time interval, MTTF is calculated based on their

thermal reliability model and current chip temperature. The

cost of a CPU, a disk and a memory maintenance are $300,

$200 and $150 respectively as shown in Table 1, according

to the maintenance ranging from $300 to $150(Barroso, L. A.

and Hölzle, U. 2009). Based on the thermal reliability model,

the cost of CPU and memory maintenance in an active server

is specified as follows:

𝑅𝐶𝑆𝑒𝑟𝑣𝑒𝑟 = ∑ 𝑅𝐶𝑃𝑟𝑜𝑐𝑒𝑠𝑠𝑜𝑟(𝑖)

𝑁𝑆

𝑖=1

+ ∑ 𝑅𝐶𝑀𝑒𝑚𝑜𝑟𝑦

𝐷𝑀

𝑗=1

(𝑗) (13)

Here, the costs of DRAM and CPU maintenance are

increased by higher inlet water temperature, though the

auxiliary components are still cooled down by air cooling.

Their little heat dissipation, much lower power density and

fixed inlet air temperature result in their little cooling power

and their stable maintenance cost. Additionally, the lifetime

of hard disks is heavily impacted by server consolidations

due to hard disk limited number of lifetime start-stop cycles

(Elerath , J. G. 2000), while the impact of utilization and

temperature is still unclear (Pinheiro, Eduardo, et al. 2007).

On the other hand, switching on/off servers incurs relatively

little maintenance cost of other components such as

processors and memory compared with that of hard disks.

The cost of disk maintenance is computed by the following

equation:

𝑅𝐶𝐷𝑖𝑠𝑘 = 𝑃𝑟𝑖𝑐𝑒

𝑠𝑡𝑎𝑟𝑡 − 𝑠𝑡𝑜𝑝 𝑐𝑦𝑐𝑙𝑒𝑠(14)

As we know, the number of lifetime start-stop cycles for

hard disks is 40000 (Chen, Y. et al. 2005).

Overall, the cost of hardware maintenance of data center

is listed as follows:

𝑅𝐶 = ∑ 𝑅𝐶𝐷𝑖𝑠𝑘

𝑁𝐷

𝑛=1

[𝑁𝐴𝑆(𝑡 − 1) − 𝑁𝐴𝑆(𝑡)]+

+ ∑ 𝑅𝑆𝑒𝑟𝑣𝑒𝑟(𝑘)

𝑁𝐴𝑆

𝑘=1

(15)

[𝐴]+ = 𝐴 𝑖𝑓 𝐴 > 0 𝑜𝑟 [𝐴]+ = 0 𝑖𝑓 𝐴 ≤ 0

where ND and NAS(t) respectively denotes the number of

disks in a server and the number of active servers in the data

center at the time t. [NAS(t) − NAS(t − 1)]+ represents the

number of servers which have been turned off.

Consequently, we have set up models for electricity cost

and the cost of hardware maintenance to evaluate our

approach which optimizes the total cost. The models have

been validated with the costs of our campus datacenters.

5. WIND POWER

Figure 2: The relationship between wind speed and

power

0

1

2

3

4

5

6

0 5 10 15 20 25

Win

d P

ow

er

(KW

)

Wind Speed (m/s)

Cut-in wind Speed

Rated Wind Speed

Cut-off Wind Speed

Workload

Prediction

Request

History

Server Monitor

Server Temperature

& Server Utilization

Server Manager

Future minimal

required number of

active servers

Estimated the Cost of

Hardware Maintenance

Thermal Manager

The Cost of

cooling power

Inlet water

temperatureTurn off or on servers

Figure 4: The overview of costs optimization system

Figure 3: The mismatch between wind power and power

consumption of data center

0

0.2

0.4

0.6

0.8

1

1.2

0 100 200 300

No

rmal

ize

d P

ow

er

Hours

Demanded Power Wind Power



Wind power is captured by wind turbines which converts

kinetic energy into mechanical energy used to produce

electricity. The output power of a typical wind turbine with

respect to the wind speed is shown in Figure 2 (Patel, M.

1999). The power is determined by three important wind

speeds: cut-in wind speed, rated wind speed, and cut-off

speed, and these speeds are specific to a wind turbine. When

the wind speed exceeds cut-in wind speed, the wind turbine

starts to generate electricity. Its power grows as the wind

speed increases, until it reaches the rated wind speed. The

relation between the power and the wind speed could be

shown in the equation: P = 0.5Cp ∅Av3 , whereCp denotes

the power efficiency, ∅ is the air density, A is the rotor

swept area , and v is the wind speed. When the wind speed

is between the cut-off wind speed and rated wind speed, the

output power meets its maximum capacity. The power

sharply drops to zero for protecting its blade assembly when

the wind power exceeds the cut-off wind speed.

For most wind farm sites, the wind speed at most time is

observed between the cut-in wind speed and the rated wind

speed (Patel, M. 1999). As a result, the output power is

greatly sensitive to the wind speed due to their cubic relation.

The resulted fluctuation of the power is shown in Figure 3 of

the wind power trace used in our experiment. Although the

average power demand derived from Saskatchewan-HTTP

trace is approximate to the total wind power in the example,

a considerable mismatch is expected due to their unrelated

factors for their fluctuation: diary human activities and local

weather condition. This mismatch leads to low wind power

usage or requires a huge capacity of energy storage to

reshape the wind power. However, the energy storage incurs

additional capital costs and wastes wind energy, since

required batteries are considerably expensive and their

round-trip energy efficiency ranges from 5% to 25%.

6. COST OPTIMIZATION IN DATA

CENTERS We formulate the total cost in equation (16) based on the

equations (8) (15) with constraints. Since we only focus on

the operational cost of data centers, we pick up a typical

specification for our heuristic data center shown in Table

1.There are two important decision variables Tinlet_waterand

NAS, while other variables are determined by available

servers, server performance and characteristics of traces,

which are also treated as parameters. For example, NS

denotes the total number of servers, while MINS denotes the

minimal required number of active servers which is

determined by traces. Our objective is to minimize the total

cost with the constraints:

min {TC = ∑ 𝑅𝐶𝐷𝑖𝑠𝑘

𝑁𝐷

𝑛=1

∗ [NAS(t − 1) − NAS(t)]+

+ ∑ RCServer(i

NAS

i=1

) + K$ ∗ (PIT

+ Pcooling)}

(16)

Subject to

TC ≤ 90 °C and TM ≤ 85 °C MINS ≤ NAS ≤ NS

The space of feasible solutions of this discrete

optimization is too large, resulting in that exhaustively

searching the global optimal solution is impossible. To

optimize the total cost of electricity and hardware

maintenance, we proposed to trace local optimal solution by

dynamically manipulating Tinlet_water and NAS

corresponding to the fluctuation of workloads.

6.1 THE OVERVIEW OF COST OPTIMIZATION SYSTEM. For the manipulation of Tinletand NAS, we proposed a

structure shown in Figure 4. In this structure, there are four

modules, Workload Prediction, Server Monitor, Server

Manager and Temperature Manger, working together to

reduce the total cost. The workload prediction collects

request history and predicts future request trend based the

history. The module also can predict the future minimal

required number of active servers. The server monitor

collects the temperature and utilization information of

servers and estimates the cost of hardware maintenance.

Acquiring the average server utilization from the server

monitor, the temperature manager adjusts inlet water

temperature. The Server manager dynamic allocates servers

according to the predicted future minimal required number

of active servers.

6.2 THE IMPACT OF INLET WATER TEMPERATURE To investigate the impact of the inlet water temperature

on the total cost, we divide the total cost into two parts: the

cost of cooling power and CPU and memory maintenance

which are affected by the inlet water temperature, and the

other costs which are unaffected denoted by C.

𝑇𝐶 = 𝐾$ ∗ 𝑃𝑐𝑜𝑜𝑙𝑖𝑛𝑔 + ∑ 𝑅𝐶𝑆𝑒𝑟𝑣𝑒𝑟(𝑖)

𝑁𝐴𝑆

𝑖=1

+ 𝐶 (17)

As the inlet water temperature increases, Pcooling

decreases based on the function of COP, while RCServer

increases at the same time according to equations (9)-(13).

There should be an optimal temperature to balance the cost

of cooling power and the costs of CPU and memory

maintenance. The optimal temperature (or sweet temperature)

is adjusted according to workloads since the two costs also

vary with the change of workloads.

6.3 THE IMPACT OF SERVER CONSOLIDATION. The other substantial variable NAS is facilitated by

server consolidation which lively migrate jobs cross servers,

with the upper bound of available servers and the low bound

of service level agreement. Under these constraints, its cost

and benefit are investigated in the following.

The cost of Server Consolidation. It is well known that

server consolidation could save the electricity cost.

Unfortunately, it increases the cost of disk maintenance,

according to equation (15). Furthermore, the transition

between the active state and the sleeping state, servers wastes

energy. We formulate the cost for server consolidation

denoted by Ccs . The cost Ccs per a server is calculated as

follows:



𝐶𝑐𝑠 = ∑ 𝑅𝐶𝐷𝑖𝑠𝑘

𝑁𝐷

𝑗=1

+ 𝑃𝑚𝑎𝑥 ∗ 𝑇𝑇 ∗ 𝐾$ (18)

where TT is the time of the two transitions including two

job migrations (20 seconds for one (Chen, Y. et al. 2005))

and two transitions between the active state and sleeping

state (5 seconds for ACPI S3 state (Linux Documentation

n.d.)). Therefore, TT is estimated to be 50 seconds. Pmax and

K$ respectively represent the maximum power for a server

and denotes commercial KWH Billing Rate.

The benefit of Server Consolidation. The reward of

server consolidation depends on the length of server sleeping

time for once turning off. In other word, the benefit is

determined by the length of the period of turning off servers

without violation of user level agreement. The length of this

period is referred as available sleeping time (AST) which

indicates the maximal server sleeping time. Thus, the benefit

of turning off N servers is denoted by Bsleeping × AST × N.

Here, Bsleeping denotes the benefit of turning off a server for

a minute.

To optimize server consolidation, we define available

sleeping time threshold (ASTT) as follow:

𝐶𝑐𝑠 = ∑ 𝑅𝐶𝐷𝑖𝑠𝑘

𝑁𝐷

𝑗=1

+ 𝑃𝑚𝑎𝑥 ∗ 𝑇𝑇 ∗ 𝐾$ (19)

When the available sleep time of servers is longer than

ASTT, the servers should be turned off. Otherwise, the server

should keep running. We design an algorithm shown in

Figure 5 based on the concept. Generally, the algorithm conservatively turns off servers to mitigate the cost of server

consolidation.

In this algorithm, the decision of turning off servers

requires the knowledge of Future Minimal Required Number

of Active Servers (FMRNAS) which is bound by the

constraint of service level agreement (SLA).The

performance of this algorithm depends on how accurately

FMRNAS is predicted. Therefore, we will introduce two

different predictions combined with the algorithm in the

following sections.

ASTT-P Available sleeping time threshold based on a

perfect prediction. Firstly, we assume that we have a

perfect predictor which indicates FMRNAS accurately.

Given this knowledge, ASTT-P is designed to minimize the

total cost by selecting an available sleeping threshold without

the impact of inaccurate predictions. The exact value of

optimal available sleeping threshold is impossibly obtained

by solving equation (18) since Bsleeping is slightly affected

by other factors such as inlet water temperature.

ASTT-AR: Available Sleeping time threshold based

on the autoregressive model (AR model). The adopted prediction based on the autoregressive model (Stoffer, D. S. and. Shumway, R. H 2010) which is widely used for

pattern prediction is listed in the following equation to

estimate FMRNAS:

𝑆��(𝑇) = (𝐾 + 1)(𝐶 + ∑ 𝐴𝑖 ∗ 𝑆𝑁(𝑇 − 𝑖))

𝑎

𝑖=1

𝑖 = 1 ⋯ 𝑎 (20)

where SN(T) denotes predicted FMRNAS at time T while

SN(T − i) denotes PMRNAS at time (T − i). C and Ai are

tuned to reduce overprovision servers and guarantee the

response time in offline. K is updated according to the

percentage of requests whose response time is satisfied.

When the percentage is below the requirement, K increases

to reserve more servers to handle spike requests. Otherwise,

K is decreased. The goal in this paper is to satisfy more than

99% requests. In our paper, we focus on the benefit of ASTT

by utilizing the mature pattern prediction, though it might be

replaced by advanced tools.

In the following section, the model of a datacenter is built

up to quantitatively evaluate the benefit of sweet temperature

and ASTT.

6.4 CO-OPTIMIZATION WITH WIND POWER The proposed optimization for the wind power is

designed to increase its benefit. Rather than merely targeting

at electricity costs, the optimization reduces the server

maintenance costs at the expense of increased power

consumption. The cost of such overhead could be avoided

when the wind power is larger than the electrical demand of

data centers. It could be explained by the modified objective:

min {TC = ∑ 𝑅𝐶𝐷𝑖𝑠𝑘

𝑁𝐷

𝑛=1

∗ [NAS(t − 1) − NAS(t)]+

+ ∑ RCServer(i

NAS

i=1

) + K$ ∗ (PIT

+ Pcooling − Pwind)}

(21)

Where Pwind denote the wind power at time t. There are

two scenarios regarding to the comparison between the wind

power and the power demand of data centers:

Pwind ≥ (PIT + Pcooling) : Power Over Sufficient

Period(POS period). With over sufficient wind power, the

only concern of this optimization is to reduce the cost of

server maintenance costs by lowering the inlet water

temperature and stopping turning off active servers. The

power consumption of data centers could be increased as

long as it is less than the wind power.

//NAS : the Current Number of Active Servers if NAS < FMRNAS [T]

NAS = FMRNAS [T] Else // Turn off servers If NAS > Max(FMRNAS [T,T+ ASTT]) // Turn off(NAS - Max(FMRNAS [T,T+ ASTT])) servers NAS = Max(FMRNAS [T,T+ ASTT]) Else // Do nothing pass

Figure 5: The algorithm based on ASTT



Pwind < (PIT + Pcooling): Power Insufficient Period (PI

period). When the wind power partially compensates the

power consumption of data centers, ASST-AR can reduce

the electricity costs and server maintenance costs together by

adjusting the inlet water temperature and the number of

active servers. Since the derivative of the total cost in the

factor of them is not affected by the wind power, our method

still reach the optimal point to minimize the total costs at

each interval.

Disk replacement cost. Predicting the comparison

between the wind power and the power demand in the

following intervals is substantial to reduce disk replacement

costs by exploiting the benefit of the wind power. The disk

replacement cost is amortized over the saving of the

electricity costs in the server sleeping time. The saving could

be reduced if the sleeping time includes some POS periods.

Consequently, the longer available sleeping time is

demanded to compensate the disk replacement cost, since

electricity saving can only be gained in the PI periods. The

portion of POS periods in the following time become key to

reduce disk replacement cost with the wind power. To

further reduce disk replacement cost, we design a POS

predictor which is similar to the classical CPU branch

predictor.

Wind Power ASST-AR. ASST-AR as well as sweet

temperature is extended to fully exploit the benefit of the

wind power based on the above discussion. The optimization

of sweet temperature is intuitive; the inlet water temperature

tracks the optimal value to balance the CPU and memory

replacement costs in PI periods, otherwise, it is fixed at the

lowest temperature to minimize the server maintenance cost.

The modified ASST-AR also shows distinct policies in

different periods to minimize the electricity cost and the

replacement costs of disks shown in Figure 6. During POS

periods, turning off active servers is prohibited to avoid

incurred replacement cost; otherwise, the original ASST-AR

still works. For capturing the immediately following POS

period, we design a predictor based on the recent history,

which is widely used in CPU branch prediction in Figure

6.The M is chosen to be 8, since we discovered that it is the

optimal value for our five traces. This modified co-

optimization is referred as Wind Power ASST-AR (WP-

ASST-AR) which reduces electricity and server maintenance

costs by utilizing the wind power.

7. EXPERIMENT SETUP

7.1 DATACENTER. Recalling the models related to the costs of electricity

and hardware maintenance, we combined them with server

performance model and real traces to simulate our prototype

data center which consists of 1024 servers cooled by hybrid

cooling.

Server performance model & response time analysis.

We assume a server in our datacenter provides 2.6

Gbytes/sec service rate and the mean of response time should

be bound by 6 ms for SLA (Chen, Y. et al. 2005). To

calculate the FMRNAS at a time interval, we use GI/G/m

model (Bolch, G. et al. 1998) to determine how many servers

can satisfy a demand based on the following equation:

�� =1

𝜇 +

𝑃𝑚

𝜇(1−𝜌)∗ (

𝐶𝐴2+𝐶𝐵

2

2𝑚) (22)

Pm = ρ

m+12

if ρ ≤ 0.7

Pm = ρm + ρ

2 if ρ > 0.7

where W is the mean response time. 1 μ⁄ is the mean service

time of a server. ρ =λφ

mf is the average utilization of servers.

λ, φ, . CA and CB are derived from trace characteristics

(Ahmad, Faraz and Vijaykumar, T. N. 2010). We use this

performance server and response time model to acquire the

minimal required number of active servers at every time slot.

For a time interval, we choose 5 minutes as the minus unit

(Ahmad, Faraz and Vijaykumar, T. N. 2010).

7.2 TRACES. We use five traces downloaded from the Internet traffic

Archive (Traces in the Internet Traffic Archive n.d.):

Clarknet-HTTP, NASA-HTTP, Saskatchewan-HTTP, UC

Berkeley IP and WorldCup. The lengths of them range from

14 days to 30 days and all of trace files cover several peak

requests. We have scaled the traces to meet our datacenter

performance.

7.3 WIND POWER TRACE. We calculated the wind power based on the relation

between the wind speed and the output power of wind

turbines (Patel, M. 1999), with the specific parameters such

as power efficiency from (错误 !未找到引用源。 The

fourteen day wind speed trace is derived from (Center for

Operational Oceanographic Products and Services n.d.).

Since the average power consumption cross web traces are

//Predictor If Wind Power > power Consumption & Predictor <M Predictor = Predictor + 1 If Wind Power < power Consumption & Predictor >0 Predictor = Predictor – 1 //NAS : the Current Number of Active Servers if NAS < FMRNAS [T] NAS = FMRNAS [T] Else // Turn off servers If NAS > Max(FMRNAS [T,T+ ASTT])&& (Predictor<M/2) // Turn off(NAS - Max(FMRNAS [T,T+ ASTT])) servers NAS = Max(FMRNAS [T,T+ ASTT]) Else // Do nothing pass

Figure 6: The algorithm of Wind power ASST-AR



different due to their distinct patterns, we scale the average

wind power to match the average power consumption for

data centers for each trace. To scrutinize the benefit of our

optimization, the average wind power is scaled to 50%, 75%,

100%, 125%, and 150% of the average power demand in

each trace, which are referred as 50%, 75%, 100%, 125%,

150% Wind Power(WP). The extra power for data centers

comes from the conventional power grid when the wind

power is less than the power demand.

8. RESULTS

8.1 THE IMPACT OF THE OPTIMIZATION BASED ON

SWEET TEMPERATURE As illustrated in equation (17), when the server power is

fixed, the total cost is only related to cooling and hardware

maintenance. Figure 7 illustrates the impact of the inlet

water temperature changing from 15°C to 35°C on the

cooling cost and the cost of hardware maintenance of our

datacenter with 30% utilization. These costs are normalized

against the total costs when inlet water temperature is 15°C.

Increasing inlet water temperature reduces cooling power

especially when the temperature is below 25°C. However,

high inlet water temperature increases the cost of hardware

maintenance of CPU and memory. Observed from Figure 7,

we can find an optimal inlet water temperature (25 °C in this

case) which minimizes the total cost when utilization is fixed

at 30%. In the following context, we will refer the sweet

temperature to the optimal inlet water temperature. This

observation justifies that high inlet water temperature is

reasonable in datacenters when the current average server

utilization is low (below 30%). Otherwise, high inlet water

temperature could hurt the cost of hardware maintenance

during the high utilization.

Figure 8 shows the cooling and hardware maintenance

costs of our datacenter when its utilization varies from 0% to

100%. The right vertical axis of the figure illustrates sweet

temperatures for different utilizations. In the figure, the total

costs for all utilizations are the lowest for the datacenter

cooled by water at corresponding sweet temperatures. When

the utilization of the datacenter is low, warm inlet water

temperature offers more benefit since the cost of cooling

power is larger than the cost of hardware maintenance (e.g.

in our simulation result, the cost of cooling power is 1.65

times of the cost of hardware maintenance when the

utilization is 10%). On the other hand, as the datacenter

utilization increases, we must keep a cold chilling water to

cool down the heating hardware and slow the growth of

hardware maintenance especially when their temperatures

are close to the critical temperatures. Consequently, to

minimize the total costs, inlet water temperature should be

dynamically adjusted according to the data center utilization.

8.2 THE IMPACT OF THE OPTIMIZATION BASED ON

ASTT. The total cost by employing ASTT-P with different

ASTT (ASTT from 5 to 80 minutes) is shown in Figure 9.

The total costs of five traces with different ASTT are

normalized against the total cost of five traces when ASTT is

5 minutes. Observed from this figure, the total cost of five

traces can be reduced considerably when we select an

optimal ASTT for them, though the best ASTT for five

Figure 7: The impact of inlet water temperature on the costs

of cooling power and hardware maintenance

0

0.2

0.4

0.6

0.8

1

1.2

15 20 25 30 35

No

rmal

ize

d C

ost

Inlet Water Temperature (°C)

Cooling costs Replacement Costs

Figure 8: The variation of Sweet Temperature and these

costs corresponding to the utilization of the data center

15

20

25

30

0

0.5

1

1.5

0 10 20 30 40 50 60 70 80 90 100

Tem

pe

ratu

re (°

C)

No

rmal

ize

d C

ost

Utilization (%)

The cost of cooling power

The cost of hardware maintenance

Figure 9: The Normalized total cost reduced by ASTT-P

when ASTT from 5 to 80 minutes in five traces

0.7

0.75

0.8

0.85

0.9

0.95

1

1.05

5 10 20 30 40 50 60 70 80

No

rmal

ize

d T

ota

l Co

st

ASTT (Minutes)

WorldCup UC Berkeley IP

Clarknet-HTTP NASA-HTTP

Saskatchewan-HTTP



traces are not the same (around 30 minutes to 50 minutes)

due to the small variation of the benefit of server

consolidation( Bsleeping ). In the following, we select 40

minutes ASTT as the optimal ASTT for ASTT-P in the five

traces. For ASTT-AR, we also obtained similar curves for

five traces, though the optimal ASTT (around 60 minutes) of

ASTT-AR for five trace is longer than that of ASTT-P due to

the inaccurate prediction and the relatively slow growth of

total cost. 60 minutes ASTT is selected as the optimal ASTT

for ASTT-AR in the five traces for the following analysis.

Figure 10 shows the comparison of the benefits of

ASTT-P (ASTT = 40 minutes) and ASTT-AR (ASTT = 60

minutes) for five traces. All the total costs are normalized

against the total cost of ASTT-P (ASTT = 5 minutes) in five

traces respectively. ASTT-P offers the most benefit

compared with ASTT-AR but requires an unreachable

perfect prediction. As a practical algorithm, ASTT-AR still

saves considerable cost while it guarantees the response time

of 99% requests in the datacenter.

8.3 JOINT OPTIMIZATION BASED ON SWEET

TEMPERATURE AND ASTT-AR



Figure 11 shows the benefit when we combine dynamic

optimal inlet water temperature (i.e. sweet temperature) and

ASTT-AR for the five traces. The total costs of five traces

are normalized against the total costs in five traces with

ASTT-P (ASTT = 5 minutes and inlet water temperature

fixed at 25 °C) as the baseline which represents a typical

scheme. Overall, the total costs of sweet temperature and

ASTT-AR offers 18% savings of total cost of five traces

compared with the baseline in arithmetic mean based on our

simulation results.

8.4 WP-ASST-AR

Figure 10: The total cost of ASTT-P with ASST (60 minutes) & fixed inlet water temperature (25 °C), ASTT-AR with

ASST (60 minutes) & fixed inlet water temperature (25 °C), and ASTT-P with ASST (60 minutes) & sweet temperature

in five traces.

0.6

0.7

0.8

0.9

1

1.1

Clarknet-HTTP NASA-HTTP Saskatchewan-HTTP UC Berkeley IP WorldCup

No

rmal

ize

d T

ota

l C

ost

Traces

ASTT-P ( T = 25 °C ) ASTT-AR (T = 25 °C) ASTT-AR with sweet temperature

Figure 11: The total cost of ASTT-P with ASST (5 minutes), ASTT-AR with ASST (60 minutes), and ASTT-P with ASST-

P with ASST (40 minutes) in five traces

0.7

0.8

0.9

1

1.1

Clarknet-HTTP NASA-HTTP Saskatchewan-HTTP UC Berkeley IP WorldCup

No

rmal

ize

d T

ota

l C

ost

Trace

ASTT-P (ASST = 5 minutes) ASTT-AR (ASST = 60 minutes) ASTT-P (ASST = 40 minutes)

Figure 12: The normalized costs in five traces of the simulated data center powered by 50%WP, 75% WP, 100% WP,

125% WP, 150% WP

0.3

0.5

0.7

0.9

1.1

Clarknet-HTTP NASA-HTTP Saskatchewan-HTTP UC Berkeley WorldCup GM

No

rmal

ize

d T

ota

l Co

st

Workloads

Baseline 50 % WP 75 % WP 100% WP 125% WP 150% WP

Figure 13: The normalized costs in five traces of the simulated data center powered by 50%WP, 75% WP, 100% WP, 125%

WP, 150% WP, and optimized by WP-ASST-AR

0.3

0.5

0.7

0.9

1.1


No

rmal

ize

d T

ota

l Co

st

Workloads

Baseline 50 % WP 75 % WP 100% WP 125% WP 150% WP



The benefit of WP-ASST-AR is revealed by the

comparison between Figure 12 and Figure 13. Figure 12

shows the normalized costs in five traces of the simulated

data center powered by 50%WP, 75%WP, and 100%WP,

and 125%WP, and 150%WP with the baseline which merely

targets electricity costs. The total costs are normalized

against those of the baseline without the wind power. The

shrinking marginal profit of increasing the wind power could

be observed from that the total costs of 50%WP, 75%WP,

100%WP, 125%WP, and 150%WP are 0.77, 0.7, 0.66, 0.63,

and 0.61 in geometric mean respectively. This trend is

confirmed by the results of five traces. Figure 13 also shows

this normalized costs but with WP-ASST-AR. The similar

decrease of the marginal profit could be observed from that

the total costs of 50%WP, 75%WP, 100%WP, 125%WP,

and 150%WP are 0.67, 0.57, 0.51, 0.47, and 0.44 in

geometric mean respectively. However, the benefit of WP-

ASST-AR grows as the wind power increases based on the

facts that with 50%WP, 75%WP, 100%WP, 125%WP, and

150%WP are 0.1, 0.13, 0.15, 0.16, and 0.17 compared with

Figure 12.

Contributing the benefit of WP-ASST-AR, its higher cost

savings of the wind power could be discovered by the

comparison between Figure 14 and Figure 15. Figure 14

shows the total cost savings of the baseline yielded by

50%WP, 75%WP, and 100%WP, and 125%WP, and

150%WP. The increase of the savings shrinks as the wind

power grows, and this trend is also perceived in Figure 15

showing the cost savings with WP-ASST-AR. More

importantly, this cost saving is increased by WP-ASST-AR,

which is consistent to the results of the total costs. The cost

saving of the wind power increases from 22%, 29%, 31%,

35%, and 37% to 27%, 37%, 45%, 50% and 54% in

geometric mean respectively. It implies that the more wind

power are supplied, the more its cost saving could be

obtained by WP-ASST-AR.

9. CONCLUSIONSThe quick growth of electricity bill drives owners of data

centers to employ server consolidation and the high

temperature of data center However, the traditional air

cooling system offers limited benefit of these two approaches

due to its low energy efficiency of cooling power especially.

We build a comprehensive framework which covers the

costs of server power, cooling power, and hardware

maintenance. Based on the models, we introduce a joint

optimization of the costs of electricity and server

maintenance. The approach gains 18% savings of the total

cost and guarantees the response time of more than 99%

requests. In the future, our framework will incorporate

elaborated reliability models for state of the art servers and

power managements which are also important for

minimizing costs of data center owners.

10. REFERENCESDRR3 DRAM. (n.d.). Retrieved from Sep 29, 2013, from

http://www.micron.com/products/dram.

Catalog of Water-Cooled Screw Compressor Chillers. (n.d.). Retrieved from Sep 29, 2013, http://chillertrader.com/manufacturer/McQuay.

2.5 MW Series Wind Turbine. (n.d.). Retrieved from Sep 29,

2013,http://site.genengy.com/prod_serv/products/wind_turbines/en/2xmw/index.htm

Figure 14: The total cost saving in five traces contributed by 50%WP, 75% WP, 100% WP, 125% WP, 150% WP

10

20

30

40

50

60

70


Tota

l Co

st S

avin

g (%

)

Workloads

50 % WP 75 % WP 100% WP 125% WP 150% WP

Figure 15: The total cost saving in five traces contributed by 50%WP, 75% WP, 100% WP, 125% WP, 150% WP

with WP-ASST-AR

10

30

50

70

Clarknet-HTTP NASA-HTTP Saskatchewan-HTTP UC Berkeley WorldCup GMTota

l Co

st S

avin

g (%

)

Workloads

50 % WP 75 % WP 100% WP 125% WP 150% WP

http://www.micron.com/products/dram

http://chillertrader.com/manufacturer/McQuay

http://site.genengy.com/prod_serv/products/wind_turbines/en/2xmw/index.htm

http://site.genengy.com/prod_serv/products/wind_turbines/en/2xmw/index.htm



Center for Operational Oceanographic Products and Services. Retrieved

from Sep 29, 2013, http://tidesandcurrents.noaa.gov

Ahmad, Faraz and Vijaykumar, T. N. (2010). Joint optimization of idle and

cooling power in data centers while maintaining response time. SIGPLAN

Not. 45, 3 (March 2010), 243-256.

Barroso, L. A. and Hölzle, U. (2009). The Datacenter as a Computer: An

Introduction to the Design of Warehouse-Scale Machines. Morgan and

Claypool Publishers.

Beitelmal, M. H. and Patel, C. D. (2006). Model-Based Approach for

Optimizing a Data Center Centralized Cooling System. Technical Report

No. HPL-2006-67, Hewlett Packard Laboratories,

Bolch, G. et al. (1998). Queueing Networks and Markov Chains: Modeling

and Performance Evaluation with Computer Science Applications, New

York: John Wiley.

Brill, K. (2007), The Economic Meltdown of Moore’s Law and the Green

Data Center, The Uptime Institute

California Energy Commission, “Summer time energy-saving tips for

businesses,” Retrieved from Sep 29, 2013, consumerener-

gycenter.org/tips/business summer.html.

Chen, Y. et al. (2005), Managing Server Energy and Operational Cost in Hosting Centers, in SIGMETRICS '05, 2005, pp. 303-314

Chang, J. et al. (2010), "Green Server Design: Beyond Operational Energy

to Sustainability," in In Proceedings of the 2010 International Conference on Power Aware Computing and Systems, HotPower’10, 2010.

EPSMA (2005), Guidelines to Understanding Reliability Prediction,

Retrieved from Sep 29, 2013, http://www.epsma.org/pdf/MTBF%20Report_24%20June%202005.pdf

Fan, X, et al,(2007) “Power provisioning for a warehouse-sized computer,” In ISCA, pages 13–23

Elerath, J. G. (2000), Specifying reliability in the Disk Drive industry: No

More MTBF, In Proceedings of the Annual Reliability and Maintainability Symposium, pp. 194-199

Elnozahy, M. et al, (2002), Energy-Efficient Server Clusters, In

Proceedings of the Second Workshop on Power Aware Computing Systems, February.

Huang, W. et al. (2012), PTL-C2D: Power and Thermal Limits from Chips

to Datacenters, in Tutorial of ISCA 2012.

Huang, W. et al. (2011), Tapo: Thermal-aware power optimization

techniques for servers and data centers, In Proc. of IGCC.

Hwang, D. C. et al (2011), Energy savings achievable through liquid cooling: A rack level case study, in ITtherm, June 2010, pp. 1-9.

Intel® Core™2 Duo Processor E8000¹ and E7000¹ Series and Intel®

Pentium® Processor E5000¹ Series Thermal and Mechanical Design Guidelines. Retrieved from Sep 29, 2013,

http://www.intel.com/content/dam/doc/design-guide/core-2-e8000-e7000-

pentium-e6000-e5000-celeron-e3000-guide.pdf

Li, S. et al. (2011), System implications of memory reliability in exascale

computing, in High Performance Computing, Networking, Storage and

Analysis (SC), 12-18 Nov. 2011, pp. 1-12.

Lin, J. et al (2007), Thermal modeling and management of dram memory

systems, in ISCA, pp. 312-322.

Lin, M. et al. (April 2011), Dynamic right-sizing for power-proportional data centers, INFOCOM, 2011 Proceedings IEEE , pp.1098-1106

Linux Documentation. latest release 2.6, Retrieved from Sep 29,

2013,http://www.kernel.org/doc/documentation/power/states.txt.

Meisner, D. et al. (2009), Powernap: Eliminating server idle power, In

ASPLOS ’09: pages 205–216.

Meisner, D. and Wenisch, T. F. (2012), DreamWeaver: Architectural Support for Deep Sleep, In ASPLOS,

Moore, J. et al. (2005), "Making Scheduling "Cool": Temperature-Aware

Workload Placement in Data Centers," in Proceedings of USENIX

Patel, M. (1999), Wind and Solar Power Systems, CRC Press

Patterson, M. K. (2008), The Effect of Data Center Temperature on Energy

Efficiency, in ITHERM, May 2008, pp. 1167-1174.

Pelley, S. et al. (2009) , Understanding and absracting total data center

power, in WEED ’09: Workshop on Energy Efficient Design, 2009.

Pinheiro, Eduardo, et al. (2007), Failure trends in a large disk drive population, In FAST

Qouneh, A, et al. (2011), A quantitative analysis of cooling power in

container-based data centers, in IISWC, 2011,pp. 61-71

Ranganathan, P. et al. (2006), Ensemblelevel power management for dense

blade servers, in ISCA ,34(2):66–77, 2006.

Raghavendra, R. et al.(2008), No "power" struggles: coordinated multi-level power management for the data center , in ASPLOS '08, 36(1):48-59,

March 2008

Rubenstein, B. A. et al (2010), "Hybrid cooled data center using above ambient liquid cooling," in ITherm, pp.1-10, 2-5 June 2010

Schroeder, B. and Gibson, G. A. (2007), Disk failures in the real world: what

does an MTTF of 1,000,000 hours mean to you? in FASTT '07, 2007.

Srikantaiah, S. et al. (2008), Energy aware consolidation for cloud

computing, In Proceedings of the 2008 conference on Power aware

computing and systems (HotPower'08), 2008

Srinivasan, J. et al. (2005) , Lifetime reliability: toward an architectural

solution, in Micro, vol. 25, no. 3, pp. 70-80, May 2005.

Stoffer, D. S. and Shumway, R. H (2010), Time Series Analysis and Its Applications, New York:Springer Verlag,

Zimmermann, Severin, et al (July 2012), Aquasar: A hot water cooled data

center with direct energy reuse, Energy, 43(1): 237-245, July 2012

Traces in the Internet Traffic Archive. Retrieved from Sep 29 2013,

http://ita.ee.lbl.gov/

Reliability Prediction of Electronic Equipment. Military Handbook,

Retrieved from Sep 29 2013,

http://snebulos.mit.edu/projects/reference/MIL-STD/MIL-HDBK-217F-Notice2.pdf.

Authors

Shaoming Chen received the bachelor’s and

master’s degrees in Electronic and

Information Engineering from Huazhong

University of Science and Technology,

China. He is currently a PhD student in

Electrical and Computer Engineering, Louisiana State

University, majoring in Computer architecture. His research

interests cover the cost optimization and Green Energy in

data centers.

Yue Hu received his Bachelor’s degree in

Electronic Information Science and

Technology from Central South University,

China, in June 2009. He is currently a PhD

student in Electrical and Computer Engineering, Louisiana

State University. His research interests include branch

predictors, cooling techniques for microprocessors and data

centers, and energy-efficient microprocessor design

Lu Peng received the bachelor’s and

master’s degrees in computer science and

engineering from Shanghai Jiao Tong

University, China, and the PhD degree in

http://tidesandcurrents.noaa.gov/

http://ita.ee.lbl.gov/



computer engineering from the University of Florida,

Gainesville, in April 2005. He is currently an associate

professor with the Division of Electrical and Computer

Engineering at Louisiana State University. His research

focuses on memory hierarchy systems, reliability, power

efficiency, and other issues in processor design. He also has

interest in network processors. He received an ORAU Ralph

E. Powe Junior Faculty Enhancement Award in 2007 and a

Best Paper Award from the IEEE International Conference

on Computer Design in 2001.



A BROKER BASED CONSUMPTION MECHANISM FOR

SOCIAL CLOUDS Ioan Petri1, Magdalena Punceva2, Omer F. Rana1, George Theodorakopoulos1 and Yacine Rezgui3

1 School of Computer Science & Informatics, Cardiff University, UK 2 Institute for Computer and Communication Systems, University of Applied Sciences, Western Switzerland

3 School of Engineering, Cardiff University, UK contact author: [email protected]

Abstract The new consumption without ownership paradigm is leading towards a “rental economy” where people can now rent and use various services from third-parties within a market of “shared” resources. The elimination of ownership has increased the marginal utility of consumption and reduced the risks associated with permanent ownership. In the absence of ownership the consumption in the global marketplace has become more dynamic and has positively impacted various economic and social sectors. The concept of “consumption without ownership” can also be used in the area of cloud computing where the interaction between clients and providers generally involves the use of data storage and computational resources. Although a number of commercial providers are currently on the market, it is often beneficial for a user to consider capability from a number of different ones. This would prevent vendor lock-in and more economic choice for a user. Based on this observation, work on “Social Clouds” has involved using social relationships formed between individuals and institutions to establish Peer-2-Peer resource sharing networks, enabling market forces to determine how demand for resources can be met by a number of different (often individually owned) providers. In this paper we identify how trading and consumption within such a network could be enhanced by the dynamic emergence (or identification) of brokers – based on their social position in the network (based on connectivity metrics within a social network). We investigate how offering financial incentives to such brokers, once discovered, could help improve the number of trades that could take place with a network, thereby increasing consumption. A social score algorithm is described and simulated with PeerSim to validate our approach. We also compare the approach to a distributed dominating set algorithm – the closest approximation to our approach.

Keywords: Cloud computing; Social Networks; Dominating Set; Economic Models; Consumption; Ownership.

__________________________________________________________________________________________________________________

1. INTRODUCTIONIn recent years the mode of acquisition and use of resources

has changed significantly, with consumers expecting to use

a product from one vendor for a short amount of time, and

renting rather than owning the product. Resources/products

which fall within this remit have ranged from cars to movies,

games and music recordings. Such a change in emphasis has

been influenced by variability in markets affected by aspects

such as seasonality and the temporary nature of exchange.

Consumers are therefore motivated to participate in a

leasing economy where products are used for a shorter

period significantly preferring to rent than to purchase

(Bendell, 2007), (Levenson, 2007). The ability to participate

in such a sharing economy also provides greater choice for

both the consumer and the provider, enabling a much

greater flexibility in being able to switch between multiple

market offerings, thereby also likely to increase

consumption from consumers by not being restricted to

products or price constraints from a single vendor. The

absence of ownership also enables access to some existing

services that may be inaccessible previously due to high cost

of ownership (Living Planet, 2012). By engaging in such a

non-ownership market, consumers can have access to

greater and increased social status with less cost (Moore,

2008), (Russell, 2007).

Consumption without the cost of ownership has been

identified by economists (Winsper, 2007), (Zukin, 2008) as

a new paradigm in the emerging “sharing economy”.

Increasing the use of social networks, data

mashing/aggregation, availability of software platforms that

facilitate such service/data aggregation and the availability

of handheld devices providing easy access to such platforms

enable users and providers of resources/services to discover

each other and utilize trust relationships developed over

time. Such trust relationships are often encoded in

interaction patterns and behaviours that can be derived from

(on-line) social networks. These relationships therefore

provide the basis for evaluating people that one can trade

with. Increasingly, there is also reluctance in making large

capital purchases of equipment and hardware, making it

more lucrative for users to monetize their time and assets.

Trust and reputation play a central role in an economy based

on the “consumption without ownership” model, therefore

these pre-established relationships would be essential to

encourage greater transactions between participants.

doi: 10.29268/stcc.2014.2.1.3

mailto:[email protected]



In a sharing market being proposed in this paradigm, market

actors (users requesting services and providers offering

them) can also act as micro-entrepreneurs or brokers.

Enabling discovery of suitable providers able to meet

particular, often specific and individual demand from

consumers, becomes an essential tenet in such systems,

especially with the ability to also associate some degree of

confidence in the likelihood of the provider being able to

meet their advertised capability. Consumers become

independent contractors, working for themselves with

control over their working time and working conditions. An

example is the case of ride-sharing companies such as Lyft,

Sidecar or UberX which own no cars themselves but they

sign up instead ordinary car owners: when people need a

ride, they can use mobile apps to find a driver nearby and

ask to be picked up. Airbnb represents another example of

over 300,000 listings from people making their apartments

and homes available for short-term rent, similarly

SnapGoods makes it possible for people to borrow

consumer goods from other people in their neighbourhood

or social network. A variety of examples exist today in other

sectors (Benny, 1973), (Surowieski, 2013).

As brokers play a key role in such a “consumption without

ownership” paradigm, identifying where such brokers

should be situated and how many are needed become

important challenges. Such brokers should also enable trust

relationships to be established between consumers and

providers to allow concerns about liability and competence

to be addressed. Ride sharing companies such as Sidecar,

Lyft and Uber often need to also implement and conform to

certain safety and driver regulations. We believe an

equivalent capability is needed for other domains.

As the demand for data and computational services

increases, the benefits of Cloud computing become

substantial. However, Cloud computing capabilities (as

currently provisioned) can prove to be limited when

accessed through a single provider. Due to vendor lock-in

and specialist data models required from a single vendor, it

is in a user’s interest to explore and interact with multiple

possible Cloud providers. Extending capabilities of Clouds

by using user owned and provisioned devices can address a

number of challenges arising in the context of current Cloud

deployments – such as data centre power efficiency,

availability and outage management. We have investigated

such “Social Clouds” in a number of contexts previously

(Chard et al., 2011), (Petri et al., 2012). Social Clouds are

developed using the observation that like any community,

individual users of a social network are bound by finite

capacity and limited capabilities. In many cases however,

other members (friends) may have surplus capacity or

capabilities that, if shared, could be used to meet the

fluctuating demand. A social cloud makes use of trust

relationships between users to enable mutually beneficial

sharing. Social Clouds are defined in (Chard et al., 2011) as

“a resource and service sharing framework utilizing

relationships established between members of a social

network.” The availability of storage resources and access

latency are also significantly improved – as storage

resources may be found in closer proximity to a user. The

establishment of such Peer-to-Peer (P2P) community

Clouds requires a robust mechanism for controlling

interactions between end-users and their access to services.

For instance, in the context of such a Cloud model, end-

users can contribute with their own resources in addition to

making use of resources provided by others (at different

times and for access to differing services) (Grivas et al.,

2010). There is also increasing interest in developing

“distributed Cloud” platforms, which are able to orchestrate

capability across multiple federated Cloud systems, see for

instance work on such a Cloud orchestration system from

Ericsson in the European UNIFY project (UNIFY, 2013).

In previous work, we have also investigated incentive

models for users to provide services to others (Chard et al.,

2011), (Punceva et al., 2012) – which can range from

bartering of resources, improving the social standing of a

participant within a community or obtaining a financial

reward. We focus on the last of these incentives in this work.

Often in such markets it is necessary for a client to discover

suitable providers of interest. This is generally undertaken

through the use of either a registry service (centralized) to

the use of a discovery request being propagated across the

network (a variety of approaches have been considered,

ranging from flooding, controlled “gossiping” to multiple

federated registries). We propose a decentralized approach

whereby some sellers or buyers may become brokers (or

“traders”) in order to improve their own revenue within a

market place, based on their social connectivity within a

network. We consider a number of graph theoretic measures

(such as connectivity degree, centrality, etc) to identify how

nodes within a social Cloud which were initially buyers or

sellers could turn into brokers – to improve their own

revenue and satisfy service requests within the market. We

map our problem into a dominating set problem in graph

theory and show how our results compare with a distributed

implementation of this algorithm.

In section II we identify the role of brokers within a P2P

market – and how the number of brokers influences the

interaction dynamics within the network. The main concepts

of our approach are outlined in section III. In section IV we

outline our overall methodology, with a description of the

social score algorithm and the metrics (degree & centrality)

used within the algorithm to identify nodes that could be

potential brokers. A description is also provided of the

PeerSim simulator we used to evaluate various scenarios.

Results are presented in section V, with Conclusions and

future work in section VI.



2. RELATED WORKIntermediaries or brokers bring together participants (users

and providers) who have not directly interacted with each

other. Brokers have been used extensively in service-based

systems, primarily as an alternative to service “exchanges”

and registry services. Such brokers may be managed by

external third parties, who have an economic incentive to

provide accurate matchmaking support (utilizing both the

functional capability provided/required including other non-

functional attributes which have been acquired by a broker

over time, such as failure rate, performance, cost, etc)

between the capabilities of a provider and the demands of a

user. Brokering solutions are now beginning to emerge in

Cloud systems also – especially with the emergence of Web

sites such as CloudHarmony (which can support

performance benchmarking across over 100 different Cloud

providers). With Social Clouds (as identified in section I),

broker-based interaction becomes even more important, as

providers can exist over shorter time frames and offer

specialist capability (Sundareswaran, 2012), (Nair et al.,

2010).

Sotiriadis et al. (Sotiriadis et al., 2013) propose a meta-

brokering decentralized approach to manage interactions

between interconnected Clouds. The objective is to support

Cloud interoperability and resource sharing. In this

framework a broker acts on behalf of the user and generates

requests for resources from the Cloud system, based on the

contacted SLAs. The authors demonstrate that the meta-

broker model outperforms a standard broker when the

system contains a high number of concurrent users and

cloudlets submissions.

Sundareswaran et al. (Sundareswaran et al., 2012) propose a

broker-based architecture where brokers help end users

select and rank Cloud service providers based on prior

service requests, enabling users to negotiate SLA terms with

providers. An efficient indexing structure called the CSP

(Cloud Service Provider) index is used to manage the

potentially large number of service providers, utilizing

similarity between various properties of service providers.

The CSP-index can subsequently be used for

service/provider selection and service aggregation.

STRATOS (Pawluk et al., 2012) is a broker service to

facilitate multi-cloud, application topology platform

construction and runtime modification in accordance with a

deployer’s objectives. STRATOS allows an application

deployer to specify what is important in terms of Key

Performance Indicators, so that Cloud system offerings can

be compared and ranked based on these indicators. The

authors demonstrate how an application distributed across

multiple Clouds can decrease the cost of deployment. Duy

et al. (Duy et al., 2012) propose a benchmark-based

approach to evaluate and compare cloud brokers. A

benchmark called Cloud Broker Challenge (CBC) is

employed to describe the cloud providers, cloud consumers,

across 5 difficulty levels – inspired by the successes and

impact of Semantic Web Service Challenge (a set of

benchmark problems in mediating, discovering, and

composing web services) . By introducing difficulty levels

for Cloud brokering and associated scenarios, the authors try

to abstract the fundamental properties of various Cloud

providers to better understand how broker-based solutions

could be applied across multiple providers simultaneously

(Leskovec, 2010).

Our approach complements these situations, in that we

already assume that brokers play an important role within a

Cloud-based resource sharing environment. Our key

objective, instead, is to understand how many brokers

should co-exist within a system to enable better interaction

between users and providers, whilst at the same time

ensuring that the number of brokers is limited.

3. APPROACHA resource trading network has a particular relevance in

Social Clouds – as some resource users & providers may

have a more dominant position in the system, with greater

access to social opportunities for intermediation. The

question of where brokers should be placed within such a

social network becomes significant – primarily to: (i)

increase the flow of ‘goods’ (i.e. facilitate resource

exchange); (ii) increase social welfare within the community.

Social welfare, in this case, measures the number of

potential resource users who are able to find providers that

match their requirements, within their budgets. We consider

a marketplace where buyers and sellers can interact through

an intermediate broker T. The broker receives commission

for each transaction that it facilitates – the broker’s

objectiveis therefore to increase the number of transactions

they participate in and the commission per transaction that

they receive.

Our approach focuses on not having a pre-defined list of

brokers – but understanding how the strategic position of a

node within a network can lead it to be become a broker –

which we refer to as “broker emergence”. Our approach

makes use of two stages to achieve this:

1) Node selection – Select nodes with the highest social

score (as described in section IV-A).

2) Risk assessment – Evaluating the broker’s capacity

of making profit and the associated (financial) risk to

lose the investment.

Broker emergence may be formulated as a dominating set

problem. A dominating set for a graph G = (V, E) is a subset

D of V such that every vertex not in D is joined to at least

one member of D by some edge. The problem minimum



Dominating Set (MDS) requires finding a dominating set of

minimum size. Our goal is to find a set of nodes that will act

as brokers among a network of socially connected nodes

here every node is either a buyer or a seller. The selected set

of brokers should satisfy the following condition:

Market Accessibility: Every non-broker node should be

connected to at least one broker.

Our Market Accessibility condition ensures that every non-

broker node can participate in a transaction either as a buyer

or as a seller. Therefore our problem can be formulated as

finding a Minimum Dominating Set (MDS): given a graph

G(V, E) that represents a social network where vertices

represent users and edges represent friendship links, finding

the set of broker nodes corresponds to finding the minimum

dominating set. The dominating set problem is a classical

NP-complete problem and several approximation algorithms

exist for finding MDS. Kuhn et al. (Kuhn, 2005) propose a

distributed approximation algorithm for finding a

dominating set of minimum size which means every node

uses only local information when executing the algorithm.

This algorithm is particularly suitable for large-scale

decentralised networks and we use it here.

As nodes can change their roles of buyers/sellers, a broker

may be connected to buyers only or sellers only. We

consider two alternatives to overcome this: (i) brokers will

attempt to connect to other brokers (perturbing the original

social structure); (ii) apply the (approximation) algorithm

for finding a connected minimum dominating set instead of

a minimum dominating set (which may not be connected).

Such algorithm although not distributed is presented in

Guha et al (Guha, 1998). It ensures that every broker node is

connected to at least one other broker node.

4. METHODOLOGY We consider a network with an associated set of peer-nodes

P={p1, p2, p3,…,pn}, and a sub-set

S={p1, p2, p3, ..., pm}, m < n, S ⊂ P , where S represents

the set of non-leaf nodes from P . We use two algorithms

for selecting broker nodes: social score algorithm and

dominating set algorithm.

A. Social Score Algorithm

We apply the social score selection algorithm over the set

of non-leaf nodes S. The selection process can be modeled

as a function f (x) : S → T , where the result is a sub-

set of peer-nodes T with the highest social score which we

consider as brokers. The selection protocol for brokers is

built around the notion of social score.

Figure 1. The selection

We use social score as a metric to evaluate nodes and select

brokers. Social score is calculated as an average of three

metrics used to assess the connectivity of a node within a

graph – a node that has greater potential to link other nodes

with each other has a higher social score. The metrics we

use are a node’s: (1) Degree Centrality, (2) Betweenness

Centrality and (3) Eigenvector centrality. Within a graph

G(V, E) where V is the set containing the number of

vertices and E is the set containing the number of edges, we

define the following metrics:

Node’s degree centrality – is simply the number of links

incident to the node:

deg(v) = DC(v)

Node’s betweenness centrality – Betweenness centrality

quantifies the number of times a node acts as a bridge along

the shortest path between two other nodes and is calculated

as the fraction of shortest paths between node pairs that pass

through the node of interest:

Vtstsp

vtspvBC ,)

),(

)/,(()(

where p(s, t/v) is the number of shortest paths between users

s and t that pass through node/user v, and p(s, t) is the

number of all the shortest paths between the two users s and

t.

Node’s Eigenvector centrality – defines the influence of the

node within a network – i.e. it measures how closely a node

is connected to other well connected nodes. It assigns

relative scores to all nodes in the network based on the

concept that connections to high-scoring nodes contribute

more to the score of the node in question than equal



connections to low-scoring nodes. Hence the objective is to

make xi proportional to the centralities of its n neighbors,

i.e.:

n

j

jjii xAxEC1

,

1)(

λ is a constant. In vector notation this can be written as X =

λ Ax, where λ is an eigenvalue of matrix A if there is a non-

zero vector x, such that Ax = λx. Thus, we classify nodes

from the perspective of their social score which is calculated

as:

3

ECDCBCSS

Figure 1 illustrates the set of non-leaf nodes for which we

calculate the social scores as a basis to support broker

selection. Each selected node is given a certain amount of

capital which can either be the same for all nodes, or in

proportion to their social score. Algorithm 1 explains how

brokers are selected based on the social score associated

with nodes. The variable degree represents the number of

current connections whereas capacity represents the

maximum number of connections a node can support. This

variable can be either specified in a configuration file or set

as the maximum degree of the social graph: capacity =

max(degree).

Algorithm 1: Brokers Selection

1: len:=node.capacity; 2: pos:=0; 3: set:=null; 4: for i := 0 to networkSize by 1 do 5: len := pos; max := maxRounds; found := false; 6: while (!found) and (len>0) and (max>0) do 7: max−− ; r[i] := selectNewNode(); rpeer := null; 8: size:=getNodeDegree(r[i]); 9: brokerObserver.calculateScore(r[i].getIndex());

10: if (r[i].isActive()) and (capacity < r[i].capacity) then 11: rpeer := getNodeId(r[i]); 12: markNode(rpeer,r[i]); 13: addToSet(rpeer,r[i],set); 14: found:=true; 15: else 16: if (degree ≤ r[i].degree) and (r[i].degree <

rpeer.getTarget()) then 17: markNode(rpeer,r[i]); 18: addToSet(rpeer,r[i],set); 19: found := true; 20: end if 21: end if 22: end while 23: if (rpeer := null) or (r[i].IsBroker()) or (r[i].Size ≥

rpeer.getTarget()) or (r[i].isActive()) then 24: removeNode(rpeer); 25: end if 26: end for

We use a set variable for storing nodes and a marking

mechanism markNode for identifying all those brokers over

a set of simulation rounds maxRounds. The algorithm starts

by excluding the leaf nodes and calculating the social score

for each the non-leaf nodes. The brokers are then selected as

the nodes with the highest social score out of all non-leaf

nodes.

Algorithm 1 attempts to identify a minimum number of

brokers within the network. The algorithm uses a

classification criteria based on the social score measure

introduced above: nodes with higher social score are

considered better candidates as brokers. The target set of

brokers is composed by the minimum set of nodes with

highest social score whose total capacity is sufficient to

cover all the remaining nodes (sellers and buyers).

B. Dominating Set Approximation: Distributed

Algorithm

The dominating set distributed algorithm is an

approximation algorithm for solving the dominating set

problem from (Kuhn, 2005). The algorithm relies on a

linear programming (LP) formulation of the problem and

consists of two parts/algorithms: first algorithm calculates

the fractional solution to the LP problem and the second

algorithm does the rounding part. The algorithm runs in

constant time and has a provable approximation ratio.

Algorithm 2: LPMDS Approximation 1: x i := 0;

2: calculateδ(i) ;

3: γ(2 )(vi) := δi + 1; δ(vi) := δi + 1;

4: for l:=k-11 to 0 by -1 do 5: (∗ δ(vi) ≤ (∆ + 1)

(l+1)/k, zi := 0∗ ;

6: for m:=k-1 to 0 by -1 do 7: if (δ(vi) ≥ γ2

(vi)l/l+1

) then 8: send ’active node’ to all neighbors; 9: end if

10: a(vi) := |j ∈ Ni|vj is activenode | ; 11: if colori = gray then then 12: a(vi) = 0; 13: end if 14: a(vi) to all neighbors; 15: a(1 )(vi) := maxj∈Nia(vi) ; 16: ∗ a(vi) , a(1 )

(vi) ≤ (∆ + 1)m+1/k∗

17: if δ(vi) ≥ γ(2 )(vi)

l/l+1 then 18: x i := maxxi, a(1 )

(vi)(−

m+1

);

19: end if 20: send x i to all neighbors; 21: if 22: colori := gray ; 23: send colori to all neighbors; 24: δ(vi) := |j ∈ Ni|colorj = white |; 25: end if 26: ∗ zi ≤ (1 + (∆ + 1)

1/k)/γ

(1 )(vi)

(l/l+1)∗ 27: send δ(vi) to all neighbors; 28: γ(1 )(vi) := maxj∈Niδ(vj) ; 29: send γ(1 )

(vi) to all neighbors; 30: γ(2 )(vi) := maxj∈Niγ

(1 )(vj) 31: end for 32: end for

For an arbitrary possibly constant parameter k and

maximum node degree ∆, the algorithm computes the



dominating set of expected size O (k∆2k log(∆)|DSOPT |) in

O(k2) rounds. Where |DSOPT | represents the size of the

optimal dominating set.The output of the algorithm is the

vector x defined for all vi∈V which has values 0 or 1 and

indicates whether a node is in the dominating set or not: if xi

= 1 then node vi is in the dominating set and if xi = 0 the

node is not in the dominating set. Initially each node

independently runs the first part of the algorithm (Algorithm

3 from Kuhn et al. (Kuhn, 2005)) and as a result returns a

fractional value between 0 and 1 for xi variables. In

accordance with this approach we use the following

notations. Initially all nodes are colored white. A node is

colored gray if the sum of the weights of xj for vj ∈ Ni

exceeds 1, i.e., as soon as node is covered. The degree of a

node vi is denoted δi. The largest degree in the network

graph is denoted ∆. The notation jNj i max)1(

is

the maximum degree of all nodes in the closed

neighborhood Ni of vi. Similarly )1()2( max jNj i

is

maximum degree of all nodes at distance at most two from

vi. Note that it is assumed each node knows its 2-hop

neighbors and these values therefore can be computed in at

most two communication rounds. A dynamic degree of a

node vi is denoted by and represents the number of white

odes in Ni, the neighborhood of vi. The output of the first

part Algorithm 3 in (Kuhn, 2005)) are fractional values for

xi and the second part (Algorithm 1 in (Kuhn, 2005)) does

the rounding: it takes fractional xi values as input and

rounds them to 0 or 1.

C. Modeling trading process

In our framework each trade is defined as a function f(t) : S

D where S represents a domain containing originating

nodes and D the domain containing destination nodes. Each

transaction f(t) brings an associated revenue for brokers and

is scheduled to happen at a specific simulation cycle. Within

the protocol we enable peer-nodes to change roles over time

such that buyers and sellers can become brokers or brokers

can become buyers or sellers. In order to validate our

hypotheses, PeerSim (Jelasity et al., 2010) was chosen as a

framework for simulating a number of different scenarios.

The PeerSim simulator uses separate source files for

programming different needed controllers of the simulation

process. We therefore employ a number of different

parameters and controllers for simulating the scenarios

reported in section V. We use an initialization controller

defining the various types of events that can happen during

the simulation and which need to be scheduled during the

simulation. Another controller is used for defining the

network variation at each simulation cycle (e.g. how the

network changes when adding new nodes to the network)

for each round of trading.

An additional controller is allocated as an observer that

collects the results for each experiment. The configuration

file also contains a number of simulation parameters:

•cycles: defines the maximum number of simulation

cycles for each experiment.

•maxCapacity defines the maximum number of

connections allowed for any given node.

•minCpacity defines the minimum number of

connections allowed for any given node.

•minTrades defines the minimum number of trades

scheduled to be run within the system as a whole.

•maxTrade defines the maximum number of trades

scheduled to be run within the system as a whole.

To support a dynamic network formulation – whereby nodes

may be added or removed from the network, we used an

additional network dynamics module.

• control.c1 peersim.dynamics.DynamicNetwork

• control.c1.type vtype

• control.c1.maxsize vmax

• control.c1.add vadd

• control.c1.step vstep

• control.c1.from vfrom

• control.c1.until vuntil

The DynamicNetwork is a module provided within PeerSim

which enables us to define a simulation with a differing

number of nodes at each simulation cycle. It includes

various Java packages initializing a network or modifying it

during simulation. The type parameter represents the type of

the node to be added, the maxsize parameter represents the

maximum number of nodes that one simulation process can

use; the add parameter defines the number of nodes injected

at each step; the step parameter defines the frequency in

cycles for each injected node. The parameter from specifies

the starting number of nodes to simulate while the until

parameter defines the maximum limit on the number of

nodes that the simulation can use.

Table 1. Simulation data set from epinions.com

Nodes 75879

Edges 508837

Nodes in largest WCC 75877 (1.000)

Edges in largest WCC 508836 (1.000)

Nodes in largest SCC 32223 (0.425)

Edges in largest SCC 443506 (0.872)

Average clustering

coefficient

0.2283

Number of triangles 1624481

Fraction of closed triangles 0.06568

Diameter (longest shortest

path)

13

90-percentile effective

diameter

5



We use two different metrics to measure the status of the

system:

(i) Volume of trades: defining the total number of

trades f(t) taking place within the system;

(ii) Average revenue: measures the average revenue

per broker. In equation 5 n defines the number

of trades within the system, m defines the

number of brokers and val(f(ti)) represents the

associated revenue of each trade.

n

i

itfvalm

AR1

)))(((1

5. SIMULATION AND RESULTS Our simulation makes use of social network data about

epinions.com obtained from the Stanford Network Analysis

Platform (SNAP) project (SNAP, 2012).

We use this particular data set as it exposes trust

relationships that are formed within a social network for

product recommendation, exposing the Web of Trust

between individuals. We also felt that this data set is

representative of the types of buyer-seller-broker

relationship that we could foresee within a Social Cloud –

based on the referrals or recommendations made between

people. Table I provides a description of the Epinions data

set.

Experiment 1: This experiment measures how the number of

brokers evolves during the simulation in relation to the

initial network configuration – containing only buyers and

sellers.

Figure 5a. Brokers emergence

The number of nodes, number of edges, average network

degree provided in table I are used to initially start the

network in bi-partite (buyer, seller only) mode.

Brokers are gradually selected based on the algorithm 1

presented in section IV-A. From figure 5a it can be observed

that the simulator needs around 6 simulation cycles to select

brokers. During simulation, the process of broker selection

works in parallel to the actual trading (i.e. trading starts

when the first broker has been identified and continues

during simulation). After 6 simulation cycles the number of

brokers within the system becomes stable – although trading

within the network still continues to take place.

Figure 5b. Degree of brokers

Experiment 2 – This experiment presents how the average

number of nodes connected to brokers evolves during the

simulation – with the initial setup provided in table I.

The average numbers of nodes connected to brokers identify

sellers and buyers within the network.

From figure 5b we observe that the number of nodes

connected to brokers gradually increases within the first 6

cycles of the simulation. Hence, as the number of brokers

increases, the number of nodes (buyers/sellers) associated

with a broker changes. This process of a change in node

interactions (buyer/broker, seller/broker) is strongly related

to the process of broker emergence.

Experiment 3 – Volume of trades when increasing the

number of brokers. In this experiment we measure the

volume of trades when increasing the number brokers

within the network but keeping a fixed number of buyers

and sellers. For running this experiment we extended the

capacity of the network by adding new brokers to the

simulation process. This is ensured by the dynamics



controller presented in section IV-C with the specific

parameter type specifying broker as the type of node to be

added.

Figure 5c. Volume of trades with brokers

From 5c it can be observed that an increase in the number of

brokers by 25% has a direct impact on the volume of trades.

When adding more brokers to the network, the routes

between sellers and buyers increases significantly, thus an

additional volume of trades is identified. In this experiment

the initial setup and the associated number of brokers are

specified by the social score calculated for each node.

Figure 5d. Volume of traders with buyers and sellers


number of buyers and sellers. In this experiment we

measure the volume of trades when increasing the number

of sellers and buyers within the network but keeping a fixed

number of brokers.

Experiments 3 and 4 are the only two simulations where we

change the structure of the network during the simulation to

better understand the impact of: (i) varying number of

intermediaries (Exp. 3); (ii) a change in demand/ supply

over time (Exp. 4). Whereas experiment 3 investigates how

an increase in the number of brokers impacts the volume of

trades, in this experiment we evaluate how the volume of

trades change when expanding the number of buyers and

sellers. As in the previous experiment, the increase of nodes

is handled by employing the dynamics controller with the

specific parameter type set to node. As illustrated in figure

5d an increase of 25% in the number of buyers and sellers

causes an increase in volume of trades. When more buyers

and sellers are added, the number of possible trade options

increases.

However, even if the increase of buyers or sellers causes an

increase in volume of trades, as the number of brokers and

the associated capital are limited the impact on volume of

trades is less significant than the increase in brokers

presented in previous experiment.

Figure 5e. Volumes of trades at broker degrees

Experiment 5 – Volume of trades with regard to broker

degrees. In this experiment we investigate how the volume

of trades evolves with reference to a (broker) node degree.

Brokers are selected according to their social score.

However, each broker has an associated degree parameter

specifying the number of current connections.



Figure 5f. Average revenue per broker

In this experiment we analyze the relationship between the

average broker degree and the volume of trades. Figure 5e

presents various levels of trades with reference to the degree

of a broker. It can be observed that the volume of trades

increases for brokers with higher degrees such as 50 or 75

connections. The impact on volumes of trades for those

brokers with lower degrees is reduced as identified for

degrees of 15 and 25.

Experiment 6 – This is used to measure the average broker

revenue with regard to broker degrees. In addition to

measuring the volumes of trades we also try to quantify the

revenue for each broker. As it can be observed from figure

5f, the average revenue for brokers is strongly related to

their network degree. Whereas for brokers with degree of 15

respectively 25 connections, the impact is low, for brokers

with higher connectivity average revenue is significantly

increased. A higher degree for a broker gives an increased

number of options for performing trades thus leading to an

increased revenue.

Experiment 7 – The number of brokers compared to the

number of nodes in the dominating set when comparing the

Social Score Algorithm with The Dominating Set algorithm.

In this experiment we compare the performance of the

Social Score Algorithm with Dominating Set Algorithm

from the perspective of number of brokers respectively

number of nodes in dominating set. As presented in figure

5g, the dominating set has better performances than the

social score algorithm. It can be observed that at cycle 5 the

number of nodes in dominating set is with 11% lower than

the number of brokers whereas at cycle 30 the difference is

around 12.5%. The performance differences are determined

by two important particularities: (i) graph properties and (ii)

evaluation metrics.

Figure 5g. Number of brokers: Social score vs. Dominating

set problem

Experiment 8 – Volume of trades when comparing the

Social Score Algorithm with The Dominating Set Algorithm.

In this experiment we evaluate the social score algorithm

and the dominating set algorithm from the perspective of the

volume of trades they generate. Figure 5h shows that the

social score algorithm generates a higher volume of trades

than the dominating set algorithm.

Figure 5h. Volume of trades: Social score vs. Dominating

set algorithm



This happens because the social score algorithm considers a

number of different network metrics for selecting the

brokers applied on a predefined social graph whereas the

dominating set algorithm seeks to optimize the dominating

set for an ad-hoc network.

Experiment 9 – Volume of trades when comparing the

Social Score Algorithm with the approach when we

incentivize peer-nodes to become brokers. For running this

experiment we assign an incentive to each peer-node in

order to become a broker. Hence, a peer-node i can decide

to become a broker because according to its subjective

decision function f(x), the broker role enables it to

maximize its revenue. Figure 5i shows that the social score

algorithm generates a higher volume of trades than the

incentivising approach.

Figure 5i. Volume of trades: Social Score vs. Incentivising

peers approach

This happens because the social score algorithm considers a

number of different network metrics for selecting the

brokers applied on a predefined social graph whereas in the

incentive approach some of the brokers can derive from

peer-nodes with poor network attributes such as low

connectivity, low centrality degree, etc. It can be also

observed that in the incentive approach the volume of trades

starts to decay after a certain simulation cycle.

This happens because the brokers derived from peers with

low connectivity are unable to generate a constant volume

of trades within the system.


demand. In this experiment we measure the volume of

trades when increasing demand within the system but

keeping a fixed number of buyers and sellers.

Figure 5k. Average revenue per broker when increasing

demand

In previous experiments we used a fixed demand identifying

a process where one broker can intermediate a single trade

between a buyer and a seller. Here, we consider that

between each buyer and each seller more than one broker-

intermediated trade can take place. Figure 5j illustrates a

comparison between the base case where there is a regular

demand and the cases where we increase the demand by

25% and 50%, respectively. We observe that the highest

differential increase in volume of trades is identified when

increasing demand by 25%.

This differential increase is determined by the capacity

parameter associated with every broker. In this experiment,

we assume that one broker has a configured capacity of

trades that can be intermediated. When increasing the

demand by 25% there is still enough capacity for brokers to

intermediate trades whereas when increasing the demand by

50% the brokers, due to limited capacity, cannot

intermediate all the trades. The request for resources

increases when the demand is increased, hence brokers will

intermediate more trades generating an increase in volume

of trades.

Experiment 11 – Average revenue per broker with an

increase in demand. In this experiment we investigate how

demand impacts the average revenue per broker. As outlined

in experiment 10, an increase in demand leads to an increase

in trade volume, and is affected by the degree of the broker

(Figure 5k). When a broker has a degree of 25, it can be

observed that the impact of demand on average revenue is

limited. When using a broker degree of 50 it can be



observed that the demand correspondingly impacts average

revenue.

Figure 5j. Volume of trades when increasing demand

This difference between various levels of demand in terms

of average revenue is determined by the broker degree.

Although the demand is increased, in some cases a broker

can support only a specific number of trades in relation to

the configured capacity, thus the impact is often limited in

practice.

6. CONCLUSION Consumption without ownership represents an emerging

economical approach with applicability in many contexts.

Enhancing consumption with a broker based market

intermediation is a process commonly used in various

market scenarios to enable better interaction between buyers

and sellers. This concept has found applicability in P2P

markets with extension to Social Clouds where a number of

sellers and buyers are able to use and provide resources –

driven primarily by economic incentives and their reputation

in the market. We investigate a specific mechanism of

broker emergence – whereby nodes in a Social Cloud can

change role from buyers or sellers to brokers – in order to

improve their revenue. We identify the associated benefits

for supporting such broker emergence within a P2P

environment. We also describe how the identification of

such brokers can lead to an improved social welfare within a

community.

A number of scenarios are simulated in PeerSim, by

employing a heuristic social score algorithm for determining

the number of brokers within the network and the associated

generated volume of trades. We investigate how the

algorithm performs when adding more brokers respectively

buyers/sellers by measuring the volume of trades and the

average revenue. In addition we compare the social score

algorithm with a distributed dominating set algorithm.

Broker emergence provides a useful alternative to the pre-

identification of “brokers” within a Cloud system – and

could lead to a dynamic environment which adapts the

number and types of brokers available over time (as the

system connectivity and trade volumes (based on

supply/demand) change.

7. REFERENCES

Stelios Sotiriadis, Nik Bessis, Nick Antonopoulos, Decentralized meta-brokers for inter-cloud: Modeling brokering coordinators for interoperable

resource management, 9th International Conference on Fuzzy Systems and

Knowledge Discovery, pp. 2462-2468, Chongqing, China,2012.

Sundareswaran, S., Squicciarini, A., Lin, D., ”A Brokerage- Based

Approach for Cloud Service Selection,” 2012 IEEE 5th International Conference on Cloud Computing (CLOUD) , pp.558-565, 24-29 June 2012.

Pawluk P., Simmons B., Smit, M. Litoiu M., Mankovski S.; ”Introducing STRATOS: A Cloud Broker Service,” Cloud Computing (CLOUD), 2012

IEEE 5th International Conference on Cloud Computing (CLOUD) , vol.,

no., pp.891-898, 24-29 June 2012.

Ngan Le Duy, S. Tsai Flora, Keong Chan Chee, Kanagasabai

Rajaraman; , ”Towards a Common Benchmark Framework for Cloud Brokers,” 2012 IEEE 18th International Conference on Parallel and

Distributed Systems (ICPADS), vol., no., pp.750- 754, 17-19 Dec. 2012

Fabian Kuhn and Roger Wattenhofer, Constant-Time Distributed

Domianting Set Approximation, Springer Journal for Distributed

Computing, Volume 17, Number 4, May 2005.

S. Guha and S. Kuhler, Approximation Algorithm for Connected

Dominating Sets, Algorithmica, Volume 20, Number 4, 1998.

SNAP, http://snap.stanford.edu/index.html, Last accessed: October 2012.

J. Leskovec, D. Huttenlocher, J. Kleinberg: Signed Networks in Social

Media. 28th ACM Conference on Human Factors in Computing Systems (CHI), 2010.

M ark Jelasity, Alberto Montresor, Gian Paolo Jesi and Spyros Voulgaris. “The Peersim Simulator”. Downloadable at: http: //peersim.sf.net, 2010.

Chard, K. Bubendorfer, S. Caton and O. Rana, “Social Cloud Computing: A Vision for Socially Motivated Resource Sharing”, IEEE Transactions on

Services Computing, 2011. IEEE Computer Society Press.

Ioan Petri, Omer F. Rana, Yacine Rezgui & Gheorghe Cosmin Silaghi,

“Trust Modelling and analysis in peer-to-peer Clouds”, International

Journal of Cloud Computing, No. 2/3, 2012. Inderscience.

Magdalena Punceva, Ivan Rodero, Manish Parashar, Omer F. Rana & Ioan

Petri, “Incentivising Resource Sharing in Social Clouds”. 21st IEEE International Workshop on Enabling Technologies: Infrastructure for

Collaborative Enterprises, WETICE 2012, Toulouse, France, June 25-27,

IEEE Computer Society Press, 2012, pp 185-190.

Living Planet Report 2012,

http://www.wwf.org.uk/what_we_do/about_us/living_planet_report_2012/

Berry, L. L., & Maricle, K. E., Consumption without ownership: What it

means for business. Management Review, 62(9), 44, 1973..

http://www.wwf.org.uk/what_we_do/about_us/living_planet_report_2012/



James Surowieski, Uber Alles, The New Yorker, Financial Page, 2013.

Grivas, S.G.; Kumar, T.U.; Wache, H., "Cloud Broker: Bringing

Intelligence into the Cloud, IEEE 3rd International Conference on," Cloud

Computing (CLOUD), vol., no., pp.544,545, 5-10 July 2010

Nair, S.K.; Porwal, S.; Dimitrakos, T.; Ferrer, A.J.; Tordsson, J.; Sharif, T.;

Sheridan, C.; Rajarajan, M.; Khan, A.U., "Towards Secure Cloud Bursting, Brokerage and Aggregation," IEEE 8th European Conference on Web

Services (ECOWS), vol., no., pp.189,196, 1-3 Dec. 2010

Belk, Russell W.., “Why Not Share Rather Than Own?,” The ANNALS of

the American Academy of Political and Social Science, 611, 126 – 140,

2007.

Bendell, Jem and Anthony Kleanthous, “Deeper Luxury: Quality and Style

When the World Matters,” 2007, ttp://www.wwf.org.uk/deeperluxury/_downloads/DeeperluxuryReport.pdf

Levenson, Eugenia, “The Fractional Life: With jewelry, yachts, and vineyards available by the slice, even the superrich are learning to share,”,

2007,

http://money.cnn.com/magazines/fortune/fortune_archive/2007/03/05/8401282/index.htm

Moore, Amy and Michael Taylor, “Why buy when you can rent? A brief investigation of difference in acquisition mode based on duration,” Applied

Economics Letters, pp. 1-3, 2008

Trendwatching.com, “Transumers: Consumers Driven by Experiences”,

2006, http://www.trendwatching.com/trends/transumers.htm

Winsper, Jeff, “The 6 P’s of Luxury Marketing: The Advanced Model for

Measuring Consumer’s Buying Behavior for Luxury Brands,” , 2007, http://www.winsper.com/6Ps/

Zukin, Sharon and Jennifer Smith Maguire, “Consumers and

Consumption,” Annual Review of Sociology, 30, 173 – 197, 2008.

European FP7 UNIFY Project – “Unifying Cloud and Carrier Networks”. Available at: http://www.fp7-unify.eu/. Last accessed: December 20, 2013.

Authors

Ioan Petri is a computer scientist

in School of Computer Science

& Informatics at Cardiff

University. He holds a PhD in

''Cybernetics and Statistics'' from

Babes-Bolyai University of Cluj-

Napoca, Romania. He has

worked in industry, as a software

developer at Cybercom Plenware

and then as a research assistant on

several projects funded by the Romanian Authority of

Research. Starting with 2009, he collaborated with the

School of Computer Science & Informatics, Cardiff

University, as an internship researcher in Distributed and

Parallel Computing. Between 2011 and 2014 he has worked

as a research associate in School of Engineering of Cardiff

University. Currently he is a member of BRE Institute of

Sustainable Engineering and a research associate in School

of Computer Science & Informatics, Cardiff University. His

research interests are cloud computing, peer-to-peer

economics and distributed systems.

Omer F. Rana is a

Professor of Performance

Engineering in School of

Computer Science &

Informatics at Cardiff

University and Deputy

Director of the Welsh e-

Science Centre. He holds a

Ph.D. in ‘‘Neural

Computing and Parallel

Architectures’’ from

Imperial College (University of London). He has worked in

industry, as a software developer at Marshall

BioTechnology Limited and then as an advisor to Grid

Technology Partners. His research interests extend to three

main areas within computer science: problem solving

environments, high performance agent systems and novel

algorithms for data analysis and management.

Magdalena Punceva is a Senior

Scientists at the Institute for

Computer and Communication

Systems (ISIC), HE-Arc, HES-

SO, Switzerland. She holds a

PhD in peer-to-peer networks

and distributed information

systems from the Swiss Federal

Institute of Technology in

Lausanne (EPFL). She has

worked as a Postdoctoral

Researcher at CERN and spent a

year as a Fulbright Visiting Reaserch Scholar at Rutgers

University, New Jersey, US. Her research interests are in the

area of large-scale networks and algorithms including social

networks, cloud computing and distributed systems.

George Theodorakopoulos is a

Lecturer at the School of

Computer Science & Informatics,

Cardiff University, since 2012.

From 2007 to 2011, he was a

Senior Researcher at the Ecole

Polytechnique Federale de

Lausanne (EPFL), Switzerland.

He is a coauthor (with John Baras)

of the book Path Problems in

Networks (Morgan & Claypool, 2010). He received his

Ph.D. (2007) in electrical and computer engineering from

the University of Maryland, College Park, MD, USA. His

research interests are in privacy, security and trust in

networks.

http://www.trendwatching.com/trends/transumers.htm

http://www.winsper.com/6Ps/

http://www.fp7-unify.eu/



Professor Yacine Rezgui is a

Professor in School of

Engineering at Cardiff

University and a BRE (Building

Research Establishment) Chair

in 'Building Systems and

Informatics'. He is a qualified

architect with an MSc (Diplôme

d’Etudes Approfondies) in

“Building Sciences” (obtained

from Université Jussieu - Paris

6) and a PhD in Computer Science applied to the

construction industry, obtained from ENPC (Ecole

Nationale des Ponts et Chaussées). He has then worked as a

researcher for CSTB (Centre Scientifique et Technique du

Bâtiment) and was involved in a number of national and EU

research projects in the field of document engineering

(DOCCIME), product modelling and Computer Integrated

Construction (ATLAS).

Call for Articles International Journal of Services Computing

Mission The International Journal of Services Computing (IJSC) aims to be a reputable resource providing leading technologies, development, ideas, and trends to an international readership of researchers and engineers in the field of Services Computing. To ensure quality, IJSC only considers extended versions of papers published at reputable international conferences such as IEEE ICWS.

From the technology foundation perspective, Services Computing covers the science and technology needed for bridging the gap between Business Services and IT Services, theory and development and deployment. All topics regarding Web-based services lifecycle study and management align with the theme of IJSC. Specially, we focus on: 1) Web-based services, featuring Web services modeling, development, publishing, discovery,composition, testing, adaptation, and delivery, and Web services technologies as well as standards; 2)services innovation lifecycle that includes enterprise modeling, business consulting, solution creation,services orchestration, services optimization, services management, services marketing, businessprocess integration and management; 3) cloud services featuring modeling, developing, publishing,monitoring, managing, delivering XaaS (everything as a service) in the context of various types ofcloud environments; and 4) mobile services featuring development, publication, discovery,orchestration, invocation, testing, delivery, and certification of mobile applications and services.

Topics The International Journal of Services Computing (IJSC) covers state-of-the-art technologies and best practices of Services Computing, as well as emerging standards and research topics which would define the future of Services Computing. Topics of interest include, but are not limited to, the following:

-Services Engineering-XaaS (everything as a service)-Cloud Computing for Internet-based services-Big Data services-Internet of Things (IoT) services-Pervasive and Mobile services-Social Networks and Services-Wearable services-Web 2.0 and Web X.0 in Web services-Service-Oriented Architecture (SOA)-RESTful Web Services-Service modeling and publishing-Service discovery, composition, and recommendation-Service operations, management, and governance-Services validation and testing-Service privacy, security, and trust-Service deployment and evolution-Semantic Web services-Scientific workflows-Business Process Integration and management-Service applications and implementations-Business intelligence, analytics and economics for Services

58

Call for Articles International Journal of Big Data

Mission Big Data has become a valuable resource and mechanism for the practitioners and researchers to explore the value of data sets in all kinds of business scenarios and scentific investigations. New computing platforms such as cloud computing, mobile Internet, social network are driving the innovations of big data. From government initiative perspective, Obama Administration in United States launched "Big Data" initiative that announces $200 Million in new R&D investments on March 29, 2012. European Union also announced "Big Data at your service" on July 25, 2012. From industry perspective, IBM, SAP, Oracle, Google, Microsoft, Yahoo, and other leading software and internet service companies have also launched their own innovation initiatives around big data.

The International Journal of Big Data (IJBD) aims to provide the first Open Access publication channel for all authors working in the field of all aspects of Big Data. Big Data is a dynamic discipline. One of the objectives of IJBD is to promote research accomplishments and new directions. Therefore, IJBD welcomes special issues in any emerging areas of big data.

Topics IJBD includes topics related to the advancements in the state of the art standards and practices of Big Data, as well as emerging research topics which are going to define the future of Big Data. Topics of interest to include, but are not limited to, the following:

Big Data Models and Algorithms (Foundational Models for Big Data, Algorithms and Programming Techniques for Big Data Processing, Big Data Analytics and Metrics, Representation Formats for Multimedia Big Data)

Big Data Architectures (Cloud Computing Techniques for Big Data, Big Data as a Service, Big Data Open Platforms, Big Data in Mobile and Pervasive Computing)

Big Data Management (Big Data Persistence and Preservation, Big Data Quality and Provenance Control, Management Issues of Social Network enabled Big Data)

Big Data Protection, Integrity and Privacy (Models and Languages for Big Data Protection, Privacy Preserving Big Data Analytics Big Data Encryption)

Security Applications of Big Data (Anomaly Detection in Very Large Scale Systems, Collaborative Threat Detection using Big Data Analytics)

Big Data Search and Mining (Algorithms and Systems for Big Data Search, Distributed, and Peer-to-peer Search, Machine learning based on Big Data, Visualization Analytics for Big Data)

Big Data for Enterprise, Government and Society (Big Data Economics, Real-life Case Studies, Big Data for Business Model Innovation, Big Data Toolkits, Big Data in Business Performance Management, SME-centric Big Data Analytics, Big Data for Vertical Industries (including Government, Healthcare, and Environment), Scientific Applications of Big Data, Large-scale Social Media and Recommendation Systems, Experiences with Big Data Project Deployments, Big Data in Enterprise Management Models and Practices, Big Data in Government Management Models and Practices, Big Data in Smart Planet Solutions, Big Data for Enterprise Transformation)

59

ISSN:2326-7542(Print) ISSN:2326-7550(Online)A Technical Publication of the Services Society

hipore.com

Date post:	08-Oct-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times

International Journal of Cloud Computing (ISSN 2326-7550) Vol. 2, No. 1, January-March 2014 i IJCC...

Documents