The nom Profit-Maximizing Operating System
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan Tsafrir
Lightbits Labs Technion—Israel Institute of Technology
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 1 / 43
This talk is about money
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 2 / 43
In the clouds
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 3 / 43
It all began a few years ago, . . .
The Resource-as-a-service (RaaS) Cloud[HotCloud’12,VEE’14,CACM]
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 4 / 43
Trend towards RaaS: Temporal Granularity
3 years on average: buying hardwareMonths: web hostingHours: EC2 on-demand (pay-as-you-go)5 minutes: EC2 Spot Instances (pay-as-you-go)3 minutes: GridSpot1 minute: Profitbricks, Google Compute Engine. . .
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 5 / 43
Trend towards RaaS: Spatial Granularity
Amazon allows clients to dynamically add and remove I/O devicesAmazon allows clients to set a desired I/O rate on aper-block-instance basisGridSpot, ProfitBricks and CloudSigma let their clients compose aflexible bundle—with prices depending on current cost ofresources
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 6 / 43
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 7 / 43
RaaS leads to the need in automatic economic mechanisms inside themachine, and it’s happening today: CloudSigma’s burst pricing
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 8 / 43
CloudSigma’s burst prices, a typical day in July 2012
Image by Kristof Kovacs, taken fromhttp://kkovacs.eu/cloudsigma-burst-price-chart
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 9 / 43
CloudSigma’s burst RAM prices, May 2015–Jan 2016
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 10 / 43
CloudSigma’s burst CPU prices, May 2015–Jan 2016
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 11 / 43
Dynamic pricing is great. . .
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 12 / 43
Except for the fly in the ointment
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 13 / 43
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 14 / 43
For traditional operating systems, performance is everything
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 15 / 43
Costs? They don’t care about no costs
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 16 / 43
Dynamic pricing clouds mandate change
Resource ownership and control the OS is no longer sole ownerEconomic model Cloud provider vs. various clients
Resource granularity fine-grained resource allocationArchitectural support physical and virtual hardware
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 17 / 43
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 18 / 43
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 19 / 43
A profit maximizing operating system
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 20 / 43
Requirements
Maximize profit optimize for both performance and costExpose resources move the kernel out of the way
Isolate applications . . . with a little help from hardware friends
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 21 / 43
nom
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 22 / 43
Design and Implementation
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 23 / 43
Networking: private application stacks
Applications run with private network stacks and device drivers.nom provides default TCP/IP stack and a device driver for the virtiovirtnet virtual device.Applications configure the stack and driver independently.Potentially: application-specific optimizations to stacks anddrivers.Applications choose polling vs. interrupt-driven operations: tradeoff CPU cycles and power for better performance
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 24 / 43
Networking: batching delay
Application controls network stack and device driver behavior onreceive and transmit via the batching delayBatching delay 0: no delay, every packet is run to completionBatching delay Wµsec : batch packets together but don’t let anypacket wait more than WµsecBalancing throughput, latency, and jitterThroughput increases with W (good) up to a pointLatency and jitter increase with W (bad!)What’s the right batching delay? Depends!
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 25 / 43
Utility of network bandwidth=Income-Expenses
Goal: Maximize the profit of a server running in the guest OS.Income: depends on its performance and client SLA.
Expenses: depend on its resource consumption and on currentprices.
Given: price of bandwidth and application loadMeans: throughput, latency, and jitter
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 26 / 43
Assumptions on Utility
Assumption: network server’s utility increases as throughput rises1 Can serve more users2 Can serve users faster
Assumption: server’s utility decreases as latency and jitter rise1 Lower quality of service2 More likely to violate SLO’s
Assumption: server’s utility decreases as the price of bandwidth rises1 Bandwidth becomes more expensive2 Expenses rise⇒ lower profits
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 27 / 43
The bonus utility function
Ubonus = T · (α+γ
L− P)
Key idea: users give the server bonuses for better latency
T throughput in Gbits or application operations/second
α server’s valuation of useful bandwidth ($/Gbit or $/operation)
γ size of the bonus
P bandwidth price in $/Gbit or $/operation
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 28 / 43
The refund utility function
Urefund = T · (max(0, α− β ·L)− P)
Key idea: the server gives its users refunds as latency increases
T throughput in Gbits or application operations/second
α server’s valuation of useful bandwidth ($/Gbit or $/operation)
β extent of the refund
P bandwidth price in $/Gbit or $/operation
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 29 / 43
The penalty utility function
Upenalty = T · (α · (1−min(1,X · N (L0,L, σ)))− P)
Key idea: server pays penalties when it fails to meet SLAs
T throughput in Gbits or application operations/second
α server’s valuation of useful bandwidth ($/Gbit or $/operation)
X penalty factor for not meeting users’ SLOs
L mean latency (µsecs)
L0 the maximal latency allowed by the SLAs
σ the latency’s standard deviation (jitter)
N (L0, L, σ) the probability that a latency sample will not meet the latency SLO
P bandwidth price in $/Gbit or $/operation
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 30 / 43
Implementation
x86-64 SMP virtual machinesDirect access to I/O devicesRuns on bare-metal tooC and (a little) assemblyApplications:
memcached key-value storenhttpd web serverNetPIPE network benchmark
Penalty, refund, and bonus utility
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 31 / 43
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 32 / 43
Methodology: key questions
1 Does optimizing for cost preclude optimizing for performance?2 Does optimizing for both cost and performance improve profit?3 Is changing behavior at runtime important for maximizing profits?
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 33 / 43
Methodology: experiment design
memcached, nhttpd, and NetPIPE running in Linux, OSv, andnom virtual machines with the same∗ resourcesEach experiment runs for 120 secondsFirst 60 seconds:
Serving many users togetherCloud provider charges $1/Gb for bandwidth
Second 60 seconds:Serving a single important user at a timeCloud provider charges $10/Gb for bandwidth
Results reported are averages of 5 runsnom applications know throughput, latency, jitter as a function ofbatching delay and change it when conditions change
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 34 / 43
Effect of batching on throughput and latency
24
26
28
30
32
34
36
38
40
0 5 10 15 20 25 30 35 40 60
65
70
75
80
85
90
95
100
105
110
thro
ughput
[1K
ops/
s]
late
ncy
[use
c]
batching delay 'w' [usec]
throughputlatency
Figure : memcached throughput and latency as a function of batching delay
1 Throughput rises with batching delay up to a point2 Latency is best with batching delay 0 (no delay)
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 35 / 43
Pareto frontier
60
65
70
75
80
85
90
95
100
105
110
24 26 28 30 32 34 36 38 40
late
ncy
[use
c]
throughput [1K ops/s]
024
68
101214
32
Figure : The memcached throughput/latency Pareto frontier
1 No single “best” batching delay setting2 ⇒ No single “best” stack or driver3 ⇒ Applications can pick the right working point according to
current price and loadMuli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 36 / 43
How well does nom perform?
0
10
20
30
40
50
thro
ug
hp
ut
[1K
op
s/s]
memcached many
0.93x 1.01x
0 3 6 9
12 15 18
memcached single
0.93x
1.28x
0
100
200
300
400
500
Linux
OS
v
nom
late
ncy
[use
c] 0.93x 1.01x
0
24
48
72
96
120
Linux
OS
v
nom
0.93x
1.29x
Up to 1.29x better throughput andlatency than Linux
0
2
4
6
8
10
thro
ug
hp
ut
[1K
req
s/s] nhttpd many
0.32x
1.23x
0
1.4
2.8
4.2
5.6
7
nhttpd single
0.91x1.16x
0
1
2
3
4
5
Linux
OS
v
nom
late
ncy
[se
c] 0.27x
2.47x 0
50
100
150
200
250
Linux
OS
v
nom
late
ncy
[use
c]
1.11x 1.22x
1.2x–3.9x better throughput and up to9.1x better latency than Linux and OSv
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 37 / 43
What makes nom fast?
1 Hypervisor friendly: 10K-20K exits vs. 43K-90K for Linux and OSv2 Polling: 1K interrupts vs. 12K-22K for Linux and OSv3 Zero-copy transmit and receive, no allocations on data path4 Application-specific configuration of stacks and drivers
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 38 / 43
Does nom profit more?
0
0.25
0.5
0.75
1
1.25
Linux
OS
v
nom
Profit
[$1
M]
penalty utility
0.00x
11.14x
0
0.25
0.5
0.75
1
1.25
Linux
OS
v
nom
refund utility
0.93x1.04x
0
0.3
0.6
0.9
1.2
1.5
Linux
OS
v
nom
bonus utility
0.91x1.12x
memcached: nom yields up to 11x more profit
0
12
24
36
48
60
Linux
OS
v
nom
Profit
[$1
K]
penalty utility
1.00x
1.30x
0
55
110
165
220
275
Linux
OS
v
nom
refund utility
0.40x
1.81x
0
65
130
195
260
325
Linux
OS
v
nom
bonus utility
0.57x
1.25x
nhttpd: nom yields up to 1.8x more profit
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 39 / 43
What makes nom profitable?
0
0.25
0.5
0.75
1
1.25
ad
p
tpt
lat
Profit
[$1
M]
penalty utility
0.82x
0.14x 0
0.25
0.5
0.75
1
1.25
ad
p
tpt
lat
refund utility
0.97x
0.69x
0
0.3
0.6
0.9
1.2
1.5
ad
p
tpt
lat
bonus utility
0.92x0.73x
memcached static vs. adaptive
0
12
24
36
48
60
ad
p
tpt
lat
Profit
[$1
K]
penalty utility
0.73x
0.94x
0
55
110
165
220
275
ad
p
tpt
lat
refund utility
0.87x0.78x
0
55
110
165
220
275
ad
p
tpt
lat
bonus utility
0.90x0.80x
nhttpd static vs. adaptive
Runtime adaptation is a must for maximizing profit
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 40 / 43
Conclusions
Clouds with dynamic pricing presentchallenges and opportunities.Traditional OSes are a poor fit for suchclouds because they ignore costs.We present nom, a profit-maximizingoperating system.In nom there is no “best” software stack:Application-specific stacks and driversUse utility functions to adapt runtimebehavior and maximize profit.
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 41 / 43
Thank you! Questions?
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 42 / 43
Related work
Userspace I/O and virtual machine device assignmentThe Exokernel [SOSP’95,SOSP’97,TOCS’02]Library operating systems [VEE’07,ASPLOS’11]Dune [OSDI’12] and IX [OSDI’14]Mirage [HotCloud’10], [ASPLOS’13]NoHype [ISCA’10]Arrakis [HotOS’13], [OSDI’14]RaaS [HotCloud’12],[VEE’14],[CACM]
Muli Ben-Yehuda, Orna Agmon Ben-Yehuda, Dan TsafrirThe nom Profit-Maximizing OS VEE, April, 2016 43 / 43