Kaleidoscope: Cloud Micro-Elasticity via VM State ColoringRoy Bryant*, Alexey Tumanov†, Olga, Irzak*, Adin Scannell*, Kaustubh Joshi§, Matti Hiltunen§, H. Andrés Lagar-Cavilla§, Eyal de Lara**University of Toronto, †Carnegie Mellon University, §AT&T Labs Research
• Long instantiation times• Coarse granularity of footprint• Network bandwidth & latency overhead
• Common practice: dedicated migration bandwidth!• Wide-range variability in instantiation times
• Fresh workers → cold state• Number of servers with runtime > month – doubled
• Strong need for elasticity• 15.3% of peak capacity –
average usage• Wide-range short-term
variability in workload• Elastic workers – short lived
• 85% of workers – live < 1hr• 10 min → mean worker lifetime
• Allocate resources proportionally to the working set• Stateful replication of VMs (inherit expensive warm state)• Color-guided state propagation
• proactive prefetch• reactive (on-demand)
• Continuum between full copy and minimal/on demand copy
1. VM memory state coloring• Semantically-aware state propagation• Architectural & introspective runtime memory state
information2. Time & space efficient implementation of coloring3. Micro-elastic cloud server
• Color-aware replication & sharing• Near-ideal post-clone QoS• Resource consumption proportional to demand
4. Real world datacenter savings• Fundamental tradeoff:
• Instantiation time vs. warmup period• Continuum between lazy and
eager replication• Continuum between minimal
and maximal footprint• Architecture-based coloring
• Performed on translated page table pages undergoing mandatory canonicalization at clone time
• Extracting the NX bit (executable)• Identifying the kernel/user-space split
• Introspective coloring• Performed on a "frozen" memory image of the master• File cache radix trees• Frame page structure: used/unused physical pages
• State prefetch of consecutive pages in color space• Significant reduction in bandwidth consumption
• Per-color differentiation in the width of the prefetch window• Insight: reduced window size for executable state• User/kernel data windows : 3-4 times wider than executable
• Color-directed memory deduplication• Per-color probabilities of inter-VM page similarities• One-time, color guided calculation of hashes for the master
memory state • Definition: ability to consume physical resources at sub-VM granularity, tracking effective resource demand• Physical host memory• Network bandwidth
• Description: Kaleidoscope workers allocate memory as necessary, an advantage when spikes are short lived.
• Blocking time reduction• Elastic footprint / resource preservation• Scalability• Runtime• QoS
Freq
uenc
y
200018001600140012001000800600400200
0 0 10 20 30 40 50 60
100.00%
80.00%
60.00%
40.00%
20.00%
0.00%
Clone Lifetime (min)
FrequencyCumulative %
footprintmax
SnowFlock
lazy
min
livemigration
replication
eagerKaleidoscope
0
0.2
0.4
0.6
0.8
1.0
0 10 20 30 40 50 60 70 80Frac
tion
of A
lloca
tion
Use
d
Time (seconds)
SupportBanking
EcommerceHttperfOLAP
0
0.2
0.4
0.6
0.8
1
0 20 40 60 80 100 120
Frac
tion
Spen
t Blo
cked
Time (seconds)
0
0.5
1
1.5
2
2.5
3
3.5
4
0 1 2 3 4 5 6 7 8
OLA
P T
hrou
ghpu
t(q
uerie
s pe
r sec
ond)
Simultaneous New Workers
KaleidoscopeCold Standby
Current Elasticity Models: Issues
0 5
10 15 20 25 30 35 40 45
Support Ecommerce Banking
SP
EC
web
"Deg
rade
d" D
urat
ion
(sec
onds
)
Cold StandbyMinimal Clone
Conservative CloneAggressive Clone
KaleidoscopeWarm Static
Kaleidoscope Design Space
Uses of Color
Evaluation Categories
Motivation
Kaleidoscope Contributions
VM Memory State Coloring
Fractional Footprint
Evaluation Results
Kaleidoscope Vision: Cloud u-elasticity
0
20
40
60
80
100
120
140
OLAP Httperf
Run
Tim
e (s
econ
ds)
Cold StandbyMinimal Clone
Conservative CloneAggressive Clone
KaleidoscopeBlocked
Warm Static
0
50
100
150
200
250
300
Httperf Support Ecommerce Banking OLAP
Num
ber o
f Unf
etch
ed
Pag
es (,
000s
)
FreeFile System Cache
User DataKernel DataUser Code
Kernel Code
0
0.2
0.4
0.6
0.8
1
0 5 10 15 20 25 30 35 40
Frac
tion
Spe
nt B
lock
ed
Time (seconds)
Kaleidoscope Worker Blocked for File System CacheUser Data
Kernel DataUser Code
Kernel CodeFree
Reduction of page faults and blocking Near-optimal QoSPrefetching reduces the length of time that prototypes fail to meet SPECweb's minimum acceptable QoS.
Resource preservationKaleidoscope workers avoid unnecessary pages. Unfetched pages save precious bandwidth and are mostly file system cache and a free list.
RuntimeRuntime approaches near-optimal (warm static) as VCPU blocking and page faults are anticipated by colordirected prefetch and eliminated from the critical path.
ScalabilityThe database's OLAP throughput scales well with the number of simultaneous clones
Before
After
Traces in this figure include transferred state plus new allocations.