Post on 20-Apr-2020
transcript
2DFQ: Two-Dimensional Fair Queuing for Multi-Tenant Cloud Services
Jonathan MaceBrown University
Peter BodikMicrosoft
Madanlal MusuvathiMicrosoft
Rodrigo FonsecaBrown University
Krishnan VaradarajanMicrosoft
2
Containers / VMs
2
Containers / VMs
Shared Systems:Storage, Database,
Queueing, etc.
2
Containers / VMs
Shared Systems:Storage, Database,
Queueing, etc.
Shared Process
3
Shared ProcessTenants
Request Queues
…
3
Shared ProcessTenants
Request Queues
time
Threads
…
3
Shared ProcessTenants
Request Queues
time
Threads
…
3
Shared ProcessTenants
Request Queues
time
Threads
…
FIFO or ad-hoc
3
Shared ProcessTenants
Request Queues
time
Threads
…
4
Shared ProcessTenants
Request Queues
time
Threads
…
Flows Packet
4
Shared ProcessTenants
Request Queues
time
Threads
…
Flows Packet Switch
Link
4
Shared ProcessTenants
Request Queues
time
Threads
…
Flows Packet Switch
Link
Weighted Fair Queueing (WFQ)
4
Shared ProcessTenants
Request Queues
time
Threads
…
Weighted Fair Queueing (WFQ)
Multiple threadsMultiple concurrent requests
4
…
5
…
5
…
Two-Dimensional Fair Queueing (2DFQ)
5
6
Asize =1
(fast database read)
6
Dsize =10
Asize =1
(fast database read) (slow database scan)
6
Dsize =10
Asize =1
One thread:
time
(fast database read) (slow database scan)
6
Dsize =10
Asize =1
One thread:
time
(fast database read) (slow database scan)
6
Dsize =10
Asize =1
One thread:
time
(fast database read) (slow database scan)
6
Dsize =10
Asize =1
…
One thread:
time
(fast database read) (slow database scan)
6
Dsize =10
Asize =1
…
One thread:
time
high-throughput burst
zero-throughput lull
thro
ugh
pu
t
(fast database read) (slow database scan)
6
Dsize =10
Asize =1
…
One thread:
time
high-throughput burst
zero-throughput lull
Ideal:
thro
ugh
pu
t
(fast database read) (slow database scan)
6
Dsize =10
Asize =1
…
One thread:
time
high-throughput burst
zero-throughput lull
Ideal:
thro
ugh
pu
t
(fast database read) (slow database scan)
6
Dsize =10
Asize =1
…
One thread:
time
high-throughput burst
zero-throughput lull
Ideal:
thro
ugh
pu
t
(fast database read) (slow database scan)
6
Dsize =10
Asize =1
7
Dsize =10
Asize =1
CB
7
Dsize =10
Asize =1
CB
time
7
Dsize =10
Asize =1
CB
time
7
Dsize =10
Asize =1
CB
time
7
Dsize =10
Asize =1
CB
time
time
7
Dsize =10
Asize =1
CB
time
time
7
Dsize =10
Asize =1
CB
time
time
7
Dsize =10
Asize =1
CB
time
time
7
Dsize =10
Asize =1
CB
time
time
thro
ugh
pu
tth
rou
ghp
ut
7
Dsize =10
Asize =1
CB
time
time
thro
ugh
pu
tth
rou
ghp
ut
WFQ, SFQ, …Weighted Fair Queuing
Start-time Fair Queueing
7
Dsize =10
Asize =1
CB
time
time
thro
ugh
pu
tth
rou
ghp
ut
WF²QWorst-Case Fair
Weighted Fair Queuing
WFQ, SFQ, …Weighted Fair Queuing
Start-time Fair Queueing
7
Dsize =10
Asize =1
CB
8
Dsize =10
Asize =1
CB
time
8
Dsize =10
Asize =1
CB
time
8
Dsize =10
Asize =1
CB
time
8
Dsize =10
Asize =1
CB
time
8
Dsize =10
Asize =1
CB
time
time
8
Dsize =10
Asize =1
CB
time
time
8
Dsize =10
Asize =1
CB
time
time
8
Dsize =10
Asize =1
CB
time
time
8
Dsize =10
Asize =1
CB
time
time
Ideal:
8
Dsize =10
Asize =1
CB
time
time
More threads → Opportunity to reduce burstiness
8
Dsize =10
Asize =1
CB
time
time
More threads → Opportunity to reduce burstiness
WFQ, SFQ, WF²Q, …Weighted Fair QueueingStart-time Fair Queueing
Worst-case Fair Weighted Fair Queueing
8
Dsize =10
Asize =1
CB
time
time
More threads → Opportunity to reduce burstiness
WFQ, SFQ, WF²Q, …Weighted Fair QueueingStart-time Fair Queueing
Worst-case Fair Weighted Fair Queueing
2DFQTwo-dimensional Fair
Queueing
8
Challenges
9
A size =1
Tenants with small requests are affected
10
C size =10A size =1
Tenants with small requests are affected
Burstiness is proportional to size of large requests
10
C size =10A size =1
Cloud services:
4+ orders of magnitude variation in cost
Tenants with small requests are affected
Burstiness is proportional to size of large requests
10
C size =10A size =1
Cloud services:
4+ orders of magnitude variation in cost
Tenants with small requests are affected
Burstiness is proportional to size of large requests
size =10A size =1
Size is used by scheduler to make scheduling decisions
C
10
C size =10A size =1
Cloud services:
4+ orders of magnitude variation in cost
Tenants with small requests are affected
Burstiness is proportional to size of large requests
A size =?size =?
Size is used by scheduler to make scheduling decisions
C
10
C size =10A size =1
Cloud services:
4+ orders of magnitude variation in cost
Tenants with small requests are affected
Burstiness is proportional to size of large requests
A
Cloud services:
Estimation using model or moving averages
size =?size =?
Size is used by scheduler to make scheduling decisions
C
10
Key
11
1st quartile 3rd quartile
distribution shape
1st percentilemedian 99th percentile
outliers
Key
11
1st quartile 3rd quartile
distribution shape
1st percentilemedian 99th percentile
outliers
12
UnpredictabilityCost Variation
Quality OfService
12
UnpredictabilityCost Variation
Quality OfService
12
UnpredictabilityCost Variation
Quality OfService
12
Two-DimensionalFair Queueing
13
Dsize =10
Asize =1
CB
time
WFQ
14
Dsize =10
Asize =1
CB
time
WFQ
14
Dsize =10
Asize =1
CB
0 0 0 0
time
WFQ
14
Dsize =10
Asize =1
CB
0 0 0 0
time
WFQ
15
Dsize =10
Asize =1
CB
0 0 0 0
time
WFQ
15
Dsize =10
Asize =1
CB
0 0 0 0
time
WFQ
15
Dsize =10
Asize =1
CB
0 0 0 0
time
WFQ
15
Dsize =10
Asize =1
CB
1 0 0 0
time
WFQ
16
Dsize =10
Asize =1
CB
1 1 0 0
time
WFQ
17
Dsize =10
Asize =1
CB
1 1 0 0
time
WFQ
18
Dsize =10
Asize =1
CB
1 1 0 0
time
v(t)=½
WFQ
18
Dsize =10
Asize =1
CB
2 2 0 0
time
v(t)=1
WFQ
19
Dsize =10
Asize =1
CB
3 3 0 0
time
v(t)=1½
WFQ
20
Dsize =10
Asize =1
CB
4 4 0 0
time
v(t)=2
WFQ
21
Dsize =10
Asize =1
CB
5 5 0 0
time
v(t)=2½
WFQ
22
Dsize =10
Asize =1
CB
6 6 0 0
time
v(t)=3
WFQ
23
Dsize =10
Asize =1
CB
7 7 0 0
time
v(t)=3½
WFQ
24
Dsize =10
Asize =1
CB
8 8 0 0
time
v(t)=4
WFQ
25
Dsize =10
Asize =1
CB
9 9 0 0
time
v(t)=4½
WFQ
26
Dsize =10
Asize =1
CB
10 10 0 0
time
v(t)=5
WFQ
27
Dsize =10
Asize =1
CB
10 10 0 0
time
v(t)=5
WFQ
27
Dsize =10
Asize =1
CB
10 10 10 10
time
v(t)=10
WFQ
28
Dsize =10
Asize =1
CB
0 0 0 0
time
v(t)=0
WF²Q
29
Dsize =10
Asize =1
CB
0 0 0 0
time
v(t)=0
WF²Q
29
Dsize =10
Asize =1
CB
1 1 0 0
time
v(t)=½
WF²Q
30
Dsize =10
Asize =1
CB
1 1 0 0
time
v(t)=½
WF²Q
30
Dsize =10
Asize =1
CB
1 1 0 0
time
v(t)=½
WF²Q
30
Dsize =10
Asize =1
CB
1 1 10 10
time
v(t)=5½
WF²Q
31
Dsize =10
Asize =1
CB
2 2 10 10
time
v(t)=6
WF²Q
32
Dsize =10
Asize =1
CB
3 3 10 10
time
v(t)=6 ½
WF²Q
33
Dsize =10
Asize =1
CB
4 4 10 10
time
v(t)=7
WF²Q
34
Dsize =10
Asize =1
CB
5 5 10 10
time
v(t)=7 ½
WF²Q
35
Dsize =10
Asize =1
CB
6 6 10 10
time
v(t)=8
WF²Q
36
Dsize =10
Asize =1
CB
7 7 10 10
time
v(t)=8½
WF²Q
37
Dsize =10
Asize =1
CB
8 8 10 10
time
v(t)=9
WF²Q
38
Dsize =10
Asize =1
CB
9 9 10 10
time
v(t)=9½
WF²Q
39
Dsize =10
Asize =1
CB
10 10 10 10
time
v(t)=10
WF²Q
40
Dsize =10
Asize =1
CB
11 11 10 10
time
v(t)=10½
WF²Q
41
Dsize =10
Asize =1
CB
11 11 10 10
time
v(t)=10½
WF²Q
41
Dsize =10
Asize =1
CB
11 11 10 10
time
v(t)=10½
WF²Q
41
Dsize =10
Asize =1
CB
0 0 0 0
time
v(t)=0
2DFQ
42
Dsize =10
Asize =1
CB
0 0 0 0
time
v(t)=0
2DFQ
42
Dsize =10
Asize =1
CB
1 1 0 0
time
v(t)=½
2DFQ
43
Dsize =10
Asize =1
CB
1 1 0 0
time
v(t)=½
2DFQ
43
Dsize =10
Asize =1
CB
1 1 10 0
time
v(t)=½
2DFQ
44
Dsize =10
Asize =1
CB
1 1 10 0
time
v(t)=½
2DFQ
44
Dsize =10
Asize =1
CB
1 1 10 0
time
v(t)=½ 1 1
2DFQ
44
Dsize =10
Asize =1
CB
2 1 10 0
time
v(t)=1
2DFQ
45
Dsize =10
Asize =1
CB
2 1 10 0
time
v(t)=1
Eligible 𝑟𝑗 , 𝑖 = Start 𝑟𝑗 − 𝑖𝑛× size 𝑟𝑗
2DFQ
45
Dsize =10
Asize =1
CB
2 2 10 0
time
v(t)=1½
Eligible 𝑟𝑗 , 𝑖 = Start 𝑟𝑗 − 𝑖𝑛× size 𝑟𝑗
2DFQ
46
Dsize =10
Asize =1
CB
3 2 10 0
time
v(t)=2
Eligible 𝑟𝑗 , 𝑖 = Start 𝑟𝑗 − 𝑖𝑛× size 𝑟𝑗
2DFQ
47
Dsize =10
Asize =1
CB
3 3 10 0
time
v(t)=2½
Eligible 𝑟𝑗 , 𝑖 = Start 𝑟𝑗 − 𝑖𝑛× size 𝑟𝑗
2DFQ
48
Dsize =10
Asize =1
CB
4 3 10 0
time
v(t)=3
Eligible 𝑟𝑗 , 𝑖 = Start 𝑟𝑗 − 𝑖𝑛× size 𝑟𝑗
2DFQ
49
Dsize =10
Asize =1
CB
4 4 10 0
time
v(t)=3½
Eligible 𝑟𝑗 , 𝑖 = Start 𝑟𝑗 − 𝑖𝑛× size 𝑟𝑗
2DFQ
50
Dsize =10
Asize =1
CB
5 4 10 0
time
v(t)=4
Eligible 𝑟𝑗 , 𝑖 = Start 𝑟𝑗 − 𝑖𝑛× size 𝑟𝑗
2DFQ
51
Dsize =10
Asize =1
CB
5 5 10 0
time
v(t)=4½
Eligible 𝑟𝑗 , 𝑖 = Start 𝑟𝑗 − 𝑖𝑛× size 𝑟𝑗
2DFQ
52
Dsize =10
Asize =1
CB
6 5 10 0
time
v(t)=5
Eligible 𝑟𝑗 , 𝑖 = Start 𝑟𝑗 − 𝑖𝑛× size 𝑟𝑗
2DFQ
53
Dsize =10
Asize =1
CB
6 6 10 0
time
v(t)=5½
Eligible 𝑟𝑗 , 𝑖 = Start 𝑟𝑗 − 𝑖𝑛× size 𝑟𝑗
2DFQ
54
Dsize =10
Asize =1
CB
6 6 10 0
time
v(t)=5½
Eligible 𝑟𝑗 , 𝑖 = Start 𝑟𝑗 − 𝑖𝑛× size 𝑟𝑗
2DFQ
54
Dsize =10
Asize =1
CB
6 6 10 0
time
v(t)=5½
Eligible 𝑟𝑗 , 𝑖 = Start 𝑟𝑗 − 𝑖𝑛× size 𝑟𝑗
2DFQ
54
Dsize =10
Asize =1
CB
6 6 10 0
time
v(t)=5½
Eligible 𝑟𝑗 , 𝑖 = Start 𝑟𝑗 − 𝑖𝑛× size 𝑟𝑗
2DFQ
6 654
Dsize =10
Asize =1
CB
7 6 10 10
time
v(t)=6
Eligible 𝑟𝑗 , 𝑖 = Start 𝑟𝑗 − 𝑖𝑛× size 𝑟𝑗
2DFQ
55
Dsize =10
Asize =1
CB
7 7 10 10
time
v(t)=6½
Eligible 𝑟𝑗 , 𝑖 = Start 𝑟𝑗 − 𝑖𝑛× size 𝑟𝑗
2DFQ
56
Unknown Costs
57
Unknown Costs
57
Unknown Costs
57
Unknown Costs
57
Unknown Costs
57
Unknown Costs
58
Unknown Costs
58
Unknown Costs
Pessimistic cost estimation
58
Evaluation
Compare 2DFQ to WFQ and WF²Q
Discrete event simulator with Azure Storage workloads
More experiment results in the paper, evaluating:• Burstiness
• Fairness
• Tail latency
59
60
A
50 tenants with size ≈ 1
60
A
50 tenants with size ≈ 1
C
50 tenants with size ≈ 1000
60
A
50 tenants with size ≈ 1
C
50 tenants with size ≈ 1000
16 threads1000 units/second
Costs known by scheduler
60
CA
50 tenants with size ≈ 1 50 tenants with size ≈ 1000
16 threads1000 units/second
Costs known by scheduler
60
A’s
Ser
vice
Rat
e
CA
50 tenants with size ≈ 1 50 tenants with size ≈ 1000
16 threads1000 units/second
Costs known by scheduler
60
A’s
Ser
vice
Rat
e
CA
50 tenants with size ≈ 1 50 tenants with size ≈ 1000
16 threads1000 units/second
Costs known by scheduler
60
A’s
Ser
vice
Rat
e
CA
50 tenants with size ≈ 1 50 tenants with size ≈ 1000
16 threads1000 units/second
Costs known by scheduler
60
A’s
Ser
vice
Rat
e
CA
50 tenants with size ≈ 1 50 tenants with size ≈ 1000
16 threads1000 units/second
Costs known by scheduler
60
A’s
Ser
vice
Rat
e
CA
50 tenants with size ≈ 1 50 tenants with size ≈ 1000
16 threads1000 units/second
Costs known by scheduler
60
A’s
Ser
vice
Rat
e
250 Azure Storage tenants32 threads
1 million units/secondCosts known by scheduler
61
250 Azure Storage tenants32 threads
1 million units/secondCosts known by scheduler
61
250 Azure Storage tenants32 threads
1 million units/secondCosts known by scheduler
T1
61
250 Azure Storage workloads32 threads
1 million units/secondCosts known by scheduler
T1
62
250 Azure Storage workloads32 threads
1 million units/secondCosts known by scheduler
T1
62
250 Azure Storage workloads32 threads
1 million units/secondCosts known by scheduler
T1
62
250 Azure Storage workloads32 threads
1 million units/secondCosts known by scheduler
T1
62
250 Azure Storage workloads32 threads
1 million units/secondCosts known by scheduler
T1
62
250 Azure Storage workloads32 threads
1 million units/secondCosts known by scheduler
T1
62
250 Azure Storage workloads32 threads
1 million units/secondCosts known by scheduler
T1
62
250 Azure Storage workloads32 threads
1 million units/secondCosts known by scheduler
T1
62
Unpredictable WorkloadsWidely Varying Costs
Quality OfService
63
Unpredictable WorkloadsWidely Varying Costs
Quality OfService
63
Unpredictable WorkloadsWidely Varying Costs
Quality OfService
63
Unpredictable WorkloadsWidely Varying Costs
Quality OfService
63
Predictable
2/3 predictable1/3 unpredictable
1/3 predictable2/3 unpredictable
T1
T1
T1
64
Predictable
2/3 predictable1/3 unpredictable
1/3 predictable2/3 unpredictable
T1
T1
T1
64
Predictable
2/3 predictable1/3 unpredictable
1/3 predictable2/3 unpredictable
T1
T1
T1
64
Predictable
2/3 predictable1/3 unpredictable
1/3 predictable2/3 unpredictable
T1
T1
T1
64
Predictable
2/3 predictable1/3 unpredictable
1/3 predictable2/3 unpredictable
T1
T1
T1
64
Predictable
2/3 predictable1/3 unpredictable
1/3 predictable2/3 unpredictable
65
Predictable
2/3 predictable1/3 unpredictable
1/3 predictable2/3 unpredictable
65
Predictable
2/3 predictable1/3 unpredictable
1/3 predictable2/3 unpredictable
65
Predictable
2/3 predictable1/3 unpredictable
1/3 predictable2/3 unpredictable
65
Two-Dimensional Fair Queueing
66
More threads → Opportunity to reduce burstiness
Two-Dimensional Fair Queueing
66
More threads → Opportunity to reduce burstiness
Two-Dimensional Fair Queueing
Partitions requests across threads by size
66
More threads → Opportunity to reduce burstiness
Two-Dimensional Fair Queueing
Partitions requests across threads by size
Co-locates unpredictable and expensive workloads
66
More threads → Opportunity to reduce burstiness
Two-Dimensional Fair Queueing
Partitions requests across threads by size
Co-locates unpredictable and expensive workloads
Unpredictable WorkloadsWidely Varying Costs
Quality OfService
66
More threads → Opportunity to reduce burstiness
Two-Dimensional Fair Queueing
Partitions requests across threads by size
Co-locates unpredictable and expensive workloads
Less burstiness
Reduced tail latency
Unpredictable WorkloadsWidely Varying Costs
Quality OfService
66