Post on 29-Sep-2020
transcript
Load Balancing for Interdependent IoT Microservices
Ruozhou Yu, Vishnu Teja Kilari, Guoliang XueArizona State University
Dejun YangColorado School of Mines
Outlines
2
Background and Motivation
System Modeling
Algorithm Design and Analysis
Performance Evaluation
Discussions, Future Work and Conclusions
IoT: The Future Internet
3
q IoT is the future Internet that connects every aspect of ourwork and life.
Environment
Agriculture
Shopping
Manufacturing
Transportation
Home
Healthcare
Travel
Security
Example IoT Applications
4
TrafficAnalysis
ImageProcessing
ObjectDetection
TemporalAnalysis
TrafficPrediction
Monolithic Applications
5
TrafficAnalysis
ImageProcessing
ObjectDetection
TemporalAnalysis
TrafficPrediction
Crowd
Detection
SignalProcessing
DeviceMatching
TemporalAnalysis
PopulationEstimation
Criminal
Searching
ImageProcessing
ObjectDetection
FaceRecognition
FaceComparison
TemporalAnalysis
FaceComparison
TrafficPrediction
PopulationEstimation
DeviceMatching
SignalProcessing
Microservices
6
TrafficAnalysis
Crowd
Detection
Criminal
Searching
ImageProcessing
ObjectDetection
FaceRecognition
Microservices vs. Edge Computing
7
High delayHigh bw overheadSingle point of failureSecurity concerns
ResponsiveLow bw overheadDistributedSecure
Cloud-based
Edge-based
The Microservice Load Balancing Problem
q Edge-based microservices can be easily saturated.
8
ImageProcessing
ImageProcessing
1
ImageProcessing
The Microservice Load Balancing Problem
q Edge-based microservices can be easily saturated.
9
ImageProcessing
2
The Microservice Load Balancing Problem
q Edge-based microservices can be easily saturated.
q Challenge: interdependent microservices.
10
ImageProcessing
1
ImageProcessing
2
TemporalAnalysis
1Traffic
Prediction
ObjectDetection
1
ObjectDetection
2
ObjectDetection
3
TemporalAnalysis
2
Our Approach: Overview
11
Problem Modeling1) DAG-based interdependency graph (App-Graph).2) Compactly modeled infrastructure (Inf-Graph).3) Flexible application instantiation (Real-Graph).4) Joint instantiation finding & load allocation.5) Application QoS requirements.
Algorithmic Results1) Optimal algorithm for QoS-agnostic problem.2) NP-hardness for QoS-aware problem.3) FPTAS for QoS-aware problem.
Next Steps (Future Work)1) Network-aware load balancing.2) Reliability and security.3) Economics-aware microservice composition.
Outlines
12
Background and Motivation
System Modeling
Algorithm Design and Analysis
Performance Evaluation
Discussions, Future Work and Conclusions
Application with Interdependent Microservices
q General DAG-based application graph (App-Graph).v Captures complex interdependencies, unlike existing line graph-based models.
13
Cloud StorageCloud
Storage
Video Proc.Video Proc.
Motion Detect.Motion Detect.
Anomaly Detect.
Anomaly Detect.
Smart Home
Control
Smart Home
ControlData Proc.Data Proc.
Cmd. CenterCmd.
Center
Security Cameras
Motion Sensors
Ambient Sensors
Data InputData Input API CallAPI CallMicroserviceMicroserviceMicroserviceData Input API CallMicroservice
Interdependency edge• Data distribution ratio: output / input.
Microservice
IoT Infrastructure in the Application’sView
q Inf-Graph: deployed microservice instances & their interactions.
14
AA
BB
CC
DD EE
0.5
0.5
1.0
1.0
1.0
1.0
App-Graph
A1A1
B1B1
C1C1
D1D1 E1E1
D2D2 E2E2δ=2.0
(2, 1)
(1, 5)
(1, 4)
(2, 3)
(2, 6)
(2, 4)
(2, 6)
A1
B1
C1
D1 E1
D2 E2δ=2.0
(2, 1)
(1, 5)
(1, 4)
(2, 3)
(2, 6)
(2, 4)
(2, 6)
Inf-Graph
Microservice Instance• Capacity: input data volume.• Processing delay.
Inter-Instance Communication Channel• Transmission delay (=0 in example).
External Demand
Application Instantiation
q Real-Graph: instantiating the App-Graph in the Inf-Graph.
15
A1A1
B1B1
C1C1
D1D1 E1E1
D2D2 E2E2δ=2.0
(2, 1)
(1, 5)
(1, 4)
(2, 3)
(2, 6)
(2, 4)
(2, 6)
A1
B1
C1
D1 E1
D2 E2δ=2.0
(2, 1)
(1, 5)
(1, 4)
(2, 3)
(2, 6)
(2, 4)
(2, 6)
A1A1
B1B1
C1C1
D1D1 E1E1
D2D2 E2E2δ=2.0
(2, 1)
(1, 5)
(1, 4)
(2, 3)
(2, 6)
(2, 4)
(2, 6)
A1
B1
C1
D1 E1
D2 E2δ=2.0
(2, 1)
(1, 5)
(1, 4)
(2, 3)
(2, 6)
(2, 4)
(2, 6)
Inf-Graph
A1
B1
C1 D1 E1
D2 E2
(1, 1)
(0.5, 6)
(0.5, 5)
(1, 9)
(1, 13)
(1, 7) (1, 13)φ=2.0
A1
B1
C1 D1 E1
D2 E2
(1, 1)
(0.5, 6)
(0.5, 5)
(1, 9)
(1, 13)
(1, 7) (1, 13)φ=2.0
Real-Graph
Selected Instance• Impact ratio: how much data this instance
gets if 1 unit is allocated to the source.• Max cumulative delay from source.• Both can be computed based on this graph.
Source Demand Allocation
Source of Demand• A real-graph has a single source of demand.
(There could be multiple sources in Inf-Graph.)
Problem Statement: Overview
q Inputs:v App-Graph: microservices, interdependencies, data distribution ratiosv Inf-Graph: instances, communication channels, capacities, and delays
q Outputs:v A set of Real-Graphs.v External demand allocation for each Real-Graph.
q Constraints:v Load balancing: total load <= instance capacity * 𝛹.v QoS awareness: maximum delay <= D.
q Objective (optimization version):v Minimize maximum delay of all Real-Graphs.
16
Outlines
17
Background and Motivation
System Modeling
Algorithm Design and Analysis
Performance Evaluation
Discussions, Future Work and Conclusions
BLB: Basic (QoS-agnostic) Load Balancing
q Without delay constraint, the problem can be formulated as LP.
18
Variables: Per-link demand allocation function.
Demand per node = external + flow-in.
find f : L 7! R⇤
s.t. �n = �extn +X
l2Lin(n)
f(l), 8n 2 N ;
�n · cn, 8n 2 N ;
r(vn,w)�n =X
l2Lout(n,w)
f(l), 8n,w 2 Vout(vn).
<latexit sha1_base64="m9PQWmRwqT5p5Kb9x2EeaIVayvI=">AAADQnicjVJLb9QwEHbCqyyPbuHIZcQKtAsoyiIkXkKq4MKhQgti26L1NnIcp7Xq2MF2Cqso/40Lf4Abf4ALB0BcOeA8JNrtHhjJ8qeZ7/OMZybOBTc2DL96/pmz585fWLvYu3T5ytX1/sa1baMKTdmUKqH0bkwME1yyqeVWsN1cM5LFgu3Ehy/q+M4R04Yr+dYucjbPyL7kKafEOle04b3Dln20ZcplUgHcxu8LkkD6BLYAZyQ3VtW3PYjj8k21dwcA416rMIEN/ilwwoQlkYRnHdwrW5o7VeX8dwGbIotKAZhL2Iq6MJdVNZSjCtKhGN2D7rFUaSIEyIb76mmTdTkRFgzwxHDANFEWaCT/R66jcnhUUz+4nMeLXlWcKmxdXUteWaALNZLtkxKXYRRE/UEYhI3BaTDuwAB1Non6X3CiaJExaakgxszGYW7nJdGWU8GqHi4Mywk9JPts5qAkGTPzstmBCm45jxuc0u5I147ae1xRksyYRRY7Zj1Psxyrnatis8Kmj+ZuTnlhmaRtorQQ4DajXihIuGbUioUDhGruagV6QDSh1q1dzzVhvPzl02B6P3gchK8fDDafd91YQzfQTTREY/QQbaKXaIKmiHqfvG/eD++n/9n/7v/yf7dU3+s019EJ8//8BUIOB0A=</latexit><latexit sha1_base64="m9PQWmRwqT5p5Kb9x2EeaIVayvI=">AAADQnicjVJLb9QwEHbCqyyPbuHIZcQKtAsoyiIkXkKq4MKhQgti26L1NnIcp7Xq2MF2Cqso/40Lf4Abf4ALB0BcOeA8JNrtHhjJ8qeZ7/OMZybOBTc2DL96/pmz585fWLvYu3T5ytX1/sa1baMKTdmUKqH0bkwME1yyqeVWsN1cM5LFgu3Ehy/q+M4R04Yr+dYucjbPyL7kKafEOle04b3Dln20ZcplUgHcxu8LkkD6BLYAZyQ3VtW3PYjj8k21dwcA416rMIEN/ilwwoQlkYRnHdwrW5o7VeX8dwGbIotKAZhL2Iq6MJdVNZSjCtKhGN2D7rFUaSIEyIb76mmTdTkRFgzwxHDANFEWaCT/R66jcnhUUz+4nMeLXlWcKmxdXUteWaALNZLtkxKXYRRE/UEYhI3BaTDuwAB1Non6X3CiaJExaakgxszGYW7nJdGWU8GqHi4Mywk9JPts5qAkGTPzstmBCm45jxuc0u5I147ae1xRksyYRRY7Zj1Psxyrnatis8Kmj+ZuTnlhmaRtorQQ4DajXihIuGbUioUDhGruagV6QDSh1q1dzzVhvPzl02B6P3gchK8fDDafd91YQzfQTTREY/QQbaKXaIKmiHqfvG/eD++n/9n/7v/yf7dU3+s019EJ8//8BUIOB0A=</latexit><latexit sha1_base64="m9PQWmRwqT5p5Kb9x2EeaIVayvI=">AAADQnicjVJLb9QwEHbCqyyPbuHIZcQKtAsoyiIkXkKq4MKhQgti26L1NnIcp7Xq2MF2Cqso/40Lf4Abf4ALB0BcOeA8JNrtHhjJ8qeZ7/OMZybOBTc2DL96/pmz585fWLvYu3T5ytX1/sa1baMKTdmUKqH0bkwME1yyqeVWsN1cM5LFgu3Ehy/q+M4R04Yr+dYucjbPyL7kKafEOle04b3Dln20ZcplUgHcxu8LkkD6BLYAZyQ3VtW3PYjj8k21dwcA416rMIEN/ilwwoQlkYRnHdwrW5o7VeX8dwGbIotKAZhL2Iq6MJdVNZSjCtKhGN2D7rFUaSIEyIb76mmTdTkRFgzwxHDANFEWaCT/R66jcnhUUz+4nMeLXlWcKmxdXUteWaALNZLtkxKXYRRE/UEYhI3BaTDuwAB1Non6X3CiaJExaakgxszGYW7nJdGWU8GqHi4Mywk9JPts5qAkGTPzstmBCm45jxuc0u5I147ae1xRksyYRRY7Zj1Psxyrnatis8Kmj+ZuTnlhmaRtorQQ4DajXihIuGbUioUDhGruagV6QDSh1q1dzzVhvPzl02B6P3gchK8fDDafd91YQzfQTTREY/QQbaKXaIKmiHqfvG/eD++n/9n/7v/yf7dU3+s019EJ8//8BUIOB0A=</latexit>
Capacity (load balancing) per node.Flow conservation: sum flow towards all
instances of a downstream microservice w= input data * data distribution ratio of w.
BLB: Basic (QoS-agnostic) Load Balancing
q Without delay constraint, the problem can be formulated as LP.
19
Variables: Per-link demand allocation function.
Demand per node = external + flow-in.
find f : L 7! R⇤
s.t. �n = �extn +X
l2Lin(n)
f(l), 8n 2 N ;
�n · cn, 8n 2 N ;
r(vn,w)�n =X
l2Lout(n,w)
f(l), 8n,w 2 Vout(vn).
<latexit sha1_base64="m9PQWmRwqT5p5Kb9x2EeaIVayvI=">AAADQnicjVJLb9QwEHbCqyyPbuHIZcQKtAsoyiIkXkKq4MKhQgti26L1NnIcp7Xq2MF2Cqso/40Lf4Abf4ALB0BcOeA8JNrtHhjJ8qeZ7/OMZybOBTc2DL96/pmz585fWLvYu3T5ytX1/sa1baMKTdmUKqH0bkwME1yyqeVWsN1cM5LFgu3Ehy/q+M4R04Yr+dYucjbPyL7kKafEOle04b3Dln20ZcplUgHcxu8LkkD6BLYAZyQ3VtW3PYjj8k21dwcA416rMIEN/ilwwoQlkYRnHdwrW5o7VeX8dwGbIotKAZhL2Iq6MJdVNZSjCtKhGN2D7rFUaSIEyIb76mmTdTkRFgzwxHDANFEWaCT/R66jcnhUUz+4nMeLXlWcKmxdXUteWaALNZLtkxKXYRRE/UEYhI3BaTDuwAB1Non6X3CiaJExaakgxszGYW7nJdGWU8GqHi4Mywk9JPts5qAkGTPzstmBCm45jxuc0u5I147ae1xRksyYRRY7Zj1Psxyrnatis8Kmj+ZuTnlhmaRtorQQ4DajXihIuGbUioUDhGruagV6QDSh1q1dzzVhvPzl02B6P3gchK8fDDafd91YQzfQTTREY/QQbaKXaIKmiHqfvG/eD++n/9n/7v/yf7dU3+s019EJ8//8BUIOB0A=</latexit><latexit sha1_base64="m9PQWmRwqT5p5Kb9x2EeaIVayvI=">AAADQnicjVJLb9QwEHbCqyyPbuHIZcQKtAsoyiIkXkKq4MKhQgti26L1NnIcp7Xq2MF2Cqso/40Lf4Abf4ALB0BcOeA8JNrtHhjJ8qeZ7/OMZybOBTc2DL96/pmz585fWLvYu3T5ytX1/sa1baMKTdmUKqH0bkwME1yyqeVWsN1cM5LFgu3Ehy/q+M4R04Yr+dYucjbPyL7kKafEOle04b3Dln20ZcplUgHcxu8LkkD6BLYAZyQ3VtW3PYjj8k21dwcA416rMIEN/ilwwoQlkYRnHdwrW5o7VeX8dwGbIotKAZhL2Iq6MJdVNZSjCtKhGN2D7rFUaSIEyIb76mmTdTkRFgzwxHDANFEWaCT/R66jcnhUUz+4nMeLXlWcKmxdXUteWaALNZLtkxKXYRRE/UEYhI3BaTDuwAB1Non6X3CiaJExaakgxszGYW7nJdGWU8GqHi4Mywk9JPts5qAkGTPzstmBCm45jxuc0u5I147ae1xRksyYRRY7Zj1Psxyrnatis8Kmj+ZuTnlhmaRtorQQ4DajXihIuGbUioUDhGruagV6QDSh1q1dzzVhvPzl02B6P3gchK8fDDafd91YQzfQTTREY/QQbaKXaIKmiHqfvG/eD++n/9n/7v/yf7dU3+s019EJ8//8BUIOB0A=</latexit><latexit sha1_base64="m9PQWmRwqT5p5Kb9x2EeaIVayvI=">AAADQnicjVJLb9QwEHbCqyyPbuHIZcQKtAsoyiIkXkKq4MKhQgti26L1NnIcp7Xq2MF2Cqso/40Lf4Abf4ALB0BcOeA8JNrtHhjJ8qeZ7/OMZybOBTc2DL96/pmz585fWLvYu3T5ytX1/sa1baMKTdmUKqH0bkwME1yyqeVWsN1cM5LFgu3Ehy/q+M4R04Yr+dYucjbPyL7kKafEOle04b3Dln20ZcplUgHcxu8LkkD6BLYAZyQ3VtW3PYjj8k21dwcA416rMIEN/ilwwoQlkYRnHdwrW5o7VeX8dwGbIotKAZhL2Iq6MJdVNZSjCtKhGN2D7rFUaSIEyIb76mmTdTkRFgzwxHDANFEWaCT/R66jcnhUUz+4nMeLXlWcKmxdXUteWaALNZLtkxKXYRRE/UEYhI3BaTDuwAB1Non6X3CiaJExaakgxszGYW7nJdGWU8GqHi4Mywk9JPts5qAkGTPzstmBCm45jxuc0u5I147ae1xRksyYRRY7Zj1Psxyrnatis8Kmj+ZuTnlhmaRtorQQ4DajXihIuGbUioUDhGruagV6QDSh1q1dzzVhvPzl02B6P3gchK8fDDafd91YQzfQTTREY/QQbaKXaIKmiHqfvG/eD++n/9n/7v/yf7dU3+s019EJ8//8BUIOB0A=</latexit>
Capacity (load balancing) per node.Flow conservation: sum flow towards all
instances of a downstream microservice w= input data * data distribution ratio of w.
Theorem 1:BLB is optimally solvable in polynomial time.
Real-Graph DecompositionTheorem
q Why do we need such a theorem?
1. Transform any solution of BLB into a set of implementable real-graphs.
2. Define QoS of a load balancing plan (max delay of all real-graphs).
20
Theorem 2:Every demand allocation function f so defined can be decomposed intoat most |N| + |L| real-graphs with positive source demands.
QLB: QoS-aware Load Balancing
q Fully Polynomial-Time Approximation Scheme (FPTAS)can achieve the best trade-off between time and accuracyv Approximation ratio: (1+𝜖) – For maximization problemv Time complexity: O(poly(1/𝜖) × poly(input))v In practice, one can arbitrarily tune 𝜖 to get best accuracy within time limit.
21
Theorem 3:QLB (optimization version) is NP-hard.
Theorems 4&5:QLB (optimization version) admits an FPTAS.
A Brief Overview of Our FPTAS
q Idea: v Pseudo-polynomial time algorithm:
Ø Expand Inf-Graph into a delay-layered graph.Ø Run BLB LP on the expanded graph.
v Discretization via approximate testing:Ø Find delay lower & upper bounds (UB, LB) s.t. UB <= poly(input) * LB.Ø Discretize delay values based on (UB, LB).Ø Run pseudo-polynomial time algorithm.Ø Refine (UB, LB) based on output.
v Efficiency enhancement:Ø Approximate testing to shrink initial bound s.t. UB <= constant * LB.
22
Outlines
23
Background and Motivation
System Modeling
Algorithm Design and Analysis
Performance Evaluation
Discussions, Future Work and Conclusions
Simulation Settings
q Simulated network scenarios:v App-Graph:
Ø 5-layered applications, layer-1 being the source layerØ 10-70 microservices: 10% in layer-1, uniformly distributed in other layersØ 4 in-going edges per microservice in layers 2-5Ø Data distribution ratio: uniformly generated
v Inf-Graph:Ø 1 instance per microservice in source layer, 5-15 in othersØ Linking probability (between interdependent instances): 0.3Ø Source demands: 100-900 unitsØ Node capacities: 10-90 unitsØ Node/Link delays: 0-500/1000 ms
v Load balancing goal: optimal load under BLB, or 2 x optimal load under BLBv Approximation parameter: 𝜖=0.5
q Comparisons:v QLBv BLBv QHU: QoS-aware heuristic, solving BLB minimizing demand-weighted delay
24
Comparison Results
25
Running timePolynomially increased
Max delay (QoS)QLB has improved delayover the other solutions.
Outlines
26
Background and Motivation
System Modeling
Algorithm Design and Analysis
Performance Evaluation
Discussions, Future Work and Conclusions
Other Perspectives and Beyond
q So far, we’ve talked aboutv Basic model: DAG-based apps, Real-Graphsv Processing capacities and delaysv Network delays
q What we didn’t consider in this workv Network topologyv Network capacities & congestionv Routingv Reliability: microservice instance failuresv Incentives, pricingv Payment methods
q A unified approach is still in need for high-performance IoT.
27
Computing Perspective
Networking Perspective
Security Perspective
Economics Perspective
Our Conclusions
q Load Balancing for Interdependent IoT Microservicesv DAG model for applications:App-Graph and Inf-Graphv Application realization with Real-Graph abstractionv System-wide load balancing with QoS (delay) constraints
q Algorithmic solutionsv Optimal solution for QoS-agnostic problemv FPTAS for (NP-hard) QoS-aware problem
q Future directionsv Unified framework for IoT performance optimization
28
Thank you very much!Q&A?
29
NP-Hardness Proof
v0v0 u1u1 v1v11.0 1.0 u2u2 v2v2 uκuκ vκvκ
1.0 1.0 1.0vκ-1vκ-11.0v0 u1 v1
1.0 1.0 u2 v2 uκ vκ1.0 1.0 1.0vκ-1
1.0
n0n0
m1m1
m1m1
n1n1
m2m2
m2m2
n2n2
mκmκ
mκmκ
nκnκ
0
1
0 0
1 1
δ=2.0
nκ-1nκ-1n0
m1
m1
n1
m2
m2
n2
mκ
mκ
nκ
0
1
0 0
1 1
δ=2.0
nκ-1
d=axκd=axκd=ax2d=ax2d=ax1d=ax1
App-graph:
Inf-graph:
30