Advanced Technology LaboratoriesAdvanced Technology Laboratories
Traffic Matrix Estimation in Non-Stationary Environments
Presented by R. L. Cruz
Department of Electrical & Computer EngineeringUniversity of California, San Diego
Joint work with
Antonio NucciNina Taft
Christophe Diot
NISS Affiliates Technology Day on Internet TomographyMarch 28, 2003
page 2Advanced Technology LaboratoriesAdvanced Technology Laboratories
The Traffic Matrix Estimation Problem
• Formulated in
Y. Vardi, “Network Tomography: Estimating Source-Destination Traffic From Link Data,” JASA, March 1995, Vol. 91, No. 433, Theory & Methods
page 3Advanced Technology LaboratoriesAdvanced Technology Laboratories
The Traffic Matrix Estimation Problem
ingress
egress
Xj
Xj
Yi
PoP (Point of Presence)
Y = A X
Link Measurement Vector
Routing Matrix
“Traffic Matrix”
page 4Advanced Technology LaboratoriesAdvanced Technology Laboratories
The Traffic Matrix Estimation Problem
• Importance of Problem: capacity planning, routing protocol configuration, load balancing policies, failover strategies, etc.
• Difficulties in Practice– missing data – synchronization of measurements (SNMP)– Non-Stationarity (our focus here)
• long convergence time needed to obtain estimates
page 5Advanced Technology LaboratoriesAdvanced Technology Laboratories
What is Non-Stationary?
•Traffic Itself is Non-Stationary
page 6Advanced Technology LaboratoriesAdvanced Technology Laboratories
What is Non-Stationary?
• Also, Routing is Non-Stationary– e.g. Due to Link Failures– Essence of Our Approach
• Purposely reconfigure routing in order to help estimate traffic matrix
– More information leads to more accurate estimates
• Effectively increases rank of A• We have developed algorithms to
reconfigure the routing for this purpose (beyond the scope of this talk)
page 7Advanced Technology LaboratoriesAdvanced Technology Laboratories
Outline of Remainder of Talk
• Describe the “Stationary” Method– Stationary traffic, non-stationary routing– Stationary traffic assumption is reasonable if we
always measure traffic at the same time of day (e.g. “peak period” of a work day)
• Briefly Describe the “Non-Stationary” Method– Both non-stationary traffic and non-stationary
routing– More complex but allows estimates to be obtained
much faster
page 8Advanced Technology LaboratoriesAdvanced Technology Laboratories
Network and Measurement Model
• Network with L links, N nodes, P=N(N-1) OD pair flows
– K measurement intervals, 1 ≤ k ≤ K – Y(k) is the link count vector at time k: (L x 1)– A(k) is the routing matrix at time k: (L x P)– X(k) is the O-D pair traffic vector at time k: (P
x 1)• X(k) = (x1(k) , x2(k) , … xP(k))T
Y(k) = A(k) X(k)
k [1,K]
Y(k) and A(k) can be truncated to reflect missing and redundant data
page 9Advanced Technology LaboratoriesAdvanced Technology Laboratories
Traffic Model: Stationary Case
x i(k) x i wi(k) k [1,K]
k
)(kxi
ix
)(kwi
• X(k) is the O-D pair traffic vector at time k: (P x 1)
X(k) = (x1(k) , x2(k) , … xP(k))T
X(k) = X + W(k)
W(k) : “Traffic Fluctuation Vector• Zero mean, covariance matrix B• B = diag(X)
page 10Advanced Technology LaboratoriesAdvanced Technology Laboratories
Matrix Notation
CWAXY
KkkWkAXkAkY
],1[)()()()(
)(
...
)1(
...
)(000
0...00
00)2(0
000)1(
)(
...
)1(
)(
...
)1( 1
KW
W
W
x
x
X
kA
A
A
C
kA
A
A
kY
Y
Y
P
where:
Linear system of equations:
[LK] [LK][P] [LK][KP] [KP][P]
Choose Routing Configurations such that
Rank(A) = P
page 11Advanced Technology LaboratoriesAdvanced Technology Laboratories
Traffic matrix Estimation-Stationary Case
• Initial Estimate: Use Psuedo-Inverse of A:
- does not require statistics of W (covariance B)• Gauss-Markov Theorem: Assume B is known
- Unbiased, minimum variance estimate- Coincides with Maximum Likelihood Estimate if W is Gaussian
Y = AX + CW
ˆ X (0) (AT A) 1 ATY
ˆ X (AT (CBCT ) 1 A) 1 AT (CBCT ) 1Y
page 12Advanced Technology LaboratoriesAdvanced Technology Laboratories
Traffic matrix Estimation-Stationary Case
Y = AX + CW
• Minimum Estimation Error:
(assumes B is known)
11 ))((])ˆ)(ˆ[( ACBCAXXXXE TTT
ˆ X (AT (CBCT ) 1 A) 1 AT (CBCT ) 1Y
page 13Advanced Technology LaboratoriesAdvanced Technology Laboratories
Traffic matrix Estimation-Stationary Case
• Recall we assume B = cov(W) satisfies B = diag(X)• Set
ˆ B (k ) diag( ˆ X (k ))
ˆ X (k1) (AT (C ˆ B (k )CT ) 1 A) 1 AT (C ˆ B (k )CT ) 1Y
• Recursion for Estimates:
page 14Advanced Technology LaboratoriesAdvanced Technology Laboratories
Traffic matrix Estimation-Stationary Case
• Our estimate is a solution to the equation:
ˆ X (AT (C diag( ˆ X ) CT ) 1 A) 1 AT (C diag( ˆ X ) CT ) 1Y
• Open questions for fixed point equation:- Existence of Solution?- Uniqueness?- Is solution an un-biased estimate?
page 15Advanced Technology LaboratoriesAdvanced Technology Laboratories
Numerical Example-Stationary case
N=10 nodes, L=24 links and P=90 connections.
Three set of OD pairs with mean x equal to:
– 500 Mbps, 2 Gbps and 4 Gbps.
Gaussian Traffic Fluctuations:
bx 20
page 16Advanced Technology LaboratoriesAdvanced Technology Laboratories
Stationary case: b=1 Samples/Snapshot=1
page 17Advanced Technology LaboratoriesAdvanced Technology Laboratories
Stationary case: b=1 Samples/Snapshot=1
page 18Advanced Technology LaboratoriesAdvanced Technology Laboratories
Stationary case: b=1 Samples/Snapshot=15
page 19Advanced Technology LaboratoriesAdvanced Technology Laboratories
Stationary case: b=1 Samples/Snapshot=15
page 20Advanced Technology LaboratoriesAdvanced Technology Laboratories
Stationary case: b=1.4 Samples/Snapshot=1
page 21Advanced Technology LaboratoriesAdvanced Technology Laboratories
Stationary case: b=1.4 Samples/Snapshot=1
page 22Advanced Technology LaboratoriesAdvanced Technology Laboratories
Stationary case: b=1.4 Samples/Snapshot=15
page 23Advanced Technology LaboratoriesAdvanced Technology Laboratories
Stationary case: b=1.4 Samples/Snapshot=15
page 24Advanced Technology LaboratoriesAdvanced Technology Laboratories
Stationary and Non-Stationary traffic
20 snapshots / 4 samples per snapshot / 5 min per sample
• Stationary Approach: 20 min per day (same time) / 20 days• Non-Stationary Approach: aggregate all the samples in one window time large 400 min (7 hours)
page 25Advanced Technology LaboratoriesAdvanced Technology Laboratories
Traffic Model: Non-Stationary Case
)()( kTxkx ii
• Each OD pair is cyclo-stationary:
• Each OD pair is modeled as:
• Fourier series expansion:
],1[)()()( Kkkwkmkx iii
)/2sin(
)/2cos()(
)(ˆ)(ˆ2
0
Nkn
Nknkb
kbmkm
n
N
nnini
b
]2,1[],0[
],1[],0[
bb
b
NNnNk
NnNk
)(kxi )(kxi
kk
)(kwi )(kwi
)(kmi)(kmi
page 26Advanced Technology LaboratoriesAdvanced Technology Laboratories
Mean estimation Results-Non Stationary case
Three set of OD pairs
where are linear independent Gaussian variables with:
)()2sin()( twfaxtx iiiiii
bix 20
3/2
3/
0
4
2
500
ii
Gbps
Gbps
Mbps
x
)(twi
page 27Advanced Technology LaboratoriesAdvanced Technology Laboratories
Non Stationary case: b=1 Link Count
page 28Advanced Technology LaboratoriesAdvanced Technology Laboratories
Non Stationary case: b=1 Mean estimation