Date post: | 22-Dec-2015 |
Category: |
Documents |
View: | 216 times |
Download: | 0 times |
October, 1998 DARPA / B. Melamed 1
High-Fidelity Real-Time Modeling and Simulation
of Network Traffic Processes
Khosrow SohrabyComputer Science TelecommunicationsUniversity of Missouri-Kansas City5100 Rockhill Rd.Kansas City, MO 64110
Benjamin MelamedRutgers UniversityFaculty of ManagementDept. of MSIS94 Rockafeller Rd.Piscataway, NJ 08854
DARPA/ITO BAA 97-04 AON F316
October, 1998 DARPA / B. Melamed 2
• Emerging high-speed telecommunications networks increasingly carry highly bursty traffic
• compressed video• file transfer
• Network modeling and analysis technologies are urgently needed (witness Internet congestion)
• network control (admission and congestion)• network provisioning and planning
• Problem: traditional analytical/simulation models are unsuitable for emerging networks
• traffic models• queueing models
MOTIVATION
October, 1998 DARPA / B. Melamed 3
• Traffic is modeled as a time series (stochastic process)
• interarrival intervals time series (between jobs)• variable bit rate (VBR) time series (e.g., compressed
VBR video)
• Traditional analysis assumes traffic time series is iid (independent identically distributed)
• assumptions ignore dependencies to simplify analysis
• But real-life traffic processes are not independent• traffic time series tend to be heavily autocorrelated• traditional analysis produces wrong predictions• autocorrelations must be incorporated into modeling!
ENTER AUTOCORRELATED TRAFFIC...
October, 1998 DARPA / B. Melamed 4
• Correlation is a measure of linear dependence between random variables
• the correlation coefficient of random variables X and Y is Corr(X,Y) = ( E[XY] - E[X]E[Y] ) / sqrt(V[X]V[Y])
• Autocorrelation function of a stationary random process {Xk} maps time lags between its random variables to their correlation coefficients
• acf(n) = Corr(Xk,Xk+n), n = 0,1,2• n is the lag
• The autocorrelation function, acf(n), captures
temporal (time) dependence• correlation / autocorrelation is one aspect of dependence• used routinely as a good proxy for temporal dependence
WHAT ARE AUTOCORRELATIONS?
October, 1998 DARPA / B. Melamed 5
IMPACT OF AUTOCORRELATIONS!!!
6000%
4000%
2000%
Acf(1)
Source : M. Livny, B. Melamed and A.K. Tsiolis,“The Impact of Autocorrelation on Queueing Systems”, Management Science 21(3), 322--339, 1993
25000%
20000%
15000%
10000%
5000%
0%
-.55 -.4 -.25 0 .25 .5 .75 .85
% error of mean waiting timeof TES/M/1 relative to M/M/1
Utilization = 80%
Acf(1)
10000%
8000%
0%
-.55 -.4 -.25 0 .25 .5 .75 .85
% error of mean waiting timeof TES/M/1 relative to M/M/1
Utilization = 25%
October, 1998 DARPA / B. Melamed 6
• The candidate model should be selected from a versatile class of stationary stochastic processes
• general marginal distributions• wide variety of autocorrelation functions (e.g., monotone, oscillatory, alternating, etc.)• broad qualitative range of sample path behavior (e.g., cyclical, non-directional, etc.)
• The candidate model should satisfy:• the marginal distribution of the model should match the empirical distribution (histogram)• the autocorrelation function of the model should approximate the empirical autocorrelation function• Monte Carlo simulated model paths (histories) should
“resemble” the empirical data
MODEL GOODNESS-OF-FIT CRITERIA
October, 1998 DARPA / B. Melamed 7
• TES is a new modeling methodology• designed to satisfy the 3 goodness-of-fit criteria• fast generation of TES sample paths• fast computation of TES autocorrelation functions• negligible memory for these computations• however, model search is not yet real-time
• QTES (Quantized TES) modeling methodology is a new discrete version of TES modeling methodology
• reduces the continuous TES state space to a finite space• integration operators reduce to finite matrices• can be used to solve queueing models with accurate traffic (arrival) processes, directly from empirical data
records of measurements
TES / QTES MODELING METHODOLOGIES
October, 1998 DARPA / B. Melamed 8
• Inversion Method• let X be an arbitrary random variable with cumulative
distribution function (cdf) F (and inverse F -1) • Let U be a Uniform random variable (available on most computers)• then Y = F -1(U) is a random variable with distribution F
• Iterated Uniformity• let <x> be the fractional part of x (modulo-1 operator)• let U be a random variable, uniform on [0,1)• let V be any random variable, independent of U• then, <U + V > is a random variable, uniform on [0,1), regardless of the distribution of V !!! • Therefore, choosing V selects a dependence structure without changing the (uniform) distribution!!!
TES MODELING ELEMENTS
October, 1998 DARPA / B. Melamed 9
• TES terminology • let H be the empirical histogram cdf and H -1 its inverse
• let Sxi be a stitching transformation, with xi in [0,1],
where Sxi(y) = y / xi, for y in [0,xi)
Sxi(y) = (1 - y) / (1 - xi), for y in [xi,1)
• let {Vn} be an innovation sequence (iid random variables,
independent of a uniform [0,1) random variable U0 )
• let D(x) = H -1(Sxi (x)) be the corresponding distortion
• Define two TES background (auxiliary) sequences• TES+: U0
+ = U0,; Un+ = < Un-1
+ + Vn >
• TES-: Un- = Un
+ for n even; Un- = 1 - Un
+ for n odd
• Define two TES foreground (target) sequences• TES+: Xn
+ = D(Un+ ) = H -1(Sxi (Un
+ ))
• TES-: Xn- = D(Un
- ) = H -1(Sxi (Un- ))
TES PROCESSES
October, 1998 DARPA / B. Melamed 10
• Geometric interpretation
TES+ BACKGROUND PROCESSES
Step-function Innovation density
Un+< Un + Ln>+ < Un + Rn>+
Unit circle
October, 1998 DARPA / B. Melamed 11
THE TES MODELING PARADIGM
stitching parameter
0
1
1xi
stitching transformation
y
Sxi(y)
+Sxi(Un )
+Un
unit circle
previousbackground
variatenext
background variate
+Un-1
Un = < Un-1 + Vn >+ +
Inversehistogram
cdf
nextforeground
variate
0 1+Sxi(Un )
Xn = H-1(Sxi(Un ))+
H -1(x)
x
October, 1998 DARPA / B. Melamed 12
• Basic results • every background TES process is a Markov sequence, uniformly distributed on [0,1)• using the inversion method, a TES foreground sequence can be endowed with any prescribed distribution, regardless of its autocorrelation structure !!!• the TES modeling methodology searches for pairs (xi,fV) (stitching parameter and innovation density) that approximate the empirical autocorrelation function
• Conclusion• TES modeling effectively decomposes the fitting of the empirical autocorrelation function and the fitting of the empirical distribution• experience shows that it often produces high-fidelity models, both quantitatively and qualitatively
TES FACTS
October, 1998 DARPA / B. Melamed 13
• QTES terminology • let M >1 be a positive integer, representing a partition of the unit circle into M equal slices of length h = 1 / M • identify each slice with a state in the set S = {0, 1 ,…, M -1}• let <n>M = n (mod M ) (smallest residual of n modulo M )
• let {Jn} be an innovation sequence (iid random variables
over S, independent of a uniform {0, 1,…, M -1} variate K0 )
• let {Wn(j)
} be an iid sequence uniform on slice [hj, h(j+1) )
• Interpretation• each slice is “collapsed” into a single state, resulting in a finite state space• values within a slice are “indistinguishable”, since as slices get small, these values lie “near” each other• the underlying transition structure (among slices) is finite (in fact, a finite-state Markov process)
QTES PROCESSES
October, 1998 DARPA / B. Melamed 14
• Define two QTES background (auxiliary) sequences• QTES+: K0
+ = K0; Kn+ = < Kn-1
+ + Jn >M
• QTES-: Kn- = Kn
+ for n even; Kn- = M - 1 - Kn
+ for n odd
• Define two QTES foreground (target) sequences• QTES+: Xn
+ = H -1(Sxi (Wn(Kn+ )))
• QTES-: Xn- = H -1(Sxi (Wn(Kn
- )))
• Interpretation• QTES background processes are random walks on a “circular lattice”, S, of integers (residuals)• QTES foreground sequences “randomize” the discrete state (slice index) to obtain a continuous state space• however, the underlying transition structure is finite! • nevertheless, QTES satisfies the 3 goodness-of-fit criteria
QTES PROCESSES (Cont.)
October, 1998 DARPA / B. Melamed 15
• Geometric interpretation
QTES+ BACKGROUND PROCESSES
Sliced unit circle
previousbackground
variate
nextbackground
variate
+Kn-1
Kn = < Kn-1 + Jn >M+ +
slice/state0
slice/state1
slice/stateM-1
slice/statek
October, 1998 DARPA / B. Melamed 16
• Basic results • every background QTES process is a Markov sequence, uniformly distributed on the integers {0, 1, … , M -1}• the randomization step results in a process which is distributed uniformly on [0,1) • thus, a QTES process can match to any prescribed distribution, and simultaneously approximate a large variety of autocorrelation functions !!!• the TES modeling methodology searches for pairs (xi,fJ)
(stitching parameter and innovation density) that approximate the empirical autocorrelation function
• Conclusion• QTES modeling enjoys all the benefits of TES modeling• however, it has a discrete transition structure which make QTES traffic models it amenable to fast queueing analysis
QTES FACTS
October, 1998 DARPA / B. Melamed 17
• TES • B. Melamed, "An Overview of TES Processes and Modeling Methodology", in Performance Evaluation of Computer and Communications Systems, (L. Donatiello and R. Nelson, Eds.), 359--393, Lecture Notes in Computer Science, Springer-Verlag, 1993• D.L. Jagerman and B. Melamed, "The Transition and Autocorrelation Structure of TES Processes Part I: General Theory", Stochastic Models 8(2), 193--219, 1992• D.L. Jagerman and B. Melamed, "The Transition and Autocorrelation Structure of TES Processes Part II: Special Cases", Stochastic Models 8(3), 499--527, 1992
• QTES• P. Jelenkovic and B. Melamed, "Algorithmic Modeling of TES Processes", IEEE Trans. on Automatic Control 40(7), 1305--1312, 1995
REFERENCES
October, 1998 DARPA / B. Melamed 18
EXAMPLE: H.261 COMPRESSED VIDEO
October, 1998 DARPA / B. Melamed 19
EXAMPLE: MPEG COMPRESSED VIDEO
October, 1998 DARPA / B. Melamed 20
EXAMPLE: JPEG “STAR WARS” VIDEO
October, 1998 DARPA / B. Melamed 21
• Numbers • empirical data set size: 500-1000 observations and up• modeling time: 5-10 minutes• analysis time: seconds• Monte Carlo traffic generation: can support 1000-10,000 traffic streams per second of CPU
• Goals• speed up modeling search to seconds (new algorithms and representations, parallelize algorithms)• real time / near real time procedure from traffic measurements to performance predictions
• Status (as of 10/97)• design and implementation of serial version for modeling testbed has begun• serial version of analysis engine is complete
• available in public domain as TELPACK (TELetraffic PACKage) at http://www.cstp.umkc.edu/org/tn/telpack/home.html (information) ftp://ftp.cstp.umkc.edu/telpack/software/ (anonymous FTP)
PROJECT INFORMATION
October, 1998 DARPA / B. Melamed 22
PROJECT SUMMARY