IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. IT-18, NO. 1, JANUARY 1972 133

[6] R. S. Bennett, “The intrinsic dimensionality of signal collections,” cardiograms by orthogonalized exponentials,” IRE Znt. Corn.. IEEE Trans. Inform. Theor?, vol. IT-15, Sept. 1969, pp. 517-525. Rec., pt. 9, 1961, pp. 145-153.

(71 J. W . Sammon, Jr., “A nonlinear mapping for data structure anal- [I l] -, “On the representation of electrocardiograms,” IEEE Trans. ysis,” IEEE Trans. Cornput., vol. C-18, May 1969, pp. 401-409.

[8] G. V. Trunk, “Statistical estimation of the intrinsic dimensionality Bio-Med. Electron., vol. BME-10, July 1963, pp. 86-95.

of data collections,” [ 121 E. L. Lehmann, Testing Statistical Hypotheses. New York : Wiley,

Inform. Contr., vol. 12, May/June 1968, 1959, ch. 6. pp. 508-525.

[9] y, “Representation and analysis of signals, Part XXIV. [13] W . H. Kautz, “Transient synthesis in the time domain,” IRE

StatIstical estimation of intrinsic dimensionality and parameter Trans. Circuit Theory, vol. CT-l, Sept. 1954, pp. 29-39.

identification,” Gen. Syst., vol. 13? 1968, pp. 49-76. [14] R. N. Shephard, “Analysis of proximities as a technique for the

[IO] T. Y. Young and W . H. Huggms, study of information processing in man,” Human Factors, vol. 5,

“Representation of electro- 1963, pp. 33-48.

A Critical Statistic for Channels W ith Memory JEAN-PIERRE A. ADOUL, MEMBER, IEEE, BRUCE D. FRITCHMAN, MEMBER, IEEE,

AND LAVEEN N. KANAL, SENIOR MEMBER, IEEE

Abstract-We present a new descriptive statistic for channels with memory and show its utility a) in evaluating and comparing existing models for such channels and b) as a theoretical tool in defining the error-gap distribution characteristics of real channels. We demonstrate that certain kinds of real channel behavior cannot be adequately described by previously proposed models and offer an example of a better model that includes many of the earlier models as special cases.

I. INTRODUCTION

I N RECENT years many models have been proposed to characterize the error sequences encountered in real

digital communication links [I]-[lo]. For the most part these models have been developed to represent certain ex- perimentally measured statistics, though inferences are often made about their applicability to more general situations. Unfortunately, it is not always easy to ferret out all of the implicit assumptions about a model in order to determine its applicability to a specific channel. Furthermore, different models, which apparently represent the same channel, can lead to different conclusions about the behavior of com- munication processes. The reason is that the models do not in reality adequately represent some statistic that is critical in the analysis.

To circumvent such difficulties, in this paper we examine a number of basic properties that a model must satisfy if it is to represent adequately a real channel or class of real channels. A consequence of our analysis is the demonstra- tion that certain kinds of real channel behavior cannot be properly described by previously developed models. We

Manuscript received March 3, 1971 ; revised May 20, 1971. This research was supported by the Mathematical and Information Sciences Directorate, Air Force Office of Scientific Research, Air Force Systems Command, Arlington, Va., under Grants Af-AFOSR 68-1390B to Lehigh University and AFOSR 71-1982 to the University of Maryland.

J.-P. A. Adoul is with the Department of Electrical Engineering, University of Sherbrooke, Sherbrooke, Que., Canada.

B. D. Fritchman is with the Department of Electrical Engineering, Lehigh University, Bethlehem, Pa. 18015.

L. N. Kanal is with the Computer Science Center, University of Maryland, College Park, Md. 20742.

then offer an alternative model as an example. This model is a function of a slowly spreading Markov chain and co- incidently includes many of the previously proposed models as special cases.

Though a number of important statistical properties of real channels are uncovered in the following analysis, it is by no means exhaustive. However, it should go a long way toward the improvement of the process of selecting models to represent various classes of real communication channels. It should also lend insight into the varied behavior of real channels.

Consider the channel model shown in Fig. 1. It consists of an input and output alphabet whose symbols are the q elements of the Galois field GF(q). Between the input and output the channel introduces a random discrepancy, which is represented mathematically as the addition of a noise symbol, i.e.,

yi = Xi + ni, (1)

where yi and xi are, respectively, the output and input symbols, n, is the noise symbol, which is also an element of GF(q), and + is addition over the same field.

The noise sequence {n,} can be thought to originate from a hypothetical random generator called the noise source. Throughout it will be assumed that the input sequence {xi} and noise sequence {ni} are statistically independent of each other, implying that the statistical properties of the channel are exhibited in the statistical properties of the noise source.

Whenever /vi is different from zero, an error is said to occur. In this way the error source is distinguished from the noise source. The channel error source generates the error sequence {e,}, which is a mapping of the noise sequence {n,} onto {O,l}, i.e.,

ei = e(n,) = 0, if ni=O 1, otherwise. (2)

134 IEEE TRANSACTIONS ON INFORMATION THEORY, JANUARY 1912

TRANSMITTER RECEIVER

Alphahet of q symbols

Channel Error Source

Alphabet of q symbols

ei = 0 if xi = yi

1 otherwise

Fig. 1. Digital communication channel.

In the common case of binary transmission the noise and error sequences are equivalent. Both the error source and noise source are discrete-time stochastic processes.

II. THE ERROR-GAP PROCESS

The state space of the error process is composed of two states: error or 1 and error free or 0. A positive integer exponent is used to indicate the number of consecutive symbols of the same type. In this notation the sequence 10000000010001100 is written as 1081031202.

In probability expressions we use the notation,

Pr{e, = 0, e, = O;.*,e,-, = O,e, = l} = P(O”-‘1). (3)

A conditional probability is also written in terms of se- quences: Pr {sequence B 1 sequence A}. If no other indica- tion is given in the conditional part, it is understood that sequence B directly follows sequence A.

Sequences of zeros between two errors (1 states) are called error gaps (also error-free runs). The length of a gap is defined as one plus the total number of zeros in the sequence between two 1’s. By defining the gap length in this way the sum of all gap lengths equals the total length of the error sequence.

Corresponding to the error process, the gap process {G,} can be introduced by treating the binary error process as a succession of gaps of length G,. The state space of this new process is the denumerable set of positive integers.

The probability

Pr {G, = m} = P(X,+, = O;.*,

x,+,-1 = 0, Xn+m = 1 I X” = 1) (4)

for all positive integers m is called the error-gap probability mass function (egpmf).

Assuming stationarity, we have

Pr {G,, = m} = P(O"-'1 11). (5) Let

F(m + 1) 4 Pr {G, 2 m + I} = P(O"'I l), (6)

where

P(Orn 1 1) = z P(Okl 1 1). k=m

(7)

P(Om 1 1) is called the error-gap or error-free run distribu- tion (egd). From the definition it is observed that P(0”’ 1 1) is a monotonically decreasing function of m. Moreover, P(Oml 1 1) = P(0” 1 1) - P(Om+l 1 1) and the expectation

E(G,) = 2 (m + l)P(OY 1 1) m=O

= $, ( m + l)[P(Om I 1) - P(Om+ 1 ) l)]

= f, p&y 1 1) = f p<lo”>. m=O P(1) (8)

From the stationarity assumption it follows that P(10’) = P(O’1). As P(O’1) is the probability that beginning with any symbol in the error sequence the first error will not be en- countered for i symbols, the events i = 0,1,2, * * . are mutually exclusive and exhaustive. Thus if and only if P(Ornl) = 0,

Therefore

f, P(10”) = 1. (9)

@G-T) = c,“=o ‘(“*) ’

P(1) = p(1) ’ (10)

where P(1) is the probability of error and is equal to E(e,). For real channels the probability of an error is greater than zero, which implies that

E(G,) = f P(0” I 1) < co. m=O

(11)

E(G,) is the expected number of symbols between two errors, i.e., the average number of symbols that will be transmitted before the recurrence of an error. Consequently,

ADOUL et al. : STATISTICS FOR CHANNELS WITH MEMORY 135

when E(G,) exists and is independent of n, it will be called the recurrence time R,. On the average the number of non- errors associated with a single event G, is R,, which implies the number of events in the corresponding gap process is reduced by a factor R, over the number of events in the binary error process. For good communication channels the probability of error P(1) is smaller than 10m3, which means R, 2 103. For such channels the number of events dealt with in the corresponding gap process is reduced by a factor of at least lo3 over the events in the error process.

III. THE DESCRIPTIVE STATISTIC

We now introduce a new statistic, defined as the slope of the error-gap distribution plotted in logarithmic coordinates. We show how it can be used a) in evaluating and comparing existing models both of the generative and descriptive type and b) as a theoretical tool in defining the error-gap distribution characteristics of real channels.

The error-gap distribution is often determined during the measurement of channel statistics. Consider a mono- tonically decreasing continuous function P”(Om 1 l), which for all nonnegative integers m is equal to P(Om 1 1); a sketch of P”(0”’ I 1) as a function of m is shown in Fig. 2. Suppose this function is plotted in logarithmic coordinates as follows :

y = log, p”(Orn 1 l),

x = log, m.

Now define the function

(12)

(13)

cc(x) A - dye ax Note that in this x - y coordinate system, the terms of the harmonic series, i.e., P(Om 1 1) = l/m, fall on a straight line of slope equal to - 1.

Now it is an elementary fact that if lamI < c, form 2 m*, where m” is some fixed integer, and if C c, converges, then C a, converges. Also, for nonnegative a,, if a, 2 d,,, 2 0 for m 2 m”, and if C d, diverges, then C a, diverges. But, as just demonstrated in (1 l), for real channels E(G,) < co. Therefore any model having an egd P(Om I 1) < l/m” +&), with E > 0 a fixed constant, after some value of m = m*, will have an x - y characteristic slope that is asymptotically less than - 1, as shown in Fig. 3 and will satisfy (11). The condition that an error source model have an a-function that is asymptotically greater than 1 is sufficient to guarantee convergence of E(G,). A necessary condition for conver- gence is obtained in Section VII.

Since P(Om I 1) is always a monotonically decreasing function of m, the slope of P(Om I 1) will always be negative, implying U(X) will be a nonnegative function of x. The a-function for the harmonic series equals unity for all values of x, and so if P(Om I 1) < l/rn(‘+‘) for all m > m*, the cc-function will fall in the region X(X) > 1, for x > x* = log, m*.

Pd-71)

1 2 3

Fig. 2. Function equal to P(Om 1 1) at integer m.

0 x

“1 SLOPE -I

r ‘\ ‘\

Fig. 3. Real channel gap distribution.

IV. THE ~-FUNCTION OF SOME SPECIAL MODELS

The preceding considerations suggest that real channels might be expected to yield cc-functions that asymptotically take on values greater than 1. Examination of the cr-func- tion of some models proposed to represent real channels will bring to light some interesting behavior.

Binary Symmetric Channel

The simplest model, and the one most commonly used to represent error sequences, is the binary symmetric chan- nel (BSC). If p = 1 - q is the probability of an error of either type (1 + 0 or 0 -+ l), P(Om I 1) = qm and y = log, P(Om 1 1) = m log, q. Since x = log, m, m = eX, and y = er log, q, we have

dy 1 m(X) = - z = log, - eX. ( 1 9

The factor log, l/q is positive and constant, and so the BSC has an exponential cr-function.

Pareto Model

Berger and Mandelbrot [3] proposed a model with an egd given by a Pareto distribution, i.e., they let

P(Om I 1) = l/m’, (16)

where 0 is a positive constant. Computation of U(X) for this case yields the value 8.

The authors of [3] claim that for small m, the Pareto distribution is a good approximation to actual measured distributions. From experimental measurements they found values of 0 = 0.5 as typical. Sussman [l l] also found values of 8 = 0.11 and 0.3 to represent some channel measurements.

136 IEEE TRANSACTIONS ON INFORMATION THEORY, JANUARY 1972

The previous discussion showed channels with

lim E(X) < 1 x-+co

do not have finite recurrence times, since the series C,,, P(Om 1 1) diverges. Berger and Mandelbrot resolved this problem by letting 0 take on a new constant value greater than unity at some point m = m” ; the point m* becomes a parameter of the model. In our terminology they assume that for values of x greater than x” = log, m”, a takes on a constant value greater than 1. This is illustrated in Fig. 4 along with the a-function of the harmonic series and the BSC.

Finite-State Markov Models

A state model for the binary discrete-time error process is a set of states, together with a mapping 4 from these states onto the set {O,l}. The function 4-r brings about a partitioning of the state space: those states that are mapped into 0 and those that are mapped into 1.

Gilbert [l] initiated the application of finite-state Markov models to the representation of error sequences by proposing a model composed of a good state G, which is error free, and a bad state B, in which the channel has error probability /r. If the state sequence is represented by the discrete-time process {Z,}, then the transition probabilities between states are defined by

Pr{Z,EG\Z,-lEB} =p

Pr(Z,EB(Z,-rEG) = P. (17)

The model is shown in Fig. 5(a). Since even in the bad state there is a probability of having

no error, the mapping $ cannot be directly applied. How- ever, the Gilbert model can be transformed into a three- state Markov chain as shown in Fig. 5(b); the new states are called G, B,, and B,. The function 4 mapping the states onto the error sequence can be defined as

4(G) = 0, W,) = 0, WI) = I.

Thus only B, is an error state. Fritchman [S] extended Gilbert’s results by studying

the general case of finite-state models (Fig. 6) with k error- free states and N - k error states. For such models he showed that the egd can be written as

P(Orn 1 1) = i f(i)&“, (18) i=l

where li are the eigenvalues of the matrix of transition probabilities among the k error-free states and f(i) is a function of the transition probabilities among all states. Ordering the set Ai by decreasing magnitude, i.e., 11,l 2 l/1,1 2 I/z,\ 2 . . . 2 l&l, for large m and aperiodic chains we get asymptotically

P(0” 1 1) N f(l)&“. (19)

44

1 __-_

0 -- x' x

Fig. 4. Truncated Pareto distribution.

P

(4

(I-PM

(I-P)

(b)

Fig. 5. Gilbert model.

Q @ . . . . (g @ . . . . . . @ \ I I

ERROR-TREE ST1lES ERROR STATES

Fig. 6. Fritchman’s finite state model.

Consequently

y = log, P(Om 1 1) N log,f(l) + m log, I, (20)

since m = eX,

dy (y = --.-N dx

This demonstrates that, regardless of the number of states, an aperiodic finite-state model always yields an a-function which is asymptotically exponential as shown in Fig. 7.

Renewal Process

A renewal process can be viewed as a series of trials in which the probability of success, i.e., no-error, at a certain

ADOUL et al.: STATISTICS FOR CHANNELS WITH MEMORY

trial is solely a function of the number of successes since the last failure, i.e., error. When an error finally occurs, the process “starts” all over again, thus giving rise to the name renewal. A renewal process is uniquely defined by the egd.

As noted in the discussion of the Pareto model, Berger and Mandelbrot proposed that the egd be described by a Pareto distribution. In this model they further assumed statistical independence between successive error gaps, as did Sussman, thereby defining a renewal process. It is obvious that this renewal process cannot be modeled by a finite-state Markov chain for, as already demonstrated, that would lead to an cc-function that is asymptotically exponential. We next show that a denumerably infinite- state Markov model, termed a slowly spreading chain, does allow modeling of this renewal process as well as more general ones.

V. SLOWLY SPREADING MARKOV CHAINS

Renewal processes have received considerable attention in the context of point processes (see, e.g., Smith [12]). The development of discrete-time renewal processes is, however, meager, except in relation to a class of denumerably infinite-state Markov chains, which Kemeny [13] calls slowly spreading chains of the first kind. Representation of the general discrete renewal process via slowly spreading Markov chains is now considered.

The denumerable state space is labeled by a nonnegative integer and the following mapping relates the chain states to the states {O,l} of the renewal process:

If the discrete process {Z,,} represents the state sequence, the transition probabilities can be defined as

Pr {Z, = i 1 Z,-, = i - I} = ji,

Pr {Z, = 0 I Z,-, = i} = Qi. (24)

All other transitions have zero probability of occurrence. This slowly spreading chain is shown in Fig. 8.

Letting

(25)

it is assumed /Ii # 0, i.e., all states are communicating. If Q is defined as the probability of returning to state 0 beFore reaching state n, then

Qon = 41 + $142 + Ai%43

+ ***+ j1j32***Pn-llgn

= 41 + P142 + hi13 +. . * + fin-142

(26)

where PO a 1. The individual terms of the series can be

1

0

Fig. 7. Asymptotically exponential a for finite-state models.

Fig. 8. Slowly spreading chain.

interpreted as follows :

Bi$i+ 1

probability of reaching = state i from state 0 1 ( ’

probability of returning from state i to state 0 ’

(27)

Thus pi can be interpreted as the probability of reaching state i when the process is making a round trip to state 0. Also, since gk = 1 - fik and Pk = Bk- iflk, substitution into the preceding expression yields Qon = 1 - p,. Similarly the probability of returning to any state i before reaching a state y1 > i, pin, can be expressed as

Qin = 1 - PnlBi, i = 0,1,2;... (28)

Here and throughout the paper PO & 1. If hii is defined as the probability of eventually returning to state i, then for a slowly spreading chain

1 . hii = ,llf”, Qin = 1 - pi )J: Pn, i = 0,1,2;**. (29)

A denumerable chain is said to be recurrent if the probability of eventually returning is unity. For the slowly spreading chain this is equivalent to requiring lim /J’, = 0 as n + co.

Although certain, the return to state i can take a very large number of steps. To insure that on the average this number of steps is finite, it is necessary for the chain to meet the more restrictive ergodic condition that follows. Let R,, be the expected time for the first return to state 0.

138 IEEE TRANSACTIONS ON INFORMATION THEORY, JANUARY 1912

Then from Fig. 8 it is seen that

M 00 = 41 + 2fi1Q2 + 3618243 + * * *

The conditions imposed on LX(X) by ei-godicity of degree r are now considered. The rth moment of the O-state recur- rence time b,,(l) is given by

bo(‘) = E[T,‘] = E[(m + l>r]

= k$o Pk. $0 ( m + l)‘P(OY ( 1)

= g,( m + l>‘[P(Om 1 1) - P(Om+ 1 I l]

State 0 and hence all states’ will be ergodic if and only if

kgo pk < *. (31)

+ l)* - m’]P(O” 1 1)

Referring to the transformation of chain states into the states 0 and 1, given by (23), it is clear that

pi = P(0’ 1 l), (32) + r<r-- ‘)m*-: +. . .

2! 1 I - m* P(Om 1 l), (34)

and so that for large m P(O’1 1 1) = pi - pi+1. (33)

In other words, pi is the egd of the chain and this model E(To’) - 2 m’-lP(O” I 1). (35) m=l

allows the specification of a renewal process having any desired egd or equivalently any desired cc-function. The If the rth moment is required to be finite, then clearly only constraint on the specification of pi is that

goPi< O”. The slowly spreading model includes as special cases the renewal models previously discussed. For example, if we let pi = (1 - p)‘, the resulting model is just the BSC having error rate p. Alternatively, if we set fii = l/i’, the result is the Pareto model. Between these two extremes a renewal process with almost any desired a-function can be specified.

VI. CONDITIONS ON u FOR ERGODICITY OF DEGREE Y

The preceding discussion has centered around chains with finite-mean recurrence times or equivalently chains with ergodic state probabilities. This is actually the weakest ergodic condition that can be imposed. With To denoting the O-state recurrence time, Kemeny et al. [14] define a chain to be ergodic of degree r if bo@) = E(T,‘) < co but bg+ ‘) = co, where r is an integer. They prove the notable fact that a finite-state Markov chain is ergodic of degree infinity. Therefore more restrictive conditions can be im- posed on a chain by requiring the first r moments of the O-state recurrence time To to be finite, and E( T ‘0’ ‘) = co.

For a slowly spreading Markov chain to be ergodic of at least degree 1, it was shown in the preceding section that

2 rn*-‘P(0” 1 1) < co. m=l

There are thus two conditions for ergodicity of degree r, namely,

z rn*-‘P(0” I 1) < co, (37) m=l

2 mrP(O” I 1) = co. m=l

(38)

These conditions can be directly related to the asymp- totic behavior of the a-function. First observe that if rn*-‘P(0” I 1) < l/m for all values of m > m*, where m* is any positive integer, then condition (37) is met, i.e., the rth moment is finite if

P(Om 1 1) < l/mr. (39)

Taking the natural logarithm of both sides of this expression and using the logarithmic variables x and y, differentiating and changing sign, gives the result that if a(x) is greater than r for all values of x > x* = log, m*, then the rth moment exists. Alternately, if

P(Om ( 1) > l/m’+‘, (40)

then the (r + 1)th moment does not exist. Sufficient con- ditions to guarantee ergodicity of degree r are therefore

r I cc(x) < r + 1.

On the U(X) diagram the asymptotic behavior of U(X) can 1 The rth moment of the mean recurrence time, bk(‘) = E(T,‘), be categorized as shown in Fig. 9. An cc-function that for

where Tk is the random duration needed to go from state k back to state k. Kemeny et al. [14] have shown that if b,(‘) < co for some state

x greater than some X* always remains between lines r and S,,, then bj(‘) < co for all states j. Consequently the conditions that r + 1 corresponds to a process having ergodicity of degree guarantee that the rth moment of state 0 is finite are the same for all states. We make use of this reasoning throughout the remainder of the

r. Processes whose cc-functions do not asymptotically ap- paper. preach finite slopes will have infinite ergodic degree. For

ADOUL et al. : STATISTICS FOR CHANNELS WITH MEMORY

1

0 x

Fig. 9. a-function and ergodicity degree.

example, processes having a-functions that are asymp- totically logarithmic or exponential have infinite ergodic degree. Processes having cc-functions that asymptotically oscillate cannot be classified by this procedure. The classifi- cation of such processes will be considered in Section VIII.

VII. RELATION BETWEEN E(X) AND THE RECURRENCE TIME

In Section I it was shown that

E(G,) = 2 P(Om 1 1) = R,. m=O

(42)

From the plot of P(0”’ I 1) versus m shown in Fig. 10 it is observed that R, is the area contained in the rectangles of height P(Om I 1) and of unit width. P”(0”’ 1 1) is defined to be a continuous function of m, which has the same values as P(Om I 1) when m is a positive integer. Referring to Fig. 10, the area in the first rectangle is always unity. Therefore the area in the first rectangle plus the area under P(O”’ I 1) for values of m > 1 will be a lower bound on the total area in all the rectangles, i.e.,

s m P(Om 1 1) dm =

1 s m e(y+X) dx. (47)

0

To help in understanding the meaning of the right-hand integral, a new function A(x) is defined as

A(x) A s x [u(p) - 11 dp. 0

(48)

s

m l+ P”(0”’ 1 1) dm < R,. (43)

1

This is just the area between U(X) and 1 for values between 0 and x. Recalling that a(x) = -dy/dx and substituting into the integral yields

A(x) = y(0) - y - x

As indicated in the figure, p(O”- ’ I 1) is an upper bound on the rectangles. Therefore the area in the first rectangle plus the area in the second rectangle plus the area under P(OmM1 I 1) for m > 2 will give an upper bound on the area in all the rectangles, i.e.,

or equivalently

Therefore

y + x = y(0) - A(x). (49)

R, I 1 + P(0 I 1) + s

0-J p(O”‘-l 1 1) dm. (44)

2

But

s

m p(Om-lI1)dm= m

s P(O”’ 1 1) dm

2 1

and so

s O3 e(Y+X) dx = eY(o)

0 s

‘x e-A(x) dx. (50)

0

Now x = 0 when m = 1, so y(0) = log, P(0 1 1) and P(0 1 1) = e y(o) The mean recurrence time is therefore . bounded by

m 1 + P(0 I 1) s

emAcX) dx < R, < 1 + P(0 0

I 1)

1+ s

m p(O’” 1 1) dm < R, < 1 + P(0 I 1) 1

+ P(0 I 1) s

O” e 0

-A(x) dx (51)

+ s

co &O”’ 1 1) dm. (45) 1

and is closely approximated by

The upper and lower bounds differ only by P(0 I l), a quantity less than 1. Since it can be expected that R, > 10

R * E 1 + p<o!1> + p(0 1 1) s

m e-A(x) dx. 2

(52) 0

Since both the upper and lower bounds are infinite when for real channels and more usually R, > lOA, R, is closely the integral is infinite and finite when it is finite, a necessary

139

Fig. 10. Bounds for the mean recurrence time.

approximated by

R, g 1 + ‘(0 + m P”(Om I 1) dm. 2 s

(46) 1

The integral on the right-hand side can now be expressed in terms of the logarithmic variables x and y. In the x - y plane P”(Om I 1) = ey and dm = eX dx, so that

140

condition for the convergence of R, is that

s

02 e-A(x) dx < co. (53)

0

This is equivalent to requiring that the area A(x) between U(X) and 1 be greater than E > 0 for all values of x greater than some value x*, i.e.,

A(x) > & > 0 for x > x*. (54)

Moreover, observing that this condition is satisfied if the previous sufficient condition is satisfied, i.e., that U(X) > 1 for x > x*, the above condition on A(x) is both necessary and sufficient.

VIII. A NECESSARY CONDITION ON CI FOR ERGODICITY OF DEGREE r

Recall in the discussion of Section VI that ergodicity of degree Y required that the conditions given by (40) be satisfied. This led to a sufficient condition on c1 that guaranteed ergodicity of degree r, namely, that Y < a(x) < r + 1. Following a procedure similar to the one used in the preceding section, which led to a necessary and sufficient condition on CI for ergodicity of degree 1, a necessary and sufficient condition on CI for ergodicity of degree r will now be obtained.

The area to be considered is

2 m*- ‘lyom 1 1). (55) m=l

Once again the upper and lower bounds differ only by a finite constant. If

s

cc mr-lP(Om 1 1) dm < co, (56)

t

both upper and lower bounds are finite, but if this integral is infinite, then so are both bounds. The convergence of this integral is a necessary condition for ergodicity of at least degree r.

Defining the auxiliary function A,(x) such that

4(x) A s

x [c@) - rl dpc, (57) 0

it is observed that A,(x) is the area between the function E(X) and the line r along the x axis between 0 and x. The necessary condition for the existence of the rth moment of the recurrence time, and hence ergodicity of at least degree r, is that

f

m e -‘r(*) dx < co. (58)

0

Consequently necessary and sufficient conditions for er- godicity of degree r are

A,(x) > 0, x > x1*,

A,+,(x) < 0, x > x2*, (59)

where x1 * and x2 * are any positive real numbers.

IEEE TRANSACTIONS ON INFORMATION THEORY, JANUARY 1972

10 IO2 Id lo4 m

Fig. 11. a-function-Cl.

Summarizing, the E(X) diagram can be used to determine the degree of ergodicity of a process as follows. If the area between E(X) and each of the lines 1,2, * * *,r is positive, but the area between U(X) and the line r + 1 is negative, then the process is ergodic of degree r. In order to guarantee a finite-mean recurrence time R,, the process must be ergodic of at least degree 1. For processes that satisfy this condition, R, can be closely approximated in terms of the area A(x) between a(x) and 1.

IX. DATA ANALYSIS

To gain a better understanding of the behavior of the a-function of real channels, three troposcatter channels were examined.’ We were particularly interested in the behavior of E(X) for large values of x, i.e., for large m, necessitating the examination of very long error sequences. The largest sequence examined was approximately 7 x lo9 bits, which in comparison to previous analyses is indeed long. The three channels are labeled Cl, C2, and C3. Some general features of these channels are as follows.

Cl c2

Bits per second Error rate Duration of run

(min) Total number

of bits Total number

of errors Mean recurrence

time Longest gap

length

307 ooo 614 000 2.355 x lo- 2 1.347 x 10-5

0.41 102

7 63.5 632 378 266 112

179 828 50 936

42.46 74 263

10 088 11 306 822

c3

1 000 000 2.949 x 1O-5

119.76

7 125322352

210 153

33 905

3 585 916

The cr-function for these three channels is shown in Figs. 11-13. Channels Cl and C2 appear to exhibit ex- ponential or nearly exponential asymptotic behavior, which suggests they might be represented by finite-state Markov models. However, channel C3 exhibits a most unusual be-

2 The data were provided by the U.S. Army Electronic Command Fort Monmouth N.J.. March 1970.

ADOUL etal. :~TATI~~CSFOR CHANNELS WITHMEMORY 141

ti

_-_-. /

I 1x

1 10 IO3 Id 105 lob m

Fig. 12. a-function-&.

Fig. 13. a-function-C,.

havior. Its cc-function continues to vary from very small values to large values and back again. It appears to be barely ergodic, though the area A(x) between N(X) and the line x = 1 is still positive for m > 106. Such a channel clearly cannot be represented by finite-state models or the Pareto model, but could be represented by the slowly spreading chain model discussed in Section V, provided, of course, that error gaps were not correlated.

X. SUMMARY AND CONCLUSIONS

The a-function is capable of revealing a number of in- teresting properties of real communication channels and of uncovering some of the underlying behavior of channel models. From the cc-diagram of a real channel it is possible to obtain an indication of its degree of ergodicity. More-

over, since it was shown that finite-state models lead to a-functions having an exponential asymptotic behavior, the a-function can be used to determine when a finite-state model can or cannot be expected to characterize a channel.

As a consequence of our investigation of the cc-function of models previously developed to characterize real channel behavior, it was discovered that certain a-function behavior could not be described, even though real channels can ex- hibit such behavior as evidenced by channel C3 investigated in Section IX. This led to the consideration of a slowly spreading Markov model capable of exhibiting almost any a-function behavior desired. A model consisting of two coupled infinite-state slowly spreading Markov chains, which is capable of accounting for a wide variety of proper- ties of real channels, has been developed by the authors. This will be a subject of a future paper.

Finally, it should be pointed out that though the present analysis has been directed toward the representation of error sequences, the results are applicable to any binary discrete-time random processes, such as, for example, those generated by binary data sources.

ACKNOWLEDGMENT

The authors wish to thank B. Goldberg, U.S. Army Electronics Command, for his help in securing the error data used in the calculation of the a-function in Section IX.

REFERENCES [l] E. N. Gilbert, “Capacity of a burst-noise channel,” Bell Syst.

Tech. J., vol. 39, Sept. 1960, pp. 1253-1266. . . [2] P. Mertz, “Model of error-burst structure in data transmission,”

in Proc. Nat. Electronics Con/I, vol. 16, Oct. 1960, pp. 232-240. 131 J. M. Beraer and B. Mandelbrot. “A new model for error cluster- - -

ing in telephone circuits,” IBM’J. Res. Dev., vol. 7, July 1963, pp. 224236.

[4] E. 0. Elliott, “A model of the switched telephone network for data communications,” Be/l Syst. Tech. J., vol. 44, Jan. 1965, pp. 89-109.

[5] J. J. Metzner, “An interesting property of some infinite-state channels,” IEEE Trans. Inform. Theory (Corresp.), vol. IT-II, Apr. 1965, pp. 310-312.

[6] S. Berkovits, E. L. Cohen, and N. Zierler, “A model for digital error distributions,” 1st IEEE Annu. Communicat ion Conv. Proc., June 1965, pp. 103-111.

[7] P. A. W . Lewis and D. R. Cox, “A statistical analysis of telephone circuit error data.” IEEE Trans. Commun. Technol.. vol. COM- 14, Aug. 1966, pp‘. 382-389.

[8] B. D. Fritchman, “A binary channel characterization using partitioned Markov chains,” IEEE Trans. Inform. Theory, vol. IT-13, Apr. 1967, pp. 221-236.

[9] N. Muntner and J. K. Wolf, “Predicted performances of error- control techniques over real channels,” IEEE Trans. Inform. Theorv. vol. IT-14. Sent. 1968. DD. 640-650.

[IO] R. H: ’ McCullough, ‘“The binary regenerative channel,” Be/l Syst. Tech. J., vol. 47, Oct. 1968, pp. 1713-1735.

[12] S. Sussman, “Communication channel attributes as related to error control,” 1st IEEE Annu. Communicat ion Conv. Proc., June 1965, pp. 5-13.

[13] J. G. Kemeny, “Slowly spreading chains of the first kind,” J. Math. Anal. Appl., vol. 15, Aug. 1966, pp. 295-310.

[14] J. G. Kemeny, J. L. Snell, and A. W . Knapp, Denumerable Markov Chains. Princeton, N.J.: Van Nostrand, 1966, pp. 269-273.

[15] W . L. Smith, “Renewal theory and its ramifications,” J.R. Statist. Sot. B, vol. 20, 1958, pp. 243-302.