1266 IEbE TRANSACTIONS ON ACOUSTICS. SPEECH. AND SIGNAL PROCESSING. VOI. 38. NO 7 JULY IYYO Identification and Estimation of Non-Gaussian ARMA Processes KEH-SHIN LII Abstract-Finite parameter models of ARMA type have been used extensively in many applications. Under the usual Gaussian assump- tion, the second-order analysis will not he able to discriminate among competing models which give the same correlation structure. In many applications the underlying process is non-Gaussian. In this case, anal- ysis using higher order cumulants will identify the model uniquely without the usual invertibility assumption. This, in turn, will affect the interpretation as well as the forecasting based on the non-Gaussian model. We present a method which uses bispectral analysis and the Pade approximation. We show that the method will consistently iden- tify the order of the ARMA model and estimate the parameters of the model. One could also deconvolve the process to estimate the distri- bution of the input process which will provide information for possibly more efficient maximum likelihood estimation of the parameters. Var- ious asymptotic distributions are given to facilitate the model identifi- cation and parameter estimation. A few examples are presented to il- lustrate the effectiveness of the method. The procedure is modified to handle the case when there is additive Gaussian noise. The modified procedure is asymptotically consistent in the estimation of orders and parameters of the ARMA model when Gaussian noise is present. I. INTRODUCTION INITE parameter autoregressive moving average F models have been used extensively in time series modeling, forecasting, and control. Most of the literature is concerned with Gaussian processes. Let random vari- ables e,, t = * , - 1, 0, 1, . . . be independent and identically distributed with mean zero Ee, = 0, and vari- ance one Ee: = 1. Let {U,} be a sequence of real con- stants with - Consider the linear process generated by { U,} and { eJ ] m X, = C J =z -02 The frequency response function is given by If the process X, is normally distributed, then its full prob- ability structure is completely determined by its spectral Manuscript received July 1, 1989; revised December 19, 1989. This work was supported in part by the Office of Naval Research Under Contract The author is with the Department of Statistics, University of Califor- nia, Riverside, California, 9252 I. IEEE Log Number 9035663. NO00 14-85-0468, density function Hence the phase information in A (e-") is not identifiable in the Gaussian case. If A(z) is a rational function, with z on the complex plane A(z) = Qq(z)/f',,(z) (1.4) with 4 Qq(z> = C qtz', qo f 0 '=U P f',(Z) = I c =o PJ', Po = 1 (1.5) then we say that the process { X, } satisfies a finite param- eter autoregressive moving average model or simply ARMA (p, 9). Usually we write P,(B)X, = Q,(B)e, (1.6) where B is the backshift operator. There are two related problems to be considered here. The first is to determine the orders of the polynomials P,,(z) and Qq(z). This is the model identification problem. The second is the prob- lem of estimating the coefficients in P,, (z) and Q(, (z) after the model has been identified. Given a model of the form (I .6), most of the literature assumes that P,, (z) and Qq (z) have no roots on the unit disk 1 z 1 5 1. The condition P,, (z) # 0 for all I z 1 5 1 is called the realizability con- dition so that X, has a one-sided infinite moving average representation m X, = A(B)e, = i c = 0 u,ie,_J with m A(B) = c uJBJ. / = 0 (1.7) This is the same as saying U, = 0 for all j < 0 in (1.1). The condition Q, (z) # 0, 1 z 1 I 1, called the invertibil- ity condition, is not needed for stationarity. If { X,} sat- isfies (1.6) and is a Gaussian process then it is well known that any real root r/ # 0 of P,(z) or Q,(z) can be re- 0096-35 18/90/0700- 1266$0 I .OO @ 1990 IEEE
Transcript
1266 IEbE TRANSACTIONS ON ACOUSTICS. SPEECH. AND SIGNAL PROCESSING. VOI. 38. NO 7 JULY I Y Y O
Identification and Estimation of Non-Gaussian ARMA Processes
KEH-SHIN LII
Abstract-Finite parameter models of ARMA type have been used extensively in many applications. Under the usual Gaussian assump- tion, the second-order analysis will not he able to discriminate among competing models which give the same correlation structure. In many applications the underlying process is non-Gaussian. In this case, anal- ysis using higher order cumulants will identify the model uniquely without the usual invertibility assumption. This, in turn, will affect the interpretation as well as the forecasting based on the non-Gaussian model. We present a method which uses bispectral analysis and the Pade approximation. We show that the method will consistently iden- tify the order of the ARMA model and estimate the parameters of the model. One could also deconvolve the process to estimate the distri- bution of the input process which will provide information for possibly more efficient maximum likelihood estimation of the parameters. Var- ious asymptotic distributions a re given to facilitate the model identifi- cation and parameter estimation. A few examples are presented to il- lustrate the effectiveness of the method. The procedure is modified to handle the case when there is additive Gaussian noise. The modified procedure is asymptotically consistent in the estimation of orders and parameters of the ARMA model when Gaussian noise is present.
I. INTRODUCTION INITE parameter autoregressive moving average F models have been used extensively in time series
modeling, forecasting, and control. Most of the literature is concerned with Gaussian processes. Let random vari- ables e,, t = * , - 1, 0, 1, . . . be independent and identically distributed with mean zero Ee, = 0, and vari- ance one Ee: = 1. Let { U , } be a sequence of real con- stants with
-
Consider the linear process generated by { U , } and { eJ ] m
X , = C J =z - 0 2
The frequency response function is given by
If the process X , is normally distributed, then its full prob- ability structure is completely determined by its spectral
Manuscript received July 1 , 1989; revised December 19, 1989. This work was supported in part by the Office of Naval Research Under Contract
The author is with the Department of Statistics, University of Califor- nia, Riverside, California, 9252 I .
IEEE Log Number 9035663.
NO00 14-85-0468,
density function
Hence the phase information in A ( e - " ) is not identifiable in the Gaussian case. If A ( z ) is a rational function, with z on the complex plane
A ( z ) = Q q ( z ) / f ' , , ( z ) (1.4)
with
4
Q q ( z > = C qtz', qo f 0 ' = U
P
f ' , ( Z ) = I c = o P J ' , Po = 1 (1.5)
then we say that the process { X , } satisfies a finite param- eter autoregressive moving average model or simply ARMA ( p , 9 ) . Usually we write
P , ( B ) X , = Q,(B)e, (1 .6)
where B is the backshift operator. There are two related problems to be considered here. The first is to determine the orders of the polynomials P , , ( z ) and Q q ( z ) . This is the model identification problem. The second is the prob- lem of estimating the coefficients in P,, ( z ) and Q(, ( z ) after the model has been identified. Given a model of the form ( I .6), most of the literature assumes that P,, ( z ) and Qq ( z ) have no roots on the unit disk 1 z 1 5 1. The condition P,, ( z ) # 0 for all I z 1 5 1 is called the realizability con- dition so that X, has a one-sided infinite moving average representation
m
X, = A ( B ) e , = i c = 0 u,ie,_J
with m
A ( B ) = c uJBJ . / = 0
(1.7)
This is the same as saying U, = 0 for all j < 0 in (1.1). The condition Q, ( z ) # 0, 1 z 1 I 1, called the invertibil- ity condition, is not needed for stationarity. If { X,} sat- isfies (1.6) and is a Gaussian process then it is well known that any real root r / # 0 of P , ( z ) or Q , ( z ) can be re-
0096-35 18/90/0700- 1266$0 I .OO @ 1990 IEEE
L11: I D E N T I F I C A T I O N A N D E S T I M A T I O N OF N O N - G A U S S I A N A R M A P R O C E S S E S I261
placed by its inverse r,- ’ and paired conjugate complex roots can be replaced by their conjugate inverses rJ-‘ with- out changing the correlation structure of { X I }. This means that if all the roots are real and distinct then there are 2 ” + y different ways to specify the roots and they are indistin- guishable by examining the autocorrelation function. Since different set of roots correspond to different set of coefficients, it is customary to assume that all roots of P,, ( z ) and Qq ( z ) are outside the unit circle and to estimate the coefficients P,, ( z ) and Qq ( z ) under this condition. We will present a method that can be applied without impos- ing the invertibility assumption.
There are various procedures in the literature concern- ing the identification of the orders p of P,,(z) and q of Qq ( z ) . Most of these procedures involve the examination of the residuals or estimates of e,’s. In doing so, inverti- bility is assumed. The distribution of e, is assumed to be Gaussian or a known one so that maximum likelihood es- timation of the coefficients can be carried out. Box and Jenkins [9] considered an iterative procedure by examin- ing the autocorrelation function and partial autocorrela- tion function. In a series of papers, Akaike [1]-[3] pro- posed a final prediction error criteria (FPE), an information criteria (AIC), and a Bayesian version of it (BIC). These methods are based on multiple decision pro- cedure and were studied by others, (see Priestly [35]). Hypothesis testing methods were considered by Godfrey [16] and Poskitt and Tremayne [34]. Gray er al. [17] con- sidered the S array method using a pattern recognition technique. More recently Woodward and Gray [41] pro- posed a generalized partial autocorrelation method. Tiao and Tsay [38] proposed an iterative regression approach based on extended autocorrelation function. Hannan and Rissanen [20] considered a recursive method to identify an ARMA model.
Parameter estimation methods have been developed by Hannan [19], Box and Jenkins [9], Anderson [5], Ansley [6], and Kreiss [23].
If the process { X , } is non-Gaussian, Lii and Rosenblatt [27] proved that, under broad conditions, (1.2) is identi- fiable up to a sign change and/or index shift of the aJ’s requiring only that P,(z ) and Q,(z ) have no root of ab- solute value one. Later this condition was weakened in
Higher order spectral methods were used in the identi- fication and estimation problem of ARMA model in [26] without detailed statistical analysis. Many other authors including Giannakis and Mendel, Nikias, Tugnait among others considered similar problems using higher order cu- mulmts. An excellent survey is given by Nikias and Ra- ghuveer. In this paper, we propose a method to identify the orders p and q of the model (1.6) and to estimate the corresponding coefficients without the usual invertibility assumption. In particular, asymptotic distributions of var- ious statistics are derived to facilitate statistical tests. In Section 11, we adopt the higher order spectrum method proposed in [27] to estimate the a,’s in (1.7) and obtain their asymptotic distributions. In Section 111 we introduce
~301.
the C-table and the Pade table and give a method to iden- tify the model and to estimate the underlying parameters. Asymptotic distributions are derived to facilitate the model identification and parameter estimation. Section IV consists of a few examples and a discussion. The additive Gaussian noise case is discussed briefly in Section V.
The method is essentially that of obtaining impulse re- sponse via a higher order spectrum and does not require the minimum phase assumption. Related methods devel- oped by Glover, Kung, and Chan and Wood are based on second-order methods which require the minimum phase assumption. The rank computation of the Hankel matrices using singular value decomposition (SVD) described by these authors are numerically more stable than the deter- minant method used in this paper. However, no statistical analysis of the SVD method is available to facilitate test- ing. To ascertain whether a random matrix has full rank, we will have to have some knowledge of the statistical distribution of a computed random quantity such as the smallest singular value in magnitude or the determinant. The determinant by itself might not be a good indicator of the rank, but when its distribution is given, there is a sense of scale and a statistical test can be carried out. This added information should be crucial and supplemental to other information available. Computationally, when the sum of the orders of an ARMA ( p , q ) model is moderate in size, the largest size of the Hankel matrix required is moderate and the numerical instability is not serious. Moreover, the C-table and Pade table can be used to- gether with the derived statistical distributions to ascer- tain the model order and parameters. Further discussion of the interplay between the C-table and Pade table are given in [28]. The method can be adapted to the case with additive Gaussian noise which need not be white, to ob- tain asymptotically consistent results.
11. ASYMPTOTICS OF THE HIGHER ORDER SPECTRAL
Let the frequency response function from (1.2) be METHOD
A(e-1’) = ) 2 ? r f ( ~ ) l ” * exp ( i h ( A ) } . (2 .1)
There are many references concerning the estimation of the spectral density functionf( A ) . (See Anderson [5] or Jenkins and Watts [22] .) Lii and Rosenblatt [27] proposed a method to estimate the phase information h ( A ) when the process X I (and hence the process e , ) is non-Gaussian. Some basic results from this paper are summarized in the following Lemmas 2.1 and 2.2.
Lemma 2.1: Let { X I } be a non-Gaussian linear process given in (1.1) with the independent random variable { e, } having all moments finite. Assuming that
c I j l laJl < 00 with a. finite,
A(eJA) + o for all A and
h ( 0 ) = 0. (2 .2)
1268 IEEE TRANSACTIONS ON ACOUSTICS. SPEECH, AND SIGNAL PROCESSING. VOL. 38. NO. 7. JULY 1990
Then the phase h ( X ) in (2.1) is given by
h ( A ) = hl( X ) - X h , ( a ) / a + U X (2.3)
where a is an integer and x
h , ( X ) = 1, ( h ’ ( u ) - h ’ ( 0 ) ) du (2 .4 )
with
h ’ ( 0 ) - h’( X )
- h ( h + ( m - 2 ) A ) } (2 .5 )
where m > 2 is an integer such that C,,,, the mth order cumulant of { X , } , is nonzero and
h ( X , ) + * * + h ( X,- , ) - h( X I + - * +
( 2 . 6 ) where b ( ) is the mth order cumulant spectral density of the process { X , } discussed in [IO].
Remark 1: From (1.2) and (2.1) we have
~ ( 1 ) = x u , = 127rf(o)/l’~ exp { i h ( 0 ) 1
Since the a,’s are real, we have either EaJ > 0 or Ea, < 0. The assumption h (0) = 0 in Lemma 2.1 represents an arbitrary choice of the signs of a,’s. Observing X,’s only, the signs of the a,’s are intrinsically undecidable since we can multiply all u,’s and er’s by minus one without chang- ing ( I . I ) .
The integer a in (2.3) is intrinsically undecidable also since it corresponds to reindexing the X,’s.
Remark 2: However, in the usual normalization of model (1.6) or (1.7) we assume a0 > 0. Under this as- sumption we can use Theorem 2.1 (to be proved later) to ascertain the first nonzero a, and shift the index as well as adjust the sign accordingly. We will use the case when m = 3 to illustrate the techniques of the method. Similar techniques can be used to derive results form 2 4. Some aspects of the case when m = 4 are discussed in [29] and [391.
Remark 3: The assumption that A (e -”) # 0 for all X can be weakened: A(e-’’) may have finitely many zeros such that each zero is of finite order. This is discussed in detail in [30] with algorithms. The modified algorithm has the same asymptotics as the ones reported here. The same paper also proved that if X, is band limited, in particular, if A ( e-”) = 0 for X in an open neighborhood of K, then the process is unidentifiable without further assumptions. Matsuoka and Ulrych [31] gave a least square method to estimate the phase function h based on ( 2 . 6 ) with no dis- cussion of its convergence properties. If asymptotic dis- tributions of the estimates of the phase function h based
upon the least square method are available, they can be used in place of ( 2 . 8 ) and the rest of the derivations can be modified by simple substitution. The method to handle the band-limited case in the same paper required the ad- ditional assumption that the phase function h ( A ) = 0 for
= SUP{PIL(P) f 0 , 0 5 P < K } Ll
wheref, is the spectral density function of { X , }. We usu- ally do not have this information.
Lemma 2.2: Under the assumptions of Lemma 2.1. An estimate of h l ( A ) is, from (2 .4 - 2 .6 )
k - I
H , , ( x ) = - C arg b , , ( j ~ , A ) (2 .7) J = 1
where kA = A, and it is understood that the bispectral estimates b,, ( ) based on a sample of size n are weighted averages of third-order periodogram values. If b ( X , p ) c C 2 and the weight function W is symmetric and band limited with bandwidth A , then asymptotically
Hn(X) - h I W = R’,(X) + ~ , ( H , f ( X ) - W)) with
E ( % ( 1 ) = AG(X) + . ( A )
and cov(Rn(Q3 R t h . 4 )
for A ( n ) -+ 0, A2n -+ 00 as n -, 00 where G(X) is a function involving b ( A, p ) . Further, EH,, ( A, ) + h l (A, ) and the H,, ( XJ )’s are asymptotically jointly normally dis- tributed with covariances given by (2.8). Since
a = - j A(e-’A)e”h d h 1 2a 0
an estimate of a, is given by I 02?r
which by symmetry can be written as
= - 2 L c (2KX1(Xk))”* Mk=l
cos ( H , , ( X k ) - y hk +A) (2 .9 )
LII: I D E N T I F I C A T I O N A N D E S T I M A T I O N OF N O N - G A U S S I A N A R M A PROCESSES I269
where 2L = M = 2a/A and Ak represent a discretization andf, is an estimate of f ( . ) similar to that of b( * ), A is an estimate of A . For a given sample of size n, let the bandwidth of the weight function W , in A, ( A ) be A I and the bandwidth of the weight function W2 in b, ( A , p ) be A2. We now derive the asymptotic joint distribution of the bj’s given in (2.9) to first order.
It is proved in [lo] that, if for i = 1 , 2, A, ---* 0 and nA? + 00 as n + 00, then asymptotically as n + 03,
f , (Ak) and bn(AJ, p i ) are independent and normally dis- tributed with
Since H, is a function of b,, H,, andf,, are also asymptot- ically independent. Let
4 . k = f : ” (M cos (Z,(X/) + k h ) with
with
C(1, k ; j , m ) = f’/’(Xl)f’/’(Aj) sin (Z(A,) + khl)
. sin (z(A,) + mAj )
{min(A/, h j ) - A , A ~ / T ) (2.10)
and
K = - [ 21r2 W i ( U , U ) du dv. A:nC:
To see this, consider
F ( 0 ) = ( F l ( 0 ) , F 2 ( 0 ) ) = V
where F , is defined by the first component of Vand F2 is defined by the second component of V . It is noted that F(8 , ) = V,. It is clear that F is a continuously differen- tiable function of 8 (withf, ( A ) bounded away from zero). Obviously the distribution of F ( 0 , ) has mean F ( 0 ) = V asymptotically. Note that
where
- - cov ( f ( A / ) 3 .Lo,)) tl,, = cov (H,,( A,), Hn( A,)), with A, = T .
We note that the magnitude of sI./ is smaller than that of r / % , . To the first order, s entries in D , can be set to zero in the following derivation. An application of a multivariate &method (see [8, p. 4931) shows that the asymptotic dis- tribution of V, = ( d / . k , d,,,,) is bivariate normal with mean
V ( f 1 l 2 ( A / ) COS (Z(A/) + k A / ) ,
f”’(’J) ‘Os (z(AJ) + mA,) )
where
Z(A/) = h , ( A / ) - A / h I ( d / n -
and covariance matrix
where Z/,k = Z(A,) + kh/ , Z,.,,, = Z ( A j ) + mAj. D2 is obtained by evaluating ( a F / a 8 ) D 1 ( a F / a e ) ’ with sub- stitution of (2.8) into t terms in D , . Simple algebra gives D2. Using this and (2.9) we have
L L 8a cov ( a k , 2,) = - C C cov (d/,k, dJ,,,,) M 2 / = I j = ~
L L 87rK = __ C C C(1, k; j , m ) . M~ / = I j = l
Therefore, we have the following theorem. 77worem 2.1: Under the assumptions of Lemma 2.1,
(iik - ~ k ) , k = 1, * * , K are asymptotically jointly Gaussian with means zero, and covariances given by
cov ( L i k , am> =
L L
- C C ( L k ; j , m ) (2 .11) / = I j = I
where C(1, k ; j , m ) is given in (2.10). We will now assume that the stationary process { X,)
satisfies (1.6) with a representation given in (1.7) such that a. > 0. As usual, we assume P,, ( z ) and Qq ( z ) given in (1.5) have no common factor with po = 1 and qo > 0. Under these assumptions, (2.11) can be used to estimate
. 1270 IEEE T R A N S A C T I O N S ON ACOUSTICS. SPEECH. A N D S I G N A L PROCESSING. VOL. 38. NO. 7. J U L Y 1990
the variance of 211 withf( A ) and h l ( A ) estimated b y i , ( A ) and H,, ( A ) , respectively. These results can be used to es- timate the smallest integer k such that Eiih # 0. We then reindex the a,’s and change their signs if necessary. This gives a complete procedure to estimate aJ’s in (1.7) con- sistently. We use these estimated bJ’s to identify and es- timate the polynomials P,,(z) and Qy(z) by the C-table and the Pade table as discussed in [28] which deals with a distributed lag model.
111. ASYMPTOTICS OF THE C-TABLE A N D THE PADE APPROXIMANT
Given a pair of nonnegative integers q and p , we denote the Pade rational approximants to a formal power series A ( z ) = C;=O a J z / , by [ q / p l = Q,(z)/P,(z), where Q,(z) and P,, ( z ) are polynomials of degrees at most q and p , respectively. We assume P,,(O) = 1, and Q,(z) and Pp ( z ) have no common factors. The coefficients of Qq ( z ) and P,,(z) are determined by A ( z ) - (Q,(z)/P,(z)) =
0 ( z p + + I ). The following three lemmas can be found in
Lemma 3.1: When it exists, the [ q / p ] Pade approxi- [71.
mant for A ( z ) is uniquely determined. Further
Q,(z) = det
and
. . . aq-pi I a y - p + 2
a, -1’ + 2 ay -1’ + 3 . . .
. . . a, ay+ I
Y Y
C a,-,z’ C ai-Pi I z ~ * J = P j = p - I
- ay+ I
ay+2
ay + I ’
C aizJ Y
- j = O
where a, = 0 i f j < 0. and the summation is set to zero if the lower index on a sum exceeds the upper index. Given nonnegative integers r and s, we define
c,, =
I . ( 3 . 3 )
The C-table, which is a doubly infinite array, is defined by C = ( Cr.,):,=O. We further define Cr,O = I.
Lemma 3.2: i) C(,,,, # 0 implies that [ q / p ] exists. ii) Every zero entry in the C-table for a formal power series A ( z ) = 1 + CJ”= I a,zJ occurs in a square block of zero entries and is completely bordered by nonzero entries.
Lemma 3.3: Given a formal power series A ( z ) , the fol- lowing three conditions are equivalent
1) A ( z ) = c;=,c,zJ/( 1 + d , z l ) 2 ) [ 4 / p 1 = A ( z ) for all q 2 1 a n d p L m 3 ) C,,,,, f 0 and C,,\ = 0 for all r > I and s > m.
If condition ( 3 ) in Lemma 3.3 is satisfied, we call the entry ( I + 1, m + 1 ) in the C-table the “breaking point.”
Lemmas 3.1, 3 .2 , and 3.3 lead to the following. Theorem 3.1: The process {X,} given in (1.7) is
ARMA ( p , q ) given in (1.6) if and only if the C-table associated with A ( z ) has the breaking point ( p + 1, q + 1 ). Further, Ccl.l, # 0 and the coefficients of P,,(z) and Q,(z) in (1.6) are obtained from (3.1) and (3 .2) . To nor- malize these coefficients we divided both P,, ( z ) and Qy ( z ) by Cq,, so that p o = 1. Whether the roots of Q,(z) are inside or outside of the unit circle is immaterial here.
This theorem provides a consistent procedure to iden- tify the model by determining the orders p and q, and to estimate the Rarameters of the identified model. The pro- cedure is
i) Use the 2,’s f o r j = 0, . * . , K , from (2 .9) to con- struct estimates i?r,.y of c,,,, for r = 0, . . . , K / 2 , s = 1, - . . , K / 2 , and form a truncated e-table. (See Table I). K is chosen somewhat larger than a guessed max { 2p, :q}. If there is information on the bounds of p and q, the C-table could be a rectangular one to save some computation.
ii) Use theorem 3.2 to calculate the estimated standard deviation u(cf . , .y ) for each entry I ? ~ , ~ of the e-table in i) (see Table I).
iii) Form the resolution table with entries Rr.,y =
1 er,.\ 1 /U( er,$) (see Table 11). Use Rr,,T to test whether Cr,,, = 0 or not. For example, Rr,,s can be compared with 1.96 for CY = 0.05 approximate significance level or 3 (for 3 u ) if one wants to be more conservative.
iv) If the model is ARMA ( p , q ) then all entries Rr.,q in the resolution table will be less than 2 asymptotically, for r > q and s > p . Here both Lemmas 3.2 and 3.3 can be used to ascertain the breaking point. In practice, we look for a global pattern of zeros indicated by Lemma 3.3 ( 3 ) rather than for individual zeros.
v) Qnce ( p , q ) are identified, (3 .1) and (3.2) in Lemma 3.1 are used to compute the [ q / p ] Pade approximant, i .e., the coefficients of the ARMA ( p , q ) model. Gen- erally the Pade approximants [ S / Y ] should be computed f o r r = 0, - , K / 2 , s = 1, , K / 2 , as well as each coefficient’s estimated standard deviation (from Theorem 3.3) to form a Pade table. This table is used together with the C-table.
L11: IDENTIFICATION A N D ESTIMATION OF N O N - G A U S S I A N A R M A PROCESSES 1271
TABLE I 6-TABLE FOR EXAMPLE I , 6, ~ IS AT T H P TOP A N D T H E ESTIMATE S T A N D A R D
DEVIATION ( C, ,) I S AT T H F BOTTOM OF EACH ENTRY
There is some further discussion of this in [28]. The following lemmas can now be proved with simple modi- fications of the proofs given in [28, sec. 41.
Lemma 3.4: If { b, }f;= I are asymptotically jointly Gaussian for a fixed integer K with mean { a, },"= I and co- variance matrix C ( C,l ) where C,, = n-& 4 0 for a fixed 6 > 0 as the s!mple size n + 00, then the asymptotic dis- tribution of R, the determinant of the M X M matrix
is Gaussian with mean M
R = det [(af , ) , , ,=,1
and variance U: = GC ( C,l ) G' where G' is the transpose
We note that g, is the sum of the cofactors of aJ in the
Pro08 Note that each a',, = a/ for some I = 1 , . . . , K . Hence, in the expansion of the determinant by cofactors we may collect terms according to the order of the product of (6, - a J ) ' s . We can verify that
of G = ( g , , . , g K ) with gJ = ( a / a u J ) R .
matrix [ (aC) l .J= 11'
M
d = O E N l = d
where NJ's are nonnegative integers. Therefore, to first order, R - R is asymptotically dis-
tributed as Cj"i I ( ij - a,) ( a / & , ) R and the result follows.
1272 IEEE T R A N S A C T I O N S ON ACOUSTICS. SPEECH. A N D SIGNAL PROCESSiNG. VOL. 38. NO. 7. JULY 1990
The following result is an immediate application of this
Theorem 3.2: If the estimates of { a , } in (1.7) are given lemma.
by { 6, ] obtained from (2.9), then for fixed r and s \ ( \ - I ,/?
e r 3 = ( -1 ) det [ (‘l f l - J ) : , = I ]
is asymptotically normally distributed with mean C, , given in (3.3) and variance GC G‘ where G = ( g L , g L + I , . . . , gu) , with L = r - s + 1, U = r + s - 1, g, = (d/da,)C,,,, and C is the covariance matrix of ( d L , . . . , d u ) from (2.11). Here we use the convention that a, and 6, are zero f o r j < 0.
This theorem gives a method to construct the e-table and to find the breaking point ( q + 1 , p + l ) .AI f the breaking point cannot be uniquely determined, the C-table will still reduce the number of possible competing models to only a few for further testing. If the process { X, } does not have a rational frequency response function [ q / p ] , the &able will suggest a possible ARMA ( p , q ) approx- imation to A ( z ) using the principle of parsimony. Once we have identified the model to be ARMA ( p , q ) , re- placing the a,’~ by their estimates 6,’s in (3.1) and (3.2), we obtain estimates $, and $, of p , and q,, respectively, by
with p;, = e,$ # 0
true model orders. When r = 0. we have a simple MA case with coefficients given by the 6,’s. Using Lemma 3.4 and (3.4)-(3.7) we can prove the following.
Theorem 3.3: For any fixed p and q which are not nec- essarily the true ARMA model orders, let L = max { 0, q - p + 1 1 and U = q + p - 1 . Then the asymptotic distributions of (pf’ - p f , - p j ) , (a,’ - p , , qi - q,) and (q,‘ - qf, qi - 4,) in [ q / p ] are and bivariate normal, with mean (0, 0 ) and covariance matrices
and
and C = cov (6L , * * . from (2.11). L. U
’ “ (3.5) To obtain the asymptotic distributions of e l ’ s and q,’s, we evaluate the determinant in (3.2) by cofactor expansion of the last row ( z p ,
A , and BI are the theoretical values of 2, and bI, respec- tively, in (3.6) and (3.7). Furthermore, the asymptotic distribution of 6, - p I and 4, - qJ are normal with mean zero and variances
q, = qyp;,, j = 0, 1, ’ * *
- . , 1). We obtain
P p ( , ) = a, - a,, + A 2 z 2 + . . * + ( - l ) p a p , ~
( 3 . 6 ) where a, is the cofactor of Z‘ in (3.2).
Similarly, we have from (3.1) 4
j = O Q&) = a,( c d’ii) - A,( j = 2 5 i i ’ - , z J )
= a,a, + (2,& - ii,A,)z To see this, note thatp: = a, in (3.6) and 4,’ = B, in (3.7). Following the proof of Lemma 3.4 we see that a,’ - p,’ and 4; - 4,’ asymptotically have the joint distribution of
- a , ) ( d / d a , ) A , and Cy=L(dJ - a,)(d/da,)B,,
Therefore, the result of (3.8). An application of the 6 method with (3.8) gives (3.9) with the note that pb # 0.
+ * * . + (6,& + . . * + ( - 1 )p64-pa,,)Z4
7 4. . , p , and j = 0, * - . ( 3 . 7 ) f o r i = 0, = BO + B , z + * * + B,zY.
The preceding discussion is valid for arbitrary ( p , q ) = ( r , s ) with r 2 1 , s 2 0, i.e., ( p , q ) need not be the
LI1. IDENTIFICATION AND ESTIMATION OF NON-GAUSSIAN AKMA PROCESSES 1273
IV. EXAMPLES A N D DISCUSSION
model of the form P , , ( B ) X , = Q , ( B ) e , with Examples in this section are simulated according to the
where the e,‘ s are independent, identical, exponentially distributed with p = Ee; = 1 and U’ = var ( e ’ ) = 1. Hence, Ee, = 0, var ( e , ) = 1. The sample size for X I is 640. The rectangular window is used with 9 point aver- aging for the spectral density estimation, and a 9 x 9 square window with constant height is used for the bi- spectrum estimation. K is set to 15. Some computational details are discussed in [25] and [27].
Example I :
Q > ( B ) = 1 - 0.6B + 0.8B’
P , ( B ) = 1 + 0.6B.
All the roots are outside the unit circle. Table I gives the e-table associated with this model. Each entry has two numbers, the upper one is e,.,, p d the lower one is the estimated standard deviation of C,., , computed from Theo- rem 3.2. We also exhibit Table I1 which gives the ratio of er,s and its estimated standard deviation (from Table I). We call Table 11, the “resolution table” of Table I. In the resolution table, entries with absolute value less than one are set to zero. It is much easier to recognize the pat- tern in a resolution table. When there is a sudden drop of resolution at entry (1, m ) and thereafter, ( 1 , m ) is likely to be the breaking point. From Table 11, it is clear that (3 , 2 ) is the breaking point and the model is correctly identified as ARMA ( 2, 1 ). The Pade approximant [ 2 / 1 ] gives, from (3.4) and (3.5)
@ ( B ) = 1.068 - 0.585 B + 0.763 B2 (0.446) (0.212) (0.097)
and
P l ( B ) = 1 + 0.594 B (0.092)
where the numbers in the parentheses are estimated stan- dard deviations from Theorem 3.3.
Example 2: In this example, both roots, -0.5 and -0.75, of
Q , ( B ) = 1 + 3.5B + 3B’
are inside the unit circle, while the roots of P , ( B ) = 1 + 0.35B + 0.5B’ are outside the unit circle. The associated e-table is Table I11 and its resolution table is Table IV. It
seems reasonable to identify the model to be ARMA (2 , 2 ) with breaking point at (3, 3 ) . In Table 111, if we just have the upper part of each entry e,., , i t might not be easy to identify the breaking point. Given the estimated a( er, , ) to form Table IV, it is rather easy to identify the breaking point. This illustrate the importance of Theorem 3.2. The Pade approximant [ 2/21 gives
@ ( B ) = 0.907 + 3.64 B + 2.868 B’ (0.94) (13.7) (10.5)
and (4.1) P 2 ( B ) = 1 + 0.55 B + 0.53 B2.
(0.60) (0.62)
The large estimated standard deviations in Example 2 may be due to the complicated formula in Theorem 3.3, and the fact that the number of parameters is large relative to the sample size. Nevertheless, the estimates of the pa- rameters provide good starting values for possibly more efficient iterative methods. We note that the usual itera- tive fitting procedures can be used here. We can decon- volve the process X , and estimate the innovation process e, by 2,. Diagnostic checking can be performed on P, to discriminate among possible competing models. The probability distribution or density function of e, can be estimated to facilitate a non-Gaussian maximum likeli- hood estimation of parameters as considered in [23] (see also [36]). It seems that in building a finite parameter ARMA model of a stationary times series { X I } , one should use the procedure suggested in [27] to deconvolve X , and see if { e, } is near Gaussian or not. If not, one should use the procedure suggested in this paper to build the ARMA model without imposing the invertibility con- dition. Alternatively, one may want to use any one of those methods mentioned in the introduction section, using mainly the second-order structure, to identify the orders of the model. However, one should still use the Pade approximant to estimate the necessary coefficients and to identify whether the roots lie inside or outside the unit circle since second-order methods are blind to inher- ent all-pass factors. Even in the Gaussian case, one may want to first fit an MA( K ) model for a moderate integer K (say 15). Then, following the procedure in Section 111, one can identify the parsimonious ARMA ( p , q ) model and obtain estimates of the parameters. As a comparison, we employed the usual Box-Jenkins type estimation pro- cedure, as implemented in the subroutines NSPE and NSLSE of the International Mathematical and Statistical Library (ISML). Given the right orders in the model. we obtain estimates
Q,(B) = 1.0 + 1.2257 B + 0.3580 B 2 (0.016) (0.016)
and
P , ( B ) = 1.0 + 0.3853 B + 0.4769 B 2
with estimated white noise variance U : = 8.789.
(0.034) (0.034) (4 .2)
1274 IEEE TRANSACTIONS ON ACOUSTICS. SPEECH. A N D SIGNAL PROCESSING. VOL 38. NO. 7. JULY 1990
TABLE 111 C-TABLE FOR EXAMPLE 2; 6, ~ IS A T T H F TOP A N D T H E ESTIMATE STANDARD
DEVIATION ( c, $ ) IS AT T H E BOTTOM OF EACH ENTRY
1 2 3 4 5 6 7 R
0.907EC00 0.411E+00
0.313E+01 0.245EC00
0.625E+OO 0.641E+00
-0.203E+01 0.232EC00
0.795EC00 0.326E+00
0.436E+00 0.2471+00
-0.6223+00 0.136E+00
0.135E+00 0.149E+00
-0.823EC00 0.747EC00
-0.9243+01 0.124Et01
-0.675E+01 0.166E+01
-0.363EC01 0.885E+00
-0.152E+01 0.511E+00
-0.6853+00 0.251E+00
-0.328Ef00 0.177E100
0.563E-03 0.443E-01
-0.7471+00 0.102E+01
-0.255E+02 0.632E+01
-0.193E+02 0.6791+01
0.144E+01 0.241E+01
0.228E+00 0.978E+00
0.666E-01 0.3641+00
0.174E+00 0.129E+00
0.340E-01 0.507E-01
0.677Et00 0.123Ef01
0.688E+02 0.270E+02
0.605Ef02 0.241E+02
0.178EC01 0.6431+01
-0.290E-01 0.193E+00
-0.512E-01 0.206E+00
0.851E-01 0.872E-01
0.4748+00 0.352E+00
0.614E+00 0.139Ef01
0.184E+O3 0.102E+03
0.184Ef03 0.7781+02
-0.342E+01 0.165E+02
-0.404E+00 0.135E+01
-0.764E-01 0.135E+00
-0.182E+00 0.536E+00
-0.579EC00 0.245E+01
-0.557E+00 0.152E+01
-0.491E+O3 0.3581+03
-0.567Ec03 0.2621+03
-0.483Et02 0.3608+02
-0.3383+01 0.296E+01
-0.132E+01 0.3561+01
0.133E+00 0.12OE+O1
-0.610EC01 0.198E+02
-0.506E+00 0.161E+01
-0.131Ef04 O.l18E+04
-0.162Et04 0.8021+03
0.121E+03 0.320E+02
-0.129E+03 0.957E+02
0.286Ec02 0.109E+03
-0.4422+02 0.131EC03
0.840E+02 0.145E+03
0.4593+00 0.166E+Ol
0.348E+04 0.3781+04
0.492Ef04 0.2523+04
-0.404E+04 0.219E+04
0.391E+04 O.l88E+04
-0.3711+04 0.169Ef04
0.342E+04 0.1353+04
-0.275E+04 0.132E+04
TABLE IV RESOLUTION TABLE FOR EXAMPLE 2 , EACH ENTRY IS T H E QUOTIENT OF e,,$ AND (f,,,) IN TABLE 111. I F T H E QUOTIENT IS LESS THAN 1 IN MAGNITUDE,
THE V A L U E IS S E T T O Z E R O , *INDICATE 'THE LOCATION OF T H E BREAKING POINT
1 2 3 4 5 6 7 R
0 2.204
1 12.805
2 0.000
3 - 8.771 4 2.436
5 1.768
6 - 4.566 7 0.000
-1.102
-7.448
-4.057
-4.102
-2.972
-2.731
-1.857
0.000
0.000
-4.038
-2.839
*o. 000
0.000
0.000
1.344
0.000
0.000
2.548
2.512
0.000
0.000
0.000
0.000
0.000
0.000
1.800
2.359
0.000
0.000
0.000
0.000
0.000
0.000
-1.372
-2.167
-1.341
-1.143
0.000
0.000
0,000
0.000
-1.103
-2.024
3.777
-1.349
0.000
0.000
0.000
0.000
0.000
1.956
-1.841
2.075
-2.192
2.526
-2.082
TABLE V
WHITE NOISE WITH STANDARD DEVIATION 0.4, *INDICATE THE LOCATION RESOLUTION TABLE FOR EXAMPLE 2 WHEN THERE IS ADDITIVE GAUSSIAN
OF THE BREAKING P O l V T
1 2 3 4 5 6 7 8
4.7044
16.3718
3.9904
-14.2257
3.3924
5.3379
-3.0630
0.0000
-2,3522
-7.1246
-7.8607
-6.3672
-4.7431
-3.3922
-1.3634
-1.4977
-1.5681
-2.6410
-8.0654
t1.7988
0.0000
0.0000
0.0000
0.0000
1.1761
0.0000
6.3195
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
5.2858
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
-4.5252
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000
1.2299
-3.8769
0.0000
0.0000
0.0000
0.0000
0.0000
~~
0.0000
-2.0448
3.4737
0.0000
0.0000
0.0000
0.0000
0.0000
~ ~ ~ ~
Using Q 2 ( B ) in (4.2) to interpret the model may be As an indication of the effect of additive noise, we con- quite different from that of using Q2 ( B ) in (4.1). ample 1, these IMSL routines yield the estimates:
+ 0.998 B 2 Q 2 ( B ) = 1.0 - 0.729 B (0.000006) (0.0001)
P , ( B ) = 1.0 + 0.3228 B . (0.0259)
In Ex- sider now that the observed data is given by Z , = X, + Y,, where X , is from Example 2 and Y, is zero mean white Gaussian noise with (T = 0.4. The resolution table is given in Table V which correctly identifies the orders and the Pade approximant is
Q 2 ( B ) = 1.27 + 2.967 B + 2.516 B 2 (1.95) (6.16) (4.319)
I
LII: I D E N T I F I C A T I O N A N D E S T I M A T I O N O F N O N - G A U S S I A N A R M A P R O C E S S E S I275
P , ( B ) = 1.0 + 0.405 B + 0.756 B 2 . (0.176) (0.607)
Here we see that while model identification is not per- turbed much, there is increased instability in the param- eter estimates.
Example 2 shows that we can discriminate models which are indistinguishable using only second-order prop- erties. The method proposed in this paper produce esti- mates that are consistent. For moderate sample size this method can be a valuable tool for ARMA model identi- fication and estimation.
V. THE CASE WITH NOISE We now briefly comment on the case when there is ad-
ditive Gaussian noise. The model is that we observe
2, = x, + U, (5.1) with X, given in (1.7) and Y, is Gaussian noise which is independent of { X , } .
It is clear theoretically that the bispectrum of Z and X are the same, i.e., b, (Al , A 2 ) . An estimate , , b , ( A I A ~ ) of bz( A I , A 2 ) will be a consistent estimate of b,( A,, A*) al- though it will have an inflated covariance structure com- pared to the case without noise. In fact, we can show that
* (1 + I“[?)* A ( e - ’ ” ) du s w 2 ( u , U ) du du
(5 .2)
where we assume that
Y, = c P , E , - ~ E - N ( 0 , 1) i.i.d. (5.3)
and / 3 ( e - ” ) = C/3,e-‘”. This is in contrast with the case without the noise given in (2.8).
One can still form a consistent estimate of the phase of A ( e P t A ) via the analog of (2.7). However, the second- order spectrum &( A ) = ,, fx( A) + fr( A ) will not give consistent estimates of 1 A ( e - ” ) 1 . The effect of this dis- tortion is hard to predict. One hopes that when Y, is white and the standard deviation is not large, the distortion in the estimation of u,’s will not be severe. To get a con- sistent estimate of 1 A (ep”) 1 we observe that b,( A, 0 ) =
( C 3 / ( 2 ~ ) ’ ) A ( 1 ) 1 A ( e p t h ) I*. Therefore, we have a con- sistent estimate of I A ( e - ” ) 1 given by
.bZ(A, 0)(2T)*/C3A( 1 ) (5.4)
where b, is a consistent estimate of the bispectrum b, of { Z, }. Again the covariances will be inflated in compari- son with that off,(A) in (2.9).
It is now clear that we will have estimates of a,’s from
(2.9) with appropriate substitutions up to a common con- stant C = ( ~ T ) ~ / C , A ( 1 ). To take care of this unknown constant C, we note that in (1.7) there is a natural nor- malization, say either uo = 1 or var( e , ) = 1. (We have used the condition that var(e,) = 1 in (1.3), and conse- quently u0 is not necessarily 1 in (1.17).) In the present situation, since all information is obtained from the bi- spectrum and we do not use any information of second- order statistics, we can choose to normalize a. = 1 (in- stead of 0: = 1 ) to obtain consistent estimates of ai’:. Accordingly, we can compute the analog of (2.1 I ) , C- table, and Pade table to obtain consistent estimates of p , q, as well as the coefficients, although less efficiently in comparison with the case without the noise. The preced- ing discussion gives a sketch of a procedure to handle the case with additive Gaussian noise. Their detailed prop- erties will be studied later.
REFERENCES H. Akaike, “Fitting autoregressive model for prediction,” Ann. Inst.
-. “Information theory and an extension of the maximum likeli- hood principle,” in 2nd Int. Symp. Inform. Theory, B . N. Petrov and F. Caski, Eds. - , “A Bayesian analysis of the minimum AIC procedure,” Ann. Inst. Stut. Math . , vol. 30, pp. 9-14, 1978. T . W. Anderson, The Spectral Analysis of Time Series. New York: Wiley, 1971. -, “Estimation for autoregressive moving average models in the time and frequency domains,” Ann. Stat., vol. 6, pp. 842-865, 1977. G. F. Ansley. “An algorithm for the exact likelihood of a mixed au- toregressive moving average process,” Biometrica. vol. 66, pp. 59- 65, 1979. G. A. Baker, Jr.. Essentials of Pade Approximations. New York: Academic, 1975. Y. M. M. Bishop, S . E. Fienberg, and P. W. Holland, Discrete Mul- tivariate Analysis-Theory and Practice. Cambridge, MA: M.I.T. Press, 1975. G. E. P. Box and G. M. Jenkins, Time Series Analysis: Forecasting cmd Control. D. R . Brillinger and M. Rosenblatt. “Asymptotic theory of estimates of kth order spectra,” in Specfrul Analysis of Time Series, B . Harris, Ed. Y. T . Chan and J. C. Wood, ”A new order determination technique for ARMA processes,” lEEE Trans. Acnust., Speech, Signal Pro- cessing. vol. 32, pp. 517-521. 1984. C. Chatfield, “Inverse autocorrelation,” J . Roy. Stut. Soc. A . vol. 142, pp. 363-377, 1979.
Stat. Math. . vol. 21, pp. 243-247, 1969.
Budapest: Akademia Kiado, 1973, pp. 267-281.
San Francisco: Holden-Day, 1976.
New York: Wiley, 1967. pp. 153-188.
[I31 G. B. Giannakis and J. M. Mendel, “Approximate realization and model reduction of nonminimum phase systems, ’’ in Proc. Z5fh lEEE Con$ Decision, Contr. (CDC) (Athens. Greece), 1986.
[I41 -, (1989). “Identification of nonminimum phase system using higher order statistics,” IEEE Truns. Acous t . , Speech, Signal Pro- cessing, pp. 360-377, 1989.
[IS] K . Glover, ”All optimal Hankel-norm approximations of linear mul- tivariable systems and their L,-error bounds.” lnt . J . Contr.. vol.
[I61 L. G. Godfrey, “Testing the adequacy of a time series model,” Biometrica, vol. 66, pp. 67-72. 1979.
(171 H. L. Gray, G. D. Kelley, and D. D. McIntire. “A new approach to ARMA modeling,” Commun. Stat., vol. B7, pp. 1-77, 1978.
[ 191 E. J. Hannan, “The estimation of mixed moving average autoregres- sive system,” Biometrica. vol. 56, pp. 579-594, 1969.
120) E. J . Hannan and J . Rissanen, ”Recursive estimation of mixed au- toregressive-moving average order,” Biometrica. vol. 1 , pp. 81-94. 1982.
[22] G. M. Jenkins and D. G. Watts, Spectral Anulysis and Its Applicu- t ion. San Francisco: Holden-Day, 1968.
1231 J . P. Kreiss, “On adaptive estimation in stationary ARMA pro- cesses.” Ann. Stat . , vol. 15, pp. 112-133, 1987.
39. pp. I 115-1 193. 1984.
1276 IEEE TRANSACTIONS ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, VOL. 38, NO. 7. JULY 1990
[24] S . Kung, “A new identification and model reduction algorithm via singular value decomposition,” in Proc. 12th IEEE Asilornar Conf: Circuits, Syst., Comput., 1979, pp. 705-714.
[25] K. S . Lii and H. N. Helland, “Cross-bispectrum computation and variance estimation,” ACM Trans. Math. Software, vol. 7 , pp. 284- 294, 1981.
1261 K . S . Lii, “Non-Gaussian ARMA model identification and estima- tion,” Proc. Bus., Econ. Stat., pp. 134-141, 1982.
[27] K. S . Lii and M. Rosenblatt, “Deconvolution and estimation of trans- fer function phase and coefficients for non-Gaussian linear pro- cesses,” Ann. Stat., pp. 1195-1208, 1982.
(281 K . S . Lii, “Transfer function model order and parameter estima- tion,” J . Time Series Anal., pp. 153-169, 1985.
[29] K. S . Lii and M. Rosenblatt, “A fourth-order deconvolution tech- nique for non-Gaussian linear processes, ” in Mulrivariate Analysis VI, P. R. Krishnaiah, Ed. Amsterdam: Elsevier, 1985, pp. 395- 410.
[30] -, “Estimation and deconvolution when the transfer function has zeros,” J . Theoret. Probabil., pp. 93-113, 1988.
1311 T. Matsuoka and T. J . Ulrych, “Phase estimation using the bispec- trum,” Proc. IEEE, vol. 72, pp. 1403-1411, 1984.
[32 ] C . L. Nikias, “ARMA bispectrum approach to nonminimum phase system identification,” Dep. Elec., Comput. Eng., Northeastern Univ., Boston, MA, Tech. Rep. CDSP-86-102, 1986.
[33] C. L. Nikias and M. R. Raghuveer, “Bispectrum estimation: A dig- ital signal processing framework.” Proc. IEEE, pp. 869-891, 1987.
[34] D. S . Poskitt and A. R. Tremayne, “Testing the specification of a fitted autoregressive-moving average model,” Eiometrica, vol. 67,
New pp. 359-363, 1980.
[35] M. B. Priestley, Spectral Analysis and Time Series, vol. 1. York: Academic, 1981.
[36] D. Sengupta and S . Kay, “Efficient estimation of parameters for non- Gaussian autoregressive processes,” IEEE Trans. Acoust. , Speech, Signal Processing, vol. 37, no. 6 , pp. 785-794, 1989.
[37] J . L. Shanks, “Recursion filters for digital processing,” Geophysics, vol. 32, pp. 33-51, 1967.
1381 G . C. Tiao and R. S. Tsay, “Identification of nonstationary and sta- tionary ARMA models,” Proc. Bus. Economic Stat. Sec. Arner. Stat. A s s . , 1981.
[39] J . K. Tugnait, “Identification of nonminimum phase linear stochastic systems,” Auromatica, vol. 22, pp. 457-464, 1986.
[40] J . K. Tugnait, “Identification of linear stochastic systems via second- and fourth-order cumulant processes,” IEEE Trans. Inform. Theory,
[41] W. A. Woodward and H. L. Gray, “On the relationship between the S array and the Box-Jenkins method of ARMA model identification,” J . Amer. Stat. Ass., vol. 76, pp. 579-587, 1981.
pp. 393-407, 1987.
Keh-Shin Lii received the B S degree in mathe- matics from the National Taiwan Normal Univer- sity in 1969 and the M.S and Ph.D. degrees in mathematics from the University of California, San Diego, in 1973 and 1975, respectlvely
From 1975 to 1978 he was an Assistant Pro- fessor at the Department of Mathematics of Northwestern University He joined the Depart- ment of Statistics of the University of California, Riverside, in 1978 where he is an Associate Pro- fessor. His major research interests include theory
and applications of time senes analysis, stochastic modeling, signal pro- cessing, function estimation, and higher order spectral analysis.
