Efficient implementation of the Volterra filter

M .J. Reed and M.O.J. Hawksford

Abstract: An efficient implementation of the Volterra filter is presented which uses a frequency domain representation to reduce the number of computations. The multidimensional convolution of the Volterra filter is transformed to the frequency domain giving a transformed input matrix which is sparse and obtained directly from a one-dimensional Fourier transform. In addition to the sparse nature of the transformed input matrix, symmetries in both the Volterra filter and the frequency domain representation are exploited to increase the efficiency of the algorithm. The computational saving is demonstrated by comparing it with the direct implementation of the time domain representation and another technique which uses a frequency domain representation but does not utilise symmetry.

1 Introduction

The Volterra series represents a nonlinear system as a set of multidimensional convolutions [ l ] and is used to model a range of nonlinear systems such as the auditory system [2], mechanical systems [3,4] and electromechanical loudspea- kers [ 5 ] . Possible applications of such a Volterra series model include system prediction due to arbitrary excitation and nonlinear correction [l]. For either of the mentioned applications it is required to implement the Volterra model of the nonlinear system and this is performed using a signal processing algorithm termed a Volterra filter. However, the Volterra filter requires the implementation of computation- ally intensive multidimensional convolutions that make it unsuitable for applications requiring real-time perfor- mance. This paper describes a new technique for the implementation of the Volterra filter that improves imple- mentation efficiency compared with existing techniques. One target application for this work is towards real-time implementation of nonlinear correction for loudspeaker transducers [6]. However, the technique can be generally applied to improve the efficiency of any Volterra filter implementation.

Sandberg has shown that a good approximation to a wide range of nonlinear systems is given by a Volterra filter which is a doubly finite and discrete implementation of the Volterra series that is expressed as [7]

N M - I M-I

r=l i , = O i,=O

of nonlinearity and M is the memory of the filter. The only general condition placed upon the input signal u is that the model is usually only valid over a finite input amplitude [7], for example in many systems the model is only valid at amplitudes below the occurrence of clipping.

The implementation of eqn. 1 in the direct form for models of real systems is computationally burdensome as the number of points in the kernel spaces is usually large [8]. It is possible to improve the efficiency of the Volterra filter by approximating part of the kernel spaces using a mirror filter [9] constructed from linear filters combined with multipliers. However, then the model is only valid for a limited set of signals that excite the kernel spaces for which the approximation is valid. Consequently, for accu- rate system prediction or correction with unconstrained input signals the whole kernel space rather than partial approximations to it are required [6]. This paper presents an efficient frequency domain implementation of the Volterra filter incorporating the whole kernel space. Frequency domain techniques already exist [ 10, 1 11 but, unlike the method presented, they do not make use of symmetries in the filter to improve efficiency and do not extend the result to a general order of nonlinearity.

2 Expressing the Volterra filter in vector notation

Define a column vector h, which represents all permuta- tions of points in the kernel h, from eqn. 1

u(n - i,)u(n - i2). . . u(n - i,) (1)

where u is an arbitrary input signal to the model, y the output, h, is the rth Volterra kernel, N the maximum order

h, =

0 IEE, 2000 IEE Proceedings online no. 20000183 DOI: IO. 1049/ip-vis:20000183 Paper first received 9th July and in revised form 25th November 1999 The authors are with the Department of Electronic Systems Engineering, University of Essex, Wivenhoe Park, Colchester CO4 3 S Q , UK

IEE Proc.-Vis. Image Signal Process., Vol. 147, No. 2, April 2000

h , ( M - l , M - 1, . . . , M - - 2)

h , . (M-l ,M-l , . . . , M - 1 )

The format of eqn. 2 represents the multidimensional points of a Volterra kernel (a multidimensional impulse response function) as a single vector by representing the

109

points of the kernel in natural order. For example, a second-order kernel

=

All the rth-order products of the input from u(n) to u(n - M+ 1) may be expressed as the rth-order Kronecker product (denoted as the superscript [ " I ) of U, with itself

U, @ U, @ . . . @ U, = u p P

r times

where @I is the Kronecker product between two vectors or matrices which is defined as [12, 131

- U!'

&I --

n- l --

_- &I - n - ( M - I )

where A is of dimension m x n and a i j is an element from A .

Thus, eqn. 1 can be represented in vector notation as N

(3)

y = U"'hr (4) r= 1

Thus, for a block of M samples, each order r of the direct form of the Volterra filter requires rM'+' multiplications.

3 Implementation of the Volterra filter using Fourier transform relationships

We will now show how Fourier transform relationships often applied to linear convolution can be extended to the multidimensional convolution of the Volterra filter giving rise to a sparse matrix operation.

110

Define a circulant input signal matrix C of the form

u(n - (M - 1))

u(n - 1) u(n>

u(n - (M - 1)) u(n - (A4 - 2))

. . . I u(n - ( M - 2)) u(n - 1)

u(n - 2) u(n - (M - 1)) . . .

u(n - (M - 3)) . . . u(n)

Define the Fourier transform as the matrix F of dimension M x M with the element X k of the ith row and kth column given by

where wM = ecznJiM. Define the inverse Fourier transform as FH, the conjugate transpose of F. To make use of the orthonormality of the M x M transform FHF = I with the larger dimensional Volterra filter we make use of the corollary from the following theorem. Note that while the authors do not claim that the following theorem and corollary are novel, they are not aware of related published work. Theorem 1: A[']B['] = (AB)['] Proof

A["lB['] = @I A ) (B['-'] @ B ) (5)

By application of the mixed product rule [12]

(Ars1 @ A ) (&' @ B ) = (AislBisl) @ (AB) (6)

By repeated application of eqn. 6 on the right side of eqn. 5 and eqn. 5 on the first term on the right side of eqn. 6 for s = r - 1, . . . , 2 the following is obtained

A['lB" = (AB) @ . . . 8 (AB)

= (AB)[']

From theorem 1 the following corollary is obtained: Corollary I : If A is orthonormal then A['] is also ortho- normal.

Corollary 1 follows from theorem 1 as if A is orthonor- mal AAH =IM where 1, is the M x M identity matrix and by applying theorem 1

A['I(AH)['l = ( A A f y I

= I,,

As $$ is the M" x IV identity matrix then A['] is ortho- normal.

As the defined Fourier transform relationship is ortho- normal then, from corollary 1, eqn. 4 can be expressed as

(7)

where the input matrix U is replaced by the circulant matrix C.

IEE Proc.-Vis. Image Signal Process., Vol. I47, No. 2, April 2000

To refer to the elements in the matrices or vectors which arise from the Kronecker product we define the following notation. The ith row in a matrix B is defined as the row vector bi,, the kth column as a column vector b,, and the ith element in the kth column as bjk.

If the row vector a = b @ b , where b is of dimension M, then define the vector a to be partitioned as

U = [bobo, bob,, . . . , bob,_l;blbo, b ib , , . . . , blbM-1; . . .]

so that the k2th element in the klth partition is given by bk,bkz., If u = b @ b ~ . . . @b=b['] then the vector a is recursively partitioned r times with each partition being partitioned M times. Now the kth element in a is labelled by defining the ordered set K = ( k , . . . k,) = ( k : 0 5 k < A I ) such that an element in a is given as

' K = a ( k l . . . k r ) = bk, 'k2 ' ' . bkr (8)

This notation is very useful for the Volterra filter as for example the point in the rth Volterra kernel h,.(k, . . . k,) is given by the element h, in the vector h, defined in eqn. 2.

Define the Fourier transform of the input signal as the column vector 0

0 = Fc,, = ['"I (9) 4 5 - I

Consider the operation FC{" (FH)['] given in eqn. 7. Let

A , = FC{")(FH)['] (10)

B = C{rl(FH)['] (1 1)

bo, = C[;!(FH)[r l (12)

bo, = (co,FN)['] (13)

and

The first row of B is given by

and by applying theorem 1

As c;. is the time reversed form of the input signal ce0, which we define as always real, then by the Fourier trans- form theorem for reversed time signals [14]

0 = Fc,, = (14)

and together with eqn. 13 the following is obtained

bo. = (oT)"' (15)

The Fourier transform theorem for time shifted signals states that for a signal x(n) with a Fourier transform X (e'") ~ 4 1

x(n - i>&te'"ix(&") (16)

Thus, as the ith row of C is a time shifted version of c0. by i places forward, eqn. 16 applied to eqn. 14 gives

C i . ~ H = [eo$$, elti;;, O ~ E ~ , . . . , Q~-,ET-~)~] (17)

where eqn. 13 gives

is the conjugate of w. Substituting eqn. 17 into

-(M-l)i [r] bi. = [e,$$, e ,~ ; , e 2 ~ $ , . . . , 1 (is) IEE Proc.-Vis. Image Signal Process., Vol. 147, No. 2, April 2000

Hence, applying the notation of eqn. 8 to eqn. 18 an element biK of B is given by

b , = e k , % z x @k2$$' x . . . x d k , @ z (19)

(20) - k i = e k , x 8 , x . . . x d k , w r

where k,,, = (k , + k2 + . . . + k,), using modulus M addi- tion. Thus, the Kth column of B is given by

= x . . . x 8, fzsum (21)

where fz,,, is a column from FH. From the definitions given in eqns. 10 and 11

A , = FB (22)

so that using the set notation for the column number the element aiK in A,. is given by

aiK = x o b . K (23) Substituting eqn. 21 into the above gives

aiK = mekl . . . ekrf;.f?T,, (24)

As F is orthonormal

1 if i = j

0 if i # j I J;.fY =

where f$ is the jth column of FH. Therefore the elements of the matrix A , are given by

The equation for the Volterra filter is now expressed as N

y = FHA,.F['Ih, r=O

where F['Ih, is the r-dimensional Fourier transform of h,. The matrix A,. is of dimension M x M and is partitioned, by columns, into M-' parts. From eqn. 25 the matrix A , is sparse as each column is non-zero in only one element. Additionally, note that each element in A , is formed from r - 1 multiplications from points in the one-dimensional Fourier transform of the input signal. Thus, for each order r in the implementation of eqn. 26 there are two one- dimensional Fourier transforms of order M required and r W complex multiplications. This can be compared to rW+I multiplications required for the direct form of the Volterra series given in eqn. 4 so that, assuming the Fourier transforms require comparably little computation, the implementation of eqn. 26 will require less computation than the direct form of eqn. 4.

4 Reduction due to symmetries

There are two symmetries in eqn. 26 which can be used to reduce the compdexity of the implementation as illustrated in Fig. 1 for an example of the matrix A , for M=4. To simplify the diagram the product 4 4 O i e k is replaced by the corresponding subscript numbers. The first symmetry used

111

columns which commute with symmetrical points in the kernel spoce

J

row which corresponds to negative portion of output

Example of matrix A, for M = 2 showing only the non-zero points

spectrum

Fig. 1

is that of each Volterra kernel which can always be represented as a symmetrical space [15]. This symmetry can be expressed using the set notation for a point in the Volterra kernel vector h, or a corresponding row in A represented by the set K = ( k l , . . . , k,). Any points in h, or columns in A which are represented by a set which contains the same elements, in any order, are points in symmetry. An example of the symmetry in a Volterra kernel is that it is always possible to make h2(i, j ) = h,(j, i ) without loss of generality. The symmetrical points in a kernel can be summed and the repetitions removed [ 161, the correspond- ing columns in A , can then also be removed. This changes the dimension of h, from M to

which gives a significant reduction for r > 1. A second reduction in the number of points can be

achieved due to the fact that the Fourier transform of a real sequence is ‘conjugate-symmetric [ 141. This implies that (for even A4) only MI2 + 1 of the rows of A,. are required to construct the output y giving approximately a saving of one half for large M.

The implementation of the rth-order stage of the efficient Volterra filter in eqn. 26 is carried out using two ‘real-valued’ one-dimensional Fourier transforms and an additional

Y M + r - 1 -( 2 1 complex multiplications. If the degree of the filter M is a power of two then the aforementioned ‘real-valued’ one- dimensional Fourier transforms can each be performed using a fast Fourier transform (FFT) in 2M log2M real multiplications [ 171. Thus, the number of operations (NOP) for a block of M samples using eqn. 26, with M a power of two and taking into account symmetries, is

N (M+:- ’) (27) NOP = 4M log, A4 + 2r

r= 1

The above assumes that a complex multiplication is performed using four real multiplications and a multiply accumulate is carried out in one operation.

5 Implementation using overlap and save techniques

The efficient implementation of the Volterra filter can be extended from a circulant input matrix to a generalised input signal by the overlap-and-save technique [ 14, 181. First the .memory of the filter, originally M, is doubled maintaining all the original points and setting any addi-

112

... ...

... ...

1 ,

non-shaded oreos are removed

Fig. 2 Block structure of the overlap and save technique

tional points for time samples greater than M - 1 to zero. The input matrix size must also be doubled by including the previous A4 samples and these 2M samples are the elements of the column ce0 in eqn. 7. By doubling the memory of the operation of eqn. 7 the effect of wrapping round the input signal for the last M samples is removed. Thus, although the first M samples of the block are not a valid output for the continuous time signal the last M samples are. By overlapping blocks by M samples and ‘saving’ only the last M samples the output signal is generated. The process is shown diagrammatically in Fig. 2. For linear systems the overlap and save technique has an alternative method termed ‘overlap and add’ which uses the superposition property of linear systems. As the super- position property does not apply to nonlinear systems the overlap-and-add technique is not applicable to the techni- que in this paper.

The implementation of the overlap-and-save technique increases the NOP for a block of size M ( M a power of two) to

NOP = 8M log, 2M + 2r ( 2 M + ; - 1 ) (28) r= 1

6 Comparison of direct and frequency domain methods

The frequency domain technique developed in this paper will now be compared to the direct form of the Volterra filter given in eqn. 1 and another frequency domain technique developed by Im and Powers [ 111. As all three methods are mathematically equivalent the comparison is only made on the basis of computational complexity.

To compare the direct form and the frequency domain methods a second- and third-order filter section will be considered separately; the linear case, which is a first-order Volterra system, has already been widely reported and investigated [14]. The direct form of an rth-order Volterra filter which makes use of the kernel symmetry requires Y M

Y+J- ‘> operations for an input sample block of M samples. For the frequency domain techniques it is assumed that the multi- dimensional Fourier transform of the kernel spaces required for eqn. 26 has been performed as this can be computed in advance of the filtering operation. Addition- ally, for the frequency domain technique, M is set to the next largest power of two thus enabling the use of FFT techniques using block processing. The number of opera- tions for the direct form and the frequency domain tech- nique per input sample are given in Table 1 for a second- and third-order Volterra filter. The Table also gives the

IEE Proc.-Vis. Image Signal Process., Vol. 147, No. 2. April 2000

Table 1: Number of operations per output sample for M length second- and third-order sections

Filter memory Number of operations per sample

M Direct Frequency domain Frequency domain no use of using symmetry symmetry [l I ]

Second-order section

2

4

8 16

32

64

128

256

6

20

72

272

1056

41 60

1651 2

65792

68

131

258

51 3 1024

2048

4096

81 92

32

47

78

141

268

524

1036

2060

Third-order section

2

4

8

16

32

64

128

256

12 388 60 1539

360 61 46

2448 24577

17952 98304

137280 39321 6

1 . 0 7 ~ 1 0 ~ 1 . 5 7 ~ 1 0 ~

8.48 x lo6 6.29 x l o6

106

249

728

2455

8982

34326

1341 66

530454

number of operations for a method by Im and Powers [l I ] which uses the frequency domain but has not utilised symmetries. Note that the use of symmetries gives a significant reduction in the number of operations: for the third-order case with M 2 16 it gives a factor of ten or more improvement over the other frequency domain method.

To test the efficiency of the technique the direct form and the frequency domain form using symmetry reduc- tions were implemented on a workstation in C++ running under a UNIX environment. The processor time for each method was estimated using the time() function and the result was scaled to give the equivalent number of opera- tions per sample. The results for a second- and third-order filter section are shown in Figs 3a and 36, respectively thus demonstrating that the theoretical computational requirement accurately predicts the operation count of the practical implementation. Figs. 3a and 3b show the effect of imposing a block structure for the frequency domain techniques which utilise FFTs. The consequence of the block structure is that the NOP for the frequency domain techniques only changes as A4 equals a power of two whereas the direct form increases with each increase in M.

7 Conclusions

An efficient algorithm for performing the Volterra filter using a frequency domain representation is presented. The method is developed from utilising the Kronecker product of matrices to manipulate the multidimensional Volterra filter equation. By transforming the input matrix of the Volterra filter to the frequency domain a sparse matrix is obtained. The transformed input matrix can be obtained solely from the one-dimensional Fourier transform of the

IEE Proc.-Vis. Image Signal Process.. Vol. 147, No. 2, April 2000

, ~ . ~ ,_ .,-\_,_ "'"'"'-'*Im.- '.-.,.-..*-a-..

O M I I I I

0 20 40 60 80 100 120 memory M

a

10 -

8-

6 -

1-

2 - /f . . , . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .-- _-.C_-.-. - -.

0 I I I

0 20 LO 60 80 100 120 memory M

b

Fig. 3 domain forms of second- and third-order sections ~ direct calculated . . . . . frequency calculated _ _ _ direct measured - .- . frequency measured a Second-order sections b Third-order sections

Number of operations per sample for the direct and jkequency

input signal so that multidimensional transforms of the input are not required.

The kernels of the Volterra filter and the Fourier trans- form of a real-time signal exhibit symmetries which can be exploited to further reduce the computational complexity of the technique. To demonstrate the efficiency of the techni- que it is compared with the direct time domain form and another frequency domain technique which does not exploit the symmetry properties. The comparison shows that using the symmetry properties gives approximately fourfold improvement for the second-order filter and tenfold for the third-order filter over existing frequency domain tech- niques for filter memories of 16 samples or above. Applica- tions for this efficient filter include simulation of a nonlinear system and real-time nonlinear correction.

References 1

SCHETZEN, M.: 'The Volterra and Wiener theories of nonlinear systems' (Wiley, New York, 1980) SHI, Y., and HECOX, K.E.: 'Nonlinear system identification by m-pulse sequences: Application to brainstem auditory evoked responses', IEEE Trans. Biomed. Eng., 1991, 38, (9), pp. 834845 PARKER, G.A., and MOORE, E.L.: 'Practical nonlinear system identi- fication using a modified Volterra series approach', Automatica, 1982,

WINTERSTEIN, S.R., UDE, T.C., and MARTHINSEN, T.: 'Volterra model of ocean structures -extreme and fatigue reliability', 1 Eng. Mech., A X E , 1994, 120, (6), pp. 1369-1385 KAIZER, A.J.M.: 'Modeling of the nonlinear response of an electro- dynamic loudspeaker by a Volterra series expansion', J . Audio Eng. Soc., 1987, 35, (6), pp. 421432

18, (I), pp. 85-91

113

6 REED, M.J., and HAWKSFORD, M.O.J.: ‘Nonlinear error correction of horn transducers using a Volterra filter’, 1 Audio Eng. Soc., 1997, 45, (5 ) , pp. 414, (abstract only), presented at the 102nd Convention of the Audio Eng. Soc., Munich March 1997, preprint 4468

7 SANDBERG, I.W.: ‘Uniform approximation with doubly finite Volterra series’, JEEE Trans. Signal Process, 1992, 40, (6) , pp. 1438-1441

8 REED, M.J., and HAWKSFORD, M.O.: ‘Practical modelling of nonlinear audio systems using the Volterra series’, J. Audio Eng. Soc., 1996, 44, (7/8), pp. 649-650 (abstract only), presented at the 100th Convention of the Audio Eng. Soc., Copenhagen May 1996, preprint 4264 KLIPPEL, W.: ‘The mirror filter -a new basis for reducing nonlinear distortion and equalizing response in woofer systems’, 1 Audio Eng. Soc., 1992, 40, (9), pp. 675-691

IO MORHAC, M.: ‘A fast algorithm of nonlinear Volterra filtering’, IEEE Truns. Signal Process., 1991, 39, ( IO) , pp. 2353-2356

1 1 IM, S., and POWERS, E.J.: ‘A fast method of discrete third-order

9

Volterra filtering’, JEEE Trans. Signal Process., 1996, 44, (9), pp, 2195-2208

12 BREWER, J.W.: ‘Kronecker products and matrix calculus in system theory’, IEEE Trans. Circuits Syst., 1978, CAS-25, (9), pp. 772-781

13 STEEB, W.-H.: ‘Kronecker product of matrices and applications’ (BI, Wissenschaftsverlag, 1991)

14 OPPENHEIM, A.V, and SCHAFER, R.W.: ‘Discrete-time signal processing’ (Prentice Hall, 1989)

15 EYKHOFF, P.: ‘System identification’ (Wiley, 1974) 16 REED, M.J., and HAWKSFORD, M.O.J.: ‘Identification of discrete

Volterra series using maximum length sequences’, IEE Proc., Circuits Devices Syst., 1996, 143, (5), pp. 241-248

17 SORENSEN, H.V., JONES, D.L., HEIDEMAN, M.T., and BURRUS, C.S.: ‘Real-valued fast Fourier transform algorithms’, IEEE Truns. Acoust. Speech Signal Process, 1987, ASSP-35, (6), pp. 849-863

18 IFEACHOR, E.C., and JERVIS, B.W.: ‘Digital signal processing, a practical approach’ (Addison-Wesley, 1993)

114 IEE Proc.-Vis. Image Signal Process,, &l. 147, No. 2, April ZOO0