Adaptive neural-network-based approach for the control of continuously stirred tank reactor

Y.Y. Yang D.A. Linkens

lndeainy terms: Control systems, Adaptioe control, Artificicial neural networks

Abstract: An online adaptive neural-network- based controller (OANNC) is developed in the paper. The detailed design procedures of the OANNC are given, along with an illustrative example of controlling a continuously stirred tank reactor. Simulation results show that the OANNC is successful in controlling nonlinear time-varying systems with slow dynamics. Compared with conventional neural-network controllers, the OANNC has the following advantages. First, it is capable of controlling nonlinear systems with time-varying parameters, which i s not usually the case for a nonadaptive neural-network controller. Secondly, the selection of the initial training data set is trivial due to the online adaptive training ability of the neural network. Normally, for a con- ventional neural-network structure, the selection of an initial training data set is crucial, with the requirement that the training data set should be persistently exciting, which is quite difficult in many practical situations.

1 Introduction

The modelling and control of bioreactor processes form a challenging problem for both control engineers and theorists for several reasons. Although the task does not involve many variables, its highly nonlinear properties and time-varying characteristics make it difficult t o control. For example, even a small change in some parameter value can cause the bioreactor to be unstable. Significant delays existing between system input and output add t o the diffculty. Moreover, in an industrial environment, the lack of reliable online sensors for various process parameters and states makes the infor- mation available for control inadequate. Many control schemes based on feedback technology become imposs- ible because of the lack of such feedback signals.

In recent years, there have been various control schemes developed for bioreactors, including those based on optimal control [l, 21, adaptive control [3], artificial intelligence [4, 51, expert systems [6] and neural net-

$; IEE, 1994 Paper 1384D (CY), first received 18th March and in revised form 6th June 1994 Y.Y. Yang is with the Institute of Industrial Control, Zhejiang Uni- versity, Hangzhou. China. He is currently a research fellow at the Uni- versity of Sheffield D.A. Linkens is with the Department of Automatic Control & System Engineering, University of Sheffield, Sheffield S1 4DU. United Kingdom

IEE Proc.-Control Theory Appl., Vol. 141, No. 5 , September IYY4

works [7]. However, research in this field is still a t an early stage, and it seems that much is required before a well-developed control scheme can be derived for pract- ical bioreactors with a satisfactory performance.

Artificial neural networks (ANNs) are very attractive for bioreactor control owing to their ability to learn the nonlinear input/output relationships from a given train- ing data set. However, there are also difficulties when an ANN is employed to tackle the bioreactor control problem. The first diffculty is how to generate the train- ing data set for the neural network. It has been pointed out [SI that the training data set must be sufficiently exciting if the underlying properties of the system are to be learned properly. For a nonlinear system, this means that the training data should span the whole operating range, and all the dynamic characteristics should be revealed. This is often very difficult or even unrealistic in many industrial environments. If a training data set that is only partially exciting is used to train a neural network, the results may be satisfactory only within the data covered by the training data set. When such a neural network is used beyond the area spanned by the training data set, poor results may arise because, gener- ally speaking, the neural network is usually weak in making extrapolations (i.e. generalisation capability).

Recently, the modelling and control of nonlinear systems based on artificial neural networks have received much attention, and there are already numerous pub- lications existing in the literature [9, lo]. The unique characteristics of neural networks, in particular their nonlinear-mapping and learning ability, suggest that they may be useful in many control systems. Here we shall describe briefly the basic ideas about the feedforward neural network.

There is a wide variety of artificial neural networks existing in the literature, of which the feedforward neural network is the one most commonly used in modelling and control. Feedforward neural networks, such as the multilayer perceptron, consist usually of many simple processing elements arranged in layers, as shown in Fig. I . Each element takes its input from the weighted sum of the outputs of the elements of the previous layer. This input is then passed through a nonlinear function, often called the activation function, to form the element’s output. Training of the neural network involves adjusting the weighting matrix by using one of the learning algo- rithms, e.g. back propagation, so that the network emu- lates the nonlinear function underlying the training data

This research is partially supported by the Bao Yu-Kong scholarship.

34 I

. set. It is this nonlinear-mapping property of neural net- works that is of vital importance to their application in the modelling and control of nonlinear systems.

Fig. 1 Three-layer feedforward neural network 0 Input neuron E2 bias unit 0 outpul neuron E? hidden neuron

-- - weight connection ~~~ - biasconnection .

The ability of an ANN to represent an arbitrary non- linear relationship has led to the idea of using networks directly in a model-based control strategy. This involves the training of the ANN so that the network emulates the input/output mapping and the corresponding inverse relationship hidden behind the given training data sets. These trained ANNs can then be used efficiently to form different control schemes. Some commonly used control schemes are direct inverse control, internal model control and feedforward feedback control, as shown in Fig. 2.

cont rol ler

neural-netmxk model

b

neural-network

C

Fig. 2 LI Neural-network direct inverse control h Neural-network internal model control c Neural-nelwork feedlorwardlfeedback control

Typical neural-network control structures

For further details of these control schemes, see Refer- ences 9 and 10.

In this paper, an online adaptive neural-network- based controller (OANNC) is developed based on the multilayer feedforward neural network, and the control

342

of a specific bioreactor, a continuously stirred tank reactor (CSTR), is studied under the control scheme developed. One of the advantages of using an OANNC is that it can track the time-varying properties of the con- trolled system to provide efficient control under circum- stances where the system parameters are changing. Besides, because the weighting matric6s of the neural network are continuously updated according to the most recent process states, it poses no restriction on the excita- tion condition of the initial trainingdata set.

2 Online adaptive neural-network-based

In most industrial systems, their characteristics change with time owing to components wearing out, the environ- ment changing etc. For such kinds of system, an adapta- tion mechanism is helpful to achieve a satisfactory control performance in all circumstances. In this Section, we propose an online adaptive neural-network-based control scheme that is suitable for nonlinear systems with slowly time-varying characteristics. To control a time- varying nonliner system, it seems essential that the con- troller involved should possess the ability to change its own behaviour according to the parameter and/or struc- tural variations occurring in the system. An OANNC, depicted in Fig. 3, has been devised for such purposes.

controller

. -

online learning algor i thrn

Fig. 3 Structureof OANNC F = Filtering

The OANNC is composed primarily of three main com- ponents: a control neural network to produce the suit- able control action, a model neural network to emulate the system behaviour, and the associated online learning algorithms to adjust the weighting matrices of the neural networks according to the available system information. The filter is added. to control the tracking properties and robustness of the control system.

2.1 Model neural network A three-layer feedforward neural network was adopted for modelling the system to be controlled. Let us denote the system output as y,,(t), which is an m-dimensional output vector, and the system input as u(t), which is an r-dimensional input vector. Then, we can use the samples of these input and output vectors to form the data set for training the model neural network. As the system is assumed to be not only nonlinear, but also time-varying, we need the facility of adaptability for the resulting neural-network model. In this paper, a moving-time- zone, fixed-length training data set is used to achieve adaptive training of the neural-network model. This

I E E Proc.-Control Theory Appl., Vol. 141, No. 5 , September I994

moving-time-zone, fixed-length training data set a t time t can be constructed using the following equation:

k ( t ) = {(x,(t - I,), Y d - I,)), (x,(t - I , + 11, Yrnd(t Irn + , , . 1 ( x m ( t ) % Y m d ( t ) ) }

where x,(t) is the input vector of the model neural network at time t ; y,At) is the desired output vector of the model neural network a t time t ; and I, is the fixed data length of the training data set D,(t).

For the model neural network, x,(t), ymd(t) can be formed by

x,(t) = [ U T ( t ) , ur(t - I ) , ..., U T ( t - TdJ,

Y m d ( t ) = Y ~ U ( ~ ) (2) where Tu and GY are the time delay orders to be con- sidered for the system input and output, respectively; and y,(t) is the output vector of the model neural network, with appropriate dimension.

Theoretically, we can determine Tu and Gy if we known the order of the transfer function of the equivalent linearlised system: Tdu is equal to the order of the numer- ator, and Ty is equal to the order of the denominator. In practice, it is often hardly possible to obtain the order information of the equivalent transfer function for a non- linear system, and some heuristic methods are required to settle the prameters and 7&. One suggestion is to select a small value for and Ky at the beginning and then gradually to increase their values until a satisfactory neural network is obtained based on the training data set governed by and Tdy .

Now, the aim of neural-network training at time t is to find the best value of the weighting matrix Wml2( t ) and Wmz3(t), such that the following performance index (model mismatch error) is minimised:

1 I".

E,(t)

subject to

5 1 ll~,At ~ k ) - Ym(t - k)1I2 (3) k = l

Y,(O = wlz,(t)z,(t)

z,(d =.fCWl1z(t)xm(t)I (4) where Wmlz( t ) is the weighting matrix between the input layer and the hidden layer, with appropriate dimensions; W,,,(t) is the weighting matrix between the hidden layer and the output layer, with appropriate dimension; z,(t) is the output vector of the hidden layer; superscript T denotes the transpose of a matrix or a vector; a n d j i s the vector activation function of the hidden neurons, where, in this paper, the following sigmoidal vector function is used :

where x = [xl, xz, . . . , x J r is an n-dimensional vector. For a specified time instant t , the above problem

reduces to a normal neural-network training problem, and many existing training algorithms could be used to determine the weighting matrices. In this paper, the revised back-propagation algorithm [ 131 is employed, and the weighting matrix updating process for the fixed time t is given by the following equation:

i = l , 2 j = i + l (6)

I t E Proc.-Control Theory Appl., Vol. 141, No. 5, September IYY4

where Vwm, Ek( t ) denotes the gradient of Ek(t ) with respect to Fkkijct); Ek(t) is the model mismatch error at the kth iteration defined by eqn. 3, when the weighting matrix Wiil(t) is used; and a is a learning constant.

Starting from the initial weighting matrix WiiJ{f), which is set randomly, this training procedure is carried out iteratively until the model mismatch error satisfies E:([) < E, where E is a prespecified error-tolerance level. To prevent infinite iteration for the training, a maximum iteration number K,,, is set. If k > K,,, and the model mismatch error E:(t) is still too large, the training pro- cedure is ended with the latest obtained weighting matrix W : y ( t ) as the one for the model neural network. The learned weighting matrix is denoted as W:i,(t), i.e.

W:(t) = W;(t) iff E;([) i E

W:(t) = W:-x(t) otherwise

w:(t) = C ~ k W r Wk;3(t)lr (7) As time evolves, the system's characteristics and the environment may change, so that the learning algorithm is kept active to modify the weighting matrix of the model neural network such that the model follows the changes occurring in the system. The adaptive weighting matrix updating process is carried out in the following way. Suppose that the latest updated weighting matrix for the model neural network is W:(t,). Now, for time t = t > t , , we first use Wz(tk) as the current weighting matrix to calculate the model output according to eqn. 4, and then the performance index, eqn. 3, is checked. If the model mismatch error is smaller than the specified level, then the assumption W:(t') = W:(tk) is validated, and no significant change has happened to the system. We advance t for the next time instant and perform the same calculation repeatedly as time advances. Suppose that, at time t = t", the resulting error index E,(t") is greater than the specified level when the original weighting matrix W:(tk) was used. This means that, at time instant t", some significant changes have happened to the system, and the weighting matrix needs to be updated to cope with the changes. Hence, we first form the updated training data set D,(t"), according to eqn. 1 , and then retrain the model neural network with this updated training data set, based on the method described above. After the training is fin- ished, we obtain the updated weighting matrix W:(t"). To improve the system robustness and to reduce the effects of noise and process disturbances, the following smoothed adaptive algorithm is used to obtain the final weighting matrix for the time t > t":

w:(tk + 1 ) = am w:(tk) + ( - %m) w:(t") (8) (9)

n + t k + l =

where a, is the adaptation rate; and t"' is the time instant immediately after time t".

The design parameters I,, a and a, are important for the training and adaptation process of the model neural network and need to be selected with care. Generally speaking, the fixed-length I, of the training data set should be selected such that the data contained in this time zone are rich enough to train the neural network. In other words, the data contained in this fixed zone should be representative of the process dynamics of the system. If 1, is too large, there is an increasing risk of the neural network being overtrained, and, if I, is too small, the neural network will probably be undertrained. It is obvious that the selection of I, is dependent on the char- acteristics of the system. For example, for a slow process,

343

I

1, should be selected long enough for the resulting time zone to encompass the typical behaviour cycle of the process, whereas, for a fast process, I , can be shorter, supposing that the sampling interval is the same. There is no existing rule for the determination of I,, and experi- ments are necessary to find a suitable value for each indi- vidual problem. Based on simulation studies, one suggestion is that I, should be proportional to the primary time constant of the system. The adaptation rate a, is also crucial to the final performance of the model neural network. If a, is increased, the ability to track the system parameter variation will be reduced. If E , is decreased, it will become more sensitive to the system noise and other disturbances contained in the training data set. This means that small a, will drive the neural network to represent any disturbance occurring in the system, which is, of course, not desirable. Therefore the selection of a, is a compromise between the tracking ability and robustness of the neural network. The learn- ing constant a mainly affects the training speed and con- vergence behaviour of the neural network; the larger the a, the faster the training process. However, when 1 is increased, there is an increasing risk that the training algorithm will not converge.

2.2 Control neural network Because the control neural network involves the inverse system model, attention should be paid to obtaining this inverse model. Hunt 1111 has discussed two ways of inverse modelling via neural networks, i.e. direct inverse modelling and indirect inverse modelling (also called specialise inverse modelling). In this paper, we select the latter for implementing the control neural network, as shown in Fig. 4.

~ ~ ~ ~ ~ & p v online learning

I inverse + forward model I

I model I I I

~ auxiliary control network I I

i ~ ~ _ _ _ _ _ - - _ _ _ _ _ _ ~ - ~ - - - - - J

Specialrsed inverse modelling .structure Fig. 4

The control network receives the set point y S p as its input, and its output U is then sent to the system and the model neural network simultaneously. Considering the control neural network, the desired output, which is the control action U, cannot be defined explicitly. Hence, it is very difficult to form the training data set for the control neural network alone. To avoid this difficulty, we con- struct an auxiliary control network, which is a com- bination of the control neural network and the model neural network, as shown in Fig. 4 by the dotted rect- angle. From the viewpoint of the auxiliary control network, the network output y , is identical to the output of the model neural network y , , and the desired network output ycd is equal to the current process setpoint y.,. Although the outputs of the auxiliary control network and the model neural network are identical, the desired outputs for these two networks are not the same. For the

344

model neural network, the desired output is y p o . This dif- ference represents the different roles played by the model neural network and the control neural network. The error signal for training the auxiliary control network in this case is the difference between the desired system output ysp and the output from the model neural network y , , which is an approximation to the $stem output y p o . It is clear that the aim of the training process for the auxiliary control network is to reduce the control error as much as possible by adapting the associated weighting matrices of the control neural network. In an efficient training process, the control error will eventually approach zero as the training process continues, i.e.

lim EJtJ = lim ( ~ ~ , , ( t ) ~ u,(N t -cc ,- -z

= lim (Y,,(t) - y,(t)) = 0 (10) I'm

Now let us consider the training process of the auxiliary control network in more detail. In the auxiliary control network, where the connected control neural network and the model neural network are viewed as a single neural network, the backpropagation algorithm can be readily extended to carry out the associated training process. The training data set can be obtained by sam- pling the input signal to the control neural network alone, as the desired output is identical to this input signal. The training data set for the auxiliary control network is then given by

D c ( t ) = {(xc(t - IC), ycd(f - IC))? (xc(f ~ f

y& - I, + l )b . . , > ( x c ( t ) 3 Y c d ( t ) ) }

xAr) = Y C d ( t ) = Y,,(t) (1 1) where x,(t) is the input vector of the control neural network at time t; ycd(r) is the desired output vector of the auxiliary control network at time t ; and I , is the fixed data length of the training data set D,(r).

The response of the auxiliary control network to its input x,(t) can be acquired by the forward calculation of the control neural network and the model neural network sequentially. The only difference in training this neural network appears during the back-propagation process. Here we consider the auxiliary control network as a partially determined network, with the weighting matrices relating to the model neural network being fixed. 'Thus, in the back-propagation process, only the weighting matrices relating to the control neural network are modified based on the steepest-descent method, the model neural network at this stage acting only as a bridge to back propagate the error signal. The training process can be represented by the following nonlinear optimisation problem:

subject to

= WT23 f [ W T I Z ( t ) x c ( b ) l ~

x,(t) = C U T ( t ) , Y 3 t - 113'

Y A ~ ) = y m ( t ) = WIz3 /CwI~~(t)xm(tJl 113) where y,(t) is the output of the auxiliary control network, W,,, is the weighting matrix between the input layer and the hidden layer of the control neural network; W,,, is the weighting matrix between the hidden layer and the output layer of the control neural network; and f ( x ) is the sigmoidal vector function defined by eqn. 5.

I E E Proc -Control Theory A p p l , Vol 141, No 5 , September lY94

The training process of the auxiliary control network is very similar to that of the model network, and the back-propagation-based learning algorithm can be rep- resented by the following equation:

w:;'(t) = wtij(t) - rv,, E:(t) i = l , 2 j = i + l (14)

where eij(t) is the weighting matrix between the ith layer and the j th layer of the control network in the kth iteration, with i = 1 for input layer, i, j = 2 for the hidden layer, and j = 3 for the output layer; V,<,, Et( t ) denotes the gradient of Et( t ) with respect to Cij(t); and a is the learning constant.

The final weighting matrices for the control neural network, similar to the derivation of those for the model neural network, are given by

w3tk+J = x , w:(t,j + (1 - a,)w:(t")

wm = c w:r* (t), w23 (01 (15) where a, is the adaptation rate, and other symbols have a similar interpretation to those in the case of the model neural network.

The principles for selecting the design parameters a, a,, 1, etc. are similar to the corresponding parameters contained in the model neural network. However, the final selection fo these design parameters, i.e. network topology, fixed data length, adaptation rate etc., are often based on a trial-and-error approach guided by process heuristics.

In this paper, we are interested in time-varying nonlin- ear systems. Usually, in an adaptive control system, if both the system model and the controller are adaptively changing with time, we should ensure that the adaptation rate of the controller is faster than that of the model. This requirement can be met by selecting I , to be smaller than I,. In practice, we often first set 1, = I, meaning that we are only concerned with the error resulting from the current control u(t). It seems nonsense that, a t time t , we should calculate I, (with I , > 1) control sequences {u(t), u(t ~ I ) , ..., u(t - Lc)} for most physical systems, as only the current control u(tj will be applied to the system, and other previous control actions (u(t - l), u(t - 2), . . .) are already fixed. It is unnecessary for 1, to be greater than 1 unless predictive control is to be considered. Please note that selecting 1, = 1 does not mean u(t - I ) , u(t - 2), . . . , have no effect on the control error E,(t) . Their effect on the control error is projected by the model neural network in a fixed way as the values of u(t - I), u(t - 2), . . . , are already fixed and cannot be altered by the control neural network at time t .

The OANNC algorithm can be implemented through the following iterative processes. First, the model neural network is obtained after the training process, and the control neural network is trained through the specialised inverse learning scheme. Then the control is calculated by the forward processing of the control neural network for the given set point. Later, this control is sent to both the model neural network and the system itself, simultan- eously. A one-step-ahead prediction of the system output can be obtained from the model neural network and is compared with the system output to produce the error signal, which will be used for the training of the model neural network. This procedure is repeated as time prog- resses.

Fig. 5 shows the flowchart of the OANNC algorithm. The OANNC algorithm begins with an initial neural- network-model training process, which is identical to

I E E Proc.-Control Theory Appl.. Vol. 141, N o . 5, September 1YY4

standard back propagation. This step is crucial when the initial control performance is important, as, in our control scheme, the control neural network is trained

for NM and NC

I initial neural model

training

set initial condition

one-step-ahead setpoint

fixed-length data

fixed-length data NM adaptive learning

network forward colcuhtlon model prediction Y,(k.l)

?-I process response Ypv(k.l)

fixed -length data updote k <-- k.1

Fig. 5 Flowchart oJOANNC

under the assumption that we already have a fairly accur- ate system model. Unlike the adaptive learning periods when we can only obtain the training data from the controlled-system input and output signals, in the initial training period we can design some experiments to form a well-constructed training data set with desirable properties. Based on this well-composed training data set, it is reasonable to think that a better initial system model can be obtained, resulting in an improved initial control performance.

345

We can also omit the initial neural-network training procedure in the OANNC in the following way. At the beginning of training, we gather the training data only until the length of the training data set accumulated is equal to the fixed length we have selected for the model neural network. Then the training of the model neural network and the control network is activated. However, the control neural network is not switched online before the training procedure has been stabilised and the system has been adjusted properly for closed loop control. After the training is stabilised and the system is ready for online control, we switch the control neural network from backup mode to online control mode. The above idea comes from typical process control, where the control is usually manual initially and after necessary adjustments is switched into online control.

Another point to be noted is that the error signal for training the control neural network is the difference between the output of the model neural network and the corresponding set point, rather than that between the system output and its set point. Obviously, the final purpose of the control is to drive the output of the system, rather than the output of the model neural network, to its desired set point. The reason for selecting the model output is that the system output is usually contaminated with process noise and measurement noise, resulting in a more difficult training process for the neural networks. The use of the model output also has its weak points. For example, if an inexact system model is used, the control performance will be degraded, because the controller is the inverse of the system model, rather than the inverse of the system itself. Keeping the system model as accurate as possible is one of the main tasks of the online adaptive-learning algorithm for the model neural network.

It should be pointed out that, in our control scheme, the online adaptive training of the control neural network is much more difficult than that of the model neural network, although much research has indicated that the multilayer feedforward neural network has the capability to approximate any continuous nonlinear function with arbitrary accuracy, provided that enough hidden layers and hidden neurons exist [14, 151. Let us look in more detail at the training process of the control neural network. Compared with the model neural network, which is a three-layer feedforward network, the auxiliary control network is now a six-layer feedforward neural network. This complicated network structure will naturally decrease the training efficiency. especially when the back-propagation algorithm is used. Moreover, the auxiliary control network is composed of two cascade neural networks, and in the training phase the model neural network, which is close to the output of the aux- iliary control network, is fixed, and only the weighting matrix of the control neural network, which is far from the auxiliary-network output, is able to be changed. Often, the effect of changing weights far from the network output will be alleviated owing to the filtering action of a series of nonlinear activation functions. Furthermore, the number of output elements in the control neural network is often not equal to (usually less than) the number of input elements in the model neural network, resulting in a partially connected neural network. Thus, in the aux- iliary control network there exists an independent part that cannot be affected directly by the weighting matrix adjusting scheme during training of the control neural network. If it is this independent part that is not consist- ent with the desired network, it will be very difficult for

346

the control neural network to make correcting actions. All of these factors make the training process of the control neural network much more difficult than that of the model neural network.

In the following Section, we present an illustrative example of the control of a CSTR by the method devel- oped so far.

3

Bioreaction is the main part in any biochemcal process, whereby microbial fungal mammalian or plant cells are employed for the economical manufacturing of a wide range of biological products. Generally speaking, the aim of a fermentation process is to increase the concentration of the biomass to obtain the maximum concentration of the final product through a suitable feeding of substrate and oxygen. The most important bioreactor for industrial application is the traditional mixing vessel, which has the dual advantages of low capital cost and low operating cost. According to the characteristics of the operation, bioreactors can be divided into three main categories, i.e. batch, semibatch and continuous bioreactor. In this paper, a continuously stirred tank reactor (CSTR) is con- sidered, as shown in Fig. 6.

Case study: control of a CSTR process

- - - - - - -

I”-”.-

L - - - - - - -

Fo 50

Fig. 6 Flow diayrom ofa simple C S T R (hioreactor)

In a CSTR, the reactor is continuously fed with sub- strate influent. The rate of outflow is always equal to that of the inflow, so that the volume of the medium in the tank reactor remains constant. We also assume that the medium in the reactor is well stirred, so that all the differ- ent component concentrations are uniform within the reactor. Considering a simplified fermentation process, a CSRT model can be derived based on the material balance equation and the rate equation, as follows [13]:

d x dt _ - - px + D(x, - x)

d S - dt = F; - (KIP + k,)x + D(Si - s)

x(0) = xo

$0) = so

where x and s are the cell concentration and substrate concentration of the broth, respectively; xi and si are the

I E E Pruc.-Confrol Theor ) Appl . , Vol. 141. No. 5 , September 1Y94

cell concentration and substrate concentration in the inlet Row, respectively; p is the specific growth rate of cell; K , is the yield coefficient of the substrate consump- tion by the biomass; k , is the specific maintenance rate of substrate; F , is the inlet flowrate to the tank reactor; D = Fi /V is the dilution rate; and V is the medium volume in the tank.

A full description of the specific growth rate is related to the concentration of various components in the bio- reactor broth and other environmental variables, such as pH, temperature, dissolved-oxygen concentration etc. In this paper, the following enhanced 'Monod' model is adopted to calculate the specific growth rate:

where p,(T) is the maximum specific growth rate; K,(T) is the saturation constant; and T is the temperature of the broth.

From eqns. 16 and 17, we can see clearly that the CSTR is a nonlinear time-varying process and that it is difficult to control because of the complicated character- istics involved. Here, the OANNC scheme is employed to control the CSTR process, and its control configuration is shown in Fig. 7.

X""

I / . - Fig. 7 Block diagram u / O A N N C / u r C S T R control

For the control neural network, the input signals are the desired substrate concentration set point ssp and the desired biomass concentration set point x S p , and the outputs of the controller are selected as the Concentration of the input substrate si and the dilution rate D. A single hidden layer with five neurons is devised for the control network. GY = 1 and = 0 are selected to form the training data set D,(t). This selection is derived from eqn. 16, where only a first-order derivative is involved. Thus the model neural network takes the input from the two outputs of the control network, si and D, along with two additional signals from the model output through a time- delay unit. It has two output elements, representing the estimated concentration of the substrates s( t ) and the biomass x(t), respectively. Similarly to the control network, the model network also contains a single hidden layer with five neurons. The deviation between the outputs of the CSTR and those of the neural network model, denoted as E, , is fed into the learning algorithm for the purpose of training the model neural network, and the deviation between the desired set points and the output of the model neural network, denoted as E,, is used for the adaptive training for the control neural network. The fixed data length, after a series of trial-and- error studies, is selected as I , = 50 and 1, = 1, respect- ively. The learning and adaptation rates are selected as a = 0.2, a, = 0.05, a, = 0.1. The maximum iterative

I E E Proc.-Control Theory A p p l . , Vol. 141, No. 5 , September 1994

number of K,, = 2000 is set to prevent too much iter- ative learning that might exceed the time available for real-time calculation.

For the simulation, normal CSTR model parameters were selected as pmo = 0.4, K,, = 0.4, k , , = 2.727, k,, = 0.1, and the following conditions were set for the initial training of the model neural network:

so = 0.2 xo = 0.2 (18) The real CSTR process was simulated by eqns. 16 and 17, where the time-varying parameters and the process dis- turbances were introduced by perturbations around the normal parameter level.

I t is supposed that no prior information about the CSTR is known to the neural-network controller. The only way for the OANNC t o obtain information about the system is through online learning based on the associ- ated training data set. Various simulations were designed and carried out to analyse and verify the suitability of the OANNC for controlling nonlinear time-varying systems, and some typical results are shown in Figs. 8-10.

~ ( t ) (- -set point)

0 1 I > I . I I I I I 2

0 5 0 100 150 200 250 300 350 400 450 a

:, input substrate ii .._..___ ..._....

-1.01 . I I I I I I 1 J

0 50 100 150 200 250 300 350 400 450 b

Fig. 8 a Oulput responses or CSTR b Control actions

Simulation results I : fixed set points control with disturbances

Fig. 8 shows the control performance when the set points were fixed; this is often the main control mode in the process industry, where the primary aim of the control is to keep the process output at its desired (fixed set point) level under various process and environment disturbances. A step disturbance with 10% amplitude of its normal value (0.5 for substrate concentration and 0.2 for biomass concentration, respectively) was added to the system output between t , = 107 and t , = 274, the response to the disturbance being good. Within a very short transient period, the system output returned to its desired level, and the control required, which is shown in Fig. 86, was also quite good. Similar changes were also introduced into the set points to investigate the tracking

347

ability ofthe OANNC, and the corresponding simulation results are shown in Fig. 9 for both the filtering scheme and the nonfiltering scheme. Again, system outputs are returned to their new set-point values quickly.

0 5

045

- k. .. -

, I , I

50 100 150 200 250 300 350 LOO 450

0

045- e 9 0 4 -

035-

0 3

- +. C

8

L 40 lA0 140 260 2;O '360 350 460 4;O

b

Fig. 9 0 output responses or CSTR

Simulation results 2: set points tracking control

x ( r ) ~ ~ ~ ~ set point real value set point (with filter)

set point

set point (with filter)

. . . .. real value (with filter)

~ real value $11) ~ ~ ~ ~

. . . . . real value (with filter) h Control actions

~~~~ Input substrate: Sin Sin (wnh filter) Dilution rate: D ~~

. . .. D (with tilter)

Consider Figs. 8 and 9 again, where step changes in the set point or step disturbances in the system output are introduced. The response to these step changes seems unsatisfactory in the nonfiltering scheme. At time instants t , = 107 and t , = 274, the absolute value of deviations in process output are significant, and the control actions are excessive in the nonfiltering scheme, although the process outputs return to their desired values in a very short time. Such a response may cause serious difficulties in a practical system. By selecting the filter F (filtering scheme) properly, the control responses can be improved, with a slightly longer transient time but smoother behav- iour. Fig. 9 also shows the simulation results on the set- point tracking, with a first-order filter of F(s) = l/(Ts + l), where T = 10 was employed. Obvi- ously, the response of the filtering scheme is preferred, compared with the nonfiltering one. The filter is also helpful in smoothing the output response and the control action when abrupt process disturbances are introduced.

Simulations have also been carried out to validate the adaptability of OANNC. Various perturbations on the

348

-

CSTR process parameters, e.g. the maximum specific i growth rate U, and the yield coefficient K ~ , were intro- duced. Under these parameter perturbation conditions,

0 55 ---

0 5 v x ( t ) (---setpoint)

025: s( t ) (--- setpoint)

2 5 - C

2 c ,2 s 1 5 - U

0 2 1 - e 2 0 5 - C

- U

0

b

Fig. 10 urements a Output responses 01 CSTR h Control actlOnS

Simulation results 3 : control with noise-contaminated meas-

input substrate > ................- ~ ____ >qL >.\ :L-r

dilution rate

the control behaviours of the OANNC are quite satisfac- tory. It turns out that the OANNC is able to deal with significant parameter fluctuations with reasonable per- formance. Owing to the length limit, these concrete simu- lation results are omitted here. Fig. 10 shows the control result arising from random measurement noise, where the noise-to-signal ratio was set to be 5%. Simulations show that the control performance is acceptable if the noise-to- signal ratio is not too large.

During the period that no adaptive learning is active, the OANNC runs very fast and it contains only the network forward calculations. However, when some sig- nificant changes occur in the system, the adaptive- learning procedure will be activated to train both the model neural network and the control neural network, and this time the computation load is rather heavy. The computation time required is determined by how many iterations are required before the weighting matrices con- verge to their new equilibrium values. This iteration number is related to the types of change occurring in the system, the neural network structure, the learning con- stant, the initial weighting matrices etc. Some discussion on these effects can be found in Reference 13. We can control the longest time allowed for such adaptive learn- ing by selecting a proper maximum iterative number K,,, . According to the simulations carried out in this research, the calculation is not critical for the CSTR system, which is a rather slow process. The average com- putation time for a 500 min simulation is about 29 min, far less than the maximum time allowed.

I E E Proc.-Control Theory Appl., Vol. 141, N o . 5 , September 1994

4 Conclusions

In this paper, an online adaptive neural-network-based controller (OANNC) has been proposed. The basic control structure, together with the design procedure, is described. Some remarks on the design parmeters are also given. The main reason for using an adaptive neural network rather than a nonadaptive one is that many practical industrial systems are not only nonlinear but also time-varying. It seems essential for the controller to adapt itself to cope with the time-varying characteristics of the system to obtain a consistently satisfactory control performance. Further advantages of the OANNC, besides its adaptability, are that it is not sensitive to the initial training data set and is easily implemented. The fixed- length training data sets, which are updated with the latest information available through online measure- ments, are used for training the required neural networks, which represent the most recent system characteristics.

The developed OANNC has been used to control a simulated CSTR, which has the characteristics of time variance and nonlinearlity. Various simulations have been carried out to test the control performance for dis- turbance rejection, set-point tracking, noise- contaminated measurements etc. The results obtained are good. Because no prior information about the controlled system is given to the controller, it is quite reasonable to believe that the OANNC will be able to control other similar nonlinear time-varying processes as well.

It should be pointed out that the OANNC algorithm is sometimes quite computationally expensive, because both the model neural network and the control neural network are adaptively trained online at every time con- stant. Hence, for systems with fast dynamics, if their behaviour changes drastically, this algorithm may not be applicable because of the large amount of computation involved. It seems desirable to seek other training algo- rithms with faster convergence to replace the back- propagation-based algorithm, which is rather slow because of its steepest-descent nature, to accelerate the learning process and to reduce the computation burden involved. However, for slow processes, such as the one

discussed in this paper, OANNC is suitable, owing to the relatively long time available for calculation at every time step.

5 References

1 KAMBHAMPATI, C., THAM, M.T., MONTAGUE. G.A., and MORRIS, A.J. : ‘Optimizing control of fermentation process’, IEE Proc. D, 1992, 139, (l), pp. 60-66

2 OHNO, H., NAKANISHI, E., and TAKAMATSU, T.: ‘Optimal control of semihatch fermentation’, Biotechnol. Bioeng., 1976, 18, pp. 847-864

3 QUEINNEC, I., DAHHOU, B., and MSAAD, M.: ‘On adaptive control of fed-batch fermentation processes’. Europe Control Conf. 91, Grenohle, France, 1991, pp. 1634-1641

4 STEPHANOPOULOS, Ge., and STEPHANOPOULOS, Gr.: ‘Artificial intelligencx in the development and design of biochemical processes’, Tien2 Biotechnol., 1986,4, pp. 241-249

5 STEYER, J.P., QUEINNEC, I.. and SIMOES, D.: ‘Biotech: a real time aoolication of artificial intelligence for fermentation Drocesses’. Preprink IFAC/IFIP/IMACS Int.%ymp. on Artificial inteiligence in real time control, Delft, Netherlands, 1992, pp. 353-358

6 STEYER, J.P., POURCIEL, J.B., SIMOES, D.. and URIBELAR- REA, J.L.: ‘Online biotechnological process control by means of rea-time expert systems’. Europe Control Conf. 91, Grenohle, France, 1991, pp. 1232-1235

7 THIBAULT, 1.. and BREUSEGEM, V.V.: ‘Modelling, prediction and control of fermentation process via neural networks’. Europe Control Conf. 91, Grenohle, France, 1991, pp. 224-229

8 HECHT-NIELSON, R.: ‘Neurocomputing’ (Addison-Wesley Puh- lishing Co., 1990)

9 MILLER, W.T., SUTTON, R.S., and WERBOS, P.J. (Eds): ’Neural networks for control’(M1T Press, 1990)

I O WARWICK, K., IRWIN, G.W., and HUNT, K.J. (Eds.): ‘Neural networks lor control and systems’ (Peter Peregrinus Ltd., 1992)

I I HUNT, K.J., and SBARBARO, D.: ‘Neural network for nonlinear internal model control’, I E E Proc. D, 1991,138, (5). pp. 431-438

12 ICHIKAWA, Y., and SAWA, T.: ‘Neural network application for direct feedback controllers’, IEEE Trans. Neural Netw., 1992, 3, pp. 220-23 I

13 YANG, Y.Y., and LINKENS, D.A.: ‘Modelling of continuous hio- reactors via neural networks’, Trans. Instit. Mens. & Control, 1993, 15, pp. 158-169

14 CYBENKO, G.: ‘Approximation by superpositions of a sigmoid function’, Math. Conrrol. Signals Syst., 1989, 2, pp. 303-314

15 HORNIK, K., STINCHCOMBE, M.. and WHITE, H.: ‘Multistage feedforward networks are universal approximators’, Neural. Nerw., 1989.2, pp. 359-366

IEE Proc.-Control Theory Appl., Vol. 141, N o . 5, September 1994 349