ON THE COGNITIVE FUNCTION OF DETERMINISTIC CHAOS IN NEURA

GIANFRANCO BASTI and ANTONIO PERRONE Pontifical Gregorian University - Piazza della Pilotta, 4 - I 00187 - Rome (Ita1

Abstract. Recently has been produced experimental evidence in neurophysiology suggesting that the different features of the external stimuli, processed in parallel along different pathways on the spatial dimension, are integrated dynamically on the temporal dimension. For this task, the deterministic chaos, experimentally found in the os- cillatory behavior of nerve cell mays of the sensory cortex, has an essential role that is not yet clear from the theoretical standpoint. In this work we propose a first approach to this problem. By the study ofH.Sompolinsky’s theoreticalmodelof aneural net, that implements in a dynamical Hopfield net chaotic behavior, we show some proper- ties of a chaotic net with respect to more classical models, such as Rosenblatt perceptron,Hopfieldnet, Boltzmann Machine. In the same time, we enhance a theoretical research line that links all these ap- proach, that is, a satisfying solution of theinducrionproblem in neural networks. In this connection, the chaotic models, show an intrinsic superiority with respect to linear and stochastic models, in perform- ing inductive task as to complex problems for which no analytical, reductive strategy is possible. Finally, we suggest a first step toward the conhuction of a learning procedure founded on chaotic dynamics, showing some relevant properties of it, with respect to computation- al irreducibility in physical modelling, and set theory in logic.

I. JNTRODUCTION

There is a growing evidence of the presence of deterministic chaos in neural dynamics of mammals, strictly related to the integration function of the input during perceptual task [ 1-31. On the other hand, from the standpoint of a theoretical study of chaos, very interesting acognitive, properties of it are even evident. Particularly, a first demonstration of an inductive power of a chaotic dynamics [4] and the well known selective power of it (for its extreme sensibility to the initial conditions), are very interesting for us. Their interest emerges

if we compare these properties with classical perceptrons, of which learning principle is associated with a linear dynamics, more or less enriched, like in the so-called multi-layer perceptrons, by local non-linearities [5]. On the other hand:

1 ) the systematic lossof isomorphism with theinitial conditions, typi- cal of a chaotic dynamics, that makes so difficult to imagine how it could be used to implement representational tasks;

2) the presence of essentially linear nets for the feature extraction from the stimulus in different levels of the sensory cortex [3] [6-81,

suggest that must exist in mammalian sensory systems some ad- vantageous synthesis of these two so complementary, dynamical ex- pressions (the linear and the chaotic) of deterministic equations.

The hue problem at issue becomes thus to find a strategy to control the chaos in neural networks, in view of integrating the systematic deficiencies of the classical associative and/or recursive nets (both founded on the supposition of a resulting global linearity of the nets) [9], granted that a chaotic behavior can be expressed, in given condi- tions, either by a stochastic net [lo], or by a truly dynamic Hopfield net [ 111. In other terns, the essential problem is, in this light, to find

st way for interfacing a chaotic dynamics with a classical,

stochastic or linear dynamics of a neural

presence of chaos in neural ng perceptual tasks, we

spatial and on the temporal dimens same net. In a fol- lowing Section, we examine more we follow the same reasoning that led to insert a Boltzmanian statis- tics in a stochastic net [15-161, a first evidence of the superiority of a chaotic dynamics with respect to a stochastic one emerges naturally

data, hiding a determinis 1. After a presentation of

vantages that this proc computable problems to W.V.0 Quine’s the

II. NEUROPHYSI

According to the cl the integration task of of the sensory system,

the lines from the input U

tically calculates the summation of its weighted input. The absence of these ccgrandmother cells, in higher levels of sensory cortex [8], before all in visual system, rejects a priori such an interpretation. On the contrary, the anatomical and neurophysiological evidence that the different submodalities (form, color, movement) of the visual patterns are not only detected, but also processed in spatially separated areas also at the higher levels of se favors the supposition that the global perception could bution of time [7]. Indeed, the rich interconnections a1 g the different processing lines could be thought to ac trol device for such a cor- relation.

ral network approach,

1-657

Of course, the temporal coding by simultaneous activation of spa- tially distant neuronal pools has been the first, more natural supposi- tion advanced by neurophysiologists [ 141. On the contrary, W.Singer more recently demonstrated that the synchronizationof the oscillatory responses of the columnar cell assemblies in the striate cortex of cat (i.e., at the level of feature detection of the stimuli) seems to be the main physical device by which the visual systems codes the amount of coherency inherent in the external stimuli that gave rise to the responses [3]. In otherterms, the establishment of ctconceptuab maps, encoding for ccfeature constellations>> in physical reality, actually coexists, on the time dimension, with the retinotopically organized maps wich represent the topological relations among features on the space dimension.Thus, it is not required to lose retinotopy in order to decode coherencies among parts of a figure which are non-connected on the topographic map. So, for instance, also at distance as large as 7 mm, when the receptive fields are no longer overlapping, cell clusters tend to synchronize in phase their oscillatory responses only when they have similar orientation preferences and are activated by stimuli that have the same orientations and are moving in the same direction.

Moreover, just as in the formation of topographic maps there is a competition among different possible pathways according to a modified Hebbian rule [6], so also in the formation of the conceptual maps in the feature space there could be a competition among many possible constellations of features, that show some coherency in the input. Only the most consistent and frequent constellations win, so that these maps furnish the sensory system also with essential infor- mation about the statistical probability with which these constella- tions occur in the physical world.

But the most intriguing property of such a distributed oscillatory behavior of cell assemblies is that they engage also in an observed, chaotic dynamics. In other terms, the oscillatory behavior of cell as- semblies in visual cortex has an highly non-linear base. More precise- ly, the oscillatory behavior of the columns is chaotic during theresting phase. After that the stimulus has arrived, the dynamics switches into a limit cycle, that is in an ctordered* oscillation with a given period. Though Singer’s discoveries concern directly the striate cortex (i.e. the famous area 17 of the visual cortex), the oscillatory behavior is a general property of the visual cortex. In such a manner, is well founded the expectation that the above remembered function of the cell assemblies as forced oscillators, for the detection of coherencies in the feature space, is a general property of the visual system.

This evidence from the visual system fits thus very well with an analogue evidence produced by W.Freeman in the study of olfactory system in mammals [l-21. The first interpretation of such an evidence in both these authors is that a chaotic dynamics grant an immediate, bias free access to the limit cycle attractors, each representing a pat- tern of the respective sensory stimulation. In other words, a neural network, that implements fruitfully a chaotic dynamics, never ends up into Buridan’s donkey situations as an Hopfield net or a linear per- ceptron, also when it must select between very similar situations. Nevertheless, such aproperty of maximal selectivity, not only can be shared, at least in principle, by a truly stochastic net that uses for clas- sification tasks ctnoise-like, keys (performed, for instance, by D.Gabor’s convolution technique) [19] [9], but overall this property is without any practical value for the networker, till it is not clarified how a chaotic dynamics compress and process information.

TO conclude this section about the evidence of chaos in neurophysiology of sensation, we would mention also

V.Mountcastle’s recent formulation of a sort of general law in sen- sory systems [8]. According to this daw,, emerging directly from the study of the tactile system, as long as we climb up the levels of the sensory cortex, we are faced with an inverse relation between the de- gree of conservation of the isomorphism with the initial conditions of the external stimuli, and the degree of selective power of the same outputs, with respect to main characteristics of the external stimuli. Though Mountcastle does not put in relation such an evidence to the role of non-linear and chaotic dynamics within the net for the reconstrunction task of the external pattern after its segmentation at the lower levels of sensory cortex, this evidence fits very well with this role.

m. HOPFED NETS, BOLTZMA” MACHINE, CHAOTIC DYNAMICS THREESTEF!SlDWARDALEAR”GPROCEDURE

FOR COMPLEX SYSTEMS

A . The Use of Non-Linearity in the Hopfield Model and in B o l m n n Machine

When we reflect upon Singer’s discoveries about the detection of ccconceptual maps, of the features by the dynamical means of the synchronous oscillations in columnar assemblies of neural cells, we find that these further constraints on visual information processing correspond to the <tweak>> constraints in the classical optimization problems faced by the Boltzmann Machine [16]. These problems con- sist essentially in finding some dynamic procedure that enables a neural network device to make hypotheses. That is, a procedure to en- code and to weigh complex properties (Singer’s feature constella- tions) of the external input that the net is not designed to detect. The essential difference between the visual system and a Boltzmann Machine seems thus to be that: 1) in a Boltzmann Machine the task of extracting feature complexions

(i.e., the hidden constraints) from the input processed at lower levels is executed by hidden units, exclusively devoted to this task, by a statistic strategy that essentially computes a global average of the input;

2) in sensory system the inductive task of feature complexions is per- formed by a mly dynamic strategy of the same units extracting the features by their oscillatory behavior. In the Boltzmann Machine, the statistic strategy to perform induc-

tive tasks consists in associating acost function to be minimized, with an optimization problem. In linear programming there is a sharp dis- tinction between the constraints always to satisfy and the cost to min- imize. In this sense, all the constraints are ccstrong>> constraints. Here, on the contrary, we need etweak, constraints, of which satisfaction depends on the context, so that the distinction between constraints and costs becomes no longer absolute. The optimal solution is then to min- imize the total constraint violation. It has been demonstrated that, under given conditions, this minimization can be implemented as a relaxation search in a parallel network [20-211. Now, to grant that few units be fully active and the rest be inactive, it is necessary to use a strongly non-linear decision rule. But in such a manner the final out- come is strongly dependent on the initial conditions of the net, so that the convergence process for the same input leads to different states on different occasions.

Hopfield’s expedient is to assume that the problem to solve is en- coded by the initial state of the net [21]. In such a manner, each local minimum is significant because it a priori represents an already defined item. The problem becomes thus to find the local minimum

1-658

closest to the initial state of a upotentiab (Lyapunov) fu

ost function becomes here

whereTij , being Tii = 0, is the synaptic efficacy which couples the output of the (presynaptic) jth neuron to the input of the (postsynap- tic) i* neuron; Vi, Ii are the state and the threshold of the ith neuron. This function exists only for Tij = qi.

In such a manner, the net is performing here the optimization of a well-defined function. Supposed that we wish to store a set of states Vs, s = l...n, the learning rule of the model is:

As we see, the dynamical process is not inducing anything here, but its function is only to search for the best fitting between the stimulus and the learned pattern. In this sense, the classical Hopfield net is equivalent to aclassical multi-perceptron [5] , with the only difference that we are using here a non-linear optimization strategy to calculate the weights of the units. Indeed, in this net just as in linear multi-per- ceptrons, the functions must be well-defined, bringing to a fixed-point behavior only when there are not correlations in the set of the stored items. On the contrary, in the Boltzmann Machine, the only significant

minimum is a global minimum. But, although the <<energy>> function is the same of the Hopfield model, here the target of the dynamics is not to minimize the ccenergyn, but to minimize the difference G be- tween the environmental and the free-running global probability dis-

presenting different internal states of the system, G is uenergy n according to the Boltzmanian distribution:

where Pa is the probability of being in the U* global state, and Ea is the energy associated to that state.

The essential feature of the Boltzmann Machine is that the set of its hidden units is never clamped by the environment and can be used to uexplain, hidden constraints in the ensemble of the input units that cannot be represented by painvise constraint. For G we have thus the following formula:

where P(Vd is the probability of the ath state of the input units when their states are determined by the environment and P’(Va) is the prob- ability when the network is running freely with no environmental input.

To minimize G it would be necessary to calculate its gradient de- scent and hence to know the partial derivative of G with respect to each individual weight, given that P’(Vd depends on the changes of the weights. But at thermal equilibrium the principles of classical statistical mechanics lead us to agreat simplification. Indeed, because, in an uniform probability distribution, each event is equivalent to minimize G, it is sufficient to observepij (i.e., the average probability of two units both being in the on state when the environment is clamp- ing the states of the input units) andp’ij (i.e., the correspondent prob-

I-

ability when the net is running freely change each weight by an amount tween these two probabilities:

Awij ~ ( P i j - p i j )

where E scales the size of the weight change.

we see that a true integration of the features is performed only by the former, because the latter has only already structured items. Never- theless, the Boltzmann Machine has an essential limitation. The sup position that every ccatomn of the system is alike, implies the purely local character of the wei ge formula that depends only on the behavior of the two uni weight connects. The global mean- ing of this change depen n an artificial supposition of equi- probability of all the dynamic justice of the complexity of the dynamics.

ing of this model by their ccann cal optimization problem, ther noninterchangeable elements that make unlikely a regular solution founded on the analogy with the cooling of an ideal fluid [22]. But it is precisely at this point that emerges the superiority

B. The inductive ability of chaos

to extract from a

So, if we compare the Boltzman

Indeed, also Kirkpatrick and his colleagues

umber sequence such as the fol-

From the plot of this data in Fi suppose that these data obey a tion of x = 0.5310. But, by sup

a

relationship, that is the sequ a deterministic rule, a one-dimensional map is produced by the pro-

the observed value of 0.4736.

-659

n xn (xn, xn + 1)

1 2 3 4 5 6 7 8 9

10 11 12 13

N N

0.5858 0.8284 0.3431 0.6863 0.6274 0.7452 0.5097 0.9807 0.0386 0.0773 0.1546 0.3091 0.6182

Tab. 1

(0.5858,0.8284) (0.8284,0.3431) (0.3431,0.6863) (0.6863,0.6274) (0.6274,0.7452) (0.7452,0.5097) (0.5097,0.9807) (0.9807,0.0386) (0.0386,0.0773) (0.0773,0.1546) (0.1546,0.3091) (0.3091,0.6182) (0.6182,0.7632)

where Jjj is the synaptic efficacy which couples the output of the (presynaptic) j" neuron to the input ofthe (postsynaptic) i" neuron, and Jii = 0. In electrical terms, Eqs. 0) are Kirchoff equations, in which the left-hand side represents the current leakage due to the membrane capacitance; the first term in the right-hand side represents the current passing through the membrane resistance and the last term represents the input current flowing to the cell from the activity of the other cells. Moreover, the microscopic time is assumed here as unitary and the network is considered with random synaptic couplings. Each of the Jjj's is a random variable, and it is assumed for convenience that it has a Gaussian distribution. The mean of Jjj is 0 whereas the variance is [J2jj]j= J2/N. With this normalization, the intensive parameter (i.e., the control parameter that switches the dynamics from low energy attractors to high energy ones and viceversa), in the case of (6), is the demensionless gain parameter gJ.

The long-time propemes of the solutions of Eqs. (7) for large N, studied by the method of the dynamical main-field theory, originally developed for spin-glasses [24], has exact solutions in the limitN +-. The essential result of this study is that the dynamics of the system at long times can be reduced to a self-consistent equation of a single neuron, which reads:

xn

Fig.2

independent of dynamical rules. Indeed, while the harmonic analysis method has no predictive ability, unless the periodicity is already known, a procedure of the same type of the precedent (the so called aeverse procedure,) has the ability of discovering periodicity and of predicting the future of the system. Finally, in the case of nonperiodic number sequences without dynamical rule, can be demonstrated that the chaotic dynamics has at least the same ability of the statistical methods.

C. Toward a Learning Procedure Founded on Chaotic Dynamics H. Sompolinsky and his Colleagues [ 111 recently proposed a model

of modified Hopfield net to implement in it a chaotic dynamics. They studied a continuous-time dynamic model of a network of N non- linear oscillators interacting via random asymmetric couplings. The model, similar to that studied by S.I.Amari [23], is constituted by N neurons (Si ( t ) ) , i = 1, ...,N, where Si are continuous variables such that -1 I Si I 1. Associated to each neuron there is a local field hi, - - - <hi < + -. It is defined through the relationship Si (t) = $(hi(t)), where $(x) is a non linear gain function which defines the input (hi)- output (Si) characteristics of the neurons. In biological terms, hi is re- lated to the membrane potential of the neuron and Si to its activity (firing rate). The function $(x) is assumed to have a sigmoid form $(fi -) = fi 1, $(- x ) = - $(x). For concreteness, the authors choose the function:

$(x) = tanh@) (6)

where the consrant g > 0 measures the degree of non-linearity of the response. The dynamics of the network is given by N coupled fmt- order differential equations:

h.i ( t ) = - hi (t) +qi ( t )

The temqiis a time-dependent Gaussian field generated by the ran- dom inputs from the other neurons, i.e., the last term in &. (?). of course, the mean of qi is 0.

The essential information of a chaotic net such as the net studied in Sompolinsky model, is hidden in the local-field autocorrelation A(Z).

(8)

A(z) = ( hi @hi ( r + 7 ) ) (9) Indeed, the flow is chaotic, if the correlations between two points along the flow decay.

So, if for gJ < 1 the behavior of the system is characterized by the attractor of zero fixed point, for gJ > 1, the system falls in a chaotic attructor. Particularly, with N 2100, as gJ is increased above unity, the system pass through different limit cycle attractors, till for gJ >2, the system falls in a chaotic attractor. This experimental evidence, o b tained by Sompolinsky and his colleagues, fits very well with the theoretical, classical D.Ruelle's hypothesis [25] that the unstable pe- riodic statesof suchsystemsaretheequivalentofthesetoflimitcycles in which the chaotic attractor may be resolved. But, this evidence fits overall with the neurophysiological finding [ 1-31 that identifies the stimulus aniving into the net with the modification of the control parameter of the unstable dynamics.

There is another fundamental correspondence between the physiological and the evidence obtained from the study of theoretical models of chaos in neural nets. The chaotic dynamics, and generally the oscillatory behavior of the net (i.e., the study of the global net be- havior from the standpoint of the time dimension) does not affect the average distribution of weights within the net. This implication, a b solutely undesired from the standpoint of the classical neural network approach to cognitive problems, (exclusively interested to the processing of the input on the space dimension), is on the contrary the essential key principle to grant efficacy to a system that, probab- ly like the mammalian brain, executes on the time dimension the progressive integration of the features (i.e., the induction of theircom-

1-660

plexions) extracted from the input on the space dimension by the same cells. To say the same thing in W.Singer's terms [3] (see above Sect. 11). this property of oscillatory nets is the theoretical foundation of the possibility of the simultaneous coexistence on the same net of the ccconceptualn and cctopographicaln map of the input at all the levels of the input processing. So, the very same elements of the same set that individually, on the space dimension, are feature detectors, as a whole, on the time dimension, may be detectors of the internal coherency of the input that they are processing in their microstate. The oscillatory behavior of the net becomes thus the dynamical counterpart of the hidden units of the Boltzmann Machine, without sharing its essential limitations: the Boltzmann statistics is not valid here [26-271 (see above Subsect. 111 A.). Just like the landscape of the hidden units after a complete cycle of the Boltzmann Machine makes explicit, by different local minima, the hidden contraints of the input [15-161, the chaotic behavior of this model compress in a dynamical

and ccuttersn it by its oscil- latory response [121[141.

es thus the interpretation of such a language. Of course, the complete decoding of this language implies a satisfying statistical and informational study of determinis- tic chaos, that is far from being reached today.

Nevertheless, a strong means to extract essential information from the chaotic dynamics of the net is already available.

We know that the essential property of the Sompolinsky model is that, for long times, the dynamics of the system may be reduced to a self-consistent equation of a single neuron (see Eq. (8)). So, by map- ping, for instance, according to the two dimensions (hi(t)) and (hi(t + z)), different values of the global field of Eq. (8) during the chaotic phase, if there exists some dynamical rule hidden in these values. the mapping of the chaotic dynamics is able to make it ex- plicit to us, withoutknowing formally thatrule. Tharis withoutknow- h g the equation relative to this rule (see above Subsect. 111 B.). Actually, we are investigating the properties of chaos in neural net- works according to these main lines.

IV. CONCLUSIONS FROM THE *COMPUTATIONAL GEOMETRY,, THE &MPUTATIONAL DYNAMICSr OF PERCEFTRONS

Final Conparkon with Boltzrnann Machine and Classical Per- cepmn. The Question of Computability.

Reading these results in the light of neurophysiol ifwe remember that:

1) the chaotic dynamics is elicited by

2) the final new limit-cycle attractor net and

havior) is the macroscopic result on the time dimension of the inter- nal mkroprocessing of it on the space

then: 3) W.Singer supposition that the regular

self an expression of an internal cccoherence,, found by the system in the input itself (see above Sect. 11), seems to be reasonable also from the theoretical standpoint. Indeed, this (coherence>> is not granted by aformal equalion in

the mind of the human observer (or of the human designer in the case of artificial neural nets), but by a dynamical rule within the physical Process that very often cannot be ever expressed by a formal equa- tion.

SO, to conclude our comparison between theoretical and biological wdels of Oscillatory nets, we can go anoth

the po wer of such an approach to neural ne each set of these neuronal oscillators (each column of cortical cells, in mammalian brain and/or each Sompolinsky net, in the theoretical

regions of the receptive served here, the system is here inducing higher order cccoherencies,

Indeed, just as a Boltzmann Machine, cconstructs>> ad hoc its higher

higher order feature detectors din

ful as much as non-he thermodynamics in self-organization tasks

on the time dimension.

namics is more powerful than linear

So, synthetically, we see:

HIGHER ORDER FEATURE DETECTORS Hidden units Oscillatory behavior

Free-running Chaotic phase

Zero point attractor t cycle attractor

ann Machine shows n T a few above the

net does not share wi stochastic device, bu

ann Machine, neither with any other th the linear perceptron. Indeed, just

the expression of the result of the determinis e complex dynamics elicited by its input. The hus the designer of its own ccperceptron>,, without reducing arbitrarily, like in classical statis- tic optimization, the complexity of the problem.

The impossibility of equation of the determinis- tic rule hidden in a comp and hence the impossibility of designing the relative cl tron, is the consequence of the

cal physics, this notion upposition of a close cor- respondence between p and computations, any

1-66 1

towad the solution of a series of computations executed by the same physical processes.

There exist, however, physical processes for which such a shortcut does not exist. A trivial example is a computer intended as a physical device. It can detennine the outcome of its own evolution only by ex- plicitly following it through. No predictive shortcut is here possible. Such a computational irreducibility occurs every time a physical process acts as a computer: the simulation or the observation of the process is the only way to find the behavior of the system. No formal (i.e., general) predictive shortcut is possible here.

The very intriguing consequence of a learning procedure founded on the mapping of chaos is that may exist a dynamical predictive shortcut for irreducible computational problems. A shortcut that ob- viously cannot have the universality of the formal modelling, because it is constructed by the nature itself to be applied to predict the out- come ofparticular physical processes. Perhaps, the secret of the so- called ugenetic coden is related to this general property of the strongly non-linear character of the thermodynamical substratum of 1ife.We return on the logic that may be applied to this inductive property of chaotic net (see below Subsect. IV B.). In any case, this capacity of a chaotic dynamics is not a umagicn property: it is only founded on the exceptional capacity of this dynamics of compressing information rw. B. A Logical Consequence: rowani an Implementatin of WVO.Qui-

ne's logic of r v i d classesw By our approach it is thus possible to overcome the essential

criticism against neural network approach from the standpoint of the usymbolicn paradigm of AI [30]. The core of this criticism is indeed that a set of connections, constructed by the only associative principle of the classical neural networks, cannot be considered a cognitive usymbOl* because it cannot exhibit in principle any intrinsic logical uconstituencyn or ucoherencyn.

Therefore, the representational unodesn of a connection pattern cannot be considered proper parts of the representational set. Con- sider, for instance, the set costituted by the three nodes ( A m ) , (A), (B). To make this nodes xepresentative of the inference from to d n , it is necessary, according to symbolic AI, the existence of the rule prescribing that, whenever a token of the form @&Qn appears, the system produces a token of the form dJn.

In other terms, though the rules may be uhardwhdn in the machine, logical constituency implies always an ultimate explicit definition of the rype smrures. But if we grant such a condition in a neural net- work, practically this machine becomes only a particular implemen- ration of the classical symbolic paradigm.

Indeed, it is not sufficient the simple property of having the logi- calrules cchardwiredn in the physical structure. rather than represented in a software. to affm that the neural network constitutes a new pmdigm of AI !.

Another criticism against neural networks from the standpoint of Classical AI is that, owing to their ccassociationist>> character, they cannot implement cognitive inrenrwnaliry. Indeed, also the intention- al reference to a content of a given symbol can be resolved in its logi- cal constituency, according to the symbolic paradigm. The content of a umoleculm statement is thus regarded as a function of its ccatomic, constituents, according to the logical rule that constitutes it. Recent- ly H.Pumam strongly criticized such a formalistic approach to inten- tionality, vindicating a more realistic study of it [17]. Surely, futher development of neural network approach may give an essential con- tribution to this study program [27].

On the other hand, the pmblem of individuating the pmper logic of the neural networks is one of the most urgent for this approach. In this connection it is important to remember that W.V.O. Quine [31] defined, by his atvirtual theory, of classes, uweakn conditions of sethood, particularly suitable for our aims. This theory indeed defines cccontextualn conditions of set membership without supposing infinite objects (such as logical classes explicitly defined) on the right side of the membership notation c c ~ n but only symbols or ccschematan (i.e., incomplete definitions) for them [31]. Moreover, these schemata can be considered as formally equivdenr to the function for the progres- sive abstraction of the relative set. The strict relation with the induc- tive strategy of chaos founded on purely dynamical rule is evident.

Of course, there are strong limitations for such a set theory. It is really impossible to justify in it the logical quantification as well as the laws of the excluded middle. In Quine's terms, the class a m such that [Q) (x E y) ] does not exist here. In this case, there exist only sets or ufiniten objects, in the sense that each set can become part of another set. The condition of class closure with respect to the succes- sor cannot be granted here: no ceiling can be posed to the succession of induced sets. Nevertheless, this theory justifies the use of all the fundamental arithmetical and logical operations.

The relations between Quine's virtual theory in logic and the purt- ly dynamical inductive pmpemes of chaos justify, also from a logi- cal standpoint, the intuitive evidence (bound to the non-linear irreducible character of the dynamics concerned) that also a dynami- cal rule may grant some sethood conditions. These conditions, in- deed, are essential to found cognitive pretension to the models (see above Subsct EI D.). Finally, the evidence that this rule is anyhow imposed to the system by the same physical reality and not by the mind of the designer, gives a particularly strong sense to the state- ment that, in such a manner, we are really studying the qerceptionm in its typical upre-logicaln character.

So, as a conclusion of these reflexions, we can say that the true problem to solve for neural new& approach, in its plttension of constituting a new paradigm of AI, is not the construction of a com- putufional geomeny, to quote the subheading of classical Minsky- Papert book. This ucomputational geometryn has been very useful to indicate in which sense the neural network does not constitute such a paradigm. The true challenge that neural network approach must ac- cept to demonstrate its absolute originality is the construction of a computational dyMmics that, of course without absurde reductive pretensions, may enlighten the same foundations of the formal logic. without depending passively on it.

REFERENCES

[l] W. J. Freeman, "Simulation of Chaotic EEG Patterns with a Dynamic Model of the Olfactory System", Bwl. Cybern., vol. 56,

[2] Ch. Skarda and W. J. Freeman, "How Brains Make Chaos in Order to Make Sense of the World, Behavioral and Brain Sciences, vol.

[3] W. Singer, "Self-organization in Cognitive Systems", in:"The Principles of Design and Operation of the Brain". Proceedings of the study week organized by the Pontifical Academy of Sciences, Vatican City, October. 19-24,1988, J. C. Eccles and 0. Creutzfeldt Eds., Berlin-Heidelberg-New York, 1989 (in press).

[4] T. Oi, "Chaos Dynamics Executes Inductive Inference", Biol. Cybern., vol. 57, pp. 47-56.1987.

pp. 139-150,1987.

IO. pp. 161-195, 1987.

1-662

, Perceptrons. An Intro on to omputational Geometry. tion. Cambridge-Mass.,

Organizing Neural Net- work Sharing Features of the Mammalian Visual System", Biol. Cybern., vol. 55,333-343, 1987.

[7] G. Baumgartner, "Where do Visual Signals Become a Percep- tion?", in:"The Principles ofDesign and Operation of the Brain". Proceedings of the study week organized by the Pontifical Academy of Sciences, Vatican City, October, 19-24, 1988, J. C. Eccles and 0. Creutzfeldt Eds., Berlin-Heidelberg-New York, 1989 (in press).

[8] V. B. Mountcastle, "The Construction of Reality", in:"The Prin- ciples of Design and Operation of the Brain". Proceedings of the study week organized by the Pontifical Academy of Sciences, Vatican City, October, 19-24,1988, J. c. Eccles andO. Creutzfeldt Eds.. Berlin-Heidelberg-New York, 1989 (in press).

[9] T. Kohonen. Self-organization and Associative Memory. Second Edition. Berlin-Heidelberg-New York, 1988.

[ 101 P. Peretto and J. J. Niez, "Stochastic Dynamics of Neural Net- tems, Man, and Cybernetics,

[1 Sommers, "Chaos in Random Neural Networks", Phys. Rev. Len., vol. 61, pp. 259-262,1988.

[12] R. Shaw, "Strange Attractors, Chaotic Behavior, and Informa- tion Flow", ZNaturforsch., vol. 36a, pp.80-112, 1981.

[ 131 P. Grassberger, "Estimating the Fractal Dimensions and Entropies of Strange Attractors", in A. V. Holden (Ed.), Chaos. Princeton NJ, 1986, pp. 291-312.

[ 141 H. Haken, "Information Compression in Biological Systems", Biol. Cybern., vol. 56, pp.11-17, 1987.

[15] D. H. Ackley, G. E. Hinton, T. J. Sejnowski, "A Learning Al- gorithm for Boltzmann Machines", Cognitive Science, vol. 9,

. E. Hinton and T. J. Sejnowski, "Learning and Relearning in oltmann Machines", in "Parallel Distributed Processing. Ex- lorations in the Microsmarre of Cognition", J. L. McClelland d D. E. Rumelhart Eds., Cambridge Mass.-London, 1986, pp.

H. Putnam, Representation andReality. Cambridge Mass.-Lon- on, 1988. C. G. Phillips, S. Zeki, H. B. Barlow,"Localization of Functions

in the Cerebral Cortex: Past, Present, Future", Brain, vol. 107,

[ 191 S. Bottini, "An After-Shannon Measure of the Storage Capacity n Associative Noise-Like Coding Memory",Biol. Cybern., vol.

. E. Hinton, "Relaxation and its Role in Vision", Unpublished toral dissertation, University of Edimburgh. Describedin D. H.

allard and C. M. Brown (Eds.), Computer Vision, Englewood Cliffs, NJ, 1977, pp.408-430.

[21] J. J. Hopfield, "Neural Networks and Physical Systems with Emergent Collective Computational Abilities", in Proceedings of the National Academy of Sciences, USA, 1982, pp. 2554-2558.

[221 S. Kirkpamck, C. D. Gelatt Jr., M. P. Vecchi, "Optimization by Simulated Annealing", Science, vol. 220, pp. 67 1-680, 1983.

[23] S . I. Amari, IEEE Trans. Syst. Man Cybern., vol. 2, pp. 643s,

[24] A. Crisanti and H. Sompolins

[6] H. Fmhn, H. Geiger, W. S

[28] S. Wolfram, Physics", Phys. Rev. Len., vol. 54, pp. 735-738,1985.

tzmann Group, "Connec- in Ch. A. S k d a and W.

J. Freeman, "How Brains Make Chaos in Order to Make Sense of the World", BehavioralandBrain Science, vol. 10, p. 181,1987.

onism and Cognitive Ar- chitecture: A Critical A

London, 1969.

[30] J.A.Fodor-Z.W.Pylyshyn, "

[31] W.V.O.Quine, The

~p.147-169.1985.

282-317.

pp.327-361.1984.

pp.15 1- 159, 1988.

1972.

1-663