562 IEEEJOURNAL OF SOLID-STATECIRCUITS, VOL. 24, NO. 3, JUNE 1989

Neural Networks for High-StorageContent-Addressable Memory: VLSI

Circuit and Learning AlgorithmMICHEL VERLEYSEN, MEMBER,lEEE,BRUNO SIRLETTI, ANDRE M. VANDEMEULEBROECKE,

ANDPAUL G. A. JESPERS, FELLOW,IEEE

Abstract —Neural networks used as content-addressable memories show

unequaled retrieval and speed capabilities in problems srreh as vision and

pattern recognition. We propose a new implementation of a VLSI fully

interconnected neural network with only two binary memory points persynapse. The small area of single synaptic cells allows implementation of

neural networks with hundreds of neurons. Classical learning algorithms

like the Hebb’s rule show a poor storage capacity, especially in VLSIneural networks where the range of the synapse weights is limited by the

number of memory points contained in each connectiorq we propose a new

algorithm for programming a Hopfield neuraf network as a high-storagecontent-addressable memory. The storage capacity obtained with this

algorithm is very promising for pattern recognition applications.

I. INTRODUCTION

A RTIFICIAL neural networks have been studied forseveral years. These studies have sought to achieve

the fabrication of machines that share some of the abilitiesof the human brain. In 1943, MacCullogh and Pitts [1]introduced a new mathematical model of the brain’s struc-ture. Many researchers tried to use this model in order tobuild machines that could learn from experience.

The new concept has been studied from both a theoreti-cal and an experimental point of view. One of the bestknown achievements was the perception [2]. This multi-stage learning machine showed some interesting features,but Minsky and Papert pointed out some drawbacks andlimitations [3]. At the same time, new interest in the studyof artificial intelligence trapped many researchers as wellas available funds; hence, the study of artificial neural netmodels was almost forsaken. In our opinion, this wasforeseeable. Although artificial intelligence seemed to offervery promising short-term applications, the lack of simula-tion tools and appropriate mathematical background inthe field of neural networks appeared to be a majorobstacle for further developments.

When, in 1982, Hopfield proposed his new simplifiedmodel of a biological neural network [4], computer algo-

Manuscript received September 23, 1988; revised January 12, 1989, M.Verleysen and B. Sirletti were supported by IRSIA, A, M.Vandemeulebroecke was supported by FNRS.

The authors are with the Laboratory of Microelectronics, Universit6Catholique de Louvain, Louvain-la-Neuve, Belgium,

IEEE Log Number 8927704.

rithms and nonlinear systems mathematics were ready tofind interesting applications. Hopfield’s model of a neuralnetwork consists of a massive parallel array of simpleprocessing elements (neurons) connected through a cou-pling network that allows each neuron to be connected tothe others through a set of connections called synapses.The information contained in the neural network is storedin the actual connection weights, and thus, it is distributedwithin the whole system. Every single neuron computes aweighted sum of the values of the other neurons, includingsign discriminations through the synapses [5]. Hence,Hopfield’s model is fully asynchronous: the system is setfirst to its initial state by turning appropriate neurons onor off and it is then left free until it reaches a stablestate [4].

II. ADVANTAGESOFANANALOGVLSIIMPLEMENTATIONFORNEURALNETWORKS

In this section, we will compare the strengths and weak-nesses of several realizations of neural networks: digital aswell as analog VLSI implementations are contrasted topoint out some of the advantages of analog VLSI circuitsfor synthetic neural networks.

A. Continuous Synaptic Weights

First, we will suppose that each synaptic weight can takeany value between two predefine bounds. Several solu-tions have been proposed to realize such networks: special-ized hardware or software to interface with conventionalcomputers is available (HNC, Neural Ware, Neuronics,etc. [6]). Even though this is very attractive for simulationand experimentation tasks, the sequential algorithms pro-grammed on classical Von Neumann machines completelyruin the speed properties of neural networks.

Dedicated processors, both analog and digital, also exist.In analog solutions, synaptic weights can be stored on afloating-gate structure, but slow charge variation on thesecapacitors represents a major drawback. In digital solu-tions, many bits are necessary to keep the accuracy neededfor synaptic coefficients; the area occupied by synapses

0018-9200/89/0600-0562$01 .00 01989 IEEE

VERLEYSENet al.: NEURALNETWORKSFOR CONTENT-ADDRESSABLEMEMORY 563

and neurons rapidly becomes too large to realize neuralnetworks with a great number of neurons.

B. Discrete Synaptic Weights

We now examine the case of a neural network in whichthe synaptic weights can take only a restricted number ofvalues. Although this restriction limits the performances ofneural networks, it allows the realization of much largernetworks, which compensates for other drawbacks. Fur-thermore, at the end of this paper, we will show that it ispossible to find learning rules that are well adapted to thiskind of circuit with almost the same properties as if thesynaptic weights had been continuous. We will supposethat each synaptic weight can take only three differentvalues: +1, O, and – 1.

One of the advantages of digital realizations of neuralnetworks stems from well-established state-of-the-art digi-tal VLSI design tools. A digital VLSI chip can be designedwith available CAD tools; standard cells and libraries canbe used, and digital chips are tolerant to parameter disper-sion and noise. However, as mentioned above, a fullydigital neural network must be synchronous in practice.Moreover, synapses and neurons occupy a large chip areadue to the arithmetic processors. Finally, a large amount ofmemory is necessary to store the weights of synapses aswell as neuron values with an additional area penalty.

The asynchronism of natural neural networks can bekept in analog neural networks. For the same functional-isty, basic cells (neurons and synapses) are much smallerthan those of digital neural networks. In Table I, we showthe approximate synapse density that has been achievedfor some recent VLSI realizations (the synapse density isdefined as the number of synapses per square millimeter).

Although it is very difficult to compare realizations forconnection processes that can differ substantially, one mayconclude from this nonexhaustive table that analog cellsare indeed smaller than their digital counterparts. More-over, one of the unique features of neural nets— they arefault-tolerant— is lost to some extent because digital im-plementations consume a lot of area for an accuracy whichis not always needed.

It seems obvious that the smaller area of analog realiza-tions is an important advantage for the implementation oflarge neural networks. To realize large digital networks,one solution, however, could be to replace spatial complex-ity by temporal complexity (clocks, phases, multiplexors,etc.); but the introduction of synchronism and multiplex-ing in a neural network has a severe drawback on thecomputation speed. In the next section, we will examineanalog asynchronous networks, which are more appropri-ate and much faster than conventional computers forsolving, for instance, optimization and vision problems.Since only three different synaptic weights are required foreach synapse, digital memory points will be preferred tofloating-gate structures; the problems encountered in ana-log memories are consequently suppressed, and the arealoss is not very significant. Another interesting approach

TABLE ISYNAPSEDENSITY

Mumiy [7] 12.5 dsgml (ptdse-ssrtarn) (2II meta12)Blayo [8] digital (2AOletaU)WeinfeId [9] ;.:7 digital (2)Imetai2)Graf [10] 67 analog (2Wmeta12)Verleysen[11] 33 analog (3y metal1)

could be the use of digital EEPROMS, but because suchcircuits need dedicated technological processes, they willnot be studied in this paper.

III. HOPFIELD’SNETWORK

A. Description of the Network

Neural networks can solve a wide variety of problems;they can be used in vision (i.e., pattern recognition), inoptimization (i.e., traveling salesman problem), in CAD(placing, routing), and in any other problem where percep-tion is more important than a huge amount of accuratecomputation [12]. For pattern recognition and associativememories, an excellent trade-off between silicon area andcomputation efficiency is offered by Hopfield’s model.

Hopfield’s network consists of an array of fully inter-connected synapses and neurons connected to each otherthrough programmable connections (Fig. 1). The output ofeach neuron is fed back into the network; each synapsecomputes a new output value according to the output ofthe control neuron and the connection weights stored inthe synapses. The outputs of all the synapses connected tothe same neuron input are then summed; this sum deter-mines the neuron activity and fixes the neuron outputthrough its nonlinear transfer function.

We define:

~ value of neuron i, i.e., its state after the nonlinearactivation function of the neuron has taken place,

xi neuron activity, i.e., the input of the neuron prior tothe activation function,

~, synaptic weight, i.e., the strength of the connectionbetween neurons i and j,

N number of neurons in the network, andI, input of neuron i.

Further, let us assume

The equations of the network are then

with

y=qx,)

where F is the activation function.

564

I II I

I II I

I I II I

i II I

.-— — —

-—— ——1Iout ~ Iout ‘2 I out ~ I ~“t

N

v neuron o synapse

Fig. 1. Hopfield’s network.

If the connection weights ~j are adequately chosen bymeans of some appropriate “learning rule,” the conver-gence process of the network can be predicted. This pro-cess can be described as follows: the values of the neuronsare first set to an initial state; neuron activities are thencomputed via the connection network; and the new neuronvalues are determined by the activation function. Thisprocess is repeated until a stable state is reached.

B. Learning Rules for Hopfield’s Network

Neural networks often are used in associative memories.With adequate connection weights, the final neuron valuescan be associated with initial input data. Of course, thecapacity of the network, i.e., the number of patterns thatcan be memorized into the network as final neuron valuesafter convergence has taken place, is limited and dependson the number of neurons in the network and on thelearning rule used. For example, consider the well-knownHebb’s learning rule. It can be formulated by

where 1s k s p, p is the number of patterns to memorize,~~ is the bit i of pattern k to memorize, and ~~ = 1 or~~ = – 1. This learning rule is very simple: each weight ?Jis increased if bits i and j are equal in pattern k; it Ndecreased in the opposite case. It has been shown that thecapacity of the network is limited to about O.15N arbitrarypatterns [4].

Other learning rules have been proposed, including theprojection rule [13], which seems more adapted to corre-lated patterns; the storage capacity is increased, but ahigher accuracy is needed for the coefficients [14].

IEEE JOURNAL OF SOLID-STATECIRCUITS, vOL. 24, NO. 3, JUNE 1989

=-isyn.j

•1

+syn.k

+D

Ineuron i I

Fig. 2, P-and n-type current sources.

IV. THE VLSI ANALOGNEURALNETWORK

A. Drawbacks of Conventional Implementations

Our goal was to realize an analog neural network with asmuch storage capacity as possible. Because the capacityincreases with the number of neurons, the dimension ofthe network is the most important factor; since the mainpart of the circuit consists of synapses, we have tried toreduce their complexity and their area as much as possible.In digital implementations, the size of the circuit, i.e., thenumber of neurons in the network, has no influence on thesynapse structure: the same design can be used in aten-neuron network as well as in a 500-neuron network.We will see this is not true in analog implementations.

The basic idea underlying the proposed circuit was thateach synapse should be able to source or sink current tothe input line of the neuron to which it is connected,depending on the value ~j~, i.e., the product of thesynapse strength times the output of the neuron to whichthe synapse is connected. One realization, represented inFig. 2, was proposed by Graf [10].

The drawback of this architecture is that one can neverassume the excitatory and inhibitory currents to be exactlythe same; even with adjustments of the size of the p- andn-type transistors, there is always a risk of mismatchingbetween the sourced and sunk currents because of thetechnological mobility differences between the two typesof transistors. Since the function of each neuron is todetect the sign of the weighted sum of the other neuronsvalues, the mismatch between sourced and sunk currents isincreased with larger numbers of synapses. In order tomake a neuron able to discriminate the sign of its inputeven when the difference between excitatory and in-hibitory currents equals only a single synaptic current, thelatter must exceed n times the difference between the p-and n-t ype current sources; this considerably limits thesize of the network that can be implemented with theprinciple shown in Fig. 2.

B. New Circuit Design

The problem can betransistors to sink andthe neuron. To realize

avoided by using the same type ofsource current on the input line ofthis, we used two distinct lines to

VERLEYSENet al.: NEURAL NETWORKSFOR CONTENT-ADDRESSABLEMEMORY 565

out neuron J

H

mem2

T*.

m ~meml

v+ v-

neuron I+ neuron I-

Fig. 3. Synapse

Vdd

A

--

It

v+

——44+

syn J syn j+2

neuron i+

Fig, 5. Feedback loop.

neuron i+ neuron i.

Fig. 4. Neuron.

sum the currents: one for excitatory currents and the otherfor inhibitory currents.

Each synapse is a programmable current source control-ling a differential pair (see Fig. 3). Three connection valuesare allowed in each synapse. If mend =1, current is deliv-ered to one of the 1wo lines with the sign of the connectiondetermined by the product of mem 2 and the output of theneuron to which the synapse is connected. If meml = O, noconnection exists between neurons i and j, and no currentflows to either excitatory or to inhibitory lines.

Depending on the state of the XORfunction, the currentmay be sourced either on the line i + or on the line i –. Inthe neuron, the comparison of the two total currents onthe lines i + and i – must be achieved. This is done bymeans of the current reflector shown in Fig. 4. The cur-rents on the lines are converted into voltages across tran-sistors T3 and T4; these voltages themselves are comparedin the differential input reflector formed by transistorsT5– T9. Because of the two-stage architecture of the neu-ron, the gain may be very large, and the output (out) iseither 5 V if the current in neuron i – is greater than theone in neuron i +, or O V in the opposite case.

With increasing numbers of synapses connected to thesame neuron, voltage drops V + and V – across T3 andT4 tend to increase. Because P’+ and V – are in fact thedrain voltages of transistors T1 and T2 (see Fig. 3),saturation of the synaptic transistors must be ayoided. Afeedback loop was introduced to keep the voltage V* (seeFig. 5) fixed to V,ef.Because no high gain is needed for thefeedback loop, the amplifier shown in Fig. 5 can be verysimple (even a single transistor can be used).

v~ts ( m 0, OIJ-r

I , ! , r

4s\

40

35

30

25

20

15

10

05

1I 1 1 , , , 1 , ? [16 1,

05 10 15 20 25 30 35 40 45

Fig. 6. Neuron output.

C. Experimental Results

Fig. 6 shows the output characteristic of one neuron.For this simulation, 512 active synapses are connected tothe neuron. The X-axis represents the number of synapses(p) connected to the positive input line of the neuron; theothers (512-p ) are connected to the negative input line.The diagram shows that the output of the neuron is alwayssaturated when there is a sufficient difference between thetwo inputs; this voltage can thus be directly fed back tothe synapse inputs through the connectiofi network.

With a smaller difference between the two inputs, theneuron output voltage varies between O and 5 V. Thissituation is incompatible with the synapse structure thatneeds a digital input. A buffer providing a binary outputhas therefore been inserted between the neuron output andthe synapse inputs.

In order to determine the buffer characteristics and themaximum number of synapses allowed in the network, wemeasured the output voltage of the neuron (before thebuffer) with an increasing number of active connectedsynapses. Fig. 7(a) and (b) shows the experimental resultswithout and with the feedback loop, respectively, describedpreviously (for this experience, a single-transistor feedbackloop was used). The X-axis represents the number N ofsynapses connected to the neuron, and the Y-axis repre-sents the neuron output voltage. This analysis has been

566 IEEEJOURNAL OF SOLID-STATECIRCUITS, VOL. 24, NO. 3, JUNE 1989

5.0 . . . . . . .

1.0 ““”””””

O.dN

2 48 16 32 64 128256 512

1!, !,6jl.o: ::::: “::

1::;””””..”.24 8 16 32 64 128256512

N

0 i->i+ ❑ i+>i- (a) ~ i->i+ ❑ i+>i- (b)

Fig. 7. Neuron output: (a) without feedback loop, and (b) with feedback loop

Fig. 8. Synapse.

Fig. 10. Chip

Fig. 9. Neuron,

made under worst-case conditions (i.e., when N/2 – 1synapses are excitatory and N/2 + 1 inhibitory (i – > i +),

or the opposite (i + > z – )). We see in Fig. 7 that the

dynamic output range decreases with the increasing num-

ber of synapses. This is due to the common-mode voltage

of the neuron amplifier: if the two input currents increase

simultaneously, the amplifier gain will decrease, and the

output will no longer be saturated. It seems obvious thatthe difference between the two curves must be large enoughto correctly turn the buffer on or off (a voltage differenceof 1 V is acceptable). With the feedback loop, a largernumber of synapses (more than 500) is obviously possible.

In our circuit, the single synaptic current equals 10 pA;

to achieve this, the synaptic transistor connected to memlis long (W/L = 0.1), and the one connected to mem2 isminimal (W/L = 1.5); such a current is acceptable in aneural network with a restricted number of neurons, asillustrated below. In larger networks, it has to be reducedfor power density reasons. For example, the power dissi-pated by a 128-neuron network with synaptic currentsequal to 1 pA will be about 100 mW; such a circuit can bemade in a 64-mm2 chip with a CMOS 2-pm double-metaltechnology (power density is about 1 mW/mm2).

Speed properties are one of the most interesting featuresof neural networks. Experience showed that a change in asynapse value introduces a change in the correspondentneuron value and is fed back in the synapse in about 30 ns.Practically, it means that it takes about 120–150 ns for a128-neuron network to converge to a stable state.

In order to verify the performances of synaptic currentsand neuron response time, a small test chip was realized,which contained 14 neurons and 196 synapses. In order tomake the circuit fully programmable and to exploit itslearning capability, two memory points were embedded ineach synapse. Figs. 8, 9, and 10, respectively, show amicrograph of a synapse, a neuron, and the complete chip.

VERLEYSEN L?ld.: NEURAL NETWORXS FOR CONTENT-ADDRESSABLEMEMORI 567

In this part of this paper, we will consider an algorithm

that allows programming of the circuit as a content-

addressable memory. Since only three different connection

weights are allowed (O, 1, and – 1), the algorithm is

adapted to this restriction.

V. A LEARNING ALGORITHM FORCONTENT-ADDRESSABLEMEMORIES

Many learning rules have been reported in the literature[4], [13]. They all have respective advantages and draw-

backs, but few seem really suited for VLSI circuits. Indeed,

in such realizations, the number of possible connection

values is limited by the number of memory points con-

tained in each synapse: if one synapse contains two mem-ory points, only three or four connection values will bepermitted (for example, – 1, 0, and 1). Synaptic coeffi-cients computed by a classical learning algorithm like theHebb’s rule have a larger dynamic range (for example, if kpatterns are stored in an N-neuron network using Hebb’srule, each connection can take as much as 2 k + 1 different

values). To adapt these algorithms to VLSI circuits with

2-bit connections, the coefficients are simply truncated.

This causes an important decrease in the storage capacity

of the network.

The object of the second part of this paper is to describe

a new learning rule that allows a good storage capacity ofpatterns with only three different connection weights. Toachieve this goal, the restriction regarding the number ofconnection weights was included in the algorithm, ratherthan truncating the values after computation. It is impor-

tant to note that the number of different connection values(in this case, three) does not rely on the number ofrecorded patterns. (This is not true for the Hebb’s rule,

which requires 2k + 1 possible values per synapse.)We propose a new way to compute the connection

strengths using a linear algebra optimization method(known as the simplex method) in order to maximize thestability of the recorded patterns. Connections betweenneurons in HopfieM’s model are therefore represented by a

(n x n) matrix in which element ~, is the value of the

connection between neuron i and neuron j (our algorithm

allows ~, to be different from ~i). If we make ~. theBoolean state of the ith neuron and @ the threshold value,

the dynamic behavior of the network can be described by

The network reaches a stable state when

In order to program stable states into the network and

appropriate connection strengths, the following procedureis used. Let us suppose we want to store k n-bit patterns in

a n-neuron Hopfield network and consider a single col-umn (number one for instance) to illustrate the algorithm.

0/0

. . . . . . .

50 . . . .

01

01234567

Hamming distancee conv ok % ❑ conv. falae % ● no converg %

Fig. 11. Hebb’s rule.

Let us call 1) ~ i the value of the ith neuron

8910

from the ,jthpattern to me”rnorize (1s is n, 1s js k, KJ = 1or ~~ =– 1); 2) T,l the value of the connection between neurons rand 1 (1 < r < n); and 3)Sl~ = XTrlVr~the input of the firstneuron when the network outputs correspond to the train-ing pattern k.

Assume that each neuron acts as a Boolean thresholdfunction whose output is 1 in the case of a positive input

and – 1 in the other cases (@ thus is set to O). The

problem is to choose the set T,l in order to maximize the

difference between Sl~ and the threshold of the neuron. If

the sign of Sl~ is forced to be the same as the one of Vl~,we obtain the highest possible stability for bit 1 of pattern

k. To ensure the right sign to Sl~, the quantity to maximizeis actually Zl~ = Sl~Vl~. This has to be done simultane-ously for all values of k. The equation to solve by the

simplex method is then

maximize M where M = min ( Zl~ ) for all values of k.

To avoid unbounded solutions (M ~ co), cl is bounded

by the inequalities:

–l<ql<l.’

Practical algorithms to solve such simplex problems are

described in the literature [15].

The linear algebra theory assumes that at least n – kcoefficients will take the maximum values – 1 or +1. In

addition, experience showed that statistically all the othercoefficient values were near – 1, 0, or +1. Hence, this

learning rule is well suited for such VLSI implementationas the one considered in the first part. Simulations showedthat the results obtained with synaptic weights restrictedonly to – 1, 0, and + 1 are practically the same as withcontinuous values.

Let us now compare our results with those of the Hebb’srule. Experiments with 12-bit patterns were carried out inorder to compare the efficiency of both rules to discrimi-nate several patterns. Three target patterns were taught tothe network using both Hebb’s rule and the new rulerespectively. The network was then run with the 212 possi-ble different input patterns. Finally, the output of the

568 IEEEJOURNAL OF SOLID-STATECIRCUITS, VOL. 24, NO. 3, JUNE 1989

REFERENCES

%

. .

i

70. .

\$’

. . . . . .

130 . . . . . . .

501 . . . .Vi

40 . . .

30 . . . [\

A.::::

+ 1

0123456 78910

Hamming distanceo conv, ok % ❑ conv. false % ● no converg O/.

Fig. 12. New algorithm.

network was compared with the closest target pattern. We

show, in Figs. 11 and 12, the percentage of well retrieved,

nonretrieved, and unstable patterns, respectively, for the

Hebb’s rule and for the one presented here. In pattern

recognition problems, observations are restricted to the left

side of the diagram, i.e., where input patterns are close torecorded ones (the X-axis represents the Hamming dis-tance or number of different bits between these two pat-terns). The conclusion was that with the new learning rule,more than 95 percent of the input patterns were correctlyretrieved, and unstable patterns were almost nonexistentwithin the Hamming distance of 2. In the new learningrule, since the number of connection weights is small and

independent from the number of memorized patterns, it iswell - adaptedpaper.

to VLSI circuits, as stated before in this

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]Collier-MacMillan, 1970

VI. CONCLUS1ON

A neural array that realizes a. content-addressable mem-

ory using Hopfield’s network is described. Its analog ”VLSIimplementation is also described. The synapse design is

extremely simple for density reasons. Simulations haveshown that circuits based on this architecture can containup to hundreds of neurons. Although the circuit wasinitially designed for content-acldressable memory applica-tions, it can be used in a large number of other applica-tions including optimization, pattern recognition and im-age processing.

A new learning rule also is described. The storage capac-ity of neural networks is increased when compared withrespect to the classical Hebb’s and projection rules. Fur-

thermore, this rule is well adapted to VLSI circuits wherethe number of possible synaptic weights is restricted forcompaction reasons.

The algorithm and the clescribed circuit seem to bepromising for the realization of high-storage content-addressable memories and applications using the speedand retrieval properties of neural networks.

W. MacCullogh and W. Pitts, “A logicaf calculus of the ideasimmanent in nervous activity,” Bull. Math. Biophys., vol. 5, pp.115-133.1943.

R. Lipprnann, “An introduction to computing with neuraf nets,”IEEE ASSP Magazine, vol. 4, pp. 4-22, Apr. 1987.M. Minsky and S. Papert, Perceptions: An Introduction to Corrrpu-tational Geometry. Cambridge, MA.: M.I.T. Press, 1969.J. HoDfield. “Neural networks and ~hvsical svstems with emerszentcollective c’omputatiortal abilities,”’ P;oc. N&l. A cad. Sci. U-SA,vol. 79, pp. 2554–2558, Apr. 1982.J. Hopf&id, “Neurons wi{h graded response have collective compu-tational properties like those of two-state neurons;’ Proc. Natl.A cad. Sci. USA, vol. 81, pp. 3088-3092, may 1984.R. Hecht-Nielsen, “ Neurocomputing: Picking the human brain,”IEEE Spectram, vol. 25, pp. 36-41, Mar. 1988.A. Murray, A. Smith, and Z. Butler, “VLSI bit-seriaf neural net-work,” in Proc. Int. Workshop VLSI for Artificial Intelligence(Univ. of Oxford, U.K.), July 1988, pp. F4\l-F4\9.F. Blayo and P. Hurst, “A VLSI systolic array dedicated toHop field neural network,” in Proc. Int. Workshop VLSI A rtificialIntelligence (Univ. of Oxford, U.K.), July 1988, pp. E2\l-E2\10.M. Weinfeld, “A fully digital integrated CMOS Hopfield networkincluding the learning algorithm,” in Proc. Int. Workshop VLSIArtificial Intelligence (Univ. of Oxford, U.K.), July 1988, pp.El /l-El /10.H.’ Graf &d P. de Vegvar, “A CMOS implemention of a neuralnetwork model,” in Proc. Stanford Conf. Adv. Res. in VLSI.Cambridge, MA: M.I.T. Press, 1987.M. Verleysen, B. Sirletti, and P. Jespers, “A new VLSI architecturefor neuraf associative memories,” in Proc. nEuro 88 (Pans, France),June 1988.E. Fahlman and G. Hinton, “ Connectionist architectures for artifi-cial intelligence,” IEEE Conrput., pp. 100-109, Jan. 1988.L. Personnaz, I. Guyon, and G, Dreyfus, “Information storage andretrievaf in spin-glass lrke neural networks,” .J. Phys, Lett., no. 46,pp. 359-365, 1985.D. Gonze, E. Charlier, and E. Gilissen, “R&eaux de neurones:+tude de m6moires associative et conception cf’un circuit CMOSprogrammable,” Eng. thesis, Universit6 Catholique de Louvain,Belgium, 1988.L. Lasdon, Optimization Theory for Large Systems, London:

works, content-addressable memories, and ana-log integrated circuits and systems.

Bruno Sirletti obtained the engineering degreein electronical engineering at the Universit&Catholique de Louvain, Louvain-la-Neuve, Bel-gium, in 1987.

He joined the Microelectronics Laboratory ofthe Universit& Catholique de Louvain where hestarted a new research activity on neural net-works. Since October 1987 he has been workingin the field of analog circuits and artificial neuralnetworks with a fellowship from IRSIA.

VERLEYSEN d a[. :NEURAL NETWORKS FOR CONTENT-ADDRESSABLEMEMORY 569

Andr6 M. Vandemeulebroecke was born in Tour-nay, Belgium, on August 29, 1961. He receivedthe engineering degree from the Universit6Catholique de Louvain, Louvain-la-Neuve, Bel-gium, in 1983. From 1983 to 1985, he was grantedan IRSIA fellowship and worked in the field ofsilicon compilation for digitaf signal processors.In 1985, he was granted an FNRS fellowshipwhile working towards the Ph.D. degree in thefield of the theory and applications of redundantnumbers systems at the Universit4 Catholique de

Louvain, Laboratoire de MicroLlectronique. His research activities andinterest are centered around computational techniques for full-customintegrated circuits and applications to A\D conversion, cryptography,image processing, and neuraf networks.

Paul G. A. Jespers (M60–SM65–F’82) was bornin Belgium on September 30, 1929. He receivedthe engineering degree from the Universit& Librede Bruxelles in 1953 and the Ph.D. degree fromthe Universit6 Catholique de Louvain, Louvain-la-Neuve, Belgium, in 1958.

He first joined the Laboratoire Central d’Elec-tricit&, Brussels, working in RFI measurementsuntil 1959. Since then he has been with theDepartment of Electrical Engineering, Universit4Catholique de Louvain, heading the Laboratoire

de Microilectronique. He was a Visiting Professor at Stanford University,Stanford, CA, from September 1967 to January 1968. His current interestis in MOS integrated circuits and systems.

Dr. Jespers is Vice-Chairman of the Steering Committee of the Euro-pean Solid-State Circuits Conference. He was appointed IEEE RegionalDirector of Region 8 from 1971 to 1972.