Proc. of Neural Computation (NC2000), Berlin, Germany, May 2000 Abstract A new multi-dimensional interpolation method for func- tion approximation using Self-Organizing Maps (SOMs) [1] is proposed. The output is continuous and infinitely differentiable in all the interpolation area, and the error function has no local minima insuring the convergence of the learning toward the minimal error. The complexity is O(n) in the general case (where n is the number of neurons). The novelty of this method lies in two facts. First, it uses directly the Euclidean distances already computed to self-organize the map in the input space. Second, it uses the neighborhood relationships between the neurons to create “neighboring” influence kernels. The interpolation leans on the Local Linear Mapping (LLM) [2] computed by each neuron for which the “neighboring” influence kernel is activated by the input vector. The results obtained in different approximation tests show that this new interpolation method improve the approximation quality of standard SOMs using LLM. A comparison between such a Continuous SOM (CSOM) and a Multi-Layered Perceptron (MLP) is also presented showing it has equivalent performances in term of accuracy and it is liable to limit the interference phenomenon [3] thanks to its local representation. This method could be applied fruitfully to other local net- works such as Neural Gas (NG) [4][5] or Growing Neu- ral Gas (GNG) [6][7] networks. Keywords: Multi-dimensional interpolation; Neural networks; Function approximation; Self-organizing maps. 1. Introduction and state of the art Although their universal approximation capabilities has been proved [8], Multi-Layered Perceptrons (MLPs) or Radial Basis Functions (RBFs) neural networks present some limits for function approximation. One limit is related to the interference [3] which occurs during the learning phase when the learning process in one area of the input space causes a loss of memory in another area. Another one is that the learning algorithm can get stuck into local minima of the error function. On another hand, the Self-Organizing Map (SOM) [1] has the characteristic of being a local framework liable to limit the interference phenomenon and preserving the topology of the data using neighborhood links between the neurons. SOMs have been used in [2][9][10][11] with a first order expansion around each neuron in the output space, called Local Linear Mapping (LLM), and in [12] with a “topological” interpolation technique in order to solve function approximation problems. But the presence of discontinuities of the output function for input vectors lying along the border of Voronoï regions in such approaches put a strain on their approximation quality. We have proposed in [13][14] another interpolation method based on LLM technique which uses neighbor- hood informations and a continuous projection tech- nique to get a Continuous SOM (CSOM). Although we succeeded in suppressing the discontinuities along the border of Voronoï regions of neighboring neurons of the map, that work was limited to one-dimensional SOMs. Interpolation is a wide domain of computational geome- try. It exists a lot of different interpolation techniques adapted to specific applications. Splines and B-Splines techniques and their numerous extensions are often used in Computer Aided Design applications [15][16][17]. A statistical technique called “kriging” is able to deal with environmental data while no gradient at each support vector is specified [18]. A “nature-like” inspired interpo- lation technique called “natural neighbors” [19][20] uses the Voronoï tessellation of the set of support vectors to build the interpolation. However, most of these techniques are computationally expensive and limited to low dimensional input spaces. We get interested in SOMs, NG and GNG because the Function Approximation with Continuous Self-Organizing Maps using Neighboring Influence Interpolation Michaël Aupetit Pierre Couturier Pierre Massotte LGI2P - Site EERIE - EMA Parc Scientifique Georges Besse F 30035 Nîmes, France e-mail: {Michael.Aupetit, Pierre.Couturier, Pierre.Massotte}@site-eerie.ema.fr
Transcript
Proc. of Neural Computation (NC2000), Berlin, Germany, May 2000
Abstract
A new multi-dimensional interpolation method for func-tion approximation using Self-Organizing Maps (SOMs)[1] is proposed. The output is continuous and infinitelydifferentiable in all the interpolation area, and the errorfunction has no local minima insuring the convergenceof the learning toward the minimal error. The complexityis O(n) in the general case (where n is the number ofneurons). The novelty of this method lies in two facts.First, it uses directly the Euclidean distances alreadycomputed to self-organize the map in the input space.Second, it uses the neighborhood relationships betweenthe neurons to create “neighboring” influence kernels.The interpolation leans on the Local Linear Mapping(LLM) [2] computed by each neuron for which the“neighboring” influence kernel is activated by the inputvector. The results obtained in different approximationtests show that this new interpolation method improvethe approximation quality of standard SOMs usingLLM. A comparison between such a Continuous SOM(CSOM) and a Multi-Layered Perceptron (MLP) is alsopresented showing it has equivalent performances interm of accuracy and it is liable to limit the interferencephenomenon [3] thanks to its local representation. Thismethod could be applied fruitfully to other local net-works such as Neural Gas (NG) [4][5] or Growing Neu-ral Gas (GNG) [6][7] networks.
Although their universal approximation capabilities hasbeen proved [8], Multi-Layered Perceptrons (MLPs) orRadial Basis Functions (RBFs) neural networks presentsome limits for function approximation. One limit is
related to theinterference [3] which occurs during thelearning phase when the learning process in one area ofthe input space causes a loss of memory in another area.Another one is that the learning algorithm can get stuckinto local minima of the error function.On another hand, the Self-Organizing Map (SOM) [1]has the characteristic of being a local framework liableto limit the interference phenomenon and preserving thetopology of the data using neighborhood links betweenthe neurons. SOMs have been used in [2][9][10][11]with a first order expansion around each neuron in theoutput space, called Local Linear Mapping (LLM), andin [12] with a “topological” interpolation technique inorder to solve function approximation problems. But thepresence of discontinuities of the output function forinput vectors lying along the border of Voronoï regionsin such approaches put a strain on their approximationquality.We have proposed in [13][14] another interpolationmethod based on LLM technique which uses neighbor-hood informations and a continuous projection tech-nique to get a Continuous SOM (CSOM). Although wesucceeded in suppressing the discontinuities along theborder of Voronoï regions of neighboring neurons of themap, that work was limited to one-dimensional SOMs.Interpolation is a wide domain of computational geome-try. It exists a lot of different interpolation techniquesadapted to specific applications. Splines and B-Splinestechniques and their numerous extensions are often usedin Computer Aided Design applications [15][16][17]. Astatistical technique called “kriging” is able to deal withenvironmental data while no gradient at each supportvector is specified [18]. A “nature-like” inspired interpo-lation technique called “natural neighbors” [19][20] usesthe Voronoï tessellation of the set of support vectors tobuild the interpolation.However, most of these techniques are computationallyexpensive and limited to low dimensional input spaces.We get interested in SOMs, NG and GNG because the
Function Approximation with Continuous Self-Organizing Mapsusing Neighboring Influence Interpolation
Michaël Aupetit Pierre Couturier Pierre Massotte
LGI2P - Site EERIE - EMAParc Scientifique Georges Besse
F 30035 Nîmes, Francee-mail: {Michael.Aupetit, Pierre.Couturier, Pierre.Massotte}@site-eerie.ema.fr
Proc. of Neural Computation (NC2000), Berlin, Germany, May 2000
predefined topology of SOMs or the adaptive topologyof NG and GNG, tend to reflect the intrinsic topology ofthe input data set by self-organization. This topologicalinformation about the data set is interesting because itindicates which input kernel vector (i.e. support vector incomputational geometry) can be considered as a neigh-bor of which other, while a simple distance criterion (e.g.nearest neighbors) is not sufficient to do so.We present here-after two considerations that inspired usto create a new multi-dimensional interpolation tech-nique specifically adapted to SOMs and NGs:•Competitive neural networks compute the Euclideandistance between the input vector and all the neurons’input kernel vectors of the network in order to determinethe closest neuron (the “winner”) and eventually itsneighbors, and to organize the neurons into the inputdata set. This model of learning is inspired from biologi-cal neural networks [1] where an “activity bubble” emer-ges from lateral inhibitory or excitatory connectionsbetween neighboring neurons. We propose these lateralconnections could be used just as well for topology pre-serving as for generating the output value from the win-ning neuron and its neighbors which participate to the“activity bubble”. In that way, each neuron could coope-rate with its neighbors to compute a better output valuethan a single neuron does.•The calculation of all the Euclidean distances betweenthe input vector and each neuron’s input kernel vector incompetitive learning, generates an overabundance ofvalues that contain implicitly the information about theposition of the input vector in the input space. Thesevalues could be used directly to compute an interpola-tion.According to these considerations, we propose a newmulti-dimensional interpolation method valid for SOMs,NG and GNG neural networks and based on the “inversedistance interpolation” first proposed by Shepard in [21].In this paper, we emphasize on a SOM using that tech-nique.Our method is based on the calculation of the Euclideandistance between the input vector and each neuron’sinput kernel vector of the SOM. It leans on the LLMvalue computed by each neuron and on the predefinedtopology of the SOM. That topology tends to reflect thetopology of the input data set by self-organization of theneurons in the input space, and defines a neighborhoodwhich enables to decide through “neighboring” influ-ence kernels how much a neuron has to participate to theinterpolation.The output of the SOM is the weighted sum of the linearoutput of each neuron where each weight depends on theneurons’ “neighboring” influence kernels.The original equation which does not consider the“neighboring” influence kernels, and its general proper-
ties are presented in Section 2. In Section 3, a way tolocalize the original equation is proposed using the“neighboring” influence kernels. Experiments andresults are presented in Section 4. At last, we discuss andconclude in Section 5.
2. Interpolation with Euclidean distances
In that section, we present the general principle of theinterpolation method detailed in [21], so that we do notconsider neighborhood relationships between the neu-rons.Given a set of neurons at a specific place in the inputspace according to their own input kernel vector; giventhe gradient and the output value associated to each neu-ron (i.e. to each input kernel vector) by learning; weexpect the interpolated output function to present the fol-lowing properties:•it respects the output values given by the neurons attheir corresponding place in the input space.•it respects the output gradient given by the neurons attheir corresponding place in the input space.•it is continuous and infinitely differentiable.We consider the following general equation:
where n is the total number of neurons, M is the inputvector, S(M) is the interpolated output value associatedto M, Si(M) is the linear output value associated to M bythe neuron i (i.e. Local Linear Mapping [2]) such as:
where Ni[out] is the output value and Ni[in] the input ker-nel vector of the neuron i, and Ai is the gradient vectorlearnt by the neuron i.
In (1), ωi(M) is a weight value depending on M andassociated to the neuron i, such as S(M) has the expectedproperties described above and formalized as follows:
where Nm[in] is the input kernel vector of the neuron m,p is the dimension of the input space and xNm[in] [h] is the
hth coordinate of the input kernel vector Nm[in] of theneuron m in the p-dimensional input space.
S M( ) ωi M( ) Si M( )⋅i 1=
n
∑ (1)=
Si M( ) Niout[ ]
A iT M Ni
in[ ]–( )+=
m 1 n,[ ]∈ h 1 p,[ ] S Nmin[ ]( ) Sm Nm
in[ ]( )=,∈∀,∀
xNm
in[ ]h[ ]
∂∂ S Nm
in[ ]( )xNm
in[ ]h[ ]
∂∂ Sm Nm
in[ ]( )=
and ωi M( )i 1=
n
∑ 1=
Proc. of Neural Computation (NC2000), Berlin, Germany, May 2000
Shepard has proposed to define eachωi(M) as follows:
where dα(M,Nk) is a term of distance (e.g. the square ofthe Euclidean distance forα=2), n is the number of neu-rons and p is the dimension of the input space.The following properties have been demonstrated:•Whatever the number n of neurons and the dimension pof the input space, the complexity to compute S(M) isO(n.(n+p)).•S takes the output value provided by each neuron ineach of their respective position in the input space.
•S respects the (α-1)th partial derivatives given by eachneuron in each of their respective position in the inputspace.•S is continuous and infinitely differentiable if and onlyif no neuron is exactly at the same place as another onein the input space.•If α=2, S tends asymptotically towards the average ofthe linear outputs of all the neurons. The hyper-surfacewhich represents S in the input-output space, looks like a“rubber” canvas stretched out to pass through each neu-ron respecting their gradient (Figure 1).•According to theα value, S is not biased in the interpo-lation of constant, linear or quadratic functions if theneurons give the exact corresponding output value, gra-dient and curvature.
•the weightsωi(M) are scale invariants, rotation inva-riants and translation invariants. That means the interpo-lation S only depends on the relative positions of theneurons between them and the input vector whatever thesystem of linear coordinates.By experimenting this interpolation function, we foundthe results are not satisfactory. The “rubber” effect ofthat function tends to bring back to the average the out-put value associated to input vectors far from any neuron(e.g. between them inside the cloud of neurons or out-side the cloud (Figure 1)). Another problem is the highcomplexity required to compute S(M) which is at least
O(n2). The reason for these two problems is that all theneurons participate to the interpolation. To come over
this problem, we propose in the next section to define aninfluence kernel function associated to each neuron andbased on their neighborhood relationships to decide howmuch a neuron has to contribute to the interpolation.
Figure 1: The “rubber” effect. The linear output of fiveneurons showing the edges of the Voronoï regions (top).S leaning on these neurons (bottom at a larger scale)
tends asymptotically toward the average of their linearoutputs (dz/dx=0.2,dz/dy=-0.2).
3. Neighboring influence kernel
In order to get a better interpolation which takes intoaccount only the relevant neurons, we should computethe Delaunay triangulation of the neurons’ input kernelvectors and consider those which are the vertex of a sim-plex which encompasses the input data. This kind ofcomputation is quite expensive (O(n.log(n)) (for n neu-rons) and we want to use as much as possible, all theinformations we already have. These informations arethe distances which are useful to self-organize the neu-rons in the input space and the neighborhood links whichreflect the topology of the input data set.Thus, for each neuron we define a new kind of influenceregion. The influence of a neuron is no more limited toits Voronoï region like in standard SOMs, but extends toa larger neighborhood which directly depends on theposition in the input space of the predefined neighbors ofthis neuron in the SOM. We call this influence kernel a“neighboring” influence kernel (Figure 2).In a two dimensional input space, the neighboring influ-ence kernel of a neuron resembles a hill with a plateau atthe height 1, covering the fully activated input region
Proc. of Neural Computation (NC2000), Berlin, Germany, May 2000
which contains this neuron’s input kernel vector andthose of its neighbors, and smoothly falling down to theground at the height 0 around the plateau (Figure 3).Whatever the dimension of the input space, it is built tohave the following properties:•its fully activated region (value 1) is an approximationof the convex hull of the predefined neighbors of a neu-ron in the input space. In that way, the neurons which areactivated by the input data thanks to their “neighboring”influence kernel, are approximately the vertex of a sim-plex which encompasses the input data (Figure 2a).•In order to keep the first order continuity of the originalequation, we use a natural cubic spline function (NCS)to pass continuously from the 1-region to the 0-region(Figure 2b).Now, we give the formal definition of the neighboringinfluence kernel. We define a natural cubic spline as:
where a and b are real numbers.
Second, we define the partσi,h(M) of the neighboringinfluence function of the neuron i with its neighbor haccording to the input vector M as: (Figure 3)
whereγ set the width of the area between 0 and 1 of theneighboring influence kernel andµ set the width of thefully activated region around the neighbors (Figure 3).
Figure 2: The neurons A;B;C fully involved (black)
(ϕ(M)=1) in the interpolation of M with the 1-region oftheir respective neighboring influence kernel. The other
neurons are less (grey) or not at all (white) involved
according to theirϕ value (a). The NCS function (b).
Finally, the neighboring influence functionϕi(M) of theneuron i with all its neighbors according to the input vec-tor M is defined as follows: (Figure 3)
where v is the number of predefined neighbors of theneuron i on the SOM.Then, we modify the general form of the interpolationfunction taking into account the neighboring influencekernels as follows:
This new function S has the same properties as beforewith α=2 but the “rubber” effect is reduced leading to amore convenient result (Section 4). Moreover, the com-
plexity of the algorithm is O(n.(p+v)+p.(i2+v2)) where nis the total number of neurons, p is the dimension of theinput space, v is the maximum number of neurons’neighbors and i is the maximum number of neuronsinvolved in the interpolation according to their neighbor-
ing influence kernel (ϕ(M)>0). In the general case, p; vand i are far less than n leading to a O(n) complexity. Atlast, S(M) can be set equal to zero while M is outside thecloud of neurons using the above expression. In thatway, we have the possibility to know whether the net-work is extrapolating or not, which is a useful propertyin applications such as control of complex systemswhere it is preferable to do nothing instead of doing badthings. It could also be used in incremental networks todecide whether the network needs a new neuron or not.
4. Experiments
We emphasize that in this paper, we do not considerSOMs as projection tools or classifiers, but just as a sup-port able to organize the neurons over the input distribu-tion and to map the topological relationships of the inputdata essential to support our interpolation method.
Proc. of Neural Computation (NC2000), Berlin, Germany, May 2000
Experiment 1In this part, we define a function to approximate andcompare the performances of a SOM; a Linear SOM(LSOM) using LLM; and a Continuous SOM (CSOM)using the interpolation method presented.We define a two-dimensional function to approximate:
for x and y ranging over [-1;1](Figure 5).We define a set of 500 random input vectors uniformlydistributed with coordinates x and y ranging over [-1;1].A learning epoch consists in passing once the whole setof input vectors to the network. We use a two-dimen-sional square SOM which is exactly suited to such a dis-tribution and should be able to reflect its topology and toplace the neurons uniformly over it, creating ideal condi-tions to compare the different methods.The training algorithm for SOM; LSOM and CSOM isgiven with the corresponding complexity of each step(Section 3). More details about the learning rule of thegradient for LSOM can be found in [7] and for CSOM in[2] and [9], which respectively give the best results. Theerror surface of CSOM corresponding to the weights
Ni[out] and Ai, is quadratic and so, has only one mini-mum (CSOM’s output is the weighted sum of the linearoutput of each neuron). Each step has to be followedonly by the methods indicated between parentheses:0 (CSOM) - Initial step performed only once: compute the
Euclidean distance between each neuron’s input kernel vector
and those of its neighbors on the map (O(n.v.p));
1 (all) - present a p-dimensional input vector M (O(1));
2 (all) - compute the square of the Euclidean distance from M
to each neuron’s input kernel vector and find the winning neu-
ron Nwin and its neighbors Nv on the map such as (O(n.p)):
win=argmink in [1,n](d2(Nk[in],M));
3 (SOM&LSOM) - compute the corresponding output of the
network for k in {win;v}:
Sk(M)=Nk[out] (SOM); (O(1))
or Sk(M)=Nk[out]+AkT(M-Nk[in]) (LSOM); (O(p))
3 (CSOM) - compute the neighboring influence value of each
neuron according to M and using already computed squared
Euclidean distance values (O(n.v)): eq. (5);
4 (CSOM) - compute the interpolated output S(M) using only
neighboring influence values not equal to zero (O(p.i2)): eq.(6);
5 (all) - compute the output error for k in {win,v} (O(1)):
Errk=f(M)-Sk(M); (SOM&LSOM) or Err=f(M)-S(M); (CSOM)
6 (all) - adapt the input weights of the winner and its neighbors
(k in {win;v}) to self-organize the map using the input vector M
(O(p.v)):
∆Nk[in]=αk. (M-Nk[in]);
7 (SOM&LSOM) - adapt the output weights of the winner and
its neighbors (k in {win;v}) using the output error (O(p.v)):
∆Nk[out]=βk . Errk; (SOM&LSOM)
∆Ak=βk. (M-Nk[in]).Errk; (LSOM)
7 (CSOM) - adapt the output weights of all the neurons Ni acti-
vated (ϕi(M)>0) using a backpropagation of the output error
Err through the weights ωi (O(p.i)):
∆Ni[out]=βwin .ωi(M).Err;
∆Ai=βwin .ωi(M).(M-Ni[in]).Err/d2(Ni[in],M);
8 (all) - adapt the learning rates LR of the training step k:
where LR is αwin and αv. (βwin and βv are constant)
9 (CSOM) - recompute the Euclidean distance between the
input kernel vector of the moving neurons (the winner and its
neighbors) and those of their neighbors on the map (O(p*v2))
10 (all) - go to step 1 while the total number of training steps
kmax is not reached.
Neighboring links Plateau atheight 1
Ground at
N3
N2 N1
N5
N4
height 0
N2
µγ
x
y
z
x
y
x
y
x
y
z
Figure 3: Considering the five neurons of the Figure 1: the partσ1,2(M) of the neighboring influence function of the
central neuron N1 with its left neighbor N2 (Left part: 3D view and contour lines) and the final neighboring influence
functionϕ1(M) including its four neighbors (Right part: 3D view and contour lines).The middle stamp shows the
parametersµ andγ respectively setting the extension of the plateau and the width of the intermediate area.
N2N1
N3
N5
N4
The partσ1,2(M) of the neighboring influence kernel. The final neighboring influence kernelϕ1(M).
Spl x y,( ) NCS 2 x 0.5–( )2 4 y 0.5–( )2 0.5 0.5,–⋅+⋅( )=
f x y,( ) Spl x y,( ) x 0.5–( )2– y 0.5–( )2
–[ ] 0.3⁄( )exp⋅1 Spl x y,( )–( ) 0.2 x y+( )⋅ ⋅( )+
=
LR LRstartLRend
LRstart----------------------
k
kmax----------------
×=
Proc. of Neural Computation (NC2000), Berlin, Germany, May 2000
Table 1 shows the Root Mean Square Error (RMSE)obtained by 6*6 SOM, LSOM and CSOM in the approx-imation of the function f. The neurons get self-organizedin the input space only during the first 6 epochs (α goesfrom αstart to αend and then is set to zero) while the lear-ning of the output continues until 40 epochs. The lear-ning rates and other parameters[αwin_start; αwin_end;αv_start; αv_end; βwin; βv; γ; µ] are set respectively to[0.99; 0.01; 0.9; 0.005; 0.1; 0.01; 0.8; 1.1]. The inputkernel vectors are initialized at random and their corre-sponding output values and gradients start from zero.The RMSE for the whole approximation is computedusing the subset of a regular grid of input vectors M( x,y) with x and y ranging over [-1, 1] from 0.05 to 0.05,such as at least one neuron Ni is fully activated
(ϕi(M)=1). The (x,y) solutions of the inequation:(x+y<=0) define the linear part (L) of f and the non-solu-tions define its non-linear part (NL). We compute theRMSE for these parts separately and together.Table 1 shows that CSOM performs better than LSOMand SOM. CSOM is able to approximate the L part of fas well as LSOM but matches the NL part of f with a fargreater accuracy. Figure 5 shows the different results forthe three algorithms at the end of the learning phase. Therepresentations clearly shows the discontinuities of SOMand LSOM set around the border of Voronoï regions,while CSOM gives a smoother result.
Experiment 2In this part, we apply CSOM and MLP to the previousapproximation test and compare their qualitative behav-ior when a change occurs in the function to approximateduring the learning phase. The MLP is defined using thefunction “newff” of the Matlab V5.3 Neural Net toolbox,with 2 inputs, 23 sigmoïdal “tansig” neurons in the hid-den layer and a single linear “purelin” neuron in the out-put. The training algorithm is “trainlm” using theLevenberg-Markardt method. We found that this net-work is able to give roughly the same RMSE as the 6*6CSOM in the same number of epochs. The RMSE of theMLP is computed with the input vectors M(x,y) with xand y ranging over [-0.8, 0.8] from 0.05 to 0.05 in orderto be comparable to the subset defined by CSOM. Figure4 shows their resulting error approximating the functionf during the 20 first epochs according to its L and NL
parts and then approximating g defined as:
Both methods approximate g with the same accuracy.However, thanks to the local representation of CSOM,the change affects the RMSE only in the region L whereit occurs, while the MLP does not present such localproperties. This locality depends on the number of neu-rons and is liable to limit theinterference phenomenon.
Figure 4: Comparison between CSOM and MLP.(Exp.2)
5. Conclusion and perspectives
A new multi-dimensional interpolation method for func-tion approximation has been presented. A SOM usingthis method (called CSOM) performs better than a stand-ard SOM or a SOM using LLM on the presented test.The comparison with a MLP shows that the locality ofthe representation in CSOM can limit theinterferencephenomenon. Moreover, CSOM has no local minimainsuring the convergence of the learning toward the min-imal error permitted by the number and the configurationof the neurons’ input kernel vectors in the input space. Ituses the Euclidean distances already computed to self-organize the map and the SOM’s neighboring links, tocompute the neighboring influence kernels. However, theway to set the parameters which characterize the neigh-boring influence kernels needs further investigations.Although SOMs are known for their rigidity because oftheir predefined topology, we use it as a support becausewe want to focus on the interpolation method itself andto avoid additional setting of learning parameters. Weintend to use this new method with Neural Gas [4][5]and Growing Neural Gas [6][7] networks which could beviewed as generalized SOMs. These neural networkshave the ideal property of preserving the local topologyof the input distribution approximating a multi-dimen-sional Delaunay triangulation of the neurons so that theygive a perfect support for our interpolation method.
Part of f L NL whole
SOM 0.0336 0.1264 0.0931
LSOM 0.0099 0.0760 0.0546
CSOM 0.0092 0.0280 0.0210
Tableau 1: RMSE of SOM, LSOM and CSOM. (Exp.1)
g x y,( ) f x y,( ) x 0.5+( )2
– y 0.5+( )2–[ ] 0.05⁄( )exp+=
0 5 10 15 20 25 30 35 400
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
RMSE
RMSE
Epoch
CSOM
MLPwhole L part NLpart
Epoch
whole L part NLpart
Proc. of Neural Computation (NC2000), Berlin, Germany, May 2000
[6] B.Fritzke, A Growing Neural Gas Network LearnsTopologies, G.Tesauro,D.S Touretzky and T.K.Leeneditors, Advances in Neural Information ProcessingSystems 7, MIT Press, Cambridge MA, 1995.
[7] B.Fritzke, Incremental Learning of Local LinearMappings, F.Fogelman and P.Gallinari editors,ICANN'95, 217-222, Paris, France, 1995.
[8] G.Cybenko, Approximation by superpositions of asigmoidal function,Mathematics of Control, Signalsand Systems, 2(4), 1989.
[9] D.Moshou & H.Ramon, Extended Self-OrganizingMaps with Local Linear Mappings for FunctionApproximation and System Identification,WSOM'97,http://www.cis.hut.fi/wsom97/progabstracts/31.html,1997.
[10] T.Martinetz, S.Berkovich & K.Schulten, «Neural-Gas» Network for Vector Quantization and its Applica-tion to Time-Series Prediction,IEEE Transactions onNeural Networks, 4(4), 1993.
[12] J.Göppert & W.Rosenstiel, Topology-preservinginterpolation in self-organizing maps,NeuroNimes’93,425-434, 1993.
[13] M.Aupetit, P.Massotte & P.Couturier, A Conti-nuous Self-Organizing Map using Spline Technique forFunction Approximation,AICS’99, 163-169, 1999.
[14] M.Aupetit, P.Massotte & P.Couturier, C-SOM: AContinuous Self-Organizing Map for Function Approx-imation,to be appear in proceedings of ISC’99, 1999.
