Www.spatialanalysisonline.com Chapter 8 Geocomputation Part B: Artificial Neural Networks (ANNs) &...

www.spatialanalysisonline.com

Chapter 8

Geocomputation Part B:

Artificial Neural Networks (ANNs) & Genetic Algorithms (GAs)

3rd edition www.spatialanalysisonline.com 2

Geocomputation: ANNs

In this presentation on geocomputation:ANNs discussed include

Multi-level perceptrons (MLPs) Radial basis function neural networks (RBFNNs) Self organising feature maps (SOFMs)

ANNs are particularly concerned with Function approximation and interpolation Image analysis and classification Spatial interaction modelling


Geocomputation: Evolutionary computing

In this presentation on geocomputation:

EC elements discussed include Genetic algorithms (GAs) Genetic programming (GP)

EC is particularly concerned with Complex problem solving using GAs Model design using GP methods


Geocomputation

Artificial Neural Networks (ANNs) A computational model based on emulating

biological neural networks A form of non-linear modelling tool Often a 3-layer network structure is used:

input, hidden, output The output layer of such structures are typically

modified weighted sums of intermediate layers, which are modified weighted sums of the input layer


Artificial Neural Networks

Hence at each output node (hidden or final) a two-step process takes place:

g


Artificial Neural Networks Simple 3-layer feedforward ANN Fully inter-connected; each connection

is given a weight, w Known as a Multi-level perceptron

(MLP) In this case: 3 input nodes, 5 hidden

nodes, 2 output nodes and 2 bias nodes (bias, B, is similar to the constant term in regression models)

At hidden node 1 we have:

where the wij are weights to be determined, b1=1, and the xi are the observed input values

2

1

3

1

2

3

4

5

1

2

INPUT HIDDEN OUT PUT

B1 B2

n=3 m=5 p=2

1

*01 1 11 1 21 2 31 3h w b w x w x w x



is simply a linear weighted sum of the inputs. To generate a non-linear output it must be modified by some (well behaved) non-linear function, g(), e.g. the logistic function:

i.e.

1 01 1 11 1 21 2 31 3h g w b w x w x w x

1

*01 1 11 1 21 2 31 3h w b w x w x w x

1( )

1 hg h

e

0

0.5

1

-5 -4 -3 -2 -1 0 1 2 3 4 5

Logistic (1+Tanh)/2 Step Linear

Sample activation functions



We can now compute the output layer values as the weighted sum

Suppose we have known input values x1=1, x2=-3, x3=5, and known outputs of 0 and 1. Can we select the weights to ensure the inputs generate the known outputs?

Suggestion: <build your own worked example & program here>

*1 01 2 11 1 21 2 31 3 41 4 51 5

*1 1

, and thusy z b z h z h z h z h z h

y g y



Learning Supervised learning

Split training/test data sets (control data) Known inputs and output (target) values for training data (Network output-Target output) = Error signal, e Systematically adjust weights to minimise sum of e2

Adjustment typically based on backpropagation and gradient descent

Used in many classification/pattern recognition applications and in function approximation

Unsupervised learning No training data Must create clusters by analysing dataset for patterns/clusters



Some basic issues: local vs global minimisation Initialisation and selection Data normalisation and coding Momentum Model design and over-fitting Overtraining Interpolation vs Extrapolation/Forecasting



MLP: Example 1 function approximation

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0 100 200 300 400 500 600 700 800 900 10000

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

Epochs

RM

SE

(R

oot

mea

n sq

uare

d er

ror)

source data fitted solution curve RMSE vs epochs



MLP Example 2: LCM



MLP Example 2: LCM



MLP Example 2: LCM



MLP Example 2: LCM

weights matrix



MLP Example 3: Spatial interaction model Generalised model: Tij=f(Oi,Dj,dij) Sample data format

(log transformed):



MLP Example 3: Spatial interaction model



Radial Basis Function NetworksBasic functional form:

Gaussian RBF:

1

( ),

where denotes the Euclidean norm.

n

k ij i ji

w

y x c

2 2/ 2

2

1( )

2

xx e



Self organising function maps SOM as an output space Neighbourhood relations Grid size, form and topology



Self organising function maps Dimensional reductions Mapped output – similar vectors (units) are

close to each other Typically an unsupervised procedure Spatial mapping of SOM can follow using

simple assignment to best matching unit (BMU)



Self organising function maps Choose a grid size, form and topology Train the network

Identify the best matching units Modify the BMU and its neighbours (spatially biased

learning rule) Map the trained network



Self organising function maps – some issues Initialisation Pre-processing Normalisation Missing data Masking and weighting Learning and tuning Distance metrics Neighbourhood functions (kernels) Learning rate functions



Self organising function maps – Idrisi



Self organising function maps – Idrisi


Genetic Algorithms

Solutions are represented as individuals Individuals are modelled as chromosomes Chromosomes are comprised of genes Genes have values known as alleles Chromosomes have a measurable fitness New chromosomes (children) are created by reproduction and mutation processes

The fittest individuals survive The creation process is then iterated


Genetic Algorithms

GAs: Example 1 - TSP

chromosome

genes

allele=12 (ID of town in TSP problem set)

Each chromosome contains complete list of towns• create a set of m randomly permuted strings and compute lengths, d• evaluate the fitness of each string (e.g. 1/d)• select random pairs of tours (biased by fitness)• combine pairs by crossover operation• evaluate fitness of offspring• apply replacement rule (fittest retained) and iterate till stable


Genetic Algorithms

GA components Encoding or representation – binary, list, tree etc Fitness function selection – use of rank transforms Population initialisation Selection: roulette, tournament, uniform random Reproduction Crossover e.g. A = [a b c d e f g h] B = [1 2 3 4 5 6 7 8]

and the crossover point is 3, the following children are generated:

child 1 = [a b c 4 5 6 7 8] child 2= [1 2 3 d e f g h] Mutation Local search Termination


Genetic Algorithms

GAs: application areas TSP (as above) Clustering Map labelling Optimum location with capacity constraints Concept can be extended to alleles that are

expressions or program elements rather than numerical values Genetic programming

Date post:	26-Mar-2015
Category:	Documents
Upload:	grace-clayton
View:	221 times
Download:	6 times

Www.spatialanalysisonline.com Chapter 8 Geocomputation Part B: Artificial Neural Networks (ANNs) &...

Documents