A Bi-population Particle Swarm Optimizer for Learning Automata based Slow

Intelligent System

Mohammad H. Mofrad

University of Pittsburgh

Thursday, December 08, 2016

1hasanzadeh@cs.pitt.edu

CS 2310 - Multimedia Software Engineering

Preface

2

• Slow Intelligence System (SIS)

• Particle Swarm Optimization (PSO)

• Learning Automata (LA)

• Adaptive Intelligence Optimizer (AIO)

Slow Intelligence System (SIS)

S.-K. Chang, “Slow intelligence systems,” in International Conference on Multimedia Modeling, 2010 3

• Enumeration (-enum<) of the different available solutions until finding the optimal solution

• Propagation (=prop+) of the achieved new information from the new solutions within a body of feasible solutions.

• Adaptation (+adap=) of the current solutions using the effective information gained from the elite solutions.

• Elimination (>elim-) of the worst solutions that exist in the problem space.

• Concentration (>conc=) on the elite solutions to produce new promising solutions.

Particle Swarm Optimization (PSO)

J. Kennedy, “Swarm intelligence,” Handb. Nat.-Inspired Innov. Comput., pp. 187–219, 2006. 4

• Inspired from movement of animals

• Vi = w Vi + c1 r1 (pbesti - Xi) + c2 r2 (gbest - Xi)

• Xi = Xi + Vi

vi n

vi n, pbest pbesti

n

vi n, gbest

gbesti

xi n

Vi+1 n

xi +1n

Learning Automata (LA)

M. Thathachar and P. S. Sastry, “Varieties of learning automata: an overview,” Syst. Man Cybern. Part B Cybern. IEEE Trans. On, vol. 32, no. 6, 2002. 5

• Belongs to the reinforcement learning family

Environment

Learning Automata

SignalAction

11

1 |

j j

j

j

p n a p n j ip n

p n a j j i

Reward signal

11

1 1 |

j

j

j

p n b j ip n

b r b p n j j i

Penalty signal

Adaptive Intelligence Optimizer (AIO)

6

• 2 PSO populations

• SIS framework

• LA framework

SIS + PSO

7

• Mapping SIS’s operators to PSO formula• cycle1: [guard1,2] P0 –enum< P1 =prop+ P2 >elim- P3 >conc= P4

• cycle2: [guard2,1] P0 –enum< P1 =prop+ P2 +adap= P3 >elim- P4 >conc= P5

• Simulating SIS’s slow & quick decision cycles• w = wmax – (0.75i (wmax - wmin)/imax)

• w = wmax – (i (wmax - wmin)/imax)

Implementing SIS’s TDR system using AIO

8

LA1 LA2

d2d1

s1

sk

LAn

dn

{0,1}

{0,1} {0,1}

{0,1} {0,1}

{0,1}

T D R

Implementing SIS Controller Component

9

SIS Controller

LA1 LA2 LAn

Swarm Membership

(LA Action selection)

D1 D2 Dn

s1 sk

PSO Population

CV (Si, gbesti)

Swarm Refinement

(LA Probability Update)

β1

βn

β2

αn

α2

α1

The Isolation of PSO populations

10

{p0, p1}

LA1 LA2

d2d1

s1

sk

LAn

dn

{0,1}

{0,1} {0,1}

{0,1} {0,1}

{0,1}

LA1

LAk {p0, p1}

Bi-population PSO

CV (Si, gbesti)β1βn

α1

αn

s1 sk

Implementation

11

• Python 3.4 (we extensively use NumPy package)

• Github repository @ https://goo.gl/V0vTRM

• Code• PSO ~200 lines• AIO ~400 lines

• XML interface for reading configurations

• 5 benchmark functions• Sphere, Rosenbrock, Ackley, Griewanks, Rastirign• 30 dimensions• 50 particles• …

Results – Sphere Benchmark

12

1E-294

1E-282

1E-270

1E-258

1E-246

1E-234

1E-222

1E-210

1E-198

1E-186

1E-174

1E-162

1E-150

1E-138

1E-126

1E-114

1E-102

1E-90

1E-78

1E-66

1E-54

1E-42

1E-30

1E-18

1E-06

Fit

nes

s

Iteration

PSO AIO

Results – Rosenbrock Benchmark

13

1E-08

0.0000001

0.000001

0.00001

0.0001

0.001

0.01

0.1

1

10

100

1000

10000

100000

1000000

10000000

100000000

1E+09

1E+10

1E+11

Fit

nes

s

Iteration

PSO AIO

Results – Ackley Benchmark

14

0.0001

0.001

0.01

0.1

1

10

100

Fit

nes

s

Iteration

PSO AIO

Results – Griewanks Benchmark

15

0.001

0.01

0.1

1

10

100

Fit

nes

s

Iteration

PSO AIO

Results – Rastrigin Benchmark

16

1E-14

1E-13

1E-12

1E-11

1E-10

1E-09

1E-08

0.0000001

0.000001

0.00001

0.0001

0.001

0.01

0.1

1

10

100

1000

10000

100000

Fit

nes

s

Iteration

PSO AIO

Future Work

17

• Clustering

• Dimensionality reduction

• Feature selection

• Parameter estimation

Questions?

18