http://metalab.uniten.edu.my/~farrukh/Utim/Utimoptim.zipOptimization with MATLABA Hands-On WorkshopFarrukh NagiUniversiti Teknologi MARA , Shah Alam Campus25-26 June, 2014*

PART I *

PART 1 INTRODUCTION TO OPTIMIZATION

Introduction to OptimizationFundamental of OptimizationMathematical BackgroundUnconstrained Optimization MethodsGradient Descent Methods (Steepest Descent)Least Square MethodsSimplex Methods4. Constrained Optimization

Contents*

PART 2 - MATLAB OPTIMIZATION

Matlab/Simulink Optimization MethodsFunction OptimizationMinimization AlgorithmsUnconstrained Optimization fminunc, fminsearchOptimization Options SettingsConstrained OptimizationMulti-objective Optimization - lsqnonlin Optimization Toolbox >>optimtool %GUI Environmental Science Optimization

4.

*

PAGE

function A=fence(x)

A=-1*x(1)*x(2);

function [c, ceq]=fencecon(x)

c=[x(1)+2*x(2)-500];

ceq=[];

%startfence.m

x0=[1,1];

options=optimset('LargeScale','off');

[x,fval]=fmincon(@fence,x0,[],[],[],[],[],[],@fencecon,options)

function C=box(x)

w=x(1);

h=x(2);

l=3*w;

C=10*(2*l*w)+6*(2*w*h + 2*l*h);

function [c, ceq]=boxcon(x)

w=x(1);

h=x(2);

l=3*w;

%non-linear inequality constraint

c=[];

%non-linear equality constraint

ceq=[l*w*h-50 ];

%startbox.m

clc

clear

x0=[1,1];

options=optimset('LargeScale','off');

[x,fval]=fmincon(@box,x0,[],[],[],[],[],[],@boxcon,options);

w=x(1),l=3*x(1),h=x(2)

function [C,d]=boxgrad(x)

w=x(1);

h=x(2);

l=3*w;

C=10*(2*l*w)+6*(2*w*h + 2*l*h);

% syms w h

% v=[w h];

% f=10*(2*l*w)+6*(2*w*h + 2*l*h);

% d=jacobian(f,v);

d= [120*w+48*h, 48*w ];

%startboxgrad.m

clc

clear

x0=[1,1];

options=optimset('LargeScale','off');

options=optimset(options,'GradObj','on','GradConstr','on','Display','iter');

[x,fval]=fmincon(@boxgrad,x0,[],[],[],[],[],[],@boxcongrad,options)

w=x(1),l=3*x(1),h=x(2)

[c, ceq]=boxcongrad(x)

function A=rect(z)

x=z(1);

y=z(2);

%

A=-4*(x)*(y);

function [c, ceq]=rectcon(z)

x=z(1);

y=z(2);

%non-linear inequality constraint

c=[];

%non-linear equality constraint

ceq=[x^2+y^2-16];

%startrect.m

clc

clear

x0=[1 0];

options=optimset('LargeScale','off','Display','iter','Maxiter',20,'MaxFunEvals',100,'TolX',1e-3,'TolFun',1e-3); %options=optimset('MediumScale','on');

[x,fval]=fmincon(@rect,x0,[],[],[],[],[],[],@rectcon,options)

Example 4 Determine the point(s) on y = x2 + 1 that are

closest to (0,2).

Solution

Again, lets get a quick sketch.

Minimization with equality constrain

function D=parab(z) %Objective function

x=z(1);

y=x^2+1; %Given function, relation b/w y depend upon x

D=x^2+(y-2)^2;

function [c, ceq]=parabcon(z)

x=z(1);

y=z(2);

%y=x^2+1;

%non-linear inequality constraint

c=[];

%non-linear equality constraint

ceq=[y-x^2-1];

%startparab.m

clc

clear

x0=[1 1];

options=optimset('LargeScale','off','Display','iter','Maxiter', 200,'MaxFunEvals',100,'TolX',1e-8,'TolFun',1e-8);

%options=optimset('MediumScale','on');

[x,fval]=fmincon(@parab,x0,[],[],[],[],[],[],@parabcon,options)

function h = height(V,r)

h = (V-2*pi*r.^3/3)./(pi*r.^2);

function cost = tower(r)

h = height(500,r);

cost = 600*pi*r.*h+800*pi*r.^2;

The session is:

>>optimum r = fminbnd(tower,0,100)

optimum r =

4.9237

>>min cost = tower(optimum r)

min cost =

9.1394e+4

>>optimum h = height(500,optimum r)

optimum h =

3.2825

So the optimum radius is 4.9237 meters; the optimum height is

3.2825 meters, and theminimum cost is $91,394.

function cost = fence(R)

A = 1600;

L = (A-0.5*pi*R.^2./(2*R);

cost = 30*(2*R+2*L)+40*pi*R;

The session is

>> optimum_R = fminbnd('fence08',0,100)

optimum_R =

18.6137

>> min_cost = fence08(optimum_R)

min_cost =

5.1575e+003

>> optimum_L = (A-0.5*pi*optimum_R.^2)./(2*optimum_R)

optimum_L =

28.3599

The optimum radius is 18.6137 feet, and the corresponding length

is 28.3599 feet. The minimum cost is $5,157.50.

Box Example and Solution-2

Example 1

Fence Example and Solution

Rectangle Example and Solution-3

PARABOLA MATLAB OPTIMIZATION EXAMPLE-4

Parabola Example and Solution -4

below

IRRIGATION PROBLEM EXAMPLE 5 Unconstrained minimization

TANK PROBLEM EXAMPLE 6 Bounded Minimization

FENCE II PROBLEM - 7

The function file is

function [L, total] =

field (W,A)

L = (A-W.^2/8)./W;

Total = 2*L+W+2*W/sqrt(2)

The session is:

>> [L, total] = field(6,80)

The results are L = 12.5833 meters and total = 39.6519 meters.

FENCE III PROBLEM 8 Bounded Minimization

RAIN EXAMPLE 9 Find the zero of function

The function files are

function deltaV = volume(t)

global r x

V = 1e+9+1e+8*(1-exp(-t/100))-r*t;

deltaV = V-0.01*x*1e+9;

function time = decrease(x,r)

global r x

time = fzero(volume,1);

The session is

>>time = decrease(50,1e+7)

time =

54.1832

Thus it will take about 54 days.

PAPER PROBLEM EXAMPLE 10

Bounded Minimization

PAPER PROBLEM EXAMPLE - 11

a) The function is

function [V, A] = torus(a,b)

V = pi^2*(a+b).*(b-a).^2;

A = pi^2*(b.^2-a.^2);

The session is

>>a = [0.25:0.01:4];

>>b = a+2;

>>[V,A] = torus(a,b);

>>subplot(2,1,1),plot(a,V),xlabel(a (in.)),ylabel(V (cu. in.))

_subplot(2,1,2),plot(a,A),xlabel(a (in.)),ylabel(A (sq. in.))

function [c, ceq, DC, DCeq]=boxcongrad(x)

w=x(1);

h=x(2);

l=3*x(1);

%non-linear inequality constraint

c=[];

DC=[];

%non-linear equality constraint

ceq=[l*w*h-50 ];

DCeq=[6*w*h;

3*w^2];

BOX MATLAB OPTIMIZATION EXAMPLE-2 GRAD

Gradient (Jacobian) - Minimization with equality constrain

RECTANGLE MATLAB OPTIMIZATION EXAMPLE - 3

Maximization with equality constrain

BOX MATLAB OPTIMIZATION EXAMPLE-2

Minimization with equality constrain

FENCE MATLAB OPTIMIZATION Example-1 Maximization with inequality constrain

OPTIMIZATION

BASIC PHYSICS EXAMPLES

and

PROBLEMS

10

Page

PART 3 - SIMULINK OPTIMIZATION RESPONSE /IEEE_optim/

Parametric Modeling Parameter IdentificationParameter Passing with Component Block InputSimulink Optimization Design (SOD) GUI9. Optimizer Output10. Simulink Examples List

REFERENCES

*

What is Optimization? Optimization is an iterative process by which a desired solution(max/min) of the problem can be found while satisfying all its constraint or bounded conditions. Optimization problem could be linear or non-linear.Non linear optimization is accomplished by numerical Search Methods.Search methods are used iteratively before a solution is achieved. The search procedure is termed as algorithm.Figure 2: Optimum solution is found while satisfying its constraint (derivative must be zero at optimum).*

Optimization Methods One-Dimensional Unconstrained OptimizationGolden-Section SearchQuadratic InterpolationNewton's Method Multi-Dimensional Unconstrained OptimizationNon-gradient or direct methodsGradient methods Linear Programming (Constrained)Graphical SolutionSimplex Method

Genetic Algorithm (GA) Survival of the fittest principle based upon evolutionary theory \IEEE_OPTIM_2012\GA\GA Presentation.ppt

Particle Swarm Optimization (PSO) Concept of best solution in the neighborhood : \IEEE_OPTIM_2012\PSO\ PSO Presentation.ppt

Others .*

The cycle of a Genetic Algorithms is presented below

Each cycle in Genetic Algorithms produces a new generation of possible solutions for a given problem

The elements of the population are encoded into bit-strings, called chromosomes.

The performance of the strings, often called fitness, is then evaluated with the help of some functions f(x) , representing the constraints of the problem.

Genetic Algorithm Steps

The crossover operation that recombines the bits (genes) of each two selected strings (chromosomes) . The crossover helps to span over the solution space

In Mutation the bits at one or more randomly selected positions of the chromosomes are altered. The mutation process helps to overcome trapping at local maxima

Mutation of a chromosome at the 5th bit position.

Genetic Algorithm Steps

Example: The Genetic Algorithms cycle is illustrated in this example for maximizing a function f(x) = x2 in the interval 0 = x = 31.

In this example the fitness function is f (x) itself. Starts with 4 initial strings. The fitness value of the strings and the percentage fitness of the total are estimated in Table A.

Table A

Table B

Since fitness of the second string is large, we select 2 copies of the second string and one each for the first and fourth string in the mating pool. The selection of the partners in the mating pool is also done randomly

Here in table B, we selected partner of string 1 to be the 2-nd string and partner of 4-th string to be the 2nd string. The crossover points for the first-second and second-fourth strings have been selected after 0-th and 2-nd bit positions respectively in table B.

Example

The second generation of the population without mutation in the first generation is presented in table C.

Table C

>> gatool %- Solution% gaex.m function f=gaex(x)f=-x*x; %Maximize

>> load gaexoptim >> gaexoptions

Example 1 -\IEEE_OPTIM_2012\GA\gaex.m

Example 1- \IEEE_OPTIM\_2012\GA \gaex.m

0-4 => 5 bits

25 =32 => [0;31]

Fitness f(x)x -variable

1. Use Gatool and maximize the quadratic equation f(x) = x2 +4x-1 within the range 5 x0.

2. Use Gatool and maximize the functionf(x1, x2, x3)= 5 sin(x1) sin(x2) sin(x3) + (- sin(5x1) sin(5x2)sin(x3))where 0 GATOOL% Import from workspace 'optimproblem'

function fposition=Live_fn_gatool(x)fposition=(x(1)+1.42513)^2+(x(2)+.80032)^2;

Example 2 -\IEEE_OPTIM_2012\GA\Live_fn_gatool.m

function funct=motor_select_ga_fun(x) Ki=x(1);Ka=x(2); assignin('base','Ki', Ki); assignin('base','Ka', Ka); [t, xout, err]=sim('lead_screw_model_opt_Motor_ga',[0 10]);funct=sqrt(sum(err).^2);

Eample 3 - Genetic Algorithm with Simulink \IEEE_OPTIM_2012\GA\motor_select_opt_ga.m%lead_screw_model_opt_Motor_ga.mdl%motor_select_ga_fun.m%load gaoptions.mat clcKi=0.1, Ka=1;u0=[Ki,Ka];options=gaoptimset(options,'PopulationSize',10,'TimeLimit',100, 'Generations',200,'StallTimeLimit',20,'TolCon',1e-6,'TolFun',1e-6) [x,funct]=ga(@motor_select_ga_fun,2,[],[],[],[],[0.001 0.001],[],[],options)

hold offclfclc[y, fposition]=ga(@Live_fn_ga, 2);x=yfposition;%3-D plot.[X,Y]=meshgrid(-5:0.1:5,-5:0.1:5);Z=(X+1.42513).^2+(Y+.80032).^2;surf(X,Y,Z), holdplot(y(1),y(2),'or')

function fposition=Live_fn_ga(x) t=1; u=[x(1),x(2)]; dt=t;[t, xout, y]=sim('Live_fn_simGA',t,simset('MaxDatapoints',1),[t,u]);fposition=y;

Example 4 \IEEE_OPTIM\_2012\GA \Live_fn_ga.m

Particle Swarm optimisation

Maurice.Clerc@WriteMe.com

These slides are adapted from a presentationby Maurice.Clerc@WriteMe.com - one of themain researchers in PSO

PSO invented by Russ Eberhart (engineering Prof) and James Kennedy (social scientist)in USA

Maurice.Clerc@WriteMe.com

Cooperation example

Maurice.Clerc@WriteMe.com

Particle Swarm optimisation

The basic idea

Each particle is searching for the optimum Each particle is moving and hence has a velocity.Each particle remembers the position it was in where it had its best result so far (its personal best)But this would not be much good on its own; particles need help in figuring out where to search.

Particle Swarm optimisation

The basic idea II

The particles in the swarm co-operate. They exchange information about what theyve discovered in the places they have visited The co-operation is very simple. In basic PSO it is like this: A particle has a neighbourhood associated with it. A particle knows the fitnesses of those in its neighbourhood, and uses the position of the one with best fitness.This position is simply used to adjust the particles velocity

Initialization. Positions and velocities

Maurice.Clerc@WriteMe.com

Particle Swarm optimisation

What a particle does

In each timestep, a particle has to move to a new position. It does this by adjusting its velocity. The adjustment is essentially this:The current velocity PLUS A weighted random portion in the direction of its personal best PLUSA weighted random portion in the direction of the neighbourhood best.Having worked out a new velocity, its position is simply its old position plus the new velocity.

Neighbourhoods

geographical

social

Maurice.Clerc@WriteMe.com

Neighbourhoods

Global

Maurice.Clerc@WriteMe.com

The circular neighbourhood

Virtual circle

Particle 1s 3-neighbourhood

Maurice.Clerc@WriteMe.com

Particles Adjust their positions according to a ``Psychosocial compromise between what an individual is comfortable with, and what society reckons

Here I am!

The best perf. of my neighbours

My best perf.

x

pg

pi

v

Maurice.Clerc@WriteMe.com

Particle Swarm optimisation

Pseudocodehttp://www.swarmintelligence.org/tutorials.php

Equation (a)v[] = c0 *v[] + c1 * rand() * (pbest[] - present[]) + c2 * rand() * (gbest[] - present[]) (in the original method, c0=1, but many researchers now play with this parameter)

Equation (b)present[] = present[] + v[]

Particle Swarm optimisation

Pseudocodehttp://www.swarmintelligence.org/tutorials.php

For each particle Initialize particle END Do For each particle Calculate fitness value If the fitness value is better than its peronal best set current value as the new pBest End Choose the particle with the best fitness value of all as gBest For each particle Calculate particle velocity according equation (a) Update particle position according equation (b) End While maximum iterations or minimum error criteria is not attained

Particle Swarm optimisation

Pseudocode

Particles' velocities on each dimension are clamped to a maximum velocity Vmax. If the sum of accelerations would cause the velocity on that dimension to exceed Vmax, which is a parameter specified by the user. Then the velocity on that dimension is limited to Vmax.

Animated illustration

Global optimum

Maurice.Clerc@WriteMe.com

Particle Swarm optimisation

Parameters

Number of particlesC1 (importance of personal best)C2 (importance of neighbourhood best)

How to choose parameters

Maurice.Clerc@WriteMe.com

Particle Swarm optimisation

Parameters

Number of particles (1050) are reported as usually sufficient.C1 (importance of personal best)C2 (importance of neighbourhood best)Usually C1+C2 = 4. No good reason other than empiricismVmax too low, too slow; too high, too unstable.

Some functions often used for testing real-valued optimisation algorithms

Maurice.Clerc@WriteMe.com

... and some typical results

Optimum=0, dimension=30

Best result after 40 000 evaluations

Maurice.Clerc@WriteMe.com

30D function

PSO Type 1"

Evolutionary algo.(Angeline 98)

Griewank [300]

0.003944

0.4033

Rastrigin [5]

82.95618

46.4689

Rosenbrock [10]

50.193877

1610.359

Adaptive swarm size

There has been enough improvement

but there has been not enough improvement

although I'm the worst

I'm the best

I try to kill myself

I try to generate a new particle

Maurice.Clerc@WriteMe.com

How and when should an excellent algorithm terminate?

Like this

Maurice.Clerc@WriteMe.com

Particle Swarm optimisation

function fposition=Live_fn(x) p=0;q=0; for k=1:5 p=p+k*cos((k+1)*x(1)+k); q=q+k*cos((k+1)*x(2)+k); end t=1; u=[p,q,x(1),x(2)]; dt=t;[t, xout, y]=sim('Live_fn_sim',t,simset('MaxDatapoints',1),[t,u]); %simulink filefposition=y;% fposition=p*q+(x(1)+1.42513)^2+(x(2)+.80032)^2; %Matlab function f(x)

Examples 1 Program \IEEE_OPTIM_2012\PSO\PSO.m\IEEE_OPTIM_2012\Live_fn.m

Particle Swarm optimisation

function fposition=Live_fn_motor(x) Ki=x(1); Ka=x(2); assignin('base','Ki', Ki); assignin('base','Ka', Ka); t=10; % u=[Ki,Ka]; dt=t;%[t, xout, y]=sim('lead_screw_model_opt_Motor_pso',t,simset('MaxDatapoints',1),[t,u]);[t, xout, y]=sim('lead_screw_model_opt_Motor_pso',t,simset('MaxDatapoints',1)); %fposition=abs(y);fposition=sqrt(sum(abs(y));

Examples 2 Program \IEEE_OPTIM_2012\PSO\PSO_motor.m\IEEE_OPTIM_2012\Live_fn_motor.m

*

*

To illustrate what cooperation means in PSO, here is a simplistic example.

As usually, the big fish is difficult to catch, hidden in the deepest part of the pond. At each time step, each fisherman tells to the other how deep the pond is at his place. At the very begininng, as the depths are quite similar, they both follow their own ways. Now, Fisherman 2 seems to be on a better place, so Fisherman 1 tends to go towards him quite rapidly. Now, the decision is a bit more difficult to make. On the one hand Fisherman 2 is still on a better place, but on the other hand, Fisherman 1s position is worse than before. So Fisherman 1 comes to a compromise: he still goes towards Fisherman 2, but more slowly than before. As we can see, doing that, he escapes from the local minimum.Of course, this example is a caricatural one, but it presents the main features of a particle in basic PSO: a position, a velocity (or, more precisely an operator which can be applied to a position in order to modify it), the ability to exchange information with its neighbours, the ability to memorize a previous position, and the ability to use information to make a decision. Remember, though, all that as to remain simple.

Lets now see more precisely these points.

*

Here you have another nice search space.First step: you put some particles on it. You can do it at random or on a regular way, or both. How many? In practice, for most real problems with dimension between 2 and 100, a swarm size of 20 particles works quite good. There are some mathematical ways to give an estimation, but a bit beyond the scope of this talk. Also, as we will see some variants use an adaptive swarm size.Second step: you define a velocity for each particle, usually at random. You can set all initial velocities to zero but, experimentally, it is usually not the best choice.Remember that what we call velocity is in fact a move, just because time is discretized.

*

Now, for each particle, we define what is called a neighbourhood. Although some variants use a geographical neighbourhood, that is to say compute distances and take the nearest particles, the most widely used neighbourhood is a social one: just a list of neighbours, regardless where they are. So, you do not need to define a distance and that is a great advantage, for in some cases, particularly for discrete spaces, such a definition would be quite arbitrary.Note that it can be proved (and it is intuitively quite obvious) that if the process converges any social neighbourhood tends to be also a geographical one.Usually, in practice, social neighbourhoods are defined just once, at the very beginning, which is consistent with the principle simple rules for simple agents.Now, the size of the neighbourhood could be a problem. Fortunately, PSO is not very sensitive to this parameter and most of users just take a value of 3 or 5 with good results.Unlike for the swarm size, there is no mathematical formula, but like for the swarm size, there are some adaptive variants.

*

Now, for each particle, we define what is called a neighbourhood. Although some variants use a geographical neighbourhood, that is to say compute distances and take the nearest particles, the most widely used neighbourhood is a social one: just a list of neighbours, regardless where they are. So, you do not need to define a distance and that is a great advantage, for in some cases, particularly for discrete spaces, such a definition would be quite arbitrary.Note that it can be proved (and it is intuitively quite obvious) that if the process converges any social neighbourhood tends to be also a geographical one.Usually, in practice, social neighbourhoods are defined just once, at the very beginning, which is consistent with the principle simple rules for simple agents.Now, the size of the neighbourhood could be a problem. Fortunately, PSO is not very sensitive to this parameter and most of users just take a value of 3 or 5 with good results.Unlike for the swarm size, there is no mathematical formula, but like for the swarm size, there are some adaptive variants.

*

The most commonly used neighbourhood is the circular one. The picture is almost self explanatory. Each particle is numbered, put on a virtual circle according to its number and the neighbourhood of a given particle is built by taking its neighbours on this circle.An important point for rule simplicity is that each particle belongs to its neighbourhood. For example if a rule says I have to check all my neighbours, there is no need to add and I have to check myself. We will see that more precisely later.

*

This may be the most important slide of this presentation, for it summarizes the core of the method.Lets take a bit time to comment it.You are a particle. Sorry, I dont mean you are quite stupid, but it is just to explain how it works.(By the way, Jim Kennedy has designed a nice game in which you compete with such stupid particles. I have it here, and if we have time, you will see it is almost impossible to beat it.) You can compute how good is your position (that is to say you can compute the objective function at the place you are). You remember the best position you ever found (and the objective function value). You can ask your neighbours for this information they also have memorized, and choose the best one.Now, you have three tendancies, - audacious, following your own way (just using your own velocity)- conservative, going back more or less towards your best previous position- sheeplike, going more or less towards your best neighbourWhat PSO formalizes is how to combine these tendancies in order to be globally efficient.

*

At the very beginning, particles are randomly put on the search space.There is just one global optimum and several local ones.Let us now run the player. As you can see, the particles are quite rapidly forming some sub swarms around sub optimums. As soon as a particle is quite near of the global optimum, the others go also towards it, so the probability to really find it does increase.Of course, if the run was longer all particles would finally go towards this global optimum

*

*

I am sorry but in this part we have to do a bit more maths.What I call here parameters are in fact just the coefficients in the formula which indicates how to update the velocity of a given particle.Swarm size and neighbourhood size are also parameters but less important. I mean you can perfectly use a swarm size of 20 and a neighbourhood of 3 for a large range of problems with good results.On the contrary, PSO is more sensitive to the parameters we examine now.

*

Let us see now a few examples with some very well known test functions.Of course far more tests have been done and there is now absolutely no doubt that PSO is efficient.

*

For each test the dimension of the search space is 30, and we stop the run after 40000 evaluations.As you can see, PSO is not bad at all, compared to a classical evolutionary algorithm.Of course, this example is quite old, and there is now some better evolutionary or genetic algorithms, but there is also some better PSO versions. As usually, it does not make sense to simply say PSO is better, or GAs are better, so, please, consider this slide just as an illustration of the fact that PSO is at least as good as some better known methods.

*

This is a more recent and sophisticated attempt.I dont give here the precise formulas, just the underlying metaphors.On this slide improvement means improvement in the particles neighbourhood, and I try means success is depending on the current swarm size, according to a probability rule.The good point is you dont have anymore to guess what could be the best swarm size, or to launch a lot of runs to find it, for you can perfectly begin with a very small swarm, and let it increase and decrease by itself.Note that by using the same kind of metaphor, it is also possible to adapt the neighbourhood size.

Introduction to Simulated Annealing

Difficulty in Searching Global Optima

startingpoint

descenddirection

local minima

global minima

barrier to local search

Intuition of Simulated Annealing

Origin:

The annealing process of heated solids.

Intuition:

By allowing occasional ascent in the search process, we might be able to escape the trap of local minima.

Consequences of the Occasional Ascents

Help escaping the local optima.

desired effect

Might pass global optima after reaching it

adverse effect

Control of Annealing Process

Acceptance of a search step (Metropolis Criterion):

Assume the performance change in the search direction is .

Accept a ascending step only if it pass a random test,

Always accept a descending step, i.e.

At each temperature, search is allowed to proceed for a certain number of steps, L(k).

Control of Annealing Process

Cooling Schedule:

T, the annealing temperature, is the parameter that control the frequency of acceptance of ascending steps.

We gradually reduce temperature T(k).

The choice of parametersis called the cooling schedule.

Simulated Annealing Algorithm

0) k = 0;

1) Search (i j), performance difference ;

2) If 0 then accept, elseif exp(-/T(k)) > random[0,1) then accept;

3) Repeat 1) and 2) for L(k) steps;

4) k = k+1;

5) Repeat 1) 4) until stopping criterion is met.

Implementation of Simulated Annealing

Select a local search scheme

Determine the cooling schedule

For example:

Set L = n, the number of variables in the problem.Set T(0) such that exp(-/T(0)) 1.Set T(k+1) = T(k), where is a constant smaller but close to 1.

Implementation of Simulated Annealing

Understand the result:

This is a stochastic algorithm. The outcome may be different at different trials.Convergence to global optima can only be realized in asymptotic sense.

Reference:

P.J.M. van Laarhoven, E.H.L. Aarts, Simulated Annealing: Theory and Applications, Kluwer Academic Publisher, 1987.

A. A. Zhigljavsky, Theory of Global Random Search, Kluwer Academic Publishers, 1991.

*

*

Local search techniques, such as steepest descend method, are very good in finding local optima. However, difficulties arise when the global optima is different from the local optima. Since all the immediate neighboring points around a local optima is worse than it in the performance value, local search can not proceed once trapped in a local optima point. We need some mechanism that can help us escape the trap of local optima. And the simulated annealing is one of such methods.

*

The name of simulated annealing origins from the simulation of annealing process of heated solids. In condensed matter physics, annealing denotes a physical process in which a solid in a heat bath is heated up by increasing the temperature of the heat bath to a maximum value at which all particles of the solid randomly arrange themselves in the liquid phase, followed by cooling through slowly lowering the temperature of the heat bath. In this way, all particles arrange themselves in the low energy ground state of a corresponding lattice. (quoted from Simulated Annealing: Theory and Applications)

In solving combinatorial optimization problems, we make an analogy to the aforementioned process. The basic idea is that by allowing the search process to proceed in an unfavorable direction occasionally, we might be able to escape the trap of local optima and reach the global optima.

*

However, like swords have two edges, there are two consequences of allowing occasional ascent steps. On one hand, it fulfills our desire to let the algorithm proceed beyond local optima. On the other hand, we might miss the global optima by allowing the search process to pass through it. To maintain the desired effect and reduce the adverse effect, we need a sophisticated scheme to control the acceptance of occasional ascents, which is the heart of simulated annealing.

*

Suppose that a step in the search direction produce a difference of in the performance value. The acceptance criterion, which is often referred to as Metropolis Criterion, is as follows:If it is a favorable direction, say 0, we always accept it. Otherwise, this step is accepted with a probability exp(-/T), where T is a parameter.

*

It is obvious that the parameter T plays a pivotal role in the acceptance criterion. The smaller the T is, the less likely an unfavorable step will be accepted. To obtain a desirable result, the parameter T will be gradually reduced as the algorithm proceeds. So at the beginning, T is large and thus the search can easily escape the trap of local optima. And later on, by reducing T, we allow the algorithm to converge. It is common practice that at each choice of T, we will allow the algorithm to proceed a certain steps, L(k). And the choice of the sequence of parameters {T(k), L(k)} is often called the cooling schedule.

*

This is an abstract description of a simulated annealing algorithm. With proper selection of parameters, it is proven that it can converge to a global optima with probability 1.

*

The implementation of simulated annealing algorithm is problem dependent. First, a proper local search scheme must be chosen. For continuous problems, steepest descend is often used. For discrete problems, a neighborhood structure is defined. And usually search is carried out by randomly selecting one of the neighbors of the current design.

Similarly, there is no fixed formula for determining the cooling schedule. A guideline is that: the number of search steps at each temperature setting should be sufficient, usually correlated to the size of the problem; the initial temperature should be high enough to allow frequent acceptance; and finally the decrease of the temperature should not be too fast. We provide a particular choice of cooling schedule here as an example.

*

Finally, we like to emphasize the interpretation of the algorithm output. Simulated annealing is a stochastic algorithm. Because random variables are used in the algorithm, the outcome of different trials may vary even for the exact same choice of cooling schedule. Moreover, the convergence to the global optima of simulated annealing is only achieved when algorithm proceeds to infinite number of iterations.

*

The solution of the linear problem lies on boundaries of the feasible region. Non-linear problem solution lies within and on the boundaries of the feasible region.Figure 3: Solution of linear problemFigure 4: Three dimensional solution of non-linear problemFundamentals of Non-Linear Optimization*

Constraints Inequality EqualityFundamentals of Non-Linear Optimization Single Objective function f(x) Maximization Minimization Design Variables, xi , i=0,1,2,3.. Figure 5: Example of design variables and constraints used in non-linear optimization. Optimal points Local minima/maxima points: A point or Solution x* is at local point if there is no other x in its Neighborhood less than x* Global minima/maxima points: A point or Solution x** is at global point if there is no other x in entire search space less than x** Contd*

Figure 6: Global versus local optimization.Figure 7: Local point is equal to global point if the function is convex.Fundamentals of Non-Linear Optimization Contdconvexconvexnot convexnot convexnot convexA set S is convex if the line segment joining any two points in the set is also in the set.*

Function f is convex if f(Xa) is less than value of the corresponding point joining f(X1) and f(X2).

Convexity condition Hessian 2nd order derivative) matrix of function f must be positive semi definite ( eigen values +ve or zero).Fundamentals of Non-Linear Optimization Figure 8: Convex and nonconvex set Figure 9: Convex function Contd*

Hessian Second derivative of f of several variables Second order condition (SOC) Eigen values of H(X*) are all positive Determinants of all lower order of H(X*) are +ve First order Condition (FOC)Optimality Conditions*

a.) Indirect approach by transforming into unconstrained problem.

b.) Exterior Penalty Function (EPF) and Augmented Lagrange Multiplier

c.) Direct Method Sequential Linear Programming (SLP), SQP and Steepest Generalized Reduced Gradient Method (GRG)Figure 10: Descent Gradient or LMSOptimization Methods Constrained*

Steepest descent method*

Example*

Example (cont.)*

Matlab steepest descent- fminunc %startpresentgrad.mclcclfhold offx0=[5,1];options = optimset('OutputFcn', @outfun); %for graphic displayoptions=optimset('LargeScale','off','Display','iter-detailed'); %for iteration options=optimset(options,'GradObj','on'); %for enabling gradient objectoptions = optimset(options,'OutputFcn', @outfun);[x,fval]=fminunc(@myfuncon2,x0,options);

function [f,g]=myfuncon2(x)function stop = outfun(x, optimValues, state)f=0.5*x(1)^2+2.5*x(2)^2; ww1= -6:0.05:6;if nargout > 1 ww2=ww1;g(1)=x(1); %gradient 1 supplied [w1,w2]=meshgrid(ww1,ww2); g(2)=5*x(2); %gradient2 suppliedJ=-1*(0.5*w1.^2+2.5*w2.^2);foo=max(abs(g)); cs=contour(w1,w2,J,20);hold ongridEndstop=false%cs=surf(w1,w2,J);hold on;Gridplot(x(1),x(2),bl+);drawnow

*

Non-Linear least squares*

Non-Linear least squares*

Simplex MethodsMinimize*

Derivative-free optimizationDownhillsimplexmethod*

Example: constrained optimization*

Example (cont.)*

Example (cont.)*

Environmental Sciences Optimization Case Studies*1. Fish Harvesting/Utim/fishharvester/2. River Pollution../Utim/WWTP_RivPol/3. Noise Pollution./Utim/machine_noise/

Data Fitting1. Hydrology .../Utim/hydrology/2. Anthropometric.../Utim/anthropometry/

PAGE

1

FISH HARVESTING EXAMPLE - SIMULINK

In this example, the objective is to develop an optimal strategy for harvesting a renewable resource (i.e., fish). From the dynamics of a logistic population model, a cost must be minimized with respect to accessible system parameters.

Let x(t) be the number of fish in a lake at time t, then the rate of change

is a conservation equation for the population. The birth and death terms both depend on x. One possibility is

Here r is the birth rate per capita, the number of baby fish hatched per year divided by the total number of fish. The fish also die, of course, from disease, from being eaten by predators which constitutes the term p1x2 , p1 is the measure of vulnerability- weakness of the fishes. The larger the p1 more fish dies.

If there is no fishing then then the system will be at equilibrium and

. Solving we get

We call xe the EQULIBRIUM POPULATION. If we know from zoological research the birth rate per capita r and weakness of the fish, p1 , then we can compute xe . Notice if p1 weak fishes is large equilibrium population xe is small

Now comes the fisherman. We have to advise them how many they can catch so that so that the fishes sustain themselves without our intervention. Statistically, if we let p2 be the fraction of the fishes to be caught per year to sustain the fishes . Then x p2 fishes are caught per year.

Fisheries experts call this FLEXIBLE HARVESTING. Naturally p2 has to be less than 1. Our equation for x is now

The equilibrium is now given as

Equilibrium xe is smaller now than before fishing.

Note equation (2) is non-linear and the solution for x(0)= x0 is given as

The following quantities are given:

x0 = 100 (initial fish population)

r = 0.8/month = 0.5

xd =65 (desired fish population)

= 0.9

tf = 10 (months)

= 0.035

The equation (2) is simulated in SIMULINK and no analytical solution is required in MATLAB

The simulink results shows that the papameter selected above can sustain itself and fishing harvesting would be successful in this period.

Unconstrained Minimization

function J = fishes1_new(pond)

p1=pond(1);

p2=pond(2);

xd=65;

r=0.8;

assignin('base','p1',p1);

assignin('base','r',r);

assignin('base','p2',p2);

[t,x,yout]=sim('fishpond1_new',[0 20]);

J=sum(yout)/length(yout);

%% STARTUP PROGRAM

clc

clear

u0=[0; 0] ;

options=optimset('LargeScale','off','Display','iter','Maxiter', 50,'MaxFunEvals',100,'TolX',0.001,'TolFun',0.001);

[p] = fminsearch('fishes1_new', u0,options)

p1=p(1);

p2=p(2);

sim('fishpond1_new',[0 20]

fishpond1_new.mdl

_1239798307.unknown

_1239801325.unknown

_1239801363.unknown

_1239801379.unknown

_1239800627.unknown

_1239798236.unknown

Waste Water Treatment Plants (WWTP)

[Type a quote from the document or the summary of an interesting point. You can position the text box anywhere in the document. Use the Text Box Tools tab to change the formatting of the pull quote text box.]

Four cities (i) are located on a river and its tributary.

Pi=14 (mg/day) is the pollutants of the cities dump into the river.

Waste Water Treatment plant in each city remove (clean) fraction xi=14 of Pi4 pollutants.

Remaining discharge in the river Wi =(1 xi)*Pi

Concentration at discharge point , where Qu is upstream flow (litres/day) and cu is upstream concentration (mg/Litres)

Ri is the reduction in pollutant concentration by biological decomposition

Different treatment cost di ($1000/mg removed) at different plants

Objective Function for total treatment cost on daily basis at four plants

Minimize ($1000/day)

xi=14 , variable is the fraction of pollutants removed by the WWTPs

Constraint pollutants concentration in the river should be less than 20(mg/litres)

function J= riverpol(x)

%City Pollution mg/day

P1=1e+9;

P2=2e+9;

P3=4e+9;

P4=2.5e+9;

% Waste treatment Cost

d1=2e-6;

d2=2e-6;

d3=4e-6;

d4=4e-6;

J= d1*P1*x(1)+d2*P2*x(2)+d3*P3*x(3)+d4*P4*x(4);

function [C,Ceq]=rivercon(x)

%City Pollution mg/day

P1=1e+9;

P2=2e+9;

P3=4e+9;

P4=2.5e+9;

%River Flow litre/day

Q13=1e+7;

Q23=5e+7;

Q34=1.1e+8;

Q45=2.5e+8;

%Decomposition Removal

R13=0.5;

R23=0.35;

R34=0.6;

%concentration mg/Litre

Ceq(1)=-20+(((1-x(1))*P1)/Q13);

Ceq(2)=-20+(((1-x(2))*P2)/Q23);

Ceq(3)=-20+((R13*Q13*Ceq(1)+R23*Q23*Ceq(2)+(1-x(3))*P3)/Q34);

Ceq(4)=-20+((R34*Q34*Ceq(3)+(1-x(4))*P4)/Q45);

C=[];

River Pollution GUI

R23

Q23c2

Qucu

cs4 =20

cs3 =20

cs2 =20

cs1 =20

Anthropometry

On average, the surface area A of human beings is related to weight W, and height, H. Measurement on a number of individuals of height 180cm and different weights (kg) gives values of A (m2) in the following table

W (kg)

70

75

77

80

82

84

87

90

A(m2)

2.10

2.12

2.15

2.20

2.22

2.23

2.26

2.30

Show that a power law , A=aWb fits the data reasonably well. Find the constant a and b.

% Anthropometric Data

function y= anthrop(x)

W = [70 75 77 80 82 84 87 90]'; %weight

A = [2.10 2.12 2.15 2.20 2.22 2.23 2.26 2.30]'; %surface area of human being

a=x(1);

b=x(2);

A1= a*W.^b;

y=A-A1

Hydrology Data

The following data was collected for the slope, hydraulic radius and velocity of water flowing in the canal:

S, m/m

0.0002

0.0002

0.0005

0.0005

0.001

0.001

R, m

0.2

0.5

0.2

0.5

0.2

0.5

U, m/s

0.25

0.5

0.4

0.75

0.5

1

There are theoretical reason to believe that this data can be fit to a power model of the form

U=SR

Where , , are empirically derived coefficients. Find the coefficients.

%Hydrology Data

function UT = hydrology(x)

S=[0.0002 0.0002 0.0005 0.0005 0.001 0.001] % Canal slope m/m

R=[ 0.2 0.5 0.2 0.5 0.2 0.5] % radiud m

U= [ 0.25 0.5 0.4 0.75 0.5 1] % water flow velocity

%Data Fit relatioship

U1= x(1)*(S.^x(2)).*R.^x(3);

UT=U-U1;

NOISE CONTROL OPTIMIZATION OF PLANT USING SIMULATED ANNEALING

Tian-Syung et.al , Taiwan 2007.

An optimization of noise control for the m-noise plant is shown Figure 1. To evaluate the influence of equipment to the environment, an n-point monitoring station uniformly around the boundary line has been introduced and shown in Figure 2. For a sound wave propagating in a free sound field, the effects of distance decay and air absorption have been considered. For a specified receiving point i, the total sound pressure level- SPLi radiated and accumulated from m's equipments is expressed as

1. Simulated Annealing %MeshsoundPrOptmSA.m

2. Genetic Algorithm %MeshsoundPrGAOptm.m

% Objective Function

function FF = soundPr2(r)

global SWL

phi=0.2;

zr=[1.5 1.5 1.5 1.5 1.5 1.5 1.5 1.5 1.5 1.5 1.5 1.5 1.5 1.5 1.5 1.5 1.5 1.5 1.5 1.5];

yr=[0 0 0 0 0 0 10 20 30 40 50 50 50 50 50 50 40 30 20 10];

xr=[0 10 20 30 40 50 50 50 50 50 50 40 30 20 10 0 0 0 0 0];

c=0;

f=1000;

m=4;

n=20;

FFOldi=0;

FFOldj=0;

for i=1:n % no. of points

for j= 1:4 % no. of machine

R(i,j)=sqrt((xr(i)-r(1+c)).^2 + (yr(i)-r(2+c)).^2 + (zr(i)-r(c+3)).^2);

SPL=SWL(j)-7.4*(10^-8)*(((R(i,j)).*f^2)/phi)-20*log10(R(i,j))-11;

if (j==1)

SPL=SWL(j)-7.4*(10^-8)*(((R(i,j)).*f^2)/phi)-20*log10(R(i,j))-11;

elseif (abs(R(i,j)-R(i,j-1))

MATLAB OptimizationPART II *

MATLAB/SIMULINK OPTIMIZATION METHODS*

Function Optimization Optimization concerns the minimization or maximization of functions Standard Optimization Problem:Equality ConstraintsSubject to:Inequality ConstraintsSide Constraintsis the objective function, which measure and evaluate the performance of a system. In a standard problem, we are minimizing the function. For maximization, it is equivalent to minimization of the ve of the objective function.Where:is a column vector of design variables, which canaffect the performance of the system.*

Function Optimization (Cont.)Equality Constraints Inequality ConstraintsSide ConstraintsMost algorithm require less than!!! Constraints Limitation to the design space. Can be linear or nonlinear, explicit or implicit functions*

Optimization Toolbox Is a collection of functions that extend the capability of MATLAB.

The toolbox includes routines for: Unconstrained optimization Constrained nonlinear optimization, including goal attainment problems, minimax problems, and semi-infinite minimization problems Quadratic and linear programming Nonlinear least squares and curve fitting Nonlinear systems of equations solving Constrained linear least squares Specialized algorithms for large scale problems*

Unconstrained Minimization Consider the problem of finding a set of values [x1 x2]T that solves Steps: Create an M-file that returns the function value (Objective Function). Call it objfun.m Then, invoke the unconstrained minimization routine. Use fminunc*

Step 1 Obj. Functionfunction f = objfun(x)

f=exp(x(1))*(4*x(1)^2+2*x(2)^2+4*x(1)*x(2)+2*x(2)+1);

Objective function*

Step 2 Invoke Routinex0 = [-1,1]; options = optimset(LargeScale,off);[xmin,feval,exitflag,output]=fminunc(objfun,x0,options);Output argumentsInput argumentsStarting with a guessOptimization parameters settings*

xmin = 0.5000 -1.0000feval = 1.3028e-010exitflag = 1output = iterations: 7 funcCount: 40 stepsize: 1 firstorderopt: 8.1998e-004 algorithm: 'medium-scale: Quasi-Newton line search'Minimum point of design variablesObjective function valueExit flag tells if the algorithm is converged.If exit flag > 0, then local minimum is foundSome other informationResults*

[xmin,feval,exitflag,output,grad,hessian]= fminunc(fun,x0,options,P1,P2,)fun: Return a function of objective function.x0: Starts with an initial guess. The guess must be a vector of size of number of design variables.Option: To set some of the optimization parameters. (More after few slides)P1,P2,: To pass additional parameters.More on fminunc Input*

[xmin,feval,exitflag,output,grad,hessian]= fminunc(fun,x0,options,P1,P2,)xmin: Vector of the minimum point (optimal point). The size is the number of design variables.feval: The objective function value of at the optimal point.exitflag: A value shows whether the optimization routine is terminated successfully. (converged if >0)Output: This structure gives more details about the optimizationgrad: The gradient value at the optimal point.hessian: The hessian value of at the optimal pointMore on fminunc Output*

Options = optimset(param1,value1, param2,value2,)Options Setting optimset The routines in Optimization Toolbox has a set of default optimization parameters. However, the toolbox allows you to alter some of those parameters, for example: the tolerance, the step size, the gradient or hessian values, the max. number of iterations etc. There are also a list of features available, for example: displaying the values at each iterations, compare the user supply gradient or hessian, etc. You can also choose the algorithm you wish to use.*

Options = optimset(param1,value1, param2,value2,)LargeScale - Use large-scale algorithm if possible [ {on} | off ]The default is with { }Parameter (param1)Value (value1)Options Setting (Cont.) Type help optimset in command window, a list of options setting available will be displayed. How to read? For example:*

LargeScale - Use large-scale algorithm if possible [ {on} | off ]Since the default is on, if we would like to turn off, we just type:Options = optimset(LargeScale, off)Options = optimset(param1,value1, param2,value2,)and pass to the input of fminunc.Options Setting (Cont.)*

Display - Level of display [ off | iter | notify | final ] MaxIter - Maximum number of iterations allowed [ positive integer ] TolCon - Termination tolerance on the constraint violation [ positive scalar ] TolFun - Termination tolerance on the function value [ positive scalar ] TolX - Termination tolerance on X [ positive scalar ]Highly recommended to use!!!Useful Option Settings*

fminunc and fminsearchfminunc uses algorithm with gradient and hessian information. Two modes: Large-Scale: interior-reflective Newton Medium-Scale: quasi-Newton (BFGS) Not preferred in solving highly discontinuous functions. This function may only give local solutions.. fminsearch is generally less efficient than fminunc for problems of order greater than two. However, when the problem is highly discontinuous, fminsearch may be more robust. This is a direct search method that does not use numerical or analytic gradients as in fminunc.

This function may only give local solutions.*

[xmin,feval,exitflag,output,lambda,grad,hessian] = fmincon(fun,x0,A,B,Aeq,Beq,LB,UB,NONLCON,options,P1,P2,)Vector of LagrangeMultiplier at optimal pointConstrained Minimization*

Subject to:function f = myfun(x)f=-x(1)*x(2)*x(3);Example*

ForCreate a function call nonlcon which returns 2 constraint vectors [C,Ceq]function [C,Ceq]=nonlcon(x)

C=2*x(1)^2+x(2);Ceq=[];Remember to return a nullMatrix if the constraint doesnot applyExample (Cont.)*

x0=[10;10;10];A=[-1 -2 -2;1 2 2];B=[0 72]';LB = [0 0 0]';UB = [30 30 30]';[x,feval]=fmincon(@myfun,x0,A,B,[],[],LB,UB,@nonlcon)Initial guess (3 design variables)CAREFUL!!!fmincon(fun,x0,A,B,Aeq,Beq,LB,UB,NONLCON,options,P1,P2,)Example (Cont.)*

Example (Cont.)*

>> optimtool - % GUI*

Constrained Optimization

An optimization algorithm is large scale when it uses linear algebra that does not need to store, nor operate on, full matrices. In contrast, medium-scale methods internally create full matrices and use dense linear algebra

The definition is based on the Karush-Kuhn-Tucker (KKT) conditions. The KKT conditions are analogous to the condition that the gradient must be zero at a minimum, modified to take constraints into account. The difference is that the KKT conditions hold for constrained problems. The KKT conditions use the auxiliary Lagrangian function:*

fmincon Algorithms

fmincon has four algorithm options: a) interior-point, b) active-set, c) SQP, d) trust-region-reflective

a) An Interior point method is a linear or nonlinear programming method (Forsgren et al. 2002) that achieves optimization by going through the middle of the solid defined by the problem rather than around its surfaceb) Active Set Approach Equality constraints always remain in the active set Sk. The search direction dk is calculated and minimizes the objective function while remaining on active constraint boundaries. *

c) Sequential quadratic programming (SQP): is an iterative method for nonlinear optimization . SQP methods are used on problems for which the objective function and the constraints are twice continuously differentiable.

If the problem is unconstrained, then the method reduces to Newton's method for finding a point where the gradient of the objective vanishes. If the problem has only equality constraints, then the method is equivalent to applying Newton's method to the first-order optimality conditions, or KarushKuhnTucker conditions , of the problem.

d) Trust-Region Reflective: The basic idea is to approximate f with a simpler function q, which reasonably reflects the behavior of function f in a neighborhood N around the point x. This neighborhood is the trust region. A trial step s is computed by minimizing (or approximately minimizing) over N. This is the trust-region sub problem

The current point is updated to be x + s if f(x + s) < f(x); otherwise, the current point remains unchanged and N, the region of trust, is shrunk and the trial step computation is repeated. Sequential Quadratic Programming and Trust Region*

lsqnonlin in Matlab Multi Objective Curve fittingLink to this Page

clc;%recfit.mclear; global data; data= [ 0.6000 0.9990.6500 0.998 0.7000 0.997 0.7500 0.995 0.8000 0.982 0.8500 0.975 0.9000 0.932 0.9500 0.862 1.0000 0.714 1.0500 0.520 1.1000 0.287 1.1500 0.134 1.2000 0.0623 1.2500 0.0245 1.3000 0.0100 1.3500 0.0040 1.4000 0.0015 1.4500 0.0007 1.5000 0.0003 ]; % experimental data,`1st coloum x, 2nd coloum R x=data(:,1); Rexp=data(:,2); plot(x,Rexp,'ro'); % plot the experimental data hold on b0=[1.0 1.0]; % start values for the parameters b=lsqnonlin('recfun',b0) % run the lsqnonlin with start value b0, returned parameter values stored in b Rcal=1./(1+exp(1.0986/b(1)*(x-b(2)))); % calculate the fitted value with parameter b plot(x,Rcal,'b'); % plot the fitted value on the same graph Find b1 and b2

>>recfit>>b = 0.0603 1.0513 %recfun.mfunction y=recfun(b) global data; x=data(:,1); Rexp=data(:,2); Rcal=1./(1+exp(1.0986/b(1)*(x-b(2)))); % the calculated value from the model %y=sum((Rcal-Rexp).^2); y=Rcal-Rexp; % the sum of the square of the difference %between calculated value and experimental value*

SIMULINK OPTIMIZATION RESPONSE

PART III*

SIMULINK & OPTIMIZATION DESIGNSIMULINK MODEL OPTIMIZATIONSimulink Block Objective function (PSO,GA)*

SIMULINK MODEL FILE Ports input/output/Utim_Optim/PSO/Live_fn_sim.mdl

* /Utim_Optim/PSO/PSO.m>> PSO % start program, Line 40 current_fitness(i) = Live_fn(current_position(:,i));

*Genetic Algorithm Objective function

/Utim_Optim/GA/Live_fn_simGA.mdl

SIMULINK OPTIMIZATION EXAMPLES

PARTICLE SWARM OPTIMIZATION Particle Swarm Optimization function (\Utim\PSO\Live_fn_sim.mdl)______________________________________________________________________________________________________________________________________

B. GENETIC ALGORITHM GA function optimization (\Utim\GA\Live_fn_simGA.mdl)______________________________________________________________________________________________________________________________________C. Fish Harvesting Simulink Optimization Design (\Utim\fishharvester\fishpond2.mdl)

*

4. DC Motor Parameter Optimization with GA (IEEE_OPTIM_2012\Diodes_amps\lead_screw_model_opt_Motor_ga.mdl)\______________________________________________________________________________________________________________________________________

C. TRADITIONAL OPTIMIZATION METHODS5

*

8. Buck Boost converter optimization (IEEE_OPTIM_2012\Buck_Boost\Buck_PWM_OPT12_Fs_Optim.mdl) (IEEE_OPTIM_2012\Buck_Boost\Buck_PWM_OPT12_Ti_L.mdl)______________________________________________________________________________________________________________________________________9. Operational Amplifier Optimization (IEEE_OPTIM_2012\Diodes_amps\Op_amp_opt.mdl)___________________________________________________________________10. Diode Voltage Doubler (IEEE_OPTIM_2012\Diodes_amps\diode_2.mdl___________________________________________________________________D. REAL DATA PARAMETERIZING 11. VCB Motor Parameter Optimization (IEEE_OPTIM_2012\lead screw\vcb_motor_opt_Motor_trk.mdl)______________________________________________________________________________________________________________________________________12. Parmeterizing Gas turbine Fuel Positioner (IEEE_OPTIM_2012\Gas_bio_Turbine\GT_Biofuel_present_b.mdl)___________________________________________________________________ _____________________________________________

*

REFERENCES1. Optimization toolbox for use with MATLAB, User Guide, The MathWorks Inc. 2006 2. Applied Optimization with MATLAB Programming, P. Venkataraman, Wiley Inter Science, 2002 3. Optimization for Engineering Design, Kalyanmoy Deb, Prentice Hall, 1996. 4. Convex Optimization, Stephen Boyd and Lieven Vandenberghe, CUP 2004. 5. Numerical Recipes in C (or C++): The Art of Scientific Computing, W. H. Press, Brain P. F. Saul A. T. W. T. Vetterling CUP ,1992/2002 6. http://users.powernet.co.uk/kienzle/octave/optim.html 7. http://www.cse.uiuc.edu/eot/modules/optimization/SteepestDescent/*

Math 685/CSI 700 Spring 08George Mason University, Department of Mathematical Sciences