OPT Matlab[1]

8/3/2019 OPT Matlab[1]

1/19

OutlineOverview

Optimization ToolboxGenetic Algorithm and Direct Search Toolbox

Function handlesGUI

Homework

Optimization in Matlab

Kevin Carlberg

Stanford University

July 28, 2009

Kevin Carlberg Optimization in Matlab
http://find/http://goback/


2/19

OutlineOverview


Function handlesGUI

Homework

1 Overview

2 Optimization Toolbox

3 Genetic Algorithm and Direct Search Toolbox

4 Function handles

5 GUI

6 Homework

http://find/


3/19

OutlineOverview


Function handlesGUI

Homework

OverviewMatlab has two toolboxes that contain optimization algorithmsdiscussed in this class

Optimization ToolboxUnconstrained nonlinearConstrained nonlinearSimple convex: LP, QPLeast SquaresBinary Integer ProgrammingMultiobjective

Genetic Algorithm and Direct Search Toolbox: generaloptimization problems

Direct search algorithms (directional): generalized patternsearch and mesh adaptive searchGenetic algorithm

Simulated annealing and Threshold ac ceptanceKevin Carlberg Optimization in Matlab


4/19

OutlineOverview


Function handlesGUI

Homework

Problem types and algorithmsContinuous

Convex, constrained (simple)LP: linprog

QP: quadprogNonlinear

Unconstrained: fminunc , fminsearchConstrained: fmincon , fminbnd , fseminf

Least-squares (specialized problem type): minx F (x ) 2F (x ) linear, constrained: lsqnonneg , lsqlinF (x ) nonlinear: lsqnonlin , lsqcurvefit

Multiobjective: fgoalattain , fminimaxDiscrete

Linear, Binary Integer Programming: bintprog



5/19

OutlineOverview


Function handlesGUI

Homework

Continuous, nonlinear algorithms

We will focus only on the following algorithms for continuous,nonlinear problemsUnconstrained: fminunc , fminsearchConstrained: fmincon


O li


6/19

OutlineOverview


Function handlesGUI

Homework

Nonlinear, unconstrained algorithms

fminunc : a gradient-based algorithm with two modesLarge-scale : This is a subspace trust-region method (see p.

7677 of Nocedal and Wright). It can take a user-suppliedHessian or approximate it using nite differences (with aspecied sparsity pattern)Medium-scale : This is a cubic line-search method. It usesQuasi-Newton updates of the Hessian (recall thatQuasi-Newton updates give dense matrices, which areimpractical for large-scale problems)

fminsearch : a derivative-free method based on Nelder-Meadsimplex


O tli


7/19

OutlineOverview


Function handlesGUI

Homework

Nonlinear constrained algorithm: fmincon

fmincon : a gradient-based framework with three algorithmsTrust-region reective : a subspace trust-region methodActive Set : a sequential quadratic programming (SQP)method. The Hessian of the Lagrangian is updated usingBFGS.Interior Point : a log-barrier penalty term is used for the

inequality constraints, and the problem is reduced to havingonly equality constraints


Outline


8/19

OutlineOverview


Function handlesGUI

Homework

Algorithms

Algorithms in this toolbox can be used to solve generalproblemsAll algorithms are derivative-free methodsDirect search: patternsearchGenetic algorithm: gaSimulated annealing/threshold acceptance: simulannealbnd ,threshacceptbndGenetic Algorithm for multiobjective optimization:gamultiobj


Outline


9/19

OutlineOverview


Function handlesGUI

Homework

Function handlesFunction handle : a MATLAB value that provides a means of calling a function indirectly

Function handles can be passed in calls to other functionsFunction handles can be stored in data structures for later useThe optimization and genetic algorithm toolboxes makeextensive use of function handles

Example: Creating a handle to an anonymous functionbowl = @(x,y)x^2+(y-2)^2;ezsurf(bowl)

! 6! 4

! 20

24

6

! 5

0

5

0

20

40

60

80

00

x

x2+(y! 2)2

y


Outline


10/19

OutlineOverview


Function handlesGUI

Homework

Example: creating a handle to a named functionAt the command line, typeedit bowlNamed;In an editor, create an m-le containingfunction f = bowlNamed(x,y)f = x^2+(y-2)^2;At the command line, typebowlhandle = @bowlNamed;ezsurf(bowlhandle)

! 6! 4

! 20

24

6

! 5

0

5

0

20

40

60

80

00

x

x2+(y! 2)2

y


Outline


11/19

OutlineOverview


Function handlesGUI

Homework

Function handles for optimization

For the optimization toolbox, only one vector-valued inputargument should be used

Example: creating a handle to an anonymous function withone vector-valued input variablebowlVec = @(x)x(1)^2+(x(2)-2)^2;

Example: creating a handle to a named function with two

scalar-valued input variablesbowlVecNamed = @(x)bowlNamed(x(1),x(2));ezsurf cannot accept handles with vector-valued arguments(stick with examples on previous pages)


Outline


12/19

OverviewOptimization Toolbox

Genetic Algorithm and Direct Search ToolboxFunction handles

GUIHomework

Supplying gradients

It may be desirable to analytically specify the gradient of thefunctionTo do this, the named function must return two outputs: thefunction value and the gradient


Outline


13/19



GUIHomework

Complete example: creating a handle to a named function,plotting it, and specifying the handle for optimizationAt the command line, typeedit bowlNamed;In the editor, create an m-le containingfunction [f,g] = bowlNamed(x,y)f = x^2+(y-2)^2;g(1) = 2*x;g(2) = 2*(y-2);At the command line, type

bowlhandle = @bowlNamed;ezsurf(bowlhandle);bowlhandleOpt = @(x) bowlNamed(x(1),x(2))

bowlhandleOpt can now be used as the argument to aMatlab optimization function with supplied gradients


Outline


14/19



GUIHomework

GUIThe optimization toolbox includes a graphical user interface(GUI) that is easy to useTo activate, simply type optimtool at the command line


Outline


15/19



GUIHomework

GUI options

We would like to track the progress of the optimizerUnder options , set Level of display: iterativeUnder plot functions , check: function valueWhen ga is used, check Best tness, and Expectation totrack the tness of the best member of the population andthe average tness of the population


OutlineO i


16/19



GUIHomework

For the functions on the following pages, do the following:1 Create a function that takes in two scalar-valued arguments

and outputs both the function and gradient2 Create a handle for this function and use ezsurf to plot the

function3 Create an optimization-ready handle for this function and

solve using different starting points using:fminunc , medium scale, derivatives approximated by solverfminunc , medium scale, gradient suppliedfminsearchga

4 Compare the algorithms on the following measures:1 Robustness: ability to nd a global optimum and dependence

of performance on initial guess2 Efficiency: how many function evaluations were required?


OutlineO i


17/19



GUIHomework

Problem 1Consider a convex function with constant Hessian

f (x 1 , x 2) = 4 x 21 + 7( x 2 4)2 4x 1 + 3 x 2

! 6! 4

! 20

2

46

! 6! 4

! 20

24

6

0

200

400

600

800

y

prob 1

x

y

prob 1

! 6 ! 4 ! 2 0 2 4 6

! 6

! 4

! 2

0

2

4

6

Figure: Surface and contour plot

Also, nd the analytical solution to this pr ob l emKevin Carlberg Optimization in Matlab

OutlineOverview


18/19



GUIHomework

Problem 2Consider the Rosenbrock function, a non-convex problem thatis difficult to minimize. The global minimum is located at(x 1 , x 2) = (0 , 0)

f (x 1 , x 2) = (1 x 1)2 + 100( x 2 x 21 )2

! 1

! 0.5

0

0.5

1

! 1! 0.5

00.51

0

100

200

300

400

500

xx

y

! 1 ! 0.8 ! 0.6 ! 0.4 ! 0.2 0 0.2 0.4 0.6 0.8 1! 1

! 0.8

! 0.6

! 0.4

! 0.2

0

0.2

0.4

0.6

0.8

1

Figure: Surface and contour plotKevin Carlberg Optimization in Matlab

OutlineOverview


19/19



GUIHomework

Problem 3Consider the Rastragins function, an all-around nastyfunction. The global minimum is located at (x 1 , x 2) = (0 , 0)

f (x 1 , x 2) = 20 + x 21 + x 22 10(cos 2 x 1 + cos 2 x 2)

! 6! 4

! 20

24

6

! 5

0

5

0

20

40

60

80

100

xy x

y

! 6 ! 4 ! 2 0 2 4 6

! 6

! 4

! 2

0

2

4

6

Figure: Surface and contour plot


Date post:	06-Apr-2018
Category:	Documents
Upload:	nadia-dridi
View:	235 times
Download:	0 times

OPT Matlab[1]

Documents