CAAM 353 -Solving Syst of NL Equations

7/28/2019 CAAM 353 -Solving Syst of NL Equations

1/32

CAAM 353

Solving Systems of Non-Linear

Equations


2/32

f(x)=0

Weve discussed so far:

One nonlinear equation, one unknown

n linear equations, n unknowns

Now, we move on to: n nonlinear equations, n unknowns

f(x)=0, where f is a set of n equations, and x is

a vector of n unknowns In words, want each to simultaneously

equal zero when we plug in the vector x


3/32

Nonlinear Equations

Nonlinear equations dont follow the same

pattern of linear equations of:

0, 1 or infinite solutions

You can have any number of solutions!

1


4/32

Application

When do we need to solve a system of

nonlinear equations?

Optimization problems!

Ex. Given a function f(x), (where x is a vector

of several variables), what values of x

maximize or minimize f?

To solve, you write down all partial derivatives

of f w.r.t. each and set equal to 0


5/32

Applications

MAX

Global vs. Local Maxs or Mins


6/32

Real Example

Modeling the cytoskeleton:

cross-liked fiber network of

actin filaments

To simulate motion and

deformation, fibers

modeled as elastic spring

network

Hookes law type behavior http://scienceblogs.com/transcript/2007/01/cytoskeleton.php


7/32

Fiber Network

Total potential energy in the system is given

by: ()

= for each fiber i in the

system with n fibers

= current fiber length, relaxed length

Question is: where should the fiber endpoints

be to minimize the stored energy in the

system?


8/32

Fiber Network

To find position of minimalenergy for all fiber

endpoints, we must find the

minimum of the energy

equation

Solve system:

0 with

partials for ys and zs also


9/32

Solution Approach

Iterative approach: Begin with initial guess and (hopefully) walk towards the solution

Unfortunately, no bisection-type method

exists for systems of nonlinear equations Newton does extend!

Can get convergence if our initial guess is

close to the true one But no bisection to help us narrow down

where to start!


10/32

Newton for Systems

Newtons method derived from Taylor series,

Systems equivalent can be derived from multivariableTaylor series

2 variable example: , , , f a, b y b

1

2

,

,

,


11/32

Newton for Systems

General case:

(|||

Where , , , , , , , (, , , ) is a direction vector

= Jacobian matrix, matrix of partialderivatives


12/32

Jacobian Matrix


13/32

Newton for Systems

initial guess vector 0 +

Solve Linear system: () (Gives new direction vector p)

Update +

Repeat! Until when?


14/32

Stopping Criteria

Recall the stopping criteria for the one

variable cases:

< 1 < 2 < 3 < 4 (1 )

Simply replace absolute value with normas


15/32

Newton for Systems

One issue: Method will only work if J(x) is

nonsingular

Will depend on current x vector

Inverse Function Theorem: (from calculus), if

functions in vector f are continuously

differentiable, and the Jacobian is nonsingular

at a point (vector) x* then the Jacobian will

be nonsingular in a neighborhood of x*


16/32

Convergence

Like in the one function, one variable case,

Newtons method for systems converges

quadraticallyprovided the Jacobian has a

bounded inverse (is nonsinular) and providedour initial guess is close to the root

+ <


17/32

Example

4 cos()

newtonsys.m


18/32

Unconstrained Optimization

Problem:

min

()

System of equations: 0() Hessian matrix of 2nd derivatives, (at a

minimum, this matrix is symmetric positive definite) Algorithm:

Solve () +


19/32

Example

Fiber network (mass-spring) application

Move fibers to minimize potential energy


20/32

Example

Compute partial derivative of E w.r.t x1 x4and y1 y4

Compute Hessiam of 2nd partials (8 x 8 matrix)

Initial Stretched State Min. Energy State

basicsprings.m


21/32

Pros/Cons to Newton

Pros:

Easy Equivalent of the One Variable Case

Cons:

You have to compute Hessiam matrix every step

You have to do a linear solve every step

We have no control over convergence


22/32

Alternatives

General idea behind optimization iteration:

+ (walk step length a in the pdirection, to update x vector)

() (choose a new searchdirection)

Gradient Descent Method

Quasi-Newton Methods


23/32

Gradient Descent

Given a surface in defined by a functionf(x), the gradient of f(x) at a point x* always

points in the direction ofsteepest ascent

f(x) points in the direction of steepestdescent

Its a great choice for p (search direction)

+ f(x)


24/32

Gradient Descent

springsmin.m

Initial Position Ending Position


25/32

Step Size

How far should you walk in the p direction

to settle on a new position x?

Line Search Strategies:

Start with a max step size (like a=1), if f(x+ap) f(x) then

try a smaller step size: , , . etc.

Get 3 (a,f(x+ap)) coordinate pairs, do a quadraticinterpolation and find the minimum, utilize that a

value for the step size


26/32

Gradient Descent

Pros

No need for linear solve

No need to compute Hessian

Cons

Can be very slow

Must decide on step size every time


27/32


Systems equivalent of the secant method

Instead of computing the Jacobian every time,can we just estimate it? Or update it?

Better yet, can we update/estimate

()(inverse of the Hessiam matrix) to avoid thelinear solves?


28/32

Quasi-Newton methods

Taylors expansion for the gradient () + +

Let:

B (The Hessian matrix) + +

So, B must satisfy:


29/32


Its the problem Ax=b except x and b are

known and A is not!

Under-determined system.

But +shouldnt be too different from It just has to be adjusted to satisfy

(Just one direction) Class of methods that rely on this idea or rank-

one updates of B


30/32

BFGS Method

Popular Choice: Bryoden-Fletcher-Goldfarb-

Shannon method

Actually updates instead of

Algorithm: Choose and (usually identity matrix) Choose step size a,

+

, +

+


31/32

BFGS Method

springsbfgs.m

Initial Position Ending Postion


32/32

Local vs. Global Min/Max

We usually want a global min/max (the overall

smallest or largest value of our function f(x))

But we may end up at a local one

Typical Solution?

Try different starting

guesses and run

minimization routine

Choose the best

minimum from the set

Date post:	03-Apr-2018
Category:	Documents
Upload:	jemmettdwyer1033
View:	217 times
Download:	0 times

CAAM 353 -Solving Syst of NL Equations

Documents