1

A Matrix Free Newton /Krylov Method For Coupling Complex Multi-Physics Subsystems

Yunlin Xu

School of Nuclear EngineeringPurdue University

October 23, 2006

2

Content Introduction MFNK and Optimal Perturbation Size Fixed Point Iteration (FPI)

for coupling subsystems – A Matrix Free Newton/Krylov method based on FPI

– Local Convergence analysis of MFNK

– Truncation and Round-off Error

– Estimation of Optimal Finite Difference Perturbation

Global Convergence strategies – Line search

– Model trust region

Numerical Examples Summary

3

Features of Multi-Physics Subsystems

Multiple nonlinear subsystems are coupled together:

The solution of each subsystem depends on some external variables which come from the other system

0),( iii yxf

inix : internal variables

)...,,...,,( 1121 piiii xxxxxy : external variables

Each subsystem can be solved with reliable methods as long as they remain decoupled

4

Two General Approaches for Coupling Subsystems

Analytic Approach: reformulate the coupled system into a larger system of equations– Standard Newton-type methods can be applied

Synthetic Approach: combine the subsystem solvers for the coupled system– utilize the well-tested and reliable solutions of each of the

subsystems because:• It may be too expensive to reformulate the coupled system and

forego the significant investment in developing reliable solvers for each of the subsystems.

• One of the subsystems may be solved using commercial software that prevents access to the source code which makes it impossible to reformulate the coupled equations for the analytic approach

5

Coupled Subsystem Example:Nuclear Reactor Simulation

6

Time Advancement:Marching vs Nest Scheme

TRAC-E

PARCS

step n step n+1

– The time steps must be kept small for accuracy and stability concerns

Marching

NestedTRAC-E

PARCS

step n step n+1

Converged?

N

YConverged

?

N

YConverged

?

N

Y

– Computational cost for each time step increased– Numerical Stability and accuracy can be improved– Time step size may be extended

7

Ringhals BWR Stability

48 hours on 2 GHz machine for initialization!

128 Chans

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

1 S8 (12) 854 854 154 154 154 154 S1 (19)

2 854 854 853 153 153 153 154 154

3 854 854 854 854 853 853 151 152 152 153 154 154 154 154

4 854 854 853 853 852 852 852 153 151 152 152 152 153 153 154 154

5 854 854 853 852 852 853 851 852 109 110 151 153 152 153 152 152 154 154

6 854 854 853 852 852 853 852 853 801 110 109 153 151 153 152 153 151 152 254 254

7 S7 (20) 754 854 852 853 852 853 851 853 805 806 103 104 111 112 151 153 155 156 252 252 254 254 S2 (12)

8 754 754 752 852 851 853 852 853 803 806 805 104 101 112 111 152 151 156 155 252 251 253 254 254

9 754 754 752 752 755 756 851 853 807 808 801 804 105 106 103 103 113 113 251 253 252 253 251 252 254 254

10 754 753 752 751 756 755 852 801 808 807 804 803 106 105 103 101 113 113 203 251 253 252 252 252 253 254

11 754 753 752 753 751 752 704 704 801 803 802 804 101 104 107 108 203 204 205 206 251 253 252 253 253 254

12 754 754 752 753 752 753 751 703 704 803 803 803 802 103 103 108 107 204 201 206 205 203 251 253 252 252 254 254

13 754 754 753 752 751 753 712 713 711 711 708 709 801 804 102 103 201 203 202 203 203 204 207 208 251 253 252 252 254 254

14 754 753 752 752 752 751 713 712 711 710 709 708 804 803 104 102 204 202 204 202 204 201 208 207 203 251 253 252 253 254

15 754 753 751 752 714 714 701 703 706 707 702 703 705 705 901 902 305 305 301 303 306 307 303 303 314 314 351 353 353 354

16 754 753 753 751 714 714 703 703 707 706 703 701 705 705 902 901 305 305 303 302 307 306 303 301 314 314 352 351 353 354

17 654 653 652 653 651 603 607 608 601 604 602 604 602 604 502 504 403 404 308 309 310 311 312 313 351 352 352 352 353 354

18 654 654 652 652 653 651 608 607 604 603 603 602 603 601 503 502 404 401 309 308 311 311 313 312 353 351 352 353 354 354

19 654 654 652 652 653 651 603 605 606 601 604 507 508 503 503 402 403 403 403 304 303 351 353 352 353 352 354 354

20 654 653 653 652 653 651 606 605 604 603 508 507 504 501 404 402 403 401 304 304 352 351 353 352 353 354

21 654 653 652 652 652 653 651 603 513 513 501 503 505 506 403 404 407 408 401 452 355 356 351 352 353 354

22 654 654 652 651 653 652 653 651 513 513 503 503 506 505 404 401 408 407 453 451 356 355 352 352 354 354

23 654 654 653 651 652 555 556 551 552 511 512 501 504 405 406 403 453 452 453 451 452 352 354 354

24 S6 (12) 654 654 652 652 556 555 553 551 512 511 504 503 406 405 453 451 453 452 453 452 454 354 S3 (20)

25 654 654 552 551 553 552 553 551 553 509 510 401 453 452 453 452 452 453 454 454

26 554 554 552 552 553 552 553 551 510 509 452 451 453 452 452 453 454 454

27 554 554 553 553 552 552 552 551 553 452 452 452 453 453 454 454

28 554 554 554 554 553 552 552 551 453 453 454 454 454 454

29 554 554 553 553 553 453 454 454

30 S5 (19) 554 554 554 554 454 454 S4 (12)

8

Synthetic Approaches

Nested Iteration– Subsystems are chained in block Gauss-Seidel or block Jacobi iteration

– Convergence is not guaranteed. Matrix Free Newton/Krylov Method

– Approximate Mat-Vec by quotient: )()(

)('

kj

k

jk

xFvxFvxF

– The system Jacobian is not constructed

– Local Convergence guaranteed

Problems with Direct Application of MFNK for Coupling of Subsystems– Solvers for the subsystems are not fully utilized

– Difficult to find a good preconditioner for MFNK

– In some cases, it is not possible to obtain residuals for a subsystem if the solver of subsystem is commercial software which can be used only as a “black box”.

9

Objectives of Research

Propose a general approach to implement efficient matrix free Newton/Krylov methods for coupling complex subsystems with their respective solvers

Identify and address specific issues which arise in implementing MFNK for practical applications– Local convergence analysis of the matrix free Newton/Krylov

method – Optimal perturbation size for the finite difference

approximation in MFNK – Globally convergent strategies

10

Fixed Point Iteration for Coupling Subsystems

Block IterationN kpiyxxx k

iki

ti

ki

ki

i ,1),,()(1

)),,((),(,iterations denotes 1 ki

ki

ki

tii

ki

ki

tii

ti yyxyxt iii

Block Iteration for coupling subsystems are fixed point iterations

p

j

nx1

,:, innn n

),...,(),,...,(),( 21211

ppkk xxxxxx

The condition for convergence of FPI is ||Φ(x*)||<1. If ||Φ(x*)||>1, then the FPI may diverge

11

Matrix Free Newton Krylov Method Based on a Fixed Point iteration

Define a nonlinear system: F(x)= Φ(x) -x =0 The solution of this system is the fixed point of function Φ(x),

which is also the solution of original coupled nonlinear system MFNK algorithm:

At kth Newton step, do

1) Find ks satisfies '( ) ( )k k k k kIt FDF x s F x r r

by a Krylov iteration, in which mat-vec, jk vxF )(' , be approximated by:

,)()()()(

)(' j

kj

kkj

k

jk v

xvxxFvxFvxF

where kItr is the iterative residual,

kFDr is the residual due to the finite difference approximation.

2) Set 1k k kx x s

12

Local Convergence of INM

The convergence of INM depends on the inner residual, assume– If p2, the INM has local q-quadratic convergence. – If 1<p<2, INM converges with q-order at least p.– If p=1 and , the INM has local q-linear convergence.

pkk xFr )(

1

Inexact Newton Method (INM)

pk

p

kk xxxFxxxx *||*)('||4

12*||*||

21

11 '( ) ( )k k k k kx x F x F x r

Local Convergence of INM

If )( kk xFr , 1 or pkk xFr )( , p>1, Then

13

Local Convergence of MFNK

11 '( ) ( )k k k k k kIt FDx x F x F x r r

k kFD mr E s

kItr

The inner residual consists with:

MFNK is an INM

– iterative residual – finite difference residual

Tmmm vvddE 11 There are two conflicting sources of error in finite difference:

– Truncation error

– Round off error

( ) ( )'( )

F x v F xd F x v

( ) ( ) ( ) ( ) 1( ) ( )rd

F x v F x v F x F xe F x F x v M x M v

2( ) ( ) '( )tr

F x v F x F x ve v

14

Local Convergence of MFNK (cont.)

In theory, MFNK has local q-linear convergence, if

1opt In practice, MFNK can achieve q-quadratic convergence, if

||)(|| kopt xF

The optimal should balance the round-off error and truncation error

kIt

kopt

kFD rxFr )( 1)('4 k

optopt xFm

)(4)('1 xFx

xF

vopt

15

Optimal vs Empirical Perturbation Size

The norm of the Jacobian and can be estimated with information provided by the MFNK algorithm:

22

2

1)('

)()(max

~xF

v

xFvxFM

mj

2

2

22

2

2

1

5.0

)(')5.0(')(2~k

kkkkk

k

k

s

sxFssxF

s

xF

~

)(2~

~1~ xF

xM

vopt or

xxM

vopt

~

2~

~1~

An empirical prescription was proposed attempt to balance the truncation and round-off errors (Dennis)

max ,emp

x typ x

v

16

Global Convergence Strategies

Solution x* of system of nonlinear equation: F(x)=0 is also the global minimizer of optimization problem:

2

2

1( ) ( )

2minnx R

f x F x

Newton step sN is the step from current solution to global minimizer of model problem:

2

2

1( ) ( ) '( )

2minn

c c c

s R

m x s F x F x s

f(xc+sN) may be larger than f(xc), due to big step sN such that m(xc+sN) is no longer a good approximation of f(xc+sN). In this case, we need globally convergent strategy to force f(xc+sN)<f(xc)

17

Descent Direction

Newton step is descent direction of both objective function and its model:

( ) ( ) '( ) ( )c c c T cf x m x F x F x ( ) ( ) '( ) ( ) ( ) 0c T N c T c N c T cf x s F x F x s F x F x

For any descent direction pk, there exist λ satisfies: (1)

( ) ( ) ( )k k k k k k T kf x p f x f x p ( ) ( ) ( )k k k k k k T kf x p f x f x p

0 1

α-condition

β-condition

A sequence {xk} generated by xk+1=xk+λkpk satisfying previous condition will converge to a minimizer of f(x). (2)

(1),(2) proofs can be found in Dennis & Schnabel’s book

18

Line Search

Take MF Newton step as descent direction, and select λ to minimize a model of following function

ˆ ( ) ( )k Mf f x s

Quadratic model2 2

1 1 1ˆ ˆ ˆ ˆ ˆˆ ( ) (0) '(0) ( ) (0) '(0) /qm f f f f f

1

1 1ˆ ˆ ˆ2 2 ( ) (0) '(0)

qf f f

λ predicted from quadratic model

19

Information Requiredin Quadratic Model

ˆ ˆ'(0) 2 (0)f f

ˆ '(0) ( ) ( ) '( )

ˆ ˆ'(0) ( ) ( ( ) ) 2 (0) ( )

k T M k T k M

k T k k T

f f x s F x F x s

f F x F x r f F x r

Two function values:

ˆ (0) ( )kf f x

One Gradient

Approximations for Gradient in MFNKˆ ˆ'(0) 2 (0) ( )k T

itf f F x r or

1 1ˆ ( ) ( )k Mf f x s

20

Model Trust Region

Minimize model function in neighborhood, trust region

xc

xN

c

xc+s(c)

sN

2 2

2

1 1( ) ( ) '( ) ( ) ( ) ( )

2 2c c c c c T cMin m x s F x F x s f x m x s s m x s

subject to 2 cs

21

Double Dogleg Curve

Approximate optimal path with double dogleg curve

C.P.

xc

xN

N

c

sN

Step along double dogleg curve.. .

.

.. . .

.

,

( ) ( ),

,

C P C PC P

C PC P N N C P C P N

N C P

N N

s if

s s s s if

s if

22

Cauchy Point

Cauchy Point is minimizer in steepest descent direction2 2

. 2 222

2

( )( )

( ) ( ) ( )

c T

C P c Tc T c c T

m x J Fs m x J F

m x m x m x J J F

Projection of Step for Cauchy Point on Krylov subspace

(Brown & Saad)

22 2

. 2 2 22 2 2

2 22

ˆ

T

m T m mC Pm m m m m m

Tm m m mm m

JV F d ds V JV F V d V d

H d R dJV JV F

1 ,m m mJV V H 1,T T

m m md JV F H e m mH QR

23

Example Problem I: Polynomials

21 1 2 1 2 1

1 2 22 1 2 1 2 2

( , ) ( 1) ( 1) ( 1)( , )

( , ) ( 1) ( 1) ( 1)

f x x a x b x c xF x x

f x x b x a x c x

Two dimensional second order polynomials

11 2

2

2'( , )

2

a cx bF x x

b a cx

* *1 2( , ) (1,1)x x

* *1 2 2

'( , ) max 2 , 2F x x a c b a c b

11 * * 11 2 2

'( , ) min 2 , 2F x x a c b a c b

* *2 1 2'( , )F x x

Solution

Jacobian

Nonlinear level 1 2

1 2

2 2,1 2

2 max{ , }max 2x x

c x xc

x x

24

PLY 1 Truncation Error Dominated Case

-20 -15 -10 -5 0 5 10 15 20-20

-15

-10

-5

0

5

10

15

2010

10

10

13

1 2 10

0.5

10

1.5, 2,

3, 2 10 ,

1.53 10opt

Input values

a

b

c

Key parameters

25

PLY 1 Errors

1.E-20

1.E-18

1.E-16

1.E-14

1.E-12

1.E-10

1.E-08

1.E-06

1.E-04

1.E-02

1.E+00

1.E+02

0 10 20 30 40 50

Newton Step

L2 n

orm

of

Err

or

Optimal Step Size Empirical Step Size

26

PLY 1 Step Sizes

0 10 20 30 40 5010

-14

10-12

10-10

10-8

10-6

Newton Step

Op

tima

l fin

ite d

iffe

ren

ce S

tep

size

: s

igm

a

Computed from JacobianEstimated by MF

0 10 20 30 40 5010

-14

10-12

10-10

10-8

10-6

Newton Step

finite

diff

ere

nce

Ste

psi

ze :

sig

ma

Computed from JacobianEmpirical

Optimal Empirical

27

PLY 1 Step Size Parametric

1.E-04

1.E-03

1.E-02

1.E-01

1.E+00

1.E+01

1.E-16 1.E-14 1.E-12 1.E-10 1.E-08

Finite Difference stepsize

Rel

ativ

e er

ror

or

resi

dau

l aft

er1

step

New

ton

iter

atio

n

ErrorOuter residualInner residual

28

PLY 2 Round Off Error Dominated Case

-20 -15 -10 -5 0 5 10 15 20-20

-15

-10

-5

0

5

10

15

20

8

8

8

8

10 8

10

10 0.01

10

2 10 , 100,

2 10 , 2 10 ,

1.772opt

Input values

a

b

c

Key parameters

29

PLY 2 Errors

1.E-201.E-181.E-161.E-141.E-121.E-101.E-081.E-061.E-041.E-021.E+001.E+021.E+041.E+061.E+081.E+101.E+121.E+14

0 10 20 30 40 50

Newton Step

L2 n

orm

of

Err

or

Optimal Step Size Empirical Step Size

30

PLY 2 Stepsizes

1 1.5 2 2.5 3 3.5 4 4.5 510

-8

10-6

10-4

10-2

100

102

Newton Step

Op

tima

l fin

ite d

iffe

ren

ce S

tep

size

: s

igm

a

Computed from JacobianEstimated by MF

0 10 20 30 40 5010

-10

10-5

100

105

1010

Newton Step

Fin

ite d

iffe

ren

ce S

tep

size

: s

igm

a

Computed from JacobianEmpirical

Optimal Empirical

31

PLY 2 Stepsize Parametric

1.E-171.E-161.E-151.E-141.E-131.E-121.E-111.E-101.E-091.E-081.E-071.E-061.E-051.E-041.E-031.E-021.E-011.E+001.E+01

1.E-12 1.E-08 1.E-04 1.E+00 1.E+04 1.E+08

Finite Difference stepsize

Re

lati

ve e

rro

r o

r re

sid

au

l aft

er1

step

New

ton

ite

rati

on

ErrorOuter residualInner residual

1.772opt

32

Numerical Examples: Navier-Stokes-Like Problem (Goyon)

),()(2

1)(2 yxquuuuu

PDE

Diffusion Convection Non-physical Force function

Boundary Condition

]1,0[]1,0[,0),( onvuu

)](2sin38)[2sin()2sin( yxyxsq

Force function),(),( sqyxq

Goyon, Precoditioned Newton Methods using Incremental Unknowns Methods for Resolution of a Steady-State Navier-Stokes-Like Problem. Applied Mathematic and Computation, 87(1997), pp. 289-311.

33

The Finite difference Equations

04

3

4

3

224

22

,1,,

21,,

2

,1,

2,1,

2,,1,1

22

jijiy

ji

yji

y

ji

y

jix

ji

xji

x

ji

xji

x

jiji

yx

svh

v

hv

h

v

h

vh

u

hv

h

u

hv

h

uu

hh

v

uw 0

),(0

0),()()(

V

U

b

b

v

u

vuA

vuAbwwAwF

v

u

v

u

022

4

3

4

3

4

22

,1,,

21,,

2

,1,

2,1,

2,1,1,

22

jijiy

ji

yji

y

ji

y

jix

ji

xji

x

ji

xji

y

jiji

yx

quh

v

hu

h

v

h

uh

u

hu

h

u

hu

h

vv

hh

34

Structure of the Matrices

vv

AAv

u

A

uv

Au

u

AA

xFv

vv

uuu

)('Jacobian Diag block

vv

AA

uu

AA

xBv

v

uu

0

0)(

35

NSL1: 50X50 meshes, =0.0015

Solving (u,v) as One Nonlinear System, w0=(1-810-3) w*

Newton I. N.(6) MFNK I. N. MFNK

Preconditioner N.A. I I B B

NNI(1) 3 18 20 3 3

NLI(2) N.A. 3600 3991 93 94

NB(3) 0 0 0 0 0

NFE(4) 4 5 4012 4 98

NMV(5) N.A. 3600 0 93 0

(1)NNI: Number of nonlinear iterations

(2) NLI: Number of cumulated Krylov iteration for Newton linear systems

(3) NB: Number of backtracking

(4) NFE: Number of function evaluation

(5) NMV: Number of Mat-Vecs in Krylov iteration

(6) Inexact Newton with GMRES as its linear system solver

36

NSL1: Newton Iterative error and residual

1.E-13

1.E-12

1.E-11

1.E-10

1.E-09

1.E-08

1.E-07

1.E-06

1.E-05

1.E-04

1.E-03

1.E-02

1.E-01

1.E+00

0 5 10 15 20

Newton Step

L2

no

rm o

f err

or

of i

tera

tive

so

lutio

n

NewtonIN(I)MFNK(I)IN(B)MFNK(B)

1.E-13

1.E-12

1.E-11

1.E-10

1.E-09

1.E-08

1.E-07

1.E-06

1.E-05

1.E-04

1.E-03

1.E-02

1.E-01

1.E+00

0 5 10 15 20

Newton Step

L2 n

orm

of

nonl

ear

syst

em r

esid

ual

NewtonIN(I)MFNK(I)IN(B)MFNK(B)

Error Residual

37

NSL1: Coupling Subsystem

FPI MFNK FPI

schemes

Steps/tolerance

for Subsystems NNI NB(1) time(s) (2) NNI NB NFE time(s)

1 60 / 0 386 460 46.64 7 0 194 24.05

2 70 / 0 388 2232 61.58 4 0 112 17.90

3 80 / 0 450 2768 89.65 4 0 111 21.45

4 70 / 10-4 358 2030 44.05 5 0 135 19.12

5 70 / 10-5 547 3509 78.83 4 0 113 17.89

6 Direct solver 186 484 -(3) 4 0 135 - (3)

(1) Maximum 8 times halving the increment of solution for each Fixed point iteration

(2) Run on Purdue SP2 cluster, the CPUs are 375 MHz PWR3.

(3) Run on PC with Matlab linear system direct solver.

Solving u and v as two subsystem, and coupled by FPI or MFNK

38

NSL1: Global convergence

Global strategy NO GS LS MTR

NNI 500 28 23

NLI 3534667 1875 1399

NB 0 61 33

NFE 50501 1965 1456

NMV 3534667 135029 99993

Time(s) 7919.09 308.21 221.08

w0=(1-810-3) w*

39

NSL1 Residual

1.E-13

1.E-12

1.E-11

1.E-10

1.E-09

1.E-08

1.E-07

1.E-06

1.E-05

1.E-04

1.E-03

1.E-02

1.E-01

1.E+00

1.E+01

1.E+02

0 5 10 15 20 25 30

Newton Step

L2 n

orm

of

Res

idua

l

NO GSLSMTR

40

NSL1: First Backtracking

The first backtracking occurred after the first Newton iteration.

The L2 norm of residual Lambdabefore 1.65150No GS 2.42510 LS 1.07990 0.316829MTR 1.03694 0.316829

41

NSL2: 1000X1000 meshes

Test cases

NSL2a (=10-3)

NSL2a (=10-3)

NSL2a (=10-3)

NSL2b (=510-4)

NSL2c (=210-4)

Scheme FPI(1) MFR(2) Opt(3) Emp(4) Opt Emp Opt Emp NNI 500(5) 500(5) 9 100 11 100 11 100 NLI 0 25000 165 1945 212 1877 420 2000 NB 4964(6) 1273(7) 0 261 6 271 8 253 NFE 5465 26774 175 2307 230 2249 440 2354 NMV 25000 0 3500 46140 4600 44980 9000 47080 Time(s) 100422 69292 9211 65690 12423 100534 24731 96791

(1) FPI with 50 steps for solving linear system of subsystems

(2) MFNK based on original nonlinear system, 50 Krylov steps for each Newton step

(3) MFNK based on FPI, with optimal perturbation.

(4) MFNK based on FPI, with empirical perturbation.

(5) 500 is limit of nonlinear iterations, NNI=500 indicates not converge

(6) At most 10 times halving for each FPI

(7) At most 3 backtrackings for each Newton step

42

NSL2: Residuals for FPI and MFR

5

6

6

7

7

8

8

9

0 10 20 30 40 50

Newton Step

L2 n

orm

of

Res

idua

l

NSL2a

NSL2b

NSL2c

0

1

2

3

4

5

6

7

8

9

10

0 10 20 30 40 50

FPI Step

L2 n

orm

of

Res

idua

l

NSL2a

NSL2b

NSL2c

FPI MFR

43

NSL 2: Optimal vs Empirical Finite Difference

Optimal Empirical

1.E-03

1.E-02

1.E-01

1.E+00

1.E+01

0 20 40 60 80 100

Newton Step

L2 n

orm

of

erro

r

NSL2a

NSL2b

NSL2c

1.E-05

1.E-04

1.E-03

1.E-02

1.E-01

1.E+00

1.E+01

0 5 10 15 20

Newton Step

L2 n

orm

of

erro

r NSL2a

NSL2b

NSL2c

44

NSL2 Step Sizes

1.E-09

1.E-08

1.E-07

1.E-06

1.E-05

1.E-04

0 5 10 15 20

Newton Step

Est

imat

ed O

ptim

al F

inite

Diff

eren

ce S

ize

NSL2aNSL2bNSL2c

Empirical finite difference size for NSL2

45

NSL2 Step Size parametric

0.E+00

1.E-05

2.E-05

3.E-05

4.E-05

5.E-05

1.E-12 1.E-11 1.E-10 1.E-09 1.E-08 1.E-07 1.E-06 1.E-05 1.E-04

Finite Difference Size

L2-n

orm

of

erro

r af

ter

1 N

ewto

n st

ep

Empirical

Estimated Optimal

46

Summary

A general approach, MFNK, was presented here for coupling subsystems with respective solvers.

Based on any FPI, a corresponding MFNK method can be constructed.

MFNK provides a more efficient method than FPI for coupling subsystems.

MFNK can converge for several cases in which the corresponding FPI diverges.

Locally, MFNK converges at least q-linearly and in many cases q-quadratically.

A more sophisticated FPI scheme provides a more efficient nonlinear system for the corresponding MFNK.

47

Summary (Cont.)

A method was proposed to estimate the optimal perturbation size for matrix free Newton/Krylov methods.

The method was based on an analysis of the truncation error and the round-off error introduced by the finite difference approximation.

The optimal perturbation size can be accurately estimated in the MFNK algorithm with almost no additional computational cost.

Numerical examples shows that the optimum perturbation size leads to improved convergence of the MFNK method compared to the perturbation determined by empirical formulas.

48

Summary (Cont.)

Line Search and Model Trust Region, were implemented within the framework of MFNK

– For the line search method, a quadratic or higher order model was used with an approximation for the gradient

– For the model trust region strategy, a double dogleg approach was implemented using the projection of a Newton step and Cauchy point within the Krylov subspace

– the model trust region strategy showed better local performance than the line search strategy

Peer-to-Peer parallel MFNK algorithm was implemented